Determination system of arthropod and gastropod

Abstract

To solve a problem that there is no method of automatically determining a type and an appearing place of an arthropod or a gastropod having a fracture part other than a part processed as desired.SOLUTION: Provided is an analyte type determination system that determines, from image information, a type of an arthropod or a gastropod having a fracture part other than a part processed as desired and an appearing place of the arthropod or the gastropod. Also provided is a portable mobile terminal mounted with the analyte type determination system.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a determination system that acquires an image of a part of an arthropod or gastropod to be determined, and determines the type of arthropod or gastropod that includes this part from the image. [Background technology]

[0002] In food processing plants, products may contain unexpected substances, including broken pieces.

[0003] Since the unexpected substance including the broken part is an unexpected substance including a broken part other than that which has been processed arbitrarily, or a melted part or a broken part caused by heat, etc., food producers must use this substance as a sample to identify the type of sample and investigate the process by which this substance became mixed into the product. In addition, in order to ensure safety and quality, it becomes necessary to recall all lots of products that contain this unexpected substance.

[0004] At food production sites and by consumers, arthropods such as insects that contaminate raw vegetable ingredients, gastropods such as snails, and some parts of arthropods such as insects that emerge when the eggs of these animals hatch inside food factories, may be recognized as unexpected substances (sometimes referred to as parts) including broken parts contained in products. In particular, when consumers find these parts in products, they may complain to producers, raise concerns about health damage, or cause so-called food loss due to the recall and disposal of the same product lot.

[0005] The parts referred to here include at least one of the wings, legs, thorax, abdomen, head (which may include antennae), and shell.

[0006] When such unexpected parts are found, since they no longer retain their original shape, it is difficult to identify the original shape or type of animal from these parts. Therefore, these unexpected parts are provided as samples to public testing and research institutions established in each prefecture, or analytical institutions that perform analysis using analytical equipment, and a request is made to identify the type of animal that contains these parts. These parts are then analyzed using various analytical equipment, and the problems are resolved or consumer complaints are responded to after the results are obtained.

[0007] However, the act of requesting an external organization to analyze the unexpected parts entails time, psychological, material, and cost issues, such as the fact that it takes a certain amount of time to obtain the results, or further time is required to resolve the problem, or it is not possible to determine in which process the unexpected part and the corresponding part occurred (became mixed in), which means that production activities cannot be carried out during that time, products cannot be sold, and consumer concerns cannot be alleviated.

[0008] In particular, when the unexpected part is expected to be an arthropod or gastropod, the response will differ depending on whether the arthropod or gastropod containing the unexpected part is one that is likely to occur within the food factory (sometimes referred to as endogenous) or one that is likely to invade from the outside (sometimes referred to as exoinvasive), so it has been necessary to quickly determine the type of arthropod or gastropod (sometimes referred to as class, order, or family) that contains the part.

[0009] For this reason, there has been a strong demand for the ability to quickly identify at least the type of unexpected area, including fractures that were not caused by arbitrary processing.

[0010] Furthermore, when identifying common arthropod and gastropod species, it takes about a week for analytical laboratories to report the results after receiving the sample, so there was a need to develop a determination method that differs from common determination methods, as well as an arthropod and gastropod determination system that can be used at production sites and is equipped with this determination method. Summary of the Invention [Problem to be solved by the invention]

[0011] The present invention has been made to solve these problems, and aims to provide an arthropod and gastropod identification system that can determine in a short period of time the type of arthropod or gastropod that contains a sample part from the sample part, and to provide this arthropod and gastropod identification system as something that can be used on the spot when discovered by production sites, inspectors, and even consumers. [Means for solving the problem]

[0012] As a result of intensive research to achieve the above object, the inventors have identified a portion including a broken portion other than that which has been arbitrarily processed, and based on the type of arthropod or gastropod including the portion identified by the results of this identification, We realized that we could determine the type of specimen from the appearance of the specimen observed at the food production site and the characteristics of the broken part of the specimen. We have developed a type determination system that extracts information for determining the type of sample, such as shape information obtained from a sample image (sometimes simply referred to as a sample image) including the broken portion and external appearance of the sample, the proportion of the sample image in the image obtained from the sample image, and color information including the so-called RGB values ​​of the sample's colors red (sometimes referred to as R), green (sometimes referred to as G), and blue (sometimes referred to as B), and determines the type of unexpected sample that includes a broken portion other than that which has been arbitrarily processed, thereby completing the present invention.

[0013] That is, the first aspect of the present invention is A system for determining arthropods and gastropods, comprising an image acquisition means, a shape and color information acquisition means, a determination means, a discrimination means for discriminating between internally occurring and externally invasive, a recording means, and a display means, the system including at least a portable terminal device, The image acquisition means is an imaging means of a portable terminal device, or a means for acquiring a sample image by selecting an already stored image, the shape and color information acquisition means is a means for acquiring at least contour information, broken portion information, and red (R), green (G), and blue (B) color information from the sample image; the determination means uses a plurality of reference object images obtained by imaging a plurality of reference objects whose types are known for each of the reference objects, and uses a trained model generated by machine learning using shape and color information obtained from the reference object images by at least the shape and color information acquisition means as feature quantities to preferentially select a highly relevant reference object image based on the shape and color information of the reference object image and a type linked to the highly relevant reference object image according to the shape and color information obtained from the sample image, and determines the type of the sample as at least one of class, order, and family; The distinguishing means is a means for distinguishing whether the animal determined by the determining means is at least endogenous or exogenous, the recording means is a means for recording the sample image, at least one of the types of sample (net, order, or family), the determination rate for each type of sample, and the classification result in association with each other; This is an arthropod and gastropod judgment system, characterized in that the display means is a means for displaying at least the type of specimen determined by the judgment means and the separation result determined by the separation means, whether the specimen is endogenous or exogenous, on the display unit of a portable terminal device or on a display terminal.

[0014] The effects of the first invention are: Even if the specimen contains broken parts that have not been arbitrarily processed, the class, order or family of the specimen can be determined in a short time from a specimen image containing shape and color information of the specimen, using a reference object image containing shape and color information such as broken parts and contours that have not been arbitrarily processed of a reference object of known class, order or family that has been accumulated in advance as training data for machine learning, the shape and color information of the reference object image, the type of reference object based on the results of scientific analysis of the reference object, and a determination means including artificial intelligence constructed based on these, and from this determination it can be distinguished whether the disease is internally occurring or externally invasive.

[0015] The second feature of the present invention is The portable terminal device is Image acquisition means; a) acquiring at least contour information, break information, and red (R), green (G), and blue (B) color information of the sample image; b) using the trained model to determine the type of sample based on the shape and color information of the sample obtained from the sample image; c) recording the sample image, the type of sample, and the determination rate in association with each other; A program for executing the above steps a) to c); A means for recording, in association with each other, a sample image of the sample determined by the program, at least one of a class of class, order, or family as the type of the sample, a determination rate for each type of sample, and a classification result; A means for displaying on a display unit a sample image of the sample determined by the program, at least one of a mesh, an order, and a family as the type of the sample, a determination rate for each type of sample, and a classification result; The present invention relates to a system for determining arthropods and gastropods, comprising:

[0016] A second advantage of the present invention is that the arthropod and gastropod determination system can be executed on a portable terminal device.

[0017] The third feature of the present invention is This is a determination system for arthropods and gastropods, characterized in that the specimen and reference are solid at room temperature, and the specimen and reference are selected to include at least one of the following parts: wings, legs, thorax, abdomen, head (which may include antennae), and shell.

[0018] A third advantage of the present invention is that it is possible to determine what type of arthropod or gastropod a solid specimen contains by including at least one of the following parts: wings, legs, thorax, abdomen, head (which may include antennae), and shell.

[0019] The fourth feature of the present invention is If the ratio of the area of ​​the sample image to the total image used for sample identification is 5% or more and 100% or less, the type of sample identification is at least: If the beetle is determined to belong to any of the following families of arthropod beetles: Staphylinidae, Dermestidae, Anobiidae, Dermestidae, Tribolidae, Bulbula, Buprestidae, or Pycnonotidae, then it is determined that there is a high possibility of internal development. If it is determined that the beetle belongs to any of the following families of arthropod beetles: Chrysomelidae, Flea Beetles, Bostrichidae, Coccinellidae, Elateridae, Scarabaeidae, Carabidae, Curculionidae, Nitidulidae, Ectocoridae, Scleractinidae, Bucconidae, Water Beetles, or Coccinellidae, it is determined that there is a high possibility of external invasiveness. If the fly is determined to belong to the arthropod order Zygidae, Drosophilidae, Phoridae, Mucoridae, Sphaeroceridae, or Drosophilidae, then it is determined that there is a high possibility of endogenous development. If the fly is determined to belong to any of the following families of arthropod flies: Chironomidae, Sciaridae, Cecidomyiidae, Crane flies, Culicidae, Mycetidae, Ceratopogonidae, Agromyzidae, Solanaceae, Muscidae, Flesh flies, Calliphoridae, Heteroptera, Anthomyiidae, Shoreflies, Shoreflies, Scytheridae, or Syrphidae, it is determined to have a high possibility of being externally invasive. If the species is determined to be either a Pyralidae or a Geometridae in the arthropod order Lepidoptera, it is determined to be highly likely to be endogenous. If the species is determined to be of the family Pieridae, Lycaenidae, or Papilionidae in the arthropod order Lepidoptera, it is determined to have a high possibility of being externally invasive. If the specimen is determined to belong to the arthropod order Hymenoptera, such as the family Amphitheatre or Formicidae, it is determined that there is a high possibility of endogenous development. If the species is determined to be of the arthropod Hymenoptera family, such as Chalcedony, Apidae, Bucconidae, Ichneumonidae, Sawfly, Anthophyllum, Cynipidae, Vespidae, Braconidae, Polistidae, or Formicidae, it is determined to have a high possibility of being externally invasive. If it is determined to be an arthropod dragonfly of the family Odonata, it is determined to have a high possibility of being an external invasive species. If the insect is determined to belong to the Hemiptera or Cicadellidae suborder of the arthropod family, either the Hemiptera, Triatomine, Miridae, or Aphididae families, it is determined to have a high possibility of being externally invasive. If the insect is determined to be either a member of the family Ephemeroptera or a member of the family Ephemeroptera, it is determined to have a high possibility of being externally invasive. If it is determined to be of the family Chrysopidae in the order Nerium, it is classified as externally invasive. If it is determined to be in the family Plecoptera of the arthropod order, it is classified as having a high possibility of being an external invasive species. If it is determined that the plant belongs to the arthropod order Isopoda, either the family Armadillididae or the family Armadillididae, it is determined that there is a high possibility that it is an external invasive plant. If the insect is determined to be of the family Orthoptera, Tetrigidae, Acrididae, Scleractinidae, Gryllidae, or Cricketidae, it is determined to have a high possibility of being invasive. If it is determined that the insect belongs to the arthropod Myriapoda, Centipede, or Centipede order, it is determined that there is a high possibility that it is invasive to the outside. If the cockroach is determined to be of the arthropod order Blattidae or Termitidae, it is determined that there is a high possibility of internal development. If it is determined to be in the family Psocidae of the arthropod order Psociridae, it is determined that there is a possibility of internal development. If it is determined to be an arthropod thrips, it is determined to have a high possibility of being externally invasive. If it is determined to be an arthropod caddisfly, it is determined to have a high possibility of being invasive. If it is determined to be an arthropod Hemiptera, it is determined to have a high possibility of being externally invasive. If it is determined to be an Arachnida of the Arthropod order, it is determined to have a high possibility of being an external invasive species. If it is determined to be an arthropod of the order Centipedia, it is determined to have a high possibility of being invasive. If it is determined to be an arthropod millipede, it is determined to have a high possibility of being invasive. If it is determined to be an arthropod centipede, it is determined to have a high possibility of being an external invasive species. If it is determined to be a member of the Gastropoda class, it is determined to be highly likely to be invasive. This is an arthropod and gastropod identification system that is characterized in that if the area of ​​the sample image used to identify the sample accounts for less than 5% of the total image, it is determined that the object is too small to be identified.

[0020] A fourth advantage of the present invention is that, as a feature, when the specimen image contained in the image is large and the specimen can be identified, a determination is made and it is determined whether the arthropod or gastropod containing the specimen is likely to be endogenous or likely to be exogenous, and when the specimen image is small and the type cannot be identified, a feature is added to determine that the type cannot be determined, thereby making it possible to prevent animals containing specimens from being mixed into products. Effect of the Invention

[0021] This arthropod and gastropod identification system (sometimes referred to as an identification system or a species identification system) has the effect of enabling simple and rapid identification of the species of arthropods and gastropods containing specimens that are unexpected substances. [Brief description of the drawings]

[0022] [Figure 1] 1 is a flowchart of a determination process of the determination system of the present invention. [Diagram 2] 1 is a schematic diagram showing the positional relationship between a portable terminal device included in the determination system of the present invention and a sample during imaging. [Diagram 3] The type of reference object image used in the machine learning of this determination system is known and is displayed on the image display means of the portable terminal device of the present invention. [Figure 4] 2 is a diagram showing an information input screen before determination of the type determination system of the present invention. [Diagram 5] 2 is an example of a specimen to be determined by the type determination system of the present invention. [Figure 6] This shows the results of Example 1 in which a specimen (head, thorax, and wings) was analyzed in the species determination system of the present invention, and the analysis results were displayed on the display means of a smartphone. [Figure 7] This shows the results of Example 2 in which a specimen (wing) was analyzed in the species determination system of the present invention, and the analysis results were displayed on the display means of a smartphone. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] The present invention will now be described in more detail with reference to the drawings.

[0024] Since the determination of the type of specimen according to the present invention is carried out according to the flow chart shown in FIG. 1, the flow chart of FIG. 1 will first be described. The flowchart in Figure 1 is used to determine the type of specimen. This determination system executes the first to fifth steps in the flowchart in Figure 1 in order. The specimen image in Figure 5 is a site that was actually recognized as a foreign body at the production site, and infrared spectroscopic analysis revealed that this type was made of protein.

[0025] <First step> In the first step, the user images the specimen 5 to be determined, which is placed on the holding table 7, using the image acquisition means 2 of the portable mobile terminal 1 (e.g., a smartphone, etc.) in FIG. 2. The light source used for imaging is not particularly limited, but a light source that can be received by the image acquisition means 2, such as natural light, a fluorescent lamp, an LED light source, or a near-infrared light source, can be preferably used. The holding table 7 on which the specimen 5 is placed is not particularly limited, but it is preferable that the holding table 7 is the same as the holding table 7 used when imaging the reference object 10 used as the teacher data as shown in FIG. 3. The holding table 7 may be placed still when the specimen 5 is imaged by the image acquisition means 2. It is more preferable that the holding table 7 is flat. It is more preferable that the holding table 7 does not reflect light. If the holding table 7 is made of a material that reflects light, it can be preferably used by adjusting the angle so that the reflected light does not affect the imaging of the specimen 5 and the reference object 10 by the image acquisition means 2. The holding table 7 can also be preferably used with an object that can hold the specimen 5, such as a finger or tweezers. In this case, the part pinched with the fingers or tweezers may or may not be included in the image 6. Note that FIG. 3 shows an image of an arthropod belonging to the family Chrysopidae of the order Nervipidae as an example of the reference object 10.

[0026] The captured specimen image 13 of specimen 5 and image 6 including the image of the holding stage 7 are digitized and displayed on the display unit 4 of a portable mobile terminal described later as image 6 including shape and color information of specimen image 13 by a programmed shape and color information acquisition means. FIG. 5 shows an image 6 including specimen image 13 captured by image acquisition means 2 on the display unit 4 of a specimen 5 that was actually recognized as a foreign body in food, after it was placed on a flat ledger paper that is the holding stage 7. This specimen 5 includes at least a head-like specimen 5a with a radius of about 3 mm, a thorax-like specimen 5b that is connected to the head-like specimen 5a and is about 5 mm long and 2.5 mm wide, a thin leg-like specimen 5c with about three legs connected to the thorax-like specimen 5b, and a wing-like part 5d that is connected to the thorax-like specimen 5b and has veins about 15 mm long or more and 3 mm to 10 mm wide. A broken portion is observed in specimen image 13, which appears to be a breast portion. In this case, the contour information indicates a contour line 8 that distinguishes specimen image 13 included in image 6 from images other than specimen image 13, and also indicates the break information of specimen image 13. As shown in FIG. 5, the contour of this broken portion is sometimes referred to as a break contour 9. This break contour 9 occurs on the contour line 8 (sometimes referred to as a boundary line) between fracture surface 12 and holding table 7, and also inside specimen image 13. The inside of specimen image 13 may include the inside of contour line 8, i.e., the break contour 9 may exist in a place that does not contact contour line 8 of specimen image 13.

[0027] The user can decide whether or not to use this image 6 as the specimen image 13 of the specimen 5. If the image 6 is to be used as the specimen image 13 of the specimen 5, the process proceeds to the next second step. If the image 6 is not to be used, the user can select to capture the specimen 5 again. If the image 6 is to be used as the specimen image 13, the user can of course write the shape and color information felt by the user as a user comment related to the specimen 5 in the comment field 3 of the display unit 4 shown in FIG. 4. In this case, the color information can be written as blue or red, the size of the specimen 5 can be written as shape information, and the light transmittance, shape and dimensions can be written. Furthermore, physical information such as hardness, softness, magnetic property, weight, the collection location of the specimen 5, the collection date and time, the impressions of the collector, what kind of thing the specimen 5 was found near, etc. can be preferably written as user comments. The state of the broken part can also be preferably written as shape information. The state of the broken part can also be preferably written as sharp, stretched, soft, hard, etc. In this case, it is preferable because it becomes easier for the analyst (sometimes referred to as the user) to judge whether the result of the judgment by the judgment means described later is strange or not. It is also preferable to select and use such comments as features by multi-layer analysis of deep learning. In addition, when using them as features, they are used as information when making judgments by artificial intelligence, which is preferable.

[0028] If it is selected to capture an image again, then in the first step, the image 6 is captured again of the specimen 5. This image 6 is not used as the specimen image 13 when, for example, the acquired specimen image 13 is out of focus, when the image capturing means 2 is subjected to vibration or movement during the capture and the specimen image 13 is blurred, when the specimen image 13 is not included in this image 6, or when the specimen 5 is mistakenly recognized as another substance and captured in the first place.

[0029] The capacity of this image 6 is not particularly limited, but a capacity of 30 kilobytes or more is preferably used. If the capacity is too small, the image 6 becomes rough and it may be difficult to determine the type. If the capacity of this image 6 is large, it is preferably used, but if the capacity of the image 6 exceeds 1 megabyte, it may take a long time to save the image 6, which will be described later.

[0030] The specimen image 13 included in the image 6 preferably includes a fractured portion other than that which has been arbitrarily processed. The fractured portion information is composed of at least one of a fractured contour 9 and a fractured surface 12. FIG. 5 shows the contour line 8, the fractured contour 9, and the fractured surface 12 of the specimen image 13 included in the image 6. The specimen image 13 in FIG. 5 has a fractured contour 9 inside the contour line 8, i.e., in a place not in contact with the contour line 8 of the specimen image 13. The specimen image 13 is included in the image 6 and does not include an image of the holding table 7. If the fractured portion exists inside the specimen image 13, it is also preferable to express the fractured portion as a fractured surface 12. When the area of ​​the image 6 is 100, the area of ​​the fractured surface 12 is preferably 0.1 to 100, more preferably 1 to 100, and even more preferably 4 to 90. If the area of ​​the fractured surface 12 is too small, it may be difficult to obtain information about the fractured surface 12. When the area of ​​the fracture surface 12 is 100, the image 6 does not include the holding table 7, and the specimen image 13 is only the fracture surface 12. In this case, it is also preferable to show the color distribution of the fracture surface 12 from the color information of the specimen image 13 like the contour lines of a map, and obtain the shape of the fracture surface 12 as the fracture contour 9 from this color distribution. In this way, the fracture portion information may consist of the fracture contour 9 and the fracture surface 12.

[0031] If the specimen 5 is small and it is difficult to identify the specimen 5 when capturing the image 6, it is preferable to enlarge and capture the specimen 5 using a microscope, a microscope, a magnifying lens, or the like. Furthermore, a saved image previously captured and recorded in a portable mobile terminal or other computer can be acquired by an image acquisition means and preferably used as the image 6. If the image 6 has a sufficient capacity and resolution for judgment, it is preferable to enlarge the specimen image 13 of the saved image and use it for judgment. When the length of the outer frame of the image 6 is 100, if the length of the broken contour 9 of the specimen image 13 is relatively short, such as about 0.1, it may be difficult to judge. The specimen 5 may be a part of the leg or wing of an arthropod, a head, or an antenna. When the length of the outer frame of the image 6 is 100, the broken contour 9 of the antenna may be less than 0.1. In such a case, it is preferable to capture the image using a magnifying lens so that the length of the broken contour 9 is 0.1 or more relative to the length of the outer frame of the image 6, which is 100. By using the enlarged specimen image 13 in this manner, it becomes easier to distinguish between the heads of Coleoptera and Hymenoptera.

[0032] <Second step> When the image 6 obtained by the image acquisition means 2 of the portable mobile terminal 1 is selected to be used for the judgment, shape and color information is acquired by a shape and color information acquisition means in the portable mobile terminal for the sample image 13 contained in this image 6. The shape and color information acquisition means is preferably a means consisting of a program stored in the portable mobile terminal, and is also preferably a program stored in a specific server, and the shape and color information of the image 6 is acquired by communication with the program in this server.

[0033] When the contour 8 includes a fracture contour 9 in the sample image 13, the shape information contained in this shape and color information includes the roughness of the fracture contour 9 and its thinness compared to other parts. Since this shape information includes information about the fractured part, this shape and color information can also be used as information about the fractured part when the shape of the fractured part is irregular or the color is non-uniform. Even when the color is less non-uniform and the color is close to a single color, or when the shape is irregular, it can be preferably handled as information about the fractured part.

[0034] As color information contained in this shape and color information, RGB values ​​and saturation of the specimen image 13 can be preferably obtained. The numerical range of the RGB values ​​can be classified into 256 types, if a dark color is set to 0 and a light color is set to 255 for each of the colors R (red), G (green), and B (blue). It is preferable to obtain the area value of the specimen image 13 contained in the image 6 from the shape and color information consisting of this color information and shape information. If the specimen image 13 contained in the image 6 has a fracture surface 12, it is also preferable to obtain the shape of the fracture surface 12 as shape information based on this color information.

[0035] The shape and color information acquisition means is preferably executed by a program. It is also preferable to select an image 6 that was previously captured and stored in the portable mobile terminal, a specific personal computer, a server, or the like, and acquire shape and color information for the sample image 13 contained in the image 6. This shape and color information can be preferably used as a feature value in the multi-layer analysis of deep learning.

[0036] <Third process> The third step is a step of determining the type of the specimen 5 by a determination means. Note that FIG. 3 shows an image of an arthropod belonging to the family Chrysopidae of the order Neraptera as an example of the reference object 10 used in machine learning. The determination means uses a plurality of reference object images 14 acquired by imaging a plurality of reference objects 10 of known types, each of which includes at least a broken portion, for each of the reference objects 10, and uses a trained model generated by machine learning that uses at least shape and color information obtained from the reference object images 14 by the shape and color information acquisition means as a feature amount to preferentially select a highly relevant reference object image 14 and a type linked to the highly relevant reference object image 14 based on the shape and color information of the reference object image 14 according to the shape and color information obtained from the specimen image 13 obtained by imaging the specimen 5, and determines the type of the specimen 5. Note that the determination may also be referred to as estimation.

[0037] This determination means can be preferably executed by an application or program stored in this portable mobile terminal 1. In addition, this determination means can also be preferably utilized by a computer or server that is capable of communicating using communication technology.

[0038] The part of the reference object 10 used in this determination means preferably includes at least one of the wings, legs, thorax, abdomen, head (which may include antennae), and shell. This part can be preferably used as training data for machine learning.

[0039] When the arthropods and gastropods determined by this determination means are determined to belong to the arthropod beetle order Coleoptera, to any of the families Staphylinidae, Dermestidae, Anobiidae, Dermestidae, Tribolidae, Platyceridae, Buprestidae, or Pycnonotidae, they are classified as having a high possibility of internal development, along with these arthropod families.

[0040] If the arthropod beetle order is determined to be one of the following families: Chrysomelidae, Flea Beetles, Bostrichidae, Coccinellidae, Elateridae, Scarabaeidae, Carabidae, Curculionidae, Nitidulidae, Ectocoridae, Scleractinidae, Bucconidae, Water Beetles, or Coccinellidae, it is classified as having a high possibility of being externally invasive, along with these arthropod families.

[0041] If the fly is determined to belong to the arthropod order Psychodidae, Drosophilidae, Phoridae, Malariae, Sphaeroceridae, or Drosophilidae, it is classified as having a high possibility of endogenous development along with these arthropod families.

[0042] If the species is determined to be of the arthropod order Fly, Chironomidae, Sciaridae, Cecidomyiidae, Crane flies, Culicidae, Mycetidae, Ceratopogonidae, Agromyzidae, Solanaceae, Muscidae, Flesh flies, Calliphoridae, Heteroptera, Anthomyiidae, Shoreflies, Cicadas, or Syrphidae, it is classified as having a high possibility of being an external invasive species, along with these arthropod families.

[0043] If the species is determined to be in the arthropod order Lepidoptera, either the Pyralidae or the Geometridae, it is classified as having a high possibility of endogenous development along with these other arthropod families.

[0044] If the species is determined to belong to the arthropod order Lepidoptera, Pieridae, Lycaenidae, or Papilionidae, it is classified as having a high possibility of being externally invasive along with these arthropod families.

[0045] If the specimen is determined to belong to the arthropod Hymenoptera family, such as the Acaridae or Formicidae, it is classified as having a high possibility of endogenous development, along with these other arthropod families.

[0046] When the species is determined to belong to the arthropod Hymenoptera family, including Chalcedonychidae, Apidae, Bucconidae, Ichneumonidae, Sawfly, Anthophyllum, Cynipidae, Vespidae, Braconidae, Polistidae, or Formicidae, it is classified as having a high possibility of being invasive, along with these arthropod families.

[0047] If an insect is determined to belong to the family Odonata in the arthropod order Odonata, it is classified as having a high possibility of being invasive to other arthropod families.

[0048] When the insect is determined to belong to the arthropod order Hemiptera or suborder Cicadellidae, either the family Hemiptera, Triatomineidae, Miridae, or Aphididae, it is classified as having a high possibility of being externally invasive, along with these arthropod families.

[0049] If the species is determined to belong to the arthropod Ephemeroptera order, either the Ephemeroptera family or the Ephemeroptera family, it is classified as having a high possibility of being externally invasive, along with these arthropod families.

[0050] If it is determined to be a member of the arthropod order Nerium and the family Chrysopidae, it will be classified as an external invasive species along with other arthropod families.

[0051] If an insect is determined to belong to the Plecoptera family of arthropods, it is classified as having a high possibility of being invasive, along with other arthropod families.

[0052] If it is determined that the insect belongs to the arthropod order Isopoda and belongs to either the family Armadillididae or the family Vulgare, it is classified as having a high possibility of being externally invasive, along with these arthropod families.

[0053] If the insect is determined to be in the Orthoptera order of arthropods, either Tetrigidae, Acrididae, Scutellaria, Gryllidae, or Cricketidae, it is classified as having a high possibility of being invasive to other species, along with these arthropod families.

[0054] If the specimen is determined to be an arthropod myriapod, belonging to the class Millipedes, Centipedes, or order Centipedia, it is classified as having a high possibility of being invasive to the outside, along with these arthropod families.

[0055] If the cockroach is determined to be in the arthropod order Blattella, either the family Blattidae or the family Termitidae, then it is classified as having a high possibility of endogenous development along with these arthropod families.

[0056] If it is determined to be a member of the arthropod order Psociridae, it is classified as possibly endogenous, along with other arthropod families.

[0057] If it is determined to be an arthropod of the order Thripida, it is classified as having a high possibility of being externally invasive, along with other arthropod families.

[0058] If it is determined to be an arthropod caddisfly, it will be classified as having a high possibility of being invasive, along with other arthropod families.

[0059] If it is determined to be an arthropod of the order Hemiptera, it will be classified as having a high possibility of being externally invasive, along with other arthropod families.

[0060] If it is determined to be an Arachnida order of arthropods, it is classified as having a high possibility of being externally invasive, along with other arthropod orders.

[0061] If it is determined to be an arthropod order, Centipedia, it will be classified as having a high possibility of being invasive to the outside world, along with other arthropod orders.

[0062] If it is determined to be an arthropod of the class Millipede, it will be classified as having a high possibility of being an external invasive species, along with other arthropod classes.

[0063] If it is determined to be an arthropod of the class Centipede, it will be classified as having a high possibility of being invasive, along with other arthropod species.

[0064] If it is determined to be a member of the Gastropoda class, it will be classified as having a high possibility of being an external invasive species, along with other arthropod classes.

[0065] This is an arthropod and gastropod identification system that is characterized in that if the area of ​​the sample image used to identify the sample accounts for less than 5% of the total image, it is determined that the sample is unidentifiable.

[0066] <Fourth step> This is a step of storing the image 6, the type of animal containing the specimen 5, the sorting result, and the matching rate in association with each other. In this fourth step, it is preferable to store and save the image 6 in a user terminal such as a portable mobile terminal, a specific personal computer, or a server, or it is preferable to store and save a screen displayed on a display unit 4 of a user terminal such as a portable mobile terminal or a personal computer, which shows the image 6, the determination result of the type of animal containing the specimen 5, the sorting result, and the matching rate, which will be described later.

[0067] <The fifth step> This is a process in which the image 6, the determination result of the type of animal containing the specimen 5, the separation result, and the match rate are displayed on a display unit 4 of a user terminal such as a portable mobile terminal, a specific personal computer, or a server.

[0068] The present invention will be described in detail below with reference to examples. However, the present invention is not limited to these examples, and various modifications can be made without departing from the scope of the present invention.

[0069] In the embodiment, a smartphone manufactured by Apple Inc. was used as the portable mobile terminal 1. Application software (sometimes referred to as application) for the type determination system incorporating a trained model was created in advance and stored in the smartphone. The application was started, and an image of the specimen 5 was acquired using the camera mounted on the smartphone as the image acquisition means 2.

[0070] <Example 1> As shown in FIG. 6, an image 6 was acquired for a specimen 5 contained in food, and the specimen was judged by this type judgment system. The specimen 5 shown in FIG. 5 was used for this specimen 5. A flat ledger paper was used as the surface of the holding table 7. As a result of judgment by the arthropod and gastropod judgment system of the present invention, the type of this specimen 5 was judged to be an arthropod belonging to the family Chrysopidae of the order Neratii with an 86% probability. In addition, since this arthropod has a high possibility of invading from the outside, it was classified as "outside", indicating external invasiveness, together with this arthropod family. For this specimen 5, in addition to the family Chrysopidae of the order Neratii, the possibility of Odonata, ... In addition, these results could be quickly shared by saving the results displayed on the display means as a report image and sending the information by e-mail. This confirmed the usefulness of the arthropod and gastropod determination system of the present invention.

[0071] <Example 2> As shown in FIG. 7, image 6 was acquired for specimen 5 that was found to be a foreign object in food, and the specimen was judged to be a foreign object by the type judgment system. The surface of the holding table 7 was made of ledger paper. As a result, the type of specimen 5 was judged to be an arthropod belonging to the family Chrysopidae of the order Neratiidae with a 67% probability. In addition, since this arthropod is highly likely to have invaded from the outside, it was classified as "outside", indicating external invasiveness, together with the family of arthropods. In addition to the family Chrysopidae of the order Neratiidae, the possibility of specimen 5 being 6% of the family Odonata, Odonata, Odonata (external invasiveness) and 4% of the family Ichneumonidae, Hymenoptera, Ichneumonidae (external invasiveness) was suggested, but all were judged to be external invasiveness. Based on this judgment result, measures to reduce intrusion into the food factory could be taken by hanging insect nets on windows, etc. to prevent intrusion from the outside. In addition, these results, which were displayed on the display means, could be saved as a report image, and the information could be quickly shared via email. From this, the usefulness of the type judgment system was confirmed.

[0072] <Example 3> From sample 5 shown in Figure 6 and sample 5 shown in Figure 7, it was determined that the arthropods containing these samples were Chrysopidae of the order Neroptera. These samples were found in the same factory, and judging from the shape and size of each part, it was inferred that these parts came from the same individual. Therefore, by discarding only the lot containing these parts, the problem of arthropods containing these parts was solved. [Industrial Applicability]

[0073] INDUSTRIAL APPLICABILITY The present invention relates to food safety, which is the basis of our lives, consumer peace of mind, and reduction of food waste, and has high industrial applicability. [Explanation of symbols]

[0074] 1 Portable terminal 2. Image Acquisition Method 3. Information input section 4 Display section 5. Specimens 5a Head-like specimen 5b Chest-like specimen 5c Leg-like specimen 5d Wing-like specimen 6 Images 7 Holding stand 8 Contour 9 Breaking Contour 10 References 12 Fracture surface 13 Sample images 14 Reference Image

Claims

1. A system for determining arthropods and gastropods, comprising an image acquisition means, a shape and color information acquisition means, a determination means, a discrimination means for discriminating between internally occurring and externally invasive, a recording means, and a display means, the system including at least a portable terminal device, the image acquisition means is an imaging means of the portable terminal device, or a means for acquiring a sample image by selecting an already stored image, the shape and color information acquisition means is means for acquiring at least contour information, broken portion information, and red (R), green (G), and blue (B) color information from the sample image; the determination means uses a plurality of reference object images acquired by imaging a plurality of reference objects whose types are known for each of the reference objects, and uses a trained model generated by machine learning that uses at least shape and color information acquired from the reference object images by the shape and color information acquisition means as feature amounts to preferentially select a reference object image that is highly related based on the shape and color information of the reference object images and a type linked to the highly related reference object image according to the shape and color information acquired from the sample image, and determines the type of the sample as at least one of class, order, and family; The distinguishing means is a means for distinguishing whether the animal determined by the determining means is at least endogenous or exogenous, the recording means is a means for recording the sample image, at least one of a class ... The arthropod and gastropod judgment system is characterized in that the display means is a means for displaying at least the type of the specimen determined by the judgment means and, as the separation result determined by the separation means, at least whether the specimen is endogenous or exogenous, on the display unit of a portable terminal device or on a display terminal.

2. The portable terminal device is The image acquisition means; a) acquiring at least the contour information, the broken portion information, and the color information of red (R), green (G), and blue (B) of the sample image; b) using the trained model to determine the type of the sample according to the shape and color information of the sample obtained from the sample image; c) recording the sample image, the type of the sample, the determination rate, and the separation result in association with each other; A program for executing the above steps a) to c); a means for recording, in association with each other, a sample image of the sample determined by the program, at least one of a class ... a means for displaying on a display unit a sample image of the sample determined by the program, at least one of a class ... The arthropod and gastropod determination system according to claim 1, further comprising:

3. The arthropod and gastropod determination system of any one of claims 1 to 2, characterized in that the specimen and the reference are solid at room temperature, and the specimen and the reference are selected so that they include at least one of the following: wings, legs, thorax, abdomen, head (which may include antennae), and shell.

4. When the ratio of the area of ​​the sample image to the entire image used for the sample determination is 5% or more and 100% or less, the type of determination of the sample and the classification result are If the beetle is determined to belong to any of the following families of arthropod beetles: Staphylinidae, Dermestidae, Anobiidae, Dermestidae, Tribolidae, Buprestidae, Buprestidae, or Pycnonotidae, then it is determined that there is a high possibility of internal development. If it is determined that the beetle belongs to any of the following families of arthropod beetles: Chrysomelidae, Flea Beetles, Bostrichidae, Coccinellidae, Elateridae, Scarabaeidae, Carabidae, Curculionidae, Nitidulidae, Ectocoridae, Scleractinidae, Bucconidae, Water Beetles, or Coccinellidae, it is determined that there is a high possibility of external invasiveness. If the fly is determined to belong to the arthropod order Zygidae, Drosophilidae, Phoridae, Mucoridae, Sphaeroceridae, or Drosophilidae, then it is determined that there is a high possibility of endogenous development. If the fly is determined to belong to any of the following families of arthropod flies: Chironomidae, Sciaridae, Cecidomyiidae, Crane flies, Culicidae, Mycetidae, Ceratopogonidae, Agromyzidae, Solanaceae, Muscidae, Flesh flies, Calliphoridae, Heteroptera, Anthomyiidae, Shoreflies, Cicadas, Syrphidae, or Syrphidae, it is determined to have a high possibility of being externally invasive. If the species is determined to be either a Pyralidae or a Geometridae in the arthropod order Lepidoptera, it is determined to be highly likely to be endogenous. If the species is determined to be of the family Pieridae, Lycaenidae, or Papilionidae in the arthropod order Lepidoptera, it is determined to have a high possibility of being externally invasive. If the specimen is determined to belong to the arthropod order Hymenoptera, such as the family Amphitheatre or Formicidae, it is determined that there is a high possibility of endogenous development. If it is determined that the insect belongs to the arthropod order Hymenoptera, such as Chalcidae, Apidae, Bucconidae, Ichneumonidae, Sawfly, Anthophyllum, Cynipidae, Vespidae, Braconidae, Polistidae, or Formicidae, it is determined that there is a high possibility of external invasiveness. If it is determined to be an arthropod dragonfly of the family Odonata, it is determined to have a high possibility of being an external invasive species. If the insect is determined to belong to the Hemiptera or Cicadellidae suborder of the arthropod family, either the family Hemiptera, Triatomine, Miridae, or Aphididae, it is determined to have a high possibility of being externally invasive. If the insect is determined to be either a member of the family Ephemeroptera or a member of the family Ephemeroptera, it is determined to have a high possibility of being externally invasive. If it is determined to be of the family Chrysopidae in the order Nerium, it is classified as externally invasive. If it is determined to be in the family Plecoptera of the arthropod order Plecoptera, it is determined to have a high possibility of being invasive. If it is determined that the plant belongs to the arthropod order Isopoda, either the family Armadillididae or the family Armadillididae, it is determined that there is a high possibility of external invasiveness. If the insect is determined to be of the family Orthoptera, Tetrigidae, Acrididae, Scleractinidae, Gryllidae, or Cricketidae, it is determined to have a high possibility of being invasive. If it is determined that the insect belongs to the arthropod Myriapoda, Centipede, or Centipede order, it is determined that there is a high possibility that it is invasive to the outside. If the cockroach is determined to be of the arthropod order Blattidae or Termitidae, it is determined that there is a high possibility of internal development. If it is determined to be in the family Psocidae of the arthropod order Psociridae, it is determined that there is a possibility of internal development. If it is determined to be an arthropod thrips, it is determined to have a high possibility of being externally invasive. If it is determined to be an arthropod caddisfly, it is determined to have a high possibility of being invasive. If it is determined to be an arthropod Hemiptera, it is determined to have a high possibility of being externally invasive. If it is determined to be an Arachnida of the Arthropod order, it is determined to have a high possibility of being an external invasive species. If it is determined to be an arthropod of the order Centipedia, it is determined to have a high possibility of being invasive. If it is determined to be an arthropod millipede, it is determined to have a high possibility of being invasive. If it is determined to be an arthropod centipede, it is determined to have a high possibility of being an external invasive species. If it is determined to be a member of the Gastropoda class, it is determined to be highly likely to be invasive. An arthropod and gastropod identification system as described in any one of claims 1 to 3, characterized in that if the area of ​​the sample image used to identify the sample accounts for less than 5% of the entire image, it is judged to be impossible to identify.

Images