Information processing device and program

Abstract

To provide an information processing device suitable for ranking objects. [Solution] The information processing device performs at least one of the following: a first inspection, which checks how much the shape of a first part included in the object differs from a predetermined first shape based on image data of the object; and a second inspection, which checks how much the shape of a second part that differs from the first part included in the object differs from a predetermined second shape. Based on the results of the performed inspections, the device ranks the object.

Description

Technical Field

[0001] This disclosure relates to an information processing apparatus and a program.

Background Art

[0002] In recent years, the labor shortage associated with the decrease in the agricultural workforce has become a serious problem. In order to improve the labor-intensive fruit selection work, for example, attempts to automate fruit selection and realize smart agriculture by utilizing technologies such as robotics and AI (Artificial Intelligence) have been made everywhere.

[0003] For example, Patent Document 1 describes an apparatus for determining the rank of shiitake mushrooms. The apparatus described in Patent Document 1 ranks shiitake mushrooms by calculating, for example, the ratio of the folds of shiitake mushrooms using image processing technology.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Improvements in technologies for ranking objects such as shiitake mushrooms are constantly required. Therefore, an object of one embodiment of this disclosure is to provide an information processing apparatus and a program suitable for ranking objects.

Means for Solving the Problems

[0006] An information processing device according to one embodiment of the present disclosure performs at least one of a first inspection, which checks how much the shape of a first part included in an object differs from a predetermined first shape based on image data of the object, and a second inspection, which checks how much the shape of a second part included in the object that differs from the first part differs from a predetermined second shape, and ranks the object based on the results of the performed inspections. [Effects of the Invention]

[0007] According to one embodiment of the present disclosure, an information processing device and program suitable for ranking objects are provided. [Brief explanation of the drawing]

[0008] [Figure 1] This is a perspective view of a fruit sorting apparatus according to one embodiment of the present disclosure. [Figure 2] This is a perspective view of a fruit sorting apparatus according to one embodiment of the present disclosure. [Figure 3] This is a block diagram showing the configuration of a fruit sorting device according to one embodiment of the present disclosure. [Figure 4] This is a block diagram showing the configuration of a terminal device according to one embodiment of the present disclosure. [Figure 5] This flowchart shows the flow of a sorting operation performed using a sorting device in one embodiment of the present disclosure. [Figure 6] This figure shows an example of a display screen in one embodiment of the present disclosure. [Figure 7] This is the subroutine for the inspection process (step S105) shown in Figure 5. [Figure 8] This is an explanatory diagram of the subroutine for the inspection process (step S105) in Figure 5. [Figure 9] This is an explanatory diagram of the subroutine for the inspection process (step S105) in Figure 5. [Figure 10] This is an explanatory diagram of the subroutine for the inspection process (step S105) in Figure 5. [Figure 11] This figure shows an example of a display screen in one embodiment of the present disclosure. [Modes for carrying out the invention]

[0009] The following description relates to an information processing apparatus and program according to one embodiment of the present disclosure. Common or corresponding elements are denoted by the same or similar reference numerals, and redundant descriptions are omitted or simplified as appropriate.

[0010] Figures 1 and 2 are perspective views of a sorting apparatus 1 according to one embodiment of the present disclosure. Figure 3 is a block diagram showing the configuration of the sorting apparatus 1 and the terminal device 2. The sorting apparatus 1 and the terminal device 2 are connected to each other so as to be able to communicate with each other, for example, by wire or wireless.

[0011] In this embodiment, the direction of gravity in the environment in which the sorting device 1 is assumed to be used is defined as downward, the opposite direction of downward is defined as upward, and the direction perpendicular to the vertical direction is defined as the horizontal direction. Furthermore, the two mutually perpendicular horizontal directions are defined as the X direction and the Y direction. The vertical direction perpendicular to both the X and Y directions is defined as the Z direction. That is, the X, Y, and Z directions are mutually perpendicular. The X direction may also be called the left-right direction. The Y direction may also be called the front-back direction. The Z direction may also be called the up-down direction. Note that the directional names are used for convenience to explain the relative positional relationships of the components and do not indicate absolute directions. For example, depending on the orientation of the device, the X direction may not necessarily be the left-right direction, but may be the front-back direction.

[0012] The sorting device 1 comprises a housing 10, a rotating support unit 20, a first drive unit 30, a second drive unit 40, a mounting table 50, and a sensor unit 60. The sorting device 1 sorts agricultural products, which are an example of target products. Agricultural products include food crops, fodder crops, green manure crops, horticultural crops, industrial crops, and special forest products such as mushrooms. Multiple mounting tables 50 are provided on the sorting device 1 in order to efficiently sort the agricultural products P that are to be sorted.

[0013] Any reference to an element using designations such as "first", "second", etc. used in this disclosure does not generally limit the quantity or order of those elements. These designations are used for convenience to distinguish between two or more elements. Therefore, references to the first and second elements do not mean, for example, that only two elements are employed, that the first element must precede the second element, and so on.

[0014] The fruit selection device 1 has a communication interface (not shown). A terminal device 2 is connected to the fruit selection device 1 wirelessly or by wire. The terminal device 2 is an example of a computer. The terminal device 2 is a controller that controls the operation of the fruit selection device 1 and is an example of an information processing device. The terminal device 2 is, for example, a PC (Personal Computer), a smartphone, or a tablet terminal. The terminal device 2 may not be a general-purpose device such as a PC but a dedicated device for the fruit selection device 1.

[0015] In another embodiment, a control unit having the same control function as the terminal device 2 may be incorporated into the fruit selection device 1. In this case, a control unit composed of, for example, LSI (Large Scale Integration) is an example of an information processing device. In addition to the control unit, an input device and a display device may be incorporated into the fruit selection device 1. Also, some components such as the sensor unit 60 may be provided as an external device. That is, the fruit selection device 1 is not limited to the configuration shown in FIG. 1 and the like. There is freedom in the design of the fruit selection device 1, and there are various modes.

[0016] FIG. 4 is a block diagram showing the configuration of the terminal device 2. As shown in FIG. 4, the terminal device 2 includes a processor 210, a memory 220, a storage 230, a communication interface 240, an input device 250, and a display device 260. Each part of the terminal device 2 is connected via a bus 270. Note that FIG. 4 only shows an example of the configuration of the terminal device 2. The terminal device 2 may include other elements (for example, a speaker) not shown in FIG. 4. The terminal device 2 may also have a configuration that does not include some of the elements shown in FIG. 4.

[0017] The processor 210 reads the programs and data stored in the storage 230. The memory 220 is, for example, a RAM (Random Access Memory). The processor 210 comprehensively controls the terminal device 2 by using the memory 220 as a work area.

[0018] The processor 210 is, for example, a single processor or a multi-processor and includes at least one processor. When configured to include a plurality of processors, the processor 210 may be packaged as a single device, or may be composed of a plurality of physically separated devices within the terminal device 2. The processor 210 may be referred to as, for example, a control unit, a CPU (Central Processing Unit), an MPU (Micro Processor Unit), or a MCU (Micro Controller Unit).

[0019] The storage 230 is, for example, a non-volatile semiconductor memory such as a flash memory, an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), a HDD (Hard Disk Drive), or a SSD (Solid State Drive).

[0020] The storage 230 stores a control program 232 for controlling the sorting device 1. The control program 232 is, for example, downloaded from an app store in advance and installed in the terminal device 2. By the processor 210 executing the control program 232, various processes according to an embodiment of the present disclosure are executed.

[0021] The control program 232 includes, for example, an inspection program for inspecting and ranking the agricultural product P. The inspection program may be a program module constituting the control program 232, or may be a program separate from the control program 232.

[0022] Some of the various processes according to one embodiment of this disclosure (such as saving inspection results) may be executed by a device other than terminal device 2 (for example, a cloud server).

[0023] The communication interface 240 is a communication interface with various media. The terminal device 2 is connected to the sorting device 1, external storage, a server on the network, etc., via the communication interface 240.

[0024] The input device 250 includes, for example, a keyboard, mouse, touch panel, operation buttons, microphone, and various sensors. The operator can operate the sorting device 1 by operating the input device 250.

[0025] The display device 260 includes a display and a driver. When the driver operates the display according to a control signal from the processor 210, a screen corresponding to the control signal is displayed. The display may be a touch panel display. The display may be, for example, an LCD (Liquid Crystal Display) or an OLED (Electro-Luminescence) display.

[0026] The display shows various types of information. For example, it shows guidance for workers, inspection results such as rank, the progress of the sorting process, and the final sorting results.

[0027] The operation of the sorting device 1 is outlined below. When the terminal device 2 receives a rotation start operation, it outputs a control signal to the first drive unit 30 to drive the stepping motor and rotate the rotation support unit 20.

[0028] The rotating support unit 20 comprises a base 21 and support members 22. The base 21 is formed in a cylindrical shape centered on a first axis AX1. The support members 22 are supported by the base 21. Multiple support members 22 are provided. The multiple support members 22 are formed at the top of the base 21, extending radially to the sides of the base 21. Each of the multiple support members 22 supports each of the multiple mounting tables 50. The multiple support members 22 are arranged at equal intervals in the circumferential direction of the base 21. Therefore, the mounting tables 50 supported by the support members 22 are also arranged at equal intervals in the circumferential direction of the base 21. When the rotating support unit 20 rotates, the mounting tables 50 supported by the rotating support unit 20 move around the rotating support unit 20. The worker places the crops P one by one onto the mounting tables 50 that are sequentially approaching them due to the rotational movement. "Placing the crops P" can be rephrased as "putting in the crops P".

[0029] In this embodiment, the crop P is a shiitake mushroom. Hereinafter, "crop P" will be referred to as "shiitake mushroom P". To ensure stability on the mounting platform 50, the worker places the shiitake mushroom P upside down (with the cap facing downwards and the stem facing upwards).

[0030] Shiitake mushrooms have a distinctive shape among mushrooms. Specifically, a cap forms and opens at the tip of a short, cylindrical stem. The shape of the cap is hemispherical in its initial stages, but becomes flat as it grows and opens. As the cap opens, the curled portion on the underside of the cap becomes smaller relative to the entire cap, and fine gills begin to appear, extending radially from the center of the cap. The gills are formed at equal intervals and densely packed in the circumferential direction. The diameter of an open cap is, for example, about 40 mm to 100 mm. Thus, shiitake mushroom P, an example of the subject, includes the curled portion of the cap (an example of the first part) and the gill portion (an example of the second part). The gill portion is the part on the underside of the cap where the gills are formed.

[0031] In this embodiment, image analysis processing is performed on the shiitake mushrooms P captured in the image to inspect their external shape (inspect the rate of deformities) and to determine the rank of the shiitake mushrooms P for shipment. As will be described in detail later, in this embodiment, an external shape inspection suitable for ranking objects such as shiitake mushrooms P is performed.

[0032] For example, inspection items may vary depending on the origin and variety. Therefore, this embodiment provides multiple inspection modes. The simplest inspection mode allows for external shape inspection based on image analysis processing. More detailed inspection modes allow for external shape inspection, as well as thickness inspection of the umbrella portion using a measurement sensor described later, and inspection of the degree of umbrella opening using AI.

[0033] The first drive unit 30 is supported inside the base 21. The first drive unit 30 includes a controller, a driver, a stepping motor, and a transmission mechanism. The controller generates a pulse signal in response to a control signal input from the terminal device 2. The driver supplies a drive current to the stepping motor in accordance with the pulse signal. The stepping motor is driven with a rotation amount and rotation speed corresponding to the supplied pulse signal.

[0034] The base 21 is provided with a shaft centered on the first axis AX1. The rotation of the stepping motor is transmitted to the shaft of the base 21 via a transmission mechanism such as gears. As a result, the rotation support section 20 (base 21 and support body 22 supported by the base 21) and the mounting table 50 supported by the support body 22 rotate around the first axis AX1 at a rotation amount and rotation speed controlled by the terminal device 2. The rotation support section 20 and the mounting table 50 rotate clockwise when viewed from above.

[0035] Multiple mounting platforms 50 revolve around the first axis AX1 and circulate around the rotating support unit 20. That is, the mounting platforms 50 circulate along a circulation path CP around the rotating support unit 20. An information acquisition point P0 is set on the circulation path CP. Multiple discharge points P1 to P3 are set downstream of the information acquisition point P0. In this embodiment, as an example, one of three ranks A to C is assigned to the shiitake mushroom P. Therefore, in this embodiment, three discharge points P1 to P3 are set, corresponding to each of ranks A to C.

[0036] The operator can set the number of ranks arbitrarily by operating terminal device 2. The discharge points are set according to the set number of ranks.

[0037] The sensor unit 60 is installed above the information acquisition point P0 of the circulation path CP. The sensor unit 60 includes an illumination light source, a camera, and a measurement sensor. The illumination light source is, for example, a ring-shaped LED (Light Emitting Diode) array light source. The image sensor and measurement sensor are installed side by side in the area surrounded by the ring-shaped illumination light source. The illumination light source irradiates light so that the environment for camera shooting is sufficiently bright. The illumination light source illuminates at least the entire mounting base 50. In this embodiment, for example, a white light source that irradiates white light having an emission spectrum in the visible light region is used. The sensor unit 60 may not include a measurement sensor and may consist only of an illumination light source and a camera.

[0038] The camera includes a lens and an image sensor. The lens has a field of view that can capture the entire mounting platform 50. The image sensor is, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor equipped with a color filter. The image sensor sequentially captures images of the shiitake mushrooms P placed on the mounting platform 50, which is then transported to the information acquisition point P0. The image sensor outputs the captured data to the terminal device 2.

[0039] More specifically, terminal device 2 outputs a control signal to the first drive unit 30 to drive the stepping motor and control its rotation amount (in other words, the movement angle of the mounting platform 50). From the movement angle of the mounting platform 50, terminal device 2 calculates the current position of the mounting platform 50 on the circulation path CP. Based on the calculated current position of the mounting platform 50, terminal device 2 detects the timing at which the mounting platform 50 is moved to the information acquisition point P0. In accordance with the detected timing, terminal device 2 instructs the sensor unit 60 to take an image. As a result, the shiitake mushroom P that has been moved to the information acquisition point P0 is imaged.

[0040] The measuring sensor is, for example, a Time of Flight (ToF) sensor that measures the distance to an object. The ToF sensor irradiates the object with ranging light (e.g., laser light) of a different wavelength than visible light, and calculates the distance to the object based on the time difference between the timing of laser light emission and the timing of light reception at the irradiation position.

[0041] When the mounting platform 50 is empty, the laser light emitted from the ToF sensor strikes the bottom surface of the mounting platform 50, reflects, and is received by the light-receiving part of the ToF sensor. When a shiitake mushroom P is placed on the mounting platform 50, the laser light emitted from the ToF sensor strikes the shiitake mushroom P placed on the mounting platform 50, reflects, and is received by the light-receiving part of the ToF sensor. Therefore, the difference between the time difference in the emission and reception timing of the laser light for an empty mounting platform 50 (in other words, the distance between the ToF sensor and the bottom surface of the mounting platform 50) and the time difference in the emission and reception timing of the laser light for a mounting platform 50 with a shiitake mushroom P placed on it (in other words, the distance between the ToF sensor and the shiitake mushroom P placed on the mounting platform 50) indicates the thickness of the shiitake mushroom P (for example, the thickness of the cap). Based on this difference, the terminal device 2 estimates the thickness of the shiitake mushroom P. The value measured by the measuring sensor is not limited to thickness; it can also be other values ​​(for example, hardness, moisture content, etc.).

[0042] Terminal device 2 may rank the shiitake mushrooms P using not only the image analysis results but also the measurements taken by the measurement sensors.

[0043] The area around the rotating support section 20 and below the circulation path CP is an empty space. Storage containers corresponding to each rank can be installed in this empty space.

[0044] The worker pre-installs storage containers below the discharge points corresponding to each set rank. For example, the worker installs storage containers 3A to 3C, corresponding to the three ranks A to C. Storage container 3A, corresponding to rank A, is installed below discharge point P1 of the circulation route CP. Storage container 3B, corresponding to rank B, is installed below discharge point P2 of the circulation route CP. Storage container 3C, corresponding to rank C, is installed below discharge point P3 of the circulation route CP.

[0045] The second drive unit 40 includes a solenoid connected to the support 22. The support 22 rotates around the second axis AX2 in response to the on / off state of the solenoid. As the support 22 rotates around the second axis AX2, the mounting table 50 supported by the support 22 rotates and tilts around the second axis AX2, causing the shiitake mushroom P placed on the mounting table 50 to fall from the mounting table 50.

[0046] More specifically, when the mounting platform 50 is moved to the discharge point corresponding to the rank assigned to the shiitake mushroom P (in other words, above the storage container corresponding to that rank), the terminal device 2 controls the second drive unit 40 to tilt the mounting platform 50. As a result, the shiitake mushroom P falls from the mounting platform 50 into the storage container corresponding to that rank. That is, each of the multiple mounting platforms 50 on which the shiitake mushroom P is placed tilts when it reaches the discharge point corresponding to the rank of the shiitake mushroom P, causing the shiitake mushroom P to fall into the storage container. In this way, the shiitake mushroom P is sorted according to its rank. After passing the discharge point, the terminal device 2 controls the second drive unit 40 to return the mounting platform 50 to its original position.

[0047] Figure 5 shows a flowchart illustrating the flow of the sorting process performed using the sorting device 1. Note that the steps in the flowchart shown in the embodiments of this disclosure may be rearranged in order, provided they are consistent with each other. For example, while the embodiments of this disclosure present various steps in an exemplary order, they are not limited to this order. Furthermore, the steps in the flowchart shown in the embodiments of this disclosure may be performed in parallel or in parallel, provided they are consistent with each other.

[0048] The operator prepares for the sorting operation (step S101). Specifically, the operator turns on the power to start the sorting device 1 and the terminal device 2. Figure 6 shows an example of the UI (User Interface) screen displayed on the display device 260 after the terminal device 2 is started. The UI screen displays the setting menu 310, the first image frame 320, the second image frame 330, the inspection items 340, and the start button 350. The setting menu 310 is a UI for making various settings. The first image frame 320 displays the image captured by the camera of the sensor unit 60. The second image frame 330 displays the image after the inspection process. The inspection items 340 displays, for example, various inspection items. The start button 350 is a button for starting the operation of the sorting device 1.

[0049] The operator operates the setting menu 310 to make various settings tailored to the shiitake mushroom P to be sorted (for example, setting the lighting source, setting the camera, determining various thresholds, setting the rank, and setting the discharge points corresponding to each rank). The operator also prepares the storage containers for each discharge point.

[0050] The operator presses the start button 350. This causes the terminal device 2 (processor 210) to output a control signal to the sorting device 1. Upon receiving the control signal, the sorting device 1 starts the rotation of the rotating support unit 20 (step S102). The mounting table 50 circulates along the circulation path CP, for example, by rotating clockwise.

[0051] The worker places each shiitake mushroom P onto a mounting platform 50 that moves sequentially closer to the worker by a rotational motion, at a position upstream of the information acquisition point P0 (step S103). The worker repeats this process until all the shiitake mushrooms P to be inspected are gone.

[0052] Guidance may be displayed on the UI screen. The guidance may include instructions on how to place the shiitake mushrooms (e.g., with the cap facing down and the stem facing up). Even workers unfamiliar with sorting can easily perform the sorting work by following the instructions in the guidance. In addition to or instead of the guidance displayed on the screen, voice guidance may be provided. Workers can easily perform the sorting work by following the instructions in the voice guidance output from the speaker of terminal device 2.

[0053] When the rotation of the rotating support unit 20 begins, the mounting platform 50 is sequentially moved to the information acquisition point P0. As described above, the terminal device 2 calculates the current position of the mounting platform 50 on the circulation path CP from the control information of the stepping motor. Based on the calculated current position of the mounting platform 50, the terminal device 2 detects the timing at which the mounting platform 50 is moved to the information acquisition point P0. The terminal device 2 instructs the sensor unit 60 to take an image in accordance with the detected timing (step S104). As a result, the shiitake mushroom P, which has been moved to the information acquisition point P0, is illuminated by the illumination light source and photographed by the camera. The sensor unit 60 outputs the captured image of the shiitake mushroom P.

[0054] The sensor unit 60 further outputs measurement data from the ToF sensor to the terminal device 2. For example, the ToF sensor repeatedly outputs measurement data to the terminal device 2 (for example, every 36 msec) from the start to the end of rotation of the rotating support unit 20.

[0055] Terminal device 2 executes the inspection process (step S105). Figure 7 shows the subroutine for the inspection process.

[0056] As shown in Figure 7, the terminal device 2 estimates the thickness of the shiitake mushroom P based on the measurement data from the ToF sensor (step S201). Specifically, in step S201, the terminal device 2 refers to approximately 10 measurement data points before and after the timing when the mounting platform 50 reaches the information acquisition point P0, and calculates the average value of the referenced values ​​as the thickness of the shiitake mushroom P.

[0057] For example, in the simplest inspection mode, the thickness of the shiitake mushroom P is not subject to inspection. In this case, the process in step S201 is omitted.

[0058] Terminal device 2 performs preprocessing on the captured image of Shiitake P (step S202). Specifically, terminal device 2 performs thresholding on the HSV (Hue-Saturation-Value) value to remove a specific color (for example, blue) from the captured image. For convenience, the captured image of Shiitake P (the image before preprocessing) is referred to as "Captured Image Ia". The captured image of Shiitake P after preprocessing is referred to as "Captured Image Ib".

[0059] Here, the support 22 and the mounting base 50 appear in the background of the shiitake mushroom P captured by the sensor unit 60. Therefore, it is desirable that the support 22 and the mounting base 50 be colors that are easily distinguishable from the shiitake mushroom P in image analysis processing and AI analysis. As an example, the support 22 and the mounting base 50 are colored blue, which is advantageous for distinguishing reflected light and shadows from the illumination light source and insect damage, etc. That is, the support 22 and the mounting base 50 are made of a material colored blue, which is easily removed as a background image. Therefore, the background (blue) is removed from the captured image Ia by thresholding. The background is an example of noise to be removed.

[0060] The thresholding process described above is insufficient to remove dust and other particles adhering to the mounting platform 50 from the captured image Ia. Therefore, the terminal device 2 performs noise reduction processing, such as morphological transformation, to remove fine information such as dust particles visible in the captured image Ia.

[0061] Terminal device 2 selects the largest area within the captured image Ia (the largest area in a single block where an object is visible). Terminal device 2 then applies noise reduction processing, such as morphological transformation, to the selected largest area. As a result, for example, the area containing the stem of the shiitake mushroom P is removed.

[0062] Terminal device 2 eliminates errors. For example, if the largest area selected from the captured image Ia is too small or too large, it is possible that some error occurred between the capture process and the pre-processing. Also, there may be cases where the shiitake mushroom P is not visible in the captured image Ia. In these cases, terminal device 2 stops the inspection process for the captured image Ia and executes the inspection process for the next captured image Ia.

[0063] Thus, before performing the inspection of the shiitake mushroom P, terminal device 2 performs preprocessing (an example of filtering) on ​​the image data of the shiitake mushroom P, removing noise components from the curled portion of the cap (an example of the first part) and the gill portion (an example of the second part).

[0064] Steps S203 to S205 in Figure 7 will be explained using Figures 8 to 10. In each of Figures 8 to 10, the upper diagram shows a single shiitake mushroom P placed with the cap facing downwards and the stem facing upwards. As shown in the upper diagram of each figure, in this position, the shiitake mushroom P has a curled-up portion P11 of the cap surrounding the gill portion P12.

[0065] In Figures 8 to 10, the middle and bottom diagrams are explanatory diagrams for the processes in steps S203 and S205, respectively. Terminal device 2 performs a first inspection on the pre-processed captured image Ib (step S203). Specifically, terminal device 2 uses a predetermined algorithm, such as a function, to find the minimum circumscribed circle a of the entangled portion P11 (an example of the first portion) captured in the captured image Ib. A concrete example of the minimum circumscribed circle a is shown by a dashed line in the middle diagrams of Figures 8 to 10.

[0066] Terminal device 2 calculates the shape difference between the determined minimum circumscribed circle a and the wrapped portion P11. Specifically, terminal device 2 calculates the ratio of the area of ​​the outer shape of the shiitake mushroom P to the area of ​​the minimum circumscribed circle a as the shape difference between the minimum circumscribed circle a and the wrapped portion P11. The area of ​​the outer shape of the shiitake mushroom P is, for example, the area of ​​the region enclosed by the outline of the wrapped portion P11.

[0067] In this embodiment, the difference in shape between the smallest circumscribed circle a and the wrapped portion P11 is, for example, called the shape variation ratio of the cap. The larger this shape difference (in other words, the smaller the ratio of the area of ​​the outer shape of the shiitake mushroom P to the area of ​​the smallest circumscribed circle a), the larger the shape variation ratio of the cap. The shape variation ratio may be expressed using other terms such as "filling rate".

[0068] The greater the degree of deformation in the cap of the shiitake mushroom P, the more distorted the outer shape of the cap will be from a perfect circle, and it will be judged as unsuitable for shipment. The smaller the degree of deformation in the cap, the closer the outer shape of the cap of the shiitake mushroom P will be to a perfect circle, and it will be judged as suitable for shipment. However, even if the outer shape of the cap is close to a perfect circle, if part of the curled portion P11 is missing or distorted, the appearance of the gill portion P12 may be poor. Therefore, it cannot be said that shiitake mushroom P is suitable for shipment simply because the outer shape of the cap is close to a perfect circle.

[0069] Therefore, terminal device 2 determines whether or not to perform a second inspection (step S205) based on the result of the first inspection (step S203) (step S204). Specifically, terminal device 2 determines whether or not the deformation rate of the umbrella portion calculated in step S203 is less than the first threshold.

[0070] If the degree of deformation of the cap is below the first threshold (Step S204: YES), the external shape of the cap of the shiitake mushroom P is normal. Note that the meaning of "normal" changes depending on the criteria used to rank (grade) the shiitake mushroom P. Here, even those with some distortion from a perfect circle are considered "normal". In this case, terminal device 2 performs a second inspection to check whether the appearance of the gill portion P12 is good or not (Step S205). If the degree of deformation of the cap is above the first threshold (Step S204: NO), the external shape of the cap has distortion that does not fall within the range of "normal". In this case, terminal device 2 does not perform the second inspection.

[0071] In this way, terminal device 2 performs a second inspection according to the result of the determination process (step S204).

[0072] The judgment conditions in step S204 are appropriately changed depending on the criteria used to rank (grade) the shiitake mushrooms P. For example, terminal device 2 performs the second inspection (step S205) if the deformation rate of the cap calculated in step S203 is equal to or greater than the first threshold, and does not perform the second inspection if this deformation rate is less than the first threshold. In other words, terminal device 2 performs the second inspection if the external shape of the cap has distortion that does not fall within the range of "normal", and does not perform the second inspection if the external shape of the cap is "normal".

[0073] In the second inspection, terminal device 2 uses a predetermined algorithm, such as a function, to determine the minimum circumscribed circle b of the pleated portion P12 (an example of the second portion) captured in the image Ib. Specific examples of the minimum circumscribed circle b are shown by dotted lines in the lower diagrams of Figures 8 to 10.

[0074] Terminal device 2 calculates the shape difference between the determined minimum circumscribed circle b and the pleated portion P12. Specifically, terminal device 2 calculates the ratio of the area of ​​the pleated portion P12 to the area of ​​the minimum circumscribed circle b as the shape difference between the minimum circumscribed circle b and the pleated portion P12. The area of ​​the pleated portion P12 is, for example, the area of ​​the region enclosed by the outline of the pleated portion P12.

[0075] In this embodiment, the difference in shape between the smallest circumscribed circle b and the pleated portion P12 is called, for example, the deformation rate of the pleated portion P12. The larger this difference in shape (in other words, the smaller the ratio of the area of ​​the pleated portion P12 to the area of ​​the smallest circumscribed circle b), the larger the deformation rate of the pleated portion P12.

[0076] Terminal device 2 performs inspections on other items (step S206). The items to be inspected differ, for example, depending on the inspection mode. In the simplest inspection mode, the process in step S206 is omitted.

[0077] As an example, terminal device 2 uses a machine learning model to inspect the degree to which the cap of the mushroom is open. The machine learning model is an AI program that analyzes the shiitake mushroom P in the captured image to inspect the degree to which the cap is open, and is implemented using a neural network.

[0078] Specifically, as a preprocessing step, terminal device 2 creates a square inscribed by the smallest circumscribed circle a, and crops the captured image Ib to fit the created square. Terminal device 2 then resizes the cropped image to the size specified by the machine learning model.

[0079] The machine learning model is provided, for example, via cloud computing. That is, terminal device 2 does not need to have a machine learning model. Terminal device 2 accesses the machine learning model located in the cloud and provides the resized image of Shiitake P to the machine learning model. Terminal device 2 obtains inspection results (for example, a value between 0 and 1 indicating the degree of cap opening) from the machine learning model on the cloud. Because the computationally intensive AI analysis can be performed on the cloud, the operator can use, for example, an inexpensive terminal device 2 with low performance.

[0080] The machine learning model can be trained using image data uploaded from each worker's terminal device 2 connected to the cloud. Because a large amount of image data can be collected as training material, the accuracy of the machine learning model improves.

[0081] For example, sorting characteristics such as the degree to which the cap of a mushroom is open are easily influenced by subjective criteria in human inspection. Therefore, it is effective to use AI that judges based on the interaction of multiple characteristics rather than simple numerical judgments. If AI is used, differences in sorting criteria that are subjective or regional will be eliminated. For example, quality will be standardized in a wide production area. This is expected to improve the overall quality of shiitake mushrooms.

[0082] Terminal device 2 ranks the shiitake mushrooms P based on the results of the inspection process (step S105) (step S106).

[0083] Specifically, terminal device 2 evaluates the shape of the shiitake mushroom P. If only the first inspection (step S205) of the two inspections (step S203) is performed by terminal device 2, the shape of the shiitake mushroom P is evaluated in multiple stages (normal, slightly deformed, significantly deformed, etc.) based on the deformation rate of the cap. If both the first and second inspections are performed by terminal device 2, the shape of the shiitake mushroom P is evaluated in multiple stages (normal, slightly deformed, significantly deformed, etc.) based on the deformation rate of the cap and the deformation rate of the gill portion P12.

[0084] In the latter case, the terminal device 2 may evaluate the shape of the shiitake mushroom P after weighting the degree of deformation of the cap portion and the degree of deformation of the gill portion P12. For example, the terminal device 2 may weight the degree of deformation of the cap portion (an example of the result of the first inspection) and the degree of deformation of the gill portion P12 (an example of the result of the second inspection) according to the origin of the shiitake mushroom P (an example of an attribute).

[0085] For example, if the appearance of the cap is prioritized, terminal device 2 multiplies the deformation rate of the cap by a coefficient α and multiplies the deformation rate of the gill portion P12 by a coefficient β (β < α). Terminal device 2 evaluates the shape of the shiitake mushroom P in multiple stages (normal, slightly deformed, significantly deformed, etc.) based on the sum of these multiplied values. The attributes considered when calculating the weights and the coefficients used in the calculation can be set as appropriate by the operator. This allows terminal device 2 to perform evaluations according to attributes (origin, variety, etc.).

[0086] Terminal device 2 ranks the shiitake mushrooms P in multiple stages (A, B, C, etc.) based on the shape evaluation results and the value indicating the degree of opening of the cap obtained from the AI ​​inspection.

[0087] In the above embodiment, the terminal device 2 performs a first inspection (inspection of the deformation rate of the cap portion), and if necessary, a second inspection (inspection of the deformation rate of the gill portion P12), before ranking the mushrooms. The operator can appropriately set the inspection conditions when ranking the shiitake mushrooms P. For example, in another embodiment, the terminal device 2 may perform a second inspection (inspection of the deformation rate of the gill portion P12), and if necessary, a first inspection (inspection of the deformation rate of the cap portion), before ranking the mushrooms.

[0088] Specifically, the terminal device 2 performs at least one of the following: a first inspection to check how much the shape of the curled portion P11 (an example of a first part included in the object) differs from the minimum circumscribed circle a (an example of a predetermined first shape) based on the captured image Ia of the shiitake mushroom P (an example of image data of the object); and a second inspection to check how much the shape of the gill portion P12 (an example of a second part that differs from the first part included in the object) differs from the minimum circumscribed circle b (an example of a predetermined second shape). Based on the results of the performed inspections, the terminal device 2 ranks the shiitake mushroom P.

[0089] In step S206, terminal device 2 may inspect the size of shiitake mushrooms P. For example, terminal device 2 calculates the diameter of the minimum circumscribed circle a and obtains the calculated value as the size of shiitake mushrooms P. In this case, in step S106, terminal device 2 ranks shiitake mushrooms P in multiple stages (A, B, C, etc.) based on the size of shiitake mushrooms P and other results (shape evaluation results and values ​​indicating the degree of cap opening obtained from AI inspection). That is, terminal device 2 ranks shiitake mushrooms P based on the size of the minimum circumscribed circle a, in addition to the inspection results (such as the degree of deformity of the cap).

[0090] Terminal device 2 associates various result information (captured image Ia, captured image Ib, various inspection results, rank, inspection time, processing time, etc.) with each shiitake mushroom P (in other words, each mounting platform 50) and saves it to storage 230 (step S107).

[0091] Terminal device 2 outputs various result information for shiitake mushrooms P (an example of information including inspection results and rank) to display device 260 (step S108). An example of the display screen at this time is shown in Figure 11. In the example display screen in Figure 11, captured image Ia, captured image Ib, various setting information 312 during inspection, various inspection results 342, and stop button 352 are displayed. Various setting information 312 include, for example, shooting conditions and rotation speed of the mounting table 50. Various inspection results 342 include rank, thickness, degree of opening, deformation rate, etc.

[0092] The rank indicates the discharge point from which the shiitake mushrooms P are discharged. Terminal device 2 discharges the shiitake mushrooms P to the discharge point corresponding to the rank (step S109). Specifically, terminal device 2 monitors the current position of each mounting platform 50. When the mounting platform 50 is moved to the discharge point corresponding to the rank assigned to the shiitake mushrooms P placed on it, terminal device 2 tilts the mounting platform 50. As a result, the shiitake mushrooms P fall from the mounting platform 50 and enter the storage container corresponding to that rank. In other words, the shiitake mushrooms P are sorted by rank. Once the mounting platform 50 has passed the discharge point, it returns to its original position.

[0093] Specifically, terminal device 2, for example, discharges a rank A shiitake mushroom P at discharge point P1. Terminal device 2 discharges a rank B shiitake mushroom P at discharge point P2. Terminal device 2 discharges a rank C shiitake mushroom P at discharge point P3.

[0094] In the sorting process shown in Figure 5, steps S103 to S109 are repeated until all of the target shiitake mushrooms P have been sorted.

[0095] Once all the target shiitake mushrooms P have been sorted (Step S110: YES), the operator presses the stop button 352. This causes the terminal device 2 to output a control signal to the sorting device 1. Upon receiving the control signal, the sorting device 1 stops the rotation of the rotating support unit 20 (Step S111). The operator then turns off the power to both the sorting device 1 and the terminal device 2 to stop them (Step S112). This completes the sorting process.

[0096] The worker collects the shiitake mushrooms P from each storage container. The worker then performs tasks such as packing and shipping the collected shiitake mushrooms P of each rank, and maintaining the sorting device 1.

[0097] The above is a description of exemplary embodiments of the present disclosure. Embodiments of the present disclosure are not limited to those described above, and various modifications are possible within the scope of the technical idea of ​​the present disclosure. For example, embodiments of the present application include combinations of embodiments explicitly shown in the specification or obvious embodiments as appropriate.

[0098] For example, in the above embodiment, various inspections are performed on the captured image of shiitake mushroom P. However, the image to be inspected is not limited to a captured image. Various inspections may also be performed on an image acquired by another means (for example, an image received via a network). [Explanation of Symbols]

[0099] 1: Fruit sorting device 2: Terminal device 10: Cabinet 20: Rotating support part 21: Base 22:Support 30: First drive unit 40: Second drive unit 50: Mounting platform 60: Sensor unit 210: Processor 220: Memory 230: Storage 232: Control program 240: Communication Interface 250: Input device 260:Display device Ia: Photographed image Ib: Captured image

Claims

1. At least one of the following is performed: a first inspection that checks how much the shape of a first part included in the object differs from a predetermined first shape based on image data of the object; and a second inspection that checks how much the shape of a second part included in the object that differs from the first part differs from a predetermined second shape. The objects are ranked based on the results of the inspections performed. Information processing device.

2. Perform the first inspection described above. Based on the results of the first inspection, a decision is made as to whether or not to perform the second inspection. The second test is performed according to the result of the determination. The information processing apparatus according to claim 1.

3. The object has the first part surrounding the second part, The predetermined first shape and the predetermined second shape are, respectively, the minimum circumscribed circle of the first part and the minimum circumscribed circle of the second part. In the first test described above, Find the smallest circumscribed circle of the first part, The difference in shape between the minimum circumscribed circle of the first part and the first part is calculated. In the second inspection described above, Find the smallest circumscribed circle of the second part mentioned above. The difference in shape between the minimum circumscribed circle of the second part and the second part is calculated. The information processing apparatus according to claim 1.

4. The difference in shape between the smallest circumscribed circle of the first part and the first part is expressed as the ratio of the area of ​​the region closed by the outline of the first part to the area of ​​the smallest circumscribed circle of the first part. The difference in shape between the smallest circumscribed circle of the second part and the second part is expressed as the ratio of the area of ​​the region closed by the outline of the second part to the area of ​​the smallest circumscribed circle of the second part. The information processing apparatus according to claim 3.

5. The objects are ranked based on the size of the smallest circumscribed circle of the first part. The information processing apparatus according to claim 3.

6. Before performing the inspection, the image data is subjected to a filter process to remove noise components from the first and second portions. The information processing apparatus according to claim 1.

7. The results of the first inspection and the results of the second inspection are weighted according to the attributes of the object. The information processing apparatus according to claim 1.

8. The aforementioned object is a shiitake mushroom. The aforementioned attribute is the origin of the shiitake mushrooms. The information processing apparatus according to claim 7.

9. The results and rank of the inspection for the object are output to a display device. The information processing apparatus according to claim 1.

10. The aforementioned object is a shiitake mushroom. The first part is the curled portion of the cap of the shiitake mushroom, The second part is the folded portion on the underside of the umbrella portion. An information processing apparatus according to any one of claims 1 to 9.

11. At least one of the following is performed: a first inspection that checks how much the shape of a first part included in the object differs from a predetermined first shape based on image data of the object; and a second inspection that checks how much the shape of a second part included in the object that differs from the first part differs from a predetermined second shape. The computer is instructed to perform a process of ranking the objects based on the results of the inspections performed. program.

Images