Eating timing determination system and eating timing determination method

Abstract

The eating timing determination system according to an embodiment of the present invention is provided with: a ripening degree determination unit for determining the ripening degree of a muskmelon on the basis of at least two elements obtained from an image of a pedicel of the muskmelon; and an eating timing determination unit that determines whether or not the muskmelon is in an eating timing state on the basis of the after-ripening degree.

Description

Technical Field

[0001] Embodiments of the present invention relate to a system and method for determining an eating timing. Background Art

[0002] In recent years, Japanese people have been experiencing a growing disconnect between fruits and vegetables. This trend is particularly pronounced for melons, with brands facing a potential brand extinction across the country. While melons enjoy high quality ratings from consignors and wholesalers, consumer demand is declining. Therefore, rebranding is crucial to avoid this extinction crisis. Farmers of fruits like melons, which ripen quickly, face a particular challenge: Consumers struggle to determine the optimal time to consume them, leading to a lack of quality assurance.

[0003] Methods for informing consumers of the best time to eat fruit include recording the expiration date of the fruit in sales channels, mechanically determining the progress of ripening, and determining the best time to eat by taking an image of the melon's skin.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 08-178912

[0007] Patent Document 2: International Publication No. 2018 / 003506

[0008] Patent Document 3: Japanese Patent Application Laid-Open No. 2006-138771

[0009] Patent Document 4: Japanese Patent Application Laid-Open No. 2006-226775

[0010] Patent Document 5: Japanese Patent Application Laid-Open No. 11-173985

[0011] Patent Document 6: Japanese Patent Application Laid-Open No. 2004-294108

[0012] Patent Document 7: Japanese Patent Application Laid-Open No. 2020-176976

[0013] Patent Document 8: Japanese Patent Application Laid-Open No. 2021-089179 Summary of the Invention

[0014] Technical problem to be solved by the invention

[0015] The progress of melon ripening varies depending on storage conditions before consumption, as well as the variety and season, making it difficult to determine the right time to eat. Furthermore, some melons lack a reticular surface pattern, while others exhibit little to no color variation, depending on the variety. This makes it difficult to determine when to eat based on the skin's condition.

[0016] The present invention has been made in view of the above-mentioned actual situation, and an object of the present invention is to provide an eating timing determination system and an eating timing determination method that can easily determine the eating timing of a melon.

[0017] Means for solving technical problems

[0018] The eating time determination system of the technical solution includes: a ripeness determination unit for determining the ripeness of the melon based on at least two elements obtained from an image of the melon stem; and an eating time determination unit for determining whether the melon is ready to be eaten based on the ripeness.

[0019] Effects of the Invention

[0020] It is easy to determine when to eat melon. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a diagram showing an example of a configuration for realizing an eating timing determination system according to an embodiment.

[0022] Figure 2 1 is a diagram showing an example of the configuration of a consumption timing determination system realized by using a dedicated application program executed by the control device 14 on the server 1 side.

[0023] Figure 3 1 is a diagram showing an example of a melon to which a piece M having a two-dimensional code indicating access to the server 1 is attached.

[0024] Figure 4A 1 is a diagram showing an example of display content displayed on the display device 25 of the terminal 2 before shooting.

[0025] Figure 4B 1 is a diagram showing an example of display content displayed on the display device 25 of the terminal 2 during imaging.

[0026] Figure 5 This is a diagram showing an example of resizing processing.

[0027] Figure 6 This is a diagram showing an example of a cutting process using the backing sheet P and the second frame L2.

[0028] Figure 71 and 2 are diagrams showing an example in which the blue portion of the background of the stem in the image P′ is replaced with black.

[0029] Figure 8 This is a diagram schematically showing an example of a procedure for moving an object and performing angle correction (tilt correction) through affine transformation.

[0030] Figure 9 1 is a diagram showing an example of the procedure of the subsequent maturity determination process performed by the subsequent maturity determination unit 33 .

[0031] Figure 10 This is a diagram showing an example of temporal changes in the stems of melons after harvest.

[0032] Figure 11 These are diagrams showing examples of images before and after the color component (yellow to brown) of the stem, which increases as ripening progresses, is deleted.

[0033] Figure 12A This is a diagram for explaining the binarization of an image when calculating a size measurement value.

[0034] Figure 12B This is a diagram for explaining the counting of the number of pixels of an image when calculating a size measurement value.

[0035] Figure 13 This is a diagram for explaining processing when the pedicle end portion in the image has a pointed shape.

[0036] Figure 14A This is a diagram showing an example of a procedure for calculating the degree of ripeness in the ripeness calculation process 33c.

[0037] Figure 14B It is schematically represented Figure 14A A diagram showing the range of b in each category shown in step S44 in FIG.

[0038] Figure 14C Yes Figure 14A FIG. 4 is a diagram of a modified example of step S44 in .

[0039] Figure 15 This is a diagram showing an example (part 1) of an image used in the degree of doneness determination process.

[0040] Figure 16 This is a diagram showing an example (part 2) of an image used in the degree of doneness determination process.

[0041] Figure 17 This is a diagram showing an example (part 3) of an image used in the ripeness determination process.

[0042] Figure 18 It means that the Figures 15 to 17Figure 1 is a table obtained by analyzing the information obtained for each image.

[0043] Figure 19 1 is a diagram showing an example of the procedure of the consumption timing determination process performed by the consumption timing determination unit 34 .

[0044] Figure 20 Yes Figure 1 FIG. 5 is a diagram of a modified example of the system described in .

[0045] Figure 21 This is a diagram schematically showing a process flow for obtaining a result of determining the degree of doneness or the timing of consumption using an AI (Artificial Intelligence) model.

[0046] Figure 22 1 is a diagram showing an example of the basic operation of the eating timing determination system 30 . DETAILED DESCRIPTION

[0047] Hereinafter, embodiments will be described with reference to the drawings.

[0048] (1) System composition

[0049] Figure 1 This is a diagram showing an example of a configuration for realizing an eating timing determination system according to an embodiment.

[0050] Figure 1 Server 1 and terminal 2 are shown. The eating timing determination system of this embodiment is installed in at least one of server 1 and terminal 2. Here, an example is shown in which the eating timing determination system is installed in server 1. The eating timing determination system is implemented using a computer program in server 1. Alternatively, some or all of the functions of the eating timing determination system may be installed in terminal 2.

[0051] The server 1 corresponds to a computer (information processing device) managed by an administrator on a network (cloud). The terminal 2 corresponds to an information device such as a smartphone, tablet terminal, or personal computer held by a consumer.

[0052] The server 1 includes a storage device 11 , a communication device 12 , a data processing device 13 , a control device 14 , and the like.

[0053] The storage device 11 stores programs and data executed by the control device 14. These programs include an application program (hereinafter referred to as the "dedicated application program") that implements the eating timing determination system. The dedicated application program has the following functions: upon access from the terminal 2 to the server 1, it prompts the user to photograph the stem of the melon, obtains the image of the photographed stem, determines the eating timing of the melon based on the image, and notifies the terminal 2 of the determination result. The eating timing here refers to the time when the melon is suitable for consumption.

[0054] The communication device 12 transmits and receives information to and from the terminal 2 .

[0055] The data processing device 13 performs various image processing and calculation processes related to the eating timing determination under the control of the control device 14 .

[0056] The control device 14 corresponds to a processor that executes a program, and controls the operations of the storage device 11 , the communication device 12 , and the data processing device 13 .

[0057] The terminal 2 includes an input device 21 , an imaging device 22 , a storage device 23 , a communication device 24 , a display device 25 , a control device 26 , and the like.

[0058] The input device 21 is a device used by the consumer to perform various input operations.

[0059] The imaging device 22 corresponds to a camera that captures an image of an object and generates an image thereof.

[0060] The storage device 23 stores programs and data executed by the control device 14 , and includes ROM (Read Only Memory), RAM (Random Access Memory), etc. The programs include an application program that accesses the server 1 based on the information of the two-dimensional code recognized by the imaging device 22 .

[0061] The communication device 24 transmits and receives information to and from the server 1 .

[0062] The display device 25 corresponds to a display that displays information such as the image captured by the imaging device 22 and the result of the eating timing determination.

[0063] The control device 26 corresponds to a processor that executes a program, and controls the operation of each of the input device 21 , the imaging device 22 , the storage device 23 , the communication device 24 , and the display device 25 .

[0064] Figure 2 1 is a diagram showing an example of the configuration of an eating timing determination system realized by using a dedicated application program executed by the control device 14 of the server 1 .

[0065] Figure 2 The illustrated eating time determination system 30 includes various functions: an image acquisition unit 31, an image preprocessing unit 32, a doneness determination unit 33, an eating time determination unit 34, and a result output unit 35. However, not all of these functions are required, and some may be omitted. For example, the eating time determination system 30 may not include the image preprocessing unit 32.

[0066] The image acquisition unit 31 performs a process for acquiring an image of the stem of the melon from the terminal 1 (image acquisition process).

[0067] The image preprocessing unit 32 performs image processing (image preprocessing) before the image acquired by the image acquisition unit 31 is used by the degree of familiarity determination unit 33. This image preprocessing includes image input processing 32a, ratio adjustment processing 32b, region clipping processing 32c, background adjustment processing 32d, and position adjustment processing 32e. However, all of these image preprocessing steps are not necessarily required, and some or all of these steps may not be performed.

[0068] The image input process 32 a is a process of inputting the image acquired by the image acquisition unit 31 into the image pre-processing unit 32 .

[0069] The ratio adjustment process 32 b is a process for making the size of the image to be processed (for example, the image input in the image input process 32 a ) match the size of a predetermined reference image.

[0070] The region clipping process 32 c is a process of clipping a predetermined region from the image to be processed (for example, the image whose size has been adjusted by the ratio adjustment process 32 b ).

[0071] The background adjustment process 32d is a process for changing the image portion corresponding to the background of the processing target image (for example, the image of the area cut out by the area cutout process 32c) to a specific color based on the color of a predetermined backing plate described later.

[0072] The position adjustment process 32e is a process of moving the pedicle in the image to be processed (for example, the image input by the image input process 32a) to the center of the image and rotating the pedicle so that the longitudinal direction of the pedicle faces a predetermined direction (for example, the horizontal direction).

[0073] The ripeness determination unit 33 performs the following processing (ripening determination processing): it obtains information about the color and shape of the stem from the stem image processed by the image preprocessing unit 32 and determines the ripeness of the melon based on this information. The ripeness determination processing includes color calculation processing 33a, size calculation processing 33b, and ripeness calculation processing 33c.

[0074] The color tone calculation process 33 a is a process of calculating the ratio of the color of the stem that increases or decreases as the ripening of the melon proceeds, using the stem image.

[0075] The size calculation process 33 b is a process of calculating the size of the stem that decreases as the melon ripens, using the stem image.

[0076] The ripeness calculation process 33 c is a process for calculating the ripeness using the result calculated by the color tone calculation process 33 a and the result calculated by the size calculation process 33 b .

[0077] The eating timing determination unit 34 performs a process of determining whether the melon is ready to be eaten based on the degree of ripeness determined by the ripeness determination unit 33 (eating timing determination process).

[0078] The result output unit 35 performs a process of notifying the terminal 1 side of the result determined by the consumption timing determination unit 34 (result output process).

[0079] Hereinafter, details of each of the image acquisition unit 31 , the image pre-processing unit 32 , the doneness determination unit 33 , the consumption timing determination unit 34 , and the result output unit 35 will be described in order.

[0080] (1-1) Image Acquisition Unit 31

[0081] As described above, the image acquisition unit 31 performs processing for acquiring an image of the stem of the melon from the terminal 1 (image acquisition processing). The image acquired by the image acquisition unit 31 is a visible image showing the stem of the melon. The visible image may be a still image obtained by photographing with a camera, or an image obtained from a moving image. It should be noted that the visible image showing the stem of the melon does not necessarily have to be a photographed image, but may be an image obtained by a method other than photographing. In the case of obtaining an image by photographing, the following methods can be cited as photographing methods.

[0082] 1) After capturing the entire melon, only the pedunculated area is captured through subsequent image preprocessing.

[0083] 2) Take the pedunculated area as the subject for imaging.

[0084] 3) A backing sheet having the same range as the area cut out by image preprocessing is attached to the stem and the image is captured. Here, an example is shown in which the backing sheet is a physical object printed on paper, etc., but the present invention is not limited to this example. The backing sheet may be a physical object printed on paper, etc., or an image displayed on a monitor, etc.

[0085] These methods can be appropriately selected and implemented according to the situation.

[0086] Here, an example of a case where imaging is performed using the above-mentioned method 3) is shown.

[0087] Below, refer to Figure 3 、 Figure 4A as well as Figure 4B , the functions of the image acquisition unit 31 are described in detail.

[0088] Figure 3 1 is a diagram showing an example of a melon to which a piece M having a two-dimensional code indicating access to the server 1 is attached. Figure 4A and Figure 4B It is a diagram showing an example of the display content before shooting and an example of the display content during shooting displayed on the display device 25 of the terminal 2 when a consumer uses the shooting device 22 of the terminal 2 to shoot the stem S of the melon according to the information guided by the dedicated application of the server 1 at the access destination.

[0089] For example, consumers follow the instructions that come with the melons they receive, such as Figure 3 As shown, an operation is performed to access the corresponding website (server 1) by reading a QR code on a piece of melon or in the instructions, etc., using the camera 22 of terminal 2. This allows access from terminal 2 to server 1, and information explaining how to determine the timing of consumption is transmitted to terminal 2 via a dedicated application on server 1 (specifically, the image acquisition unit 31 of the consumption timing determination system 30) via communication devices 12 and 24 and displayed on display device 25. Alternatively, a URL (Uniform Resource Locator) may be included on the piece of melon or in the instructions, etc., instead of the QR code. In this case, the corresponding website (server 1) can be accessed using the URL.

[0090] The instructions for implementing the eating timing determination method may include, for example, a message such as "Please attach a backing sheet behind the thin part of the stem of the melon and align the other frames in the image with the frames on the backing sheet before photographing." Additionally, sample images, illustrations, and drawings illustrating the photographing process may be included as supplementary information.

[0091] Instructions on the implementation method for consumers to judge according to the timing of consumption, such as Figure 4A As shown, the stem S and backing sheet P are photographed with a backing sheet P of a predetermined color attached to the back of the stem S. Specifically, the image is taken with the second frame L2 (guide) displayed on the display device 25 of the terminal 2 aligned with the first frame L1 pre-marked on the backing sheet P. The captured image is transmitted from the terminal 2 to the server 1.

[0092] Thus, the image acquisition unit 31 acquires an image obtained by photographing the stem S and the backing plate P with the backing plate of a predetermined color attached to the back of the stem S. Specifically, the image acquisition unit 31 acquires an image photographed so that the second frame L2 displayed on the display device 25 of the terminal 2 is aligned with the first frame L1 pre-marked on the backing plate P.

[0093] About lining

[0094] The backing sheet P is, for example, a square sheet approximately 3 to 5 cm square and is attached to the back side of the narrow portion of the stem S. The color of the backing sheet P is, for example, blue. To facilitate attachment of the backing sheet P to the stem S, the surface can be formed of a sheet-like material. Furthermore, the backing sheet P can be made of paper or other materials.

[0095] There are two purposes for using such a liner P.

[0096] The first purpose of liner P

[0097] The first purpose of the backing sheet P is to facilitate the area clipping process 32c. The area clipping process 32c specifies a predetermined area within the input visual image to be used in the ripeness determination process and clips the image of that area. Using the backing sheet P during this process allows the image to be clipped within a predetermined range, such as the size of the backing sheet P or the size of the second frame L2, thus preventing errors caused by range designation.

[0098] The second purpose of the liner

[0099] The second purpose of the backing plate P is to facilitate background adjustment processing 32d. Background adjustment processing 32 performs the following: the stem S is identified as the foreground, and the other captured parts are identified as the background. The background is then converted to a color irrelevant to subsequent processing, such as black. During this process, if a background of the same color as the stem S is reflected, it may not be recognized as the background of the stem S. Therefore, by using the backing plate P, any components other than the stem S that could constitute the background are not reflected, making background adjustment processing 32d easier. As a result, compared to conventional methods, the consumption timing determination process can be performed more easily without environmental constraints.

[0100] ・About shooting with backing board

[0101] like Figure 4A As shown, the stem S of the melon and the backing plate P are reflected in the image F displayed on the display device 25 of the terminal 2. The first frame L1 is pre-marked on the backing plate P. Figure 4B As shown, the second frame L2 is displayed at the center of the image F displayed on the display device 25 of the terminal 2. The image is captured in a state where the second frame L2 is aligned with the first frame L1.

[0102] At least one of the colors or shapes of the first and second frames L1, L2 can differ. This makes it easier for consumers to align the frames when taking photos. For example, using a complementary color to the stem as the color of the first or second frame L1 or L2 can make the frame color stand out. Furthermore, the first or second frame L1 or L2 can be in the form of a partially interrupted dotted or dashed line, or without a line except at the four corners.

[0103] By aligning the second frame L2 with the first frame L1 on the backing sheet P during imaging, the coordinate information of the four corners of the second frame L2 can be used as a reference for the area to be captured by the area capture process 32c, making it easier to perform the area capture process 32c. This information can also serve as a reference for determining how close to the object to capture during imaging. Furthermore, since the first frame L1 is positioned inward of the edge of the backing sheet P, the captured area is prevented from protruding outside the backing sheet P due to errors during imaging and cropping.

[0104] The above-mentioned imaging method is not limited to this example. For example, if the same region clipping as above can be performed in the region clipping process 32c, imaging may be performed using other methods.

[0105] (1-2) Image Preprocessing Unit 32

[0106] As described above, the image preprocessing unit 32 performs image processing (image preprocessing) before the image acquired by the image acquisition unit 31 is used by the degree of familiarity determination unit 33. This image preprocessing includes ratio adjustment 32b, region clipping 32c, background adjustment 32d, and position adjustment 32e. These processes are described in detail below.

[0107] Image input processing 32a

[0108] As described above, the image input process 32 a is a process of inputting the image acquired by the image acquisition unit 31 into the image pre-processing unit 32 .

[0109] In the image input process 32a, the image acquired by the image acquisition unit 31 is adjusted, as necessary, to a format suitable for processing in the ratio adjustment process 32b, the region clipping process 32c, the background adjustment process 32d, and the position adjustment process 32e, and then the image is input to the image pre-processing unit 32. However, if the image format is already suitable for each process, there is no need to re-adjust the format.

[0110] The adjusted format uses, for example, a general file format used for still images processed on a computer.

[0111] Regarding the ratio adjustment process 32b

[0112] As described above, the ratio adjustment process 32 b is a process for making the size of the image to be processed (for example, the image input in the image input process 32 a ) match the size of a predetermined reference image.

[0113] In the size calculation process 33b described later, when calculating the thickness of the pedicle reflected in the image, the number of pixels in the portion of the image corresponding to the pedicle is calculated. However, depending on the type of imaging device, the size of the input image varies, and it may not be possible to calculate the thickness of the pedicle based on a fixed standard. Therefore, in the ratio adjustment process 32b, the size of the image is adjusted to match the size of the reference image while maintaining its aspect ratio.

[0114] Figure 5 This is a diagram showing an example of resizing processing.

[0115] Figure 5 An example is shown in which the conversion from the pre-correction image F to the post-correction image F′ is performed based on the reference image F0.

[0116] In this example, when the horizontal and vertical pixel counts of the reference image F0 are 400×600 and the horizontal and vertical pixel counts of the pre-correction image F are 500×700, the size of the pre-correction image F is converted so that the horizontal and vertical pixel counts of the post-correction image F' become 400×560 without changing the aspect ratio.

[0117] Specifically, while maintaining the aspect ratio of the pre-correction image F, the pre-correction image F is resized at a magnification of 4 / 5 so that the number of horizontal pixels (400) in the post-correction image F' matches the number of horizontal pixels (400) in the reference image F0. At this time, the second frame L2 is also resized at the same magnification. As a result, the size of the second frame L2 displayed in the post-correction image F' becomes the same as the size of the frame L0 displayed in the reference image F0.

[0118] In this example, the reference image F0 and the corrected image F' have different aspect ratios. Therefore, while the number of pixels in the vertical direction of the reference image F0 is 600, the number of pixels in the vertical direction of the corrected image F' is 560. This results in an area G in the vertical direction of the corrected image F' that is insufficient relative to the size of the reference image F0. This insufficient area G is supplemented by pixels having a predetermined color, such as black, for example.

[0119] The following describes the basic processing steps of the resizing process.

[0120] 1) First, the size (the number of pixels in the vertical and horizontal directions) of the reference image F0 is determined, and either the number of pixels in the vertical or horizontal directions of the reference image is selected.

[0121] 2) Next, the size of the pre-correction image F (the number of pixels in the vertical and horizontal directions) is detected.

[0122] 3) Finally, the pre-correction image F is resized to match the number of pixels selected in step 1) above, without changing its aspect ratio, to obtain the corrected image F'. Furthermore, the missing region G is processed to match the size of F0 by adding pixels of a predetermined color, such as black. In this case, the addition of pixels of the predetermined color to the missing region G is not necessary, provided that this does not affect the region clipping process 32c.

[0123] By performing such a resizing process, the thickness of the pedicle can be calculated based on a certain reference in the size calculation process 33 b regardless of which device the image is captured by.

[0124] The size adjustment process shown here is not essential. If the size calculation process can be performed based on a certain standard, the size adjustment process may be omitted or other processes may be performed.

[0125] Regarding the area clipping process 32c

[0126] As described above, the region clipping process 32 c is a process of clipping a predetermined region from the image to be processed (for example, the image whose size has been adjusted by the ratio adjustment process 32 b ).

[0127] The region clipping process 32c preliminarily cuts off the range of the region to be subjected to the subsequent ripeness determination process by the subsequent ripeness determination unit 33. When imaging using the backing sheet P as described above, the backing sheet P and the second frame L2 are preferably used for the cropping process.

[0128] Figure 6 This is a diagram showing an example of a cutting process using the backing sheet P and the second frame L2.

[0129] The cutting process is performed based on the size of the backing plate P and the second frame L2. Figure 6 In the example, the cropping process is performed based on the size of the second frame L2. After the cropping process, the image P' of a predetermined area is cut out based on the coordinate information of the four corner points a, b, c, and d of the second frame L2.

[0130] By performing such a cropping process, the image region can be cut out based on a certain range, thereby preventing the occurrence of errors in range specification.

[0131] About background adjustment processing 32d

[0132] As described above, the background adjustment process 32d is a process for changing the image portion corresponding to the background of the processing target image (for example, the image of the area cut out by the area cutout process 33b) to a specific color based on the color of the backing plate.

[0133] In the background adjustment process 32d, a process is performed in which the stem captured in the image is recognized as the foreground, and the other captured contents are recognized as the background, and converted into a color irrelevant to subsequent processing, such as black.

[0134] During this process, if a background of the same color as the stem of the melon is reflected, for example, the background may not be recognized as the stem's background. Therefore, it is preferable to capture the image using the aforementioned backing plate P and then perform region clipping processing 32c to clip out the area where only the foreground is reflected. Specifically, processing is performed to replace the color of the portion of image P' that serves as the stem's background (the color of backing plate P) with a predetermined color.

[0135] Figure 7 1 and 2 are diagrams showing an example in which the blue portion of the background of the stem in the image P′ is replaced with black.

[0136] exist Figure 7 In the example shown in FIG. 3 , the color of the portion of the image P' that serves as the background is blue (the color of the backing plate P), and the color of this portion is replaced with black. By performing the region clipping process 32c, components that could serve as the background other than the backing plate P are not reflected, making it easier to perform the background adjustment process 32d.

[0137] Specifically, a threshold is set for the color information of each pixel in image P', and if there is a pixel with a value exceeding the threshold, the pixel value is converted to 0 (equivalent to a black value) (hereinafter referred to as "color threshold processing"). For example, the threshold can be pre-set so that the blue value exceeds the threshold, and a filter is applied only to the blue component to delete the blue component. In this way, an image with the remaining color components can be obtained. In this example, since a blue backing plate P is used that does not reach a value close to the color component of the base (green to yellow, brown), the color component of the base (green to yellow, brown) can be retained, and the blue component of the background can be easily deleted.

[0138] By performing such color threshold processing, it is possible to obtain information on the pedicle region excluding the pedicle background and its color components.

[0139] Furthermore, the color of the backing plate P can be any color that easily removes the background color, and is not limited to blue; other colors are also acceptable. In addition to RGB, there are color systems such as HSV and L*a*b* as methods for expressing color components. However, any color system can be used as long as it facilitates color thresholding. In this example, the color of the backing plate P is defined by the hue (H) of the HSV color space.

[0140] Regarding position adjustment processing 32e

[0141] As described above, the position adjustment process 32e is a process of moving the pedicle in the image of the processing object (for example, the image input by the image input unit 32) to the center of the image and rotating the pedicle so that the longitudinal direction of the pedicle is oriented in a predetermined direction (for example, the horizontal direction).

[0142] In this position adjustment process 32e, before the position adjustment of the pedicle is performed, the image to be processed is binarized, thereby rendering the pedicle portion white and the pedicle background black. The aforementioned threshold processing method can also be used for binarization. After binarization, the white portion of the pedicle is subjected to an affine transformation.

[0143] Figure 8 This is a diagram schematically showing an example of a procedure for moving an object and performing angle correction (tilt correction) through affine transformation.

[0144] Figure 8 Image Q1 in shows the state before the object is moved. Image Q2 shows the state after the object is moved to the center of the image. Image Q3 shows the state after the tilt of the object is corrected so that the longitudinal direction of the object is oriented horizontally.

[0145] The circle in the object shown in image Q1 represents the center of gravity of the object.

[0146] The circle located in the center of the images Q1 and Q2 indicates the center of the image.

[0147] The affine transformation here is composed of two steps: movement and angle correction (tilt correction) as shown below.

[0148] 1) Calculate the offset between the center of image Q1 and the center of gravity of the object, and move the object to the center of the image as in image Q2.

[0149] 2) Calculate the tilt of the object in image Q2 and correct the tilt.

[0150] By performing such affine transformation, the arrangement position of the stem in the image can be adjusted appropriately, and the eating timing determination process can be performed with high accuracy.

[0151] In addition, the above-mentioned movement and angle correction are not limited to affine transformation, and can also be achieved using other methods.

[0152] (1-3) Cooking degree determination unit 33

[0153] As mentioned above, Figure 1 The ripeness determination unit 33 shown in the figure performs the following processing (ripening determination processing): it obtains information about the color and shape of the stem from the stem image processed by the image preprocessing unit 32 and determines the ripeness of the melon based on the obtained color and shape information. The ripeness determination processing includes color tone calculation processing 33a, size calculation processing 33b, and ripeness degree calculation processing 33c.

[0154] Figure 9 1 is a diagram showing an example of the procedure of the subsequent maturity determination process performed by the subsequent maturity determination unit 33 .

[0155] In the hue calculation process 33a, the image generated by the image preprocessing unit 32 is used to calculate the ratio of the color of the stem that increases or decreases as the melon ripens (hereinafter referred to as the "hue ratio value"). The hue ratio value represents the ratio of the area of ​​the color region that decreases as the melon ripens relative to the overall area of ​​the stem in the image, or the ratio of the area of ​​the color region that increases as the melon ripens relative to the overall area of ​​the stem in the image.

[0156] In the size calculation process 33b, the size of the stem that decreases as the melon ripens (hereinafter referred to as the "size measurement value") is calculated using the image generated by the image pre-processing unit 32. The size measurement value represents the minimum value of the stem thickness (the size of the thinnest part).

[0157] The ripeness calculation process 33c calculates the ripeness using the hue ratio calculated in the hue calculation process 33a and the size measurement value calculated in the size calculation process 33b. In this embodiment, a larger value indicates less ripeness, while a smaller value indicates more mature ripening.

[0158] Hereinafter, the details of each of the color tone calculation process 33 a , the size calculation process 33 b , and the degree of maturity calculation process 33 c will be described in order.

[0159] Regarding Hue Calculation Processing 33a

[0160] As described above, the color tone calculation process 33 a is a process of calculating the ratio (color tone ratio value) of the stem color that increases or decreases as the ripening of the melon proceeds.

[0161] One characteristic that changes over time as a melon ripens is the change in the hue of the stem. Generally speaking, as the melon ripens, the hue value of yellow increases, while the hue value of green decreases. Hue values, as used here, refer to the value of a color in a specified color system (e.g., the hue value in the HSV color space).

[0162] Figure 10 This is a diagram showing an example of temporal changes in the stems of melons after harvest.

[0163] In the pedicels of melons, changes in color and shape (or size) are observed as the date passes after harvest.

[0164] The color of the melon stem Figure 10 The order of (a), (b), (c), and (d) is changed as follows.

[0165] "Dark green" → "Light green" → "Green mixed with yellow" → "Yellow, brown"

[0166] Therefore, it can be said that the color components of the stem shown in the image that increase or decrease as the melon ripens are correlated with the number of days to ripening. Therefore, to understand the progress of ripening, the color ratio value is calculated by dividing the area of ​​the region with increasing or decreasing color components by the area of ​​the entire stem. Each area can be expressed as a number of pixels.

[0167] Here, the yellow to brown color components (the color components of the ripened portion) are removed from the HSV color space. The ratio of the remaining green component (the color component of the unripened portion) to the overall area of ​​the stem is calculated. This is used to calculate the hue ratio value of the green component, which decreases as ripening progresses. A larger hue ratio value for the green component indicates less ripeness, while a smaller value indicates more advanced ripening. Hue ratio values ​​are expressed in a range of 0 to 1.

[0168] An example of a procedure for calculating a tone ratio value is shown below.

[0169] 1) Count the number of pixels in the entire pedicle area excluding the background.

[0170] 2) Using the color threshold processing method described above, the color components (yellow to brown) that increase as ripening progresses are eliminated.

[0171] 3) Calculate the number of pixels in the area of ​​the green component remaining after elimination.

[0172] 4) Calculate the hue ratio value of the green component using the following calculation formula.

[0173] Hue ratio value = (number of pixels in the green component area) / (number of pixels in the entire pedicle area)

[0174] Figure 11 These are diagrams showing examples of images before and after the color component (yellow to brown) of the stem, which increases as ripening progresses, is deleted.

[0175] Figure 11 The image shown in (a) is an image before the color component (yellow to brown) of the stem that increases as the ripening progresses is eliminated. In this image, the number of pixels calculated for the area excluding the background (the entire stem area) is 4627.

[0176] Figure 11 The image shown in (b) is an image after the color component (yellow to brown) of the stem that increases as the ripening progresses is eliminated. In this image, the number of pixels calculated for the remaining green component area is 3688.

[0177] The hue ratio value of the green component in this case is 0.797 (=79.7%) according to the above calculation formula.

[0178] In addition, in the above description, an example is shown in which the hue ratio value of the remaining green component is calculated after eliminating the yellow to brown color components that increase as the ripening progresses, but instead, the hue ratio value of the remaining yellow to brown color components may be calculated after eliminating the green color components that decrease as the ripening progresses.

[0179] Regarding the yellow-to-brown color component hue ratio, smaller values ​​indicate less ripeness, while larger values ​​indicate more advanced ripening. Therefore, this can be calculated by calculating "1 - (yellow-to-brown color component hue ratio)" with larger values ​​indicating less ripeness and smaller values ​​indicating more advanced ripening.

[0180] · About size calculation processing 33b

[0181] As described above, the size calculation process 33b is a process for calculating the size (measured size value) of the stem that decreases as the melon ripens.

[0182] As the characteristics of muskmelon that change over time as the melon ripens, the change in the thickness of the stem can be mentioned. Generally speaking, as the melon ripens, the thickness of the stem decreases.

[0183] like Figure 10 As shown, the size and shape of the pedicle vary with Figure 10 (a) to (d) and changes as follows.

[0184] "A thick, firm shape" → "A shape that tapers and twists thinly"

[0185] Therefore, it can be said that the thickness of the stem reflected in the image is correlated with the number of days to ripening. Therefore, in order to understand the progress of ripening, the thickness of the stem is measured and the size measurement value is calculated. The larger the size measurement value, the less mature it is, and the smaller the value, the more mature it is. In this embodiment, although the thickness of the stem is expressed in pixels, a unit other than pixels that can express length (such as millimeters) can also be used.

[0186] In the following, while referring to Figure 12A 、 Figure 12B and Figure 13 , while explaining an example of the steps for calculating the dimensional measurement value.

[0187] 1) The pedunculated image is binarized, with "0" representing the black portion of the image and "1" representing the white portion.

[0188] Assuming that the entire area of ​​the image is black, then Figure 12A As shown in (1) in the figure, the values ​​of all pixels arranged in a matrix in the image are "0". On the other hand, if the area in the image is all white, then Figure 12A As shown in (2) in , all pixels have a value of "1". If part of the area in the image is white and the rest is black, then Figure 12A As shown in (3), the values ​​of some pixels are "1" and the values ​​of the remaining pixels are "0".

[0189] 2) If Figure 12B As shown in (a) of FIG, when the image of the pedicle extending in the horizontal direction (X direction) of the image is binarized, the same as Figure 12A In this case, the pedicle portion is represented by a value of "1", and the background portion other than the pedicle is represented by a value of "0". Figure 12B As shown in (b) of the figure, the number of pixels arranged in the vertical direction (Y direction) of the image is counted for each column. This yields X count values ​​(X: the number of pixels in the horizontal direction of the image). The X count values ​​represent the thickness of each part of the pedicle.

[0190] 3) Among the X count values, the thinnest portion of the pedicle reflected in the image is determined by searching for a count value indicating a minimum value other than 0.

[0191] 4) Sometimes, the thinnest part of the pedicle shown in the image cannot be accurately determined depending on the shape of the pedicle end. Figure 13The state shown in (a) becomes the state shown in (b), but sometimes the pedicle end that appears in the image has a pointed shape as shown by the dotted line. Figure 13 As shown in (c) in FIG. 1 , if the number of pixels is counted in the vertical direction (Y direction) at position (1), the count value becomes the minimum value.

[0192] Therefore, excluding the two ends of the pedicle, the part other than the two ends of the pedicle is searched for the part showing the minimum count value other than 0. Figure 13 The position (2) shown in (c) is targeted at the central part of the stem, and a portion showing a count value of the minimum value other than 0 is searched. Then, the count value showing the minimum value other than 0 is calculated as the size measurement value.

[0193] · Regarding the calculation process of ripeness 33c

[0194] As described above, the ripeness calculation process 33 c is a process for calculating the ripeness using the color tone ratio value calculated by the color tone calculation process 33 a and the size measurement value calculated by the size calculation process 33 b .

[0195] In this ripeness calculation process 33c, a calculation process using the tone ratio value and the size measurement value is performed to calculate the ripeness expressed in a range of 0 to 1. Here, the value obtained by applying a predetermined adjustment process to the tone ratio value is represented by A, and the value obtained by applying a predetermined adjustment process to the size measurement value is represented by B. By adding A and B, the ripeness expressed in a range of 0 to 1 is calculated.

[0196] Below, refer to Figure 14A , an example of the steps of calculating the degree of ripeness in the degree of ripeness calculation process 33c is described.

[0197] 1) Set the tone scale value to a (step S41).

[0198] 2) The hue ratio value a represents the hue ratio of the green component, which decreases as the melon ripens, within a range of 0 to 1. As described above, the calculation of the degree of ripening uses not only A but also B. Therefore, the weight ratio of A and B is set to, for example, "1:1," and the range of A is adjusted to between 0 and 0.5 (hue value adjustment). Here, the weighting coefficient for a is set to "0.5," and the value obtained by calculating "a × 0.5" is set as A (step S42).

[0199] However, the example shown in 2) above is merely an example and is not limiting. For example, the weight ratio described above may be changed to a ratio other than "1:1" depending on the type of melon being processed or the harvest period. Accordingly, the weighting coefficient multiplied by a may also be changed to a value other than "0.5."

[0200] 3) The dimension measurement value is set to b (step S43).

[0201] 4) The size measurement value b is a value indicating the thickness of the stem that decreases as the melon ripens. In the embodiment, assuming that the value range of b is 50 pixels or more and 250 pixels or less, the value range of the size measurement value b is divided into multiple classes (multiple ranges), and b to B are calculated for each class. For example, Figure 14A The dimensional measurement value b is classified in advance as shown in step S44 in FIG.

[0202] Figure 14B Schematically shown Figure 14A The range of b in each category shown in step S44. Figure 14B In the example, the range of b corresponding to B=0.2, 0.3, and 0.4 is set to be narrower than the range of b corresponding to B=0.1 and 0.5, so that the progress of ripening can be distinguished more finely.

[0203] Here, as mentioned above, it is assumed that the value range of b is greater than 50 pixels and less than 250 pixels, and 5 ranges of B are set based on this assumption. However, there may be a case where b is less than 50 pixels or greater than 250 pixels. In order to appropriately determine the value of B in such a case, Figure 14A In step S44, the lower limit value of the smallest range among the five ranges of b is set to 0, and no upper limit is set for the largest range.

[0204] In addition, the following example is shown: assuming that the value range of b is greater than or equal to 50 pixels and less than or equal to 250 pixels, the possible values ​​of the size measurement value b are divided into Figure 14A The five categories (five ranges) shown in step S44 are described above, but are not limited to this example.

[0205] For example, you can also Figure 14A Step S44 in is replaced by Figure 14C The step S44' shown in FIG. 4 is to enable the case where the value range of b is not less than 50 pixels and not more than 250 pixels to be applicable (generalized). Figure 14BAs shown, the range of b within each category remains unchanged, but in step S44', the initial value of the thickness of the melon stem at the time of harvest is set as the variable w. By using an equation containing the variable w, the range of b within each category and the ratio of each range can be changed. In step S44', w is assumed to be 200, but w can be changed to a value other than 200. The appropriate value of w (as well as the appropriate range of b within each category and the appropriate ratio of each range) varies depending on the variety of melon being processed and the harvest time. If the variety and harvest time are pre-stored in the QR code described above, the burden on the user can be minimized. The value of w can also be pre-stored in a storage device accessible by the ripening determination unit 33 and updated based on the variety and harvest time obtained from the QR code. In addition, in steps S44 and S44', an example of five categories is shown, but this is not limited to this example. For example, by increasing the number of categories, B can be categorized more finely. This allows for more detailed differentiation of the progress of ripening, for example, to address situations where ripening is over-ripening or under-ripening. Increasing or decreasing the number of categories, in other words, the accuracy of ripeness determination is in a trade-off with the data processing load and data capacity. Therefore, it is preferable to consider both and appropriately set the values. Furthermore, the range of b defining each category can be freely changed based on the type and harvest period of the melon being tested. Information such as a table associating the range of b defined for each category with the melon variety and harvest period can be stored in a storage device accessible by the ripeness determination unit 33.

[0206] 5) Finally, the degree of ripeness is calculated by calculating "A+B" using A obtained in step S42 and B obtained in step S44 (step S45).

[0207] While this example illustrates the use of two indicators (criteria)—A and B—to calculate ripeness, the number of indicators is not limited to this example. Three or more indicators can also be used to calculate ripeness. This allows for an appropriate determination using the remaining two indicators, even if one indicator exhibits significant error due to individual differences. This improves accuracy.

[0208] When using three indicators, the degree of ripeness can be calculated by adding not only A and B but also another indicator, C. The degree of ripeness obtained using AI technology, described later, can also be used as C. This allows for a more accurate determination result.

[0209] The appropriate value of the weighting coefficient for A (the weighting coefficient multiplied by a used in the calculation of A) and the appropriate maximum value of each indicator, such as A and B, vary depending on the number of indicators processed and / or the ratio of each indicator. In this case, the indicators are pre-adjusted so that the sum of the indicators is 1. For example, if three indicators, A, B, and C, are used and the ratio is set to 1:1:1, the maximum value of each indicator is 1 / 3.

[0210] Figures 15 to 17 Shows some examples of images used in the degree of maturity determination process.

[0211] Figure 15 represents the image captured on the first day, Figure 16 represents the image three days after the first day of shooting. Figure 17 Indicates images taken one week after the first day of shooting.

[0212] exist Figures 15 to 17 In each figure, (a) shows an image that has undergone image preprocessing, (b) shows an image after the color components of the pedicle (yellow to brown) that increase as the ripening progresses are eliminated after image preprocessing, and (c) shows an image that has been binarized after the yellow to brown color components have been eliminated.

[0213] in addition, Figure 18 The following table summarizes the Figures 15 to 17 A table of information obtained for each image.

[0214] Figure 18 The images A, B, and C shown in the table refer to Figure 15 、 Figure 16 、 Figure 17 .

[0215] exist Figure 18 In the table, for each of images A, B, and C, the values ​​of the "foreground area after preprocessing" (i.e., the area of ​​the pedicle area obtained from the image after image preprocessing), the "area after yellow component elimination" (i.e., the area of ​​the green component remaining after eliminating the yellow to brown color components), and the "green component hue ratio value" (i.e., the hue ratio value of the green component) obtained in the hue calculation processing 33a are recorded, and the values ​​of the "size measurement value" obtained in the size calculation processing 33b are recorded, and the values ​​of the "hue value", "size value", and "maturity" obtained in the maturity calculation processing 33c are recorded.

[0216] After verifying these values, it was confirmed that good accuracy results were obtained.

[0217] (1-4) Eating Time Determination Unit 34

[0218] As mentioned above, Figure 1 The eating timing determination unit 34 shown performs a process of determining whether the melon is ready to be eaten based on the degree of ripeness determined by the ripeness determination unit 33 (eating timing determination process).

[0219] Figure 19 1 is a diagram showing an example of the procedure of the consumption timing determination process performed by the consumption timing determination unit 34 .

[0220] The degree of ripeness determination unit 33 calculates the degree of ripeness expressed in a range of 0 to 1.

[0221] In the eating time determination unit 34, it is determined which of the three categories "eating time", "wait a little longer", and "unripe" the food falls into based on the degree of ripeness. For this purpose, the range of the degree of ripeness that can be taken is divided into multiple categories (multiple ranges), and the eating time determination result ("eating time", "wait a little longer", and "unripe") is obtained based on the degree of ripeness for each category. For example, Figure 19 Classification is performed in advance as shown in step 34a in FIG. On this basis, the eating time determination result is output according to which category the ripeness belongs to.

[0222] In addition, although the classification into three categories "time to eat", "wait a little longer", and "unripe" is exemplified here, more detailed classification is also possible.

[0223] (1-5) Result output unit 35

[0224] As mentioned above, Figure 1 The result output unit 35 shown performs a process (result output process) of notifying the terminal 1 side of the result (consumption timing determination result) determined by the consumption timing determination unit 34 .

[0225] In the result output process, in addition to the result of the eating time determination, the results calculated in the ripeness determination process (ripeness, tone ratio value, size measurement value, etc.) are also output. In addition, other numerical values ​​and / or various images (such as Figures 15 to 17 These information are transmitted from the server 1 to the terminal 2 via the communication devices 12 and 24 and displayed on the display device 25.

[0226] (2) Modification

[0227] Then, Figure 1 A modified example of the system described in is described.

[0228] Figure 20 Yes Figure 1 FIG. 1 is a diagram of a modified example of the system described in FIG. Figure 20 In, with Figure 1The same reference numerals are used to designate the common elements, and redundant descriptions are omitted.

[0229] Figure 20 system and Figure 1 The difference of the system is that the server 1 further includes a storage device 15, etc. The storage device 15 stores the AI ​​model and various data related thereto, which will be described later. The data processing device 13 performs information processing using the AI ​​model or various data related thereto under the control of the control device 14. In addition, Figure 2 The AI ​​technology described below is applied to the doneness determination process and the consumption timing determination process in the consumption timing determination system 30 described in

[15] .

[0230] (2-1) Application of AI Technology

[0231] In this example, AI technology is used to achieve more accurate results for determining doneness or ready-to-eat food. To achieve this, an AI model is pre-created and fed with a large number of images for training. The AI ​​model in this context is an inference model created through deep learning using images.

[0232] In this example, we assume a task (image classification task) that performs classification based on image features (e.g., classification into "time to eat," "wait a while," or "undercooked"). Using this image classification task, we create an inference model through deep learning, and perform inference using this inference model.

[0233] When creating an AI model, it is preferable to implement it under fixed conditions, such as setting the area of ​​the image being trained on the AI ​​to the same background (e.g., using a common backing plate P) except for the object (the stem of the melon). For example, by using the backing plate P described above to capture the object and then performing subsequent image preprocessing, it is easy to only capture the object in a specified area.

[0234] By pre-storing a large number of images captured in this way in a predetermined storage device and using these images to create an AI model, the AI ​​model can be used to perform high-precision inferences, thereby obtaining high-precision results for determining the degree of doneness or timing of consumption.

[0235] The following describes the process from AI model creation to execution.

[0236] Figure 21 This is a diagram schematically showing the flow of processing for obtaining a result of determining the degree of doneness or the timing of consumption using an AI model.

[0237] Figure 21 The processing shown can be roughly divided into a learning process for building the decision-making capability of AI and an inference process for actually making the decision. Figure 21(a) in the figure represents the learning process. Figure 21 (b) in FIG. 1 shows the flow of inference processing.

[0238] In the learning process, each image is associated with a "class," which is the final answer for AI classification. A large number of images are prepared for each class. This is called a dataset.

[0239] In this example, a training data set is prepared. The training data set consists of a large number of images of melon stems that have been pre-labeled with labels such as "eat now," "wait a little longer," and "unripe."

[0240] Based on such a data set, an AI model 41 (learned model) is created using an arbitrary deep learning method in deep learning 40. The deep learning method here refers to image classification based on an image classification task.

[0241] In addition, the method of creating an AI model can be either a method of creating an AI model from scratch or a method of performing transfer learning to adapt an existing model on the network to its own requirements.

[0242] In the inference process, the created AI model 41 is used to perform inference processing on an unknown image X. Here, for an image X of a melon stem, for which the ripeness or eating time determination result is to be measured, AI model 41 is used to perform classification processing 42, thereby classifying it as "eat now," "wait a little longer," "unripe," etc., and outputting the corresponding classification ("eat now," "wait a little longer," "unripe," etc.) as the result.

[0243] (2-2) Effects of applying AI technology

[0244] This system focuses on the stem of the melon reflected in the captured image, quantifying its color and shape using image processing technology to calculate the final ripeness. Furthermore, by incorporating judgments based on AI models created using images captured under the same conditions, comprehensive judgments can be made, achieving higher accuracy and improved versatility. Furthermore, generally speaking, increasing the number of training images allows for the creation of versatile AI that can handle a wide range of situations. Therefore, increasing the number of training data sets can improve overall judgment accuracy.

[0245] (3) Operation Example of Eating Timing Determination System 30

[0246] Below, refer to Figure 22 An example of the basic operation of the eating timing determination system 30 is described with reference to the flowchart of FIG.

[0247] First, the image acquisition unit 31 acquires an image of the stem of the melon from the terminal 1 (image acquisition processing) (step S1 ).

[0248] Next, the image input unit 32 performs a process of inputting the image acquired by the image acquisition unit 31 (image input process) (step S2 ).

[0249] The image pre-processing unit 32 performs processing (image pre-processing) of processing the image input by the image input unit 32 before the familiarity determination unit 33 uses the image (step S3 ).

[0250] Next, the ripening degree determination unit 33 performs the following processing (ripening degree determination processing) (step S4): obtains the color and shape information of the stem from the stem image processed by the image pre-processing unit 32, and determines the ripening degree of the melon based on the obtained color and shape information.

[0251] Next, the eating timing determination unit 34 performs a process (eating timing determination process) of determining whether the melon is ready to be eaten based on the ripeness determined by the ripeness determination unit 33 (step S5 ).

[0252] Next, the result output unit 35 performs a process (result output process) of notifying the terminal 1 of the result determined by the consumption timing determination unit 34 (step S6 ).

[0253] (4) Effects of Implementation

[0254] The effects of the above-described embodiment will be described below.

[0255] (4-1) Improvement of the accuracy of determining when to eat

[0256] In this embodiment, ripeness is determined by focusing on the melon's stem. The stem's dimensions (thickness) and hue (color component) are digitized to determine ripeness and consumption timing. While conventional ripeness determination systems use a single criterion, this system uses two criteria (three if the aforementioned AI-based determination is included), enabling determinations based on multiple criteria. This allows for greater accuracy even when one criterion has significant errors due to individual differences.

[0257] (4-2) Judgment system for compliance with preferences

[0258] In this embodiment, the ripeness determination unit 33 calculates the ripeness of the melon based on the digitized results of the characteristics that change over time as the melon ripens. The eating timing determination unit 34 then makes a final eating timing determination based on whether the ripeness exceeds a predetermined threshold. The threshold generally refers to the average value of the time when the fruit of the melon softens and becomes easily edible. However, by setting different thresholds to suit the preferences of those who prefer softer or firmer fruit, a system for determining eating timing can be provided that caters to the diverse preferences of consumers.

[0259] (4-3) Reduction of environmental constraints

[0260] In the embodiment, one of the options is to prepare a melon with a predetermined backing plate P attached to the stem S, and then photograph the melon by aligning the guide displayed by the dedicated application with the backing plate P. The backing plate P is, for example, approximately 3 to 5 cm square and has a background color that facilitates the subsequent background adjustment process 32d. In a general determination system using an image, in order to perform the process of identifying and converting components other than the measured object reflected as background, there are sometimes restrictions on the environment during the photographing process, such as the need to photograph against a black background. In contrast, in the embodiment, by photographing the melon with the backing plate P attached, components other than the stem of the melon are no longer reflected. Therefore, compared to the past, there are no environmental restrictions, and the timing of consumption can be easily determined.

[0261] (4-4) Measurement can be performed at any timing

[0262] In the embodiment, it is possible to confirm whether the melon is ready to be eaten at that moment using the eating timing determination system. Furthermore, the eating timing determination system can be used at any time. Therefore, actual users such as consumers can easily determine whether the melon is ready to be eaten at any time.

[0263] As described above in detail, according to the embodiment, the timing of eating the cantaloupe can be easily determined.

[0264] The present invention is not limited to the above-mentioned embodiments as they are, and in the implementation stage, the constituent elements can be deformed and concretized within the scope of the main purpose. In addition, various inventions can be formed by appropriately combining the multiple constituent elements disclosed in the above-mentioned embodiments. For example, several constituent elements can also be deleted from all the constituent elements shown in the embodiments. In addition, the constituent elements of different embodiments can also be appropriately combined.

[0265] Description of Reference Numerals

[0266] 1…server, 2…terminal, 11…storage device, 12…communication device, 13…information processing device, 14…control device, 15…storage device, 21…input device, 22…shooting device, 23…storage device, 24…communication device, 25…display device, 26…control device, 30…time-to-eat determination system, 31…image acquisition unit, 32…image pre-processing unit, 32a…image input processing, 32b…ratio adjustment processing, 33b…area clipping processing, 32d…background adjustment processing, 32e…position adjustment processing, 33…ripeness determination unit, 33a…hue calculation processing, 33b…size calculation processing, 33c…ripeness calculation processing, 34…time-to-eat determination unit, 35…result output unit, L1…first frame, L2…second frame, M…sheet, P…backing plate, S…stem.

Claims

1. A system for determining when to eat, wherein: have: a ripening degree determination unit configured to determine the ripening degree of the melon based on at least two elements obtained from an image of the melon stem; and The eating time determination unit determines whether the melon is in an eating time state based on the ripeness degree.

2. The eating time determination system according to claim 1, wherein: The ripening degree determination unit performs processing for calculating a ratio of the color of the stem that increases or decreases as the ripening of the melon progresses, using the stem image.

3. The eating time determination system according to claim 1, wherein: The ripeness determination unit performs processing to calculate the thickness of the stem using the stem image.

4. The eating time determination system according to claim 1, wherein: The ripening degree determination unit acquires information on the color and thickness of the stem from the stem image, and determines the ripening degree of the melon using the acquired color and thickness information.

5. The eating time determination system according to claim 4, wherein: The ripening degree determination unit determines the ripening degree of the melon using information on the color and thickness of the stem and information on the ripening degree of the melon obtained using AI technology.

6. The eating timing determination system according to any one of claims 1 to 5, wherein: The eating timing determination system further includes an image acquisition unit that acquires an image obtained by photographing the pedicle and the backing plate in a state where a backing plate is attached to the back of the pedicle.

7. The eating timing determination system according to claim 6, wherein: The image acquisition unit acquires an image captured by aligning a second frame displayed on a display device of a terminal with a first frame marked in advance on the backing sheet.

8. The eating time determination system according to claim 6, wherein: The system further includes an image preprocessing unit that processes the image acquired by the image acquisition unit before the familiarity determination unit uses the image, and performs processing to cut out a predetermined area from the image to be processed.

9. The eating time determination system according to claim 8, wherein: The image preprocessing unit processes the image to be processed by changing the image portion corresponding to the background of the pedicle to a specific color based on the color of the backing plate.

10. The eating time determination system according to claim 8, wherein: The image preprocessing unit performs processing to move a pedicle in an image to be processed to the center of the image and to rotate the pedicle so that a longitudinal direction of the pedicle faces a predetermined direction.

11. A method for determining the timing of consumption, wherein: The steps include: The ripening degree determining unit determines the ripening degree of the melon based on at least two elements obtained from the image of the melon stem; as well as The eating time determination unit determines whether the melon is in an eating time state based on the ripeness degree.

Images