Information processing device, information processing method, information processing system, information processing program, and recording medium

Abstract

This information processing device comprises: an acquisition means for acquiring an image obtained by imaging an object to be evaluated; a detection means for detecting, from the image, the contour of the object to be evaluated and the center line of the contour of the object to be evaluated; an index calculation means for deriving the area of a region surrounded by the center line of the contour of the object to be evaluated and a straight line connecting both ends of the center line; and a determination means for determining quality evaluation of the object to be evaluated on the basis of the derived area and outputting the determination result.

Description

Information processing device, information processing method, information processing system, program, and recording medium

[0001] The present invention relates to an information processing device, an information processing method, an information processing system, a program, and a recording medium that support evaluation work related to an object.

[0002] Quality evaluation based on the appearance of an object, such as its shape and size, has traditionally been performed. For example, because the more attractive a product's appearance is, the higher its demand tends to be, shipping standards that determine the grade (quality) of the product are set based on appearance criteria such as color, size, shape, and the presence or absence of defects. Produced products are then sorted into grades according to the shipping standards and traded. However, because the quantity of produced products is enormous, manual sorting can be a heavy burden. Furthermore, sorting of the products is sometimes performed by a large number of people, which can result in inconsistent grades for the sorted products. For this reason, in recent years, technologies have been proposed that use photographed images of the products to assist in determining the grade of the products.

[0003] JP 2013-169156 A

[0004] Detecting the shape of an object using an image requires evaluating the entire object, which results in a huge processing load, and it has not always been possible to accurately evaluate the quality of an object based on its shape. The present invention aims to provide an information processing device that can accurately evaluate the quality of an object based on its appearance using an image of the object.

[0005] The information processing device of the present invention is characterized by having an acquisition means for acquiring an image of an object to be evaluated, a detection means for detecting the contour of the object to be evaluated and the center line of the contour of the object to be evaluated from the image, an index calculation means for deriving the area of ​​the region surrounded by the center line of the contour of the object to be evaluated and a straight line connecting both ends of the center line, and a judgment means for making a judgment regarding the quality evaluation of the object to be evaluated based on the derived area and outputting the judgment result.

[0006] According to the present invention, it is possible to accurately evaluate the quality of an object based on its appearance using an image of the object.

[0007] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system. FIG. 2 is a diagram illustrating an example of the hardware configuration of an information processing device. FIG. 3 is a diagram illustrating an example of the functional configuration of the information processing device. FIG. 4 is a flowchart illustrating an example of processing by the information processing device. FIG. 5A is a diagram illustrating an example of photographing an object. FIG. 5B is a diagram illustrating an example of photographing an object. FIG. 6 is a diagram illustrating an example of feature information related to shape. FIG. 7A is a diagram illustrating an example of an evaluation index. FIG. 7B is a diagram illustrating an example of an evaluation index. FIG. 7C is a diagram illustrating an example of an evaluation index. FIG. 8 is a diagram illustrating an example of output of a determination result. FIG. 9 is a diagram illustrating an example of a mask image. FIG. 10 is a flowchart illustrating an example of result change processing. FIG. 11 is a diagram illustrating an example of an evaluation index. FIG. 12A is a diagram illustrating an example of an evaluation index. FIG. 12B is a diagram illustrating an example of an evaluation index. FIG. 13A is a diagram illustrating an example of output of a determination result. FIG. 13B is a diagram illustrating an example of output of a determination result.

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0009] First Embodiment FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to an embodiment of the present invention. The information processing system according to this embodiment is an information processing system that supports evaluation work on an object to be evaluated (hereinafter also referred to as an "object"), and uses an image of the object to perform quality evaluation of the object based on the appearance of the photographed object. The information processing system according to this embodiment includes an information processing device 100, an imaging device 110, and a display device 120. The information processing device 100 is communicatively connected to each of the imaging device 110 and the display device 120. The information processing device 100, the imaging device 110, and the display device 120 may be connected by wire or wirelessly. Furthermore, the information processing device 100, the imaging device 110, and the display device 120 may be communicatively connected to each other via a network.

[0010] The information processing device 100 acquires an image of an object (target object) 130 to be evaluated from the imaging device 110, and performs a quality evaluation of the target object using the acquired image. The information processing device 100 detects feature information about the target object's appearance from the image of the target object acquired from the imaging device 110, and evaluates the quality of the target object based on the detected feature information. The feature information about the target object's appearance includes, for example, information about the target object's color, size, shape, etc. The feature information about the target object's appearance may further include information about the presence or absence of defects on the target object, the surface condition, color, gloss, etc.

[0011] The imaging device 110 is an imaging device such as a camera, and captures an image of an object (target) 130 to be evaluated. The information processing system in this embodiment may be provided with any number of imaging devices 110, and for example, multiple imaging devices 110 may be provided to capture images of the target 130 from multiple directions. Note that the imaging device 110 is not limited to a device having only an imaging function such as a camera, but may also be a device having an imaging function such as a smartphone or tablet terminal.

[0012] The display device 120 is a display device for displaying the results of quality evaluation by the information processing device 100, etc. For example, the display device 120 is a display that displays display information on a screen, or a projector that projects and displays display information. Furthermore, for example, the display device 120 may be smart glasses (AR glasses). Furthermore, for example, the functions of the imaging device 110 and the display device 120 may be realized using a smartphone, a tablet terminal, or the like, and the results of quality evaluation, etc. may be superimposed on an image of the object 130.

[0013] 2 is a diagram showing an example of the hardware configuration of the information processing device 100 according to this embodiment. The information processing device 100 includes a CPU 201, a ROM 202, a RAM 203, an auxiliary storage device 204, an output device 205, an input device 206, and a network I / F 207. The CPU 201, the ROM 202, the RAM 203, the auxiliary storage device 204, the output device 205, the input device 206, and the network I / F 207 are communicably connected via a system bus 208.

[0014] The CPU (Central Processing Unit) 201 is a central processing unit that controls various operations of the information processing device 100. For example, the CPU 201 may control the operation of the entire information processing device 100. The ROM (Read Only Memory) 202 stores control programs, boot programs, and the like that can be executed by the CPU 201. The RAM (Random Access Memory) 203 is the main storage memory of the CPU 201, and is used as a work area or a temporary storage area for expanding various programs.

[0015] The auxiliary storage device 204 stores various data, various programs, etc. The auxiliary storage device 204 is realized by a storage device that can temporarily or permanently store various data, such as a non-volatile memory such as a hard disk drive (HDD) or a solid state drive (SSD).

[0016] The output device 205 is a device that outputs various types of information and is used to present various types of information to a user. For example, the output device 205 is realized by a display device such as a display. The output device 205 may present information to a user by displaying various types of display information. As another example, the output device 205 may be realized by an audio output device that outputs sounds such as voice and electronic sounds. In this case, the output device 205 may present information to a user by outputting sounds such as voice and electronic sounds. Furthermore, the device used as the output device 205 may be changed as appropriate depending on the medium used to present information to a user.

[0017] The input device 206 is used to receive various instructions from a user. For example, the input device 206 may include an input device such as a mouse, a keyboard, or a touch panel. As another example, the input device 206 may include a sound collection device such as a microphone to collect voices uttered by the user. In this case, various analysis processes such as acoustic analysis and natural language processing may be performed on the collected voice, and the content of the voice may be recognized as an instruction from the user. Furthermore, the device used as the input device 206 may be changed as appropriate depending on the method for recognizing instructions from the user. Furthermore, multiple types of devices may be used as the input device 206.

[0018] The network I / F 207 is used for communication with external devices, etc. via a network. Note that the device used as the network I / F 207 may be changed as appropriate depending on the type of communication path and the communication method to be applied.

[0019] The CPU 201 loads a program stored in the ROM 202 or the auxiliary storage device 204 into the RAM 203 and executes the program, thereby realizing the functions and processes of the information processing device described below. The program of the information processing device 100 may be provided to the information processing device 100 by a recording medium such as a CD-ROM, or may be downloaded via a network, etc. When the program of the information processing device 100 is provided by a recording medium, the program recorded on the recording medium is installed in the auxiliary storage device 204 by inserting the recording medium into a predetermined drive device.

[0020] 2 is merely an example and does not necessarily limit the hardware configuration of the information processing device 100 in this embodiment. As a specific example, some components such as the output device 205 and the input device 206 may not be included. As another example, components according to the functions realized by the information processing device 100 may be added as appropriate.

[0021] 3 is a diagram showing an example of the functional configuration of the information processing device 100 according to this embodiment. The information processing device 100 includes an acquisition unit 301, an evaluation processing unit 302, a control unit 306, an input / output control unit 307, and a storage unit 308.

[0022] The acquisition unit 301 acquires an image of an object (target) to be evaluated from the imaging device 110. In addition to the image of the target, the acquisition unit 301 may acquire point cloud data about the target obtained by a distance measurement sensor such as LiDAR, or other information about the target obtained by a weighing scale, a saccharometer, etc. For example, by acquiring point cloud data about the target, it is possible to obtain information about the unevenness, etc. on the surface of the target.

[0023] The evaluation processing unit 302 evaluates the quality of the object based on the images, information, etc. acquired by the acquisition unit 301. For example, the evaluation processing unit 302 detects feature information about the object from the image of the object acquired by the acquisition unit 301, and evaluates the quality of the object based on the detected feature information. The evaluation processing unit 302 has a detection unit 303, an index calculation unit 304, and a determination unit 305.

[0024] The detection unit 303 detects feature information about the object from the images, information, etc. acquired by the acquisition unit 301. For example, the detection unit 303 uses a trained model obtained in advance through machine learning to detect feature information about the appearance of the object, such as its color, size, and shape, from the images of the object acquired by the acquisition unit 301. Furthermore, for example, the detection unit 303 may detect information such as the presence or absence of flaws in the object and the surface condition as feature information from the images, information, etc. acquired by the acquisition unit 301.

[0025] The index calculation unit 304 calculates an evaluation index used for evaluating the quality of the object based on the feature information about the object detected by the detection unit 303. The index calculation unit 304 calculates an evaluation index for evaluating the degree of deformation or bending of the object based on the feature information detected by the detection unit 303, for example.

[0026] The determination unit 305 determines the quality evaluation of the object based on the characteristic information detected by the detection unit 303 and the evaluation index obtained by the index calculation unit 304, and outputs the determination result. The determination unit 305, for example, compares the detected characteristic information or the obtained evaluation index with a set threshold value to determine the quality of the object. For example, if the object is an agricultural product, the determination unit 305 determines the grade of the product in terms of shipping standards, etc.

[0027] The control unit 306 is responsible for controlling each component of the information processing device 100. The input / output control unit 307 performs various processes related to presenting various information to the user via the display device 120 or the like and accepting input of information (e.g., instructions, etc.) from the user via an input device or the like. For example, the input / output control unit 307 may perform processes related to presenting a UI (User Interface) and accepting input via the UI. This enables the information processing device 100 to recognize instructions from the user and present the results of processing in accordance with those instructions to the user.

[0028] The storage unit 308 stores various data and the like used when performing processing in the information processing device 100. The storage unit 308 stores, for example, a trained model that has undergone machine learning and that is used for inferring feature information performed by the detection unit 303, a threshold value that is used for quality evaluation performed by the determination unit 305, and the like.

[0029] 4 is a flowchart showing an example of processing by the information processing device 100 according to this embodiment. In the following, an example of processing by the information processing device 100 will be described using an example in which an object to be evaluated (target object) is graded as a sweet potato, which is an agricultural product.

[0030] In step S401, the acquisition unit 301 acquires an image of a sweet potato, which is an object (target) to be evaluated, from the imaging device 110. In step S401, the acquisition unit 301 may acquire, in addition to the image of the sweet potato, which is the target, point cloud data indicating the surface of the sweet potato obtained by a distance measuring sensor or the like, or other information obtained by a weighing scale, a saccharometer, or the like. In this example of the present embodiment, the image of the sweet potato, which is the target, is captured as shown in Figures 5A and 5B.

[0031] 5A and 5B are diagrams illustrating an example of photographing a sweet potato, which is an object of interest. FIG. 5A shows a state from above of a platform 502 on which a sweet potato 501, which is an object of interest, is placed. FIG. 5B shows a cross section indicated by line II in FIG. 5A. In FIGS. 5A and 5B, the sweet potato 501 is placed on a platform 502 made of a transparent material. The platform 502 is formed in a V-shape, and when the sweet potato 501 is placed on it, it can be stably stationary at the bottom of the platform 502. In the example shown in FIGS. 5A and 5B, three imaging devices 503, 504, and 505 capable of photographing the sweet potato 501 placed on the platform 502 are arranged, and one sweet potato 501 is photographed from three different directions at the same time. The three imaging devices 503, 504, and 505 are preferably arranged on the same plane of the cross section of the table 502, offset by approximately 120 degrees, so that the shape of the target object (sweet potato 501) can be detected with high accuracy. By arranging the imaging devices 503, 504, and 505 in this manner, it is possible to capture the entire target object, sweet potato 501, including its back and sides. Furthermore, imaging devices 506 and 507 may be further arranged on the front and back sides of the table 502 (in the normal direction of the plane on which the imaging devices 503 to 505 are arranged) so that the front and back surfaces of the target object, sweet potato 501, can be captured.

[0032] 5A and 5B , it is also possible to use a single imaging device to capture an image of an object from a desired direction, without being limited to the examples shown in FIGS. 5A and 5B . For example, a track such as a rail may be installed around the object, and the imaging device may be moved along the track to capture an image of the object from a desired direction. Furthermore, some imaging devices may be replaced with components such as mirrors that capture images of the object, and the imaging devices may be positioned so that the images are captured within the angles of view of the other imaging devices, thereby reducing the number of imaging devices without reducing the number of directions from which the object can be captured. Images from multiple directions may also be captured by rolling the object down a slope or by a user directly holding and rotating the object in their hands. Furthermore, it is also possible to store which sides have been captured.

[0033] In step S402, the detection unit 303 of the evaluation processing unit 302 detects feature information about the object from the image, information, etc. acquired in step S401. For example, the detection unit 303 uses a trained model that has undergone machine learning to detect feature information about the appearance of the captured sweet potato from the image of the sweet potato, which is the object, acquired in step S401.

[0034] FIG. 6 is a diagram illustrating an example of feature information regarding the external shape of a sweet potato detected by the detection unit 303. The detection unit 303 detects an outline 601 of an object (sweet potato) from an image 600 of the object (sweet potato) using a trained model that has undergone machine learning. Furthermore, the detection unit 303 derives a rectangular region 602 that circumscribes (includes) the outline 601 of the object (sweet potato) based on the detected outline 601 of the object (sweet potato). Hereinafter, the rectangular region 602 that circumscribes the outline 601 of the object is also referred to as a "circumscribed rectangle." The circumscribed rectangle 602 is the rectangular region with the smallest area among the rectangular regions circumscribing the outline 601 of the object (sweet potato). Furthermore, the detection unit 303 derives a center line 603 of the outline 601 of the detected object (sweet potato) based on the outline 601 of the object (sweet potato). The detection unit 303 derives the center line 603 by setting the center of the contour as the midpoint of two intersections between the contour 601 and a line parallel to the short side of the circumscribing rectangle 602. In this way, the detection unit 303 detects the contour 601, the circumscribing rectangle 602, and the contour center line 603 as feature information related to the external shape for each direction in which the sweet potato, which is the target object, is photographed.

[0035] In addition, the detection unit 303 may detect information about the appearance, such as blemishes, diseases, wrinkles, and discoloration, as feature information from the image of the sweet potato, which is the object, acquired in step S401.

[0036] In step S403, the index calculation unit 304 of the evaluation processing unit 302 derives an evaluation index to be used in evaluating the quality of the object based on the feature information about the object detected in step S402. In the first embodiment, the index calculation unit 304 calculates, for example, the area ratio between the outline of the object and a circumscribing rectangle for each direction in which the object (sweet potato) is photographed based on the feature information detected in step S402 as an evaluation index for evaluating the degree of deformation of the object (sweet potato). The index calculation unit 304 determines, for example, the ratio between the number of pixels within the outline of the object and the number of pixels within the circumscribing rectangle as the area ratio between the outline of the object and the circumscribing rectangle. The evaluation index calculated in step S403 in this first embodiment will be described with reference to FIGS. 7A to 7C.

[0037] 7A to 7C are diagrams illustrating examples of evaluation indices calculated in step S403 in the first embodiment. As shown in FIG. 7A , when the difference between the area 701 within the object's outline and the area 702 of the circumscribing rectangle is small and the area ratio is close to 1, the difference in shape between the object's outline and the circumscribing rectangle is small, and the degree of deformation of the object is small. Note that in FIG. 7A , 713 is the center line of the object's outline. On the other hand, as shown in FIG. 7B , when the center line 713 of the object's outline is significantly curved and the degree of deformation of the object is large, the difference between the area 711 within the object's outline and the area 712 of the circumscribing rectangle becomes large, and the area ratio becomes a value significantly different from 1. Furthermore, as shown in FIG. 7C , when the center line 723 of the object's outline is slightly curved but the degree of deformation of the object is large, the difference between the area 721 within the object's outline and the area 722 of the circumscribing rectangle becomes large, and the area ratio becomes a value significantly different from 1. In this way, the area ratio between the outline of the object and the circumscribing rectangle is calculated as an evaluation index, and the degree of deviation of the area ratio from 1 is evaluated, thereby making it possible to evaluate the degree of deformation of the object.

[0038] Furthermore, the index calculation unit 304 may derive, as an evaluation index for each defect or disease in the object, the area ratio between the area of ​​the object estimated to be a defect or disease and the total area of ​​the object based on the characteristic information detected by the detection unit 303.

[0039] In step S404, the determination unit 305 of the evaluation processing unit 302 determines the grade of the sweet potato, which is the object, based on the evaluation index derived in step S403 and the feature information detected in step S402. The determination unit 305 evaluates the degree of deformation of the sweet potato, which is the object, and determines the grade, for example, by determining whether the area ratio between the outline of the object and the circumscribing rectangle, which was derived as the evaluation index in step S403, is within a predetermined range. At this time, the determination unit 305 determines the grade of the sweet potato, which is the object, based on the worst value (the area ratio farthest from 1) among the area ratios between the outline of the object and the circumscribing rectangle, which were derived for each direction in which the sweet potato, which is the object, was photographed.

[0040] For example, the determination unit 305 may determine the grade of the sweet potato object based on whether the area ratio of the area estimated to be flawed or diseased in the object derived as the evaluation index in step S403 to the total area of ​​the object (area estimated to be flawed or diseased / total area of ​​the object) exceeds a set threshold. The determination unit 305 may determine that an object estimated to be flawed or diseased is a grade corresponding to processing or disposal regardless of the area ratio. For example, the grade of the sweet potato object may be determined based on feature information regarding wrinkles or discoloration on the object detected in step S402. For example, a trained model may be used to classify the sweet potato object from a photographed image to determine the grade.

[0041] In step S405, the determination unit 305 outputs the determination result from step S404. The determination result output from the determination unit 305 is displayed on the display device 120, for example, via the input / output control unit 307. FIG. 8 is a diagram illustrating an example of the output of the determination result, and displays information 804 regarding the determination result, such as grade, length, and thickness, along with the outline 801, circumscribed rectangle 802, and center line 803 of the object detected as characteristic information in step S402. For example, the color of the outline 801 of the object to be displayed may be changed depending on the grade of the determination result. Note that, based on the determination result output from the determination unit 305, the sweet potatoes, which are the objects, may be sorted by grade using a sorting device (not shown) that can switch transportation routes, etc.

[0042] To enable real-time confirmation of the determination results, the determination results output from the determination unit 305 may be displayed using projection mapping. In this case, if the display position of the determination result is misaligned with the position of the target sweet potato, the image of the sweet potato projected and displayed misaligned will be captured by the imaging device, and repeated capture will result in multiple images being displayed in a misaligned, overlapping image. This can be avoided, for example, by displaying an image as shown in FIG. 9, in which a black mask is applied to all areas except the object contour 801, circumscribing rectangle 802, contour center line 803, and information about the determination result 804 shown in FIG. 8. Note that even if the entire image other than the object contour 801, circumscribing rectangle 802, contour center line 803, and information about the determination result 804 shown in FIG. 8 is not black-masked, a similar effect can be achieved by applying black masking to at least the area within the circumscribing rectangle 802 (excluding the object contour 801, circumscribing rectangle 802, and contour center line 803). In addition, the overlapping of multiple images can be avoided by displaying and photographing a projection image including a so-called AR marker, calculating a correction amount based on the position of the AR marker in the photographed image, and correcting the projection image.

[0043] Furthermore, when displaying the judgment result using projection mapping, the display of the circumscribing rectangle 802 of the object may be changed depending on the judgment result, as shown in FIG. 13A . FIG. 13A is a diagram illustrating an example of output of the judgment result. Note that while FIG. 13A illustrates the circumscribing rectangle of the object and information about the judgment result, the outline and center line of the outline of the object may also be displayed, as in the example described above. For example, if the object is judged to be a first-class product (object 1301 in the illustrated example), the circumscribing rectangle 802 may be displayed in green (circumscribing rectangle 1311 shown by a solid line in the figure); if the object is judged to be a second-class product (object 1302 in the illustrated example), the circumscribing rectangle 802 may be displayed in yellow (circumscribing rectangle 1312 shown by a dashed line in the figure); and if the object is judged to be a B-class product (object 1303 in the illustrated example), the circumscribing rectangle 802 may be displayed in red (circumscribing rectangle 1313 shown by a dashed line in the figure). Although the embodiment in which the color of the circumscribing rectangle 802 is changed according to the judgment result has been described above, the present invention is not limited to this. For example, the color of the circumscribing rectangle may be changed according to the size, length, thickness, etc. of the object. For example, if the judgment result is a first-class product and the length is equal to or greater than a predetermined length, the color of the circumscribing rectangle 802 may be displayed in green. If the judgment result is a first-class product and the length is within a predetermined length range, the color of the circumscribing rectangle 802 may be displayed in yellow. If the judgment result is a second-class product or the length is less than the predetermined length, the color of the circumscribing rectangle 802 may be displayed in light blue. If the judgment result is a third-class product, the color of the circumscribing rectangle 802 may be displayed in pink. In this way, by changing and displaying the color of the circumscribing rectangle 802 according to the judgment result, the size, length, thickness, etc. of the object, or a combination thereof, the user may be able to intuitively recognize the grade of the object, thereby improving the efficiency of sorting the objects by grade. Furthermore, as shown in FIG. 13B, the circumscribing rectangle 802 of the object may be displayed in a different color, and a legend 1321 of the color of the circumscribing rectangle 802 may be displayed by projection mapping.

[0044] If the user determines that the determination result by the determination unit 305 is inappropriate, the information processing device 100 may accept input from the user and change the determination result. Fig. 10 is a flowchart showing an example of a process for changing the determination result in the information processing device 100. By performing the process for changing the determination result described below, it becomes possible to automatically adjust the threshold value in the quality (grade) evaluation simply by inputting a change to the result without the user having to adjust detailed parameters, etc.

[0045] In step S1001, the control unit 306 determines whether or not the user has input a change to the results. If the control unit 306 determines that the user has input a change to the results (YES), the process proceeds to step S1002. On the other hand, if the control unit 306 determines that the user has not input a change to the results (NO), the change process shown in FIG. 10 ends.

[0046] In step S1002, the control unit 306 changes the judgment result for the object in accordance with the user's input to change the result. In step S1003, the control unit 306 adjusts and updates the threshold value for quality (grade) evaluation based on the judgment result changed by the user, and ends the change process shown in FIG.

[0047] According to the first embodiment, the information processing device 100 derives the area ratio between the area within the contour of the object and the area of ​​the circumscribing rectangle as an evaluation index for evaluating the degree of deformation of the object based on feature information detected from a captured image of the object. The information processing device 100 then evaluates the degree of deformation of the object by determining whether the area ratio between the area within the contour of the object and the area of ​​the circumscribing rectangle, derived as the evaluation index, is within a predetermined range, thereby making a determination regarding the quality evaluation of the object. This enables accurate quality evaluation of an object based on its appearance using a captured image of the object through processing with a relatively low processing load.

[0048] Second Embodiment In the first embodiment, in step S403 of FIG. 4, the index calculation unit 304 derives an evaluation index for evaluating the degree of deformation of the sweet potato, which is the object. In the second embodiment, the index calculation unit 304 derives an evaluation index for evaluating the curvature of the sweet potato, which is the object, instead of the evaluation index for evaluating the degree of deformation of the sweet potato, which is the object. The second embodiment is the same as the first embodiment described above except that the index calculation unit 304 derives an evaluation index for evaluating the curvature of the sweet potato, which is the object. Therefore, a description of these aspects will be omitted, and the following describes the derivation of the evaluation index in step S403 of FIG.

[0049] In the second embodiment, in step S403 of Fig. 4, the index calculation unit 304 derives an evaluation index for evaluating the curvature of the sweet potato as the object for each direction in which the sweet potato as the object is photographed, instead of an evaluation index for evaluating the degree of deformation of the sweet potato as the object. The evaluation index calculated in step S403 in the second embodiment will be described with reference to Fig. 11.

[0050] FIG. 11 is a diagram illustrating an example of an evaluation index for evaluating the curvature of a sweet potato, which is an object, calculated in step S403 in the second embodiment. In FIG. 11, 1101 to 1103 are feature information detected in step S402, where 1101 is the contour of the object, 1102 is a circumscribing rectangle, and 1103 is the center line of the contour. Based on the feature information detected in step S402, the index calculation unit 304 calculates the area of ​​the region enclosed by the center line 1103 of the contour of the object and the straight line 1104 connecting both ends of the contour center line 1103 (points 1103A and 1103B) as an evaluation index for evaluating the curvature of the sweet potato, which is an object. When the curvature of the object is small, the area of ​​the region enclosed by the center line 1103 of the object's contour and the straight line 1104 connecting both ends is small. On the other hand, if the curvature of the object is large, the area of ​​the region enclosed by the center line 1103 of the object's outline and the straight line 1104 connecting both ends of the center line 1103 will be large. Therefore, the area of ​​the region enclosed by the center line 1103 of the object's outline and the straight line 1104 connecting both ends of the center line 1103 will be calculated as an evaluation index, and the curvature of the object can be evaluated by evaluating its size.

[0051] In the second embodiment, in step S404 following step S403, the determination unit 305 evaluates the curvature of the sweet potato, which is the object, and determines its grade based on whether the area of ​​the region surrounded by the center line 1103 of the object's outline, which is derived as the evaluation index, and the straight lines 1104 connecting both ends of the center line 1103, exceeds a set threshold value. At this time, the determination unit 305 determines the grade of the sweet potato, which is the object, based on the maximum value of the area of ​​the region surrounded by the center line 1103 of the object's outline, which is derived for each direction in which the sweet potato is photographed, and the straight lines 1104 connecting both ends of the center line 1103.

[0052] According to the second embodiment, the information processing device 100 derives the area of ​​the region enclosed by the center line of the object's outline and the lines connecting both ends of the outline as an evaluation index for evaluating the curvature of the object based on feature information detected from a captured image of the object. The information processing device 100 then evaluates the curvature of the sweet potato object based on whether the area of ​​the region enclosed by the center line of the object's outline and the lines connecting both ends of the outline, derived as the evaluation index, exceeds a preset threshold, and makes a determination regarding the quality of the object. This directly evaluates the curvature of the object, enabling accurate quality evaluation of the object based on its appearance using a captured image of the object.

[0053] In the above description, the evaluation index for evaluating the curvature of the sweet potato object is calculated by calculating the area of ​​the region surrounded by the center line 1103 of the object's contour detected as feature information and the straight line 1104 connecting both ends of the center line 1103. However, this is not limited to this. For example, as shown in FIG. 12A, the index calculation unit 304 may calculate the area of ​​the region surrounded by the center line 1203 of the object's contour 1201 and the straight line 1204 that divides the circumscribing rectangle 1202 in half in the longitudinal direction as an evaluation index for evaluating the curvature of the sweet potato object. Also, for example, as shown in FIG. 12B, the circumscribing rectangle 1212 for the object's contour 1211 is divided into approximately 20 to 30 regions. The index calculation unit 304 may then calculate the maximum value of the angle 1215 between the center line 1213 of the contour 1211 of the object and a line 1214 parallel to the long side of the circumscribing rectangle 1212 as an evaluation index for evaluating the curvature of the sweet potato, which is the object.In this case, local curvature can be detected and evaluated.

[0054] The first and second embodiments may be combined. That is, the index calculation unit 304 may calculate both an evaluation index for evaluating the degree of deformation of the object and an evaluation index for evaluating the curvature of the object based on the feature information detected by the detection unit 303, and may use these evaluation indexes to evaluate the quality of the object. By using two different evaluation indexes in this way to evaluate the quality of the object, it is possible to prevent errors from occurring in the quality evaluation.

[0055] In addition, in the above-described embodiments, an example has been described in which the object (target) to be evaluated is a sweet potato, an agricultural product, and the grade is determined. However, the present invention is not limited to this, and can also be applied to the quality evaluation of other agricultural products, industrial products, daily necessities, etc. As for other agricultural products, the present invention can be applied to the quality evaluation of root vegetables and vegetables such as eggplant, cucumber, carrot, and burdock. Furthermore, as for other products, the present invention can be applied to the quality evaluation of the shape of handicrafts and food products such as bread, which are determined based on their appearance.

[0056] <Other Embodiments> The present invention can also be realized by executing the following process. That is, software (programs) that realize the functions of the above-described embodiments are supplied to a system or device via a network or various recording media. The computer (or CPU, MPU, etc.) of the system or device then reads and executes the program. Furthermore, computer-readable recording media on which the program is recorded and computer program products such as the program can also be applied as embodiments of the present invention. Examples of recording media that can be used include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs.

[0057] It should be noted that the above-described embodiments are merely examples of specific embodiments of the present invention, and the technical scope of the present invention should not be construed as being limited by these embodiments. In other words, the present invention can be embodied in various forms without departing from its technical concept or main features.

[0058] According to the present invention, it is possible to accurately evaluate the quality of an object based on its appearance using an image of the object.

Claims

1. An information processing device comprising: an acquisition means for acquiring an image of an object to be evaluated; a detection means for detecting the contour of the object to be evaluated and the center line of the contour of the object to be evaluated from the image; an index calculation means for deriving the area of ​​a region enclosed by the center line of the contour of the object to be evaluated and a straight line connecting both ends of the center line; and a judgment means for making a judgment regarding the quality evaluation of the object to be evaluated based on the derived area and outputting the judgment result.

2. The information processing device described in claim 1, characterized in that the detection means detects a rectangular area circumscribing the contour of the evaluation object, the index calculation means derives the area of ​​the area surrounded by the center line of the contour of the evaluation object and a line that divides the rectangular area in half longitudinally, instead of the area surrounded by the center line of the contour of the evaluation object and a line connecting both ends of the center line, and the judgment means makes a judgment regarding the quality evaluation of the evaluation object based on the derived area.

3. The information processing device described in claim 1 or 2, characterized in that the judgment means judges the curvature of the evaluation object based on the derived area, and makes a judgment regarding the quality evaluation of the evaluation object based on the judged curvature of the evaluation object.

4. The information processing apparatus according to claim 3, wherein said determining means determines whether said object to be evaluated is bent depending on whether said derived area exceeds a predetermined threshold value.

5. The information processing device described in claim 1, characterized in that the detection means detects a rectangular area circumscribing the contour of the object to be evaluated and a center line of the contour of the object to be evaluated, the index calculation means divides the rectangular area into a plurality of areas and, for each divided area, derives the angle between a line parallel to the long side of the rectangular area and the center line of the contour of the object to be evaluated, and the judgment means makes a judgment regarding the quality evaluation of the object to be evaluated based on the angle between the derived line parallel to the long side of the rectangular area and the center line of the contour of the object to be evaluated.

6. An information processing device according to any one of claims 1 to 4, characterized in that the acquisition means acquires images of the object to be evaluated taken from a plurality of directions, and the index calculation means derives the area for each of the plurality of directions.

7. The information processing apparatus according to claim 6, wherein said determining means makes a determination regarding the quality evaluation of said evaluation object based on the maximum value of said areas derived for each of said plurality of directions.

8. An information processing device as described in any one of claims 1 to 7, characterized in that it has a change means for accepting user input regarding a change in the judgment result, and changing the judgment result regarding the evaluation object based on the input, and changing the threshold value in the quality evaluation by the judgment means.

9. The information processing device according to any one of claims 1 to 8, wherein the object to be evaluated is an agricultural product.

10. The information processing device described in claim 9, characterized in that the detection means further detects information about at least one of defects, diseases, wrinkles, and discoloration in the agricultural product from the image, and the judgment means further makes a judgment regarding the quality evaluation of the agricultural product based on the detected information about at least one of defects, diseases, wrinkles, and discoloration in the agricultural product.

11. An information processing system comprising: an information processing device according to any one of claims 1 to 10; an imaging means for taking an image of the object to be evaluated; and a display means for displaying the judgment result.

12. The information processing system according to claim 11, characterized in that it has a plurality of said image capturing means and captures images of said evaluation object from a plurality of directions.

13. The information processing system according to claim 11 or 12, wherein the display means displays the determination result using projection mapping.

14. The information processing system according to claim 13, wherein said display means displays said determination result by masking an area other than the area showing said determination result.

15. An information processing system as described in claim 13 or 14, characterized in that the detection means detects a rectangular area circumscribing the contour of the object to be evaluated, and the display means displays a rectangle corresponding to the object based on the rectangular area detected by the detection means, and changes the color of the rectangle depending on the judgment result of the quality evaluation of the object.

16. The information processing system according to claim 15, characterized in that the display means changes the color of the rectangle and displays it according to at least one of the size, length, and thickness of the object in addition to the judgment result.

17. An information processing method executed by an information processing device, comprising: an acquisition step of acquiring an image of an object to be evaluated; a detection step of detecting the contour of the object to be evaluated and the center line of the contour of the object to be evaluated from the image; an index calculation step of deriving the area of ​​a region enclosed by the center line of the contour of the object to be evaluated and a straight line connecting both ends of the center line; and a judgment step of making a judgment regarding the quality evaluation of the object to be evaluated based on the derived area and outputting the judgment result.

18. A program that causes a computer of an information processing device to execute the following steps: an acquisition step of acquiring an image of an object to be evaluated; a detection step of detecting the contour of the object to be evaluated and the center line of the contour of the object to be evaluated from the image; an index calculation step of deriving the area of ​​the region enclosed by the center line of the contour of the object to be evaluated and a straight line connecting both ends of the center line; and a judgment step of making a judgment regarding the quality evaluation of the object to be evaluated based on the derived area and outputting the judgment result.

19. A computer-readable recording medium having recorded thereon a program that causes a computer of an information processing device to execute the following steps: an acquisition step of acquiring an image of an object to be evaluated; a detection step of detecting the contour of the object to be evaluated and the center line of the contour of the object to be evaluated from the image; an index calculation step of deriving the area of ​​the region enclosed by the center line of the contour of the object to be evaluated and a straight line connecting both ends of the center line; and a judgment step of making a judgment regarding the quality evaluation of the object to be evaluated based on the derived area and outputting the judgment result.

Images