Information processing system, information processing method and program

Abstract

An information processing system (100) comprises a feature extraction unit (112) and an estimation unit (113). The feature extraction unit (112) extracts features of an object shown in an image. The estimation unit (113) uses the extracted features to estimate a type of the object and a shape type and shape parameters corresponding to the object.

Description

Information processing system, information processing device, information processing method, and recording medium

[0001] The present disclosure relates to an information processing system, an information processing device, an information processing method, and a recording medium.

[0002] Various techniques have been proposed for recognizing objects contained in images.

[0003] For example, Patent Document 1 discloses a technology for recognizing objects that are not registered in training data. In the technology described in Patent Document 1, a known object recognition unit recognizes known objects registered in training data from an image. A generalized object recognition unit recognizes a generalizable generalized object by combining known objects registered in the training data.

[0004] For example, Patent Document 2 describes a technique for determining an approximate range (object range) in which a detected object exists from an image to be recognized. In the technique described in Patent Document 2, an object detection means performs a recognition process to determine whether a predetermined object exists in an input image. An object range determination means determines the object range of the detected object based on the detection result of the object detection means.

[0005] Patent Document 2 describes that the object range determination means uses, for example, the position and size of the object output by the object detection means to expand the detection window in which the object is detected by a predetermined ratio vertically and horizontally to determine the object range. According to Figure 7(A) of Patent Document 2, the detection window is rectangular.

[0006] JP 2019-220014 A JP 2016-018538 A

[0007] The present disclosure aims to improve upon the techniques described in the prior art documents mentioned above.

[0008] According to one aspect of the present disclosure, an information processing system is provided, comprising: a feature extraction means for extracting features of an object shown in an image; and an estimation means for estimating the type of the object, and the figure type and figure parameters corresponding to the object, using the extracted features.

[0009] According to one aspect of the present disclosure, there is provided an information processing device comprising: a feature extraction means for extracting features of an object shown in an image; and an estimation means for estimating the type of the object, and the figure type and figure parameters corresponding to the object, using the extracted features.

[0010] According to one aspect of the present disclosure, an information processing method is provided, which includes at least one computer extracting features of an object shown in an image, and using the extracted features to estimate the type of the object, and the figure type and figure parameters corresponding to the object.

[0011] According to one aspect of this disclosure, a recording medium is provided having recorded thereon a program for causing at least one computer to extract features of an object shown in an image, and use the extracted features to estimate the type of the object, and the figure type and figure parameters corresponding to the object.

[0012] 8 is a diagram illustrating an overview of an information processing system according to a first embodiment. FIG. 9 is a diagram illustrating an overview of an information processing device according to the first embodiment. FIG. 10 is a diagram illustrating an overview of information processing according to the first embodiment. FIG. 11 is a diagram illustrating an example of the configuration of an information processing system according to the first embodiment. FIG. 12 is a diagram illustrating an example of the physical configuration of an information processing device according to the first embodiment. FIG. 13 is a flowchart illustrating an example of information processing according to the first embodiment. FIG. 14 is a diagram illustrating a first example of an object type, figure type, and figure parameters to be estimated for an object. FIG. 15 is a diagram illustrating a second example of an object type, figure type, and figure parameters to be estimated for an object. FIG. 16 is a diagram illustrating a third example of an object type, figure type, and figure parameters to be estimated for an object. FIG. 17 is a diagram illustrating an example of the configuration of an information processing system according to a second embodiment. FIG. 18 is a diagram illustrating an example of the functional configuration of a figure estimation unit according to the second embodiment. FIG. 19 is a diagram illustrating an example of association information. FIG. 10 is a diagram illustrating first to third regions in the second example shown in FIG. 11. FIG. 12 is a flowchart illustrating an example of information processing according to the second embodiment. FIG. 13 is a flowchart illustrating an example of figure estimation processing according to the second embodiment. FIG. 14 is a diagram illustrating an example of the functional configuration of a parameter estimation unit 233 according to the second and third embodiments. FIG. 10 is a flowchart showing an example of a parameter estimation process according to embodiments 2 and 3. FIG. 11 is a diagram showing an example of a configuration of an information processing system according to embodiment 5. FIG. 12 is a flowchart showing an example of information processing according to embodiment 5. FIG. 13 is a diagram showing an example of a configuration of an information processing system according to embodiment 6. FIG. 14 is a flowchart showing an example of authentication processing according to embodiment 6. FIG. 15 is a diagram showing an example of a configuration of an information processing system according to modified example 1.

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all drawings, like components are designated by like reference numerals, and descriptions thereof will be omitted where appropriate.

[0014] 1 is a diagram showing an overview of an information processing system 100 according to embodiment 1. The information processing system 100 includes a feature extraction unit 112 and an estimation unit 113.

[0015] The feature extraction unit 112 extracts feature quantities of an object shown in an image. The estimation unit 113 uses the extracted feature quantities to estimate the type of the object and the figure type and figure parameters corresponding to the object.

[0016] According to this information processing system 100, it is possible to recognize a general shape that adequately approximates the outer edge of an object contained in an image.

[0017] 2 is a diagram showing an overview of the information processing device 102 according to embodiment 1. The information processing device 102 includes a feature extraction unit 112 and an estimation unit 113.

[0018] The feature extraction unit 112 extracts feature quantities of an object shown in an image. The estimation unit 113 uses the extracted feature quantities to estimate the type of the object and the figure type and figure parameters corresponding to the object.

[0019] According to this information processing device 102, it is possible to recognize a general shape that satisfactorily approximates the outer edge of an object contained in an image.

[0020] FIG. 3 is a diagram illustrating an overview of information processing according to the first embodiment.

[0021] The feature extraction unit 112 extracts the feature amount of the object shown in the image (step S102).

[0022] The estimation unit 113 uses the extracted feature amount to estimate the type of object, and the figure type and figure parameters corresponding to the object (step S103).

[0023] This information processing makes it possible to recognize a general shape that closely approximates the outer edge of an object contained in an image.

[0024] A detailed example of the information processing system 100 according to the first embodiment will be described below.

[0025] (Details) Generally, when recognizing the shape of an object in an image, recognizing the precise shape of the object often places a heavy processing load on the system and can be difficult. In addition, it is sometimes preferable to recognize the general shape of the object in order to grasp its shape.

[0026] However, Patent Document 1 does not disclose a technique for recognizing the general shape of such an object.

[0027] In response to this, Patent Document 2 discloses that a rectangular detection window is expanded vertically and horizontally at a predetermined ratio to define the object range, as described above. Patent Document 2 also discloses that the reason for expanding the detection window to a predetermined range rather than defining the object range as is is that the area of ​​the human body may easily extend beyond the object detection window depending on the posture of the limbs of the human body.

[0028] However, as exemplified in Patent Document 2, the shape of an object can vary depending on the state, even for the same object, as the human body area can extend beyond the rectangular detection window depending on the posture. Therefore, for example, depending on the posture of the human body, even if the aspect ratio of the rectangular detection window is widened, the human body area may extend significantly beyond the detection window, or the area other than the human body area may become large within the detection window.

[0029] As described above, with the techniques described in Patent Documents 1 and 2, it is difficult to recognize a general shape that adequately approximates the outer edge of an object included in an image.

[0030] In view of these circumstances, one example of the purpose of this disclosure is to provide an information processing system, an information processing device, an information processing method, a recording medium, etc. that solve the problem of recognizing a rough shape that closely approximates the outer edge of an object contained in an image.

[0031] (Configuration Example of Information Processing System 100 According to Embodiment 1) Fig. 4 is a diagram showing a configuration example of the information processing system 100 according to Embodiment 1. The information processing system 100 is a system for recognizing the outline of an object included in an image to be processed (hereinafter also referred to as "processing target image").

[0032] The "object" is, for example, an object that appears in an image. The object includes, for example, at least one of a person and an object. Note that the object may also be, for example, a predetermined part of an object, such as an iris or a face.

[0033] The information processing system 100 includes an image storage device 101 and an information processing device 102 .

[0034] The image storage device 101 and the information processing device 102 are connected to each other via a network N that may be wired, wireless, or a combination of these, and transmit and receive information to and from each other via the network N.

[0035] (Example of functional configuration of image storage device 101 according to embodiment 1) The image storage device 101 is a device for storing an image to be processed. The image to be processed is, for example, an image generated by capturing an image using an imaging device such as a camera. The image storage device 101 may store the image to be processed in advance.

[0036] The information processing system 100 may include an image capturing device connected to the network N instead of the image storage device 101 or in addition to the image storage device 101 .

[0037] (Example of functional configuration of information processing device 102 according to embodiment 1) The information processing device 102 is a device that performs information processing to recognize the general shape of an object included in an image to be processed. Functionally, the information processing device 102 includes an image acquisition unit 111, a feature extraction unit 112, an estimation unit 113, and an output unit 114.

[0038] The image acquisition unit 111 acquires the processing target image. The image acquisition unit 111 according to this embodiment acquires the processing target image from the image storage device 101.

[0039] The feature extraction unit 112 extracts, from the processing target image, the feature amount of each of one or more objects that appear in the processing target image.

[0040] The estimation unit 113 uses the features extracted by the feature extraction unit 112 for each of one or more objects to estimate the type of object and the figure type and figure parameters corresponding to the object for each of one or more objects.

[0041] The "type of object" may be determined in advance, and in this embodiment, it is the type of object. Examples of the type of object in this embodiment include a person, an umbrella, a bag, a dog, a car, a bicycle, etc.

[0042] The figure type and figure parameters corresponding to an object are information for specifying a shape according to the outline of the object. The "outline of the object" is the general shape indicated by the outer edge of the object.

[0043] "Graphic type" is a predetermined type of graphic.

[0044] Examples of the shape type according to this embodiment include polygons (for example, triangles, rectangles, pentagons, etc.), circles, ellipses, curves, closed curves, and straight lines.

[0045] The triangle may be, in more detail, an equilateral triangle, an isosceles triangle, etc. Similarly, the quadrilateral may be a square, a rectangle, a parallelogram, a trapezoid, etc.

[0046] A closed curve shape refers to a shape formed by a closed curve. A closed curve is a closed line that includes a curve at least in part, that is, a line that has a common start point and end point and includes a curve at least in part. A curve or a curve included in a closed curve is, for example, a Bezier curve or a spline curve.

[0047] The graphic type may include at least one of polygon, circle, ellipse, curve, closed curve, and straight line.

[0048] The "figure parameters" are parameters for specifying the figure indicated by the figure type corresponding to the target in the image to be processed. In other words, the figure parameters are values ​​for specifically defining the figure indicated by the figure type in the image to be processed.

[0049] Examples of the graphic parameters according to this embodiment include the size and rotation angle of the graphic indicated by the graphic type, and the position in the image. The rotation angle may be expressed as an angle of a reference direction that indicates a reference orientation that is predetermined for each graphic type.

[0050] The graphic parameters preferably include at least one of the size and rotation angle of the graphic indicated by the graphic type, and the position in the image.

[0051] The output unit 114 outputs the object type, figure type, and figure parameters estimated by the estimation unit 113 for each of one or more objects. The output method may be, for example, display or transmission to another device (not shown) such as a terminal. However, the output method is not limited to these.

[0052] Up to this point, the functional configuration example of the information processing system 100 according to the first embodiment has been mainly described. From here, a physical configuration example of the information processing system 100 according to the first embodiment will be described.

[0053] (Example of physical configuration of information processing system 100 according to embodiment 1) The information processing system 100 is physically configured from an image storage device 101 and an information processing device 102 connected via a network N. Each of the image storage device 101 and the information processing device 102 is configured, for example, from a single physically different device.

[0054] The image storage device 101 and the information processing device 102 may be physically configured as a single device, in which case the image storage device 101 and the information processing device 102 may be connected using an internal bus 1010 (described later) instead of the network N. Also, one or both of the image storage device 101 and the information processing device 102 may be physically configured as multiple devices connected via an appropriate communication line such as the network N.

[0055] The image storage device 101 according to this embodiment may be physically configured in the same manner as the information processing device 102. An example of the physical configuration of the information processing device 102 will be described with reference to the drawings.

[0056] 5 is a diagram showing an example of the physical configuration of the information processing apparatus 102 according to embodiment 1. The information processing apparatus 102 physically includes, for example, a bus 1010, a processor 1020, a memory 1030, a storage device 1040, a network interface 1050, an input interface 1060, and an output interface 1070.

[0057] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, network interface 1050, input interface 1060, and output interface 1070. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0058] The processor 1020 is implemented as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit).

[0059] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0060] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read-only memory (ROM), or the like. The storage device 1040 stores program modules for realizing the functions of the device that includes the storage device 1040. The processor 1020 loads each of these program modules into the memory 1030 and executes them to realize the function corresponding to that program module.

[0061] The network interface 1050 is an interface for connecting a device having the network interface 1050 to the network N.

[0062] The input interface 1060 is an interface for a user to input information, and is configured with one or more of, for example, a touch panel, a keyboard, a mouse, and the like.

[0063] The output interface 1070 is an interface for presenting information to the user, and is configured, for example, by a liquid crystal panel, an organic EL (Electro-Luminescence) panel, or the like.

[0064] Up to now, the description has been mainly given of an example of the configuration of the information processing system 100 according to the first embodiment. From here, an example of the operation of the information processing system 100 according to the first embodiment will be described.

[0065] (Example of Operation of Information Processing System 100 According to First Embodiment) The information processing device 102 executes information processing for recognizing the outline of an object included in an image to be processed. The information processing is started, for example, in response to a user instruction.

[0066] The information processing may include a process of generating a processing target image by photographing (photographing process). In this case, the information processing may be repeatedly executed in real time.

[0067] FIG. 6 is a flowchart illustrating an example of information processing according to the first embodiment.

[0068] The image acquisition unit 111 acquires the image to be processed from the image storage device 101 via the network N (step S101).

[0069] The feature extraction unit 112 extracts, from the processing target image acquired in step S101, feature amounts of each of one or more objects appearing in the processing target image (step S102).

[0070] In step S102, the technology for extracting feature quantities of a target from the processing target image may be a general technology. For example, the feature extraction unit 112 receives the processing target image as input and extracts feature quantities of the target using an image processing model. The image processing model is a machine learning model that has been trained to extract feature quantities of a target contained in an image from the image. When training the image processing model, it is preferable to perform supervised learning to extract feature quantities of a target contained in the training image.

[0071] The estimation unit 113 uses the feature amount extracted in step S102 to estimate the type of object and the figure type and figure parameters corresponding to the object for each of one or more objects (step S103).

[0072] 7 is a diagram showing a first example of the object type, figure type, and figure parameters estimated for an object. The example shown in FIG. 7 shows a person holding an umbrella extracted from the image to be processed, and includes two objects P1 and P2. The type of object P1 is a person. The type of object P2 is an umbrella.

[0073] The figure type associated with the target P1 is a rectangle F1. The figure parameters associated with the target P1 include parameters (values) for identifying the rectangle F1 associated with the target P1 in the processing target image. These parameters include, for example, the position of the center of gravity G1 of the rectangle F1, the horizontal and vertical lengths of the rectangle F1, and the rotation angle. In the example shown in FIG. 7 , the reference direction V1 for the rectangle F1 is defined parallel to the base, and the rotation angle is 0 degrees (for example, the same direction as the X-axis of the coordinate system defined for the processing target image).

[0074] The figure type associated with the target P2 is an isosceles triangle F2. The figure parameters associated with the target P2 include parameters (values) for identifying the isosceles triangle F2 associated with the target P2 in the image to be processed. These parameters include, for example, the position of the center of gravity G2 of the isosceles triangle F2, the lengths of the base and height of the isosceles triangle F2, and the rotation angle. In the example shown in FIG. 7 , the reference direction V2 for the isosceles triangle F2 is defined parallel to the base, and the rotation angle is 0 degrees (for example, the same direction as the X-axis of the coordinate system defined for the image to be processed).

[0075] 8 is a diagram showing a second example of the object type, figure type, and figure parameters estimated for an object. In the example shown in FIG. 8, a seated person is extracted and included as an object P3.

[0076] The figure type associated with the target P3 is an isosceles triangle F2. The figure parameters associated with the target P3 include parameters (values) for identifying the isosceles triangle F2 associated with the target P3 in the image to be processed. These parameters include, for example, the position of the center of gravity G3 of the isosceles triangle F2, the lengths of the base and height of the isosceles triangle F2, and the rotation angle. In the example shown in Figure 8, the reference direction V2 for the isosceles triangle F2 is defined parallel to the base, and the rotation angle is 0 degrees (for example, the same direction as the X-axis of the coordinate system defined for the image to be processed).

[0077] 9 is a diagram showing a third example of the object type, figure type, and figure parameters estimated for an object. In the example shown in FIG. 9, an umbrella in an umbrella stand is extracted and included as an object P4.

[0078] The figure type associated with the target P4 is a rectangle F1. The figure parameters associated with the target P4 include parameters (values) for identifying the rectangle F1 associated with the target P4 in the image to be processed. These parameters include, for example, the position of the center of gravity G4 of the rectangle F1, the horizontal and vertical lengths of the rectangle F1, and the rotation angle. In the example shown in FIG. 9 , the reference direction V1 for the rectangle F1 is defined parallel to the base, and the rotation angle is θ degrees. θ is, for example, the angle around a predetermined direction relative to the positive direction of the X-axis of a coordinate system defined for the image to be processed.

[0079] The centers of gravity G1 to G4 are examples of representative positions that are predetermined for each graphic type. Note that the representative positions are not limited to the centers of gravity, and may be determined for any graphic indicated by the graphic type.

[0080] In this way, for each of one or more objects, the estimation unit 113 identifies a figure type corresponding to the object from among predetermined figure types for identifying the general shape of the object in addition to the type of the object. Furthermore, the estimation unit 113 estimates parameters for identifying the figure indicated by the identified figure type in the processing target image. This makes it possible to estimate the general shape of the object in the processing target image using various shapes included in the figure type.

[0081] As can be seen from these examples, the number of graphic parameters may vary depending on the graphic type. When the number of graphic parameters varies depending on the graphic type, the graphic parameters may be represented by a vector whose length corresponds to the number of graphic parameters, or by a fixed-length vector that is common to all graphic types. When a fixed-length vector is used, the size of the vector should be the same as the maximum number of graphic parameters among all predetermined graphic types. Unused elements of the fixed-length vector should be filled with predetermined values ​​such as blanks or null values.

[0082] Referring again to Fig. 6, the output unit 114 outputs the object type, figure type, and figure parameters estimated for each of the one or more objects in step S103 (step S104).

[0083] For example, when the output is a display, the output unit 114 displays a figure specified by the figure type and figure parameters estimated for each of one or more objects superimposed on the image to be processed, thereby enabling a figure indicating the outline of each object appearing in the image to be processed to be superimposed on the image to be processed.

[0084] (Actions and Effects) As described above, according to this embodiment, the information processing system 100 includes the feature extraction unit 112 and the estimation unit 113. The feature extraction unit 112 extracts feature amounts of an object shown in an image. The estimation unit 113 uses the extracted feature amounts to estimate the type of the object and the figure type and figure parameters corresponding to the object.

[0085] This allows the general shape of an object in the image to be processed to be estimated using one of the various shapes included in the figure type, thereby making it possible to recognize a general shape that closely approximates the outer edge of the object in the image.

[0086] According to this embodiment, the shape type includes at least one of a polygon, a circle, an ellipse, a curve, a closed curve, and a line. The shape parameters include at least one of a size and a rotation angle of the shape indicated by the shape type, and a position in the image.

[0087] This allows the general shape of an object in the image to be processed to be estimated using one of the various shapes included in the figure type, thereby making it possible to recognize a general shape that closely approximates the outer edge of the object in the image.

[0088] According to this embodiment, shape parameters are represented as fixed length vectors that are common to the shape type.

[0089] This allows the shape parameters to be treated as variables of the same size regardless of the shape type, facilitating the process of recognizing an outline that adequately approximates the outer edge of an object contained in an image.

[0090] Second Embodiment A first detailed example of a method in which the estimation unit estimates the type of an object and the figure type and figure parameters corresponding to the object will be described.

[0091] In this embodiment, for the sake of simplicity, differences from the first embodiment will be mainly described, and descriptions that overlap with the first embodiment will be omitted as appropriate.

[0092] 10 is a diagram showing an example of the configuration of an information processing system 200 according to the second embodiment. The information processing system 200 includes an information processing device 202 that replaces the information processing device 102 according to the first embodiment. The information processing device 202 functionally includes an estimation unit 213 that replaces the estimation unit 113 according to the first embodiment. Except for these, the information processing system 200 according to this embodiment may be configured similarly to the information processing device 102 according to the first embodiment.

[0093] The estimation unit 213 has the same functions as the estimation unit 113 according to the first embodiment. The estimation unit 213 functionally includes an object estimation unit 221 and a figure estimation unit 222.

[0094] The object estimation unit 221 estimates the type of object using the feature amount extracted by the feature extraction unit 112 for each of one or more objects.

[0095] The figure estimation unit 222 estimates a figure type and figure parameters corresponding to the object based on the object type estimated by the object estimation unit 221 for each of one or more objects.

[0096] 11 is a diagram illustrating an example of the functional configuration of the figure estimation unit 222 according to the second embodiment. The figure estimation unit 222 functionally includes an association information storage unit 231, a figure type estimation unit 232, a parameter estimation unit 233, and a determination unit 234.

[0097] The association information storage unit 231 is a storage unit for storing association information 231a.

[0098] 12 is a diagram showing an example of the association information 231a. The association information 231a is information that associates object types with one or more graphic types. As a result, the association information 231a is information that specifies, for each object type, a graphic type used to recognize the outline of the object.

[0099] 12 includes object types "person" and "umbrella." The object type "person" is associated with the graphic types "rectangle," "isosceles triangle," and "oval." The object type "person" is associated with the graphic types "rectangle" and "isosceles triangle."

[0100] Referring again to Fig. 11, the figure type estimation unit 232 estimates one or more figure types corresponding to the object based on the object type estimated by the object estimation unit 221 for each of the one or more objects.

[0101] The graphic type estimation unit 232 in this embodiment estimates one or more graphic types corresponding to each of one or more objects based on the type of object estimated by the object estimation unit 221 and the correspondence information 231a.

[0102] The parameter estimation unit 233 estimates, for each of one or more objects, graphic parameters corresponding to the object for one or more graphic types estimated by the graphic type estimation unit 232. For example, the parameter estimation unit 233 may use a compatibility score, which will be described later, to estimate the graphic parameters.

[0103] When there are multiple estimated graphic types, the determination unit 234 determines the graphic type and graphic parameters corresponding to the target from among the combinations of the multiple estimated graphic types and the graphic parameters estimated for each of them.

[0104] The determination unit 234 uses the compatibility score to determine a combination of a figure type and figure parameters corresponding to the target. That is, the determination unit 234 determines a figure type and figure parameters corresponding to the target based on the figure parameters estimated by the parameter estimation unit 233 for each of the estimated figure types and the compatibility score.

[0105] The fit score is a value that indicates the degree to which the shape fits the outer edge of the object. For example, the greater the fit of the shape to the outer edge of the object, the higher the fit score, and the less the fit of the shape to the outer edge of the object, the lower the fit score.

[0106] The matching score is calculated according to a general shape estimation criterion defined using the areas of at least two of the first to third regions.

[0107] Fig. 13 is a diagram showing the first to third regions in the second example shown in Fig. 8. The first region is the region where the figure and the object overlap. The second region is the region within the object that extends beyond the figure. The third region is the region within the figure that extends beyond the object.

[0108] For example, the general shape estimation criteria may be defined such that the area of ​​the first region or the ratio of the first region to the area within the figure is the largest, and the fit score of the figure that most encompasses the object is the largest.

[0109] For example, the general shape estimation criterion may be a criterion such that the larger the area of ​​the first region, the larger the matching score, and the smaller the area of ​​the second region (or the third region), the larger the matching score. For example, the general shape estimation criterion may be a criterion such that the smaller the difference in area between the second region and the third region, the larger the matching score.

[0110] In this embodiment, the determination unit 234 calculates a compatibility score for each of a plurality of combinations of the estimated figure type and figure parameters, and then determines the combination of the figure type and figure parameters that has the highest compatibility score as the figure type and figure parameters corresponding to the target.

[0111] So far, the functional configuration example of the information processing system 200 according to the second embodiment has been mainly described. The information processing system 200 may be physically configured similarly to the information processing system 100 according to the first embodiment. From here, an operation example of the information processing system 200 according to the second embodiment will be described.

[0112] (Example of Operation of Information Processing System 200 According to Second Embodiment) Similar to the information processing device 102 according to the first embodiment, the information processing device 202 executes information processing for recognizing the outline of an object included in an image to be processed.

[0113] Fig. 14 is a flowchart showing an example of information processing according to the second embodiment. The information processing according to the second embodiment includes an estimation process (step S203) instead of the estimation process (step S103) according to the first embodiment. Except for this, the information processing according to the second embodiment may be similar to the information processing according to the first embodiment. Fig. 14 shows details of the estimation process (step S203).

[0114] The estimation unit 213 estimates the type of object and the figure type and figure parameters corresponding to the object using the feature amount extracted in step S102, similar to step S103 according to the first embodiment (step S203).

[0115] As shown in the figure, the object estimation unit 221 estimates the type of object using the feature amount extracted for each of one or more objects in step S102 (step S211).

[0116] In step S211, the technology for estimating the type of an object using the feature quantities of the object may be a common technology. For example, the object estimation unit 221 receives the feature quantities of the object as input and extracts the type of the object using a type estimation model. The type estimation model is a machine learning model that has been trained to estimate the type of the object based on the feature quantities of the object. When training the type estimation model, it is preferable to perform supervised learning to estimate the type of the object using the feature quantities of the object extracted from the training image.

[0117] The figure estimation unit 222 estimates a figure type and figure parameters corresponding to the target based on the object type estimated for each of one or more targets in step S211 (step S212; figure estimation process).

[0118] FIG. 15 is a flowchart showing an example of the figure estimation process (step S212) according to the second embodiment.

[0119] The graphic type estimation unit 232 estimates one or more graphic types corresponding to the object based on the object type estimated for each of the one or more objects in step S211 (step S221).

[0120] In more detail, for example, the graphic type estimation unit 232 refers to the association information 231a to identify one or more graphic types associated with each object type estimated in step S211. In this way, the graphic type estimation unit 232 estimates one or more graphic types corresponding to the object.

[0121] For example, when the estimated object type is "person," by referring to the association information 231a illustrated in FIG. 12, the graphic type estimation unit 232 estimates a rectangle, an isosceles triangle, and an ellipse as graphic types.

[0122] Referring again to Fig. 15, the parameter estimation unit 233 estimates graphic parameters corresponding to the one or more objects for the one or more graphic types estimated for each of the one or more objects in step S211 (step S222).

[0123] The method for estimating the graphic parameters in step S222 may be various, such as a method of changing the graphic parameters or a method of using a machine learning model.

[0124] For example, when using the method of changing graphic parameters, the parameter estimation unit 233 estimates graphic parameters for each of the estimated one or more graphic types so as to maximize the compatibility score. Specifically, for example, the parameter estimation unit 233 refers to the image to be processed, changes each graphic parameter for each graphic type, for example, in predetermined increments, and obtains the compatibility score for each graphic parameter. Then, the parameter estimation unit 233 estimates graphic parameters that maximize the compatibility score for each graphic type.

[0125] For example, when a machine learning model is used, the parameter estimation unit 233 receives the estimated one or more graphic types and the processing target image as input, and estimates graphic parameters corresponding to the target for each of the estimated one or more graphic types using the parameter estimation model. The parameter estimation model is a machine learning model that has been trained to estimate graphic parameters corresponding to the target based on the graphic type.

[0126] When training the parameter estimation model, it is preferable to perform supervised learning using training images and one or more figure types estimated for the objects included in the training images to estimate figure parameters corresponding to the objects for each of the one or more figure types. In this learning, it is preferable to perform learning to estimate figure parameters that maximize the matching score, for example.

[0127] The determination unit 234 determines whether or not a plurality of figure types have been estimated in step S221 (step S223). If a plurality of figure types have not been estimated (step S223; No), the determination unit 234 ends the figure estimation process (step S212) and returns to information processing. As shown in Fig. 14, step S104, which is the same as in the first embodiment, is then executed.

[0128] Referring again to Fig. 15, if a plurality of figure types have been estimated (step S223; Yes), the determination unit 234 determines the figure type and figure parameters corresponding to the target (step S224), and ends the figure estimation process (step S212). The determination unit 234 returns to the information processing, and step S104, similar to that of the first embodiment, is subsequently executed, as shown in Fig. 14.

[0129] In more detail, for example, the determination unit 234 refers to the image to be processed and calculates a compatibility score for each of the combinations of the plurality of figure types and figure parameters estimated in steps S211 to S212. The determination unit 234 determines the combination of the figure type and figure parameters that results in the highest compatibility score as the figure type and figure parameters corresponding to the target. In this way, the determination unit 234 determines the figure type and figure parameters corresponding to the target from the combinations of the plurality of figure types and figure parameters estimated in steps S211 to S212.

[0130] The parameter estimation unit 233 may be configured to output the combination of the figure type and the figure parameters that maximizes the compatibility score as the estimation result. In this case, the information processing device 202 may not include the determination unit 234. Furthermore, the information processing may not include steps S223 to S224.

[0131] (Actions and Effects) As described above, according to this embodiment, estimation unit 213 includes object estimation unit 221 and figure estimation unit 222. Object estimation unit 221 estimates the type of object using the extracted feature amount. Figure estimation unit 222 estimates a figure type and figure parameters corresponding to the object based on the estimated object type.

[0132] This allows the general shape of an object in the image to be processed to be estimated using one of the various shapes included in the figure type, thereby making it possible to recognize a general shape that closely approximates the outer edge of the object in the image.

[0133] According to this embodiment, the figure estimation unit 222 includes a figure type estimation unit 232 and a parameter estimation unit 233. The figure type estimation unit 232 estimates one or more figure types corresponding to the object based on the estimated type of the object. The parameter estimation unit 233 estimates figure parameters corresponding to the object for the estimated one or more figure types.

[0134] This allows the general shape of an object in the image to be processed to be estimated using one of the various shapes included in the figure type, thereby making it possible to recognize a general shape that closely approximates the outer edge of the object in the image.

[0135] According to this embodiment, the graphic type estimation unit 232 estimates one or more graphic types corresponding to the object based on the estimated type of object and association information 231a that associates the type of object with one or more graphic types.

[0136] This allows the general shape of an object in the image to be processed to be estimated using one of the various shapes included in the figure type, thereby making it possible to recognize a general shape that closely approximates the outer edge of the object in the image.

[0137] According to this embodiment, the figure estimation unit 222 further includes a determination unit 234. When a plurality of figure types are estimated, the determination unit 234 determines the figure type and figure parameters corresponding to the target based on the figure parameters estimated for each of the plurality of figure types and the compatibility score indicating the degree to which the figure matches the outer edge of the target.

[0138] This allows the general shape of the object in the image to be processed to be estimated using the shape that best matches the outer edge of the object from among the various shapes included in the shape type, making it possible to recognize a general shape that closely approximates the outer edge of the object in the image.

[0139] According to this embodiment, the matching score is a value calculated according to an outline estimation standard defined using the areas of at least two regions among (A) the first region, (B) the second region, and (C) the third region. (A) The first region is a region where the figure and the object overlap. (B) The second region is a region within the object that extends beyond the figure. (C) The third region is a region within the figure that extends beyond the object.

[0140] This allows the matching score to be used to estimate a general shape that closely approximates the outer edge of an object included in the image to be processed, thereby making it possible to recognize a general shape that closely approximates the outer edge of an object included in the image.

[0141] Third Embodiment A second detailed example of a method in which the estimation unit estimates the type of an object and the figure type and figure parameters corresponding to the object will be described.

[0142] In this embodiment, for the sake of simplicity, differences from other embodiments will be mainly described, and descriptions that overlap with other embodiments will be omitted as appropriate.

[0143] 16 is a diagram showing an example of the configuration of an information processing system 300 according to the third embodiment. The information processing system 300 includes an information processing device 302 that replaces the information processing device 102 according to the first embodiment. The information processing device 302 functionally includes an estimation unit 313 that replaces the estimation unit 113 according to the first embodiment. Except for these, the information processing system 300 according to this embodiment may be configured similarly to the information processing device 102 according to the first embodiment.

[0144] The estimation unit 313 has the same functions as the estimation unit 113 according to the first embodiment. The estimation unit 313 functionally includes an object estimation unit 221 and a figure estimation unit 322 similar to those in the second embodiment.

[0145] The figure estimation unit 322 estimates a figure type and figure parameters corresponding to one or more objects based on the feature amount extracted by the feature extraction unit 112 for each of the objects.

[0146] 17 is a diagram showing an example of the functional configuration of the figure estimation unit 322 according to the third embodiment. The figure estimation unit 322 functionally includes a figure type estimation unit 332, and a parameter estimation unit 233 and a determination unit 234 similar to those in the second embodiment.

[0147] The figure type estimation unit 332 receives the feature quantities extracted by the feature extraction unit 112 for each of one or more objects and estimates one or more figure types corresponding to the objects using a figure type estimation model. The figure type estimation model is a machine learning model that has been trained to estimate at least one figure type corresponding to an object shown in an image. It is preferable that multiple types of figures are predetermined as the figure types estimated by the figure type estimation model.

[0148] When training the figure type estimation model, it is preferable to perform supervised learning to estimate one or more figure types corresponding to the object using the feature amounts of the object extracted from the training image.

[0149] So far, the functional configuration example of the information processing system 300 according to the third embodiment has been mainly described. The information processing system 300 may be physically configured similarly to the information processing system 100 according to the first embodiment. From here, an operation example of the information processing system 300 according to the second embodiment will be described.

[0150] (Example of Operation of Information Processing System 300 According to Third Embodiment) Similar to the information processing device 102 according to the first embodiment, the information processing device 302 executes information processing for recognizing the outline of an object included in an image to be processed.

[0151] Fig. 18 is a flowchart showing an example of information processing according to embodiment 3. The information processing according to embodiment 3 includes an estimation process (step S303) instead of the estimation process (step S103) according to embodiment 1. Except for this, the information processing according to embodiment 3 may be similar to the information processing according to embodiment 1. Fig. 18 shows details of the estimation process (step S303).

[0152] The target estimation unit 221 executes step S211 similar to that in the second embodiment.

[0153] The figure estimation unit 322 estimates a figure type and figure parameters corresponding to the target based on the feature amount extracted for each of one or more targets in step S102 (step S312; figure estimation process).

[0154] FIG. 19 is a flowchart showing an example of the figure estimation process (step S312) according to the third embodiment.

[0155] The graphic type estimation unit 332 receives the feature values ​​extracted for each of one or more objects in step S102 as input, and estimates one or more graphic types corresponding to the objects using a graphic type estimation model (step S321).

[0156] The parameter estimation unit 233 and the determination unit 234 return to the information processing by executing steps S222 to S224 similar to those in the second embodiment. Subsequently, step S104 similar to that in the first embodiment is executed as shown in FIG.

[0157] (Actions and Effects) As described above, according to this embodiment, estimation unit 313 includes object estimation unit 221 and figure estimation unit 322. Object estimation unit 221 estimates the type of object using the extracted feature amount. Figure estimation unit 322 estimates the figure type and figure parameters corresponding to the object based on the extracted feature amount.

[0158] This allows the general shape of an object in the image to be processed to be estimated using one of the various shapes included in the figure type, thereby making it possible to recognize a general shape that closely approximates the outer edge of the object in the image.

[0159] According to this embodiment, the figure estimation unit 322 includes a figure type estimation unit 332 and a parameter estimation unit 233. The figure type estimation unit 332 estimates one or more figure types corresponding to the object based on the extracted feature quantities. The parameter estimation unit 233 estimates figure parameters corresponding to the object for the estimated one or more figure types.

[0160] This allows the general shape of an object in the image to be processed to be estimated using one of the various shapes included in the figure type, thereby making it possible to recognize a general shape that closely approximates the outer edge of the object in the image.

[0161] According to this embodiment, the figure type estimation unit 332 uses the extracted features as input and estimates one or more figure types corresponding to the object using a figure type estimation model that has been trained to estimate at least one figure type corresponding to the object shown in the image.

[0162] This allows the general shape of an object in the image to be processed to be estimated using one of the various shapes included in the figure type, thereby making it possible to recognize a general shape that closely approximates the outer edge of the object in the image.

[0163] Fourth Embodiment There are various details regarding the configuration of the parameter estimation unit 233 and the graphic parameter estimation process (step S222) executed by the parameter estimation unit 233 described in the second and third embodiments. In the fourth embodiment, a detailed example of the parameter estimation unit 233 and the graphic parameter estimation process (step S222) will be described.

[0164] 20 shows an example of the functional configuration of the parameter estimation unit 233 according to the second and third embodiments. The parameter estimation unit 233 includes first to Nth figure type-specific estimation units 233A_1 to 233A_N (N is an integer equal to or greater than 1) and a selection unit 233B.

[0165] When the "first to Nth graphic type-specific estimating units 233A_1 to 233A_N" are not particularly distinguished from one another, they are also referred to as "graphic type-specific estimating units 233A."

[0166] The first to N-th graphic type-specific estimation units 233A_1 to 233A_N are respectively associated with predetermined first to N-th graphic types. The predetermined first to N-th graphic types correspond to, for example, all of the graphic types included in the association information 231a. The first to N-th graphic type-specific estimation units 233A_1 to 233A_N estimate graphic parameters corresponding to the target for the graphic types associated with them, respectively.

[0167] For example, suppose that the association information 231a includes three types of figure types: a rectangle, an isosceles triangle, and an ellipse. In this case, the parameter estimation unit 233 includes first to third figure-type estimation units 233A_1 to 233A_3 associated with the rectangle, the isosceles triangle, and the ellipse, respectively.

[0168] That is, in this case, the first geometrical type-specific estimation unit 233A_1 is a geometrical type-specific estimation unit 233A associated with a rectangle and estimates geometrical parameters for the rectangle. The second geometrical type-specific estimation unit 233A_2 is a geometrical type-specific estimation unit 233A associated with an isosceles triangle and estimates geometrical parameters for the isosceles triangle. The third geometrical type-specific estimation unit 233A_3 is a geometrical type-specific estimation unit 233A associated with an ellipse and estimates geometrical parameters for the ellipse.

[0169] The method by which each of the graphic type estimation units 233A estimates graphic parameters may be any of various methods, such as the method of changing graphic parameters as described above or the method of using a machine learning model.

[0170] When a machine learning model is used, each of the figure-type-specific estimation units 233A, for example, receives an image to be processed or an image of a target included in the image to be processed as input, and estimates figure parameters corresponding to the target for the associated figure type using a figure-type-specific parameter estimation model. The figure-type-specific parameter estimation model is a machine learning model that has been trained to estimate figure parameters corresponding to the target for the associated figure type.

[0171] When training the parameter estimation model for each figure type, it is preferable to perform supervised learning using training images or target images included in the training images to estimate figure parameters corresponding to the target for the associated figure type. For example, this learning may be performed to estimate the figure parameters so as to maximize the matching score.

[0172] The selection unit 233B selects the graphic type-specific estimation unit 233A to be used to estimate graphic parameters corresponding to the target, based on one or more graphic types estimated by the graphic type estimation unit 232.

[0173] For example, the selection unit 233B selects a graphic type estimation unit 233A corresponding to each of one or more graphic types estimated by the graphic type estimation unit 232 from the first to Nth graphic type estimation units 233A_1 to 233A_N.

[0174] FIG. 21 is a flowchart showing an example of the parameter estimation process (step S222) according to the second and third embodiments.

[0175] The selection unit 233B selects the graphic type-specific estimation unit 233A to be used to estimate graphic parameters corresponding to the target, based on one or more graphic types estimated in step S221 or S321 (step S222a).

[0176] In detail, for example, the selection unit 233B selects the graphic type-specific estimation unit 233A to which one or more estimated graphic types are associated.

[0177] The graphic type-specific estimation unit 233A selected in step S222a estimates graphic parameters corresponding to the target for the associated graphic type (step S222b), and returns to the graphic estimation process (step S212 or S312).

[0178] (Operations and Effects) As described above, according to this embodiment, the parameter estimation unit 233 includes first to Nth graphic type-specific estimation units 233A_1 to 233A_N and a selection unit 233B. The first to Nth graphic type-specific estimation units 233A_1 to 233A_N estimate graphic parameters corresponding to the target for graphic types associated with the predetermined first to Nth graphic types, respectively. The selection unit 233B selects one of the graphic type-specific estimation units 233A_1 to 233A_N to be used to estimate graphic parameters corresponding to the target, based on the estimated one or more graphic types.

[0179] This allows the estimation of graphic parameters using the graphic type-specific estimation unit 233A specialized for each graphic type. Therefore, it is possible to estimate graphic parameters that better fit the outline of the object in the image to be processed. It becomes possible to recognize an outline that better approximates the outer edge of the object included in the image.

[0180] Fifth Embodiment A third detailed example of a method in which the estimation unit estimates the type of an object and the figure type and figure parameters corresponding to the object will be described.

[0181] In this embodiment, for the sake of simplicity, differences from other embodiments will be mainly described, and descriptions that overlap with other embodiments will be omitted as appropriate.

[0182] 22 is a diagram showing an example of the configuration of an information processing system 500 according to the fifth embodiment. The information processing system 500 includes an information processing device 502 that replaces the information processing device 102 according to the first embodiment. The information processing device 502 functionally includes an estimation unit 513 that replaces the estimation unit 113 according to the first embodiment. Except for these, the information processing system 500 according to this embodiment may be configured similarly to the information processing device 102 according to the first embodiment.

[0183] The estimation unit 513 has the same functions as the estimation unit 113 according to the first embodiment. The estimation unit 513 functionally includes an object estimation unit 221 and a figure estimation unit 522 similar to those in the second embodiment.

[0184] Figure estimation unit 522 receives the feature values ​​extracted by feature extraction unit 112 as input and estimates a figure type and figure parameters corresponding to the target using a figure estimation model. The figure estimation model is a machine learning model that has been trained to estimate the general shape of an target shown in an image. The figure estimation model may estimate a plurality of figure types in advance. The figure estimation model may also receive the target image as input.

[0185] When training the figure estimation model, it is preferable to perform supervised learning to estimate figure types and figure parameters corresponding to objects included in the training images using features extracted from the training images. When training the figure estimation model, training images may also be input.

[0186] In this learning, for example, learning may be performed to estimate the figure type and figure parameters so as to maximize the above-mentioned matching score. That is, the figure estimation model may be a machine learning model that has been trained to use the feature quantities of an object included in the learning image as input and to output the figure type and figure parameters that indicate the outline of the object so as to increase the matching score, which indicates the degree to which the figure matches the outer edge of the object.

[0187] So far, the functional configuration example of the information processing system 500 according to the fifth embodiment has been mainly described. The information processing system 500 may be physically configured similarly to the information processing system 100 according to the first embodiment. From here, an operation example of the information processing system 500 according to the fifth embodiment will be described.

[0188] (Example of Operation of Information Processing System 500 According to Fifth Embodiment) Similar to the information processing device 102 according to the first embodiment, the information processing device 502 executes information processing for recognizing the outline of an object included in an image to be processed.

[0189] Fig. 23 is a flowchart showing an example of information processing according to embodiment 5. The information processing according to embodiment 5 includes an estimation process (step S503) instead of the estimation process (step S103) according to embodiment 1. Except for this, the information processing according to embodiment 2 may be similar to the information processing according to embodiment 1. Fig. 23 shows details of the estimation process (step S503).

[0190] The estimation unit 513 estimates the type of object and the figure type and figure parameters corresponding to the object using the feature amount extracted in step S102, similar to step S103 according to the first embodiment (step S503).

[0191] As shown in the figure, the target estimation unit 221 executes step S211 similar to that of the second embodiment.

[0192] The figure estimation unit 522 uses the feature extracted in step S102 and the figure estimation model to estimate the figure type and figure parameters corresponding to the target (step S512). Subsequently, step S104 is executed as in the first embodiment.

[0193] (Actions and Effects) As described above, according to this embodiment, the figure estimation unit 522 uses the extracted features as input and estimates the figure type and figure parameters corresponding to the object using a figure estimation model that has been trained to estimate the general shape of the object shown in the image.

[0194] This allows the general shape of an object in the image to be processed to be estimated using one of the various shapes included in the figure type, thereby making it possible to recognize a general shape that closely approximates the outer edge of the object in the image.

[0195] According to this embodiment, the figure estimation model is a machine learning model that is trained to use the feature quantities of an object contained in a training image as input and to output a figure type and figure parameters that indicate the outline of the object so as to obtain a high matching score. The matching score indicates the degree to which the figure matches the outer edge of the object.

[0196] This allows the outline of the object in the image to be processed to be estimated using a graphic that closely matches the outer edge of the object, thereby making it possible to recognize an outline that closely approximates the outer edge of the object contained in the image.

[0197] Sixth Embodiment The estimation results regarding the outline of the object described in the other embodiments may be used for authentication. For example, the estimation results may be used to extract a predetermined object area, such as a human face or iris, included in the image to be processed, and perform biometric authentication such as face authentication or iris authentication. In the sixth embodiment, a case where the object is a face (head) or iris will be described as an example. Such an object is typically a part of a human, but may also be a part of an animal, such as a dog or a snake.

[0198] In this embodiment, for the sake of simplicity, differences from the first embodiment will be mainly described, and descriptions that overlap with the first embodiment will be omitted as appropriate.

[0199] 24 is a diagram showing an example of the configuration of an information processing system 600 according to the sixth embodiment. The information processing system 600 includes an information processing device 602 that replaces the information processing device 102 according to the first embodiment. Functionally, the information processing device 602 includes a region extraction unit 631 and a matching unit 632 in addition to the components included in the information processing device 102 according to the first embodiment. Except for these components, the information processing system 200 according to this embodiment may be configured similarly to the information processing device 102 according to the first embodiment.

[0200] The region extraction unit 631 extracts an object region, which is an image region in which an object appears, from the processing target image based on the figure type and figure parameters estimated by the estimation unit 113 .

[0201] The object according to this embodiment may be, for example, a face (head) or an iris. For example, if the object is an iris, the graphic type may include a double circle, a shape in which the top and bottom of the double circle are partially cut out, and the like.

[0202] The matching unit 632 uses the extracted target region to perform a matching process for authentication. The authentication may be, for example, biometric authentication. In more detail, for example, the authentication may be face authentication if the target is a face, or iris authentication if the target is an iris. A general technique may be used for the matching process for such authentication.

[0203] So far, the functional configuration example of the information processing system 600 according to the sixth embodiment has been mainly described. The information processing system 600 may be physically configured similarly to the information processing system 100 according to the first embodiment. From here, an operation example of the information processing system 600 according to the sixth embodiment will be described.

[0204] (Example of operation of information processing system 600 according to embodiment 6) An information processing device 602 executes information processing. The information processing according to this embodiment includes authentication processing in addition to the information processing executed by the information processing device 102 according to embodiment 1. The authentication processing is processing for authenticating a person who has been registered in advance, based on an image to be processed.

[0205] FIG. 25 is a flowchart illustrating an example of authentication processing according to the sixth embodiment.

[0206] The region extraction unit 631 extracts a target region, which is an image region in which a target appears, from the processing target image based on the figure type and figure parameters output in step S104, for example (step S601).

[0207] The matching unit 632 performs matching processing for authentication using the target area extracted in step S601 (step S602).

[0208] In more detail, for example, the matching unit 632 holds registration information including feature amounts of people registered in advance. The matching unit 632 extracts feature amounts of the target region extracted in step S601. The matching unit 632 matches the extracted feature amounts with feature amounts included in the registration information. As a result of the matching, the matching unit 632 generates matching result information indicating, for example, whether the similarity of the matched feature amounts is equal to or greater than a predetermined threshold. The matching process described here is one example, and is not limited to this.

[0209] The collation unit 632 outputs the result of the collation performed in step S602 (step S603). This output may be displayed or transmitted to another device (not shown).

[0210] According to this authentication process, the target area can be extracted using the figure type and figure parameters estimated in step S103. Therefore, by including figures corresponding to various states of face shapes, iris shapes, etc. in the figure type, the target area can be extracted easily and with high accuracy. Therefore, accurate matching is possible.

[0211] For example, if the target is a face, various states include a state in which the target image contains multiple people and the faces of the multiple people overlap, etc. For example, if the target is an iris, various states include a blinking state, a wide-open eye state, etc.

[0212] (Actions and Effects) As described above, according to this embodiment, the information processing system 600 further includes a region extraction unit 631 and a matching unit 632. The region extraction unit 631 extracts a target region, which is an image region in which a target appears, from an image based on the estimated figure type and figure parameters. The matching unit 632 performs matching processing for authentication using the extracted target region. The target is a face or an iris.

[0213] This allows the target region to be extracted easily and accurately, as described above, and therefore enables matching with high accuracy.

[0214] According to this embodiment, the image includes multiple people with overlapping faces.

[0215] Even in such an image, as described above, the facial region, which is the target region, can be extracted with high accuracy and ease, thereby enabling accurate matching for face authentication.

[0216] 26 is a diagram showing an example of the configuration of an information processing system 600 according to Modification 1. The information processing system 600 according to Modification 1 includes an authentication device 603 in addition to the image storage device 101 and information processing device 102 similar to those of Embodiment 1. The authentication device 603 includes a region extraction unit 631 and a matching unit 632 similar to those of Embodiment 6.

[0217] That is, in this modification, an authentication device 603 separate from the information processing device 102 has a function of executing authentication processing. This also provides the same effects as in the sixth embodiment.

[0218] Although the embodiments and modifications of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various configurations other than those described above can also be adopted.

[0219] In addition, although the flowcharts used in the above description show multiple steps (processes) in a sequential order, the order of steps performed in each embodiment is not limited to the order shown. In each embodiment, the order of steps shown in the drawings can be changed as long as it does not cause any problems in terms of content. Furthermore, the above-described embodiments and variations can be combined as long as the content is not contradictory.

[0220] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0221] 1. An information processing system comprising: feature extraction means for extracting features of an object shown in an image; and estimation means for estimating the type of the object and the graphic type and graphic parameters corresponding to the object using the extracted features. 2. The information processing system described in 1., wherein the estimation means includes: object estimation means for estimating the type of the object using the extracted features; and graphic estimation means for estimating the graphic type and graphic parameters corresponding to the object based on the estimated object type or the extracted features. 3. The information processing system described in 2., wherein the graphic estimation means includes: graphic type estimation means for estimating one or more graphic types corresponding to the object based on the estimated object type or the extracted features; and parameter estimation means for estimating the graphic parameters corresponding to the object for the estimated one or more graphic types. 4. The information processing system described in 3., wherein the graphic type estimation means estimates the one or more graphic types corresponding to the object based on the estimated object type and association information associating the object type with the one or more graphic types. 5. The information processing system described in 3., wherein the graphic type estimation means estimates the one or more graphic types corresponding to the object using a graphic type estimation model that has been trained to estimate at least one graphic type corresponding to an object shown in an image using the extracted feature amount as input. 6. The information processing system described in any one of 3. to 5., wherein the parameter estimation means includes: first to Nth graphic type-specific estimation means for estimating the graphic parameters corresponding to the object for graphic types associated with predetermined first to Nth graphic types, respectively; and selection means for selecting a graphic type-specific estimation means to be used to estimate the graphic parameters corresponding to the object based on the estimated one or more graphic types.7. The information processing system described in any one of 3. to 6., wherein the graphic estimation means further includes determination means, when there are multiple estimated graphic types, for determining the graphic type and the graphic parameters corresponding to the object based on graphic parameters estimated for each of the multiple graphic types and a compatibility score indicating the degree to which the graphic matches the outer edge of the object. 8. The information processing system described in 2., wherein the graphic estimation means estimates the graphic type and the graphic parameters corresponding to the object using a graphic estimation model that has trained to estimate the general shape of an object shown in an image using the extracted feature amounts as input. 9. The information processing system described in 8., wherein the graphic estimation model is a machine learning model that has trained to use feature amounts of the object included in training images as input and to output the graphic type and the graphic parameters indicating the general shape of the object so as to increase the compatibility score indicating the degree to which the graphic matches the outer edge of the object. 10. The compatibility score is a value calculated according to an outline estimation standard defined using the areas of at least two of: (A) a first region, which is a region where the graphic and the object overlap; (B) a second region, which is a region within the object that extends beyond the graphic; and (C) a third region, which is a region within the graphic that extends beyond the object. 7. The information processing system described in or 9.. 11. The information processing system described in any one of 1. to 10., wherein the graphic type includes at least one of polygon, circle, ellipse, curve, closed curve, and straight line, and the graphic parameters include at least one of the size and rotation angle of the graphic indicated by the graphic type, and its position in the image. 12. The information processing system described in 11., wherein the graphic parameters are expressed as a fixed-length vector common to the graphic types. 13. The information processing system according to any one of 1. to 12., further comprising: an area extraction unit that extracts an object area, which is an image area in which the object appears, from the image based on the estimated figure type and figure parameters; and a matching unit that performs matching processing for authentication using the extracted object area, wherein the object is a face or an iris. 14. The information processing system according to 13., wherein the image includes multiple people whose faces overlap.15. The information processing system according to any one of 1. to 14., further comprising an image storage device for storing the image. 16. An information processing device comprising: feature extraction means for extracting features of an object shown in an image; and estimation means for estimating the type of the object, and the graphic type and graphic parameters corresponding to the object, using the extracted features. 17. An information processing method, including at least one computer extracting features of an object shown in an image, and estimating the type of the object, and the graphic type and graphic parameters corresponding to the object, using the extracted features. 18. The information processing method according to 17., wherein the estimating the type of the object, and the graphic type and graphic parameters corresponding to the object includes estimating the type of the object using the extracted features, and estimating the graphic type and the graphic parameters corresponding to the object based on the estimated object type or the extracted features. 19. The information processing method according to 18., wherein estimating the graphic type and the graphic parameters includes estimating one or more graphic types corresponding to the object based on the estimated object type or the extracted feature amount, and estimating the graphic parameters corresponding to the object for the estimated one or more graphic types. 20. The information processing method according to 19., wherein estimating the graphic type includes estimating the one or more graphic types corresponding to the object based on the estimated object type and association information associating the object type with the one or more graphic types. 21. The information processing method according to 19., wherein estimating the graphic type includes estimating the one or more graphic types corresponding to the object using a graphic type estimation model that has trained to estimate at least one graphic type corresponding to an object shown in an image using the extracted feature amount as input.22. The information processing method of any one of 19. to 21., wherein estimating the graphic parameters includes estimating the graphic parameters corresponding to the object using first to Nth graphic type-specific estimation means associated with predetermined first to Nth graphic types, respectively, and selecting a graphic type-specific estimation means to be used for estimating the graphic parameters corresponding to the object based on the one or more estimated graphic types. 23. The information processing method of any one of 19. to 22., wherein estimating the graphic type and the graphic parameters further includes, if there are multiple estimated graphic types, determining the graphic type and the graphic parameters corresponding to the object based on graphic parameters estimated for each of the multiple graphic types and a compatibility score indicating the degree to which a graphic matches the outer edge of the object. 24. The information processing method of 18., wherein estimating the graphic type and the graphic parameters includes estimating the graphic type and the graphic parameters corresponding to the object using a graphic estimation model that has trained to estimate the general shape of an object shown in an image using the extracted feature as input. 25. The information processing method described in 24., wherein the figure estimation model is a machine learning model that has been trained to use feature quantities of an object included in a training image as input and to output the figure type and the figure parameters indicating the outline of the object so as to increase the fit score, which indicates the degree to which the figure fits the outer edge of the object. 26. The fit score is a value calculated according to an outline estimation standard defined using the areas of at least two regions among (A) a first region, which is the region where the figure and the object overlap, (B) a second region, which is a region within the object that extends beyond the figure, and (C) a third region, which is a region within the figure that extends beyond the object. 27. The information processing method described in any one of 17. to 26., wherein the figure type includes at least one of a polygon, a circle, an ellipse, a curve, a closed curve, and a straight line, and the figure parameters include at least one of the size and rotation angle of the figure indicated by the figure type and its position in the image. 28. 27. The information processing method according to 27., wherein the shape parameters are represented by fixed-length vectors that are common to the shape types.29. The information processing method described in any one of 17. to 28., further comprising extracting an object region, which is an image region in which the object appears, from the image based on the estimated graphic type and graphic parameters, and performing a matching process for authentication using the extracted object region, wherein the object is a face or an iris. 30. The information processing method described in 29., wherein the image includes a plurality of people whose faces overlap. 31. A recording medium having recorded thereon a program for causing at least one computer to: extract features of an object appearing in an image, and use the extracted features to estimate the type of the object, and the graphic type and graphic parameters corresponding to the object. 32. The recording medium described in 31., wherein the estimating the type of the object, the graphic type and the graphic parameters corresponding to the object includes: estimating the type of the object using the extracted features; and estimating the graphic type and the graphic parameters corresponding to the object based on the estimated object type or the extracted features. 33. The recording medium described in 32., wherein estimating the graphic type and the graphic parameters includes estimating one or more graphic types corresponding to the object based on the estimated object type or the extracted feature amount, and estimating the graphic parameters corresponding to the object for the estimated one or more graphic types. 34. The recording medium described in 33., wherein estimating the graphic type includes estimating the one or more graphic types corresponding to the object based on the estimated object type and association information associating the object type with the one or more graphic types. 35. The recording medium described in 33., wherein estimating the graphic type includes estimating the one or more graphic types corresponding to the object using a graphic type estimation model that has trained to estimate at least one graphic type corresponding to an object shown in an image using the extracted feature amount as input.36. The recording medium described in any one of 33. to 35., wherein estimating the graphic parameters includes estimating the graphic parameters corresponding to the object using first to Nth graphic type-specific estimation means associated with predetermined first to Nth graphic types, respectively, and selecting a graphic type-specific estimation means to be used to estimate the graphic parameters corresponding to the object based on the one or more estimated graphic types. 37. The recording medium described in any one of 33. to 36., wherein estimating the graphic type and the graphic parameters further includes, if there are multiple estimated graphic types, determining the graphic type and the graphic parameters corresponding to the object based on graphic parameters estimated for each of the multiple graphic types and a compatibility score indicating the degree to which a graphic matches the outer edge of the object. 38. The recording medium described in 32., wherein estimating the graphic type and the graphic parameters includes estimating the graphic type and the graphic parameters corresponding to the object using a graphic estimation model that has trained to estimate the general shape of an object shown in an image using the extracted feature as input. 39. The recording medium described in 38., wherein the figure estimation model is a machine learning model that has been trained to input feature quantities of an object included in a training image and output the figure type and the figure parameters indicating the outline of the object so as to increase a fit score indicating the degree to which the figure fits the outer edge of the object. 40. The fit score is a value calculated according to an outline estimation standard defined using the areas of at least two of: (A) a first region that is an area where the figure and the object overlap; (B) a second region that is an area within the object that extends beyond the figure; and (C) a third region that is an area within the figure that extends beyond the object. 41. The recording medium described in any one of 31. to 40., wherein the figure type includes at least one of a polygon, a circle, an ellipse, a curve, a closed curve, and a straight line, and the figure parameters include at least one of the size and rotation angle of the figure indicated by the figure type and its position in the image. 42. The recording medium described in 41., wherein the figure parameters are expressed as a fixed-length vector common to the figure types.43. The recording medium according to any one of 31. to 42., further comprising: extracting an object region from the image, which is an image region in which the object appears, based on the estimated figure type and figure parameters; and performing a matching process for authentication using the extracted object region, wherein the object is a face or an iris. 44. The recording medium according to 43., wherein the image includes multiple people with overlapping faces.

[0222] 100, 200, 300, 500, 600 Information processing system 101 Image storage device 102, 202, 302, 502, 602 Information processing device 111 Image acquisition unit 112 Feature extraction unit 113, 213, 313, 513 Estimation unit 114 Output unit 221 Object estimation unit 222, 322, 522 Figure estimation unit 231 Correspondence information storage unit 231a Correspondence information 232, 332 Figure type estimation unit 233 Parameter estimation unit 233A Figure type-specific estimation unit 233B Selection unit 234 Decision unit 603 Authentication device 631 Area extraction unit 632 Matching unit

Claims

1. feature extraction means for extracting feature quantities of an object shown in an image; and an estimation means for estimating the type of the object and the figure type and figure parameters corresponding to the object using the extracted feature amount. Information processing system.

2. The estimation means an object estimation means for estimating the type of the object using the extracted feature amount; and a figure estimation means for estimating the figure type and the figure parameters corresponding to the object based on the estimated type of the object or the extracted feature amount. The information processing system according to claim 1 .

3. The figure estimation means a figure type estimation means for estimating one or more figure types corresponding to the object based on the estimated object type or the extracted feature amount; and parameter estimation means for estimating the graphic parameters corresponding to the object for the estimated one or more graphic types. The information processing system according to claim 2 .

4. The graphic type estimating means estimates the one or more graphic types corresponding to the object based on the estimated object type and association information that associates the object type with the one or more graphic types. The information processing system according to claim 3 .

5. The graphic type estimation means estimates the one or more graphic types corresponding to the object by using a graphic type estimation model that has been trained to estimate at least one graphic type corresponding to the object shown in the image, using the extracted feature amount as an input. The information processing system according to claim 3 .

6. The parameter estimation means first to Nth figure type estimating means for estimating the figure parameters corresponding to the target for each of first to Nth predetermined figure types; and a selection means for selecting a figure type-specific estimation means to be used for estimating the figure parameters corresponding to the object based on the estimated one or more figure types. The information processing system according to any one of claims 3 to 5.

7. The graphic estimation means further includes a determination means for determining, when a plurality of graphic types are estimated, the graphic type and the graphic parameters corresponding to the object based on the graphic parameters estimated for each of the plurality of graphic types and a compatibility score indicating the degree to which the graphic matches the outer edge of the object. The information processing system according to any one of claims 3 to 5.

8. The figure estimation means estimates the figure type and the figure parameters corresponding to the object using a figure estimation model that has been trained to estimate the general shape of the object shown in the image, with the extracted feature amount as an input. The information processing system according to any one of claims 2 to 5.

9. At least one computer Extract the features of the object in the image, and estimating the type of the object and the figure type and figure parameters corresponding to the object using the extracted feature amount. Information processing methods.

10. At least one computer Extract the features of the object in the image, A program for executing the steps of: using the extracted feature amount to estimate the type of the object, and the figure type and figure parameters corresponding to the object.

Images