Information processing systems, information processing methods, and programs

Abstract

An information processing system (100) comprises a feature extraction unit (112) and an estimation unit (113). The feature extraction unit (112) extracts features of an object shown in an image. The estimation unit (113) uses the extracted features to estimate a type of the object and a shape type and shape parameters corresponding to the object.

Description

[[Technical Field]]

[0001] The present disclosure relates to an information processing system , love an information processing method and program . [[Background Art]]

[0002] Various techniques have been proposed for recognizing objects included in an image.

[0003] For example, Patent Document 1 discloses a technique for recognizing an object that is not registered in learning data. In the technique described in Patent Document 1, a known object recognition unit recognizes a known object registered in learning data from an image. A generalized object recognition unit recognizes a generalizable generalized object by combining known objects registered in learning data.

[0004] For example, Patent Document 2 describes a technique for determining an approximate range (object range) in which a detected object exists from an image to be recognized. In the technique described in Patent Document 2, an object detection unit performs recognition processing as to whether or not a predetermined object exists in an input image. An object range determining unit determines an object range of the detected object based on a detection result of the object detection unit.

[0005] Patent Document 2 describes that the object range determining unit, for example, uses the position and size of the object output by the object detection unit to expand the detection window in which the object is detected vertically and horizontally at a predetermined ratio, and sets this as the object range. According to FIG. 7(A) of Patent Document 2, the detection window is rectangular. [[Prior Art Documents]] [[Patent Documents]]

[0006] [[Patent Document 1]] Japanese Unexamined Patent Application Publication No. 2019-220014 [[Patent Document 2]] Japanese Unexamined Patent Application Publication No. 2016-018538 [[Summary of the Invention]] [Problems that the invention aims to solve]

[0007] This disclosure aims to improve upon the technology described in the prior art documents mentioned above. [Means for solving the problem]

[0008] According to one aspect of this disclosure, A feature extraction means for extracting feature quantities from objects shown in an image, Using the extracted features, the type of object and the corresponding object Diagram Shape type and Diagram It comprises a shape parameter and an estimation means for estimating it. An information processing system is provided.

[0010] According to one aspect of this disclosure, At least one computer, Extract the feature quantities of the objects shown in the image, Using the extracted features, the type of object and the corresponding object Diagram Shape type and Diagram This includes estimating the shape parameters. Information processing methods are provided.

[0011] According to one aspect of this disclosure, On at least one computer, Extract the feature quantities of the objects shown in the image, Using the extracted features, the type of object and the corresponding object Diagram Shape type and Diagram A program to perform the estimation of shape parameters. Mu It will be provided. [Brief explanation of the drawing]

[0012] [Figure 1] This diagram shows an overview of the information processing system according to Embodiment 1. [Figure 2] FIG. 1 is a diagram illustrating an overview of an information processing apparatus according to a first embodiment. [Figure 3] FIG. 2 is a diagram illustrating an overview of information processing according to the first embodiment. [Figure 4] FIG. 3 is a diagram illustrating a configuration example of an information processing system according to the first embodiment. [Figure 5] FIG. 4 is a diagram illustrating a physical configuration example of an information processing apparatus according to the first embodiment. [Figure 6] FIG. 5 is a flowchart illustrating an example of information processing according to the first embodiment. [Figure 7] FIG. 6 is a diagram illustrating a first example of estimated object type, graphic type and graphic parameter for an object. [Figure 8] FIG. 7 is a diagram illustrating a second example of estimated object type, graphic type and graphic parameter for an object. [Figure 9] FIG. 8 is a diagram illustrating a third example of estimated object type, graphic type and graphic parameter for an object. [Figure 10] FIG. 9 is a diagram illustrating a configuration example of an information processing system according to a second embodiment. [Figure 11] FIG. 10 is a diagram illustrating a functional configuration example of a graphic estimation unit according to the second embodiment. [Figure 12] FIG. 11 is a diagram illustrating an example of association information. [Figure 13] FIG. 12 is a diagram illustrating first to third regions in the second example shown in FIG. 8. [Figure 14] FIG. 13 is a flowchart illustrating an example of information processing according to the second embodiment. [Figure 15] FIG. 14 is a flowchart illustrating an example of graphic estimation processing according to the second embodiment. [Figure 16] FIG. 15 is a diagram illustrating a configuration example of an information processing system according to a third embodiment. [Figure 17] FIG. 16 is a diagram illustrating a functional configuration example of a graphic estimation unit according to the third embodiment. [Figure 18] FIG. 17 is a flowchart illustrating an example of information processing according to the third embodiment. [Figure 19] FIG. 18 is a flowchart illustrating an example of graphic estimation processing according to the third embodiment. [Figure 20] Examples of the functional configuration of the parameter estimation unit 233 according to embodiments 2 and 3 are shown. [Figure 21] This flowchart shows examples of parameter estimation processes according to Embodiments 2 and 3. [Figure 22] This figure shows an example configuration of the information processing system according to Embodiment 5. [Figure 23] This flowchart shows an example of information processing according to Embodiment 5. [Figure 24] This figure shows an example configuration of the information processing system according to Embodiment 6. [Figure 25] This flowchart shows an example of the authentication process according to Embodiment 6. [Figure 26] This figure shows an example configuration of an information processing system related to Modification Example 1. [Modes for carrying out the invention]

[0013] The embodiments of this disclosure will be described below with reference to the drawings. In all drawings, similar components are denoted by the same reference numerals, and their descriptions are omitted where appropriate.

[0014] <Embodiment 1> (overview) Figure 1 is a diagram showing an overview of the information processing system 100 according to Embodiment 1. The information processing system 100 comprises a feature extraction unit 112 and an estimation unit 113.

[0015] The feature extraction unit 112 extracts feature quantities of objects that appear in the image. The estimation unit 113 uses the extracted feature quantities to estimate the type of object, the corresponding graphic type, and the graphic parameters.

[0016] This information processing system 100 makes it possible to recognize a general shape that closely approximates the outer edge of an object contained in an image.

[0017] Figure 2 shows an overview of the information processing device 102 according to Embodiment 1. The information processing device 102 comprises a feature extraction unit 112 and an estimation unit 113.

[0018] The feature extraction unit 112 extracts feature quantities of objects that appear in the image. The estimation unit 113 uses the extracted feature quantities to estimate the type of object, the corresponding graphic type, and the graphic parameters.

[0019] This information processing device 102 makes it possible to recognize an approximate shape that closely approximates the outer edge of an object included in an image.

[0020] Figure 3 is a diagram showing an overview of the information processing according to Embodiment 1.

[0021] The feature extraction unit 112 extracts feature quantities of objects that appear in the image (step S102).

[0022] The estimation unit 113 uses the extracted features to estimate the type of object, the corresponding figure type, and the figure parameters (step S103).

[0023] This information processing makes it possible to recognize the general shape that closely approximates the outer edge of an object contained in an image.

[0024] The following describes a detailed example of the information processing system 100 according to Embodiment 1.

[0025] (detail) Generally, when recognizing the shape of objects in an image, recognizing the precise shape of an object often places a heavy processing load on the system and can be difficult. In some cases, recognizing the general shape of an object is preferable to understanding its form.

[0026] However, Patent Document 1 does not disclose any technology for recognizing the general shape of such an object.

[0027] In contrast, Patent Document 2 discloses, as mentioned above, that a rectangular detection window is expanded vertically and horizontally by a predetermined ratio to form the object range. Furthermore, Patent Document 2 discloses that the reason for expanding the detection window to a predetermined range rather than using the detection window as is is that, depending on the posture of the limbs of the human body, the area of ​​the human body may easily extend beyond the object detection window.

[0028] However, as illustrated in Patent Document 2, the shape of an object can vary depending on its state, even if it is the same object, as the area of ​​the human body may extend beyond the rectangular detection window depending on its posture. Therefore, depending on the posture of the human body, for example, even if the aspect ratio of the rectangular detection window is widened, the area of ​​the human body may extend significantly beyond the detection window, or the area other than the human body may become large within the detection window.

[0029] Thus, with the techniques described in Patent Documents 1 and 2, it is difficult to recognize a general shape that closely approximates the outer edge of an object included in an image.

[0030] One example of the purpose of this disclosure is, in view of these circumstances, to provide an information processing system, information processing device, information processing method, recording medium, etc., that solves the problem of recognizing an outline that closely approximates the outer edge of an object included in an image.

[0031] (Example of the configuration of the information processing system 100 according to Embodiment 1) Figure 4 shows an example of the configuration of the information processing system 100 according to Embodiment 1. The information processing system 100 is a system for recognizing the general shape of an object contained in an image to be processed (hereinafter also referred to as the "image to be processed").

[0032] The "object" is, for example, an object that appears in the image. The object includes, for example, a person or at least one of two things. The object may also be a specific part of an object, such as the iris or face.

[0033] The information processing system 100 comprises an image storage device 101 and an information processing device 102.

[0034] The image storage device 101 and the information processing device 102 are connected to each other via a network N that is configured by wire, wireless, or a combination thereof, and they send and receive information to each other via the network N.

[0035] (Example of the functional configuration of the image storage device 101 according to Embodiment 1) The image storage device 101 is a device for storing images to be processed. Images to be processed are, for example, images generated by a camera or other imaging device. The image storage device 101 is preferably pre-stored with images to be processed.

[0036] The information processing system 100 may also include, in place of, or together with, the image storage device 101, an imaging device connected to the network N.

[0037] (Example of the functional configuration of the information processing device 102 according to Embodiment 1) The information processing device 102 is a device that performs information processing to recognize the general shape of an object contained in an image to be processed. Functionally, the information processing device 102 comprises an image acquisition unit 111, a feature extraction unit 112, an estimation unit 113, and an output unit 114.

[0038] The image acquisition unit 111 acquires the image to be processed. In this embodiment, the image acquisition unit 111 acquires the image to be processed from the image storage device 101.

[0039] The feature extraction unit 112 extracts the feature quantities of one or more objects that appear in the image to be processed from the image to be processed.

[0040] The estimation unit 113 uses the feature quantities extracted by the feature extraction unit 112 for each of one or more objects to estimate the type of object, the corresponding graphic type, and the graphic parameters for each of one or more objects.

[0041] The "type of object" can be predetermined, and in this embodiment, it is the type of object. Examples of the type of object in this embodiment include a person, an umbrella, a bag, a dog, a car, a bicycle, and so on.

[0042] The shape type and shape parameters corresponding to the object are information used to identify the shape according to the general shape of the object. The "general shape of the object" refers to the approximate shape indicated by the outer edge of the object.

[0043] "Shape type" refers to a predetermined type of shape.

[0044] Examples of geometric shapes according to this embodiment include polygons (e.g., triangles, quadrilaterals, pentagons, etc.), circles, ellipses, curves, closed curves, and straight lines.

[0045] Triangles can be further defined as equilateral triangles, isosceles triangles, etc. Similarly, quadrilaterals can be squares, rectangles, parallelograms, trapezoids, etc.

[0046] A closed curve shape refers to the shape formed by a closed curve. A closed curve is a closed line that contains a curve in at least part of it; that is, a line whose start and end points are common and which contains a curve in at least part of it. Examples of curves, or curves contained within a closed curve, include Bézier curves and spline curves.

[0047] The shape type should include at least one of the following: polygon, circle, ellipse, curve, closed curve, or straight line.

[0048] "Geometric parameters" are parameters used to identify the geometric shape represented by the corresponding geometric type in the image being processed. In other words, geometric parameters are values ​​that specifically define the geometric shape represented by that geometric type in the image being processed.

[0049] Examples of graphic parameters according to this embodiment include the size and rotation angle of the graphic type, and its position in the image. The rotation angle may be expressed, for example, as the angle of a reference direction that indicates a predetermined reference orientation for each graphic type.

[0050] Furthermore, the shape parameters should include at least one of the following: the size and rotation angle of the shape indicated by the shape type, and its position in the image.

[0051] The output unit 114 outputs the type of object, graphic type, and graphic parameters estimated by the estimation unit 113 for each of one or more objects. The output method may be, for example, display, or transmission to another device (not shown), such as a terminal. However, the output method is not limited to these.

[0052] Up to this point, we have mainly described the functional configuration example of the information processing system 100 according to Embodiment 1. From here, we will describe the physical configuration example of the information processing system 100 according to Embodiment 1.

[0053] (Example of physical configuration of the information processing system 100 according to Embodiment 1) The information processing system 100 consists of an image storage device 101 and an information processing device 102 that are physically connected via a network N. Each of the image storage device 101 and the information processing device 102 consists of, for example, a physically different single device.

[0054] The image storage device 101 and the information processing device 102 may be physically composed of a single device. In this case, the image storage device 101 and the information processing device 102 may be connected using an internal bus 1010, which will be described later, instead of a network N. Alternatively, one or both of the image storage device 101 and the information processing device 102 may be physically composed of multiple devices connected via an appropriate communication line, such as a network N.

[0055] The image storage device 101 according to this embodiment may be physically configured in the same way as the information processing device 102. An example of the physical configuration of the information processing device 102 will be described with reference to the figure.

[0056] Figure 5 shows an example of the physical configuration of the information processing device 102 according to Embodiment 1. The information processing device 102 physically includes, for example, a bus 1010, a processor 1020, a memory 1030, a storage device 1040, a network interface 1050, an input interface 1060, and an output interface 1070.

[0057] Bus 1010 is a data transmission path for the processor 1020, memory 1030, storage device 1040, network interface 1050, input interface 1060, and output interface 1070 to send and receive data to and from each other. However, the method of connecting the processor 1020 and other components to each other is not limited to bus connection.

[0058] Processor 1020 is a processor implemented in components such as the CPU (Central Processing Unit) and GPU (Graphics Processing Unit).

[0059] Memory 1030 is a main memory device implemented using RAM (Random Access Memory), etc.

[0060] The storage device 1040 is an auxiliary storage device implemented as an HDD (Hard Disk Drive), SSD (Solid State Drive), memory card, or ROM (Read Only Memory). The storage device 1040 stores program modules for realizing the functions of the device equipped with it. The processor 1020 loads each of these program modules into memory 1030 and executes them, thereby realizing the functions corresponding to those program modules.

[0061] The network interface 1050 is an interface for connecting a device equipped with it to network N.

[0062] The input interface 1060 is an interface for the user to input information. The input interface 1060 consists of one or more components, such as a touch panel, keyboard, and mouse.

[0063] The output interface 1070 is an interface for presenting information to the user. The output interface 1070 is composed of, for example, a liquid crystal panel, an organic EL (Electro-Luminescence) panel, etc.

[0064] Up to this point, we have mainly described an example configuration of the information processing system 100 according to Embodiment 1. From here, we will describe an example of operation of the information processing system 100 according to Embodiment 1.

[0065] (Example of operation of the information processing system 100 according to Embodiment 1) The information processing device 102 performs information processing to recognize the general shape of an object included in the image to be processed. Information processing is initiated, for example, by user instruction.

[0066] Furthermore, the information processing may include the process of generating the image to be processed through photography (photography process). In this case, the information processing may be performed repeatedly in real time.

[0067] Figure 6 is a flowchart showing an example of information processing according to Embodiment 1.

[0068] The image acquisition unit 111 acquires the image to be processed from the image storage device 101 via the network N (step S101).

[0069] The feature extraction unit 112 extracts the feature quantities of one or more objects that appear in the image to be processed acquired in step S101 from the image to be processed (step S102).

[0070] In step S102, the technique for extracting the target features from the image to be processed can be a general technique. For example, the feature extraction unit 112 takes the image to be processed as input and extracts the target features using an image processing model. The image processing model is a machine learning model that has been trained to extract the target features contained in the image. When training the image processing model, it is preferable to perform supervised learning to extract the target features contained in the training images.

[0071] The estimation unit 113 uses the features extracted in step S102 to estimate the type of object, the corresponding graphic type, and the graphic parameters for each of one or more objects (step S103).

[0072] Figure 7 shows a first example of the estimated object type, shape type, and shape parameters for an object. In the example shown in Figure 7, a person holding an umbrella is extracted from the image to be processed and includes two objects P1 and P2. The type of object P1 is a person. The type of object P2 is an umbrella.

[0073] The shape type associated with target P1 is rectangle F1. The shape parameters associated with target P1 include parameters (values) for identifying rectangle F1 associated with target P1 in the image to be processed. These parameters include, for example, the position of the centroid G1 of rectangle F1, the horizontal and vertical lengths of rectangle F1, and the rotation angle. In the example shown in Figure 7, the reference direction V1 for rectangle F1 is defined as parallel to the base, and the rotation angle is 0 degrees (for example, in the same direction as the X-axis of the coordinate system defined for the image to be processed).

[0074] The geometric type associated with target P2 is an isosceles triangle F2. The geometric parameters associated with target P2 include parameters (values) for identifying the isosceles triangle F2 associated with target P2 in the image to be processed. These parameters include, for example, the position of the centroid G2 of the isosceles triangle F2, the lengths of the base and height of the isosceles triangle F2, and the rotation angle. In the example shown in Figure 7, the reference direction V2 for the isosceles triangle F2 is defined as parallel to the base, and the rotation angle is 0 degrees (for example, in the same direction as the X-axis of the coordinate system defined for the image to be processed).

[0075] Figure 8 shows a second example of the estimated object type, shape type, and shape parameters for an object. In the example shown in Figure 8, a seated person is extracted and included as object P3.

[0076] The shape type associated with target P3 is an isosceles triangle F2. The shape parameters associated with target P3 include parameters (values) for identifying the isosceles triangle F2 associated with target P3 in the image to be processed. These parameters include, for example, the position of the centroid G3 of the isosceles triangle F2, the lengths of the base and height of the isosceles triangle F2, and the rotation angle. In the example shown in Figure 8, the reference direction V2 for the isosceles triangle F2 is defined as parallel to the base, and the rotation angle is 0 degrees (for example, in the same direction as the X-axis of the coordinate system defined for the image to be processed).

[0077] Figure 9 shows a third example of the estimated object type, shape type, and shape parameters for an object. In the example shown in Figure 9, an umbrella placed in an umbrella stand is extracted and shown, and this umbrella is included as object P4.

[0078] The shape type associated with target P4 is rectangle F1. The shape parameters associated with target P4 include parameters (values) for identifying rectangle F1 associated with target P4 in the image to be processed. These parameters include, for example, the position of the centroid G4 of rectangle F1, the horizontal and vertical lengths of rectangle F1, and the rotation angle. In the example shown in Figure 9, the reference direction V1 for rectangle F1 is defined as parallel to the base, and the rotation angle is θ degrees. θ is, for example, the angle made around a predetermined direction with respect to the positive X-axis of the coordinate system defined for the image to be processed.

[0079] The centroids G1 to G4 are examples of representative positions predetermined for each geometric type. Note that the representative position is not limited to the centroid; it should be determined for the geometric shape represented by the geometric type.

[0080] In this way, for each of one or more objects, the estimation unit 113 identifies a type of shape appropriate to the object from among predetermined shape types used to identify the general shape of the object, in addition to the type of object. Furthermore, the estimation unit 113 estimates parameters for identifying the shape represented by the identified shape type in the image to be processed. This makes it possible to estimate the general shape of the object in the image to be processed using the various shapes included in the shape type.

[0081] Furthermore, as these examples show, the number of geometric parameters may vary depending on the geometric type. When the number of geometric parameters varies depending on the geometric type, the geometric parameters may be represented by vectors of a length corresponding to the number of geometric parameters, or by fixed-length vectors common to all geometric types. If fixed-length vectors are used, the size of these vectors should be the same as the maximum number of geometric parameters among all predetermined geometric types. Unused elements of the fixed-length vectors may be stored with predetermined values ​​such as blanks or null values.

[0082] Refer to Figure 6 again. The output unit 114 outputs the type of object, graphic type, and graphic parameters estimated for each of the one or more objects in step S103 (step S104).

[0083] For example, if the output is a display, the output unit 114 overlays the shapes identified by the estimated shape type and shape parameters for each of the one or more objects onto the image to be processed. This allows the shapes representing the general shapes of each object in the image to be processed to be displayed together with the image to be processed.

[0084] (Effects / Actions) As described above, according to this embodiment, the information processing system 100 comprises a feature extraction unit 112 and an estimation unit 113. The feature extraction unit 112 extracts feature quantities of objects depicted in the image. The estimation unit 113 uses the extracted feature quantities to estimate the type of object, the corresponding graphic type, and the graphic parameters.

[0085] This allows the general shape of an object in the image being processed to be estimated using one of the various shapes included in the shape type. Therefore, it becomes possible to recognize a general shape that closely approximates the outer edge of an object contained in the image.

[0086] According to this embodiment, the shape type includes at least one of polygons, circles, ellipses, curves, closed curves, and straight lines. The shape parameters include at least one of the size and rotation angle of the shape represented by the shape type, and its position in the image.

[0087] This allows the general shape of an object in the image being processed to be estimated using one of the various shapes included in the shape type. Therefore, it becomes possible to recognize a general shape that closely approximates the outer edge of an object contained in the image.

[0088] According to this embodiment, the geometric parameters are represented by fixed-length vectors common to all geometric types.

[0089] This allows geometric parameters to be treated as variables of the same size, regardless of the geometric type. Consequently, it becomes easier to process images to recognize a general shape that closely approximates the outer edge of an object within the image.

[0090] <Embodiment 2> A first detailed example of how the estimation unit estimates the type of object, the corresponding geometric type, and the geometric parameters is described below.

[0091] In this embodiment, in order to simplify the explanation, we will mainly describe the differences from Embodiment 1, and explanations that overlap with Embodiment 1 will be omitted as appropriate.

[0092] Figure 10 shows an example configuration of the information processing system 200 according to Embodiment 2. The information processing system 200 includes an information processing device 202 that replaces the information processing device 102 according to Embodiment 1. Functionally, the information processing device 202 includes an estimation unit 213 that replaces the estimation unit 113 according to Embodiment 1. Aside from these, the information processing system 200 according to this embodiment may be configured in the same way as the information processing device 102 according to Embodiment 1.

[0093] The estimation unit 213 has the same functions as the estimation unit 113 according to Embodiment 1. Functionally, the estimation unit 213 includes a target estimation unit 221 and a graphic estimation unit 222.

[0094] The target estimation unit 221 estimates the type of target using the feature quantities extracted by the feature extraction unit 112 for each of one or more targets.

[0095] The shape estimation unit 222 estimates the shape type and shape parameters corresponding to each of one or more objects based on the type of object estimated by the object estimation unit 221.

[0096] Figure 11 shows an example of the functional configuration of the figure estimation unit 222 according to Embodiment 2. Functionally, the figure estimation unit 222 includes a correspondence information storage unit 231, a figure type estimation unit 232, a parameter estimation unit 233, and a determination unit 234.

[0097] The association information storage unit 231 is a storage unit for storing the association information 231a.

[0098] Figure 12 shows an example of the correspondence information 231a. The correspondence information 231a is information that associates the type of object with one or more graphic types. In this way, the correspondence information 231a defines the graphic types used to recognize the general shape of the object for each type of object.

[0099] The example of mapping information 231a shown in Figure 12 includes "person" and "umbrella" as object types. Furthermore, the object type "person" is associated with the geometric types "rectangle," "isosceles triangle," and "ellipse."

[0100] Refer to Figure 11 again. The shape type estimation unit 232 estimates one or more shape types corresponding to each of the one or more objects, based on the type of object estimated by the object estimation unit 221 for each of the objects.

[0101] In this embodiment, the graphic type estimation unit 232 estimates one or more graphic types corresponding to each of one or more objects based on the type of object estimated by the object estimation unit 221 and the correspondence information 231a.

[0102] The parameter estimation unit 233 estimates the geometric parameters corresponding to each of the one or more objects for each of the one or more geometric types estimated by the geometric type estimation unit 232. The parameter estimation unit 233 may, for example, use a fitting score, as described later, to estimate the geometric parameters.

[0103] When there are multiple estimated geometric types, the determination unit 234 determines the geometric type and geometric parameters corresponding to the target from among the combinations of the multiple estimated geometric types and the geometric parameters estimated for each of them.

[0104] The determination unit 234 uses a fit score to determine the combination of graphic type and graphic parameters corresponding to the target. That is, the determination unit 234 determines the graphic type and graphic parameters corresponding to the target based on the graphic parameters estimated by the parameter estimation unit 233 for each of the estimated multiple graphic types, and the fit score.

[0105] The fit score is a value that indicates the degree to which a shape fits the outer edge of an object. For example, the greater the degree to which a shape fits the outer edge of an object, the higher the fit score; and the less the degree to which a shape fits the outer edge of an object, the lower the fit score.

[0106] The fit score is determined according to a general shape estimation criterion defined using the areas of at least two of the three regions (Region 1 to Region 3).

[0107] Figure 13 shows the first to third regions in the second example shown in Figure 8. The first region is the area where the figure and the object overlap each other. The second region is the area within the object that extends beyond the figure. The third region is the area within the figure that extends beyond the object.

[0108] For example, the general shape estimation criteria should be defined such that the area of ​​the first region or the ratio of the first region to the area within the figure is maximized, and the fit score of the figure that best encompasses the subject is maximized.

[0109] For example, the general shape estimation criterion may be one in which the larger the area of ​​the first region, the higher the fit score, and the smaller the area of ​​the second region (or third region), the higher the fit score. Alternatively, for example, the general shape estimation criterion may be one in which the smaller the difference in area between the second region and the third region, the higher the fit score.

[0110] In this embodiment, the determination unit 234 calculates a fit score for each of the multiple combinations of estimated graphic types and graphic parameters. The determination unit 234 then determines the combination of graphic types and graphic parameters that yields the highest fit score as the graphic type and graphic parameters corresponding to the target.

[0111] Up to this point, we have mainly described an example of the functional configuration of the information processing system 200 according to Embodiment 2. Physically, the information processing system 200 should be configured in the same way as the information processing system 100 according to Embodiment 1. From here, we will describe an example of the operation of the information processing system 200 according to Embodiment 2.

[0112] (Example of operation of the information processing system 200 according to Embodiment 2) The information processing device 202 performs information processing to recognize the general shape of an object included in the image to be processed, similar to the information processing device 102 according to Embodiment 1.

[0113] Figure 14 is a flowchart showing an example of information processing according to Embodiment 2. The information processing according to Embodiment 2 includes an estimation process (step S203) that replaces the estimation process (step S103) according to Embodiment 1. Except for this, the information processing according to Embodiment 2 may be the same as the information processing according to Embodiment 1. Figure 14 shows the details of the estimation process (step S203).

[0114] The estimation unit 213, similar to step S103 in Embodiment 1, uses the feature quantities extracted in step S102 to estimate the type of object, the corresponding graphic type, and the graphic parameters (step S203).

[0115] As shown in the figure, the target estimation unit 221 estimates the type of target using the feature quantities extracted for each of one or more targets in step S102 (step S211).

[0116] In step S211, the technique for estimating the type of object using the object's features can be a general technique. For example, the object estimation unit 221 takes the object's features as input and extracts the type of object using a type estimation model. The type estimation model is a machine learning model that has been trained to estimate the type of object based on the object's features. When training the type estimation model, it is preferable to perform supervised learning to estimate the type of object using the object's features extracted from training images.

[0117] The shape estimation unit 222 estimates the shape type and shape parameters corresponding to each of the objects based on the type of object estimated for each of the one or more objects in step S211 (step S212; shape estimation process).

[0118] Figure 15 is a flowchart showing an example of the shape estimation process (step S212) according to Embodiment 2.

[0119] The shape type estimation unit 232 estimates one or more shape types corresponding to each of the objects based on the type of object estimated for each of the one or more objects in step S211 (step S221).

[0120] In detail, for example, the shape type estimation unit 232 refers to the correspondence information 231a and identifies one or more shape types associated with each type of object estimated in step S211. This allows the shape type estimation unit 232 to estimate one or more shape types corresponding to the object.

[0121] For example, if the estimated type of object is "person," and referring to the correspondence information 231a illustrated in Figure 12, the shape type estimation unit 232 estimates a rectangle, an isosceles triangle, and an ellipse as the shape types.

[0122] Refer to Figure 15 again. The parameter estimation unit 233 estimates the geometric parameters corresponding to each of the one or more geometric types estimated for each of the one or more objects in step S211 (step S222).

[0123] The method for estimating the geometric parameters in step S222 can vary, including methods for changing the geometric parameters or methods for using machine learning models.

[0124] For example, when using a method that modifies the geometric parameters, the parameter estimation unit 233 estimates the geometric parameters for each of the estimated one or more geometric types in such a way that the fit score is maximized. Specifically, for example, the parameter estimation unit 233 refers to the image to be processed and modifies each geometric parameter for each geometric type, for example, in predetermined increments, and obtains the fit score for each geometric parameter. Then, for each geometric type, the parameter estimation unit 233 estimates the geometric parameters that maximize the fit score.

[0125] For example, when using a machine learning model, the parameter estimation unit 233 takes one or more estimated shape types and the image to be processed as input and uses the parameter estimation model to estimate the shape parameters corresponding to the target for each of the one or more estimated shape types. The parameter estimation model is a machine learning model that has been trained to estimate the shape parameters corresponding to the target based on the shape type.

[0126] When training a parameter estimation model, it is advisable to perform supervised learning to estimate the geometric parameters corresponding to each of the one or more geometric types, using training images and one or more geometric types estimated for the objects contained in those training images. In this learning, for example, it is advisable to train the model to estimate the geometric parameters in a way that maximizes the fit score.

[0127] The determination unit 234 determines whether there are multiple figure types estimated in step S221 (step S223). If there are not multiple figure types estimated (step S223; No), the determination unit 234 terminates the figure estimation process (step S212) and returns to information processing. As shown in Figure 14, step S104, which is the same as in Embodiment 1, is then executed.

[0128] Refer to Figure 15 again. If there are multiple estimated figure types (step S223; Yes), the determination unit 234 determines the figure type and figure parameters corresponding to the target (step S224), and terminates the figure estimation process (step S212). The determination unit 234 returns to information processing, and step S104, similar to that in Embodiment 1, is executed as shown in Figure 14.

[0129] In detail, for example, the determination unit 234 refers to the image to be processed and calculates a fit score for each of the multiple combinations of graphic types and graphic parameters estimated in steps S211 to S212. The determination unit 234 determines the combination of graphic type and graphic parameters that yields the highest fit score as the graphic type and graphic parameters corresponding to the target. In this way, the determination unit 234 determines the graphic type and graphic parameters corresponding to the target from among the multiple combinations of graphic types and graphic parameters estimated in steps S211 to S212.

[0130] The parameter estimation unit 233 may be configured to output the combination of graphic type and graphic parameters that maximizes the fit score as the estimation result. In this case, the information processing device 202 does not need to include the determination unit 234. Also, the information processing does not need to include steps S223 to S224.

[0131] (Effects / Actions) As described above, according to this embodiment, the estimation unit 213 includes a target estimation unit 221 and a shape estimation unit 222. The target estimation unit 221 estimates the type of target using the extracted features. The shape estimation unit 222 estimates the shape type and shape parameters corresponding to the target based on the estimated type of target.

[0132] This allows the general shape of an object in the image being processed to be estimated using one of the various shapes included in the shape type. Therefore, it becomes possible to recognize a general shape that closely approximates the outer edge of an object contained in the image.

[0133] According to this embodiment, the shape estimation unit 222 includes a shape type estimation unit 232 and a parameter estimation unit 233. The shape type estimation unit 232 estimates one or more shape types corresponding to an object based on the estimated type of object. The parameter estimation unit 233 estimates the shape parameters corresponding to the object for the estimated one or more shape types.

[0134] This allows the general shape of an object in the image being processed to be estimated using one of the various shapes included in the shape type. Therefore, it becomes possible to recognize a general shape that closely approximates the outer edge of an object contained in the image.

[0135] According to this embodiment, the graphic type estimation unit 232 estimates one or more graphic types corresponding to an object based on the estimated object type and the correspondence information 231a that associates the object type with one or more graphic types.

[0136] This allows the general shape of an object in the image being processed to be estimated using one of the various shapes included in the shape type. Therefore, it becomes possible to recognize a general shape that closely approximates the outer edge of an object contained in the image.

[0137] According to this embodiment, the figure estimation unit 222 further includes a determination unit 234. When there are multiple estimated figure types, the determination unit 234 determines the figure type and figure parameters corresponding to the target based on the estimated figure parameters for each of the multiple figure types and a fit score indicating the degree to which the figure fits the outer edge of the target.

[0138] This allows the general shape of an object in a processed image to be estimated using a shape that best fits the object's outer edge among the various shapes included in the shape type. Therefore, it becomes possible to recognize a general shape that closely approximates the outer edge of an object in an image.

[0139] According to this embodiment, the fit score is a value obtained according to a general shape estimation criterion defined using the areas of at least two of the following regions: (A) the first region, (B) the second region, and (C) the third region. (A) The first region is the region where the figure and the object overlap each other. (B) The second region is the region within the object that extends beyond the figure. (C) The third region is the region within the figure that extends beyond the object.

[0140] This allows the system to estimate a general shape that closely approximates the outer edge of an object in the image being processed, using the fitting score. Therefore, it becomes possible to recognize a general shape that closely approximates the outer edge of an object in the image.

[0141] <Embodiment 3> A second detailed example of how the estimation unit estimates the type of object, the corresponding geometric type, and the geometric parameters is described below.

[0142] In this embodiment, in order to simplify the explanation, we will mainly describe the differences from other embodiments, and explanations that overlap with other embodiments will be omitted as appropriate.

[0143] Figure 16 shows an example configuration of the information processing system 300 according to Embodiment 3. The information processing system 300 includes an information processing device 302 that replaces the information processing device 102 according to Embodiment 1. Functionally, the information processing device 302 includes an estimation unit 313 that replaces the estimation unit 113 according to Embodiment 1. Aside from these, the information processing system 300 according to this embodiment may be configured in the same way as the information processing device 102 according to Embodiment 1.

[0144] The estimation unit 313 has the same functions as the estimation unit 113 according to Embodiment 1. Functionally, the estimation unit 313 includes a target estimation unit 221 and a graphic estimation unit 322, similar to those in Embodiment 2.

[0145] The shape estimation unit 322 estimates the shape type and shape parameters corresponding to each of the one or more objects based on the feature quantities extracted by the feature extraction unit 112 for each of the objects.

[0146] Figure 17 shows an example of the functional configuration of the shape estimation unit 322 according to Embodiment 3. Functionally, the shape estimation unit 322 includes a shape type estimation unit 332 and a parameter estimation unit 233 and determination unit 234 similar to those in Embodiment 2.

[0147] The shape type estimation unit 332 takes the feature quantities extracted by the feature extraction unit 112 for each of the one or more objects as input and uses a shape type estimation model to estimate one or more shape types corresponding to the objects. The shape type estimation model is a machine learning model that has been trained to estimate at least one shape type corresponding to an object in an image. It is preferable that multiple types of shapes be predetermined for the shape type estimation model to estimate.

[0148] When training a shape type estimation model, it is advisable to perform supervised learning to estimate one or more shape types corresponding to an object using the feature quantities of the object extracted from the training images.

[0149] Up to this point, we have mainly described an example of the functional configuration of the information processing system 300 according to Embodiment 3. Physically, the information processing system 300 should be configured in the same way as the information processing system 100 according to Embodiment 1. From here, we will describe an example of the operation of the information processing system 300 according to Embodiment 2.

[0150] (Example of operation of the information processing system 300 according to Embodiment 3) The information processing device 302 performs information processing to recognize the general shape of an object included in the image to be processed, similar to the information processing device 102 according to Embodiment 1.

[0151] Figure 18 is a flowchart showing an example of information processing according to Embodiment 3. The information processing according to Embodiment 3 includes an estimation process (step S303) that replaces the estimation process (step S103) according to Embodiment 1. Except for this, the information processing according to Embodiment 3 may be the same as the information processing according to Embodiment 1. Figure 18 shows the details of the estimation process (step S303).

[0152] The target estimation unit 221 performs the same step S211 as in Embodiment 2.

[0153] The shape estimation unit 322 estimates the shape type and shape parameters corresponding to each of the one or more objects based on the feature quantities extracted for each of the objects in step S102 (step S312; shape estimation process).

[0154] Figure 19 is a flowchart showing an example of the shape estimation process (step S312) according to Embodiment 3.

[0155] The shape type estimation unit 332 takes the feature quantities extracted for each of the one or more objects in step S102 as input and uses a shape type estimation model to estimate one or more shape types corresponding to the objects (step S321).

[0156] The parameter estimation unit 233 and the determination unit 234 perform steps S222 to S224, similar to those in Embodiment 2, and then return to information processing. Subsequently, as shown in Figure 18, step S104, similar to that in Embodiment 1, is performed.

[0157] (Effects / Actions) As described above, according to this embodiment, the estimation unit 313 includes a target estimation unit 221 and a shape estimation unit 322. The target estimation unit 221 estimates the type of target using the extracted features. The shape estimation unit 322 estimates the shape type and shape parameters corresponding to the target based on the extracted features.

[0158] This allows the general shape of an object in the image being processed to be estimated using one of the various shapes included in the shape type. Therefore, it becomes possible to recognize a general shape that closely approximates the outer edge of an object contained in the image.

[0159] According to this embodiment, the shape estimation unit 322 includes a shape type estimation unit 332 and a parameter estimation unit 233. The shape type estimation unit 332 estimates one or more shape types corresponding to the target based on the extracted features. The parameter estimation unit 233 estimates the shape parameters corresponding to the target for the estimated one or more shape types.

[0160] This allows the general shape of an object in the image being processed to be estimated using one of the various shapes included in the shape type. Therefore, it becomes possible to recognize a general shape that closely approximates the outer edge of an object contained in the image.

[0161] According to this embodiment, the shape type estimation unit 332 takes the extracted feature quantities as input and uses a shape type estimation model that has been trained to estimate at least one shape type corresponding to an object in the image to estimate one or more shape types corresponding to the object.

[0162] This allows the general shape of an object in the image being processed to be estimated using one of the various shapes included in the shape type. Therefore, it becomes possible to recognize a general shape that closely approximates the outer edge of an object contained in the image.

[0163] <Embodiment 4> The configuration of the parameter estimation unit 233 and the details of the geometric parameter estimation process (step S222) performed by the parameter estimation unit 233, as described in Embodiments 2 and 3, vary. Embodiment 4 will describe a detailed example of the parameter estimation unit 233 and the geometric parameter estimation process (step S222).

[0164] Figure 20 shows an example of the functional configuration of the parameter estimation unit 233 according to Embodiments 2 to 3. The parameter estimation unit 233 includes first to Nth figure type estimation units 233A_1 to 233A_N (where N is an integer of 1 or more) and a selection unit 233B.

[0165] If there is no particular distinction between the "Figure Type Estimation Units 233A_1 to 233A_N" from the 1st to the Nth, they will also be referred to as "Figure Type Estimation Unit 233A".

[0166] The first to nth figure type estimation units 233A_1 to 233A_N are associated with each of the first to nth predetermined figure types. The first to nth predetermined figure types correspond to, for example, each of the figure types included in the association information 231a. The first to nth figure type estimation units 233A_1 to 233A_N estimate the figure parameters corresponding to the target for each associated figure type.

[0167] For example, suppose the geometric types included in the correspondence information 231a are three types: rectangle, isosceles triangle, and ellipse. In this case, the parameter estimation unit 233 includes, for example, first to third geometric type estimation units 233A_1 to 233A_3, each associated with a rectangle, isosceles triangle, and ellipse.

[0168] In other words, in this case, the first figure type estimation unit 233A_1 is a figure type estimation unit 233A associated with a rectangle, and estimates the figure parameters for the rectangle. The second figure type estimation unit 233A_2 is a figure type estimation unit 233A associated with an isosceles triangle, and estimates the figure parameters for the isosceles triangle. The third figure type estimation unit 233A_3 is a figure type estimation unit 233A associated with an ellipse, and estimates the figure parameters for the ellipse.

[0169] Various methods can be employed for each of the shape type estimation units 233A to estimate the shape parameters, such as methods for changing the shape parameters as described above, or methods for using machine learning models.

[0170] When using a machine learning model, each of the shape type estimation units 233A takes, for example, the image to be processed or an image of an object contained in the image to be processed as input, and uses a shape type parameter estimation model to estimate the shape parameters corresponding to the object for the associated shape type. The shape type parameter estimation model is a machine learning model that has been trained to estimate the shape parameters corresponding to the object for the associated shape type.

[0171] When training a parameter estimation model for each shape type, it is advisable to perform supervised learning to estimate the shape parameters corresponding to the object for each associated shape type, using training images or images of the object contained within those training images. In this learning, for example, it is advisable to perform learning to estimate the shape parameters in a way that maximizes the fit score.

[0172] The selection unit 233B selects a figure type estimation unit 233A to be used to estimate figure parameters corresponding to the target, based on one or more figure types estimated by the figure type estimation unit 232.

[0173] For example, the selection unit 233B selects a figure type estimation unit 233A from among the first to Nth figure type estimation units 233A_1 to 233A_N that corresponds to each of the one or more figure types estimated by the figure type estimation unit 232.

[0174] Figure 21 is a flowchart showing an example of the parameter estimation process (step S222) according to Embodiments 2 and 3.

[0175] The selection unit 233B selects a figure type estimation unit 233A to be used to estimate figure parameters corresponding to the target, based on one or more figure types estimated in step S221 or S321 (step S222a).

[0176] In detail, for example, the selection unit 233B selects a figure type estimation unit 233A to which one or more estimated figure types are associated.

[0177] The shape type estimation unit 233A selected in step S222a estimates the shape parameters corresponding to the target for the associated shape type (step S222b), and returns to the shape estimation process (step S212 or S312).

[0178] (Effects / Actions) As described above, according to this embodiment, the parameter estimation unit 233 includes first to nth figure type estimation units 233A_1 to 233A_N and a selection unit 233B. The first to nth figure type estimation units 233A_1 to 233A_N estimate the figure parameters corresponding to the target for each of the first to nth predetermined figure types associated with them. The selection unit 233B selects one or more figure type estimation units 233A_1 to 233A_N to be used to estimate the figure parameters corresponding to the target based on the estimated figure types.

[0179] This allows for the estimation of geometric parameters using a geometric type-specific estimation unit 233A, which is specialized for each geometric type. Therefore, it becomes possible to estimate geometric parameters that better fit the general shape of the object in the image being processed. This makes it possible to recognize a general shape that more accurately approximates the outer edge of the object contained in the image.

[0180] <Embodiment 5> A third detailed example of how the estimation unit estimates the type of object, the corresponding geometric type, and the geometric parameters is described below.

[0181] In this embodiment, in order to simplify the explanation, we will mainly describe the differences from other embodiments, and explanations that overlap with other embodiments will be omitted as appropriate.

[0182] Figure 22 shows an example configuration of the information processing system 500 according to Embodiment 5. The information processing system 500 includes an information processing device 502 that replaces the information processing device 102 according to Embodiment 1. Functionally, the information processing device 502 includes an estimation unit 513 that replaces the estimation unit 113 according to Embodiment 1. Aside from these, the information processing system 500 according to this embodiment may be configured in the same way as the information processing device 102 according to Embodiment 1.

[0183] The estimation unit 513 has the same functions as the estimation unit 113 according to Embodiment 1. Functionally, the estimation unit 513 includes a target estimation unit 221 and a graphic estimation unit 522, similar to those in Embodiment 2.

[0184] The shape estimation unit 522 takes the features extracted by the feature extraction unit 112 as input and uses a shape estimation model to estimate the shape type and shape parameters corresponding to the object. The shape estimation model is a machine learning model trained to estimate the general shape of an object in an image. Multiple types of shapes may be predetermined for the shape type estimated by the shape estimation model. The shape estimation model may also be input with the image to be processed.

[0185] When training a shape estimation model, it is advisable to perform supervised learning to estimate the shape type and shape parameters corresponding to the objects contained in the training images, using features extracted from those training images. Further training images may also be input during the training of the shape estimation model.

[0186] In this learning process, for example, it is preferable to perform training to estimate the shape type and shape parameters in such a way that the aforementioned fit score is maximized. That is, the shape estimation model may be a machine learning model that takes the features of an object contained in a training image as input and outputs the shape type and shape parameters that represent the general shape of the object in such a way that the fit score, which indicates the degree to which the shape fits the outer edge of the object, is high.

[0187] Up to this point, we have mainly described an example of the functional configuration of the information processing system 500 according to Embodiment 5. Physically, the information processing system 500 should be configured in the same way as the information processing system 100 according to Embodiment 1. From here, we will describe an example of the operation of the information processing system 500 according to Embodiment 5.

[0188] (Example of operation of the information processing system 500 according to Embodiment 5) The information processing device 502 performs information processing to recognize the general shape of an object included in the image to be processed, similar to the information processing device 102 according to Embodiment 1.

[0189] Figure 23 is a flowchart showing an example of information processing according to Embodiment 5. The information processing according to Embodiment 5 includes an estimation process (step S503) that replaces the estimation process (step S103) according to Embodiment 1. Except for this, the information processing according to Embodiment 2 may be the same as the information processing according to Embodiment 1. Figure 23 shows the details of the estimation process (step S503).

[0190] The estimation unit 513, similar to step S103 in Embodiment 1, uses the feature quantities extracted in step S102 to estimate the type of object, the corresponding graphic type, and the graphic parameters (step S503).

[0191] As shown in the figure, the target estimation unit 221 performs the same step S211 as in Embodiment 2.

[0192] The shape estimation unit 522 uses the features extracted in step S102 to estimate the shape type and shape parameters corresponding to the target using a shape estimation model (step S512). Subsequently, step S104, which is the same as in Embodiment 1, is performed.

[0193] (Effects / Actions) As described above, according to this embodiment, the shape estimation unit 522 takes the extracted feature quantities as input and uses a shape estimation model that has been trained to estimate the general shape of an object in the image to estimate the shape type and shape parameters corresponding to the object.

[0194] This allows the general shape of an object in the image being processed to be estimated using one of the various shapes included in the shape type. Therefore, it becomes possible to recognize a general shape that closely approximates the outer edge of an object contained in the image.

[0195] According to this embodiment, the shape estimation model is a machine learning model that takes the features of an object contained in a training image as input and outputs a shape type and shape parameters that represent the general shape of the object in order to achieve a high fit score. The fit score indicates the degree to which the shape fits the outer edge of the object.

[0196] This allows us to estimate the general shape of an object in a processed image using a figure that closely matches the object's outer edge. Therefore, it becomes possible to recognize a general shape that closely approximates the outer edge of an object contained in an image.

[0197] <Embodiment 6> The estimation results regarding the general shape of the object, as described in other embodiments, may be used for authentication. For example, the estimation results can be used to extract predetermined target areas, such as faces or irises of people, included in the image to be processed, and biometric authentication such as facial recognition or iris recognition can be performed. Embodiment 6 describes the case where the object is a face (head) or an iris as an example. Such objects are typically parts of a person, but may also be parts of animals such as dogs or snakes.

[0198] In this embodiment, in order to simplify the explanation, we will mainly describe the differences from Embodiment 1, and explanations that overlap with Embodiment 1 will be omitted as appropriate.

[0199] Figure 24 shows an example configuration of an information processing system 600 according to Embodiment 6. The information processing system 600 includes an information processing device 602 that replaces the information processing device 102 according to Embodiment 1. Functionally, the information processing device 602 includes a region extraction unit 631 and a matching unit 632 in addition to the configuration of the information processing device 102 according to Embodiment 1. Aside from these, the information processing system 200 according to this embodiment may be configured in the same way as the information processing device 102 according to Embodiment 1.

[0200] The region extraction unit 631 extracts the target region, which is the image region in which the object is depicted, from the image to be processed, based on the shape type and shape parameters estimated by the estimation unit 113.

[0201] The object in this embodiment is, for example, a face (head) or an iris. Furthermore, if the object is an iris, the graphic type may include a double circle, a double circle with parts of the top and bottom cut out, and so on.

[0202] The matching unit 632 performs a matching process for authentication using the extracted target area. Authentication is, for example, biometric authentication. More specifically, if the target is a face, the authentication is facial recognition. If the target is an iris, the authentication is iris recognition. General techniques may be used for such matching processes for authentication.

[0203] Up to this point, we have mainly described an example of the functional configuration of the information processing system 600 according to Embodiment 6. Physically, the information processing system 600 should be configured in the same way as the information processing system 100 according to Embodiment 1. From here, we will describe an example of the operation of the information processing system 600 according to Embodiment 6.

[0204] (Example of operation of the information processing system 600 according to Embodiment 6) The information processing device 602 performs information processing. The information processing according to this embodiment includes authentication processing in addition to the information processing performed by the information processing device 102 according to Embodiment 1. The authentication processing is a process for authenticating a person or other person who has been registered in advance, based on the image to be processed.

[0205] Figure 25 is a flowchart showing an example of the authentication process according to Embodiment 6.

[0206] The region extraction unit 631 extracts the target region, which is the image region in which the object is depicted, from the image to be processed, based on the graphic type and graphic parameters output in step S104, for example (step S601).

[0207] The matching unit 632 performs a matching process for authentication using the target area extracted in step S601 (step S602).

[0208] In detail, for example, the matching unit 632 holds registration information that includes the characteristics of a person that has been registered in advance. The matching unit 632 extracts the characteristics of the target region extracted in step S601. The matching unit 632 compares the extracted characteristics with the characteristics included in the registration information. As a result of the comparison, the matching unit 632 generates matching result information that indicates, for example, whether the similarity of the compared characteristics is above a predetermined threshold. The matching process described here is just one example and is not limited to it.

[0209] The matching unit 632 outputs the result of the matching performed in step S602 (step S603). This output may be displayed or transmitted to another device (not shown).

[0210] This authentication process allows the target area to be extracted using the figure type and figure parameters estimated in step S103. Therefore, by including figures corresponding to various states of face shape, iris shape, etc., in the figure type, the target area can be extracted accurately and easily. Consequently, accurate matching becomes possible.

[0211] Various states include, for example, if the subject is a face, a state where the image contains multiple people and the faces of those multiple people overlap. Various states also include, for example, if the subject is an iris, a state where it is blinking, a state where the eyes are wide open, etc.

[0212] (Effects / Actions) As described above, according to this embodiment, the information processing system 600 further comprises a region extraction unit 631 and a matching unit 632. The region extraction unit 631 extracts a target region from the image, which is the image region in which the target is depicted, based on the estimated figure type and figure parameters. The matching unit 632 performs a matching process for authentication using the extracted target region. The target is either a face or an iris.

[0213] As a result, as described above, the target area can be extracted accurately and easily. Therefore, accurate matching becomes possible.

[0214] According to this embodiment, the image includes multiple people whose faces overlap.

[0215] Even with images like these, as described above, the target area, which is the face, can be extracted accurately and easily. Therefore, it becomes possible to perform accurate matching for facial recognition.

[0216] (Variation 1) Figure 26 shows an example configuration of the information processing system 600 according to Modification 1. The information processing system 600 according to Modification 1 includes an authentication device 603 in addition to the image storage device 101 and information processing device 102 similar to those in Embodiment 1. The authentication device 603 includes a region extraction unit 631 and a matching unit 632 similar to those in Embodiment 6.

[0217] In other words, in this modified example, an authentication device 603, separate from the information processing device 102, is equipped with the function of performing authentication processing. This also produces the same effects as in Embodiment 6.

[0218] The embodiments and modifications of the present invention have been described above with reference to the drawings, but these are merely examples of the present invention, and various other configurations can also be adopted.

[0219] Furthermore, while the flowcharts used in the above description show multiple steps (processes) in sequence, the execution order of the steps performed in each embodiment is not limited to the order in which they are described. In each embodiment, the order of the illustrated steps can be changed to the extent that it does not impede the content. Also, the above embodiments and modifications can be combined to the extent that their content is not contradictory.

[0220] Some or all of the above embodiments may also be described as follows, but are not limited to the following:

[0221] 1. A feature extraction means for extracting feature quantities from objects shown in an image, The system includes estimation means for estimating the type of object, the graphic type corresponding to the object, and the graphic parameters using the extracted features. Information processing system. 2. The estimation means is, A target estimation means for estimating the type of target using the extracted features, Includes a shape estimation means for estimating the shape type and shape parameters corresponding to the object based on the estimated type of object or the extracted feature quantities. The information processing system described in 1. 3. The aforementioned shape estimation means is A shape type estimation means for estimating one or more shape types corresponding to the object based on the estimated type of object or the extracted feature quantities, Includes parameter estimation means for estimating the geometric parameters corresponding to the target for one or more of the estimated geometric types. The information processing system described in 2. 4. The graphic type estimation means estimates the one or more graphic types corresponding to the object based on the estimated type of object and the correspondence information that associates the type of object with the one or more graphic types. The information processing system described in 3. 5. The shape type estimation means uses the extracted feature quantities as input and a shape type estimation model trained to estimate at least one shape type corresponding to an object in the image to estimate one or more shape types corresponding to the object. The information processing system described in 3. 6. The parameter estimation means is: For each of the first to n predetermined geometric types associated with a geometric type, the first to n geometric type estimation means estimates the geometric parameters corresponding to the target, Includes a selection means for selecting a figure type estimation means to be used to estimate the figure parameters corresponding to the target, based on the estimated one or more figure types. An information processing system described in any one of items 3 through 5. 7. The figure estimation means further includes, if there are multiple estimated figure types, a determination means for determining the figure type and figure parameters corresponding to the target, based on the estimated figure parameters for each of the multiple figure types and a fit score indicating the degree to which the figure fits the outer edge of the target. An information processing system described in any one of items 3 through 6. 8. The shape estimation means uses the extracted feature quantities as input and a shape estimation model trained to estimate the general shape of an object in the image to estimate the shape type and shape parameters corresponding to the object. The information processing system described in 2. 9. The shape estimation model is a machine learning model that takes the feature quantities of an object contained in a training image as input and outputs the shape type and shape parameters that represent the general shape of the object in such a way that the fit score, which indicates the degree to which the shape fits the outer edge of the object, is high. The information processing system described in 8. 10. The aforementioned fit score is a value determined according to a general shape estimation criterion defined using the areas of at least two of the following regions: (A) the first region which is the area where the figure and the object overlap; (B) the second region which is the area within the object that extends beyond the figure; and (C) the third region which is the area within the figure that extends beyond the object. The information processing system described in 7. or 9. 11. The geometric type includes at least one of a polygon, a circle, an ellipse, a curve, a closed curve, or a straight line. The aforementioned geometric parameter includes at least one of the size and rotation angle of the geometric shape indicated by the geometric type, and its position in the image. An information processing system described in any one of items 1 through 10. 12. The geometric parameters are represented by fixed-length vectors common to the geometric type. The information processing system described in 11. 13. Region extraction means for extracting a target region from the image that contains the object, based on the estimated shape type and shape parameters, The system further comprises matching means that perform matching processing for authentication using the extracted target area, The subject is the face or the iris. An information processing system described in any one of items 1 through 12. 14. The aforementioned image includes multiple people whose faces overlap. The information processing system described in 13. 15. Further comprising an image storage device for storing the aforementioned image. An information processing system described in any one of items 1 through 14. 16. A feature extraction means for extracting feature quantities of objects shown in an image, The system includes estimation means for estimating the type of object, the graphic type corresponding to the object, and the graphic parameters using the extracted features. Information processing device. 17. At least one computer, Extract the feature quantities of the objects shown in the image, This includes estimating the type of object, the corresponding graphic type, and the graphic parameters using the extracted features. Information processing methods. 18. By estimating the type of the object, the graphic type corresponding to the object, and the graphic parameters, Using the extracted features, the type of the object is estimated. Based on the estimated type of target or the extracted features, the corresponding graphic type and graphic parameters are estimated. The information processing method described in 17. 19. By estimating the graphic type and the graphic parameters, Based on the estimated type of object or the extracted feature quantities, one or more graphic types corresponding to the object are estimated. For the one or more estimated geometric types, estimate the geometric parameters corresponding to the target. The information processing method described in 18. 20. Estimating the graphic type involves estimating the one or more graphic types corresponding to the object based on the estimated type of object and the correspondence information that associates the type of object with one or more graphic types. The information processing method described in 19. 21. Estimating the shape type involves using a shape type estimation model that has been trained to estimate at least one shape type corresponding to an object in the image, taking the extracted features as input, to estimate one or more shape types corresponding to the object. The information processing method described in 19. 22. By estimating the aforementioned geometric parameters, Using the first to nth geometric type estimation means, which are associated with each of the first to nth predetermined geometric types, the geometric parameters corresponding to the target are estimated. Based on the estimated one or more geometric types, a geometric type-specific estimation means is selected to be used to estimate the geometric parameters corresponding to the target. An information processing method described in any one of items 19 to 21. 23. In estimating the graphic type and the graphic parameters, if there are multiple estimated graphic types, the graphic type and the graphic parameters corresponding to the object are determined based on the estimated graphic parameters for each of the multiple graphic types and a fit score indicating the degree to which the graphic fits the outer edge of the object. An information processing method described in any one of items 19 to 22. 24. In estimating the shape type and the shape parameters, the shape type and the shape parameters corresponding to the object are estimated using a shape estimation model that has been trained to estimate the general shape of an object in an image, with the extracted features as input. The information processing method described in 18. 25. The aforementioned shape estimation model is a machine learning model that takes the feature quantities of an object contained in a training image as input and outputs the shape type and shape parameters that represent the general shape of the object in such a way that the fit score, which indicates the degree to which the shape fits the outer edge of the object, is high. The information processing method described in 24. 26. The aforementioned fit score is a value determined according to a general shape estimation criterion defined using the areas of at least two of the following regions: (A) the first region which is the area where the figure and the object overlap; (B) the second region which is the area within the object that extends beyond the figure; and (C) the third region which is the area within the figure that extends beyond the object. The information processing method described in 23. or 25. 27. The aforementioned geometric type includes at least one of a polygon, a circle, an ellipse, a curve, a closed curve, or a straight line. The aforementioned geometric parameter includes at least one of the size and rotation angle of the geometric shape indicated by the geometric type, and its position in the image. An information processing method described in any one of items 17 through 26. 28. The geometric parameters are represented by fixed-length vectors common to the geometric type. The information processing method described in 27. 29. Based on the estimated shape type and shape parameters, extract the target region from the image, which is the image region in which the object is depicted. The process further includes performing a matching process for authentication using the extracted target area, The subject is the face or the iris. An information processing method described in any one of items 17 to 28. 30. The aforementioned image includes multiple people whose faces overlap. The information processing method described in 29. 31. On at least one computer, Extract the feature quantities of the objects shown in the image, A recording medium on which a program is stored that causes the system to perform the estimation of the type of object, the graphic type corresponding to the object, and the graphic parameters, using the extracted features. 32. By estimating the type of the object, the graphic type corresponding to the object, and the graphic parameters, Using the extracted features, the type of the object is estimated. Based on the estimated type of target or the extracted features, the corresponding graphic type and graphic parameters are estimated. The recording medium described in 31. 33. By estimating the graphic type and the graphic parameters, Based on the estimated type of object or the extracted feature quantities, one or more graphic types corresponding to the object are estimated. For the one or more estimated geometric types, estimate the geometric parameters corresponding to the target. The recording medium described in 32. 34. Estimating the graphic type involves estimating the one or more graphic types corresponding to the object based on the estimated type of object and the correspondence information that associates the type of object with one or more graphic types. The recording medium described in 33. 35. Estimating the shape type involves using a shape type estimation model that has been trained to estimate at least one shape type corresponding to an object in the image, taking the extracted features as input, to estimate one or more shape types corresponding to the object. The recording medium described in 33. 36. By estimating the aforementioned geometric parameters, Using the first to nth geometric type estimation means, which are associated with each of the first to nth predetermined geometric types, the geometric parameters corresponding to the target are estimated. Based on the estimated one or more geometric types, a geometric type-specific estimation means is selected to be used to estimate the geometric parameters corresponding to the target. A recording medium described in any one of items 33 to 35. 37. In estimating the graphic type and the graphic parameters, if there are multiple estimated graphic types, the graphic type and the graphic parameters corresponding to the object are determined based on the estimated graphic parameters for each of the multiple graphic types and a fit score indicating the degree to which the graphic fits the outer edge of the object. The recording medium according to any one of 33. to 36. 38. In estimating the graphic type and the graphic parameter, the graphic type and the graphic parameter corresponding to the object are estimated by using a graphic estimation model that has been trained to estimate the rough shape of an object appearing in an image, with the extracted feature quantity as an input. The recording medium according to 32. 39. The graphic estimation model is a machine learning model trained to output the graphic type indicating the rough shape of an object and the graphic parameter by receiving a feature quantity of the object included in a training image as an input, such that a matching score indicating a degree to which a graphic matches the outer edge of the object becomes higher. The recording medium according to 38. 40. The matching score is a value obtained in accordance with a rough shape estimation criterion defined by using areas of at least two of the following: (A) a first area which is an area where the graphic and the object overlap each other, (B) a second area which is an area within the object that protrudes from the graphic, and (C) a third area which is an area within the graphic that protrudes from the object. The recording medium according to 37. or 39. 41. The graphic type includes at least one of a polygon, a circle, an ellipse, a curve, a closed curve, and a straight line, the graphic parameter includes at least one of a size and a rotation angle of the graphic indicated by the graphic type, and a position in the image. The recording medium according to any one of 31. to 40. 42. The graphic parameter is represented by a fixed-length vector common to the graphic types. The recording medium according to 41. 43. Extracting an object area, which is an image area where the object appears, from the image based on the estimated graphic type and graphic parameter, further causing the computer to perform matching processing for authentication by using the extracted object area, and the object is a face or an iris. The recording medium according to any one of 31. to 42. 44. The image includes a plurality of people whose faces overlap each other. The recording medium according to 43. Description of Reference Signs

[0222] 100, 200, 300, 500, 600 Information processing system 101 Image storage device 102, 202, 302, 502, 602 Information processing apparatus 111 Image acquisition unit 112 Feature extraction unit 113, 213, 313, 513 Estimation unit 114 Output unit 221 Object estimation unit 222, 322, 522 Figure estimation unit 231 Association information storage unit 231a Association information 232, 332 Figure type estimation unit 233 Parameter estimation unit 233A Estimation unit for each figure type 233B Selection unit 234 Determination unit 603 Authentication apparatus 631 Region extraction unit 632 Matching unit

Claims

1. A feature extraction means for extracting feature quantities from objects shown in an image, The system includes estimation means for estimating the type of object, the shape type, and the shape parameters corresponding to the object, using the extracted features. The aforementioned geometric parameters are represented by fixed-length vectors of a common size across all predetermined geometric types, and predetermined values, including blanks or null values, are stored in the elements of the fixed-length vector that are not used in the estimated geometric type. Information processing system.

2. The estimation means is, A target estimation means for estimating the type of the target using the extracted features, Includes a shape estimation means for estimating the shape type and shape parameters corresponding to the object based on the estimated type of object or the extracted feature quantities. The information processing system according to claim 1.

3. The aforementioned shape estimation means is A shape type estimation means for estimating one or more shape types corresponding to the object based on the estimated type of object or the extracted feature quantity, Includes parameter estimation means for estimating the geometric parameters corresponding to the target for one or more of the estimated geometric types. The information processing system according to claim 2.

4. The graphic type estimation means estimates the one or more graphic types corresponding to the object based on the estimated type of object and the correspondence information that associates the type of object with the one or more graphic types. The information processing system according to claim 3.

5. The shape type estimation means uses the extracted feature quantities as input and a shape type estimation model trained to estimate at least one shape type corresponding to an object in the image to estimate one or more shape types corresponding to the object. The information processing system according to claim 3.

6. The parameter estimation means is For each of the first to n predetermined figure types associated with a figure type, the first to n figure type estimation means estimates the figure parameters corresponding to the target, Includes a selection means for selecting a figure type estimation means to be used to estimate the figure parameters corresponding to the target, based on the estimated one or more figure types. The information processing system according to any one of claims 3 to 5.

7. The shape estimation means further includes, if there are multiple estimated shape types, a determination means for determining the shape type and the shape parameters corresponding to the target, based on the estimated shape parameters for each of the multiple shape types and a fit score indicating the degree to which the shape fits the outer edge of the target. The information processing system according to any one of claims 3 to 5.

8. The shape estimation means uses the extracted features as input and a shape estimation model trained to estimate the general shape of an object in the image to estimate the shape type and shape parameters corresponding to the object. The information processing system according to any one of claims 2 to 5.

9. At least one computer, Extract the feature quantities of the objects shown in the image, This includes estimating the type of object, the corresponding graphic type, and the graphic parameters using the extracted features, The aforementioned geometric parameters are represented by fixed-length vectors of a common size across all predetermined geometric types, and predetermined values, including blanks or null values, are stored in the elements of the fixed-length vector that are not used in the estimated geometric type. Information processing methods.

10. On at least one computer, Extract the feature quantities of the objects shown in the image, Using the extracted features, the system is made to estimate the type of object, the corresponding geometric type, and the geometric parameters. The aforementioned geometric parameters are represented by fixed-length vectors of a common size across all predetermined geometric types, and predetermined values, including blanks or null values, are stored in the elements of the fixed-length vector that are not used in the estimated geometric type. program.

Images