IMAGE GENERATION SYSTEM, IMAGE GENERATION METHOD, AND RECORDING MEDIUM

The image generation system addresses detection errors by generating new images based on perturbed positional information, improving face verification and augmentation processes.

JP7593407B2Active Publication Date: 2024-12-03NEC CORP
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2022551527
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-09-25
Publication Date
2024-12-03
Estimated Expiration
2040-09-25

AI Technical Summary

Technical Problem

Existing image generation systems face challenges in accurately detecting and normalizing facial features due to errors in positional information, leading to suboptimal image generation and verification processes.

Method used

An image generation system that includes a detection unit for facial feature points, an acquisition unit to add errors to positional information, and a generation unit to create new images based on perturbed position information, addressing detection errors and enhancing image normalization.

Benefits of technology

The system generates new images that account for positional changes due to detection errors, improving robustness and accuracy in face verification and augmentation of training data, while enhancing adversarial learning and montage creation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007593407000001
    Figure 0007593407000001
  • Figure 0007593407000002
    Figure 0007593407000002
  • Figure 0007593407000003
    Figure 0007593407000003
Patent Text Reader

Abstract

An image generation system (10) that comprises a detection means (110) that detects position information about a face or about the positions of feature points of the face from an image, an acquisition means (120) that acquires perturbed position information that is generated by adding the error that occurs when the position information is detected to the position information, and a generation means (130) that generates a new image that includes the face on the basis of the perturbed position information. The image generation system makes it possible to take position estimation error into account to appropriately generate a new image.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to the technical fields of an image generating system for generating an image, an image generating method, and a recording medium. [Background technology]

[0002] As this type of system, a system that generates a face image based on facial feature points is known. For example, Patent Document 1 discloses a technology that normalizes a face so that the positions of both eyes are located at predetermined reference positions, and cuts out an image including the normalized face.

[0003] As other related techniques, for example, Patent Document 2 discloses a technique for training a neural network using features extracted from training data as adversarial features. Patent Document 3 discloses a technique for identifying a person in a face image based on a registered face template and probability distribution sample data. Patent Document 4 discloses a technique for generating a perturbed face image by rotating a face image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2007-226424 A [Patent Document 2] International Publication No. 2018 / 167900 [Patent Document 3] JP 2005-208850 A [Patent Document 4] JP 2017-182459 A Summary of the Invention [Problem to be solved by the invention]

[0005] This disclosure improves upon the related art discussed above. [Means for solving the problem]

[0006] One aspect of the image generation system of this disclosure includes a detection means for detecting position information regarding the position of a face or a feature point of the face from an image, an acquisition means for acquiring perturbed position information that takes into account an error that occurs when detecting the position information, and a generation means for generating a new image including the face based on the perturbed position information.

[0007] One aspect of the image generation method disclosed herein detects position information regarding the position of a face or a feature point of the face from an image, obtains perturbed position information that takes into account an error that occurs when detecting the position information, and generates a new image including the face based on the perturbed position information.

[0008] One aspect of the recording medium of this disclosure is a computer program recorded on the recording medium to cause a computer to execute an image generation method that detects position information regarding the position of a face or a feature point of the face from an image, obtains perturbed position information that takes into account an error that occurs when detecting the position information, and generates a new image including the face based on the perturbed position information. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a hardware configuration of an image generating system according to a first embodiment. [Diagram 2] 1 is a block diagram showing a functional configuration of an image generating system according to a first embodiment. [Diagram 3] 4 is a flowchart showing the flow of operations of the image generating system according to the first embodiment. [Figure 4] 10 is a flowchart showing the flow of operations of the image generating system according to the second embodiment. [Diagram 5] FIG. 11 is a block diagram showing the functional configuration of an image generating system according to a third embodiment. [Figure 6] 13 is a flowchart showing the flow of operations of the image generating system according to the third embodiment. [Figure 7]FIG. 13 is a block diagram showing the functional configuration of an image generating system according to a fourth embodiment. [Figure 8] 13 is a flowchart showing the flow of operations of an image generating system according to a fourth embodiment. [Figure 9] FIG. 13 is a block diagram showing the functional configuration of an image generating system according to a fifth embodiment. [Figure 10] 13 is a flowchart showing the flow of operations of the image generating system according to the fifth embodiment. [Figure 11] FIG. 13 is a block diagram showing the functional configuration of an image generating system according to a sixth embodiment. [Figure 12] 13 is a flowchart showing the flow of operations of the image generating system according to the sixth embodiment. [Figure 13] FIG. 13 is a block diagram showing the functional configuration of an image generating system according to a seventh embodiment. [Figure 14] 13 is a flowchart showing the flow of operations of the image generating system according to the seventh embodiment. [Figure 15] FIG. 23 is a block diagram showing a functional configuration of an image generating system according to a modified example of the seventh embodiment. [Figure 16] 23 is a flowchart showing the flow of operations of an image generating system according to a modified example of the seventh embodiment. [Figure 17] FIG. 13 is a block diagram showing the functional configuration of an image generating system according to an eighth embodiment. [Figure 18] 13 is a flowchart showing the flow of operations of the image generating system according to the eighth embodiment. [Figure 19] FIG. 23 is a block diagram showing a functional configuration of an image generating system according to a modified example of the eighth embodiment. [Figure 20] 13 is a flowchart showing the flow of operations of an image generating system according to a modified example of the eighth embodiment. [Figure 21] FIG. 13 is a block diagram showing the functional configuration of an image generating system according to a ninth embodiment. [Figure 22] 13 is a flowchart showing the flow of operations of the image generating system according to the ninth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of an image generating system, an image generating method, and a recording medium will be described with reference to the drawings.

[0011] First Embodiment An image generating system according to a first embodiment will be described with reference to FIGS. 1 to 3. FIG.

[0012] (Hardware configuration) First, the hardware configuration of the image generation system according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the image generation system according to the first embodiment.

[0013] 1, an image generation system 10 according to the first embodiment includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14. The image generation system 10 may further include an input device 15 and an output device 16. The processor 11, the RAM 12, the ROM 13, the storage device 14, the input device 15, and the output device 16 are connected via a data bus 17.

[0014] The processor 11 reads a computer program. For example, the processor 11 is configured to read a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may read a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The processor 11 may acquire (i.e., read) a computer program from a device (not shown) disposed outside the image generating system 10 via a network interface. The processor 11 controls the RAM 12, the storage device 14, the input device 15, and the output device 16 by executing the computer program that the processor 11 reads. In particular, in this embodiment, when the processor 11 executes the computer program that the processor 11 reads, a functional block for generating a new image from an input image is realized in the processor 11. In addition, the processor 11 may be one of a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a demand-side platform (DSP), and an application specific integrated circuit (ASIC), or may be a plurality of devices in parallel.

[0015] The RAM 12 temporarily stores a computer program executed by the processor 11. The RAM 12 temporarily stores data that is temporarily used by the processor 11 when the processor 11 is executing the computer program. The RAM 12 may be, for example, a D-RAM (Dynamic RAM).

[0016] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store other fixed data. The ROM 13 may be, for example, a programmable ROM (P-ROM).

[0017] The storage device 14 stores data that is to be stored long-term by the image generating system 10. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0018] The input device 15 is a device that receives input instructions from a user of the image generation system 10. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel.

[0019] The output device 16 is a device that outputs information related to the image generation system 10 to the outside. For example, the output device 16 may be a display device (for example, a display) that can display information related to the image generation system 10.

[0020] (Functional configuration) Next, the functional configuration of the image generation system 10 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the image generation system according to the first embodiment.

[0021] 2, the image generating system 10 according to the first embodiment includes a detection unit 110, an acquisition unit 120, and a generation unit 130 as processing blocks or physical processing circuits for realizing the functions of the system. Each of the detection unit 110, the acquisition unit 120, and the generation unit 130 may be realized by, for example, the above-mentioned processor 11 (see FIG. 1).

[0022] The detection unit 110 is configured to be able to detect position information relating to the position of a face or facial feature points from an input image. A facial feature point is a point that represents a facial feature, and is set as corresponding to a specific part such as the eyes, ears, nose, and mouth. Which part is to be set as a feature point may be appropriately set by, for example, a system administrator or the like. The detection unit 110 may detect both position information relating to the position of the face and position information relating to the positions of the facial feature points. In this case, the detection unit 110 may be configured to separately include one that detects position information relating to the position of the face and one that detects position information relating to the positions of the facial feature points (for example, it may be configured as two independent detection units 110). Furthermore, the detection unit 110 may be configured to first detect position information relating to the position of the face, and then detect position information relating to the positions of the facial feature points based on the detected position information relating to the position of the face. The position information may be, for example, coordinate information or vector information. The detection unit 110 may be configured as, for example, a neural network. A specific method for detecting the position information by the detection unit 110 is not described here in detail because existing technology can be appropriately adopted. The position information detected by the detection unit 110 is output to the acquisition unit 120.

[0023] The acquisition unit 120 is configured to be able to acquire perturbed position information that takes into account an error in the position information detected by the detection unit 110. The "error" here refers to an error (i.e., a detection error) that occurs when the detection unit 110 detects the position information. The perturbed position information is information that has a perturbation according to the error, compared with the position information detected by the detection unit 110. When taking into account an error for a plurality of feature points, the error may be taken into account for all feature points, or the error may be taken into account only for some feature points. When taking into account an error only for some feature points, the feature points for which the error is taken into account may be automatically determined from a past history or the like, or may be specified by a user. A specific method for taking into account an error in the position information will be described in detail in another embodiment described later. The method for taking into account an error may be common to all feature points, or a different method may be adopted for each feature point. The perturbed position information acquired by the acquisition unit 120 is configured to be output to the generation unit 130.

[0024] The generating unit 130 is configured to be able to generate a new image including a face based on the perturbation position information acquired by the acquiring unit 120. The new image generated by the generating unit 130 is generated based on the perturbation position information, and therefore is an image having perturbation according to the error. Therefore, a difference according to the error occurs between the image input to the detecting unit 110 and the new image generated by the generating unit 130. Note that a specific method for generating an image from position information can be appropriately adopted from existing technologies, and therefore a detailed description thereof will be omitted here. The generating unit 130 has a function of outputting the generated new image. The generating unit 130 may be configured to output the generated new image to a display unit having a display, for example, and display the generated new image.

[0025] (Operation flow) Next, the flow of operations of the image generating system 10 according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of operations of the image generating system according to the first embodiment.

[0026] 3, when the operation of the image generating system 10 according to the first embodiment is started, first, an image is input to the detection unit 110 (step S11). When the image is input, the detection unit 110 detects position information related to the positions of the face or facial feature points from the image (step S12).

[0027] Next, the acquisition unit 120 acquires perturbed position information that adds an error to the position information (step S13). Then, the generation unit 130 generates a new image including a face based on the perturbed position information (step S14). The generation unit 130 outputs the generated new image (step S15).

[0028] (Technical effect) Next, technical effects obtained by the image generation system 10 according to the first embodiment will be described.

[0029] 1 to 3, in the image generating system 10 according to the first embodiment, a new face image is generated based on perturbed position information that takes into account an error during detection. Therefore, it is possible to generate a new face image including a positional variation according to the error from an input image.

[0030] <Second embodiment> An image generating system 10 according to the second embodiment will be described with reference to Fig. 4. Note that the second embodiment differs from the first embodiment only in some of the operations, and the system configuration and the like may be similar to the first embodiment (see Figs. 1 and 2). Therefore, hereinafter, the parts that differ from the first embodiment will be described in detail, and the explanation of the other overlapping parts will be omitted as appropriate.

[0031] (Operation flow) First, the flow of operations of the image generating system 10 according to the second embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of operations of the image generating system according to the second embodiment. In Fig. 4, the same processes as those shown in Fig. 3 are denoted by the same reference numerals.

[0032] 4, when the operation of the image generating system 10 according to the first embodiment is started, first, an image is input to the detection unit 110 (step S11). When the image is input, the detection unit 110 detects position information related to the positions of the face or facial feature points from the image (step S12).

[0033] Next, the acquisition unit 120 acquires perturbed position information that adds an error to the position information (step S13). Then, the generation unit 130 executes a process of normalizing the face based on the perturbed position information (step S21). The generation unit 130 outputs a face normalized image, which is an image including the normalized face, as a new image (step S22).

[0034] (Face Normalization) Next, the face normalization process executed by the image generating system 10 according to the second embodiment will be described in more detail.

[0035] Face normalization is achieved by adjusting at least one of the position, size, and angle of the face based on position information. Face normalization is achieved by appropriately adjusting the position, size, and angle of the face so that the positions of facial feature points such as the eyes, nose, and mouth are at predetermined positions. For face normalization, for example, image processing techniques such as image enlargement / reduction, rotation, and 2D / 3D conversion may be used. For face normalization, existing techniques not listed here may also be appropriately adopted.

[0036] (Technical effect) Next, technical effects obtained by the image generation system 10 according to the second embodiment will be described.

[0037] As described in Fig. 4, in the image generating system 10 according to the second embodiment, a face normalized image in which the face is normalized is generated as a new image. Here, in this embodiment, in particular, face normalization is performed based on perturbation position information. This makes it possible to generate a normalized image including positional fluctuations according to errors.

[0038] In the following embodiments, similarly to the second embodiment, the description will be given taking as an example a configuration in which the generation unit 130 generates a normalized face image.

[0039] <Third embodiment> An image generating system 10 according to the third embodiment will be described with reference to Figures 5 and 6. The third embodiment differs from the first and second embodiments only in part of the configuration and operation, and other parts may be similar to the first and second embodiments. Therefore, hereinafter, parts that differ from the first and second embodiments will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0040] (Functional configuration) First, the functional configuration of an image generation system 10 according to the third embodiment will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the functional configuration of the image generation system according to the third embodiment. In Fig. 5, the same components as those shown in Fig. 2 are denoted by the same reference numerals.

[0041] As shown in Fig. 5, the image generating system 10 according to the third embodiment includes a detection unit 110, an acquisition unit 120, a generation unit 130, and a perturbation amount generation unit 140 as processing blocks or physical processing circuits for realizing the functions of the image generating system 10. That is, the image generating system 10 according to the third embodiment is configured to further include a perturbation amount generation unit 140 in addition to the configuration of the first embodiment (see Fig. 2). The perturbation amount generation unit 140 may be realized by the above-mentioned processor 11 (see Fig. 1).

[0042] The perturbation amount generation unit 140 is configured to be able to generate a perturbation amount according to an error parameter (i.e., a value indicating the magnitude of the error). The perturbation amount generation unit 140 may generate a perturbation amount by, for example, multiplying the error parameter by a predetermined random number. In this case, the random number may be generated using a random number generator based on a normal distribution (for example, a normal distribution with a mean of 0 and a variance of 1). The error parameter may be a value specified by a user. The perturbation amount generated by the perturbation amount generation unit 140 is configured to be output to the perturbation addition unit 121 of the acquisition unit 120.

[0043] The perturbation addition unit 121 is configured to be capable of executing a process of adding a perturbation amount to a parameter (i.e., position information) indicating the position of the face or facial feature amount detected by the detection unit 110. The perturbation addition unit 121 may execute a process of adding the perturbation amount as it is, or may execute a process of multiplying the perturbation amount by a predetermined coefficient and then adding it. The acquisition unit 120 acquires the result of the addition process by the perturbation addition unit 121 as perturbation position information.

[0044] (Operation flow) Next, the flow of operations of the image generating system 10 according to the third embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the flow of operations of the image generating system according to the third embodiment. In Fig. 6, the same processes as those shown in Figs. 3 and 4 are denoted by the same reference numerals.

[0045] 6, when the operation of the image generating system 10 according to the third embodiment is started, first, an image is input to the detection unit 110 (step S11). When the image is input, the detection unit 110 detects position information related to the positions of the face or facial feature points from the image (step S12).

[0046] Next, the perturbation amount generating unit 140 generates a random number for generating a perturbation amount (step S31). Then, the perturbation amount generating unit 140 generates a perturbation amount from the generated random number and an error parameter (step S32).

[0047] Next, the perturbation adding unit 121 adds the perturbation amount to the position information detected by the detecting unit 110 to obtain perturbation position information (step S33). Then, the generating unit 130 executes a process of normalizing the face based on the perturbation position information (step S21). The generating unit 130 outputs a face normalized image, which is an image including the normalized face, as a new image (step S22).

[0048] (Technical effect) Next, technical effects obtained by the image generation system 10 according to the third embodiment will be described.

[0049] As described in Fig. 5 and Fig. 6, in the image generating system 10 according to the third embodiment, a perturbation amount is generated based on an error parameter. Then, the perturbation amount is added to the detected position information to obtain perturbation position information. In this way, it is possible to properly obtain perturbation position information that takes the error into account. Therefore, it is possible to properly generate a new face image that includes a positional variation according to the error from an input image.

[0050] <Fourth embodiment> An image generating system 10 according to the fourth embodiment will be described with reference to Figures 7 and 8. The fourth embodiment differs from the first to third embodiments in part of its configuration and operation, and other parts may be similar to the first to third embodiments. Therefore, hereinafter, parts that differ from the first to third embodiments will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0051] (Functional configuration) First, the functional configuration of an image generation system 10 according to a fourth embodiment will be described with reference to Fig. 7. Fig. 7 is a block diagram showing the functional configuration of an image generation system according to a fourth embodiment. In Fig. 7, the same components as those shown in Figs. 2 and 5 are denoted by the same reference numerals.

[0052] As shown in Fig. 7, the image generating system 10 according to the fourth embodiment includes a detection unit 110, an acquisition unit 120, a generation unit 130, a perturbation generation unit 140, and an error evaluation unit 150 as processing blocks or physical processing circuits for realizing the functions of the image generating system 10. That is, the image generating system 10 according to the fourth embodiment is configured to further include an error evaluation unit 150 in addition to the configuration of the third embodiment (see Fig. 5). The error evaluation unit 150 may be realized by the above-mentioned processor 11 (see Fig. 1).

[0053] The error evaluation unit 150 is configured to be able to evaluate an error using test data for error evaluation (i.e., to estimate an error occurring when detecting location information). The test data is data having correct answer data regarding location information. The error evaluation unit 150 may evaluate the error using a statistical method based on the deviation between the location information detected from the test data and the correct answer data. As a specific error evaluation method, existing technology can be appropriately adopted, and one example is the Mean Absolute Error (MAE). The evaluation result of the error evaluation unit 150 is configured to be output to the perturbation amount generation unit 140 as an error parameter. That is, in this embodiment, a perturbation amount is generated based on the error parameter evaluated by the error evaluation unit 150.

[0054] (Operation flow) Next, the flow of operations of the image generating system 10 according to the fourth embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the flow of operations of the image generating system according to the fourth embodiment. In Fig. 8, the same processes as those shown in Figs. 3, 4, and 6 are denoted by the same reference numerals.

[0055] 8, when the operation of the image generating system 10 according to the fourth embodiment is started, first, an image is input to the detection unit 110 (step S11). When the image is input, the detection unit 110 detects position information related to the positions of the face or facial feature points from the image (step S12).

[0056] Next, the perturbation amount generation unit 140 generates random numbers for generating perturbations (step S31). Then, the perturbation amount generation unit 140 generates perturbations from the generated random numbers and the error parameters evaluated by the error evaluation unit 150 (step S41). Note that the error evaluation by the error evaluation unit 150 may be performed separately before the series of processes shown in Fig. 8 is started. However, the error evaluation by the error evaluation unit 150 may be performed after the series of processes shown in Fig. 8 is started and before the process of step S41 is executed.

[0057] Next, the perturbation adding unit 121 adds the perturbation amount to the position information detected by the detecting unit 110 to obtain perturbation position information (step S33). Then, the generating unit 130 executes a process of normalizing the face based on the perturbation position information (step S21). The generating unit 130 outputs a face normalized image, which is an image including the normalized face, as a new image (step S22).

[0058] (Technical effect) Next, technical effects obtained by the image generation system 10 according to the fourth embodiment will be described.

[0059] As described in Fig. 7 and Fig. 8, in the image generating system 10 according to the fourth embodiment, a perturbation amount is generated based on an error parameter that is a result of error evaluation using test data. Then, the perturbation amount is added to the detected position information to obtain perturbation position information. In this way, it is possible to properly obtain perturbation position information that takes into account the error. Therefore, it is possible to properly generate a new face image that includes a positional variation according to the error from an input image.

[0060] The third and fourth embodiments described above can be combined with each other. For example, both the error parameter designated by the user and the error parameter output from the error evaluation unit 150 may be integrated to generate an integrated error parameter, and a perturbation amount may be generated from the integrated error parameter. The integrated error parameter may be, for example, an average value of each error parameter. In addition, the error parameter designated by the user and the error parameter output from the error evaluation unit 150 may be selectively used. That is, the perturbation position information may be acquired by adding one selected from the perturbation amount generated from the error parameter designated by the user and the perturbation amount generated from the error parameter output from the error evaluation unit 150. In this case, the system may automatically select the perturbation amount, or the user may select the perturbation amount.

[0061] <Fifth embodiment> An image generating system 10 according to the fifth embodiment will be described with reference to Figures 9 and 11. The fifth embodiment differs from the first to fourth embodiments described above only in part of the configuration and operation, and other parts may be similar to the first to fourth embodiments. Therefore, hereinafter, parts that differ from the first to fourth embodiments will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0062] (Functional configuration) First, the functional configuration of an image generation system 10 according to the fifth embodiment will be described with reference to Fig. 9. Fig. 9 is a block diagram showing the functional configuration of the image generation system according to the fifth embodiment. In Fig. 9, the same components as those shown in Figs. 5 and 7 are denoted by the same reference numerals.

[0063] As shown in FIG. 9, the image generating system 10 according to the fifth embodiment includes a detection unit 110, an acquisition unit 120, a generation unit 130, and a perturbation amount generation unit 140 as processing blocks or physical processing circuits for realizing the functions. That is, the image generating system 10 according to the fifth embodiment is configured to include elements similar to those of the third embodiment (see FIG. 5). However, the perturbation amount generation unit 140 according to the fifth embodiment is configured to input a deviation of a probability distribution. The deviation of the probability distribution may be, for example, a deviation of a Gaussian distribution. The perturbation amount generation unit 140 is configured to be able to generate a perturbation amount based on the deviation of the input probability distribution. For example, the perturbation amount generation unit 140 may generate a perturbation amount by multiplying the deviation of the probability distribution by a random number.

[0064] (Operation flow) Next, the flow of operations of the image generating system 10 according to the fifth embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the flow of operations of the image generating system according to the fifth embodiment. In Fig. 10, the same processes as those shown in Figs. 6 and 8 are denoted by the same reference numerals.

[0065] 10, when the operation of the image generating system 10 according to the fifth embodiment is started, first, an image is input to the detection unit 110 (step S11). When the image is input, the detection unit 110 detects position information related to the positions of the face or facial feature points from the image (step S12).

[0066] Next, the perturbation amount generating unit 140 generates a random number for generating a perturbation amount (step S31). Then, the perturbation amount generating unit 140 generates a perturbation amount from the deviation between the generated random number and a probability distribution (step S42).

[0067] Next, the perturbation adding unit 121 adds the perturbation amount to the position information detected by the detecting unit 110 (step S33) to obtain perturbation position information. Then, the generating unit 130 executes a process of normalizing the face based on the perturbation position information (step S21). The generating unit 130 outputs a face normalized image, which is an image including the normalized face, as a new image (step S22).

[0068] (Technical effect) Next, technical effects obtained by the image generation system 10 according to the fifth embodiment will be described.

[0069] As described in FIG. 9 and FIG. 10, in the image generating system 10 according to the fifth embodiment, a perturbation amount is generated based on the deviation of the probability distribution. Then, the perturbation amount is added to the detected position information to obtain perturbation position information. In this way, it is possible to properly obtain perturbation position information that takes into account an error. Therefore, it is possible to properly generate a new face image that includes a positional variation according to the error from an input image.

[0070] Sixth embodiment An image generating system 10 according to the sixth embodiment will be described with reference to Fig. 11 and Fig. 12. The sixth embodiment differs from the first to fifth embodiments in part of its configuration and operation, and other parts may be similar to the first to fifth embodiments. Therefore, hereinafter, only the parts that differ from the first to fifth embodiments will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0071] (Functional configuration) First, the functional configuration of an image generation system 10 according to the sixth embodiment will be described with reference to Fig. 11. Fig. 11 is a block diagram showing the functional configuration of the image generation system according to the sixth embodiment. In Fig. 11, the same components as those shown in Fig. 2 are denoted by the same reference numerals.

[0072] As shown in Fig. 11, the image generating system 10 according to the sixth embodiment includes a detection unit 110, an acquisition unit 120, and a generation unit 130 as processing blocks or physical processing circuits for realizing the functions. However, the detection unit 110 according to the sixth embodiment is configured to perform detection n times (n is a natural number) in a predetermined period (within a predetermined time, when a predetermined operation is being performed, from when a predetermined first operation is performed until a predetermined second operation is performed, etc.) and to output n pieces of position information. The n pieces of position information may be output sequentially or n pieces may be output collectively.

[0073] Each of the n pieces of position information detected by the detection unit 110 is configured to be output to the acquisition unit 120. Here, in particular, the acquisition unit 120 according to the sixth embodiment is configured to include a selection unit 122. The selection unit 122 is configured to randomly select one piece of position information from the n pieces of position information detected by the detection unit 110. Since the n pieces of position information are each detected at a different timing, they are pieces of information having errors when viewed as a whole. Therefore, the position information randomly selected from the n pieces of position information can be used as perturbation position information with an error added. The acquisition unit 120 is configured to output the position information selected by the selection unit 122 to the generation unit 130 as perturbation position information.

[0074] (Operation flow) Next, the flow of operations of the image generating system 10 according to the sixth embodiment will be described with reference to Fig. 12. Fig. 12 is a flowchart showing the flow of operations of the image generating system according to the sixth embodiment. In Fig. 12, the same processes as those shown in Figs. 3 and 4 are denoted by the same reference numerals.

[0075] 12, when the operation of the image generating system 10 according to the sixth embodiment is started, first, an image is input to the detection unit 110 (step S11). When the image is input, the detection unit 110 detects position information relating to the positions of the face or facial feature points from the image n times (step S50). As a result, the detection unit 110 detects n pieces of position information.

[0076] Next, the selection unit 122 randomly selects one piece of position information from the n pieces of position information (step S52).Then, the generation unit 130 executes a process of normalizing the face based on the randomly selected position information (i.e., the perturbed position information) (step S21).The generation unit 130 outputs a face normalized image, which is an image including the normalized face, as a new image (step S22).

[0077] (Technical effect) Next, technical effects obtained by the image generation system 10 according to the sixth embodiment will be described.

[0078] As described in FIG. 11 and FIG. 12, in the image generating system 10 according to the sixth embodiment, n pieces of position information are detected by the detection unit 110. Then, position information randomly selected from the n pieces of position information is used as perturbation position information for generating a new image. In this way, it is possible to obtain perturbation position information without directly calculating an error (for example, without adding a perturbation amount as in the third to fifth embodiments). Therefore, it is possible to appropriately generate a new face image including a positional variation according to the error.

[0079] Seventh embodiment An image generating system 10 according to the seventh embodiment will be described with reference to Fig. 13 and Fig. 14. The seventh embodiment differs from the first to sixth embodiments in part of its configuration and operation, and other parts may be similar to the first to sixth embodiments. Therefore, hereinafter, parts that differ from the first to sixth embodiments will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0080] (Functional configuration) First, the functional configuration of an image generation system 10 according to the seventh embodiment will be described with reference to Fig. 13. Fig. 13 is a block diagram showing the functional configuration of the image generation system according to the seventh embodiment. In Fig. 13, the same components as those shown in Figs. 2 and 11 are denoted by the same reference numerals.

[0081] As shown in FIG. 13, the image generation system 10 according to the seventh embodiment includes a plurality of detection units 110, an acquisition unit 120, and a generation unit 130 as processing blocks or physical processing circuits for realizing the functions of the image generation system 10. That is, the image generation system 10 according to the seventh embodiment is different from the configuration of the sixth embodiment (see FIG. 11) in that the image generation system 10 according to the seventh embodiment includes a plurality of independent detection units 110. Although three detection units 110 are illustrated in FIG. 13 for convenience, the image generation system 10 according to the seventh embodiment may include three or more detection units 110. In the following description, the image generation system 10 according to the seventh embodiment will be described as including N (N is a natural number) detection units 110.

[0082] Each of the position information detected by the N detection units 110 described above is configured to be output to the acquisition unit 120. Here, particularly, the acquisition unit 120 according to the seventh embodiment is configured to have a selection unit 122. The selection unit 122 is configured to randomly select one piece of position information from the N pieces of position information detected by the N detection units 110. Note that, when the N detection units 110 perform n-times detection as in the sixth embodiment described above, it is sufficient to randomly select one piece of position information from the N×n pieces of position information. Since the N pieces of position information are each detected by a different detection unit 110, they are information having errors when viewed as a whole. Therefore, the position information randomly selected from the N pieces of position information detected by the different detection units 110 can be used as perturbation position information with an error added. The acquisition unit 120 is configured to output the position information selected by the selection unit 122 to the generation unit 130 as perturbation position information.

[0083] (Operation flow) Next, the flow of operations of the image generating system 10 according to the seventh embodiment will be described with reference to Fig. 14. Fig. 14 is a flowchart showing the flow of operations of the image generating system according to the seventh embodiment. In Fig. 14, the same processes as those shown in Figs. 3 and 4 are denoted by the same reference numerals.

[0084] 14, when the operation of the image generating system 10 according to the seventh embodiment is started, first, an image is input to each of the N detection units 110 (step S11). When the image is input, each of the N detection units 110 detects position information related to the position of a face or facial feature points from the image (step S51). As a result, N pieces of position information are detected.

[0085] Next, the selection unit 122 randomly selects one piece of position information from the N pieces of position information (step S52). Then, the generation unit 130 executes a process of normalizing the face based on the randomly selected position information (i.e., perturbed position information) (step S21). The generation unit 130 outputs a face normalized image, which is an image including a normalized face, as a new image (step S22).

[0086] (Technical effect) Next, technical effects obtained by the image generation system 10 according to the seventh embodiment will be described.

[0087] As described in FIG. 13 and FIG. 14, in the image generating system 10 according to the seventh embodiment, N pieces of position information are detected by N independent detection units 110. Then, position information randomly selected from the N pieces of position information is used as perturbation position information for generating a new image. In this way, it is possible to obtain perturbation position information without directly calculating an error (for example, without adding a perturbation amount as in the third to fifth embodiments). Therefore, it is possible to appropriately generate a new face image including a positional variation according to the error.

[0088] (Modification of the seventh embodiment) Next, an image generation system 10 according to a modified example of the seventh embodiment will be described with reference to Fig. 15 and Fig. 16. Note that the image generation system 10 according to the modified example of the seventh embodiment differs from the seventh embodiment described above only in some operations, and other operations and system configurations may be similar to those of the seventh embodiment. Therefore, hereinafter, the parts that differ from the seventh embodiment already described will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0089] (Functional configuration) First, the functional configuration of an image generation system 10 according to a modified example of the seventh embodiment will be described with reference to Fig. 15. Fig. 15 is a block diagram showing the functional configuration of an image generation system according to a modified example of the fifth embodiment. In Fig. 15, the same components as those shown in Fig. 13 are denoted by the same reference numerals.

[0090] As shown in Fig. 15, the image generation system 10 according to the modification of the seventh embodiment includes, as processing blocks for realizing the functions thereof, a plurality of detection units 110, an acquisition unit 120, a generation unit 130, and a probability distribution estimation unit 160. That is, the image generation system 10 according to the modification of the seventh embodiment is configured to further include a probability distribution estimation unit 160 in addition to the configuration of the seventh embodiment described above (see Fig. 13). The probability distribution estimation unit 160 may be realized by the above-mentioned processor 11 (see Fig. 1).

[0091] The probability distribution estimation unit 160 is configured to be capable of executing a process of fitting the N pieces of position information detected by the N detection units 110 with a predetermined probability distribution. The probability distribution estimation unit 160 may, for example, fit the N pieces of position information with a Gaussian distribution. In this case, the position information may be a value expressed by the formula Gaussian model mean + Gaussian model deviation × random number.

[0092] The result of the fitting process of the probability distribution estimation unit 160 is configured to be output to the acquisition unit 120. Here, in particular, the acquisition unit 120 according to the modified example of the seventh embodiment is configured to have a sampling unit 123. The sampling unit 123 is configured to sample one piece of position information from the probability distribution output as the processing result of the probability distribution estimation unit 160. The position information sampled in this manner can be used as perturbed position information with an error added. The acquisition unit 120 is configured to output the position information sampled by the sampling unit 123 to the generation unit 130 as perturbed position information.

[0093] (Operation flow) Next, the flow of operations of the image generating system 10 according to a modified example of the seventh embodiment will be described with reference to Fig. 16. Fig. 16 is a flowchart showing the flow of operations of the image generating system according to a modified example of the fifth embodiment. In Fig. 16, the same processes as those shown in Fig. 13 are denoted by the same reference numerals.

[0094] 16, when the operation of the image generating system 10 according to the modified example of the seventh embodiment is started, first, an image is input to each of the N detection units 110 (step S11). When the image is input, each of the N detection units 110 detects position information related to the position of a face or facial feature points from the image (step S51). As a result, N pieces of position information are detected.

[0095] Next, the probability distribution estimation unit 160 executes a process of fitting the N pieces of position information with a predetermined probability distribution (step S55). Then, the sampling unit 123 samples one piece of position information from the probability distribution output as a processing result of the probability distribution estimation unit 160 (step S56).

[0096] Next, the generation unit 130 executes a process of normalizing the face based on the sampled position information (i.e., perturbation position information) (step S21). The generation unit 130 outputs a face normalized image, which is an image including the normalized face, as a new image (step S22).

[0097] (Technical effect) Next, technical effects obtained by the image generation system 10 according to the modified example of the seventh embodiment will be described.

[0098] As described in Fig. 15 and Fig. 16, in the image generating system 10 according to the modified example of the seventh embodiment, N pieces of position information are detected by N independent detection units 110. Then, the N pieces of position information are fitted with a probability distribution, and the position information sampled from the distribution is used as perturbed position information for generating a new image. In this way, the perturbed position information can be obtained from the probability distribution estimated from the N pieces of position information. Therefore, it is possible to appropriately generate a new face image including positional fluctuations according to errors.

[0099] Eighth embodiment An image generating system 10 according to the eighth embodiment will be described with reference to Fig. 17 and Fig. 18. The eighth embodiment differs from the first to seventh embodiments in part of its configuration and operation, and other parts may be similar to the first to seventh embodiments. Therefore, hereinafter, parts that differ from the first to seventh embodiments will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0100] (Functional configuration) First, the functional configuration of an image generation system 10 according to the eighth embodiment will be described with reference to Fig. 17. Fig. 17 is a block diagram showing the functional configuration of the image generation system according to the eighth embodiment. In Fig. 17, the same components as those shown in Figs. 2 and 13 are denoted by the same reference numerals.

[0101] As shown in Fig. 17, the image generating system 10 according to the eighth embodiment includes a detection unit 110, an acquisition unit 120, a generation unit 130, and a perturbation image generation unit 170 as processing blocks for realizing the functions of the image generating system 10. That is, the image generating system 10 according to the eighth embodiment is configured to further include a perturbation image generation unit 170 in addition to the configuration of the first embodiment (see Fig. 2). The perturbation image generation unit 170 may be realized by the above-mentioned processor 11 (see Fig. 1).

[0102] The perturbed image generating unit 170 is configured to be able to generate a plurality of perturbed images by applying a perturbation to an input image. The perturbed image generating unit 170 is configured to be able to apply a perturbation by processing such as image cropping, shrinking / enlarging, rotating, inverting, and changing color tone. The number of perturbed images generated by the perturbed image generating unit 170 may be fixed or variable. In the following description, it is assumed that the perturbed image generating unit 170 generates M perturbed images (M is a natural number). If the perturbed image generating unit 170 generates M perturbed images, a total of M+1 images, including one original image and M perturbed images, will be output from the perturbed image generating unit 170.

[0103] The M+1 images output from the perturbation image generating unit 170 are configured to be output to the detecting unit 110. Therefore, the detecting unit 110 according to the eighth embodiment outputs M+1 pieces of position information.

[0104] Each of the M+1 pieces of position information detected by the detection unit 110 is configured to be output to the acquisition unit 120. Here, particularly, the acquisition unit 120 according to the eighth embodiment is configured to include a selection unit 122. The selection unit 122 is configured to randomly select one piece of position information from the M+1 pieces of position information. Since the M+1 pieces of position information are each detected from a different image (i.e., a plurality of images to which perturbation has been added), the pieces of position information are information having errors when viewed as a whole. Therefore, the position information randomly selected from the M+1 pieces of position information can be used as perturbed position information with an error added. The acquisition unit 120 is configured to output the position information selected by the selection unit 122 to the generation unit 130 as perturbed position information.

[0105] (Operation flow) Next, the flow of operations of the image generating system 10 according to the eighth embodiment will be described with reference to Fig. 18. Fig. 18 is a flowchart showing the flow of operations of the image generating system according to the eighth embodiment. In Fig. 18, the same processes as those shown in Figs. 3, 4, and 14 are denoted by the same reference numerals.

[0106] 18, when the operation of the image generating system 10 according to the eighth embodiment is started, first, an image is input to the perturbed image generating unit 170 (step S61). When the image is input, the perturbed image generating unit 170 generates M perturbed images from the image (step S62).

[0107] Next, the detection unit 110 detects position information on the positions of the faces or facial feature points from each of the M+1 images (i.e., one original image plus M perturbed images) (step S63). As a result, M+1 pieces of position information are detected.

[0108] Next, the selection unit 122 randomly selects one piece of position information from the M+1 pieces of position information (step S64).Then, the generation unit 130 executes a process of normalizing the face based on the randomly selected position information (i.e., the perturbed position information) (step S21).The generation unit 130 outputs a face normalized image, which is an image including the normalized face, as a new image (step S22).

[0109] (Technical effect) Next, technical effects obtained by the image generation system 10 according to the eighth embodiment will be described.

[0110] As described in FIG. 17 and FIG. 18, in the image generating system 10 according to the eighth embodiment, M perturbed images are generated from an input image, and M+1 pieces of position information are detected. Then, position information randomly selected from the M+1 pieces of position information is used as perturbed position information for generating a new image. In this way, perturbed position information can be obtained without directly calculating an error (for example, without adding a perturbation amount as in the third and fourth embodiments). Therefore, it is possible to appropriately generate a new face image including a positional variation according to the error.

[0111] (Modification of the eighth embodiment) Next, an image generation system 10 according to a modified example of the eighth embodiment will be described with reference to Fig. 19 and Fig. 20. Note that the image generation system 10 according to the modified example of the eighth embodiment differs from the eighth embodiment described above only in some operations, and other operations and system configurations may be similar to those of the eighth embodiment. Therefore, hereinafter, only the parts that differ from the eighth embodiment already described will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0112] (Functional configuration) First, the functional configuration of an image generation system 10 according to a modified example of the eighth embodiment will be described with reference to Fig. 19. Fig. 19 is a block diagram showing the functional configuration of an image generation system according to a modified example of the eighth embodiment. In Fig. 19, the same components as those shown in Fig. 17 are denoted by the same reference numerals.

[0113] As shown in Fig. 19, the image generating system 10 according to the modification of the eighth embodiment includes a detection unit 110, an acquisition unit 120, a generation unit 130, a probability distribution estimation unit 160, and a perturbed image generation unit 170 as processing blocks for realizing the functions of the image generating system 10. That is, the image generating system 10 according to the modification of the eighth embodiment is configured to further include a probability distribution estimation unit 160 in addition to the configuration of the above-mentioned eighth embodiment (see Fig. 17). The probability distribution estimation unit 160 may be realized by the above-mentioned processor 11 (see Fig. 1).

[0114] The probability distribution estimation unit 160 is configured to be able to execute a process of fitting M+1 pieces of position information detected from M+1 images with a predetermined probability distribution. The probability distribution estimation unit 160 may, for example, fit N pieces of position information with a Gaussian distribution. In this case, the position information may be a value expressed by the formula Gaussian model mean + Gaussian model deviation + random number.

[0115] The result of the fitting process of the probability distribution estimation unit 160 is configured to be output to the acquisition unit 120. Here, in particular, the acquisition unit 120 according to the modified example of the eighth embodiment is configured to have a sampling unit 123. The sampling unit 123 is configured to sample one piece of position information from the probability distribution output as the processing result of the probability distribution estimation unit 160. The position information sampled in this manner can be used as perturbed position information with an error added. The acquisition unit 120 is configured to output the position information sampled by the sampling unit 123 to the generation unit 130 as perturbed position information.

[0116] (Operation flow) Next, the flow of operations of the image generating system 10 according to a modified example of the eighth embodiment will be described with reference to Fig. 20. Fig. 20 is a flowchart showing the flow of operations of the image generating system according to a modified example of the eighth embodiment. In Fig. 20, the same processes as those shown in Fig. 18 are denoted by the same reference numerals.

[0117] 20, when the operation of the image generating system 10 according to the modification of the eighth embodiment is started, first, an image is input to the perturbed image generating unit 170 (step S61). When the image is input, the perturbed image generating unit 170 generates M perturbed images from the image (step 62).

[0118] Next, the detection unit 110 detects position information on the positions of the faces or facial feature points from each of the M+1 images (i.e., one original image plus M perturbed images) (step S63). As a result, M+1 pieces of position information are detected.

[0119] Next, the probability distribution estimation unit 160 executes a process of fitting the M+1 pieces of location information with a predetermined probability distribution (step S65). Then, the sampling unit 123 samples one piece of location information from the probability distribution output as a processing result of the probability distribution estimation unit 160 (step S66).

[0120] Next, the generation unit 130 executes a process of normalizing the face based on the sampled position information (i.e., perturbation position information) (step S21). The generation unit 130 outputs a face normalized image, which is an image including the normalized face, as a new image (step S22).

[0121] (Technical effect) Next, technical effects obtained by the image generation system 10 according to the modified example of the eighth embodiment will be described.

[0122] As described in Fig. 19 and Fig. 20, in the image generating system 10 according to the modified example of the eighth embodiment, M perturbed images are generated by the perturbed image generating unit 170, and as a result, M+1 pieces of position information are detected. Then, the M+1 pieces of position information are fitted with a probability distribution, and the position information sampled from the distribution is used as perturbed position information for generating a new image. In this way, the perturbed position information can be obtained from the probability distribution estimated from the M+1 pieces of position information. Therefore, it is possible to appropriately generate a new face image including a positional variation according to an error.

[0123] <Ninth embodiment> An image generating system 10 according to the ninth embodiment will be described with reference to Fig. 21 and Fig. 22. The ninth embodiment differs from the first to eighth embodiments in part of its configuration and operation, and other parts may be similar to the first to eighth embodiments. Therefore, hereinafter, parts that differ from the first to eighth embodiments will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0124] (Functional configuration) First, the functional configuration of the image generation system 10 according to the ninth embodiment will be described with reference to Fig. 21. Fig. 21 is a block diagram showing the functional configuration of the image generation system according to the ninth embodiment. In Fig. 21, the same components as those shown in Figs. 2, 15, and 19 are denoted by the same reference numerals.

[0125] As shown in Fig. 21, the image generating system 10 according to the ninth embodiment includes a detection unit 110, an acquisition unit 120, and a generation unit 130 as processing blocks for realizing the functions of the system. In particular, the detection unit 110 according to the ninth embodiment is configured to include a probability distribution output unit 111. The acquisition unit 120 according to the ninth embodiment is configured to include a sampling unit 123.

[0126] The probability distribution output unit 111 is configured to output the output result of the detection unit 110 in the form of a probability distribution. Therefore, the detection unit 110 according to the ninth embodiment is capable of outputting position information related to a face or facial feature points detected from an image in the form of a probability distribution. The detection unit 110 according to the ninth embodiment may be configured as a detector that performs position estimation in the form of a probability distribution. In a detector that performs position estimation in the form of a probability distribution, it is sufficient to deterministically estimate position information according to a rule determined in advance by a user, such as a median, an average, or a mode.

[0127] The sampling unit 123 is configured to sample one piece of position information from the probability distribution output from the probability distribution output unit 111. The position information sampled in this manner can be used as perturbed position information that takes error into account. The acquisition unit 120 is configured to output the position information sampled by the sampling unit 123 to the generation unit 130 as perturbed position information.

[0128] (Operation flow) Next, the flow of operations of the image generating system 10 according to the ninth embodiment will be described with reference to Fig. 22. Fig. 22 is a flowchart showing the flow of operations of the image generating system according to the ninth embodiment. In Fig. 22, the same processes as those shown in Figs. 3 and 4 are denoted by the same reference numerals.

[0129] 22, when the operation of the image generating system 10 according to the ninth embodiment is started, first, an image is input to the detection unit 110 (step S11). When the image is input, the detection unit 110 detects position information related to the positions of the face or facial feature points from the image (step S12).

[0130] Next, the probability distribution output unit 111 outputs the detection result of the detection unit 110 in the form of a probability distribution (step S71). Then, the sampling unit 123 samples one piece of position information from the probability distribution output as the detection result of the detection unit 110 (step S72).

[0131] Next, the generation unit 130 executes a process of normalizing the face based on the sampled position information (i.e., perturbation position information) (step S21). The generation unit 130 outputs a face normalized image, which is an image including the normalized face, as a new image (step S22).

[0132] (Technical effect) Next, technical effects obtained by the image generation system 10 according to the ninth embodiment will be described.

[0133] 21 and 22, in the image generating system 10 according to the ninth embodiment, the detection result of the detection unit 110 is output in the form of a probability distribution, and position information sampled from the distribution is used as perturbed position information for generating a new image. In this way, the perturbed position information can be acquired from the probability distribution that is the detection result of the detection unit 110. Therefore, it is possible to appropriately generate a new face image including positional fluctuations according to errors.

[0134] Tenth embodiment An image generation system according to the tenth embodiment will be described. The tenth embodiment describes a specific application example of the image generation system according to the first to ninth embodiments described above, and the system configuration and operation flow may be similar to those of the first to ninth embodiments. Therefore, in the following, the parts that differ from the first to ninth embodiments described above will be described in detail, and the explanation of other overlapping parts will be omitted as appropriate.

[0135] (Increasing training data) The image generating system 10 according to the tenth embodiment may be used to increase the training data used in machine learning. According to the image generating system 10 according to the present embodiment, a different image can be newly generated from one image included in the training data, so that the number of images included in the training data can be increased. By increasing the training data in this way, generalizability and robustness can be improved.

[0136] As a method for increasing the training data, it is possible to adopt processes such as image cropping, shrinking / enlarging, and rotation. However, even if the training data is increased in this way, it is difficult to improve the robustness against positional fluctuations in face normalization, for example.

[0137] However, in the image generating system 10 according to the present embodiment, as already described, perturbations are applied in accordance with errors that occur when detecting position information of a face or facial feature points. This makes it possible to improve the robustness of position fluctuations in the normalization process that uses position information of a face or facial feature points.

[0138] (Application to generative adversarial networks) The image generation system 10 according to the tenth embodiment may be used for virtual adversarial training. Specifically, the image generation system 10 may be configured to generate adversarial examples in adversarial training.

[0139] When generating adversarial examples, artificial small noise is added to the training data to make it difficult for machines to recognize them. However, if the generated data does not take into consideration the distribution of the training data, noisy data that does not actually exist may be generated. Adversarial examples generated in this way cannot be said to contribute to improving the learning of neural networks.

[0140] However, in the image generation system 10 according to the present embodiment, perturbations are applied according to errors that occur when detecting position information of a face or facial feature points. Therefore, by combining the adversarial examples for feature point positions obtained by adversarial learning with the perturbation position information acquired by the image generation system 10, it is possible to reliably improve the effect of adversarial learning. Note that the above-mentioned combination method may use known techniques for vector / sequence integration, such as averaging and projection.

[0141] (Person matching) The image generating system 10 according to the tenth embodiment can also be applied to person matching (so-called face recognition). For example, by using a plurality of images (images to which perturbations according to errors are given) generated by the image generating system 10 according to the present embodiment, it is possible to suppress a decrease in authentication accuracy.

[0142] When matching a person, there is a risk that matching will not be performed correctly depending on how the face is captured in the image. In such cases, for example, a person who should normally be authenticated may not be authenticated, or conversely, a person who should not be authenticated may be authenticated.

[0143] However, in the image system 10 according to this embodiment, a different face image can be newly generated from one face image. Therefore, it is possible to perform person matching using multiple face images (or by selecting one appropriate face image from multiple face images). Furthermore, particularly in this embodiment, perturbation is applied according to an error that occurs when detecting position information of a face or facial feature points. Therefore, it is possible to generate an appropriate image taking into consideration the positional fluctuation of feature points used for person matching. As a result, it is possible to effectively improve the accuracy of authentication by person matching.

[0144] (Maximize matching accuracy) In matching a person, a perturbation amount that maximizes matching accuracy when matching is performed in a specific environment may be calculated. Specifically, when calculating the perturbation amount by multiplying the error parameter by a coefficient between 0 and 1, the coefficient that maximizes matching accuracy may be obtained.

[0145] (Feature integration) In person matching, an integrated feature may be obtained by integrating N feature amounts. Specifically, first, the image generating system 10 according to this embodiment generates N face normalized images from N pieces of perturbation position information. Then, face feature extraction is performed for each of the N face normalized images to generate N feature amounts. Finally, one integrated feature is generated from the N feature amounts using any feature integration method. Note that, as the feature integration method, existing techniques such as average and Gaussian estimation can be appropriately adopted.

[0146] (Selection of match image) In matching a person, an image to be used for matching may be selectable. Specifically, a plurality of generated face normalized images are displayed on a display and presented to a user (i.e., a person to be authenticated, or a system administrator, etc.). The user may then select an image to be used for authentication from the plurality of presented face normalized images. In this case, the images may be ranked so that the more suitable an image is for matching, the higher it is displayed. Also, only a predetermined number of images suitable for matching may be displayed. In addition to displaying the images, a sentence such as "Are your eyes misaligned?" may be displayed to obtain a response from the user.

[0147] (montage creation) The image generating system 10 according to the tenth embodiment can be used to create a montage. When creating a montage, parts of the face are changed little by little, but if the changes cannot be made appropriately, the image cannot be created efficiently. For example, if changes that are impossible for a human face are made, an appropriate montage cannot be created.

[0148] However, in the image generating system 10 according to the present embodiment, perturbations are applied according to errors that occur when detecting position information of a face or facial feature points. This makes it possible to impart realistically expected appropriate variations to a face image. Therefore, if the images generated by the image generating system 10 according to the present embodiment are applied to the creation of a montage, it becomes possible to create a montage more efficiently.

[0149] <Additional Notes> The above-described embodiment may be further described as follows, but is not limited to the following.

[0150] (Appendix 1) The image generation system described in Appendix 1 is an image generation system characterized by comprising a detection means for detecting position information regarding the position of a face or a feature point of the face from an image, an acquisition means for acquiring perturbed position information that takes into account an error that occurs when detecting the position information, and a generation means for generating a new image including the face based on the perturbed position information.

[0151] (Appendix 2) The image generation system described in Appendix 2 is the image generation system described in Appendix 1, characterized in that the generation means generates, as the new image, a normalized image in which at least one of the position, size, and angle of the face is adjusted based on the perturbation position information.

[0152] (Appendix 3) The image generation system described in Appendix 3 is the image generation system described in Appendix 1 or 2, further comprising a calculation means for calculating a perturbation amount corresponding to the error, and the acquisition means acquires the perturbation position information by adding the perturbation amount to the position information.

[0153] (Appendix 4) The image generation system described in Supplementary Note 4 is the image generation system described in Supplementary Note 3, characterized in that the calculation means calculates the perturbation amount according to the error specified by a user.

[0154] (Appendix 5) The image generation system described in Appendix 5 is the image generation system described in Appendix 3, further comprising an error evaluation means for evaluating the error using a test image having correct answer data for the position information, and the calculation means calculates the perturbation amount according to the evaluated error.

[0155] (Appendix 6) The image generation system described in Supplementary Note 6 is the image generation system described in Supplementary Note 3, characterized in that the calculation means calculates the perturbation amount according to a deviation of a probability distribution of a plurality of pieces of position information.

[0156] (Appendix 7) The image generation system described in Appendix 6 is the image generation system described in Appendix 1 or 2, characterized in that the acquisition means acquires the perturbation position information based on multiple pieces of position information detected by one of the detection means.

[0157] (Appendix 8) The image generation system described in Appendix 8 is the image generation system described in Appendix 7, characterized in that the acquisition means acquires the perturbation position information based on multiple pieces of position information detected by multiple different detection means.

[0158] (Appendix 9) The image generation system described in Appendix 9 is the image generation system described in Appendix 7 or 8, further comprising a probability distribution estimation means for estimating a probability distribution from a plurality of the position information, and the acquisition means acquires the perturbed position information by sampling from the estimated probability distribution.

[0159] (Appendix 10) The image generation system described in Appendix 10 is the image generation system described in Appendix 1 or 2, further comprising a perturbed image generation means for applying a perturbation to the image to generate a plurality of perturbed images, and the acquisition means is characterized in that it acquires the perturbed position information based on a plurality of pieces of position information detected for each of the plurality of perturbed images.

[0160] (Appendix 11) The image generation system described in Appendix 11 is the image generation system described in Appendix 1 or 2, characterized in that the detection means outputs the position information in the form of a probability distribution, and the acquisition means acquires the perturbed position information by sampling from the output probability distribution.

[0161] (Appendix 12) The image generation method described in Appendix 12 is an image generation method characterized by detecting position information regarding the position of a face or a feature point of the face from an image, obtaining perturbed position information that takes into account an error that occurs when detecting the position information, and generating a new image including the face based on the perturbed position information.

[0162] (Appendix 13) The recording medium described in Supplementary Note 13 is a recording medium having a computer program recorded thereon to cause a computer to execute an image generating method for detecting position information on a face or a feature point of the face from an image, acquiring perturbed position information that takes into account an error that occurs when detecting the position information, and generating a new image including the face based on the perturbed position information.

[0163] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and the image generation system, image generation method, and recording medium incorporating such modifications are also included in the technical idea of ​​this disclosure. [Explanation of symbols]

[0164] 10 Image Generation System 11 Processors 110 Detection unit 111 Probability distribution output unit 120 Acquisition Department 121 Perturbation Addition Unit 122 Selection section 123 Sampling section 130 Generation part 140 Perturbation amount generation section 150 Error evaluation section 160 Probability distribution estimation unit 170 Perturbation Image Generation Unit

Claims

1. a detection means for detecting position information relating to the position of a face or a feature point of the face from an image; a calculation means for calculating a perturbation amount corresponding to an error that occurs when detecting the position information with respect to the position information; an acquisition means for acquiring perturbed position information by adding the perturbation amount to the position information; a generating means for generating a new image including the face based on the perturbation position information; An image generating system comprising:

2. The image generation system according to claim 1 , wherein the generating means generates, as the new image, a normalized image in which at least one of the position, size, and angle of the face is adjusted based on the perturbation position information.

3. 2. The image generating system according to claim 1, wherein said calculation means calculates said perturbation amount in accordance with said error designated by a user.

4. further comprising an error evaluation means for evaluating the error by using a test image having correct answer data of the position information; 2. The image generating system according to claim 1, wherein the calculation means calculates the amount of perturbation in accordance with the evaluated error.

5. 2. The image generating system according to claim 1, wherein the calculation means calculates the amount of perturbation in accordance with a deviation of a probability distribution of a plurality of pieces of position information.

6. Detecting position information relating to the position of a face or a feature point of the face from the image; Calculating a perturbation amount corresponding to an error that occurs when detecting the position information with respect to the position information; adding the perturbation amount to the position information to obtain perturbed position information; Generate a new image including the face based on the perturbed position information.

13. An image generating method comprising:

7. Detecting position information relating to the position of a face or a feature point of the face from the image; Calculating a perturbation amount corresponding to an error that occurs when detecting the position information with respect to the position information; adding the perturbation amount to the position information to obtain perturbed position information; Generate a new image including the face based on the perturbed position information. A computer program that causes a computer to execute the image generation method.

Citation Information

Patent Citations

  • Face image recognition device and face image recognition program

    JP2005208850A

  • Image recognition apparatus, method and program

    JP2007004767A

  • Feature point detecting method, device and program

    JP2007226424A

  • Face image processor

    JP2012173812A

  • Face image authentication device

    JP2017182459A