Image generation apparatus, program, and image generation method
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- NIKON CORP
- Filing Date
- 2021-09-29
- Publication Date
- 2026-08-04
Smart Images
Figure 0007899522000001 
Figure 0007899522000002 
Figure 0007899522000003
Abstract
Description
Technical Field
[0001] The present invention relates to an image generation apparatus, a program, and an image generation method.
Background Art
[0002] Conventionally, there has been known a technique for hiding the face of a user by synthesizing a stamp image, a mosaic image, or the like on a part of the user's face or the like, or blurring an image of the user being photographed. As a document disclosing such a technique, for example, Patent Document 1 can be cited.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
[0004] One aspect of the present invention includes a determination unit that determines whether or not a specific person is included in an image, and a generation unit that generates a processed image in which at least a part of the face of the specific person is removed from the image when it is determined that the specific person is included in the image, based on the image, such that the face of the specific person cannot be discriminated. The image includes a first image captured from a first viewpoint and a second image captured from a second viewpoint different from the first viewpoint. The determination unit determines whether the processed image can be generated based on the first image and whether the processed image can be generated based on the second image. The generation unit determines that the processed image cannot be generated based on the first image and that the processed image can be generated based on the second image, and generates the processed image based on the second image. It is an image generation apparatus. One aspect of the present invention includes a determination unit that determines whether or not a specific person is included in an image, and a generation unit that generates a processed image in which at least a part of the face of the specific person is included in a part of the image when it is determined that the specific person is included in the image, by overlapping, based on the image, such that the face of the specific person cannot be discriminated. Images of a different subject from the aforementioned image, taken at the same time as the aforementioned image. It is an image generation apparatus including the above. One aspect of the present invention is an image generation device comprising: a determination unit that determines whether or not a specific person is included in an image, and whether or not a person other than the specific person is continuously captured in the image for a predetermined period of time; and a generation unit that, if it is determined that the image is included in a specific person, generates a processed image based on the image in which the face of the specific person is indistinguishable, and if it is determined that a person other than the specific person is continuously captured in the image for a predetermined period of time, generates a processed image in which at least a part of the face of the person other than the specific person is included.
[0005] One aspect of the present invention is a computer that determines whether or not a specific person is included in an image. Furthermore, to determine whether a person other than the specific person is continuously captured in the image for a predetermined period of time. If it is determined that the image contains the specific person, a processed image is generated based on the image in which the face of the specific person is indistinguishable. Furthermore, if it is determined that a person other than the specific person is continuously captured in the image for a predetermined period of time, the processed image is generated which includes at least a portion of the face of the person other than the specific person. This is a program that makes that possible.
[0006] One aspect of the present invention is determining whether or not a specific person is included in an image. Furthermore, it is determined whether a person other than the specific person is continuously captured in the image for a predetermined period of time. If it is determined that the image contains the specific person, a processed image is generated based on the image in which the face of the specific person is indistinguishable. Furthermore, if it is determined that a person other than the specific person is continuously captured in the image for a predetermined period of time, the processed image is generated which includes at least a portion of the face of the person other than the specific person. This is an image generation method. [Brief explanation of the drawing]
[0007] [Figure 1] This figure shows an example of the hardware configuration of the image generation device according to the embodiment. [Figure 2] This figure shows an example of the software configuration of the image generation device according to the embodiment. [Figure 3] This figure shows an example of an image according to the embodiment. [Figure 4] This figure illustrates an example of a method by which an image generation apparatus according to the embodiment generates a processed image. [Figure 5] This figure shows an example of an image according to the embodiment. [Figure 6]This figure illustrates an example of a method by which an image generation apparatus according to the embodiment generates a processed image. [Figure 7] This figure illustrates an example of a method by which an image generation apparatus according to the embodiment generates a processed image. [Figure 8] This figure illustrates an example of a method by which an image generation apparatus according to the embodiment generates a processed image. [Figure 9] This figure illustrates an example of a method by which an image generation apparatus according to the embodiment generates a processed image. [Figure 10] This flowchart shows an example of a process performed by the image generation device according to the embodiment. [Figure 11] This flowchart shows an example of a process performed by the image generation device according to the embodiment. [Figure 12] This flowchart shows an example of a process performed by the image generation device according to the embodiment. [Figure 13] This flowchart shows an example of a process performed by the image generation device according to the embodiment. [Modes for carrying out the invention]
[0008] The technology disclosed in Patent Document 1 mentioned above can prevent the public disclosure of the face of a person whose public disclosure is restricted in at least one of the moving images and still images in which the face is captured, by using stamp images, mosaic images, or blurring. However, the technology disclosed in Patent Document 1 may give viewers of at least one of the moving images and still images an inappropriate impression, such as that the person on whom the stamp image, mosaic image, or blurring was used has caused a problem. Furthermore, because the technology disclosed in Patent Document 1 uses stamp images, mosaic images, or blurring, it may reduce the sense of realism in at least one of the moving images and still images, or give viewers of at least one of the moving images and still images an unnatural impression.
[0009] Therefore, an embodiment according to the present invention provides an image generation device, a program, and an image generation method that can generate at least one of a live-action moving image and a still image while preventing the face of a person whose public disclosure of at least one of the moving image and the still image in which the face is imaged is restricted from being publicly disclosed.
[0010] The image generation device, program, and image generation method according to the embodiment will be described with reference to FIGS. 1 to 9. In the description of this embodiment, a moving image in which a youth soccer game is being imaged will be taken as an example for explanation.
[0011] First, the hardware configuration of the image generation device according to the embodiment will be described with reference to FIG. 1. FIG. 1 is a diagram showing an example of the hardware configuration of the image generation device according to the embodiment. As shown in FIG. 1, the image generation device 10 includes a processor 11, a main storage device 12, a communication interface 13, an auxiliary storage device 14, an input / output device 15, and a bus 16.
[0012] The processor 11 is, for example, a CPU (Central Processing Unit), reads and executes a program, and realizes each function of the image generation device 10 and the functions necessary for realizing these functions.
[0013] The main storage device 12 is, for example, a RAM (Random Access Memory), and stores in advance a program read and executed by the processor 11.
[0014] The communication interface 13 is an interface circuit for executing communication with other devices via a network. The network is, for example, a WAN (Wide Area Network), a LAN (Local Area Network), the Internet, or an intranet. For example, as shown in FIG. 1, the communication interface 13 executes communication with cameras 20-1,... and camera 20-k (k: an integer of 1 or more).
[0015] At least one of cameras 20-1, …, and camera 20-k may be used, for example, by a guardian of a child who is a player participating in a youth soccer game to image their own child. Alternatively, at least one of cameras 20-1, …, and camera 20-k may be a camera installed in a stadium where a youth soccer game is being held. Cameras 20-1, …, and camera 20-k image one youth soccer game from different viewpoints. Also, cameras 20-1, …, and camera 20-k may all have fixed viewpoints, or the viewpoints may move as needed. Note that cameras 20-1, …, and camera 20-k all transmit the captured moving images to the image generation device 10 via a network.
[0016] The auxiliary storage device 14 is, for example, a hard disk drive (HDD), a solid state drive (SSD), a flash memory, or a ROM (Read Only Memory).
[0017] The input / output device 15 is, for example, an input / output port, to which an input device and an output device are connected. The input device is, for example, a touch panel display, a mouse, or a keyboard, and is used for operating the image generation device 10 and inputting data to the image generation device 10. The output device is, for example, a touch panel display or a speaker, and is used for the image generation device 10 to present information to the user.
[0018] The bus 16 connects the processor 11, the main storage device 12, the communication interface 13, the auxiliary storage device 14, and the input / output device 15 so that data can be transmitted and received between them.
[0019] Next, the software configuration of the image generation device will be described with reference to Figure 2. Figure 2 is a diagram showing an example of the software configuration of an image generation device according to an embodiment. As shown in Figure 2, the image generation device 10 includes a determination unit 101, a generation unit 102, and an editing unit 103. The determination unit 101, the generation unit 102, and the editing unit 103 are all realized by the processor 11 reading and executing a program stored in the main memory 12.
[0020] The determination unit 101 determines whether or not a specific person is captured in the image. This image is a frame image that makes up a video of a youth soccer match. Alternatively, this image is a still image of a youth soccer match. The specific person is, for example, a child who is a player belonging to a youth soccer team and whose public release is restricted for at least one of the video and / or still images in which their face is captured.
[0021] Furthermore, the determination unit 101 acquires pre-prepared data indicating a specific person and determines whether or not a specific person is captured in the image based on this data. The data indicating a specific person may be, for example, a facial photograph of each person and data indicating that at least one of the video and still images containing that person's face has been refused publication. In the case of players participating in a youth soccer match, the data indicating a specific person may be each player's jersey number and data indicating that at least one of the video and still images containing that player's face has been refused publication.
[0022] Furthermore, data identifying a specific individual may also indicate that the disclosure of at least one of the video and still images is being refused, depending on the scope of disclosure. The scope of disclosure may be, for example, a group formed on a social networking service (SNS). For example, if a player's guardian has an account on a social networking service and the player's face is not publicly displayed within the group to which the guardian belongs, the data identifying a specific individual would indicate that the disclosure of video and still images containing the player's face is being refused. Alternatively, if neither the player nor the player's guardian has an account on a social networking service, the data identifying a specific individual would indicate that the disclosure of video and still images containing the player's face in a manner that allows for identification of the player's face is being refused.
[0023] Specifically, the determination unit 101 determines whether each object captured in the image is a specific person. For example, the determination unit 101 applies face recognition and object recognition using machine learning to the image and determines whether each of these objects is a specific person. The objects referred to here are people, objects, etc., captured in the image. If the determination unit 101 determines that at least one of the objects captured in the image is a specific person, it determines that a specific person is captured in the image. On the other hand, if the determination unit 101 determines that none of the objects captured in the image are specific people, it determines that a specific person is not captured in the image.
[0024] If the generation unit 102 determines that a specific person is captured in an image, it generates a processed image based on that image in which the face of the specific person is indistinguishable. Specifically, if the generation unit 102 determines that a specific person is captured in an image, it generates a processed image based on that image in which at least a part of the face of the specific person is not included. For example, the generation unit 102 generates a processed image in which the face of the specific person cannot be recognized by cutting out the necessary parts from the image and removing at least a part of the face of the specific person to the extent that it is unrecognizable, removing a rectangular part of the image to remove at least a part of the face of the specific person, or overlaying another image on at least a part of the face of the specific person.
[0025] Furthermore, the determination unit 101 may determine whether or not a person other than the specific person is captured in the image. A person other than the specific person is, for example, a child who is a player belonging to a youth soccer team and whose public release of video and still images in which their face is captured is not restricted.
[0026] Furthermore, the determination unit 101 acquires data indicating a person different from a specific person that has been prepared in advance, and determines whether or not a person different from the specific person is captured in the image based on this data. The data indicating a person different from the specific person may be, for example, a facial photograph of each person and data indicating that permission has been granted to publish at least one of a video or still image containing that person's face. In the case of players participating in a youth soccer match, the data indicating a person different from the specific person may also be the jersey number of each player and data indicating that permission has been granted to publish at least one of a video or still image containing that person's face.
[0027] Furthermore, data indicating a person different from a specific individual may also be data indicating permission for publication depending on the scope of publication of at least one of the video and still images. The scope of publication may be, for example, a group formed on a social networking service. For example, if a player's guardian has an account on a social networking service and the player's face is made public within the group to which the guardian belongs, the data indicating a specific individual would indicate permission for publication of video and still images containing the player's face within that group.
[0028] Specifically, the determination unit 101 determines whether each object captured in the image is a different person from the specific person. For example, the determination unit 101 applies face recognition and object recognition using machine learning to the image and determines whether each of these objects is a different person from the specific person. The objects referred to here are people, objects, etc., captured in the image. If the determination unit 101 determines that at least one of the objects captured in the image is a different person from the specific person, it determines that a person different from the specific person is captured in the image. On the other hand, if the determination unit 101 determines that none of the objects captured in the image are the specific person, it determines that the specific person is not captured in the image.
[0029] If the generation unit 102 determines that an image contains a person other than the specific person, it generates a processed image based on the image that includes the face of the person other than the specific person. For example, the generation unit 102 generates a processed image in which at least a portion of the face of the person other than the specific person is retained in a manner that allows the person other than the specific person to be recognized.
[0030] Figure 3 shows an example of an image according to the embodiment. Of the four players included in image P3 shown in Figure 3, the first, third, and fourth players from the left are examples of specific individuals. On the other hand, of the four players included in image P3 shown in Figure 3, the second player from the left is an example of a person different from the specific individual.
[0031] Figure 4 is a diagram illustrating an example of how the image generation apparatus according to the embodiment generates a processed image. For example, the generation unit 102 removes the area with diagonal hatching from the image P3 shown in Figure 4 to generate a processed image P4. In the images P3 shown in Figures 3 and 4, the vertical position of the image P3 of a specific person's face is different from the vertical position of the image P3 of a person other than that specific person. Therefore, the generation unit 102 can generate a processed image P4 by removing at least the area above the head of the second player from the left in image P3. Furthermore, it is preferable that the aspect ratio of the processed image P4 is equal to the aspect ratio of image P3.
[0032] Figure 5 shows an example of an image according to the embodiment. Of the four players included in image P5 shown in Figure 5, the first, third, and fourth players from the left are examples of specific individuals. On the other hand, of the four players included in image P5 shown in Figure 5, the second player from the left is an example of a person different from the specific individual.
[0033] Figure 6 is a diagram illustrating an example of how the image generation device according to the embodiment generates a processed image. For example, the generation unit 102 removes the area with diagonal hatching from the image P5 shown in Figure 6 to generate a processed image P6. In the images P5 shown in Figures 5 and 6, the vertical position of the image P5 of a specific person's face is close to the vertical position of the image P5 of a person other than the specific person. Therefore, the generation unit 102 can generate a processed image P6 by removing at least the area to the left and the area to the right of the second player from the left in image P5. Furthermore, the processed image P6 may be an image with a shape other than a rectangle.
[0034] Furthermore, the determination unit 101 may determine whether a person other than the specific person is continuously captured in the image for a predetermined period of time. The predetermined period of time here is, for example, a length of time that does not give a viewer an unnatural impression of a processed image generated based on the image or a video containing the image.
[0035] If the generation unit 102 determines that a person other than the specific person is continuously captured in the image for a predetermined period of time, it generates a processed image that includes at least a portion of the face of the person other than the specific person. For example, in such a case, the generation unit 102 generates a processed image in which at least a portion of the face of the person other than the specific person is retained in a manner that allows the person other than the specific person to be recognized.
[0036] Furthermore, the above-mentioned image may include a first image captured from a first viewpoint and a second image captured from a second viewpoint different from the first viewpoint. The first image is, for example, an image captured by camera 20-1, ... or camera 20-k. On the other hand, the second image is an image captured by a camera different from the camera that captured the first image. In this case, the determination unit 101 determines whether or not there is a second image captured from a second viewpoint different from the first viewpoint. The determination unit 101 then determines whether or not a processed image can be generated based on the first image and whether or not a processed image can be generated based on the second image.
[0037] For example, the determination unit 101 determines that a processed image can be generated based on the first image if a part of the first image can be removed so that a specific person cannot be identified in the first image. The same applies to the determination regarding the second image.
[0038] More specifically, the determination unit 101 determines that it is possible to generate a processed image in which the face of a specific person cannot be recognized, by processes such as removing a rectangular portion of the first image to remove at least a portion of the face of a specific person, or overlaying another image on at least a portion of the face of a specific person. The same applies to the determination regarding the second image.
[0039] Then, if the generation unit 102 determines that it cannot generate a processed image based on the first image, and determines that it can generate a processed image based on the second image, it generates a processed image based on the second image. For example, in such a case, the generation unit 102 generates a processed image in which the face of the specific person cannot be recognized by removing a rectangular portion of the second image to remove at least a portion of the face of the specific person, or by overlaying another image on at least a portion of the face of the specific person.
[0040] Furthermore, the generation unit 102 may generate a processed image that includes characteristic objects captured in the image. Characteristic objects here refer to parts or objects other than characteristic human faces that represent the scene captured in the image. For example, if the image captures a scene from a youth soccer match, the characteristic objects would be the feet of the multiple players vying for the soccer ball and the soccer ball itself.
[0041] Figure 7 is a diagram illustrating an example of how the image generation apparatus according to the embodiment generates a processed image. For example, the generation unit 102 removes the area with diagonal hatching from the image P5 shown in Figure 7 and generates a processed image P7. The processed image P7 is a processed image that includes the feet of the four players and the soccer ball included in image P5.
[0042] Furthermore, the determination unit 101 may determine whether or not a processed image can be generated by inserting an image of a different object from the specific person into the portion of the first or second image that includes at least a part of the specific person's face.
[0043] Furthermore, if the generation unit 102 determines that it can generate a processed image by this method, it may generate the processed image by inserting an image of a different subject from the specific person into the portion of the image that includes at least a part of the specific person's face. The image of a different subject from the specific person is, for example, an image of a person or object other than the specific person's face. For example, if the image captures a scene from a youth soccer match, the image of a different subject from the specific person would be an image of the stands at the match, an image of the scoreboard at the match, etc.
[0044] Figure 8 is a diagram illustrating an example of how the image generation device according to the embodiment generates a processed image. For example, the generation unit 102 generates a processed image P81 by superimposing images P82 and P83 onto the area to the right of the second player from the left, which is an example of a specific person, excluding the area with diagonal hatching in the image P5 shown in Figure 8. Note that the generation unit 102 only needs to generate a processed image in which the face of a specific person cannot be recognized, so it does not need to superimpose image P83 at the feet of the specific person, and image P83 may be superimposed on at least a part of the face of the specific person.
[0045] Figure 9 is a diagram illustrating an example of how the image generation device according to the embodiment generates a processed image. For example, the generation unit 102 generates a processed image P91 by superimposing image P92 onto the area to the right of the second player from the left, which is an example of a specific person, excluding the area with diagonal hatching in image P5 shown in Figure 9.
[0046] Furthermore, when the generation unit 102 generates a processed image by inserting an image of a different object into the portion of an image that includes at least a part of the face of a specific person, it is preferable that the time at which the image containing at least a part of the face of the specific person was captured and the time at which the image of the different object was captured are the same. In particular, for example, as shown in Figure 9, when the image of the different object is an image taken from a bird's-eye view of a soccer field where a youth soccer match is being played, it is preferable that these two times are the same.
[0047] Editorial Department 103, if a processed image is generated, will adopt the processed image as one of the images that make up the video or still image presented to the user. Furthermore, Editorial Department 103 will adopt the first image if it is determined that all of the objects captured in the first image, taken from the first viewpoint, are not a specific person. Similarly, Editorial Department 103 will adopt the second image if it is determined that all of the objects captured in the second image, taken from the second viewpoint, are not a specific person. In addition, if Editorial Department 103 cannot adopt the image as is and cannot generate a processed image, it will not adopt either the original image or the processed image. The image adopted by Editorial Department 103 will be displayed on the screen as part of the video or still image presented to the user.
[0048] Next, an example of the processing performed by the image generation apparatus according to the embodiment will be described with reference to Figures 10 to 13. Figures 10 to 13 are flowcharts of an example of the processing performed by the image generation apparatus according to the embodiment. The flowchart shown in Figure 10, the flowchart shown in Figure 11, the flowchart shown in Figure 12, and the flowchart shown in Figure 13 are connected by connectors A, B, C, D, and E.
[0049] In step S101, the determination unit 101 determines whether or not a specific person is captured in the first image captured from the first viewpoint. For example, the determination unit 101 determines whether or not each object captured in the first image captured from the first viewpoint is a specific person. If the determination unit 101 determines that at least one of the objects captured in the first image is a specific person (step S101: YES), the process proceeds to step S102. On the other hand, if the determination unit 101 determines that none of the objects captured in the first image are specific people (step S101: NO), the process proceeds to step S105.
[0050] In step S102, the determination unit 101 determines whether or not a person other than the specific person is captured in the first image captured from the first viewpoint. For example, the determination unit 101 determines whether or not each object captured in the first image captured from the first viewpoint is a person other than the specific person. If the determination unit 101 determines that at least one of the objects captured in the first image is a person other than the specific person (step S102: YES), the process proceeds to step S103. On the other hand, if the determination unit 101 determines that none of the objects captured in the first image are different people from the specific person (step S102: NO), the process proceeds to step S106.
[0051] In step S103, the determination unit 101 determines whether or not a processed image can be generated based on the first image. If the determination unit 101 determines that a processed image can be generated based on the first image (step S103: YES), the process proceeds to step S104. On the other hand, if the determination unit 101 determines that a processed image cannot be generated based on the first image (step S103: NO), the process proceeds to step S106.
[0052] For example, the determination unit 101 determines that it can generate a processed image based on the first image if it is possible to remove a part of the first image in such a way that a person different from the specific person is identifiable in the first image, while leaving that person in the first image identifiable in the first image, while making the specific person in the first image identifiable in the first image. On the other hand, the determination unit 101 determines that it is not possible to generate a processed image based on the first image if it is not possible to remove a part of the first image in such a way that a person different from the specific person is identifiable in the first image, while making that person in the first image identifiable in the first image.
[0053] In step S104, the generation unit 102 generates a processed image based on the first image in which at least part of the face of a person different from the specific person is not included, and at least part of the face of a person different from the specific person is included. For example, the generation unit 102 generates the processed image by removing a part of the first image so that the person different from the specific person is left in the first image in a manner that makes them identifiable, while the specific person is left in the first image in a manner that makes them identifiable.
[0054] In step S105, the editorial department 103 selects the first image, the second image, or the processed image as images that constitute a moving or still image to be presented to the user, and terminates the processing. For example, if the editorial department 103 determines in step S101 that all of the objects captured in the first image, which was captured from the first viewpoint, are not a specific person, it selects the first image. Also, for example, if the processed image is generated in step S104 based on the first image, the editorial department 103 selects the processed image. Also, for example, if the editorial department 103 determines in step S107 (described later) that all of the objects captured in the second image are not a specific person, it selects the second image. Also, for example, if the processed image is generated in step S111, step S114, or step S116 (described later), the editorial department 103 selects the processed image.
[0055] In step S106, the determination unit 101 determines whether or not there is a second image captured from a second viewpoint different from the first viewpoint. If the determination unit 101 determines that there is a second image captured from a second viewpoint different from the first viewpoint (step S106: YES), the process proceeds to step S107. On the other hand, if the determination unit 101 determines that there is no second image captured from a second viewpoint different from the first viewpoint (step S106: NO), the process proceeds to step S113.
[0056] In step S107, the determination unit 101 determines whether a specific person is captured in the second image, which is captured from a second viewpoint different from the first viewpoint. For example, the determination unit 101 determines whether each object captured in the second image, which is captured from the second viewpoint, is a specific person. If the determination unit 101 determines that at least one of the objects captured in the second image is a specific person (step S107: YES), the process proceeds to step S108. On the other hand, if the determination unit 101 determines that none of the objects captured in the second image are specific people (step S107: NO), the process proceeds to step S105.
[0057] In step S108, the determination unit 101 determines whether or not a person other than the specific person is captured in the second image captured from the second viewpoint. For example, the determination unit 101 determines whether or not each object captured in the second image captured from the second viewpoint is a person other than the specific person. If the determination unit 101 determines that at least one of the objects captured in the second image is a person other than the specific person (step S108: YES), the process proceeds to step S109. On the other hand, if the determination unit 101 determines that none of the objects captured in the second image are different people from the specific person (step S108: NO), the process proceeds to step S113.
[0058] In step S109, the determination unit 101 determines whether a person other than the specific person is continuously captured in the second image for a predetermined period of time. If the determination unit 101 determines that a person other than the specific person is continuously captured in the second image for a predetermined period of time (step S109: YES), the process proceeds to step S110. On the other hand, if the determination unit 101 determines that a person other than the specific person is not continuously captured in the second image for a predetermined period of time (step S109: NO), the process proceeds to step S113.
[0059] In step S110, the determination unit 101 determines whether or not a processed image can be generated based on the second image. If the determination unit 101 determines that a processed image can be generated based on the second image (step S110: YES), the process proceeds to step S111. On the other hand, if the determination unit 101 determines that a processed image cannot be generated based on the second image (step S110: NO), the process proceeds to step S112.
[0060] For example, the determination unit 101 determines that it can generate a processed image based on the second image if it is possible to remove a part of the second image in such a way that a person different from the specific person remains identifiable in the second image, while making the specific person unidentifiable in the second image. On the other hand, the determination unit 101 determines that it is not possible to generate a processed image based on the second image if it is not possible to remove a part of the second image in such a way that a person different from the specific person remains identifiable in the second image, while making the specific person unidentifiable in the second image.
[0061] In step S111, the generation unit 102 generates a processed image based on the second image. For example, the generation unit 102 generates the processed image by removing a part of the second image so that a person different from the specific person is left in the second image in a manner that allows identification of that person, while making the specific person unidentifiable in the first image.
[0062] In step S112, the generation unit 102 selects an image different from the first image and the second image that was the subject of determination in step S110 as a new second image, and returns the process to step S110.
[0063] In step S113, the determination unit 101 determines whether it is possible to generate a processed image in which a characteristic object is visible and at least part of the face of a specific person is not visible. If the determination unit 101 determines that it is possible to generate a processed image in which a characteristic object is visible and at least part of the face of a specific person is not visible (step S113: YES), the process proceeds to step S114. On the other hand, if the determination unit 101 determines that it is not possible to generate a processed image in which a characteristic object is visible and at least part of the face of a specific person is not visible (step S113: NO), the process proceeds to step S115.
[0064] In step S114, the generation unit 102 generates a processed image in which the characteristic object captured in the first or second image is visible, but at least a part of the face of a specific person is not visible. For example, the generation unit 102 generates the processed image by removing a part of the first or second image in such a way that the characteristic object captured in the first or second image remains, while making it impossible to identify a specific person.
[0065] In step S115, the determination unit 101 determines whether a processed image can be generated by inserting an image of a different subject into the portion of the first or second image that includes at least a part of the face of the specific person. If the determination unit 101 determines that a processed image can be generated by inserting an image of a different subject into the portion of the first or second image that includes at least a part of the face of the specific person (step S115: YES), the process proceeds to step S116. On the other hand, if the determination unit 101 determines that a processed image cannot be generated by inserting an image of a different subject into the portion of the first or second image that includes at least a part of the face of the specific person (step S115: NO), the process proceeds to step S117.
[0066] In step S116, the generation unit 102 generates a processed image by inserting an image of a different object from the specific person into the portion of the first or second image that includes at least a part of the specific person's face.
[0067] In step S117, editorial department 103 does not adopt the first image, the second image, or the processed image.
[0068] The image generation apparatus, program, and image generation method according to the embodiment have been described above.
[0069] If the image generation device 10 determines that a specific person is captured in an image, it generates a processed image based on that image in which the face of the specific person is indistinguishable. For example, if the image generation device 10 determines that a specific person is captured in an image, it generates a processed image in which at least a part of the face of the specific person is not included in a manner that makes the face of the specific person identifiable. In this way, the image generation device 10 can generate at least one of a moving image and a still image that has a sense of realism and naturalness, while preventing the face of a person whose public release is restricted from being revealed in at least one of a moving image and a still image in which the face is captured.
[0070] Furthermore, if the image generation device 10 determines that an image contains a person other than the specific person, it generates a processed image that includes at least a portion of the face of the person other than the specific person. As a result, the image generation device 10 can include at least a portion of the face of the person other than the specific person in the processed image, and generate at least one of a moving image and a still image with a higher sense of realism.
[0071] Furthermore, if the image generation device 10 determines that a person other than the specific person is continuously captured in the image for a predetermined period of time, it generates a processed image that includes at least a portion of the face of the person other than the specific person. This allows the image generation device 10 to include the person other than the specific person in the moving image in a recognizable manner for a predetermined period of time, thereby generating at least one of the moving image and still image with a higher sense of realism. In addition, this prevents the image generation device 10 from generating at least one of the moving image and still image that is momentary and difficult to view because the person other than the specific person is included for less than a predetermined period of time in a recognizable manner.
[0072] Furthermore, if the image generation device 10 determines that it cannot generate a processed image based on the first image, but determines that it can generate a processed image based on the second image, it generates the processed image based on the second image. As a result, the image generation device 10 can generate a processed image based on the second image even if it cannot generate a processed image based on the first image.
[0073] Furthermore, the image generation device 10 generates a processed image that includes characteristic objects captured in the image. As a result, even when it is difficult to generate a processed image that does not include at least a part of a specific person's face by removing a part of the image, the image generation device 10 can generate a processed image with a high sense of realism by including objects that represent the characteristics of the scene captured in the image.
[0074] Furthermore, the image generation device 10 generates a processed image by inserting an image of a different object into the portion of the image that contains at least a part of the face of a specific person. As a result, even when it is difficult to generate a processed image that does not include at least a part of the face of a specific person by removing a portion of the image, the image generation device 10 can generate a processed image that has a sense of realism and looks natural.
[0075] In the embodiments described above, the example given was that video footage of a youth soccer match was captured by cameras 20-1, ... and 20-k, but the invention is not limited to this. The image generation device 10 can be applied not only to video footage of youth soccer matches, but also to video footage and still images of sports, parties, camping, barbecues, trips, events, etc.
[0076] Furthermore, although the above-described embodiment explained the case in which video footage of a youth soccer match captured from different viewpoints is edited by the image generation device 10, it is not limited to this. For example, free-viewpoint video generated using at least two of cameras 20-1, ... and camera 20-k may be edited by the image generation device 10.
[0077] Furthermore, in the embodiments described above, an example was given in which the image captured a scene from a youth soccer match, and the characteristic objects were the feet of multiple players vying for the soccer ball and the soccer ball itself. However, the embodiments are not limited to this. For example, if the image captured a scene from a volleyball match, the characteristic objects could be, for example, the hands of a player receiving the ball and the volleyball itself. Alternatively, if the image captured a scene from a barbecue, the characteristic objects could be, for example, at least one of the wire grill used for the barbecue, the meat, vegetables, and seafood being cooked on the grill, and the hands of the barbecue participants.
[0078] Furthermore, the determination unit 101 may determine whether the number of people different from the specific person included in the processed image will be less than or equal to a predetermined number once the processed image is generated. If the generation unit 102 determines that the number of people different from the specific person included in the processed image will be less than or equal to a predetermined number once the processed image is generated, it generates the processed image using a process different from the process of removing the region containing the face of the specific person from the image. As a result, the image generation device 10 can generate a processed image in which the number of people different from the specific person included in the processed image exceeds a predetermined number, thereby generating a processed image with a high sense of realism.
[0079] Furthermore, if the image includes a first image captured from a first viewpoint and a second image captured from a second viewpoint different from the first viewpoint, the determination unit 101 may determine whether the subject captured in the first image is a specific person, and whether the subject captured in the second image is a specific person. If the generation unit 102 determines that the subject captured in the first image is a specific person and the subject captured in the second image is not a specific person, it generates a processed image by replacing the first image with the second image. In this way, the image generation device 10 can generate at least one of a moving image and a still image that has a sense of realism and naturalness, while preventing the public disclosure of the face of a person whose public disclosure is restricted in at least one of the moving images and still images in which the face is captured.
[0080] Furthermore, the determination unit 101 may determine whether the resolution of the processed image will be below a predetermined resolution once the processed image is generated. If the generation unit 102 determines that the resolution of the processed image will be below a predetermined resolution once the processed image is generated, it generates the processed image using a process different from the process of removing the area containing the face of a specific person from the image. As a result, the image generation device 10 can generate a processed image whose resolution exceeds the predetermined resolution.
[0081] At least some of the functions of the image generation device 10 shown in Figure 2 may be implemented by hardware including circuitry such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit). Alternatively, at least some of the functions of the image generation device 10 shown in Figure 2 may be implemented through the cooperation of software and hardware.
[0082] Although embodiments of the present invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments, and various combinations, modifications, substitutions, and design changes may be made without departing from the spirit of the present invention.
[0083] Furthermore, the problems solved by the embodiments of the present invention described above are merely examples. Therefore, the embodiments of the present invention can also solve other problems that a person skilled in the art may recognize from the description of the embodiments described above.
[0084] Furthermore, the effects of the embodiments of the present invention described above are merely examples. Therefore, the embodiments of the present invention may also produce other effects that can be recognized by those skilled in the art from the description of the embodiments above. [Explanation of Symbols]
[0085] 10…Image generation device, 11…Processor, 12…Main memory, 13…Communication interface, 14…Auxiliary memory, 15…Input / output device, 20-1,...20-k…Camera, 30…Playback display device, 101…Determination unit, 102…Generation unit, 103…Editing unit
Claims
1. A determination unit that determines whether or not a specific person is included in the image, If it is determined that the image contains the specific person, a generation unit generates a processed image based on the image in which the face of the specific person is indistinguishable by removing at least a portion of the face of the specific person from the image. Equipped with, The aforementioned image includes a first image captured from a first viewpoint and a second image captured from a second viewpoint different from the first viewpoint. The determination unit determines whether or not the processed image can be generated based on the first image, and determines whether or not the processed image can be generated based on the second image, If the generation unit determines that it cannot generate the processed image based on the first image, and determines that it can generate the processed image based on the second image, it generates the processed image based on the second image. Image generation device.
2. A determination unit that determines whether or not a specific person is included in the image, If it is determined that the image contains the specific person, a generation unit generates a processed image based on the image in which the face of the specific person is indistinguishable by superimposing an image of a different subject, captured at the same time as the image, onto the portion of the image that contains at least a part of the face of the specific person. An image generation device equipped with the following features.
3. A determination unit that determines whether or not a specific person is included in the image, and whether or not a person other than the specific person is continuously captured in the image for a predetermined period of time, If it is determined that the image contains the specific person, a generation unit generates a processed image based on the image in which the face of the specific person is indistinguishable, and if it is determined that a person other than the specific person is continuously captured in the image for a predetermined period of time, a generation unit generates a processed image in which at least a part of the face of the person other than the specific person is included. An image generation device equipped with the following features.
4. The generation unit generates the processed image in which at least a portion of the face of the specific person is not included in a manner that allows for the identification of the face of the specific person. An image generation apparatus according to any one of claims 1 to 3.
5. The aforementioned image is a frame image that makes up a video. The image generation apparatus according to any one of claims 1 to 3, wherein the generation unit processes the frame image which has been determined to contain the specific person to generate the processed image.
6. The determination unit determines whether or not a person other than the specific person is captured in the image, If the generation unit determines that a person other than the specific person is captured in the image, it generates the processed image which includes at least a portion of the face of the person other than the specific person. An image generation apparatus according to any one of claims 1 to 3.
7. The determination unit determines, when the processed image is generated, whether the number of people different from the specific person included in the processed image is less than or equal to a predetermined number. If the generation unit determines that the number of people different from the specific person included in the processed image will be less than or equal to a predetermined number when the processed image is generated, it generates the processed image by a process different from the process of removing the region containing the face of the specific person from the image. An image generation apparatus according to any one of claims 1 to 3.
8. The aforementioned image includes a first image captured from a first viewpoint and a second image captured from a second viewpoint different from the first viewpoint. The determination unit determines whether the specific person is captured in the first image, and determines whether the specific person is captured in the second image, The generation unit determines that the specific person is captured in the first image and that the specific person is not captured in the second image, and generates the processed image by replacing the first image with the second image. The image generation apparatus according to claim 2 or claim 3.
9. The generation unit generates the processed image which includes the characteristic object captured in the image. An image generation apparatus according to any one of claims 1 to 8.
10. The determination unit determines whether the resolution of the processed image becomes less than or equal to a predetermined resolution once the processed image is generated. If the generation unit determines that the resolution of the processed image will be less than or equal to a predetermined resolution when the processed image is generated, it generates the processed image using a process different from the process of removing the region containing the face of the specific person from the image. An image generation apparatus according to any one of claims 1 to 9.
11. On the computer, To determine whether a specific person is included in the first image captured from a first viewpoint and the second image captured from a second viewpoint different from the first viewpoint, To determine whether it is possible to generate a processed image in which the face of the specific person is unrecognizable based on the first image, To determine whether it is possible to generate a processed image in which the face of the specific person is unrecognizable based on the second image, If it is determined that the second image contains the specific person, and that the processed image cannot be generated based on the first image, and that the processed image can be generated based on the second image, then the processed image is generated based on the second image by removing at least a portion of the face of the specific person from the second image. A program that makes this possible.
12. Determine whether a specific person is included in a first image taken from a first viewpoint and a second image taken from a second viewpoint different from the first viewpoint. Based on the first image, it is determined whether or not a processed image can be generated in which the face of the specific person is indistinguishable. Based on the second image, it is determined whether or not a processed image can be generated in which the face of the specific person is indistinguishable. If it is determined that the second image contains the specific person, and that the processed image cannot be generated based on the first image, and that the processed image can be generated based on the second image, then the processed image is generated based on the second image by removing at least a portion of the face of the specific person from the second image. Image generation method.
13. On the computer, Determining whether or not a specific person is included in the image, If it is determined that the aforementioned image contains the aforementioned specific person, a processed image in which the face of the aforementioned specific person is indistinguishable is generated based on the aforementioned image by superimposing an image of a different subject, captured at the same time as the aforementioned image, onto the portion of the aforementioned image that contains at least a part of the face of the aforementioned specific person. A program that makes this possible.
14. Determine whether or not a specific person is included in the image. If it is determined that the image contains the specific person, a processed image is generated based on the image in which the face of the specific person is indistinguishable by superimposing an image of a different subject, taken at the same time as the image, onto the portion of the image that contains at least a part of the face of the specific person. Image generation method.
15. On the computer, The process involves determining whether a specific person is included in the image, and determining whether a person other than the specific person is continuously captured in the image for a predetermined period of time or longer. If it is determined that the image contains the specific person, a processed image is generated based on the image in which the face of the specific person is indistinguishable, and if it is determined that a person other than the specific person is continuously captured in the image for a predetermined period of time, a processed image is generated that includes at least a part of the face of the person other than the specific person. A program that makes this possible.
16. Determine whether a specific person is included in the image, and determine whether a person other than the specific person is continuously captured in the image for a predetermined period of time. If it is determined that the image contains the specific person, a processed image is generated based on the image in which the face of the specific person is indistinguishable. If it is determined that a person other than the specific person is continuously captured in the image for a predetermined period of time, the processed image is generated in which at least a portion of the face of the person other than the specific person is included. Image generation method.