Electronic apparatus

The electronic device uniformly applies image processing to all detected individuals by analyzing and enhancing unprocessed areas, addressing the inconsistency in existing technologies and ensuring natural-looking images.

JP2024159041APending Publication Date: 2024-11-08CANON KK
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023074773
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-28
Publication Date
2024-11-08

Smart Images

  • Figure 2024159041000001_ABST
    Figure 2024159041000001_ABST
Patent Text Reader

Abstract

To provide an electronic apparatus capable of preferably performing an image processing of all of persons detected from an image, and in addition, obtaining an image not inducing sense of incongruity.SOLUTION: An electronic apparatus of the present invention, includes: detection means for detecting a subject region as a region of at least one part of a person from a captured image; analysis means for analyzing the subject region detected by the detection means; and processing means for performing, on the subject region having succeeded in the analysis of the analysis means, image processing in accordance with a result of an analysis, meanwhile, performing, on the subject region having not succeeded in the analysis of the analysis means, image processing based on the analysis result of the subject region having succeeded in the analysis.SELECTED DRAWING: Figure 5
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to electronic devices, and in particular to image processing. [Background technology]

[0002] Imaging devices such as digital cameras and smartphones (smartphone cameras) generally have not only still image capture functions but also video capture functions. In addition, with the spread of video distribution services via the Internet, it has become common for individuals to distribute live video. As a result, there are an increasing number of imaging devices that can not only write captured images to a recording medium but also distribute (live distribute) them to the outside.

[0003] Also proposed is an imaging device that is capable of detecting a human region from a captured image, performing image processing on that region, and performing live view display of the processed image.

[0004] Patent document 1 discloses a digital camera that performs facial recognition on captured images, determines conditions for skin beautifying processing for each recognized person based on the results of the facial recognition, and performs skin beautifying processing for each recognized person based on the determined conditions. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2010-211497 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, in the technology disclosed in Patent Document 1, skin beautification (image processing) is performed only on people whose faces have been successfully recognized, and skin beautification is not performed on people whose faces have not been successfully recognized (people who have simply been detected). As a result, people who have been subjected to skin beautification and people who have not been subjected to skin beautification are mixed together, and an unnatural image (an image in which the skin conditions of people are significantly different) may be obtained. In addition, an unnatural image may be obtained due to a change from a state in which skin beautification has not been performed to a state in which skin beautification has been performed. In particular, in situations where the situation changes in real time, such as during video shooting and live streaming, an unnatural image is likely to be obtained.

[0007] An object of the present invention is to provide an electronic device that can perform image processing appropriately for all people detected from an image, thereby obtaining an image that does not look unnatural. [Means for solving the problem]

[0008] The electronic device of the present invention is characterized by having a detection means for detecting a subject area which is at least a portion of a person from a captured image, an analysis means for analyzing the subject area detected by the detection means, and a processing means for performing image processing on subject areas which have been successfully analyzed by the analysis means in accordance with the results of the analysis, and for subject areas which have not been successfully analyzed by the analysis means in accordance with the analysis results of the subject areas which have been successfully analyzed. Effect of the Invention

[0009] According to the present invention, image processing can be suitably performed on all people detected from an image, and an image that does not look unnatural can be obtained. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram of a digital camera. [Diagram 2] FIG. 1 is an external view of a digital camera. [Diagram 3]FIG. 2 is a schematic diagram showing a screen to be displayed. [Figure 4] 13 is a flowchart of a moving image recording mode process. [Diagram 5] 13 is a flowchart of an image processing process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a block diagram showing the configuration of a digital camera 100 (imaging device) as an example of a device (electronic device) to which the present invention can be applied, and Fig. 2 is an external view of the digital camera 100.

[0012] In Fig. 1, the photographing lens 104 is a lens group including a zoom lens and a focus lens. The shutter 105 is a shutter with an aperture function. The imaging unit 106 is an imaging element (image sensor) composed of a CCD or CMOS element that converts an optical image into an electrical signal. The A / D converter 107 converts an analog signal output from the imaging unit 106 into a digital signal. The barrier 103 covers the imaging system including the photographing lens 104 of the digital camera 100 to prevent the imaging system including the photographing lens 104, the shutter 105, and the imaging unit 106 from becoming dirty or being damaged.

[0013] The image processing unit 102 performs a predetermined process (such as pixel interpolation, resizing such as reduction, and color conversion) on the data from the A / D converter 107 or the data from the memory control unit 108. The image processing unit 102 also performs a predetermined calculation process using the captured image data, and the system control unit 101 performs exposure control and distance measurement control based on the calculation results obtained by the image processing unit 102. This allows TTL (through-the-lens) AF (autofocus) processing, AE (autoexposure) processing, and EF (pre-flash) processing to be performed. The image processing unit 102 further performs a predetermined calculation process using the captured image data, and performs TTL AWB (auto white balance) processing based on the calculation results obtained. The image processing unit 102 can also perform image processing (image processing) such as skin correction, face correction, and contour correction. The method of these image processing is not particularly limited, and a known method may be applied as a method of image processing. The skin correction may be skin beautifying processing (imparting a skin beautifying effect), and the face correction may be resizing of facial organs such as the eyes, nose, mouth, and ears.

[0014] The memory control unit 108 controls the transmission and reception of data between the A / D converter 107, the image processing unit 102, and the memory 109. The output data from the A / D converter 107 is written to the memory 109 via the image processing unit 102 and the memory control unit 108. Alternatively, the output data from the A / D converter 107 is written to the memory 109 via the memory control unit 108 without via the image processing unit 102. The memory 109 stores image data obtained by the imaging unit 106 and converted into digital data by the A / D converter 107, and image data to be displayed on the display unit 111. The memory 109 has a storage capacity sufficient to store a predetermined number of still images and a predetermined period of moving images and audio. The memory 109 also serves as a memory for displaying images (video memory). The D / A converter 110 converts the image data for display stored in the memory 109 into an analog signal and supplies it to the display unit 111. The image data for display written in the memory 109 is then displayed on the display unit 111 via the D / A converter 110 .

[0015] The display unit 111 is a display such as an LCD or an organic EL display, and performs display according to an analog signal from the D / A converter 110. The digital signal that has been A / D converted by the A / D converter 107 and stored in the memory 109 is converted to analog by the D / A converter 110, and is sequentially transferred to and displayed on the display unit 111, thereby functioning as an electronic viewfinder and performing live view display (LV display). Hereinafter, an image displayed in live view display will be referred to as a live view. This is called a LV image.

[0016] The non-volatile memory 114 is an electrically erasable and recordable memory, such as an EEPROM. Constants and programs for the operation of the system control unit 101 are stored in the non-volatile memory 114. The programs referred to here are programs for executing various flowcharts described later.

[0017] The system control unit 101 is a control unit that is composed of at least one processor and / or at least one circuit, and controls the entire digital camera 100. The system control unit 101 is both a processor and a circuit. The system control unit 101 executes programs recorded in a non-volatile memory 114 to realize various processes described below. The system memory 112 is, for example, a RAM, and the system control unit 101 loads constants and variables for the operation of the system control unit 101, programs read from the non-volatile memory 114, and the like in the system memory 112. The system control unit 101 also performs display control by controlling the memory 109, the D / A converter 110, the display unit 111, and the like. The system timer 113 is a timekeeping unit that measures the time used for various controls and the time of a built-in clock.

[0018] The image processing unit 102 has a subject recognition unit 128. The subject recognition unit 128 detects a subject area having a predetermined characteristic, for example, an area of ​​at least a part of a person (a person, a face, a facial organ, etc.) from a live view image (detection process). The subject recognition unit 128 also analyzes the detected subject area. For example, the subject recognition unit 128 detects feature points from the detected face area (face area) and performs recognition process (face recognition) to recognize the characteristics of the person corresponding to the face area (for example, at least one of gender, age, race, and skin condition (roughness degree, etc.)). The subject recognition unit 128 also performs authentication process (face authentication) to compare (collate) the detected face area with a plurality of face information (a plurality of face information corresponding to a plurality of people) registered in advance in the memory 109 or the non-volatile memory 114 and identify the person corresponding to the detected face area. This makes it possible to perform AF on the detected subject, the recognized subject, or the authenticated subject, improving the convenience of shooting.

[0019] The detection process (face detection), the recognition process (face recognition), and the authentication process (face authentication) are not particularly limited in method, and any known method may be applied as the method. Although the authentication process (face authentication) may be performed regardless of whether the recognition process (face recognition) is successful or not, in this embodiment, the authentication process (face authentication) is performed only for the subject area where the recognition process (face recognition) is successful.

[0020] The shutter button 115, the mode switching dial 118, the power button 119, and the operation unit 120 are input units that accept operations from the user (user operations) and are used to input various operation instructions to the system control unit 101. The shutter button 115 includes a first shutter switch 116 and a second shutter switch 117.

[0021] The mode switching dial 118 is an operating member for switching the operation mode of the system control unit 101 to one of a still image recording (still image shooting) mode, a moving image recording (moving image shooting) mode, a playback mode, and the like.

[0022] The first shutter switch 116 is turned on by half-pressing (instruction of preparation for shooting) during operation of the shutter button 115, and generates a first shutter switch signal SW1. In response to the first shutter switch signal SW1, the system control unit 101 starts preparation operations for shooting, such as AF (autofocus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (pre-flash) processing.

[0023] The second shutter switch 117 is turned on when the operation of the shutter button 115 is completed, that is, when the shutter button 115 is fully pressed (photographing instruction), and generates a second shutter switch signal SW2. The system control unit 101 starts a series of photographing processing operations, from reading out a signal from the imaging unit 106 to writing the captured image to the recording medium 124 as an image file, based on the second shutter switch signal SW2.

[0024] The power supply control unit 121 is made up of a battery detection circuit, a DC-DC converter, a switch circuit for switching between blocks to which electricity is applied, and detects the state of the power button 119, whether a battery is installed, the type of battery, and the remaining battery level. The power supply control unit 121 also controls the DC-DC converter based on the detection results and instructions from the system control unit 101, and supplies the necessary voltage for the necessary period to each unit including the recording medium 124. The power button 119 is an operating member for switching the power supply of the digital camera 100 on and off.

[0025] The power supply unit 122 is composed of a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, etc. In this embodiment, a case where a secondary battery (hereinafter referred to as a battery) is used as the power supply unit 122 will be described.

[0026] The recording medium I / F 123 is an interface with a recording medium 124 such as a memory card or a hard disk. The recording medium 124 is a recording medium such as a memory card for recording captured images, and is composed of a semiconductor memory, a magnetic disk, or the like.

[0027] 2, the power supply unit 122 (battery) and the recording medium 124 can be inserted into the digital camera 100 from the bottom side of the digital camera 100. The insertion portions for the power supply unit 122 (battery) and the recording medium 124 can be covered with an openable cover 208.

[0028] The speaker 126 outputs sounds such as warning sounds, etc. The sound control unit 125 controls the sound output by the speaker 126.

[0029] The operation unit 120 includes one or more operation members. The operation members included in the operation unit 120 are assigned appropriate functions for each scene by selecting and operating various functional icons displayed on the display unit 111, and act as various function buttons. The function buttons include, for example, an end button, a back button, an image forward button, a jump button, a filter button, and an attribute change button. For example, the operation unit 120 includes a menu button 201, a four-way key 202, a SET button 203, a controller wheel 204, an electronic dial 205, a playback button 206, and a video button 207, as shown in FIG. 2.

[0030] The menu button 201 is used to display a menu screen, etc. The four-way key 202 is an operation member configured so that the up, down, left, and right parts can be pressed, and is used to select an item, etc. The SET button 203 is used to instruct the confirmation of a selected item, etc. When the menu button 201 is pressed, a menu screen on which various settings can be made is displayed on the display unit 111. The user can intuitively make various settings using the menu screen displayed on the display unit 111 and the four-way key 202 and the SET button 203. The controller wheel 204 and the electronic dial 205 are operation members that can be rotated, and are used to select an item, etc. The playback button 206 is used to switch the operation mode of the digital camera 100 between a shooting mode and a playback mode, etc. The movie button 207 is used to instruct the start and stop of movie shooting (movie recording), etc. When the movie button 207 is pressed, recording of the captured movie starts, and when the movie button 207 is pressed again, recording of the movie stops.

[0031] 3(A) to 3(C) show examples of screens according to this embodiment. The screen displays a live view image, and the results of face detection, face recognition, and face authentication are superimposed on the live view image.

[0032] The live view image in FIG. 3A shows a person 301 whose face authentication has been successful and a person 302 whose face authentication has not been successful but whose face recognition has been successful. A face frame 301a (a frame surrounding a face area) is displayed for the person 301, and a face frame 302a is displayed for the person 302. The face frame 301a and the face frame 302a are frames of the same type, and show the face (face area) of a person whose face recognition and / or face authentication have been successful. For the person 301 whose face authentication has been successful, a character string 301b (e.g., a person's name) corresponding to the person 301 (a person identified by face authentication) is also displayed. As described above, in this embodiment, face authentication is performed only for a person whose face recognition has been successful. Therefore, a user can know whether or not face recognition has been successful depending on whether a specific face frame is displayed, and can know whether or not face authentication has been successful depending on whether a specific character string is displayed. The face frame of a person whose face recognition has been successful and the face frame of a person whose face authentication has been successful may be different from each other.

[0033] In Fig. 3(A), the face region of person 301, whose face authentication was successful, has been subjected to image processing in accordance with the face authentication result (image processing registered in advance for person 301 (person identified by face authentication), i.e., image processing dedicated to person 301). The face region of person 302, whose face authentication was not successful but whose face recognition was successful, has been subjected to image processing in accordance with the face recognition result (for example, facial feature points).

[0034] Fig. 3(B) shows a state in which a new person 303 has entered the imaging range from the state in Fig. 3(A). As the face of person 303 is in profile, face detection has been successful, but neither face recognition nor face authentication has been successful. Therefore, face frame 303a different from face frames 301a, 302a is displayed for person 303. By seeing face frame 303a different from face frames 301a, 302a, the user can understand that only face detection has been successful for person 303 (neither face recognition nor face authentication has been successful).

[0035] In the conventional technology, image processing is not performed on the face area of ​​the person 303, and the persons 301 and 302 with image processing and the person 303 without image processing are mixed, resulting in an image that looks unnatural (an image in which the face states of the persons 301 and 302 and the person 303 are significantly different). In addition, an unnatural feeling may occur due to a change from a state in which image processing is not performed to a state in which image processing is performed (a sudden change in the face state). Therefore, in this embodiment, image processing is performed on the face area of ​​a person whose analysis is unsuccessful (a person whose analysis is unsuccessful in neither face recognition nor face authentication) based on the analysis result of a person whose analysis is successful (a person whose analysis is successful in at least one of face recognition and face authentication). This makes it possible to perform image processing appropriately on all the persons detected from the image, and ultimately to obtain an image that does not look unnatural (for example, an image in which the difference in the face state between a plurality of persons is small and an image in which a sudden change in the face state is suppressed). In FIG. 3B, the face region of person 303 has been subjected to image processing based on the image processing applied to the face region of person 301 and the image processing applied to the face region of person 302.

[0036] FIG. 3(C) shows a state in which person 303 faces forward from the state in FIG. 3(B) and face authentication of person 303 has been successful. Since face authentication of person 303 has been successful, the face frame has been changed from face frame 303a to face frame 303a' which is the same as face frames 301a and 302a, and character string 303b corresponding to person 303 is displayed. By looking at character string 303b (and face frame 303a'), the user can understand that face authentication of person 303 has been successful. Then, the image processing performed on the face region of person 303 is changed from image processing based on image processing of persons 301 and 302 to image processing dedicated to person 303. In FIG. 3(C), image processing dedicated to person 303 has been applied to the face region of person 303.

[0037] Fig. 4 is a flowchart of the movie recording mode processing performed by the digital camera 100. The movie recording mode processing of Fig. 4 is realized by the system control unit 101 expanding a program stored in the non-volatile memory 114 into the system memory 112 and executing it. For example, when the movie recording mode is set, the system control unit 101 starts the movie recording mode processing of Fig. 4. Note that in the movie recording mode, processing not included in the movie recording mode processing of Fig. 4 may also be executed.

[0038] In S401, the system control unit 101 determines whether or not the moving image button 207 has been pressed. If the moving image button 207 has been pressed, the process proceeds to S402, and if not, the process proceeds to S405.

[0039] In S402, the system control unit 101 determines whether or not a moving image is being recorded. If a moving image is being recorded, the process proceeds to S403, and if not (if the camera is on standby for shooting), the process proceeds to S404. In S403, the system control unit 101 stops recording the captured moving image. In S404, the system control unit 101 starts recording the moving image.

[0040] In S405, the system control unit 101 determines whether or not the setting for performing the image processing is valid. If it is valid, the process proceeds to S406, and if not, the process proceeds to S407. In S406, the system control unit 101 performs the image processing. The image processing is a process for realizing the operation described with reference to Figs. 3(A) to 3(C). Details of the image processing will be described later with reference to Fig. 5. Note that the image processing may be performed not only during the moving image recording (moving image shooting) mode, but also during, for example, the live distribution mode or the still image recording (still image shooting) mode. The live distribution mode is an operation mode in which a captured moving image is distributed (live distribution) to the outside. In a scene where the situation changes in real time, such as during moving image shooting and live distribution, it is easy to obtain an unnatural image, so it is preferable to perform the image processing.

[0041] In S407, the system control unit 101 determines whether or not a moving image is being recorded. If a moving image is being recorded, the process proceeds to S410, and if not (if the camera is waiting to record), the process proceeds to S408. In S408, the system control unit 101 determines whether or not the menu button 201 has been pressed. If the menu button 201 has been pressed, the process proceeds to S409, and if not, the process proceeds to S410. In S409, the system control unit 101 performs various settings in response to user operations.

[0042] In S410, the system control unit 101 determines whether or not an instruction to end the movie recording mode process has been given. For example, when the mode switching dial 118 is operated, that is, when an instruction to switch to another operation mode is given, the system control unit 101 determines that an instruction to end the movie recording mode process has been given. Also, when the power button 119 is operated, that is, when an instruction to turn off the power of the digital camera 100 is given, the system control unit 101 determines that an instruction to end the movie recording mode process has been given. If an instruction to end the movie recording mode process has been given, the system control unit 101 ends the movie recording mode process, and if not, the system control unit 101 proceeds to S401.

[0043] FIG. 5 is a flowchart of the image processing performed in S406 of FIG.

[0044] In S501, the system control unit 101 performs a subject detection process to detect a subject area (a subject area) from a live view image. In the subject detection process, various subject areas are detected.

[0045] In S502, the system control unit 101 determines whether or not a subject area was detected in S501. If a subject area was detected, the process proceeds to S503. If not, the process proceeds to image processing. The processes of S503 to S514 are performed for each subject region detected in S501. When the processes of S503 to S514 have been performed for all subject regions detected in S501, the image processing ends.

[0046] In S503, the system control unit 101 determines whether or not the subject region to be processed (target region) is a human face region (face detection). If the target region is a face region, the process proceeds to S504, and if not, the process of S503 to S514 for the target region ends.

[0047] In S504, the system control unit 101 detects feature points from the target area (face area) and performs recognition processing (face recognition) to recognize the features of the person corresponding to the target area (for example, at least one of gender, age, race, and skin condition (degree of roughness, etc.)).

[0048] In S505, the system control unit 101 determines whether or not the face recognition in S504 was successful. If the face recognition was successful, the process proceeds to S506, and if not, the process proceeds to S511.

[0049] In S506, the system control unit 101 compares (matches) the target area (face area) with face information of multiple people pre-registered in the memory 109 or the non-volatile memory 114, and performs an authentication process (face authentication) to identify the person corresponding to the target area.

[0050] In S507, the system control unit 101 determines whether or not the face authentication in S506 was successful. If the face authentication was successful, the process proceeds to S508, and if not, the process proceeds to S509.

[0051] In S508, the system control unit 101 performs image processing (image processing pre-registered for the person corresponding to the target area (person identified by face authentication)) on the target area (face area) in accordance with the result of the face authentication in S506.

[0052] In S509, the system control unit 101 performs image processing on the target area (face area) according to the result of the face recognition in S504. For example, a plurality of image processings corresponding to a plurality of features are predefined, and among the plurality of image processings, an image processing corresponding to the feature of the target area is performed on the target area. The plurality of image processings may include image processing for each gender, image processing for each age group, image processing for each race, or image processing for each skin condition. The plurality of image processings may include image processing for each combination of a plurality of features (for example, a combination of gender and age group).

[0053] In S510, the system control unit 101 stores information (processing information) related to the image processing performed in S508 or S509 in the system memory 112. The processing information indicates the levels of various corrections, such as skin correction, whitening correction, contour correction, makeup correction, and facial part correction. The processing information may indicate the state of the face after correction, such as the degree of skin roughness after correction and the degree of skin shine after correction.

[0054] If the face recognition in S504 is not successful, in S511, the system control unit 101 determines whether or not a moving image is being recorded. If a moving image is being recorded, the process proceeds to S512, and if not (if the camera is waiting to shoot), the process of S503 to S514 for the target area is terminated. In the image processing process of FIG. 5, the process of S511 is performed to permit the display of an unnatural live view image and to suppress the recording of an unnatural image (frame). In the image processing process in the live distribution mode, instead of the process of S511, it may be determined whether or not live distribution is being performed. Then, if live distribution is being performed, the process proceeds to S512, and if not, the process of S503 to S514 for the target area may be terminated. By doing so, it is possible to suppress the distribution of an unnatural image (frame). The process of S511 may be omitted so that the display of an unnatural live view image is also suppressed.

[0055] In S512, the system control unit 101 determines whether or not there is a face area for which the analysis has been successful (a face area for which at least one of face recognition and face authentication has been successful). In the image processing process of Fig. 5, face authentication is performed only for persons for whom face recognition has been successful, so in S512, the system control unit 101 determines whether or not there is a face area for which face recognition has been successful. If there is a face area for which face recognition has been successful, the process proceeds to S513, and if not, the process proceeds to S514.

[0056] In S513, the system control unit 101 performs image processing on face areas not successfully analyzed (face areas not successfully subjected to face recognition (and face authentication)) based on the processing information stored in the system memory 112 in S510.

[0057] For example, the system control unit 101 performs the same image processing on a face region that has not been successfully analyzed (a face region that has been successfully analyzed in at least one of face recognition and face authentication). When the analysis of multiple face regions has been successful, the system control unit 101 may perform an image processing (representative processing) that is representative of the multiple image processing performed on each of the multiple face regions that have been successfully analyzed, on the face region that has not been successfully analyzed. The representative processing may be an average image processing of the multiple image processings, or may be an image processing with the lowest strength (level of correction) among the multiple image processings, or may be an image processing with the highest strength among the multiple image processings.

[0058] When face authentication is successful for at least one of the multiple face regions for which analysis has been successful (face regions for which at least one of face recognition and face authentication has been successful), the same image processing as that performed on the face regions for which face authentication has been successful may be performed as the representative processing. The priority of the person identified by face authentication may be determined in advance. Then, when face authentication is successful for two or more face regions among the multiple face regions for which analysis has been successful, the same image processing as that performed on the face authentication with the highest priority among the two or more face regions for which face authentication has been successful may be performed as the representative processing. In this way, the representative processing may be an image processing representative of the multiple image processing performed on each of the multiple face regions for which face authentication has been successful. The representative processing may be an image processing that is an average of the multiple image processing performed on each of the multiple face regions for which face authentication has been successful. The representative processing may be an image processing with the lowest intensity among the multiple image processing performed on each of the multiple face regions for which face authentication has been successful. The representative processing may be an image processing with the highest intensity among the multiple image processing performed on each of the multiple face regions for which face authentication has been successful. The representative processing may be image processing that represents a plurality of image processings that are performed on a plurality of face regions where face authentication has not been successful but face recognition has been successful.

[0059] In S514, the system control unit 101 performs a predetermined image processing on face areas that have not been successfully analyzed (face areas that have not been successfully face recognition (and face authentication)). By performing some kind of image processing, it is expected that the unnaturalness of the image can be reduced compared to the case where no image processing is performed. If there is no face area that has been successfully analyzed (a face area that has been successful in at least one of face recognition and face authentication), the system control unit 101 does not need to perform image processing on face areas that have not been successfully analyzed.

[0060] When a user instructs still image capture, the user can wait until face recognition (or face authentication) is successful before instructing still image capture. Therefore, while a captured video is being recorded or distributed, image processing may be performed on face areas that have not been successfully analyzed, and when a still image is being captured, image processing may not be performed on face areas that have not been successfully analyzed.

[0061] In addition, the facial recognition of S504 was successful so that all facial regions were taken into account. It is preferable to perform the processes of S512 to S514 for each face area for which the face recognition in S504 was not successful after the processes of S506 to S510 have been completed for all face areas.

[0062] The various controls described above as being performed by the system control unit 101 may be performed by a single piece of hardware, or the entire device may be controlled by multiple pieces of hardware (e.g., multiple processors or circuits) sharing the processing.

[0063] In addition, although the embodiments of the present invention have been described in detail, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Furthermore, each of the above-mentioned embodiments merely shows one embodiment of the present invention, and each embodiment can be appropriately combined.

[0064] In the above-mentioned embodiment, the present invention is applied to a digital camera (imaging device), but the present invention is not limited to this example and can be applied to any electronic device capable of processing images. For example, the present invention can be applied to personal computers, PDAs, mobile phone terminals, portable image viewers, printers, digital photo frames, music players, game consoles, electronic book readers, etc. In addition, the present invention can be applied to video players, display devices (including projection devices), tablet terminals, smartphones, AI speakers, home appliances, in-vehicle devices, etc.

[0065] The present invention is also applicable not only to the imaging device itself, but also to a control device that communicates with an imaging device (including a network camera) via wired or wireless communication and remotely controls the imaging device. Examples of devices that remotely control an imaging device include a smartphone, a tablet PC, and a desktop PC. The imaging device can be remotely controlled by notifying the control device of commands that cause the imaging device to perform various operations and settings based on operations or processes performed on the control device. Also, a live view image captured by the imaging device may be received via wired or wireless communication and displayed on the control device.

[0066] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0067] The disclosure of the present embodiment includes the following configuration, method, program, and medium. (Configuration 1) A detection means for detecting a subject area, which is at least a part of a person's area, from a captured image; an analysis means for analyzing the subject area detected by the detection means; a processing means for performing image processing on a subject region that has been successfully analyzed by the analysis means in accordance with the result of the analysis, and for performing image processing on a subject region that has not been successfully analyzed by the analysis means based on the analysis result of the subject region that has been successfully analyzed; 1. An electronic device comprising: (Configuration 2) The subject area is a face area. 2. The electronic device according to configuration 1. (Configuration 3) The analysis by the analysis means includes a recognition process for recognizing features of a person corresponding to the subject area. 3. The electronic device according to configuration 1 or 2. (Configuration 4) The characteristics of the person include at least one of gender, age, race, and skin condition. 4. The electronic device according to configuration 3. (Configuration 5) 4. The electronic device according to any one of configurations 1 to 3, wherein the analysis by the analysis means includes an authentication process for identifying a person corresponding to the subject region. (Configuration 6) the analysis means performs a recognition process for recognizing features of a person corresponding to the subject region, and an authentication process for identifying the person corresponding to the subject region; The processing means is For the subject area for which the authentication process has been successful, image processing is performed in accordance with the result of the authentication process; For a subject area where the authentication process is not successful but the recognition process is successful, image processing is performed according to the result of the recognition process; The subject area that has not been successfully analyzed by the analysis means is a subject area that has not been successfully analyzed in either the recognition process or the authentication process. 6. The electronic device according to any one of configurations 1 to 5. (Configuration 7) The processing means performs the same image processing on the subject region that has not been successfully analyzed by the analysis means as the image processing performed on the subject region that has been successfully analyzed by the analysis means. 7. The electronic device according to any one of configurations 1 to 6. (Configuration 8) When the analysis means has succeeded in analyzing a plurality of subject regions, the processing means performs image processing on the subject regions not successfully analyzed by the analysis means, the image processing performing image processing on the plurality of subject regions, the image processing performing image processing on the subject regions being processed by the analysis means. 8. The electronic device according to any one of configurations 1 to 7. (Configuration 9) The image processing representative of the plurality of image processings is an average image processing of the plurality of image processings. 9. The electronic device according to configuration 8. (Configuration 10) The image processing that represents the plurality of image processings is the image processing with the lowest intensity among the plurality of image processings. 9. The electronic device according to configuration 8. (Configuration 11) The image processing that represents the plurality of image processings is the image processing that has the highest intensity among the plurality of image processings. 9. The electronic device according to configuration 8. (Configuration 12) the analysis means performs a recognition process for recognizing features of a person corresponding to the subject region, and an authentication process for identifying the person corresponding to the subject region; The processing means is For the subject area for which the authentication process has been successful, image processing is performed in accordance with the result of the authentication process; For a subject area where the authentication process is not successful but the recognition process is successful, image processing is performed according to the result of the recognition process; The subject area that has not been successfully analyzed by the analysis means is a subject area that has not been successfully analyzed in either the recognition process or the authentication process, When the authentication process is successful for at least one of the plurality of subject regions, the processing means performs, as an image processing representative of the plurality of image processing, the same image processing as that performed for the subject region for which the authentication process is successful. 12. The electronic device according to any one of configurations 8 to 11. (Configuration 13) A priority of a person identified by the authentication process is determined in advance; When the authentication process is successful for two or more of the plurality of subject regions, the processing means performs, as an image processing representative of the plurality of image processing, the same image processing as that performed for the subject region having the highest priority among the two or more subject regions for which the authentication process is successful. 13. The electronic device according to configuration 12. (Configuration 14) The electronic device described in any one of configurations 1 to 13, characterized in that when there is no subject area that has been successfully analyzed by the analysis means, the processing means performs a predetermined image processing on the subject area that has not been successfully analyzed by the analysis means. (Configuration 15) When there is no object region that has been successfully analyzed by the analyzing means, the processing means does not process the image for the object region that has not been successfully analyzed by the analyzing means. 14. The electronic device according to any one of configurations 1 to 13. (Configuration 16) The processing means is During recording or distribution of the captured video, image processing is performed on subject areas that have not been successfully analyzed by the analysis means; When taking a still image, no image processing is performed on the subject area that has not been successfully analyzed by the analyzing means. 16. The electronic device according to any one of configurations 1 to 15. (method) A detection step of detecting a subject area, which is at least a part of a person area, from the captured image; an analysis step of analyzing the subject area detected in the detection step; a processing step of performing image processing on a subject region that has been successfully analyzed in the analysis step in accordance with the result of the analysis, and performing image processing on a subject region that has not been successfully analyzed in the analysis step based on the analysis result of the subject region that has been successfully analyzed; 13. A method for controlling an electronic device comprising: (program) A program for causing a computer to function as each of the means of the electronic device according to any one of configurations 1 to 16. (medium) A computer-readable storage medium storing a program for causing a computer to function as each of the means of the electronic device according to any one of configurations 1 to 16. [Explanation of symbols]

[0068] 100: Digital camera 101: System control unit 102: Image processing unit 128: Object recognition unit

Claims

1. A detection means for detecting a subject area, which is at least a part of a person's area, from a captured image; an analysis means for analyzing the subject area detected by the detection means; a processing means for performing image processing on a subject region that has been successfully analyzed by the analysis means in accordance with the result of the analysis, and for performing image processing on a subject region that has not been successfully analyzed by the analysis means based on the analysis result of the subject region that has been successfully analyzed; 1. An electronic device comprising:

2. The subject area is a face area.

2. The electronic device according to claim 1 .

3. The analysis by the analysis means includes a recognition process for recognizing features of a person corresponding to the subject area.

2. The electronic device according to claim 1 .

4. The characteristics of the person include at least one of gender, age, race, and skin condition.

4. The electronic device according to claim 3.

5. 2. The electronic device according to claim 1, wherein the analysis by the analysis means includes an authentication process for identifying a person corresponding to the subject area.

6. the analysis means performs a recognition process for recognizing features of a person corresponding to the subject region, and an authentication process for identifying the person corresponding to the subject region; The processing means is For the subject area for which the authentication process has been successful, image processing is performed in accordance with the result of the authentication process; For a subject area where the authentication process is not successful but the recognition process is successful, image processing is performed according to the result of the recognition process; The subject area that has not been successfully analyzed by the analysis means is a subject area that has not been successfully analyzed in either the recognition process or the authentication process.

2. The electronic device according to claim 1 .

7. The processing means performs the same image processing on a subject region that has not been successfully analyzed by the analysis means as the image processing performed on a subject region that has been successfully analyzed by the analysis means.

2. The electronic device according to claim 1 .

8. When the analysis means has succeeded in analyzing a plurality of subject regions, the processing means performs image processing on the subject regions not successfully analyzed by the analysis means, the image processing performing image processing on the plurality of subject regions, the image processing performing image processing on the subject regions being processed by the analysis means.

2. The electronic device according to claim 1 .

9. The image processing representative of the plurality of image processings is an average image processing of the plurality of image processings.

9. The electronic device according to claim 8.

10. The image processing that represents the plurality of image processings is the image processing with the lowest intensity among the plurality of image processings.

9. The electronic device according to claim 8.

11. The image processing that represents the plurality of image processings is the image processing that has the highest intensity among the plurality of image processings.

9. The electronic device according to claim 8.

12. the analysis means performs a recognition process for recognizing features of a person corresponding to the subject region, and an authentication process for identifying the person corresponding to the subject region; The processing means is For the subject area for which the authentication process has been successful, image processing is performed in accordance with the result of the authentication process; For a subject area where the authentication process is not successful but the recognition process is successful, image processing is performed according to the result of the recognition process; The subject area that has not been successfully analyzed by the analysis means is a subject area that has not been successfully analyzed in either the recognition process or the authentication process, When the authentication process is successful for at least one of the plurality of subject regions, the processing means performs, as an image processing representative of the plurality of image processing, the same image processing as that performed for the subject region for which the authentication process is successful.

9. The electronic device according to claim 8.

13. A priority of a person identified by the authentication process is determined in advance; When the authentication process is successful for two or more of the plurality of subject regions, the processing means performs, as an image processing representative of the plurality of image processing, the same image processing as that performed for the subject region having the highest priority among the two or more subject regions for which the authentication process is successful.

13. The electronic device according to claim 12.

14. 2. The electronic device according to claim 1, wherein, when there is no subject area that has been successfully analyzed by the analysis means, the processing means performs a predetermined image processing on the subject area that has not been successfully analyzed by the analysis means.

15. When there is no object region that has been successfully analyzed by the analyzing means, the processing means does not process the image for the object region that has not been successfully analyzed by the analyzing means.

2. The electronic device according to claim 1 .

16. The processing means is During recording or distribution of the captured video, image processing is performed on subject areas that have not been successfully analyzed by the analysis means; When taking a still image, no image processing is performed on the subject area that has not been successfully analyzed by the analyzing means.

2. The electronic device according to claim 1 .

17. A detection step of detecting a subject area, which is at least a part of a person area, from the captured image; an analysis step of analyzing the subject area detected in the detection step; a processing step of performing image processing on a subject region that has been successfully analyzed in the analysis step in accordance with the result of the analysis, and performing image processing on a subject region that has not been successfully analyzed in the analysis step based on the analysis result of the subject region that has been successfully analyzed; 13. A method for controlling an electronic device comprising:

18. A program for causing a computer to function as each of the means of the electronic device according to any one of claims 1 to 16.

19. A computer-readable storage medium storing a program for causing a computer to function as each of the means of the electronic device according to any one of claims 1 to 16.

Citation Information

Patent Citations

  • Digital camera and image processing program

    JP2010211497A