Imaging apparatus, method, and system

Abstract

To provide an imaging apparatus, method, and system capable of easily obtaining a desired image effect for an imaged subject.SOLUTION: An imaging apparatus (100) includes: an imaging part (115) for imaging a subject image and generating a captured image (20); an image processing part (120) for performing image processing to the captured image generated by the imaging part; and a control part (135) for controlling the image processing by the image processing part on the basis of the captured image. The control part detects a subject in a predetermined state from the captured image (S12). When the subject in the predetermined state is detected (YES in the S12), the control part allows the image processing part to perform the image processing, so as to generate a processed image (40) to which an image effect which is different between the inside and the outside of a subject area corresponding to the subject in the predetermined state in the captured image is imparted (S14).SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to an imaging device, method, and system for performing image processing on captured images. [Background technology]

[0002] Patent Document 1 discloses an imaging device that aims to enable a user to easily perform color conversion during shooting, retaining a color freely designated by the user and erasing all other color components. In this imaging device, the user's designated color is determined based on color information contained in a predetermined area of ​​an image displayed on an electronic viewfinder, which displays an image based on image data captured by an imaging unit and output from an image processing unit. The imaging device then sets parameters for the image processing unit to perform color conversion that erases all colors other than the designated color. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-211525 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides imaging devices, methods, and systems that can facilitate achieving desired image effects for imaged subjects. [Means for solving the problem]

[0005] In the present disclosure, an imaging device includes an imaging unit that captures an image of a subject and generates a captured image, an image processing unit that performs image processing on the captured image generated by the imaging unit, and a control unit that controls the image processing by the image processing unit based on the captured image. The control unit detects a subject in a predetermined state from the captured image. When the control unit detects a subject in the predetermined state, it causes the image processing unit to perform image processing so as to generate a processed image in which different image effects are applied between the inside and outside of a subject area in the captured image that corresponds to the subject in the predetermined state.

[0006] In the present disclosure, an imaging method is an imaging method including a step of an imaging unit capturing an image of a subject and generating a captured image, a step of an image processing unit performing image processing on the captured image generated by the imaging unit, and a step of a control unit controlling the image processing by the image processing unit based on the captured image. In this method, the control unit detects a subject in a predetermined state from the captured image, and when the subject in the predetermined state is detected, causes the image processing unit to perform image processing so as to generate a processed image in which different image effects are applied between the inside and outside of a subject area in the captured image corresponding to the subject in the predetermined state.

[0007] In the present disclosure, an imaging system includes an imaging device and an image processing device. The imaging device includes an imaging unit that captures an image of a subject and generates a captured image, and a transmission unit that transmits image data representing the captured image to the image processing device. The image processing device includes a receiving unit that receives image data from the imaging device and an image processing unit that performs image processing on the captured image represented by the image data based on the image data received by the receiving unit. The imaging device or the image processing device includes a control unit that detects a subject in a predetermined state from the captured image, and, when the subject in the predetermined state is detected, causes the image processing unit to perform image processing so as to generate a processed image in which different image effects are applied between the inside and outside of a subject area corresponding to the subject in the predetermined state in the captured image. [Effects of the Invention]

[0008] The imaging device, method, and system disclosed herein can facilitate achieving a desired image effect for an imaged subject. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an imaging system according to a first embodiment of the present disclosure. [Figure 2] FIG. 1 is a diagram illustrating the configuration of a digital camera in an imaging system; [Figure 3] FIG. 1 is a diagram illustrating a configuration of an image processing server in an imaging system. [Figure 4] FIG. 1 is a diagram for explaining the operation of the imaging system according to the first embodiment. [Figure 5] 1 is a flowchart illustrating an example of the operation of a digital camera in the imaging system of the first embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of the data structure of shot information in the imaging system. [Figure 7] FIG. 1 is a diagram for explaining a detection operation of a subject area in an imaging system. [Figure 8] 10 is a flowchart illustrating an example of automatic editing processing in the imaging system of the first embodiment. [Figure 9] FIG. 1 is a diagram for explaining image processing in an imaging system. [Figure 10] 10 is a flowchart illustrating the operation of a digital camera in an imaging system according to a second embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0011] (Embodiment 1) 1. Configuration An imaging system according to a first embodiment of the present disclosure will be described with reference to FIG.

[0012] 1, an imaging system 10 according to this embodiment includes a digital camera 100, an image processing server 200, and an image editing terminal 300. In this system 10, the image processing server 200 is, for example, a cloud server, and is connected to the digital camera 100 and the image editing terminal 300 via a communication network 15 such as the Internet so as to enable data communication. The digital camera 100 and the image editing terminal 300 may also be connected via a wired or wireless connection so as to enable data communication.

[0013] This system 10 can be used, for example, when a user creates a desired video work by shooting and editing multiple videos using a digital camera 100. This system 10 provides useful information support for a series of workflows, such as when a user creates a scenario that outlines the concept of the video work, repeatedly shoots videos according to multiple cuts that divide the scenario, and edits the multiple shot videos.

[0014] The image editing terminal 300 may be various information terminals such as a personal computer (PC), a smartphone, or a tablet terminal. In the present system 10, the image editing terminal 300 may be omitted. Alternatively, the image editing terminal 300 may be an example of an image processing device. The image editing terminal 300 and the image processing server 200 may be configured integrally.

[0015] 1-1.Digital camera configuration The configuration of the digital camera 100 according to this embodiment will be described with reference to FIG.

[0016] 2 is a diagram illustrating an example of the configuration of a digital camera 100 in the system 10. The digital camera 100 is an example of an imaging device in this embodiment. The digital camera 100 of this embodiment includes an image sensor 115, an image processing engine 120, a display monitor 130, and a control unit 135. The digital camera 100 also includes a buffer memory 125, a card slot 140, a flash memory 145, an operation unit 150, a communication module 155, and a microphone 160. The digital camera 100 also includes, for example, an optical system 110 and a lens drive unit 112.

[0017] The optical system 110 includes a focus lens, a zoom lens, an optical image stabilization lens (OIS), an aperture, a shutter, etc. The focus lens is a lens for changing the focus state of the subject image formed on the image sensor 115. The zoom lens is a lens for changing the magnification of the subject image formed by the optical system. Each of the focus lens, etc. is composed of one or more lenses.

[0018] Lens driving unit 112 drives the focus lens and the like in optical system 110. Lens driving unit 112 includes a motor, and moves the focus lens along the optical axis of optical system 110 under the control of control unit 135. The configuration for driving the focus lens in lens driving unit 112 can be realized by a DC motor, a stepping motor, a servo motor, an ultrasonic motor, or the like.

[0019] The image sensor 115 captures an image of a subject formed via the optical system 110 and generates imaging data. The imaging data constitutes image data representing an image captured by the image sensor 115. The image sensor 115 generates new frame image data at a predetermined frame rate (e.g., 30 frames / second). The timing of generating imaging data and the electronic shutter operation in the image sensor 115 are controlled by the control unit 135. The image sensor 115 can be any of a variety of image sensors, such as a CMOS image sensor, a CCD image sensor, or an NMOS image sensor.

[0020] The image sensor 115 performs operations such as capturing still images and through images. Through images are mainly moving images, and are displayed on the display monitor 130 so that the user can determine the composition for capturing the still image. The through images and still images are each an example of a captured image in this embodiment. The image sensor 115 is an example of an imaging unit in this embodiment.

[0021] The image processing engine 120 performs various processes on the imaging data output from the image sensor 115 to generate image data, and performs various processes on the image data to generate an image to be displayed on the display monitor 130. The various processes include, but are not limited to, white balance correction, gamma correction, YC conversion processing, electronic zoom processing, compression processing, and expansion processing. The image processing engine 120 may be configured with a hardwired electronic circuit, or may be configured with a microcomputer, processor, or the like using a program.

[0022] In this embodiment, the image processing engine 120 includes an image recognition unit 122 that realizes a function for detecting an area where a specific subject, such as a person, is located by image recognition of a captured image. The image recognition unit 122 includes a trained model constructed to perform image recognition such as segmentation using machine learning such as a neural network. The image processing engine 120 may be configured integrally with the control unit 135, or the image processing engine 120 may be omitted from the digital camera 100.

[0023] The display monitor 130 is an example of a display unit that displays various information. For example, the display monitor 130 displays an image (through image) represented by image data captured by the image sensor 115 and processed by the image processing engine 120. The display monitor 130 also displays a menu screen or the like that allows the user to make various settings for the digital camera 100. The display monitor 130 can be configured, for example, by a liquid crystal display device or an organic EL device.

[0024] The operation unit 150 is a general term for user interfaces such as hard keys, software keys, etc., such as operation buttons and operation levers provided on the exterior of the digital camera 100, and accepts operations by the user. The operation unit 150 includes, for example, a release button, a mode dial, and a touch panel. When the operation unit 150 accepts an operation by the user, it transmits an operation signal corresponding to the user operation to the control unit 135. The operation unit 150 is an example of a setting unit of the digital camera 100 in this embodiment.

[0025] The control unit 135 controls the overall operation of the digital camera 100. The control unit 135 includes a CPU and the like, and the CPU executes a program (software) to realize predetermined functions. Instead of a CPU, the control unit 135 may include a processor configured with a dedicated electronic circuit designed to realize the predetermined functions. In other words, the control unit 135 can be realized by various processors such as a CPU, MPU, GPU, DSP, FPGA, ASIC, etc. The control unit 135 may be configured with one or more processors. Furthermore, the control unit 135 may be configured on a single semiconductor chip together with the image processing engine 120, etc.

[0026] The buffer memory 125 is a recording medium that functions as a work memory for the image processing engine 120 and the control unit 135. The buffer memory 125 is realized by a DRAM (Dynamic Random Access Memory) or the like. The flash memory 145 is a non-volatile recording medium. Although not shown, the control unit 135 may have various types of internal memory, for example, a built-in ROM. The ROM stores various programs executed by the control unit 135. The control unit 135 may also have a built-in RAM that functions as a work area for the CPU.

[0027] The card slot 140 is a means for inserting a removable memory card 142. The card slot 140 can electrically and mechanically connect the memory card 142. The memory card 142 is an external memory equipped with a recording element such as a flash memory inside. The memory card 142 can store data such as image data generated by the image processing engine 120.

[0028] The communication module 155 is a module (circuit) that connects to an external device in accordance with a predetermined communication standard for wired or wireless communication. The predetermined communication standard includes, for example, USB, HDMI (registered trademark), IEEE802.11, Wi-Fi, Bluetooth, etc. The digital camera 100 can communicate with other devices via the communication module 155.

[0029] The microphone 160 includes, for example, multiple microphone elements built into the digital camera 100. The microphone 160 outputs an audio signal indicating audio picked up on multiple channels to the control unit 135 so that, for example, the direction from which the picked-up audio is coming can be estimated. The digital camera 100 may also use an external microphone 160. Instead of or in addition to the built-in microphone 160, the digital camera 100 may also include a connection unit such as a terminal for connecting to the external microphone 160.

[0030] 1-2. Image processing server configuration The configuration of the image processing server 200 in this embodiment will be described with reference to FIG.

[0031] 3 is a diagram illustrating an example of the configuration of the image processing server 200. The image processing server 200 illustrated in FIG.

[0032] The control unit 210 includes, for example, a CPU or MPU that cooperates with software to realize predetermined functions. The control unit 210 controls, for example, the overall operation of the image processing server 200. The control unit 210 reads data and programs stored in the storage unit 220 and performs various arithmetic processing to realize various functions.

[0033] The control unit 210 executes, for example, a program including a set of instructions for implementing each of the above functions. The program may be provided via the communication network 15 or may be stored on a portable recording medium. The control unit 210 may also be a hardware circuit such as a dedicated electronic circuit or a reconfigurable electronic circuit designed to implement each of the above functions. The control unit 210 may also be configured with various semiconductor integrated circuits such as a CPU, MPU, GPU, GPGPU, TPU, microcomputer, DSP, FPGA, and ASIC.

[0034] The storage unit 220 is a storage medium that stores programs and data necessary to realize the functions of the image processing server 200. The storage unit 220 includes a storage unit 221 and a temporary storage unit 222, as shown in FIG.

[0035] The storage unit 221 stores parameters, data, control programs, etc. for realizing predetermined functions. The storage unit 221 is configured, for example, with an HDD or SSD. For example, the storage unit 221 stores the above-mentioned programs and various image data, etc.

[0036] The temporary storage unit 222 is configured with a RAM such as a DRAM or an SRAM, and temporarily stores (i.e., holds) data. For example, the temporary storage unit 222 holds image data that is in the middle of being edited. The temporary storage unit 222 may also function as a work area for the control unit 210, and may be configured as a storage area in the internal memory of the control unit 210.

[0037] The communication unit 250 is a module (circuit) that connects to an external device in accordance with a predetermined communication standard for wired or wireless communication. The predetermined communication standard includes, for example, USB, HDMI, IEEE802.11, Wi-Fi, Bluetooth, etc. The communication unit 250 may connect the image processing server 200 to a communication network such as the Internet. The communication unit 250 is an example of an acquisition unit that receives various information from an external device or a communication network.

[0038] The image processing unit 260 generates processed image data by performing various types of image processing on image data received from, for example, the communication unit 250. The image processing unit 260 may be configured with a hardwired electronic circuit, or may be configured with a microcomputer, processor, or the like using a program.

[0039] The above-described configuration of the image processing server 200 is an example, and the configuration of the image processing server 200 is not limited to this. For example, the control unit 210 and the image processing unit 260 may be configured integrally. Furthermore, the acquisition unit in the image processing server 200 may be realized by cooperation with various software in the control unit 210, etc. The input unit in the image processing server 200 may acquire various pieces of information by reading out the various pieces of information stored in various storage media (e.g., the storage unit 221) into a working area (e.g., the temporary storage unit 222) of the control unit 210.

[0040] 2.Operation The operation of the system 10 configured as above will be described below.

[0041] The system 10 performs image processing such as color preservation processing that preserves color only in a specific subject, using, for example, the subject recognition results in the digital camera 100. An overview of the operation of the system 10 will be described with reference to FIG.

[0042] 4 is a diagram illustrating the operation of the imaging system 10 of the embodiment 1. The imaging system 10 determines the subject for which color is to be retained depending on the state of the subject, and colors a person in a predetermined state, such as speaking, singing, laughing, or gesturing, when shooting a video of a movie, a drama, a talk show, or a live performance.

[0043] Fig. 4(A) shows an example of a captured image 20 in the digital camera 100. Fig. 4(B) shows an example of a processed image 40 resulting from image processing performed on the captured image 20 in Fig. 4(A) in the system 10. Fig. 4(C) shows an example of a captured image 20 in which the state of the subject has changed from that in Fig. 4(A). Fig. 4(D) shows an example of a processed image 40 resulting from processing the captured image 20 in Fig. 4(C).

[0044] 4(A) and 4(C) show captured images 20 of different frames in a scene in which two people 31 and 32 are having a conversation as subjects in a video (hereinafter also referred to as the "original video") captured by the digital camera 100. The digital camera 100 of this embodiment sequentially detects person areas R31 and R32 that follow the outlines of the people 31 and 32, for example, while capturing the original video. These person areas R31 and R32 are examples of subject areas used for various types of capture control in the digital camera 100, such as AF (autofocus operation) or AE (auto exposure determination).

[0045] The system 10 uses the detection results of these person regions R31, R32 to perform image processing on the captured image 20 so as to highlight a subject in a predetermined state, such as a speaking person 31, as shown in Figures 4(A) and (B), to generate a processed image 40. The image processing by the system 10 is, for example, a color-preserving process that preserves the color inside the person region R31 corresponding to the speaking person 31 while achromatizing the area outside this person region R31.

[0046] For example, the system 10 detects a subject in the above-mentioned predetermined state for each frame of the original video (FIGS. 4(A) and (C)), and generates a video (FIGS. 4(B) and (D)) including a processed image 40 in which a subject to be highlighted is dynamically determined from the captured image 20. For example, from the captured image 20 of a frame in which the speaker has changed from the state of FIG. 4(A) to that of FIG. 4(C), a processed image 40 is obtained in which the person 32 who has just started speaking is highlighted in place of the person 31 who has stopped speaking, as shown in FIG. 4(D).

[0047] As described above, the system 10 determines the subject to which the highlighting visual effect is applied in accordance with the dynamically changing state of the subject, such as when the people 31 and 32 are speaking. This allows the system 10 to easily add the user's desired visual expression to the video they have shot. An example of applying the operation of the system 10 to a video production workflow will now be described.

[0048] 2-1. Overall operation The overall operation of the digital camera 100 in the present system 10 will be described with reference to FIGS.

[0049] Fig. 5 is a flowchart illustrating an example of the operation of the digital camera 100 in the present system 10. In this embodiment, an example of the operation of the digital camera 100 to perform image processing as illustrated in Figs. 4(A) to (D) will be described. Fig. 6 illustrates an example of the data structure of the shot information D1 in the present system 10.

[0050] First, in response to, for example, a user's operation, the control unit 135 of the digital camera 100 acquires cut division information D1 that manages a scenario desired by the user for each cut in the system 10 (S1). Fig. 6 shows an example of the cut division information D1 in the system 10.

[0051] As shown in Fig. 6, the cut information D1 manages a "script," "composition," "shooting time," "shooting location," "editing information," and "shooting completion flag" in association with each other for each "cut number." The cut information D1 is an example of management information in this embodiment. The cut information D1 is stored, for example, in the flash memory 145 of the digital camera 100.

[0052] "Cut number" indicates identification information of cuts corresponding to various shooting scenes in the scenario desired by the user. "Script" stores text information such as a script divided into portions for the corresponding cut in the scenario. "Composition" stores image information indicating the composition of the video recording of the corresponding cut. Image information may be input by drawing through user operation or by specifying image data.

[0053] "Shooting time" stores numerical information indicating the estimated length of time it will take to shoot the video of that cut. "Shooting location" stores location information indicating the location where the video of that cut will be shot. "Memo" stores various information desired by the user regarding the shooting of the video of that cut, such as the shooting equipment.

[0054] The "editing information" includes various information indicating the plans to edit the video of the cut using image processing, including, for example, identification information of the subject to be subjected to color preservation processing, etc. The "editing information" may also include the state of the subject to be edited (e.g., "talking") and the type of image processing (e.g., "color deepening processing"), etc.

[0055] The "shooting completion flag" is used to manage whether the shooting of the cut has been completed or not by setting it to ON / OFF. For example, in the initial state of the cut division information D1, the shooting completion flag is set to OFF for all cuts.

[0056] In the present system 10, the cut information D1 as described above may be created in the image editing PC 300 in response to a user operation, or may be managed in the image processing server 200, for example.

[0057] 5, such cut information D1 may be input to digital camera 100 via data communication with image editing terminal 300 or image processing server 200. Alternatively, the user may input cut information D1 to digital camera 100 via a portable recording medium such as memory card 142. Furthermore, digital camera 100 may be provided with a user interface for assisting in the creation of cut information D1.

[0058] Next, for example, the control unit 135 of the digital camera 100 manages the video shooting by the user for each cut by referring to the acquired cut division information D1 (S2). For example, in step S2, the control unit 135 displays a selection screen for multiple cuts included in the cut division information D1 on the display monitor 130, accepts a user operation on the operation unit 150 to select a cut from the selection screen, and shoots and records the video in association with the selected cut.

[0059] In step S2, an original video including captured images 20 such as those shown in Figures 4(A) and (C) is shot. For example, digital camera 100 detects a subject area during this video shooting (S2). The subject area detection operation in system 10 will be described with reference to Figure 7.

[0060] 7 illustrates an example of an area division map M20 of the detection result for the captured image 20 in Fig. 4(A). For example, the image recognition unit 122 of the digital camera 100 inputs the captured image 20 for each frame from the image sensor 115 into a trained model of image recognition such as segmentation technology, and sequentially generates an area division map M20 from the output of the trained model (S2).

[0061] The trained model is machine-learned to calculate, for example, a probability value for each pixel in an input image as to whether a specific type of subject, such as a person, is present. For example, in step S2, the image recognition unit 122 performs threshold determination on the probability value for each pixel in the output of the trained model to generate a region division map M20. Using this region division map M20, it is possible to detect, for example, person regions R31 and R32, each having a shape that follows the outline of each person 31 and 32, as shown in FIG. 7, corresponding to each person 31 and 32 appearing in the captured image 20 of FIG. 4(A).

[0062] The digital camera 100 of this embodiment can use, for example, this region division map M20 to use either of the person regions R31, R32 as the region for calculating an evaluation value for AF operation, or for various other shooting control (e.g., AE control) (S2). Furthermore, the control unit 135 of the digital camera 100 stores the detection results, such as the region division map M20 for each frame, in the buffer memory 125 in step S2. Alternatively, the control unit 135 may record information about the detection results in the metadata of the video file. Furthermore, during video shooting for each cut, the control unit 135 of the digital camera 100 collects sound, for example, using the microphone 160, and includes the sound data along with the image data in the recorded video file.

[0063] Next, the control unit 135 performs automatic editing image processing (S3) on the video shot in step S2, for example, based on the editing information in the cut information D1. In the automatic editing processing (S3) of this embodiment, a predetermined state of a subject specified in the editing information is detected from the original video, for example, as shown in Figures 4(A) and (C), and image processing is performed using person areas R31 and R32, as shown in Figures 4(B) and (D). The processing of step S3 will be described in detail later.

[0064] Next, for example, the control unit 135 of the digital camera 100 determines whether the user's evaluation of the video of the cuts shot in step S2 is "NG (No Good)" (S4). "NG" indicates, for example, that the user has determined that the video will not be used in a video production.

[0065] For example, in step S4, the control unit 135 causes the video resulting from the automatic editing process (S3) to be played back and displayed on the display monitor 130, and accepts a user operation on the operation unit 150 in which the user selects an evaluation of the video shooting (S4). For example, the digital camera 100 presents the user with two options for evaluating the video shooting: "OK" or "NG." "OK" indicates, for example, that the user has judged the video to be suitable for use in a video work. These evaluation options are not limited to the above two options, and for example, a further option, "KEEP," between "OK" and "NG" may be presented.

[0066] If the user's evaluation is NG (YES in S4), for example, the control unit 135 performs a process of re-shooting and recording the video associated with the NG cut (S5), and performs an automatic editing process (S3) on the re-shot video. Instead of the process of step S5, the control unit 135 may notify the user to urge them to re-shoot the NG cut, and then proceed to step S6. In this case, it may be up to the user to decide whether or not to shoot the cut thereafter.

[0067] On the other hand, if the user's evaluation is not NG (NO in S4), the control unit 135 determines whether or not video shooting of all cuts has been completed, for example, by referring to the shooting completion flag in the cut division information D1 (S6). For example, when proceeding to NO in step S4, the control unit 135 updates the shooting completion flag of the corresponding cut to ON and makes the determination in step S6. If video shooting of all cuts has not been completed (NO in S6), the processing from step S2 onwards is repeated for the cuts for which shooting has not been completed.

[0068] When the video shooting of all cuts is completed (YES in S6), the control unit 135 ends the processing of the flowchart shown in Fig. 5. After that, in the present system 10, for example, in the image editing terminal 300 or the image processing server 200, editing is performed as appropriate to complete the video work using the processed video for each cut.

[0069] According to the above processing, in the present system 10, the digital camera 100 can provide information support for improving the workflow of video production according to a scenario through the automatic editing process (S3) that uses the results of subject area detection. For example, when a user evaluates a video shot for each cut, the digital camera 100 of this embodiment can play back and display the processed video (FIGS. 4(B) and (D)) that has been subjected to the automatic editing process (S3), making it easier for the user to evaluate the video work while keeping in mind the finished product.

[0070] 2-2.Automatic editing process The automatic editing process in step S3 of FIG. 5 will be described in detail with reference to FIGS.

[0071] Fig. 8 is a flowchart illustrating an example of automatic editing processing in the present system 10. The processing shown in the flow of Fig. 8 starts in a state where, for example, one cut of video has been newly shot and recorded in step S2.

[0072] First, the control unit 135 analyzes, for example, audio data in a moving image file for one cut to be automatically edited (S11). For example, the control unit 135 detects a time period in the moving image in which speech is included in the audio data, and estimates the direction from which the speech is coming (DOA estimation). The control unit 135 may also perform individual identification of the speaker by analyzing the characteristics of the speech.

[0073] Next, the control unit 135 detects whether or not a subject is speaking, for example, for each frame of the moving image file (S12). The detection process in step S12 is performed based on, for example, the image data of the moving image file and the analysis result of the audio data (S11).

[0074] For example, in step S12, a predetermined state of the subject is detected by image analysis that analyzes the movement of a body part, such as the mouth, of the subject. For example, speaking person 31 may be detected by analyzing speech characteristics, such as whether the mouths of persons 31 and 32 are open, from captured image 20 in Fig. 4(A). Furthermore, a position where the speaker is estimated to be present on captured image 20 may be calculated by estimating the DOA of the speech from the results of voice analysis (S11).

[0075] When a speaking subject is detected (YES in S12), for example, the control unit 135 determines whether the detected speaking subject corresponds to a specific subject based on the editing information in the cut information D1 (S13). The specific subject is set in the editing information as a subject to be highlighted by image processing, for example.

[0076] The processing of step S13 makes it possible to avoid emphasizing an unexpected subject, even if, for example, an audience member who is not expected to be a performer in the scenario is speaking. The determination of step S13 may be made by image recognition for personal identification. Alternatively, it may be made by comparing the position of the performer in the composition of the cuts with the position of the person detected in the captured image 20. For example, if multiple subjects are detected as speaking in step S12 (YES in S12), and any one of the multiple subjects corresponds to a specific subject, the process proceeds to YES in step S13. The processing of step S13 may be omitted as appropriate.

[0077] If it is determined that the detected speaking subject corresponds to a specific subject (YES in S13), the control unit 135 controls the image processing engine 120 to perform image processing, such as color preservation processing, on the captured image 20 of the current frame in accordance with the person area R31 of the subject, thereby generating a processed image 40 (S14). The image processing in step S14 will be described with reference to FIG.

[0078] Fig. 9(A) shows a mask image M21 obtained by extracting the inside of the person region R31 detected in the captured image 20 of Fig. 4(A). Fig. 9(B) shows a mask image M22 obtained by extracting the outside of the person region R31 detected in the captured image 20 of Fig. 4(A).

[0079] In step S14, first, the control unit 135 controls the image processing engine 120 to intermediately generate mask images M21 and M22 by extracting the inside and outside of the detected person area R31 from the captured image 20, as shown in Figures 9(A) and (B), for example, by referring to the area division map M20 (Figure 7).

[0080] For example, in the color-retaining process (S14), the control unit 135 performs monochrome image processing to change a mask image M22 (FIG. 9(B)), which is an extracted image of the outside of the person region R31, to an achromatic color. Next, the control unit 135 controls the image processing engine 120 to combine the mask image M21 (FIG. 9(A)), which leaves the inside of the person region R31, with the mask image M22 (FIG. 9(B)), which is an extracted image of the outside of the mask image M21, to generate a processed image 40.

[0081] On the other hand, if it is determined that the detected speaking subject does not correspond to a specific subject (NO in S13), or if a speaking subject is not detected (NO in S12), the control unit 135 does not particularly perform the processing of step S14, and proceeds to, for example, step S15.

[0082] The control unit 135 determines whether or not the detection process (S12) of a subject in a predetermined state has been performed on all frames in the moving image file to be processed (S15). If the detection process of step S13 has not been performed on all frames (NO in S15), the control unit 135 repeats the processes from step S12 onwards for frames in the moving image file that have not been processed in step S12.

[0083] If the detection process in step S13 has been performed on all frames (YES in S15), for example, the control unit 135 ends the automatic editing process (S3) and proceeds to the process of step S4 in FIG.

[0084] According to the above automatic editing process (S3 in FIG. 8), when a subject in a predetermined state, such as a speaking person 31, is detected (YES in S13), image processing is performed (S14) so ​​that different image effects are applied to the inside and outside of the person area R31. As a result, for example, in a scene in which a speaking person 31 is present in a video, a highlighting effect is performed, such as by changing everything other than the speaking person 31 to monochrome (see FIG. 4(B)), making it easier to achieve a dramatic effect that attracts the viewer's attention.

[0085] The image processing in step S14 is not limited to the above example. For example, the image processing engine 120 may use an image in which the entire captured image 20 is converted into a monochrome image instead of the mask image M22 that leaves the area outside the person area R31, and then superimpose the mask image M21 on the monochrome image. In such superimposition, the mask image M21 is in a layer above the monochrome image, and the area of ​​the mask image M21 other than the corresponding part of the person area R31 is given a transparent attribute.

[0086] In step S14, image processing may be performed to impart various image effects to the mask image M21 (FIG. 9A) that leaves the inside of the person region R31. For example, different color corrections may be performed inside and outside the person regions R31 and R32, such as increasing the saturation of the mask image M21 that leaves the inside of the person region R31 and decreasing the saturation of the other mask image M22.

[0087] In step S14, for example, one frame of captured image 20 may be input to two image correction circuits, and image processing for the inside and outside of the subject area may be performed in parallel in each image correction circuit. The images thus obtained may be combined to generate processed image 40.

[0088] Also, in the above explanation, an example of operation has been described in which the area division map M20 obtained when shooting a moving image in digital camera 100 is kept and used in the processing of step S14. Digital camera 100 of this embodiment does not need to keep the area division map M20 obtained when shooting a moving image, and for example, control unit 135 may cause image recognition unit 122 to perform the object area detection operation again during the processing of step S14. This also makes it possible to efficiently realize image processing that is in line with the object area by utilizing image recognition unit 122 of digital camera 100.

[0089] 3. Summary As described above, in this embodiment, the digital camera 100, which is an example of an imaging device, includes the image sensor 115, which is an example of an imaging section, the image processing engine 120, which is an example of an image processing section, and the control unit 135. The image sensor 115 captures an image of a subject to generate a captured image 20. The image processing engine 120 performs image processing on the captured image 20 generated by the image sensor 115. The control unit 135 controls the image processing by the image processing engine 120 based on the captured image 20. The control unit 135 detects a subject in a predetermined state from the captured image 20 (S12). When the control unit 135 detects a subject in a predetermined state (YES in S12), the control unit 135 causes the image processing engine 120 to perform image processing so as to generate a processed image 40 in which different image effects are applied between the inside and outside of a subject area corresponding to the subject in the predetermined state in the captured image 20 (S14).

[0090] According to the digital camera 100 described above, it is possible to easily achieve a desired image effect for a captured subject by performing image processing that imparts different image effects to the inside and outside of a subject area corresponding to a subject in a predetermined state.

[0091] In the digital camera 100 of this embodiment, the control unit 135 detects a subject in a predetermined state by analyzing the movement of parts of the subject in the captured images 20 based on the multiple captured images 20 acquired from the image sensor 115 (S12). This makes it possible to detect a predetermined state in which parts of the subject are moving and to apply a desired image effect along the subject area.

[0092] In the digital camera 100 of this embodiment, the image effect is an effect in which the color tone is adjusted separately between the inside and outside of the subject area. The system 10 makes it easy to achieve a desired color tone effect for the captured subject.

[0093] In the digital camera 100 of this embodiment, person areas R31 and R32, which are an example of a subject area, are areas that follow the outline of people 31 and 32, which are an example of a subject, in the captured image 20. When a subject in a predetermined state is detected, the image processing engine 120 generates a processed image 40 so that the image effect varies along the outline of the subject in the predetermined state in the captured image 20. This makes it possible to vary the image effect between areas that follow the outline of the subject in the captured image 20, making it easier to achieve a desired image effect for the captured subject.

[0094] In this embodiment, the digital camera 100 further includes an image recognition unit 122 that recognizes a subject area corresponding to a subject in the captured image 20 based on the captured image 20 generated by the image sensor 115. The control unit 135 controls the image capturing operation of the image sensor 115 based on the subject area recognized by the image recognition unit 122. This makes it easier to achieve a desired image effect for the captured subject by utilizing the image recognition unit 122 of the digital camera 100.

[0095] In the digital camera 100 of this embodiment, when a subject in a predetermined state is not detected (NO in S12), the control unit 135 controls image processing by the image processing engine 120 so as not to impart different image effects to the inside and outside of a subject area corresponding to a subject not in the predetermined state in the captured image 20. For example, the digital camera 100 of this embodiment can generate a video that includes both a processed image 40 in which a subject in a predetermined state is detected and a captured image 20 in which a subject is not detected.

[0096] In this embodiment, digital camera 100 further includes flash memory 145, an example of a storage unit, that stores shot information D1, an example of management information that manages the schedule for image capture in one or more shooting scenes. Control unit 135 manages the subject in a predetermined state for each shooting scene, for example, by referencing the editing information in shot information D1. This allows the user to easily achieve the desired image effect by including the desired predetermined state of the subject in shot information D1.

[0097] The imaging method of this embodiment is an imaging method including a step (S2) in which the image sensor 115 captures an image of a subject and generates a captured image 20, a step (S3) in which the image processing engine 120 performs image processing on the captured image 20 generated by the image sensor 115, and steps (S12 to S14) in which the control unit 135 controls the image processing by the image processing engine 120 based on the captured image 20. In this method, the control unit 135 detects a subject in a predetermined state from the captured image 20 (S12), and when a subject in the predetermined state is detected (YES in S12), causes the image processing engine 120 to perform image processing (S14) to generate a processed image 40 in which different image effects are applied to the inside and outside of a subject region in the captured image 20 corresponding to the subject in the predetermined state. This method makes it easier to achieve a desired image effect for the captured subject.

[0098] A program including a group of instructions for causing a processor such as the control unit 135 to execute this method may be provided. Such a program may include a step in which the processor acquires a captured image 20 in which an image of a subject is captured.

[0099] (Embodiment 2) Hereinafter, a second embodiment of the present disclosure will be described with reference to Fig. 10. In the first embodiment, the imaging system 10 that imparts image effects for producing a video production scenario has been described, but the present disclosure is not limited thereto. In the second embodiment, such a modified imaging system 10 will be described.

[0100] Below, the description of the configuration and operation similar to those of the digital camera 100 and imaging system 10 according to the first embodiment will be omitted as appropriate, and the digital camera 100 and imaging system 10 according to this embodiment will be described.

[0101] 10 is a flowchart illustrating the operation of the digital camera 100 in the imaging system 10 of embodiment 2. In embodiment 1, image processing of a subject in a predetermined state was performed with reference to the shot information D1 that manages the scenario. In the imaging system 10 of this embodiment, for example, the digital camera 100 is provided with a plurality of operation modes dedicated to image processing of a subject in a predetermined state.

[0102] For example, in the digital camera 100 of this embodiment, the control unit 135 selects an applicable operation mode from among a plurality of dedicated operation modes in response to a user operation on the operation unit 150 (S22). In the example of Fig. 10, the dedicated operation modes include (I) a color effect operation mode and (II) a blur effect operation mode. For example, the control unit 135 displays a selection screen presenting these operation mode options (I) and (II) on the display monitor 130, and accepts a user operation on the operation unit 150 to select one of the options (S22).

[0103] When the color tone effect operation mode is selected ((I) in S21), for example, after shooting and recording a moving image (S22), when the control unit 135 detects a subject in a predetermined state for each frame (YES in S23), it sequentially performs image processing to apply different color tone effects to the inside and outside of the corresponding subject area (S24, S25). Such image processing for color tone effects can be performed in the same manner as the automatic editing process of the first embodiment (FIG. 8), for example. The color tone effect may employ various color corrections such as color grading or tone mapping.

[0104] Furthermore, when the blur effect operation mode is selected ((II) in S21), the control unit 135 performs steps S22A to S25A similar to the processing of the color tone effect operation mode (S22 to S25), for example, and performs image processing for the blur effect (S24A) instead of step S24. For example, the image processing of step S24A is performed by applying a blur effect so as to selectively blur the outside of an object region corresponding to an object in a predetermined state. This makes it possible to realize an image in which everything other than the object in the predetermined state is blurred, thereby achieving an emphasis effect that draws attention to a person who is speaking, for example.

[0105] Such image processing can be performed, for example, by using mask images M21 and M22 inside and outside the subject area, as in the example of Figure 9. The blur effect may be applied as an image effect similar to motion blur caused by movement of a subject other than the target to be highlighted. For example, an image effect that creates the appearance of motion blur may be applied between frames in which a subject in a predetermined state is detected.

[0106] The image effects applied by the present system 10 are not limited to those described above, and may be visual effects such as various video effects created by editing or processing images or videos. For example, time effects such as slow motion and time lapse, or transition effects such as whiteout and fade-in / fade-out may be employed in the present system 10. Special effects for creating scenes or images that are impossible to realize in reality, such as explosions, collapses, flight, or the appearance of ghosts or monsters, may also be included. Special effects that create realistic images or fantasy worlds using CG may also be applied to the present system 10.

[0107] By applying image effects in this system 10, it is possible to enhance the appeal and information transmission power of the image, and to express the atmosphere and emotions of the story. Furthermore, in this system 10, the dedicated operation mode is not limited to image effects, and may also specify a subject in a predetermined state to be detected.

[0108] As described above, in the imaging system 10 of this embodiment, the digital camera 100 further includes an operation unit 150, which is an example of an input unit that accepts a user operation to select one of a plurality of operation modes corresponding to a plurality of image effects. In accordance with the user operation on the input unit, the control unit 135 causes the image processing engine 120 to perform image processing so as to impart an image effect corresponding to the operation mode selected by the user operation (S21, S24, S24A). This allows the user to easily achieve the image effect desired by selecting the desired operation mode.

[0109] (Other embodiments) As described above, Embodiments 1 and 2 have been described as examples of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made as appropriate. Furthermore, it is also possible to combine the components described in Embodiments 1 and 2 above to create new embodiments.

[0110] In the first embodiment described above, an example of the operation of the imaging system 10 when a speaking person 31 is the subject of a predetermined state to be detected has been described, but the present disclosure is not particularly limited thereto. In this embodiment, the predetermined state to be detected by the imaging system 10 may be, for example, a state in which the subject person is making a gesture such as holding up a finger, or a state of a particular facial expression such as smiling. In the present system 10, the control unit 135 may detect such a predetermined state by, for example, analyzing changes in the subject's face or fingers in the captured image.

[0111] The system 10 may also detect a person exhibiting suspicious behavior (i.e., a suspicious individual) as a subject in a predetermined state. The system 10 may also be applied to crime prevention applications, such as using the digital camera 100 as a surveillance camera that captures images of a monitored space, such as a public place or a store. In this embodiment, the control unit 135 may, for example, calculate the motion vector of each person extracted from an image captured by the digital camera 100, and detect as a suspicious person a person whose motion vector deviates from a group trend obtained by statistically analyzing the motion vectors of multiple people in the past by a greater than a predetermined threshold. The system 10 may also detect as a subject in a predetermined state a person exhibiting various behaviors, such as repeatedly looking at the same product or staying in the same place for a long time.

[0112] As described above, in the digital camera 100 of this embodiment, the predetermined state may be at least one of the following: speech, gesture, smile, and specific behavior of the person as the subject. The system 10 can apply a desired image effect along the subject area according to the state of the person.

[0113] In the first embodiment, an example of an operation in which image processing such as automatic editing processing (S3 in FIG. 5) is performed in the digital camera 100 has been described. The present system 10 is not particularly limited to this, and, for example, some of the processing may be performed by the image processing server 200. For example, the image processing server 200 of this embodiment may receive original video data from the digital camera 100 via the communication network 15 or the like. The control unit 210 may cause the image processing unit 260 to perform automatic editing processing (S3 in FIG. 5) on the captured images 20 included in the original video data. The control unit 210 of the image processing server 200 may also acquire information on the subject area detection results (FIG. 7) along with the data of the captured images 20, or the control unit 210 of the image processing server 200 may perform the subject area detection operation in the same manner as the control unit 135.

[0114] As described above, the imaging system 10 of this embodiment includes a digital camera 100 and an image processing server 200, an example of an image processing device. The digital camera 100 includes an image sensor 115 that captures an image of a subject and generates a captured image 20, and a communication module 155, an example of a transmitter, that transmits image data representing the captured image 20 to the image processing device. The image processing server 200 includes a communication unit 250, an example of a receiver, that receives image data from the digital camera 100, and an image processing unit 260 that performs image processing on the captured image 20 represented by the image data based on the image data received by the receiver. The digital camera 100 or the image processing server 200 includes a control unit 135, 210 that detects a subject in a predetermined state from the captured image 20 and, when a subject in the predetermined state is detected, causes the image processing unit 260 to perform image processing to generate a processed image 40 in which different image effects are applied to the inside and outside of a subject area corresponding to the subject in the predetermined state in the captured image 20. This system 10 makes it easy to achieve a desired image effect for a captured subject.

[0115] In addition, in each of the above embodiments, an example has been described in which the subject to be subjected to image processing in the imaging system 10 is one person, but the present disclosure is not limited to this. In the imaging system 10 of this embodiment, the subject to be subjected to image processing may be multiple people, or may be a moving object such as an animal or a vehicle, without being limited to a person.

[0116] In addition, in each of the above embodiments, moving images are exemplified as the target of image processing in the imaging system 10. In the imaging system 10 of this embodiment, the target of image processing is not particularly limited to moving images, and may be still images.

[0117] In addition, in each of the above embodiments, the digital camera 100 is illustrated as including the optical system 110 and the lens driving unit 112. The imaging device of this embodiment does not need to include the optical system 110 and the lens driving unit 112, and may be, for example, an interchangeable lens camera.

[0118] In addition, although a digital camera has been described as an example of an imaging device in each of the above embodiments, the imaging device is not limited to this. The imaging device of the present disclosure may be any electronic device having an image capturing function (for example, a video camera, a smartphone, a tablet terminal, etc.).

[0119] (Example) Various aspects of the present disclosure are listed below.

[0120] A first aspect of the present disclosure is an imaging device including an imaging unit that captures an image of a subject and generates a captured image, an image processing unit that performs image processing on the captured image generated by the imaging unit, and a control unit that controls the image processing by the image processing unit based on the captured image. The control unit detects a subject in a predetermined state from the captured image. When the control unit detects the subject in the predetermined state, it causes the image processing unit to perform image processing so as to generate a processed image in which different image effects are applied between the inside and outside of a subject area corresponding to the subject in the predetermined state in the captured image.

[0121] In a second aspect, in the imaging device described in the first aspect, the control unit detects a subject in a predetermined state by analyzing the movement of parts of the subject in the captured images based on multiple captured images obtained from the imaging unit.

[0122] In a third aspect, in the imaging device according to the first or second aspect, the predetermined state is at least one of speech, gesture, smile, and specific behavior of a person as a subject.

[0123] In a fourth aspect, in the imaging device according to any one of the first to third aspects, the image effect is an effect in which color tones are adjusted separately between the inside and outside of the subject area.

[0124] In a fifth aspect, in an imaging device described in any of the first to fourth aspects, the subject area is an area that follows the outline of the subject in the captured image, and when a subject in a predetermined state is detected, the image processing unit generates a processed image so as to vary the image effect along the outline of the subject in the predetermined state in the captured image.

[0125] In a sixth aspect, the imaging device according to any one of the first to fifth aspects further includes an image recognition unit that recognizes a subject area corresponding to a subject in the captured image based on the captured image generated by the imaging unit, and a control unit that controls an image capturing operation by the imaging unit based on the subject area recognized by the image recognition unit.

[0126] In a seventh aspect, in an imaging device described in any of the first to sixth aspects, when a subject in a predetermined state is not detected, the control unit controls image processing by the image processing unit so as not to impart different image effects between the inside and outside of a subject area in the captured image corresponding to a subject that is not in the predetermined state.

[0127] In an eighth aspect, the imaging device according to any one of the first to sixth aspects further includes a storage unit that stores management information for managing a schedule for image capture in one or more photographic scenes, and the control unit manages a subject in a predetermined state for each photographic scene by referring to the management information.

[0128] In a ninth aspect, the imaging device according to any one of the first to eighth aspects further includes an input unit configured to receive a user operation for selecting one of a plurality of operation modes corresponding to a plurality of image effects, and a control unit configured to cause an image processing unit to perform image processing in accordance with the user operation on the input unit so as to impart the image effect corresponding to the operation mode selected by the user operation.

[0129] A tenth aspect is an imaging method including a step in which an imaging unit captures an image of a subject and generates a captured image, a step in which an image processing unit performs image processing on the captured image generated by the imaging unit, and a step in which a control unit controls the image processing by the image processing unit based on the captured image. In this method, the control unit detects a subject in a predetermined state from the captured image, and when the subject in the predetermined state is detected, causes the image processing unit to perform image processing so as to generate a processed image in which different image effects are applied between the inside and outside of a subject area corresponding to the subject in the predetermined state in the captured image.

[0130] An eleventh aspect is an imaging system including an imaging device and an image processing device. The imaging device includes an imaging unit that captures an image of a subject and generates a captured image, and a transmission unit that transmits image data representing the captured image to the image processing device. The image processing device includes a receiving unit that receives image data from the imaging device and an image processing unit that performs image processing on the captured image represented by the image data based on the image data received by the receiving unit. The imaging device or the image processing device includes a control unit that detects a subject in a predetermined state from the captured image, and, when the subject in the predetermined state is detected, causes the image processing unit to perform image processing so as to generate a processed image in which different image effects are applied between the inside and outside of a subject area corresponding to the subject in the predetermined state in the captured image.

[0131] As described above, the embodiments have been described as examples of the technology in the present disclosure, and for that purpose, the accompanying drawings and detailed description have been provided.

[0132] Therefore, the components shown in the accompanying drawings and detailed description may include not only essential components for solving the problem, but also components that are not essential for solving the problem in order to illustrate the above technology. Therefore, the fact that these non-essential components are shown in the accompanying drawings or detailed description should not be interpreted as immediately indicating that these non-essential components are essential.

[0133] Furthermore, since the above-described embodiments are intended to illustrate the technology of the present disclosure, various modifications, substitutions, additions, omissions, etc. may be made within the scope of the claims or their equivalents. [Industrial Applicability]

[0134] The present disclosure is applicable to various uses in which image processing is performed on an image of a captured subject. [Explanation of symbols]

[0135] 10. Imaging System 100 digital cameras 115 Image Sensor 120 Image Processing Engine 130 Display Monitor 135 Control Unit 150 Operation section 155 Communication Module 200 Image Processing Server 210 Control Unit 250 Communications Department 260 Image Processing Unit

Claims

1. an imaging unit that captures an image of a subject and generates a captured image; an image processing unit that performs image processing on the captured image generated by the imaging unit; a control unit that controls image processing by the image processing unit based on the captured image, The control unit Detecting a subject in a predetermined state from the captured image, When the subject in the predetermined state is detected, the image processing unit is caused to perform the image processing so as to generate a processed image in which different image effects are applied between the inside and outside of a subject area corresponding to the subject in the predetermined state in the captured image. Imaging device.

2. The control unit detects the subject in the predetermined state by analyzing a movement of a part of the subject in the captured images based on a plurality of captured images acquired from the imaging unit. The imaging device according to claim 1 .

3. The predetermined state is at least one of speech, gesture, smile, and specific behavior of the person as the subject. The imaging device according to claim 1 .

4. The image effect is an effect in which the color tone is adjusted separately between the inside and outside of the subject area. The imaging device according to claim 1 .

5. the subject region is a region in the captured image that follows the outline of the subject, When the subject in the predetermined state is detected, the image processing unit generates the processed image so that the image effect varies along the outline of the subject in the predetermined state in the captured image. The imaging device according to claim 1 .

6. an image recognition unit that recognizes a subject area corresponding to the subject in the captured image based on the captured image generated by the imaging unit; The control unit controls an image capturing operation by the imaging unit based on the subject area recognized by the image recognition unit. The imaging device according to claim 1 .

7. a storage unit for storing management information for managing a schedule for image capture in one or more photographic scenes; The control unit manages the subject in the predetermined state for each of the photographic scenes by referring to the management information. The imaging device according to claim 1 .

8. an input unit that accepts a user operation to select one of a plurality of operation modes corresponding to the plurality of image effects, The control unit causes the image processing unit to perform image processing in accordance with the user operation on the input unit so as to impart an image effect corresponding to the operation mode selected by the user operation. The imaging device according to claim 1 .

9. an imaging unit capturing an image of a subject and generating a captured image; an image processing unit performing image processing on the captured image generated by the imaging unit; a step in which a control unit controls image processing by the image processing unit based on the captured image, The control unit Detecting a subject in a predetermined state from the captured image, When the subject in the predetermined state is detected, the image processing unit is caused to perform the image processing so as to generate a processed image in which different image effects are applied between the inside and outside of a subject area corresponding to the subject in the predetermined state in the captured image. Imaging method.

10. An imaging system including an imaging device and an image processing device, The imaging device is an imaging unit that captures an image of a subject and generates a captured image; a transmission unit that transmits image data representing the captured image to the image processing device, The image processing device includes: a receiving unit that receives the image data from the imaging device; an image processing unit that performs image processing on a captured image indicated by the image data based on the image data received by the receiving unit, The imaging device or the image processing device is a control unit that detects a subject in a predetermined state from the captured image, and when the subject in the predetermined state is detected, causes the image processing unit to perform the image processing so as to generate a processed image in which different image effects are applied between an inside and an outside of a subject area in the captured image corresponding to the subject in the predetermined state. Imaging system.

Images