Imaging device, control method, and program

Abstract

To allow a user to select a desired image when an image that satisfies predetermined conditions extracted externally and a representative image extracted by an image capture apparatus are different.SOLUTION: An image capture apparatus that communicates with an image processing apparatus includes: transmission means which transmits a plurality of RAW images acquired by continuous shooting by imaging means to the image processing apparatus; receiving means which extracts a predetermined RAW image that satisfies a predetermined condition from the multiple RA images received from the image capture apparatus, and receives an image obtained by developing the predetermined RAW image or an extracted result; and determination means which determines, when a predetermined representative image among the plurality of RAW images acquired by the imaging means and a predetermined RAW image extracted by the image processing apparatus are different, whether or not to replace the predetermined representative image with the predetermined RAW image.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to a system for extracting and reproducing RAW images that satisfy predetermined conditions from a plurality of RAW images. [Background technology]

[0002] Due to improvements in the performance of image sensors, electronic shutter systems, which capture images by controlling the timing of charge accumulation and readout by image sensors, have become mainstream, replacing mechanical shutter systems, which capture images by mechanically operating a shutter. Unlike mechanical shutter systems, electronic shutter systems do not emit shutter sounds when shooting, making them ideal for shooting scenes in quiet environments, for example. In addition, with electronic shutter systems, it is possible to shoot several tens of frames per second due to improvements in the readout speed of charge from image sensors.

[0003] In this way, in an imaging device capable of capturing several tens of frames (e.g., 30 frames) per second, it is necessary to omit the relatively time-consuming compression and filing processes that follow capturing, and move to a state in which the next capture is possible. For this reason, several frames per second are recorded as RAW image data immediately after capture without development processing. Furthermore, even with RAW image data, thumbnail data for display is necessary, so thumbnail data that is smaller than the data size at the time of capture is embedded in each frame. In this way, the overall processing time for capture is saved.

[0004] Conventionally, RAW image data was developed using development applications on digital cameras or personal computers (PCs), but with the recent improvement of communication environments, it is now possible to perform development processing on cloud servers via networks. By performing development processing on cloud servers in this way, new services and algorithms can be adopted quickly, resulting in high-quality developed images. Furthermore, using high-performance servers can shorten the time required for development processing.

[0005] Furthermore, the high-performance server can determine whether the image is in focus, whether the composition is appropriate, and if the subject is a person, whether the person's expression is good, faster than the user can select their favorite scene, and can extract the best shot from multiple images. Patent Document 1 describes a video playback system that extracts multiple representative images corresponding to multiple scenes from a video file, displays them as a list, and allows the user to select a favorite scene and play the representative image of the selected scene. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2006-197566 A Summary of the Invention [Problem to be solved by the invention]

[0007] In a system in which a user transmits multiple RAW image data captured by an imaging device to a cloud server and the cloud server extracts the best shot from the multiple RAW image data, the best shot image extracted by the cloud server from the multiple RAW image data may differ from the representative image extracted from the RAW image data held by the imaging device. In addition, when the imaging device receives the best shot image extracted by the cloud server, the representative image extracted by the imaging device may be replaced with the best shot image extracted by the cloud server that is not intended by the user and played back.

[0008] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to realize a technology that enables a user to select a desired image when an image that satisfies certain conditions extracted externally differs from a representative image extracted by an imaging device. [Means for solving the problem]

[0009] In order to solve the above problems and achieve the object, the present invention provides an imaging device capable of communicating with an image processing device, comprising a transmitting means for transmitting a plurality of RAW images obtained by continuous shooting using an imaging means to the image processing device, a receiving means in the image processing device for extracting a predetermined RAW image that satisfies predetermined conditions from the plurality of RAW images received from the imaging device, and receiving an image or the extracted result of the predetermined RAW image, and a determining means for determining whether or not to replace the predetermined representative image with the predetermined RAW image when a predetermined representative image among the plurality of RAW images obtained by the imaging means differs from the predetermined RAW image extracted by the image processing device. Effect of the Invention

[0010] According to the present invention, when an image that satisfies a predetermined condition and that is extracted externally differs from a representative image extracted by an image capture device, a user can select a desired image. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a system configuration diagram of the present embodiment. [Diagram 2] FIG. 1 is a block diagram showing the configuration of a digital camera according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a block diagram showing the configuration of a server according to the embodiment. [Figure 4] FIG. 1A is an explanatory diagram of a bundled RAW image file and best shot determination in this embodiment, FIG. 1B is an explanatory diagram of RAW burst shooting when pre-shooting is "off" in this embodiment, FIG. 1C is an explanatory diagram of RAW burst shooting when pre-shooting is "on" in this embodiment, and FIG. 1D is a data structure diagram of a bundled RAW image file in this embodiment. [Diagram 5] 4 is a diagram illustrating an example of a screen displaying a list of a plurality of image files held by the digital camera of the present embodiment. [Figure 6] 5 is a flowchart showing processing of a server in the embodiment. [Figure 7]6A to 6C are views for explaining a notification process of a best shot extraction result in the present embodiment. [Figure 8] 4 is a flowchart showing processing of the digital camera according to the embodiment. [Figure 9] 11A and 11B are explanatory diagrams of processing during playback when a representative image and a best shot image differ from each other in this embodiment. [Figure 10] 11 is a flowchart showing the process of the digital camera when the user selects a best shot image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0013] In this embodiment, an example of a system in which a digital camera (hereinafter, camera) 200 and a server computer 300 are connected via a communication network 100 as an image generating device and an image processing device will be described.

[0014] In the system of this embodiment, development processing of RAW image data generated by the camera 200 is executed by a cloud server (hereinafter, server) 300 on a network that provides a development processing function. In this embodiment, an example is described in which the server 300 executes development processing of RAW image data, but the server 300 may also perform image processing other than development processing.

[0015] <System Configuration> First, the system configuration of this embodiment will be described with reference to FIG.

[0016] An image processing application for utilizing the development processing function of the server 300 is installed in the camera 200. The camera 200 is communicatively connected to the server 300 via a communication network 100 such as a wide area network (WAN), the Internet, or a public line.

[0017] Camera 200 generates a bundled RAW image file 50 consisting of multiple RAW image data generated by continuously shooting images using continuous shooting (RAW burst shooting), and transmits bundled RAW image file 50 to server 300. The RAW image data generated by camera 200 is image data such as still images and videos captured by a digital camera or the like, and is filed without undergoing development processing.

[0018] The camera 200 transmits RAW image data generated by the camera 200 to the server 300, and the server 300 executes a development process on the RAW image data received from the camera 200, and transmits the developed image data to the camera 200.

[0019] The server 300 extracts RAW image data determined to be the best shot from the multiple RAW image data received from the camera 200. Then, the server 300 transmits to the camera 200 information related to the RAW image data determined to be the best shot or image data 51 obtained by developing the RAW image data determined to be the best shot.

[0020] <Configuration of Camera 200> Next, the hardware configuration of camera 200 will be described with reference to FIG.

[0021] In this embodiment, an example in which the image generating device is applied to an imaging device such as a digital camera will be described, but the image generating device is not limited to this. For example, the image generating device may be an information processing device such as a smartphone, which is a type of mobile phone, a tablet device, a watch-type terminal, or a glasses-type terminal.

[0022] The control unit 201 is a central processing unit (CPU) that controls the entire camera 200, and realizes the processing of the flowchart described below by executing a program stored in a non-volatile memory 202 described below. Note that instead of the control unit 201 controlling the entire device, the entire device may be controlled by multiple hardware devices sharing the processing.

[0023] The non-volatile memory 202 stores constants, programs, etc. for the operation of the control unit 201. The programs here include programs for executing communication processing with the server 300, which will be described later in this embodiment. The non-volatile memory 202 also stores an OS (operating system), which is basic software executed by the control unit 201, and an image processing application that cooperates with the OS to realize applied functions. The processing of the camera 200 in this embodiment is realized by reading software provided by the image processing application. It is assumed that the image processing application has software for utilizing the basic functions of the OS installed in the camera 200. It is also assumed that the OS of the camera 200 has software for realizing the processing in this embodiment. The non-volatile memory 202 is, for example, an EEPROM.

[0024] The working memory 203 is used as a working area for expanding constants and variables for the operation of the control unit 201, programs read from the non-volatile memory 202, etc. The working memory 203 also includes a buffer memory for temporarily storing image data captured by the imaging unit 204, a display unit 206 It is used as image display memory.

[0025] The imaging unit 204 includes a group of lenses including a zoom lens and a focus lens, and a shutter with an aperture function. The imaging unit 204 also includes an imaging element configured with a CCD or CMOS element or the like that converts a subject image into an electric signal, and an A / D converter that converts an analog image signal output from the imaging element into a digital signal. Under the control of the control unit 201, the imaging unit 204 converts the subject image light formed by the lens included in the imaging unit 204 into an electric signal using the imaging element, performs noise reduction processing, and outputs image data consisting of a digital signal.

[0026] The control unit 201 performs various image processes, excluding development processing, on RAW image data captured by the imaging unit 204, generates an image file, and records it on the recording medium 208. In the camera 200 of this embodiment, the RAW image data is recorded on the recording medium 208 in accordance with the DCF (Design Rule for Camera File system) standard. The control unit 201 also performs a predetermined calculation process using the captured image data, and controls the focus lens, aperture, and shutter of the imaging unit 204 based on the obtained calculation result, thereby performing AF (autofocus) processing and AE (autoexposure) processing.

[0027] Operation section 205 The operation unit is made up of various switches, buttons, touch panels, and other operation members that accept various operations from the user. 205 For example, the display unit 200 includes a power button for turning the power on and off, a shutter button for taking pictures, and a mode setting button for setting the operation mode of the camera 200. 206 The touch panel formed integrally with the operation unit 205 Also, the operation section 205 The operation unit 207 includes an operation member such as a dedicated connection button for starting communication with an external device via a communication unit 207 described later. 205 The operation unit may be integrated with the camera 200 or may be an external device connected to the camera 200. 205 It would be good if you could connect it to

[0028] The shutter button is turned on by a user's half-press (instruction to prepare for shooting) during shooting and generates a first shutter switch signal SW1. In response to the first shutter switch signal SW1 ON, the control unit 201 controls the imaging unit 204 to start shooting preparation operations such as AF (autofocus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (pre-flash) processing. In addition, the shutter button is turned on by a user's full-press (instruction to start shooting) during shooting and generates a second shutter switch signal SW2. In response to the second shutter switch signal SW2 ON, the control unit 201 starts a series of shooting operations from reading out a signal from the imaging unit 204 to writing image data to the recording medium 208, and in response to the second shutter switch signal SW2 OFF (instruction to end shooting), ends the shooting operations.

[0029] The mode setting button can set the operation mode of the camera 200 to one of a number of operation modes, including a shooting mode for shooting still images and videos, a playback mode for playing back images, and a RAW burst shooting mode, which will be described later. In the RAW burst shooting mode, the shooting setting can be set to "off" or "on" pre-shooting, which will be described later.

[0030] Display section 206 The display unit 203 displays image data (including live view) captured by the image capture unit 204 and image data that has been subjected to image processing by the server 300. 206 The display section displays characters for interactive operation, such as the GUI (Graphical User Interface) of image processing applications. 206 is a display device such as a liquid crystal display or an organic electroluminescence display. 206 The display unit may be an integrated part of the camera 200 or an external device connected to the camera 200. 206 It can be connected to the display unit 206 It is sufficient that the display device has a function of controlling the display of the above.

[0031] The communication unit 207 is an interface for connecting to an external device such as a server 300. The camera 200 of this embodiment can exchange data with the external device via the communication unit 207. For example, RAW image data generated by the camera 200 can be transmitted to the external device via the communication unit 207. In this embodiment, the communication unit 207 includes a wired connection interface such as a USB cable, HDMI (registered trademark), or IEEE1394. The communication unit 207 also includes a wireless connection interface such as a wireless LAN (Local Area Network) based on the IEEE802.11 standard and Bluetooth (registered trademark) based on the IEEE802.15 standard. The control unit 201 realizes communication with the external device by controlling the communication unit 207.

[0032] Image data output from imaging unit 204 is written to recording medium 208, and previously recorded image files are read out. Recording medium 208 may be a memory card or a hard disk drive that is attached to camera 200, or may be a flash memory or a hard disk drive built into camera 200. It is sufficient that camera 200 has at least a function for accessing recording medium 208.

[0033] <Configuration of Server 300> Next, the hardware configuration of the server 300 will be described with reference to FIG.

[0034] The server 300 of this embodiment includes a control unit 301 , a non-volatile memory 302 , a working memory 303 , an image processing unit 304 , a communication unit 305 , a storage unit 306 , and an image determination unit 307 .

[0035] The control unit 301 is a central processing unit (CPU) that controls the entire server 300, and realizes the processing of the flowchart described below by executing a program stored in the non-volatile memory 302 described below. Note that instead of the control unit 301 controlling the entire device, the entire device may be controlled by multiple hardware devices sharing the processing.

[0036] The non-volatile memory 302 stores constants, programs, etc. for the operation of the control unit 301. The programs here include programs for executing communication processing with the camera 200. The non-volatile memory 302 also stores an OS (operating system) which is basic software executed by the control unit 301, and an image processing program which cooperates with the OS to realize applied functions. The processing of the server 300 in this embodiment is realized by reading software provided by the image processing program. It is assumed that the image processing program has software for utilizing the basic functions of the OS installed in the server 300. It is also assumed that the OS of the server 300 has software for realizing the processing in this embodiment. The non-volatile memory 302 is, for example, a hard disk drive (HDD), a solid state drive (SSD) composed of a flash memory, an EEPROM, a flash memory, an optical disk, etc.

[0037] The working memory 303 is used as a working area for expanding constants and variables for the operation of the control unit 301, programs read from the non-volatile memory 302, etc. The working memory 303 is also used as a buffer memory for temporarily holding RAW image data received from the camera 200.

[0038] The image processing unit 304 is a GPU (Graphics Processing Unit) that performs development processes such as pixel interpolation, luminance signal processing, and color signal processing on the RAW image data received from the camera 200 to generate developed image data.

[0039] The communication unit 305 includes software and circuits for communicating with an external device by HTTP communication or the like. The server 300 of this embodiment can exchange data with the camera 200 via the communication unit 305. For example, the server 300 can transmit image data developed by the image processing unit 304 to the external device via the communication unit 305. In this embodiment, the communication unit 305 includes an interface for communicating with the external device wirelessly or via a wired method. The control unit 301 realizes communication with the external device by controlling the communication unit 305. In this embodiment, the communication unit 305 can connect to the camera 200 using a GET / HEAD method. It is to be noted that the communication method is not limited to the GET / HEAD method, and methods such as a POST method and a PUT method may also be used. When a user connects the camera 200 to the server 300 when going out, the camera 200 can be connected to the server 300 via a public line or Wi-Fi. (Registered Trademark) etc., and when returning home, the connection may be established via an access point at home etc. Note that communication with camera 200 is not limited to this, and developed image data does not necessarily have to be transmitted to camera 200.

[0040] The storage unit 306 is a storage device such as an SSD or a hard disk that stores image data and other data received from the camera 200. Note that the device with which the server 300 communicates is not limited to the camera 200, and may be other communication devices.

[0041] The image determination unit 307 includes software and circuitry that determines the best shot from the RAW image data received from the camera 200 and stored in the storage unit 306, and extracts the RAW image data determined to be the best shot.

[0042] The image processing unit 304 performs development processing on the RAW image data determined to be the best shot by the image determination unit 307. The developed image data is transmitted to the camera 200 by the communication unit 305.

[0043] <Bundle RAW images and best shot judgement> FIG. 4A is a diagram illustrating a method for extracting an image determined to be the best shot from a plurality of pieces of RAW image data.

[0044] In this embodiment, the best shot is RAW image data that is determined to be optimal among multiple RAW images that are generated by continuously shooting the same subject in accordance with a predetermined shooting operation using RAW burst shooting. There are multiple determination conditions for the best shot, such as being in focus, the subject being captured large, and capturing a decisive moment. In this embodiment, an example in which multiple (e.g., six) images 401 to 406 are taken by continuously shooting one person will be described with reference to FIG. 4(a). In this case, image 404 shows the person smiling the most, and is determined to be the best shot among images 401 to 406.

[0045] 4(b) and (c) are diagrams illustrating RAW burst shooting.

[0046] RAW burst shooting is a method of shooting multiple RAW image data in succession, and the predetermined shooting settings are pre-shooting "off" and pre-shooting "on". Figure 4(b) shows an example of shooting operation when pre-shooting "off", and Figure 4(c) shows an example of shooting operation when pre-shooting "on". The horizontal axis of Figure 4(b) and (c) shows time.

[0047] In the pre-shooting "off" state of FIG. 4(b), continuous shooting is performed between when the second shutter switch signal SW2 is turned on (start of shooting) and when the second shutter switch signal SW2 is turned off (end of shooting), and multiple pieces of RAW image data captured during the continuous shooting are recorded as a single bundled RAW image file 410.

[0048] 4(c), in the pre-photography "ON" state, continuous shooting is performed from a point in time after the first shutter switch signal SW1 is turned on and a predetermined time before the second shutter switch signal SW2 is turned on until the second shutter switch signal SW2 is turned off, and multiple pieces of RAW image data obtained by continuous shooting are recorded as one bundled RAW image file 411. In this case, the RAW image data obtained by continuous shooting from a predetermined time (maximum 0.5 seconds) before the second shutter switch signal SW2 is turned on until the second shutter switch signal SW2 is turned off is recorded.

[0049] FIG. 4D is a diagram illustrating an example of the data structure of a bundled RAW image file generated by RAW burst shooting.

[0050] The bundled RAW image file 420 includes meta information 421, a representative image 422, and image frames 423-426.

[0051] The meta information 421 includes shooting information, representative image information, the number of frames, and best shot information.

[0052] The representative image information is information including the frame number and shooting date and time of the RAW image data corresponding to the representative image 422 described later, the resolution of the representative image, etc. The number of frames is the number of RAW image data. The best shot information is information on the frame determined to be the best shot. In this embodiment, the image determination unit 307 of the server 300 determines the best shot and transmits the determination result to the camera 200 via the communication unit 305 of the server 300, but this is not limiting, and the user himself may determine the best shot, or the camera 200 may automatically determine it.

[0053] The representative image 422 includes JPEG display data of a representative image of the entire bundle RAW image file. For example, the display image of the frame when the second shutter switch signal SW2 is on in Fig. 4(b) or 4(c) is recorded as the representative image and is used as the image displayed on the selection screen or playback screen described later in Fig. 5.

[0054] Each of the image frames 423 to 426 is a plurality of RAW image data included in the bundle RAW image file 420. The image frame 423 includes frame information of the first frame, a JPEG for display (size: small), and RAW data. The image frame 424 includes frame information of the second frame, a JPEG for display (size: small), and RAW data. The image frame 425 includes frame information of the third frame, a JPEG for display (size: small), and RAW data. The image frame 426 includes frame information of the last frame N, a JPEG for display (size: small), and RAW data.

[0055] By performing RAW burst shooting using the camera 200, one bundle RAW image file 420, which is a bundle of RAW image data captured in continuous shooting, is generated. The server 300 can obtain a best shot image by extracting one piece of RAW image data determined to be the best shot from the bundle RAW image file 420 received from the camera 200.

[0056] FIG. 5 is a diagram showing an example of a screen displaying a list of a plurality of image files held by camera 200 of this embodiment.

[0057] An example of a selection screen 430 displayed on the display unit 206 of the camera 200, in which a list of multiple image files stored in the recording medium 208 is displayed, will be described below.

[0058] Selection screen 430 in Fig. 5(a) displays a state in which four image files 431 to 434 are stored in recording medium 208. Image files 431, 432, and 433 are JPEG files, and image file 434 is a bundled RAW image file. For JPEG files, a thumbnail added to the meta information is displayed. For bundled RAW image files, a small JPEG for display held in representative image 422 is displayed.

[0059] When the bundle RAW image file 434 is selected as the file to be played back on the selection screen 430 in Fig. 5(a), a playback screen 440 shown in Fig. 5(b) is displayed on the display unit 206. A representative image 434a extracted from the bundle RAW image file 434 and a slider bar 441 are displayed on the playback screen 440. The length of the slider bar 441 corresponds to the length of the multiple pieces of RAW image data contained in the bundle RAW image file 434 when arranged in chronological order. The representative image 434a is displayed at a position corresponding to a mark 442 on the slider bar 441, and by sliding the mark 442 on the slider bar 441, it is possible to check images before and after the representative image 434a or to change the representative image.

[0060] <Processing of Server 300> Next, the best shot extraction process and development process executed by the server 300 will be described with reference to FIG.

[0061] 6 is realized by the control unit 301 of the server 300 expanding a program stored in the non-volatile memory 302 into the working memory 303, executing the program, and controlling each component. The process of FIG. 6 is started when the server 300 receives a bundled RAW image file from the camera 200.

[0062] In step S601, the control unit 301 determines whether or not all RAW image data included in the bundled RAW image file transmitted from the camera 200 has been received. If the control unit 301 determines that all data has been received, the process proceeds to step S604, and if the control unit 301 determines that all data has not been received, the process proceeds to step S602. The determination of whether or not data has been received is made by referring to the number of frames in the meta information 421 of the bundled RAW image file 420 received by the control unit 301.

[0063] In step S602, the control unit 301 determines whether or not best shot determination is possible using a portion of the RAW image data received from the camera 200. If the control unit 301 determines that best shot determination is possible, the process proceeds to step S604, and if the control unit 301 determines that best shot determination is impossible, the process proceeds to step S603. The determination of whether or not best shot determination is possible is made by, for example, determining whether a predetermined number of pieces of RAW image data or more have been received, and whether or not the tendency of the subject can be determined from the received RAW image data. In addition, in cases such as when it is determined that no more RAW image data is required, the control unit 301 determines whether or not best shots can be extracted from a portion of the RAW image data of the bundle RAW image file.

[0064] In step S603, the control unit 301 waits until the remaining RAW image data is transmitted from the camera 200, and then returns the process to step S601.

[0065] In step S604, the control unit 301 performs best shot determination by the image determination unit 307, and proceeds to step S605. The best shot determination is performed using a number of conditions, such as whether the image is in focus, whether the subject is large, whether the decisive moment is captured, etc., and can be performed by changing or combining the conditions according to the subject recognition result. Note that the best shot determination method is not limited to this, and any method may be used. For example, various algorithms may be combined using artificial intelligence (AI), or the user's shooting tendencies may be learned and reflected in the best shot determination.

[0066] In step S605, the control unit 301 performs development processing on the RAW image data determined to be the best shot using the image processing unit 304, and proceeds to step S606. Note that the parameters of the development processing performed by the image processing unit 304 can be set arbitrarily by the user, or can be automatically determined by the server 300 using artificial intelligence (AI). It is also possible to hold the RAW image data as is without performing development processing.

[0067] In step S606, the control unit 301 sets a developed flag indicating a developed state for the RAW image data developed in step S605, and proceeds to step S607. The developed flag may be stored in a dedicated storage area of ​​the non-volatile memory 302 or the working memory 303.

[0068] In step S607, the control unit 301 determines whether the development process has been completed, and if it is determined that the development process has been completed, the process proceeds to step S608, and if it is determined that the development process has not been completed, the process returns to step S604. Whether or not to end the development process is determined based on the setting of the upper limit of the number of RAW image data sheets to be determined as the best shot and developed from one bundle RAW image file, and is determined based on whether the development of the set number of RAW image data sheets has been completed. The upper limit of the number of RAW image data sheets to be determined as the best shot and developed may be set in advance in the server 300, or may be settable in the server by some method.

[0069] In step S608, the control unit 301 notifies the camera 200 of the best shot extraction result, transmits developed image data to the camera 200, and ends the process. Details of the process in step S608 will be described later with reference to FIG. 7. The notification of the best shot extraction result from the server 300 to the camera 200 may be only the best shot information, such as the frame number of the RAW image data determined by the image determination unit 307 of the server 300. Also, developed image data obtained by developing the RAW image data determined to be the best shot may be transmitted, or both may be transmitted.

[0070] <Notification of best shot selection results> Next, a process of notifying the camera 200 of the best shot extraction result from the server 300 will be described with reference to FIG.

[0071] FIG. 7(a) is a diagram for explaining the notification process of the best shot extraction result. The camera 200 inquires as to which frame of the bundled RAW image file transmitted to the server 300 has been determined to be the best shot by transmitting a best shot extraction status inquiry 701 to the server 300. The trigger for transmitting the inquiry is, for example, when the image is reproduced by the camera 200. If the camera 200 is always connected to the server 300, the timing of the connection does not need to be considered, but if the camera 200 is not connected to the server 300, it is necessary to connect to the server 300 when transmitting the best shot extraction status inquiry 701. When the server 300 receives the best shot extraction status inquiry 701 from the camera 200, it transmits the best shot extraction status 702 in the server 300 to the camera 200 as a response.

[0072] 7(b) is a diagram illustrating an example of the transition of the best shot extraction state of the server 300. The state in which the server 300 has received the bundled RAW image file and has not yet started the best shot extraction process is referred to as "unprocessed" 705. The state in which the best shot extraction process is in progress is referred to as "best shot determination in progress" 706. The state in which the best shot extraction process has ended is referred to as "best shot extraction completed" 707. Note that although three states are illustrated in the example of FIG. 7, other states may also be present.

[0073] If the best shot extraction status 702, which is a response to the best shot extraction status query 701, is unprocessed 705 or best shot determination in progress 706, the best shot extraction status query 701 is repeatedly sent to the server 300 until best shot extraction is completed 707. When the camera 200 determines that the best shot extraction status 702 of the server 300 has reached best shot extraction completed 707, it sends a best shot information acquisition request 703 to the server 300, inquiring of the server 300 about the frame extracted as the best shot. In response to the best shot information acquisition request 703, the server 300 sends best shot information 704, such as the frame number of the RAW image data determined to be the best shot from the bundled RAW image file.

[0074] <Processing of Camera 200> Next, processing executed by camera 200 will be described with reference to FIG.

[0075] FIG. 8 is a flowchart showing the process of the camera 200 when the best shot image extracted by the server 300 and the representative image extracted by the camera 200 are different.

[0076] The process in Fig. 8 is realized by the control unit 201 of the camera 200 expanding a program stored in the non-volatile memory 202 into the working memory 203, executing it, and controlling each component. The process in Fig. 8 is started in response to the user selecting the bundled RAW image file 434 as the file to be played back on the selection screen 430 shown in Fig. 5(a). The process in Fig. 8 is premised on the fact that the server 300 has received the bundled RAW images in advance from the camera 200, and is in the best shot extraction completion state 707 described in Fig. 7. The same applies to Fig. 10 described later.

[0077] In step S801, the control unit 201 starts processing to read and play back the bundled RAW image file from the recording medium 208. If the camera 200 is not connected to the server 300, then in step S802, the control unit 201 starts processing to connect to the server 300.

[0078] In step S803, the control unit 201 compares the representative image of the bundle RAW image file for which playback processing has started in step S801 with the best shot image extracted by the server 300. If the control unit 201 determines that the representative image of the bundle RAW image file for which playback processing has started in step S801 is different from the best shot image extracted by the server 300, the control unit 201 proceeds to step S804. If the control unit 201 determines that the representative image of the bundle RAW image file for which playback processing has started in step S801 is the same as the best shot image extracted by the server 300, the control unit 201 ends the process.

[0079] In step S804, the control unit 201 determines whether or not to replace the representative image of the bundled RAW image file stored on the recording medium 208 with the best shot image extracted by the server 300. In this case, for example, a warning screen may be displayed before playback starts to notify the user that there is a difference between the representative image of the bundled RAW image file stored in the camera 200 and the best shot image extracted by the server 300, and the user may select whether or not to replace the representative image.

[0080] In step S804, if the control unit 201 determines that the representative image of the bundle RAW image file stored on the recording medium 208 should be replaced with the best shot image extracted by the server 300, the process proceeds to step S805; otherwise, the control unit 201 determines that the representative image should not be replaced, the process proceeds to step S806.

[0081] In step S805, the control unit 201 replaces the representative image of the bundled RAW image file stored on the recording medium 208 with the best shot image extracted by the server 300. This replacement involves replacing the frame number of the best shot image extracted by the server 300 with the representative image information in the meta information 421 of the bundled RAW image file 420 described in FIG.

[0082] In step S 806 , the control unit 201 records the frame number of the best shot image extracted by the server 300 in the best shot information in the meta information 421 of the bundled RAW image file 420 .

[0083] FIG. 9 is an explanatory diagram of the process during playback when the representative image of the bundled RAW image file stored in the camera 200 and the best shot image extracted by the server 300 are different.

[0084] When the representative image of the bundle RAW image file stored in the camera 200 and the best shot image extracted by the server 300 are the same, an image corresponding to the representative image of the bundle RAW image file stored in the camera 200 and the position corresponding to the mark 901 of the slider bar 441 is displayed as shown in FIG. 9(a). On the other hand, when the representative image of the bundle RAW image file stored in the camera 200 and the best shot image extracted by the server 300 are different, if there is a difference between the representative image information and the best shot image information during playback, a warning screen 903 is displayed on the display unit 206 as shown in FIG. 9(b), and the user is prompted to select whether or not to display the best shot image. When the user selects "Yes" 904 on the warning screen 903, the best shot image is displayed at a position corresponding to the mark 902 on the slider bar 441 shown in FIG. 9(a). When the user selects "No" 905 on the warning screen 903, the representative image is displayed at a position corresponding to the mark 901 on the slider bar 441.

[0085] In this way, when the representative image of the bundle RAW image files stored in camera 200 and the best shot image extracted by server 300 differ, the user can select either the representative image of the bundle RAW image files stored in camera 200 or the best shot image extracted by server 300 as the image to be played back, and the representative image of the bundle RAW image files stored in camera 200 can be replaced with the best shot image extracted by server 300, so that the image intended by the user can be displayed when playback of the bundle RAW image files begins.

[0086] [Variations] In the above embodiment, an example has been described in which the server 300 extracts the best shot image from the bundled RAW images and notifies the digital camera 200 of the best shot information. However, there is also a possibility that the user performs the best shot determination without the server 300 performing the best shot determination. For this reason, the following describes the process when the user selects the best shot image from the bundled RAW image file.

[0087] FIG. 10 is a flowchart showing the process of the camera 200 when the user selects a best shot image.

[0088] In step S1001, the control unit 201 starts the process of reading out the bundled RAW image file from the recording medium 208 and playing it back.

[0089] In step S1002, the control unit 201 controls the playback screen 440 in FIG. Ida By moving the position of the mark 442 on the bar 441, the position of the best shot image selected from the bundled RAW image files is stored.

[0091] In step S1003, the control unit 201 compares the representative image of the bundled RAW image file stored in the camera 200 with the best shot image selected by the user in step S1002. If the control unit 201 determines that the representative image of the bundled RAW image stored in the digital camera 200 and the best shot image selected in step S1002 are different, the process proceeds to step S1004, but if it determines that they are the same, the process ends.

[0092] In step S1004, the control unit 201 replaces the representative image of the bundled RAW image file stored in the camera 200 with the best shot image selected in step S1002. The position of the mark on the slider bar 441 on the playback screen 440 is also changed from the representative image of the bundled RAW image file stored in the camera 200 to the position of the best shot image selected by the user.

[0093] In this way, when the user selects a best shot image without having server 300 perform a best shot determination, the representative image of the bundled RAW image file stored in camera 200 is replaced with the best shot image selected by the user, and the position of the mark on slider bar 441 on playback screen 440 is also changed, so that the image intended by the user can be instantly displayed as the display target when playback of the bundled RAW image file begins.

[0094] [Other embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of each embodiment to a system or device via a network or a storage medium, and having one or more processors of a computer in the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0095] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0096] 100: communication network, 200: camera, 201: control unit, 204: imaging unit, 207: communication unit, 300: server, 301: control unit, 304: image processing unit, 305: communication unit, 307: image determination unit

Claims

1. An imaging device capable of communicating with an image processing device, a transmission means for transmitting a plurality of RAW images acquired by successively photographing the RAW images by the imaging means to the image processing device; a receiving means for extracting a predetermined RAW image that satisfies a predetermined condition from a plurality of RAW images received from the imaging device, and receiving an image obtained by developing the predetermined RAW image or a result of the extraction; An imaging device characterized by having a determination means for determining whether or not to replace a predetermined representative image with a predetermined RAW image extracted by the image processing device when a predetermined representative image among a plurality of RAW images acquired by the imaging means differs from the predetermined RAW image extracted by the image processing device.

2. 2. The imaging device according to claim 1, further comprising a notification means for notifying a user whether or not to replace the specified representative image with the specified RAW image when the specified representative image and the specified RAW image are different.

3. The image processing apparatus further includes a playback means for playing back a plurality of RAW images, 3. The imaging device according to claim 1, wherein when the predetermined representative image is replaced with the predetermined RAW image, the playback means changes the position at which image playback starts from the predetermined representative image to the predetermined RAW image.

4. 4. The imaging apparatus according to claim 1, further comprising a storage unit that stores information about the specified raw image when the specified representative image is not replaced with the specified raw image.

5. 2. The imaging apparatus according to claim 1, wherein the predetermined RAW image is not a RAW image extracted by the image processing apparatus, but a RAW image selected by a user from among the plurality of RAW images.

6. 4. The imaging apparatus according to claim 3, wherein the playback means displays the position of the predetermined representative image among the plurality of RAW images and / or the position of the predetermined RAW image.

7. 7. The imaging device according to claim 1, wherein the predetermined RAW image is a best shot image extracted by the image processing device from a plurality of RAW images received from the imaging device.

8. A method for controlling an imaging device capable of communicating with an image processing device, comprising: transmitting a plurality of RAW images continuously captured by an imaging means to the image processing device; extracting, in the image processing device, a predetermined RAW image that satisfies a predetermined condition from the plurality of RAW images received from the imaging device, and receiving an image obtained by developing the predetermined RAW image or a result of the extraction; a step of determining whether or not to replace a predetermined representative image with a predetermined RAW image extracted by the image processing device when a predetermined representative image among a plurality of RAW images acquired by the imaging means differs from a predetermined RAW image extracted by the image processing device.

9. A program for causing a computer to function as the imaging device according to any one of claims 1 to 7.

Images