Information processing system, information processing method, program, and storage media

Abstract

To provide an information processing system capable of redoing high image quality processing by reducing a storage capacity required for storing an image before executing the high image quality processing.SOLUTION: An information processing system includes: an image processing device configured to execute image processing for improving the quality of an image captured by an imaging device; and a first storage device configured to store the image captured by the imaging device. The image processing device determines whether or not to execute the image processing. The first storage device stores the image captured by the imaging device when it is determined to execute the image processing, and not to stores the image captured by the imaging device when it is determined not to execute the image processing.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing system, an information processing method, a program, and a storage medium. [Background technology]

[0002] 2. Description of the Related Art There is known a surveillance camera system that generates video images that have been subjected to high-quality processing based on imaging data on the surveillance camera side in environments with poor visibility, such as at night or when fog is present.

[0003] Patent document 1 discloses a technology in which low-quality and high-quality video data are generated when shooting video, and the low-quality video data or the high-quality video data to be distributed is switched according to user operation. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-58994 A Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the present invention is to provide an information processing system that reduces the storage capacity required to store an image before image quality improvement processing is performed, and that allows image quality improvement processing to be redone. [Means for solving the problem]

[0006] In order to solve the above problem, an information processing system according to one aspect of the present invention is an information processing system comprising an image processing device that performs image processing to improve the image quality of an image captured by an imaging device, and a first storage device that stores an image captured by the imaging device, wherein the image processing device decides whether or not to perform the image processing, and the first storage device stores the image captured by the imaging device when it is decided to perform the image processing, and does not store the image captured by the imaging device when it is decided not to perform the image processing. Effect of the Invention

[0007] According to the present invention, it is possible to provide an information processing system that allows redoing of image quality improvement processing while reducing the capacity required to store an image before image quality improvement processing is performed. [Brief description of the drawings]

[0008] [Figure 1] A diagram showing the system configuration of an information processing system. [Diagram 2] FIG. 1 is a diagram showing a hardware configuration and a functional configuration of an imaging device. [Diagram 3] FIG. 1 illustrates a hardware configuration of an image processing apparatus. [Figure 4] FIG. 1 illustrates hardware of an information processing device. [Diagram 5] FIG. 2 is a diagram showing a functional configuration of an information processing device; [Figure 6] 1 is a flowchart showing the operation of an information processing system. [Figure 7] FIG. 1 shows images stored in a storage device and the timing of storage. [Figure 8] 1 is a flowchart showing the operation of an information processing system. [Figure 9] FIG. 1 shows images stored in a storage device and the timing of storage. [Figure 10] A flowchart showing a method for controlling the timing of transmitting an image. [Figure 11] 1 is a flowchart showing the operation of an information processing system. [Figure 12] Flowchart showing the operation of redoing the image quality improvement process [Figure 13] 1 is a flowchart showing the operation of an information processing system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the present invention will be described in detail based on preferred embodiments thereof with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the configurations shown in the drawings.

[0010] Although the embodiment describes a plurality of features, not all of these features are essential to the invention, and the plurality of features may be arbitrarily combined. Furthermore, in the accompanying drawings, the same reference numerals are given to the same or similar configurations, and duplicated explanations are omitted.

[0011] <Embodiment 1> (System Configuration) 1 is a diagram showing an example of the configuration of an information processing system according to this embodiment. The information processing system 100 is composed of an imaging device 101, a first storage device 102, an image processing device 103, a network 104, an information processing device 105, a display device 106, an input device 107, and a second storage device 108.

[0012] The imaging device 101 is a device for capturing an image of a subject. The image captured by the imaging device 101 is output to an image processing device 103. The image captured by the imaging device 101 can also be stored in a first storage device.

[0013] The first storage device 102 is a device for storing images captured by the imaging device 101. The first storage device 102 is, for example, a hard disk drive (HDD) or a solid state drive (SSD).

[0014] An external storage device may also be used to fulfill a similar role. Here, the external storage device can be realized, for example, by a medium (recording medium) and an external storage drive for realizing access to the medium. Known examples of such media include flexible disks (FDs), CD-ROMs, DVDs, USB memories, MOs, and flash memories. The external storage device may also be a server device connected via a network, or may be built into the imaging device 101.

[0015] The image processing device 103 executes image processing for improving image quality, that is, high-image-quality processing. The high-image-quality processing refers to, but is not limited to, NR (noise reduction) processing, de-misting, super-resolution, HDR, and fluctuation removal. In addition, the image processing device 103 performs a compression-encoding process (hereinafter, compression process) on an image captured by the imaging device 101 or an image on which high-image-quality processing has been performed. A known method can be used as the compression-encoding method, and the compression process can be performed in a predetermined file format such as Motion Jpeg, H264, or H265. The image processing device 103 transmits an image to the information processing device 105 via the network 104. That is, the image processing device 103 can transmit an image on which at least one of high-image-quality processing and compression processing has been performed, or an image captured by the imaging device 101 without performing any processing.

[0016] The network 104 operates according to a communication standard such as Ethernet. It is configured to include one or more routers, switches, cables, etc. Note that the network 104 may be any network with no limitations on its communication standard, scale, or configuration, as long as it is capable of executing communication between the image processing device 103 and the information processing device 105. The image processing device 103 and the information processing device 105 are connected to each other via the network 104 so as to be able to communicate with each other.

[0017] The information processing device 105 receives an image transmitted from the image processing device 103. The information processing device 105 is connected to a display device 106, an input device 107, and a second storage device 108. The information processing device 105 outputs the received image to the display device 106 to display the image on the display device 106, and executes an operation according to input information from a user via the input device 107. The information processing device 105 can also execute various image processes including compression processing, image quality improvement processing, and decoding processing performed in the image processing device 103. That is, the information processing device 105 can receive an image captured by the imaging device 101 from the image processing device 103 as is, and execute image quality improvement processing in the image processing device 103.

[0018] The display device 106 is a device equipped with a monitor or the like for displaying an image input from the information processing device 105. The input device 107 is a keyboard, a mouse, or the like, and is an interface for inputting operation information for the information processing device 105, the image processing device 103, and the imaging device 101.

[0019] The second storage device 108 is a device similar to the first storage device. The second storage device stores images received from the image processing device 103 or images that have been decoded, compressed, or image-enhancing processed by the information processing device 105.

[0020] (Hardware configuration of imaging device) 2A is a diagram showing the hardware configuration of the imaging device 101. The imaging device 101 includes an imaging unit 201, a CPU 202, a RAM 203, a ROM 204, and a communication I / F 205.

[0021] The imaging unit 201 is composed of an imaging optical system consisting of at least one lens and an imaging element, and captures an image of a subject and outputs the image as an electrical signal. Examples of lenses include a zoom lens, a focus lens, and a blur correction lens. The imaging optical system uses the lens to focus light on the imaging element, form an image of the subject, and change the focus position. The imaging element converts the image of the subject received on the imaging surface into an electrical signal and outputs it.

[0022] The CPU 202 is a central processing unit that performs overall control of the image capture device 101. The RAM 203 provides a work area used when the CPU 202 executes processing. The RAM 203 also functions as a frame memory and a buffer memory.

[0023] The ROM 204 stores a program for the CPU 202 to control the image capturing apparatus 101 and the like.

[0024] The communication I / F 205 is an interface for outputting and transmitting the captured image to the first storage device 102, the image processing device 103, etc. The communication I / F 205 is, for example, a signal transmission standard such as SDI (Serial Digital Interface), or may be a LAN terminal for communicating with the information processing device 105 via a network.

[0025] (Functional configuration of the imaging device) Fig. 2(b) is a diagram showing the functional configuration of the imaging device 101. Of the functions shown in Fig. 2(b), those that are realized by hardware are given the same reference numerals and descriptions thereof will be omitted.

[0026] The imaging device 101 includes an imaging unit 201 , an image generating unit 206 , a camera control unit 207 , a high image quality determination unit 208 , and a communication control unit 209 .

[0027] The image generation unit 206 converts the electrical signal, which is an analog signal output by the imaging unit 201, into a digital signal to generate a RAW image. The camera control unit 207 controls the imaging device 101. The control of the imaging device 101 includes, for example, zoom control, focus control, exposure control, and the like. The image generation unit 206 can also execute various development processes including color conversion processing for outputting the RAW image via SDI. Therefore, the image generation unit 206 may output a RAW image, or may output an image that has been subjected to the above development process.

[0028] The high image quality determination unit 208 determines whether or not the image generated by the image generation unit 206 requires high image quality processing by the image processing device 103. Whether or not high image quality processing is required is determined based on the generated image. For example, whether or not high image quality processing is performed is determined based on parameters related to noise in the generated image. In addition, when the high image quality determination unit 208 determines that high image quality processing is unnecessary, it also determines whether or not a predetermined condition is satisfied. Here, the predetermined condition is a condition related to at least one of the noise amount, gain, contrast, fluctuation, and size of the subject of the generated image. Specifically, a threshold is set for a parameter indicating each element, and a determination is made as to whether or not the parameter is equal to or greater than the threshold. For example, it is determined whether or not the gain is equal to or greater than a threshold, or whether or not the contrast evaluation value indicating the contrast of the image is equal to or greater than a threshold. In this way, it is determined whether or not the predetermined condition is satisfied for an image determined to not require high image quality processing. The determination result by the high image quality determination unit 208 can be transmitted to the image processing device 103 or the information processing device 105.

[0029] The communication control unit 209 controls the communication of the imaging device 101. Specifically, the communication control unit 209 changes the transmission frequency of the images generated by the image generation unit 206, controls the amount of communication of the imaging device 101, and controls the communication destination of the imaging device 101.

[0030] (Hardware configuration of image processing device) 3 is a diagram showing a hardware configuration of the image processing device 103. The image processing device 103 includes a CPU 301, a RAM 302, a ROM 303, an input I / F 306, an output I / F 307, and a communication I / F 308.

[0031] The CPU 301 is a central processing unit that controls the image processing device 103. The RAM 302 provides a work area that is used when the CPU 301 executes processing. The RAM 302 also functions as a frame memory and a buffer memory.

[0032] The ROM 303 stores programs and the like for the CPU 301 to control the image processing device 103. For example, programs and the like for executing image quality improvement processing and compression processing are stored.

[0033] The input I / F 306 is an interface that accepts input to the image processing device 103. The input I / F 306 accepts, for example, an image output from the imaging device 101, instruction information related to the imaging device 101 output from the information processing device 105, etc. Alternatively, the input I / F 306 is a terminal for connecting an input device such as a keyboard or a mouse, or an SDI terminal for receiving an image output via SDI (SDI image).

[0034] The output I / F 307 is an interface for outputting images subjected to high-quality processing, compression processing, and the like by the image processing device 103, and various parameters in the high-quality processing. Specifically, it may be an SDI terminal or a USB terminal. An output device such as a monitor may be connected to the output I / F 307.

[0035] The communication I / F 308 is an interface for realizing communication with the information processing device 105 via the network 104. The communication I / F 308 is, for example, a terminal for connecting a LAN cable.

[0036] (Hardware configuration of information processing device) 4 is a diagram showing the hardware configuration of the information processing device 105 according to this embodiment. The information processing device 105 has a CPU 401, a RAM 402, a ROM 403, an input I / F 404, an output I / F 405, and a communication I / F 406. Descriptions of the roles of each unit that overlap with those of the image processing device 103 will be omitted.

[0037] The input I / F 404 is connected to the input device 107 and the second storage device 108, and accepts instructions from a user via the input device 107. The input I / F 404 also accepts an image stored in the second storage device 108 as an input.

[0038] The output I / F 405 is connected to the display device 106 and the second storage device 108, and an image output from the information processing device 105 via the output I / F 405 can be displayed on the display device 106. Furthermore, an image that has been subjected to image processing by the information processing device 105 is stored in the second storage device via the output I / F 405. As described above, the second storage device 108 stores an image captured by the imaging device 101 (or an image that has been subjected to image quality improvement processing or compression encoding processing by the image processing device 103). Therefore, the second storage device 108 can store a RAW image or an image that has been subjected to development processing.

[0039] (Functional configuration of information processing device) Fig. 5 is a block diagram showing the functional configuration of the information processing device 105 according to this embodiment. The information processing device 105 has an image acquisition unit 501, an image distribution unit 502, an image processing unit 503, a communication control unit 504, and a search unit 505. Each function shown in Fig. 5 can be realized by the CPU 401 executing a program stored in the RAM 402 or the ROM 403.

[0040] The image acquisition unit 501 acquires an image from the image processing device 103 or the second storage device. That is, the image acquisition unit 501 acquires an image (including a RAW image) captured by the imaging device 101, and an image on which at least one of image quality improvement processing and compression encoding processing has been performed by the image processing device 103.

[0041] The image delivery unit 502 executes a process of reading an image specified by the user from the second storage device 108 and delivering it to the display device 106 .

[0042] The image processing unit 503 reads the image specified by the user from the second storage device 108, and performs image quality improvement processing with the parameter settings specified by the user. The image that has been subjected to image quality improvement processing is output to the second storage device 108 and stored therein.

[0043] The communication control unit 504 transmits and receives data to and from connected devices via the network 104 .

[0044] The search unit 505 analyzes the image received from the image processing device 103 or the image output from the image processing unit 503, and searches for a similar image or an image with a similar analysis result from among the images stored in the second storage device 108.

[0045] (Operation description) Fig. 6 is a flowchart showing the operation of the information processing system 100. Fig. 6(a) shows the operation up to the transmission of an image captured by the imaging device 101 to the information processing device 105 via the image processing device 103. Fig. 6(b) shows the operation for redoing the image quality improvement process on an image selected from the images transmitted in Fig. 6(a). Note that the flowchart shown in Fig. 6 is realized by executing a computer program by the CPU of any one of the imaging device 101, the image processing device 103, and the information processing device 105.

[0046] In step S601, an image captured by the imaging device 101 is acquired. The image acquired here may be a RAW image, or may be an image that has been developed within the imaging device 101.

[0047] In step S602, it is determined whether the acquired image is an image that requires high image quality processing. This determination process may be executed in the imaging device 101 or in the image processing device 103. Examples of indicators for determining whether high image quality processing is required include the standard deviation of noise in the image and contrast. For example, if the standard deviation of noise in the image acquired in step S601 is greater than a predetermined threshold value, or if the evaluation value indicating the contrast is equal to or less than a predetermined threshold value, it is determined that high image quality processing is required. Note that the high image quality processing may be switched ON / OFF in response to an instruction from a user. In that case, a flag indicating whether or not to execute high image quality processing is implemented, and by referring to this flag, it is determined whether or not high image quality processing is required (whether or not to execute). If it is determined that high image quality processing is required in step S602 (YES in S602), the process proceeds to step S603. On the other hand, if it is determined that high image quality processing is not required (NO in S602), the process proceeds to step S606.

[0048] In step S603, the image processing device 103 performs image quality improvement processing on the image acquired in step S601.

[0049] In step S 604 , the image that has been subjected to the image quality improvement processing is transmitted to the information processing device 105 .

[0050] In step S605, the image captured by the imaging device 101 is stored in the first storage device 102.

[0051] In step S606, an image captured by the imaging device 101 or an image compressed and encoded by the image processing device 103 is transmitted. As described above, the image captured by the imaging device 101 is a RAW image or an image that has been subjected to development processing. Since the image is transmitted via the network 104, it is preferable to transmit a compressed and encoded image in consideration of the limitations of the communication band and the amount of communication. On the other hand, if it is desired to perform high image quality processing again within the information processing device 105, it is necessary to transmit the RAW image or the image that has been subjected to development processing (SDI image). This is because the RAW image or the image that has been subjected to development processing (SDI image) is indispensable for performing high image quality processing.

[0052] In step S611, the information processing device 105 selects an image for which image quality improvement processing is to be redone from the images stored in the second storage device 108 in response to a user input via the input device 107. Metadata related to the selected image and a signal requesting redoing of the image quality improvement processing are transmitted from the information processing device 105 to the image processing device 103. This metadata is used to identify an image corresponding to the selected image among the images stored in the first storage device 102. In other words, the metadata is used to identify an original image (an image before the image quality improvement processing is performed) corresponding to the selected image. The contents of the metadata may be any data capable of identifying the original image corresponding to the selected image. For example, an identification number or a timestamp assigned to each image may be used as the metadata.

[0053] In step S612, in response to a request received from the information processing device 105, it is determined whether or not to transmit the image stored in the first storage device 102 to the information processing device 105. This determination may be made manually by the user, or may be made based on the load on the network 104 calculated from the traffic situation of the network 104 and the size of the image. For example, if the load on the network 104 does not exceed a predetermined load, it is decided to transmit the image stored in the first storage device 102. Alternatively, it may be determined whether the data amount of the image to be transmitted is equal to or less than a predetermined amount, and if it is equal to or less than the predetermined amount, it may be determined to transmit the image. If there are multiple images to be transmitted, the load and data amount are calculated based on the number of images and the total communication amount. If it is determined that the image is to be transmitted, the process proceeds to step S613, and if it is determined that the image is not to be transmitted, the process proceeds to step S618. The image stored in the first storage device 102 is an image captured by the imaging device 101 and is an image before compression encoding (for example, a RAW image). Therefore, the data size of the image is larger than that of a compression encoded image, and the load on the network 104 due to the transmission of the image is also large. Step S612 is a process for transmitting the image before being compressed and encoded so that the load on the network 104 does not exceed a predetermined load.

[0054] In step S613, among the images stored in the first storage device 102, an image corresponding to the image selected by the information processing device 105 is transmitted to the information processing device 105. As described above, the image to be transmitted is specified using metadata acquired from the information processing device 105. Alternatively, the image may be specified according to an instruction from a user. For example, among the images stored in the first storage device 102, an image captured at a specified date and time or during a specified period may be transmitted.

[0055] In step S614, the information processing device 105 stores the received image, that is, the image read from the first storage device 102, in the second storage device .

[0056] In step S615, parameters for redoing the image quality improvement process are set. The parameters for redoing the process may be, for example, parameters input by the user.

[0057] In step S616, the image processing unit 503 in the information processing device 105 performs image quality improvement processing on the received image based on the parameters set in step S615, and generates an image that has been subjected to image quality improvement processing. That is, the image quality improvement processing is redone in the information processing device 105.

[0058] In step S617, the image on which the image quality improvement process has been performed is stored in the second storage device 108. When storing, the image selected in step S611 may be overwritten and stored, or the image may be associated with the selected image and stored together (without overwriting). Alternatively, the selected image and the image on which the image quality improvement process has been redone may be displayed side by side, and the user may select the image to store.

[0059] In step S618, parameters for redoing the image quality improvement process are transmitted from the information processing device 105 to the image processing device 103. In step S619, parameters for redoing the image quality improvement process are set in the image processing device 103 based on the received parameters.

[0060] In step S620, the image quality improvement process is redone in accordance with the set parameters for the image corresponding to the image selected in step S611, among the images stored in the first storage device 102. Note that the difference between step S616 and step S620 is whether the image quality improvement process is executed in the image processing device 103 or in the information processing device 105.

[0061] In step S 621 , the image that has been subjected to the high image quality processing in the image processing device 103 is further compressed to reduce the amount of image data to be transmitted, and then transmitted to the information processing device 105 .

[0062] In step S622, the information processing device 105 receives the image received from the image processing device 103 and stores it in the second storage device 108. As in step S617, the image may be overwritten or may be stored together.

[0063] Fig. 7 is a diagram showing an example of images stored in the first storage device 102 and the second storage device 108 in the information processing system 100 according to this embodiment and the timing of storage. The horizontal axis indicates the time axis, with t0 to t4 indicating times at certain timings. Note that the reference image in Fig. 7 is an image that has been captured by the imaging device 101 and developed. Preferably, it is an image that has been compression-encoded. In other words, the reference image is the image transmitted in step S606.

[0064] At times t0 to t1, it is determined that high image quality processing is not required, so reference image 1 is transmitted and the images are stored only in the second storage device 108. At times t1 to t2, it is determined that high image quality processing is required, so RAW image 1 is stored in the first storage device 102, and high image quality processed image 1 is stored in the second storage device 108. At times t2 to t3, it is determined that high image quality processing is not required, so only reference image 2 is stored in the second storage device 108. At times t3 to t4, it is determined that high image quality processing is required, so RAW image 2 is stored in the first storage device 102, and high image quality processed image 2 is stored in the second storage device 108. The stored RAW image may be a developed image, or may be an image for SDI output, for example.

[0065] In this way, the information processing system 100 according to this embodiment stores the original image (RAW image or SDI image) corresponding to the image on which image quality improvement processing has been performed in the first storage device 102. This makes it possible to reduce the required storage capacity compared to storing all images, and to provide an information processing system that allows the image quality improvement processing to be redone. Note that it is not necessary to store images on which image quality improvement processing has not been performed. This is because the possibility of redoing image quality improvement on an image that is determined not to require image quality improvement processing is lower than on an image on which image quality improvement processing has been performed. On the other hand, an image that has been image quality improved may require redoing image quality improvement processing due to factors such as insufficient correction or overcorrection, depending on the parameter settings.

[0066] The imaging device 101, the first storage device 102, and the image processing device 103 according to this embodiment may be integrated. In that case, each operation of the flowchart shown in FIG. 6(a) may be executed by one imaging device. That is, the imaging device has an acquisition unit, an image processing unit, a decision unit, and a storage unit. The acquisition unit is an acquisition means for acquiring an image, and for example, acquires an image captured by an image sensor in the imaging device. The image processing unit is an image processing unit corresponding to the image processing device 103, and executes image quality improvement processing to improve the image quality of the image acquired by the acquisition unit. The decision unit decides whether or not to execute image quality improvement processing on the image acquired by the acquisition unit. The storage unit stores an image for which it has been decided by the decision unit to execute image quality improvement processing, and does not store an image for which it has been decided not to execute image quality improvement processing.

[0067] <Embodiment 2> In the image that has not been subjected to image quality improvement in the first embodiment, there are cases where the target of surveillance cannot be satisfactorily confirmed or analyzed due to factors such as a slight decrease in contrast. In this embodiment, a case will be described in which images that have not been subjected to image quality improvement processing but are likely to be subjected to image quality improvement processing are also stored in the first storage device 102.

[0068] The system configuration, hardware configuration, and functional configuration of the information processing system 100 according to this embodiment are similar to those of the first embodiment, and therefore will not be described.

[0069] Fig. 8 is a flowchart showing the operation of the information processing system 100 according to this embodiment. Note that, among the operations shown in Fig. 8, the same operations as those in the flowchart shown in embodiment 1 will not be described. Also, the operations shown in Fig. 8 correspond to the operations in Fig. 6(a), and the operations corresponding to Fig. 6(b) in this embodiment will not be described because they are the same. The flowchart shown in Fig. 8 is realized by executing a computer program by the CPU of any one of the imaging device 101, the image processing device 103, and the information processing device 105.

[0070] In step S801, it is determined whether the image determined in step S606 as not requiring image quality improvement processing meets a predetermined condition. Here, the predetermined condition includes a case where the gain is equal to or greater than a predetermined value, and a case where the contrast evaluation value is equal to or greater than a predetermined value. In addition, it also includes a case where the size of the subject in the image is equal to or smaller than a predetermined size, and a case where the image capture device 101 is set to telephoto. Under such conditions, a user viewing the image may desire image quality improvement. Therefore, it is desirable to store the image in the first storage device 102 so that image quality improvement processing can be performed later. If it is determined in step S801 that the predetermined condition is met, the process proceeds to step S605, and the image captured by the image capture device 101 is saved in the first storage device 102.

[0071] 9 is an explanatory diagram showing images stored in the first storage device 102 and the second storage device 108 in this embodiment and the timing of storage. The horizontal axis indicates the time axis, and t0 to t4 indicate times at certain timings.

[0072] An example of the destinations and timings of storing RAW images and reference images in this embodiment is shown with reference to FIG. 9. At times t0 to t1, it is determined that image quality improvement processing is not necessary, so only reference image 1 is stored in the second storage device 108. At times t1 to t2, it is determined that image quality improvement processing is not necessary but is highly likely to be performed, so RAW image 1 is stored in the first storage device 102, and reference image 2 is stored in the second storage device 108. At times t2 to t3, it is determined that image quality improvement processing is not necessary, so only reference image 3 is stored in the second storage device 108. At times t_3 to t_4, it is determined that image quality improvement processing is not necessary but is highly likely to be performed, so RAW image 2 is stored in the first storage device 102, and reference image 4 is stored in the second storage device 108.

[0073] According to the information processing system of the present embodiment, it is possible to perform image quality improvement processing on images with poor visibility, even if the image quality improvement processing has not been performed on the images. Also, since only images that meet a predetermined condition are stored, it is possible to reduce the storage capacity required to store images before the image quality improvement processing is performed without compressing the storage area more than necessary.

[0074] <Embodiment 3> In this embodiment, a process of transmitting a RAW image stored in the first or second embodiment to the information processing device 105 at a predetermined timing will be described. One of the predetermined timings is, for example, a timing when a user desires to redo the image quality improvement process. When a user requests to redo the image quality improvement process, priority is given to transmitting the RAW image over the congestion status of the network. Another timing is a timing when the network is not congested. When transmitting a RAW image via a network, a load is placed on the network, so the image is transmitted at a timing when the network load is low.

[0075] The system configuration, hardware configuration, and functional configuration of the information processing system 100 according to this embodiment are similar to those of the first embodiment, and therefore will not be described.

[0076] Fig. 7 is a flowchart showing a process flow in the information processing system according to this embodiment. The flowchart shown in Fig. 7 is realized by executing a computer program by the CPU of any one of the imaging device 101, the image processing device 103, and the information processing device 105.

[0077] Step S1001 is a step in which a RAW image is stored in the first storage device 102 while image quality improvement processing is being performed in the first or second embodiment or while it is determined that there is a high possibility that image quality improvement processing will be performed.

[0078] In step S1002, it is determined whether or not it is a predetermined timing. Here, the predetermined timing includes a time when the network 104 is not congested, a time designated by the user, a time when real-time monitoring is not being performed, and the like.

[0079] In step S1003, all RAW images stored in the first storage device 102 when it is determined in S702 that it is the predetermined timing are transmitted to the information processing device 105 via the network 104. Here, it is assumed that the RAW images stored in the first storage device 102 may be deleted when the transmission is completed.

[0080] In step S1004, the information processing apparatus 105 stores all the RAW images received via the network 104 in the second storage device .

[0081] In step S1005, if it is not determined in step S1002 that it is the predetermined timing, the process waits for a predetermined time and then executes step S1002 again. Here, the predetermined time may be set automatically or freely by the user.

[0082] As described above, by transmitting a RAW image to the information processing device 105 at a timing desired by the user or when the network is not congested, it is possible to improve user convenience and reduce the load on the network.

[0083] <Embodiment 4> In this embodiment, a process will be described in which a user adds image analysis results before saving a RAW image, for the purpose of assisting the user in setting parameters when re-executing image quality improvement processing.

[0084] The image analysis results include information such as the analysis results of the reference image (average brightness, contrast, etc.), the parameter settings when the image quality improvement process was performed, and other parameter setting candidates, etc. By checking the image analysis results added to the RAW image, the user can perform the image quality improvement process with other parameter setting candidates included in the image analysis results when re-executing the image quality improvement process, and can also set appropriate parameters by himself / herself based on the analysis results of the reference image.

[0085] The system configuration, hardware configuration, and functional configuration of the information processing system 100 according to this embodiment are similar to those of the first embodiment, and therefore will not be described.

[0086] Fig. 11 is a flowchart showing the flow of processing in the information processing system 100 according to this embodiment. Among the operations in the flowchart shown in Fig. 11, the same operations as those in the first embodiment are denoted by the same reference numerals, and the description thereof will be omitted. The flowchart shown in Fig. 11 is realized by executing a computer program by the CPU of any one of the imaging device 101, the image processing device 103, and the information processing device 105.

[0087] In step S1101, the image acquired in step S601 is provided to the RAW image with the analysis results obtained by the image quality improvement determination unit 208. The information to be provided includes image analysis results such as average luminance and contrast, and parameter settings and other candidates for image quality improvement processing determined by the image quality improvement determination unit 208 based on the image analysis results.

[0088] When redoing the image quality improvement process, the parameter settings and the processing method including model changes to be processed by the image processing device 103 are determined based on the parameter settings and information obtained from the image analysis results. This process may be executed automatically, or may be displayed to the user on a display unit (not shown) as change parameter candidates.

[0089] As a result, the user can quickly and easily set parameters for image quality improvement processing by checking the image analysis results added to the RAW image.

[0090] <Embodiment 5> In this embodiment, a process will be described when the information processing device 105 searches the second storage device 108 for an image similar to the image to be analyzed or an analysis result similar to the analysis result given to the image to be analyzed.

[0091] Consider a case where an image captured under similar camera settings, camera angle of view, and environment to the target image for which a user requests redoing of image quality improvement processing has been redone in the past. In that case, it is considered that the parameter settings used can also be applied to the image quality improvement processing of the target image. Therefore, when there is an image for which image quality improvement processing is to be redone, an image similar to the analysis target image or an analysis result similar to the analysis result given to the analysis target image is searched for in the second storage device 108, and image quality improvement processing is performed based on the parameter settings given to that image.

[0092] 12 is a flowchart showing the flow of processing of the information processing device 105 according to this embodiment. The operations shown in this flowchart are realized by the CPU 401 in the information processing device 105 executing a program.

[0093] In step S1201, an image to be analyzed is selected from the images that have been subjected to image quality improvement processing and are stored in the second storage device 108.

[0094] In step S1202, it is determined whether or not the image quality improvement process needs to be redone. The determination of whether or not to redo the image quality improvement process may be made according to an instruction from a user, or may be made according to a parameter indicating the visibility of the image. The parameter indicating the visibility of the image is, for example, a contrast evaluation value, and if the parameter indicating the visibility of the image is less than a predetermined threshold, it is determined that the image quality improvement process needs to be redone. If it is determined that the image quality improvement process needs to be redone, the process proceeds to step S1203.

[0095] In step S1203, an image similar to the image to be analyzed or an analysis result similar to the analysis result assigned to the image to be analyzed is searched for among images that have been subjected to image quality improvement processing and are stored in the second storage device 108. This processing is executed by the search unit 505.

[0096] In step S1204, it is determined whether an image similar to the image to be analyzed or an analysis result similar to the analysis result added to the image to be analyzed is confirmed. If there is a similar image or analysis result, the process proceeds to step S1205.

[0097] In step S1205, parameter setting candidates when performing image quality improvement processing associated with the target image are obtained.

[0098] In step S1206, the parameter setting candidates acquired in step S1205 are presented, for example, on the display device 106.

[0099] In step S1207, the parameters selected by the user from among the presented parameter setting candidates or the parameters separately set by the user are set as the parameters to be used when the image quality improvement process is executed.

[0100] In step S1208, the image processing unit 503 in the information processing device 105 executes image quality improvement processing in accordance with the set parameters. The image to be subjected to image quality improvement processing is an image corresponding to the image to be analyzed, among the images stored in the first storage device 102 or the second storage device 108.

[0101] In step S1209, the image that has undergone image quality improvement processing and the parameter settings used when the image quality improvement processing was performed are stored in the second storage device 108 in association with each other.

[0102] As a result, by making it possible to check images similar to the image to be analyzed or analysis results similar to the analysis results assigned to the image to be analyzed, it is possible to support the user in setting parameters when re-executing image quality improvement processing.

[0103] <Embodiment 6> In this embodiment, a process for reflecting the tendency of past image quality improvement processes when performing real-time image quality improvement processes will be described.

[0104] In the image quality improvement process that is automatically performed in real time, similar image quality improvement processes are performed under the same environment unless the learned model is updated or switched. For example, consider a case where the strength of the image quality improvement process was weak when the image quality improvement process was performed under a specific environment. In this case, as long as the environment is the same, the image quality improvement strength tends to be weak for all images. If a user requests redoing the image quality improvement process for this image quality improvement processed image, the image quality improvement process will be redone for all images taken under the specific environment, resulting in a great deal of rework. In order to deal with the above-mentioned case, if it is known that the strength of the image quality improvement process performed in the past under a specific environment tends to be weak, the strength is increased when the real-time image quality improvement process is performed. In this way, the tendency of redoing the image quality improvement process performed in the past is reflected in the real-time image quality improvement process. Note that the hardware configuration and functional configuration of each device constituting the information processing system according to this embodiment are the same as those of the first embodiment, and therefore will not be described.

[0105] 13 is a flowchart showing the operation of the information processing system 100 according to this embodiment. In the flowchart shown in this embodiment, the same operations as those in the first embodiment are denoted by the same reference numerals, and the description thereof will be omitted.

[0106] As shown in FIG. 13, the present embodiment differs from the first embodiment in the operations of steps S1301 to S1304.

[0107] In step S1301, it is confirmed whether or not a parameter analysis result exists in the first storage device 102. Here, the parameter analysis result is information including the parameter settings when the image quality improvement process was executed and the degree of correction of the strength of the image quality improvement process. The search unit 505 analyzes the re-image quality improvement processed image stored in the second storage device 108 at a predetermined timing, and the parameter analysis result is stored in the first storage device 102. In addition, the predetermined timing is a periodic timing such as every hour or every day, or a timing specified by the user. If the parameter analysis result exists, the process proceeds to step S1302.

[0108] In step S1302, the parameter analysis result is obtained from the first storage device 102.

[0109] In step S1303, the image processing device 103 executes image quality improvement processing based on the parameter analysis result. For example, a case will be described in which a statistical result is included that a re-image quality improvement processing with a weakened intensity is executed on an image quality improvement processed image acquired in a low-luminance environment. In this case, during real-time image quality improvement processing executed in a low-luminance environment, a correction to weaken the intensity may be executed on the determined parameter setting. In this way, the parameters used in redoing the image quality improvement processing and the change tendency of the parameters used in the previous image quality improvement processing are linked and stored. In addition, when the image quality improvement processing is redoed, parameters related to the image including luminance information and the like are linked and stored with the above-mentioned change tendency. Then, these are fed back to the parameter setting when redoing the image quality improvement processing as the parameter analysis result. These analyses can also be realized by a learning model learned by machine learning. That is, machine learning is executed using the luminance information of the image and the above-mentioned change tendency as learning data, and the parameters used in redoing the image quality improvement processing are set by the learned model.

[0110] As a result, the trends of previously performed re-image quality improvement processing can be reflected in the real-time image quality improvement processing, thereby saving the user the trouble of re-executing the image quality improvement processing, and enabling the user to obtain an image with the image quality improvement processing desired in real time.

[0111] <Other embodiments> Although the embodiment has been described in detail above, the present invention can be embodied as, for example, a system, an apparatus, a method, a program, or a recording medium (storage medium), etc. Specifically, the present invention may be applied to a system composed of a host computer, an interface device, an imaging device, a web application, etc., or may be applied to an apparatus composed of a single device.

[0112] Needless to say, the object of the present invention can be achieved by the following: That is, a recording medium (or storage medium) on which is recorded a program code (computer program) of software that realizes the functions of the above-mentioned embodiments is supplied to a system or device. The storage medium is, of course, a computer-readable storage medium. Then, a computer (or a CPU or MPU) of the system or device reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-mentioned embodiments, and the recording medium on which the program code is recorded constitutes the present invention. [Explanation of symbols]

[0113] 100 Information Processing Systems 101 Imaging device 102 First storage device 103 Image processing device 104 Network 105 Information processing equipment

Claims

1. an image processing device that performs image processing to improve the image quality of an image captured by the imaging device; a first storage device that stores an image captured by the imaging device, The image processing device determines whether to execute the image processing; the first storage device stores an image captured by the imaging device when it is determined that the image processing is to be performed; An information processing system, comprising: an image processing unit that stores an image captured by the image capturing device when it is determined that the image processing is not to be performed;

2. 2. The information processing system according to claim 1, wherein whether or not to execute the image processing is determined based on an image captured by the imaging device.

3. 3. The information processing system according to claim 2, wherein whether or not to execute the image processing is determined based on a parameter related to noise in the image captured by the imaging device.

4. 2. The information processing system according to claim 1, wherein whether or not to execute the image processing is determined in response to a request from the imaging device or the information processing device.

5. 2. The information processing system according to claim 1, wherein the image processing is executed on the image stored in the first storage device in response to a request from an information processing device.

6. the image processing device performs a compression process on an image captured by the imaging device or an image on which the image processing has been performed; 2. The information processing system according to claim 1, wherein an image on which at least one of the image processing and the compression processing has been performed or an image on which neither the image processing nor the compression processing has been performed is transmitted.

7. The information processing system according to claim 1, characterized in that, among a plurality of images captured by the imaging device, images for which it has been determined that the image processing is not to be performed and which satisfy predetermined conditions are stored in the first storage device.

8. 8. The information processing system according to claim 7, wherein the predetermined condition is a condition related to at least one of gain, contrast, fluctuation, and size of a subject of an image captured by the imaging device.

9. The information processing system according to claim 7 , wherein the predetermined condition is whether or not a gain or an amount of noise in the image captured by the imaging device is equal to or greater than a threshold value.

10. 2. The information processing system according to claim 1, wherein an image captured by said imaging device is transmitted at a predetermined timing.

11. a second storage device that stores the image transmitted at the predetermined timing; An information processing device connected to the image processing device via a network so as to be capable of communicating with the image processing device, 11. The information processing system according to claim 10, wherein the information processing device performs image processing to improve image quality on the image stored in the second storage device in response to an instruction input to an input device.

12. An acquisition step of acquiring an image; a decision step of deciding whether or not to perform image processing for improving image quality on the image acquired in the acquisition step; a storage step of storing the image acquired in the acquisition step when it is determined that the image processing is to be performed, and not storing the image acquired in the acquisition step when it is determined that the image processing is not to be performed; An information processing method comprising the steps of:

13. A program for causing a computer to execute the information processing method according to claim 12.

14. A computer-readable storage medium storing the program according to claim 13.

Images