Information Processing System, Information Processing Method, and Information Processing Apparatus

Abstract

This information processing system (1) has: a specification means (12) that, in accordance with the image-capture conditions when an image is captured by an image-capture device (20) and an item of analysis with regard to the image, which is delivered via a network, specifies a first image quality for a region of a specific site in the image that is used in analysis; and a control means (13) that performs control such that the region of the specific site in the image is delivered in the first image quality.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, and an information processing device. [Background technology]

[0002] There are known techniques for performing various analyses (medical treatment, diagnosis, examination) based on images such as still images and moving images (videos). In relation to this technique, Patent Document 1 describes a technique for simply measuring fluctuations in the blood pressure of a subject based on a video signal obtained by photographing a predetermined part of the subject. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-097757 Summary of the Invention [Problem to be solved by the invention]

[0004] However, Patent Document 1 does not consider how to handle analysis based on video that has been coded for distribution, etc. Therefore, the technology described in Patent Document 1 has a problem in that, for example, it may not be possible to properly perform analysis based on images (including still images and moving images (video)) distributed via a network.

[0005] In view of the above-mentioned problems, an object of the present disclosure is to provide a technology that can appropriately perform analysis based on images distributed via a network. [Means for solving the problem]

[0006] In a first aspect according to the present disclosure, an information processing system includes a specifying means for specifying a first image quality of a region of a specific part used for analysis in the image according to a shooting situation when an image is shot by a shooting device and an analysis item for the image distributed via a network, and a control means for performing control to distribute the region of the specific part in the image as the first image quality.

[0007] Also, in a second aspect according to the present disclosure, there is provided an information processing method for executing a process of specifying a first image quality of a region of a specific part used for analysis in the image according to a shooting situation when an image is shot by a shooting device and an analysis item for the image distributed via a network, and a process of controlling to distribute the region of the specific part in the image as the first image quality.

[0008] Also, in a third aspect according to the present disclosure, an information processing apparatus includes a specifying means for specifying a first image quality of a region of a specific part used for analysis in the image according to a shooting situation when an image is shot by a shooting device and an analysis item for the image distributed via a network, and a control means for performing control to distribute the region of the specific part in the image as the first image quality.

Advantages of the Invention

[0009] According to one aspect, analysis based on an image distributed via a network can be appropriately executed.

Brief Description of the Drawings

[0010]

Figure 1A

Figure 1B

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Mode for Carrying Out the Invention

[0011] [[ID=3X]] The principles of the present disclosure are described with reference to several exemplary embodiments. These embodiments are described for illustrative purposes only and are intended to help those skilled in the art understand and implement the present disclosure without suggesting any limitation on the scope of the present disclosure. The disclosure described herein may be implemented in various ways other than those described below. In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0012] <First Embodiment> <Configuration> Referring to FIG. 1A, the configuration of the information processing system 1 according to the embodiment will be described. FIG. 1A is a diagram showing an example of the configuration of the information processing system 1 according to the embodiment. The information processing system 1 includes a specific part 12 and a control part 13. It should be noted that in the original text, there is an unclear "3X" in the line with ID = 3X. This may be an error in the original text. The translation is based on the best understanding of the overall context.

[0013] The specific unit 12 may specify (determine, estimate) the shooting situation when the image is shot by the imaging device 20, for example, based on an image or the like that is encoded and distributed via the network N. The shooting situation may be specified (determined, estimated) by other functional units of the imaging device 20. Alternatively, the specification may be executed by an external device such as a cloud or a server. In this case, the specific unit 12 may transmit the image to the external device and acquire the specification result from the external device. Note that the shooting situation is, for example, the state (situation) of the subject, the situation around the subject, or the situation of the imaging device 20 when the subject is shot. The state of the subject may include, for example, the distance (m) from the imaging device 20 to the subject, the orientation of the subject with respect to the imaging device 20, and the size of the specific part used for analysis (the number of pixels included in the area of the specific part). In addition, the situation around the subject may include, for example, the brightness of the environment around the subject. Further, the situation of the imaging device 20 may include, for example, the performance of the imaging device 20.

[0014] Further, the specific unit 12 specifies the image quality of the area of the specific part used for the analysis of the item to be analyzed (hereinafter also appropriately referred to as "analysis target") in the image. In this case, the specific unit 12 may determine the image quality according to, for example, the shooting situation when the image is shot by the imaging device 20 and the item to be analyzed for the image distributed via the network N.

[0015] Further, the specific unit 12 may cause an analysis module or the like inside or outside the information processing device 10 to execute an analysis (inspection, analysis, estimation) based on the area of the specific part of the subject in the image. For example, the heart rate may be analyzed based on an image of the area of the subject's face. When the analysis is executed by an external device, the specific unit 12 may transmit the image to the external device and acquire the analysis result from the external device.

[0016] Further, the specific unit 12 may receive (acquire) various types of information from a storage unit inside the information processing apparatus 10 or an external device. Further, the specific unit 12 may execute various types of processing based on an image photographed by the photographing apparatus 20 and distributed.

[0017] The control unit 13 transmits (outputs) information based on the determination result by the specific unit 12 to each processing unit inside the information processing apparatus 10 or an external device. For example, the control unit 13 transmits information (command) for causing an image in which the area of a specific part is the image quality determined by the specific unit 12 to be distributed. Note that the information processing apparatus 10 may be a device to which an image photographed by the photographing apparatus 20 and encoded is distributed, or may be a device from which an image photographed by the photographing apparatus 20 and encoded is distributed.

[0018] Further, the specific unit 12 and the control unit 13 may be integrated into one device as shown in FIG. 1B. In the example of FIG. 1B, the information processing system 1 includes an information processing apparatus 10 and a photographing apparatus 20. The photographing apparatus 20 is a device that photographs a subject, and may be, for example, a camera built in a smartphone, a tablet, or the like. Further, the photographing apparatus 20 may be, for example, a camera connected to a personal computer or the like by an external bus. The information processing apparatus 10 includes a specific unit 12 and a control unit 13. Each of these units may be realized by cooperation between one or more programs installed in the information processing apparatus 10 and hardware such as a processor 101 and a memory 102 of the information processing apparatus 10.

[0019] <Processing> Next, with reference to FIG. 2, an example of the processing of the information processing system 1 according to the embodiment will be described. FIG. 2 is a flowchart showing an example of the processing of the information processing system 1 according to the embodiment.

[0020] In step S1, the specifying unit 12 determines the image quality of the area of the specific part in the image captured by the imaging device 20 according to the imaging situation when the imaging device 20 captures an image and the analysis target which is the area of the specific part of the subject in the image captured by the imaging device 20 and distributed via the network N. Subsequently, the control unit 13 transmits information for causing the image with the area of the specific part having the image quality to be distributed (step S2).

[0021] (Processing example when the information processing device 10 is the device of the image distribution destination) When the information processing device 10 is the device of the image distribution destination, the specifying unit 12 may receive the image via the network N. Then, the specifying unit 12 may determine the image quality according to the imaging situation and the analysis target. Then, the control unit 13 may transmit a command for setting (changing) the image distributed from the device of the distribution destination to the image quality to the device of the distribution destination.

[0022] (Processing example when the information processing device 10 is the device of the image distribution source) When the information processing device 10 is the device of the image distribution source, the specifying unit 12 may receive the image from the imaging device 20 built in the information processing device 10 via the internal bus. Also, the specifying unit 12 may receive the image from an external (external attached) imaging device 20 connected to the information processing device 10 by a cable or the like via an external bus (for example, a USB (Universal Serial Bus) cable, an HDMI (registered trademark) (High-Definition Multimedia Interface) cable, an SDI (Serial Digital Interface) cable). Then, the specifying unit 12 may determine the image quality according to the imaging situation and the analysis target. Then, the control unit 13 may transmit a command for setting (changing) the image distributed from the information processing device 10 to the image quality to the module that performs the encoding process inside the information processing device 10 or to the imaging device 20.

[0023] <Hardware configuration> Fig. 3 is a diagram showing an example of the hardware configuration of an information processing device 10 according to an embodiment. In the example of Fig. 3, the information processing device 10 (computer 100) includes a processor 101, a memory 102, and a communication interface 103. These components may be connected via a bus or the like. The memory 102 stores at least a part of a program 104. The communication interface 103 includes an interface required for communication with other devices having communication functions.

[0024] When the program 104 is executed by the processor 101, memory 102, and other components in cooperation with each other, the computer 100 performs at least some of the processing of the embodiments of the present disclosure. The memory 102 may be of any type suitable for a local technology network. The memory 102 may be, by way of non-limiting example, a non-transitory computer-readable storage medium. The memory 102 may also be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. While only one memory 102 is shown in the computer 100, several physically distinct memory modules may be present in the computer 100. The processor 101 may be of any type. The processor 101 may include one or more of a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), and, by way of non-limiting example, a processor based on a multi-core processor architecture. The computer 100 may have multiple processors, such as application-specific integrated circuit chips time-slaved to a clock that synchronizes the main processor.

[0025] Embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device.

[0026] The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions such as instructions included in program modules, which are executed on a device on a target actual processor or virtual processor to execute the process or method of the present disclosure. Program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The functions of the program modules may be combined or divided among the program modules as desired in various embodiments. The machine-executable instructions of the program modules can be executed within a local or distributed device. In a distributed device, the program modules can be arranged on both local and remote storage media.

[0027] The program code for executing the method of the present disclosure may be written in any combination of one or more programming languages. This program code is provided to a processor or controller of a general-purpose computer, a dedicated computer, or other programmable data processing device. When the program code is executed by the processor or controller, the functions / operations in the flowchart and / or the block diagram to be implemented are executed. The program code is executed entirely on the machine, partly on the machine as a stand-alone software package, partly on the machine, partly on a remote machine, or entirely on a remote machine or server.

[0028] The program can be stored and provided to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible recording media. Examples of non-transitory computer-readable media include magnetic recording media, magneto-optical recording media, optical disk media, and semiconductor memory. Magnetic recording media include, for example, flexible disks, magnetic tapes, and hard disk drives. Magneto-optical recording media include, for example, magneto-optical disks. Optical disk media include, for example, Blu-ray discs, CD (Compact Disc)-ROMs (Read Only Memory), CD-Rs (Recordable), and CD-RWs (Rewritable). Semiconductor memory includes, for example, solid-state drives, mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory). The program may also be provided to a computer by various types of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire or an optical fiber, or via a wireless communication path.

[0029] Second Embodiment <System configuration> Next, the configuration of the information processing system 1 according to the embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the configuration of the information processing system 1 according to the embodiment. In the example of Fig. 4, the information processing system 1 has an image capturing device 20 and an information processing device 10. Note that the number of image capturing devices 20 and information processing devices 10 is not limited to the example of Fig. 4.

[0030] The technology disclosed herein may be used, for example, to measure biometric information based on images of a patient in a video conference (video call, online medical consultation) between a doctor and a patient (human or animal). The technology disclosed herein may also be used, for example, to analyze (identify) a person and analyze (estimate) their behavior based on images from a surveillance camera. The technology disclosed herein may also be used, for example, to analyze (inspect) a product based on images from a surveillance camera in a factory or plant.

[0031] 4, the image capturing device 20 and the information processing device 10 are connected to each other so as to be able to communicate with each other via a network N. Examples of the network N include the Internet, a mobile communication system, a wireless LAN (Local Area Network), a LAN, and a short-range wireless communication system such as Bluetooth (registered trademark) Low Energy (BLE). Examples of the mobile communication system include a fifth-generation mobile communication system (5G), a fourth-generation mobile communication system (4G), a third-generation mobile communication system (3G), and the like.

[0032] The image capturing device 20 may be, for example, a device including a smartphone, a tablet, a personal computer, etc. The image capturing device 20 encodes captured images (including still images and moving images (video)) using any encoding method and distributes the encoded images to the information processing device 10 via the network N. The encoding method may include, for example, H.265 / HEVC (High Efficiency Video Coding), AV1 (AOMedia Video 1), H.264 / MPEG-4 AVC (Advanced Video Coding), etc.

[0033] The information processing device 10 may be, for example, a personal computer, a server, a cloud, a smartphone, a tablet, etc. The information processing device 10 performs analysis based on the image delivered from the image capturing device 20.

[0034] <Processing> Next, referring to FIGS. 5 to 9, an example of the processing of the information processing system 1 according to the embodiment will be described. FIG. 5 is a sequence diagram showing an example of the processing of the information processing system 1 according to the embodiment. FIG. 6 is a diagram showing an example of the specific part DB (database) 601 according to the embodiment. FIG. 7 is a diagram showing an example of the image quality setting DB 701 according to the embodiment. FIG. 8 is a diagram showing an example of the analysis result history DB 801 according to the embodiment. FIG. 9 is a diagram showing an example of the area of the specific part according to the embodiment.

[0035] Hereinafter, as an example, the case of measuring biometric information based on a patient's image in a video conference (video call, online medical treatment) between a doctor and a patient will be described. Hereinafter, it is assumed that the processing such as the establishment of the video conference session has already been completed between the patient's imaging device 20 and the doctor's information processing device 10.

[0036] In step S101, the imaging device 20 distributes (transmits) a first image obtained by encoding the area of the specific part of the subject in the captured image to the information processing device 10 via the network N. Here, the imaging device 20 may distribute a first image obtained by encoding the area of the specific part with a specific image quality and encoding the area other than the specific part with an image quality lower than the specific image quality. In other words, the imaging device 20 may encode and distribute the image with an image quality such that the area of the specific part in the captured image is clearly displayed and the area other than the area of the specific part is displayed less clearly than the area of the specific part.

[0037] Subsequently, based on the received first image and the like, the specifying unit 12 of the information processing apparatus 10 specifies the shooting situation when the first image is shot by the shooting apparatus 20 (step S102). Here, the specifying unit 12 of the information processing apparatus 10 may specify the shooting situation by, for example, an AI (Artificial Intelligence) using deep learning or the like. The shooting situation may include at least one of the distance (m) from the shooting apparatus 20 to the subject, the orientation of the subject with respect to the shooting apparatus 20, the number of pixels included in the area of the specific part used for analysis, the brightness of the environment around the subject, and the performance of the shooting apparatus 20. Further, the shooting situation may further include at least one of the encoding method (for example, H.264, H.265, etc.) when the image shot by the shooting apparatus 20 is distributed via the network N, and the bandwidth available on the network N.

[0038] The specifying unit 12 of the information processing apparatus 10 may calculate the distance from the shooting apparatus 20 to the subject based on, for example, the value of the ratio of the number of pixels included in the area of the subject to the number of pixels of the entire received frame. In this case, the specifying unit 12 of the information processing apparatus 10 may determine that, for example, the larger the value of the ratio, the smaller (closer) the distance from the shooting apparatus 20 to the subject. Further, the specifying unit 12 of the information processing apparatus 10 may use, for example, a Depth estimation technique for estimating the distance to each pixel in the image. Further, the specifying unit 12 of the information processing apparatus 10 may measure the distance by, for example, a stereo camera or LiDAR. The orientation of the subject with respect to the shooting apparatus 20 may be, for example, information indicating how much the front of the subject is displaced from the shooting apparatus 20 in at least one of the up, down, left, and right directions within the image.

[0039] The number of pixels included in the region of the specific part used for the analysis is the number of pixels included in the region of the specific part corresponding to the analysis target analyzed in step S107. Note that the analysis target may be specified (selected, set) in advance by a doctor or the like. Also, the specific part 12 of the information processing apparatus 10 may determine one or more analysis targets based on the result of a medical interview input in advance from a patient by a predetermined website or the like. The specific part 12 of the information processing apparatus 10 may determine a specific part corresponding to the analysis target, for example, by referring to the specific part DB 601. In the example of FIG. 6, the specific part DB 601 records the specific part of the subject used for the analysis in association with the analysis target. Note that the specific part DB 601 may be stored (registered, set) in the storage device inside the information processing apparatus 10, or may be stored in a DB server or the like outside the information processing apparatus 10. In the example of FIG. 6, for example, when the analysis target is the heart rate, it is recorded that the region of the face (cheek) in the image is used for the analysis. Then, the specific part 12 of the information processing apparatus 10 may detect the region of the specific part in the received image by object recognition or the like, and calculate the number of pixels in the detected region.

[0040] The brightness of the environment around the subject is the brightness of the environment around the subject photographed by the photographing apparatus 20, including ambient light and the brightness by the flash light of the photographing apparatus 20. The performance of the photographing apparatus 20 may include, for example, the focal length, the presence or absence of HDR (high dynamic range), the color depth, the resolution of a still image, the resolution of a moving image, and the maximum frame rate. Note that the specific part 12 of the information processing apparatus 10 may acquire information indicating the performance of the photographing apparatus 20 from the photographing apparatus 20. In this case, the information indicating the performance of the photographing apparatus 20 may include, for example, the model name (product name) of the photographing apparatus 20, or when the photographing apparatus 20 is built in the photographing apparatus 20, the model name of the photographing apparatus 20. In this case, the specific part 12 of the information processing apparatus 10 may acquire the value of each performance of the photographing apparatus 20 based on the model name using a table or the like in which a pre-registered model name and the value of each performance are associated with each other.

[0041] Next, the determination unit 12 of the information processing device 10 determines the first image quality of the specific body part area of the subject in accordance with the shooting conditions and the analysis target for analysis based on the specific body part area (step S103). This allows, for example, determining an image quality that allows appropriate analysis based on the image distributed via a network. For example, if the image quality of the specific body part area according to the shooting conditions and the analysis target is increased, the reliability (accuracy) of the analysis results can be improved. Furthermore, for example, if the image quality of the specific body part area according to the shooting conditions and the analysis target is decreased, the bandwidth usage of the network N can be reduced. Furthermore, for example, even for images in which the distance from the imaging device 20 to the patient is relatively large, the accuracy of analysis can be improved. Furthermore, for example, by increasing the image quality of only the specific body part area, the increase in bandwidth used for distribution can be reduced.

[0042] Here, the identification unit 12 of the information processing device 10 may, for example, refer to the specific part DB 601 of FIG. 6 and extract information on the specific part according to the analysis target. Then, the identification unit 12 of the information processing device 10 may identify the area of the specific part in the image captured by the image capturing device 20. Here, the identification unit 12 of the information processing device 10 may determine a rectangular (square or rectangular) area including a part such as a face based on the distributed image using AI or the like, and determine the rectangular area as the area of the specific part. Note that the information indicating the area of the specific part may include, for example, coordinate positions of the lower left and upper right pixels of the area. Furthermore, the information indicating the area of the specific part may include, for example, coordinate positions of any of the upper left, lower left, upper right, and lower right, and the size (e.g., height and width) of the specific part. Alternatively, the information indicating the area of the specific part may include, for example, information on a map (QP map) that sets a QP value for each specific pixel area unit (e.g., 16 pixels vertically by 16 pixels horizontally).

[0043] Then, the specifying unit 12 of the information processing apparatus 10 may determine information indicating the image quality of the region of the specific part based on the analysis target and the shooting situation. In this case, the information indicating the image quality of the region of the specific part may include at least one of, for example, the encoding bit rate, the encoding frame rate, and the encoding quantization parameter (QP value).

[0044] When hierarchical coding (SVC, Scalable Video Coding) is used as the encoding method of the image captured by the imaging device 20, the specifying unit 12 of the information processing apparatus 10 may determine to use the entire image as the base layer and the region of the specific part as the enhancement layer. In this case, the information indicating the image quality of the region of the specific part may include the bit rate of each layer of 1 or more including at least the enhancement layer.

[0045] In addition, the information indicating the image quality of the region of the specific part may include information related to the settings of the imaging device 20. The information related to the settings of the imaging device 20 may include a setting value related to the adjustment of the image quality of the image output from the imaging device 20 and a setting value related to the control of the imaging device 20. The settings related to the adjustment of the image quality of the image output from the imaging device 20 may include at least one of, for example, the bit depth (color depth), brightness, contrast, color tone, vividness, white balance, backlight correction, and gain of the image output from the imaging device 20. In addition, the settings related to the control of the imaging device 20 may include at least one of, for example, zoom, focus, and exposure. <s

[0046] (Example of determining image quality based on the correspondence table) The identifying unit 12 of the information processing device 10 may refer to the image quality setting DB 701 to determine information indicating the image quality of the specific body part area. In the example of FIG. 7, the image quality setting DB 701 sets the image quality of the specific body part area in association with a pair of the analysis target and the shooting situation. The identifying unit 12 of the information processing device 10 may refer to the image quality setting DB 701 to determine the image quality set according to the analysis target and the shooting situation as the image quality of the specific body part area. Note that the image quality setting DB 701 may be stored (registered, set) in a storage device inside the information processing device 10, or may be stored in a DB server or the like external to the information processing device 10.

[0047] (An example of determining image quality based on machine learning results) The identification unit 12 of the information processing device 10 may determine information indicating the image quality of the region of the specific part based on the machine learning result. In this case, the identification unit 12 of the information processing device 10 may use data recorded in the analysis result history DB 801 as learning data to generate a learned model in advance. Note that the analysis result history DB 801 may be stored (registered, set) in a storage device inside the information processing device 10, or may be stored in a DB server or the like external to the information processing device 10.

[0048] In the example of FIG. 8, the analysis result history DB 801 records a data set in which reliability is registered in association with a combination of an analysis target, a shooting situation, and the image quality of the region of a specific part. The reliability is the reliability (accuracy) of the analysis result for the analysis target in the shooting situation and the image quality. Note that the reliability of the analysis result is, for example, a value indicating how accurate the value of the analysis result is for a combination of a certain analysis target, a shooting situation, and the image quality of the region of a specific part. The reliability of the analysis result may be, for example, the degree of deviation (for example, the value of variance) between the value estimated by an analysis module or the like and the correct value. Note that the correct value may be, for example, a value measured by a doctor or the like in person with a patient or the like. Further, the correct value may be, for example, a value directly measured from a patient or the like using a dedicated instrument for measurement or the like. Further, the correct value may be, for example, a value inferred by an analysis module or the like based on an image whose data size is not compressed. The specifying unit 12 of the information processing apparatus 10 may perform supervised learning of a regression problem using the analysis target, the shooting situation, and the image quality recorded in the analysis result history DB 801 as explanatory variables (input variables, independent variables) and the reliability as an objective variable (correct label, response variable, dependent variable). In this case, the specifying unit 12 of the information processing apparatus 10 may perform machine learning using, for example, a neural network (NN) or a random forest. Note that the process of generating a learned model (learning phase) may be executed by an external apparatus such as a cloud.

[0049] Then, the specifying unit 12 of the information processing apparatus 10 may respectively estimate (infer) the reliability for each of a plurality of image qualities of the region of the specific part in the determined combination of the shooting situation and the analysis target. In this case, the specifying unit 12 of the information processing apparatus 10 may calculate the value of the reliability by inputting the information on the analysis target, the shooting situation, and the image quality into the learned model.

[0050] Then, the specifying unit 12 of the information processing apparatus 10 may determine the image quality of the area of the specifying part to be instructed to the imaging apparatus 20 based on the communication amount (data size, bit rate) when the image is distributed at each image quality among one or more image qualities whose estimated reliability is equal to or higher than the threshold value and the reliability at each image quality. In this case, for example, the specifying unit 12 of the information processing apparatus 10 may determine a higher priority (score) to be determined as the image quality of the area of the specifying part as the communication amount of each image quality is smaller and the reliability is higher. Then, the specifying unit 12 of the information processing apparatus 10 may determine the image quality with the highest determined priority as the image quality of the area of the specifying part to be instructed to the imaging apparatus 20. Thereby, for example, in the trade-off between the communication amount and the reliability, an optimal image quality with a relatively small communication amount and a relatively high reliability can be selected.

[0051] (Example of reducing the increase in the communication amount of an image) The specifying unit 12 of the information processing apparatus 10 may increase the image quality of the area of the specifying part (for example, the face of the patient) and reduce the image quality of the part other than the specifying part. Thereby, for example, an increase in the communication amount of the image can be reduced. In this case, for example, the specifying unit 12 of the information processing apparatus 10 may determine the area of the specifying part to be the first image quality and determine the area other than the specifying part to be the second image quality lower than the first image quality. Then, the specifying unit 12 of the information processing apparatus 10 may transmit information for causing the imaging apparatus 20 to distribute an image in which the area of the specifying part is the first image quality and the area other than the specifying part is the second image quality.

[0052] (Example of determining image quality based on the predicted value of the bandwidth) The specifying unit 12 of the information processing apparatus 10 may determine at least one of a first image quality and a second image quality based on the communication environment (for example, fluctuations in available bandwidth) of the network N to which the image captured by the imaging apparatus 20 is distributed. Thereby, when the available bandwidth is small, the image quality can be reduced to reduce video distortion. Further, the specifying unit 12 of the information processing apparatus 10 may determine at least one of the first image quality and the second image quality based on a predicted value of the available bandwidth. Thereby, for example, compared with the case where the image quality is reduced after the bandwidth has decreased, video distortion during the period from when the bandwidth has decreased until the image quality is reduced can be further reduced. Further, for example, when it is predicted that there will be no margin in the bandwidth when only the face region is to be high-quality, the face region can be high-quality and the region other than the face can be low-quality.

[0053] Note that the specifying unit 12 of the information processing apparatus 10 may perform machine learning in advance on the communication log information when an image was transmitted in the past on the network N, wireless quality information such as radio wave intensity, and the relationship between the day of the week, time, weather, and available bandwidth, and calculate the available bandwidth and the predicted value of the bandwidth.

[0054] Subsequently, the control unit 13 of the information processing apparatus 10 transmits information (command) for causing the imaging apparatus 20 to distribute a second image whose specific part region has the first image quality (step S104). Here, the command may include, for example, information indicating the region of the specific part and information indicating the image quality of the region of the specific part. Note that the second image may be the same image as the first image or a different image. For example, when the image is distributed in real time, the first image is the image captured during the process of step S101, and the second image is the image captured at a time after the process of step S104.

[0055] Next, the photographing device 20 sets (changes) the image quality of the area of the specific part of the subject in the photographed image to the first image quality based on the received command (step S105). Next, the photographing device 20 distributes (transmits) a second image, in which the area of the specific part of the subject in the photographed image is encoded at the first image quality, to the information processing device 10 via the network N (step S106). In the example of FIG. 9, an area 911 of the patient's face in the photographed image 901 is encoded at the first image quality specified by the information processing device 10. Furthermore, areas of the image 901 other than the face area 911 may be encoded at an image quality lower than the first image quality.

[0056] Next, the identification unit 12 of the information processing device 10 analyzes the subject based on the area of the specific part of the subject in the first image quality in the received second image (step S107). Here, the identification unit 12 of the information processing device 10 may measure (calculate, infer, estimate) information on various analysis targets of the subject, for example, by AI (Artificial Intelligence) using deep learning or the like. The analysis targets may include, for example, at least one of heart rate, respiratory rate, blood pressure, swelling, percutaneous arterial oxygen saturation, pupil size, throat swelling, and the degree of periodontal disease.

[0057] The determination unit 12 of the information processing device 10 may measure the heart rate based on an image of an area where the patient's skin is exposed (for example, the face area). In this case, the determination unit 12 of the information processing device 10 may measure the heart rate based on, for example, the transition (cycle) of the change in skin color.

[0058] Furthermore, the determination unit 12 of the information processing device 10 may measure the respiratory rate based on an image of the patient's chest (upper body) region. In this case, the determination unit 12 of the information processing device 10 may measure the respiratory rate based on, for example, the cycle of shoulder movement.

[0059] Further, the specifying unit 12 of the information processing apparatus 10 may measure blood pressure based on an image of an area where the patient's skin is exposed (for example, the face area). In this case, the specifying unit 12 of the information processing apparatus 10 may estimate blood pressure based on, for example, the difference and shape of the pulse waves estimated from two locations on the face (for example, the forehead and the cheek).

[0060] Further, the specifying unit 12 of the information processing apparatus 10 may measure transcutaneous arterial oxygen saturation (SpO2) based on an image of an area where the patient's skin is exposed (for example, the face area). Note that red is likely to be transmitted when hemoglobin and oxygen are bound, and blue is less affected by the binding of hemoglobin and oxygen. Therefore, the specifying unit 12 of the information processing apparatus 10 may measure SpO2 based on, for example, the difference in the degree of change between the blue and red colors of the skin near the cheekbone under the eyes.

[0061] Further, the specifying unit 12 of the information processing apparatus 10 may measure the degree of swelling based on, for example, an image of the patient's eyelid area. Further, the specifying unit 12 of the information processing apparatus 10 may measure the size of the pupil (pupil diameter) based on, for example, an image of the patient's eye area. Further, the specifying unit 12 of the information processing apparatus 10 may measure the degree of throat swelling, periodontal disease, etc. based on, for example, an image of the area inside the patient's oral cavity.

[0062] The specifying unit 12 of the information processing apparatus 10 may cause the display device to display the biological information (vital signs) of the patient, which is the analysis result. Note that the specifying unit 12 of the information processing apparatus 10 may continuously perform analysis and display the analysis result in real time.

[0063] Further, in the process of step S105, when the imaging device 20 does not support the image quality specified by the received command, the imaging device 20 may return a response indicating that to the information processing apparatus 10. In this case, the specifying unit 12 of the information processing apparatus 10 may cause a message indicating that the analysis has failed to be displayed. Thereby, the doctor can give an instruction, for example, to approach the imaging device 20 to the patient by voice during a call.

[0064] (Example of identifying a person based on the image of the imaging device 20 which is a surveillance camera) In the above-described example, an example of measuring biological information in a video conference between a doctor and a patient was explained. Hereinafter, an example of identifying a person based on the image of the imaging device 20 which is a surveillance camera will be explained. In this case, the video of the imaging device 20 may be distributed from the imaging device 20 to the information processing device 10.

[0065] First, when the specifying unit 12 of the information processing device 10 detects a person's area based on the image of the imaging device 20, the image quality of the entire image may be improved to a high image quality so that the reliability of detecting the person's area is equal to or higher than a threshold value based on the shooting situation when the image is taken by the imaging device 20. Further, when the specifying unit 12 of the information processing device 10 identifies who the person is based on the image of the imaging device 20, the area of the person's face may be improved to a high image quality so that the reliability of identifying the person is equal to or higher than a threshold value based on the shooting situation when the image is taken by the imaging device 20. Further, when the specifying unit 12 of the information processing device 10 identifies the behavior of a person based on the image of the imaging device 20, the area of the person's entire body may be improved to a high image quality so that the reliability of identifying the behavior is equal to or higher than a threshold value based on the shooting situation when the image is taken by the imaging device 20.

[0066] (Example of inspecting a product (quality inspection) based on the image of the imaging device 20) Hereinafter, an example of inspecting a product (quality inspection) based on the image of the imaging device 20 which is a surveillance camera will be explained. In this case, the video of the imaging device 20 may be distributed from the imaging device 20 to the information processing device 10.

[0067] First, when the specifying unit 12 of the information processing device 10 detects the area of a product based on the image of the imaging device 20, the image quality of the entire image may be improved to a high image quality so that the reliability of detecting the area is equal to or higher than a threshold value based on the shooting situation when the image is taken by the imaging device 20. Further, when the specifying unit 12 of the information processing device 10 inspects the product based on the image of the imaging device 20, the area of the product may be improved to a high image quality so that the reliability of the inspection is equal to or higher than a threshold value based on the shooting situation when the image is taken by the imaging device 20.

[0068] (Example of inspecting a facility using images from the imaging device 20) In the following, an example will be described in which a facility is inspected using images captured by the image capturing device 20 mounted on a drone, a robot that moves autonomously on the ground, etc. In this case, the image captured by the image capturing device 20 mounted on the drone, etc. may be distributed to the information processing device 10.

[0069] First, when the specifying unit 12 of the information processing device 10 detects the area of an object to be inspected (for example, a steel tower, a power line, etc.) based on an image from the imaging device 20, the specifying unit 12 may improve the image quality of the entire image so that the reliability of the area detection is equal to or higher than a threshold, based on the imaging conditions when the image is captured by the imaging device 20. Furthermore, when the specifying unit 12 of the information processing device 10 inspects (for example, measures damage, deterioration, etc.) a part to be inspected (for example, an insulator) based on an image from the imaging device 20, the specifying unit 12 may improve the image quality of the area of the part to be inspected so that the reliability of the inspection is equal to or higher than a threshold, based on the imaging conditions when the image is captured by the imaging device 20.

[0070] Third Embodiment 4 and 5, an example in which the image quality of a specific part or the like is determined in the information processing device 10 of the distribution destination has been described. Below, an example in which the image quality of a specific part or the like is determined in the information processing device 10 of the distribution source will be described with reference to FIGS. 10 and 11. FIG. 10 is a diagram showing an example of the configuration of an information processing system 1 according to an embodiment. FIG. 11 is a sequence diagram showing an example of processing of the information processing system 1 according to an embodiment. In the example of FIG. 10, the information processing device 10 having the imaging device 20 and the distribution destination device 30 are connected so as to be able to communicate with each other via a network N.

[0071] In step S201, the control unit 13 of the information processing apparatus 10 distributes (transmits) a first image obtained by encoding the area of a specific part of the subject in the image captured by the imaging apparatus 20 to the destination apparatus 30 via the network N. Subsequently, the specifying unit 12 of the information processing apparatus 10 specifies the imaging situation when the first image is captured by the imaging apparatus 20 based on the encoded first image and the like (step S202). Subsequently, the specifying unit 12 of the information processing apparatus 10 determines the first image quality of the area of the specific part according to the imaging situation and the analysis target for which analysis is performed based on the area of the specific part of the subject (step S203).

[0072] Subsequently, the control unit 13 of the information processing apparatus 10 sets (changes) the area of the specific part of the subject in the image captured by the imaging apparatus 20 to the first image quality (step S204). Subsequently, the control unit 13 of the information processing apparatus 10 distributes (transmits) a second image obtained by encoding the area of the specific part of the subject in the captured image with the first image quality to the destination apparatus 30 via the network N (step S205). Subsequently, the destination apparatus 30 performs analysis of the subject based on the area of the specific part of the subject with the first image quality in the received second image (step S206).

[0073] Note that the processes of step S201, step S204, and step S205 may be the same as the processes of step S201, step S105, and step S106 in FIG. 5, respectively. Also, the processes of step S202, step S203, and step S206 may be the same as the processes of step S102, step S103, and step S107 in the information processing apparatus 10B in FIG. 5, respectively. Note that in the information processing apparatus 10B as well, analysis processes similar to the processes of step S202 and step S206 may be executed in parallel.

[0074] <Modification Example> The information processing apparatus 10 may be an apparatus included in one housing, but the information processing apparatus 10 of the present disclosure is not limited thereto. Each part of the information processing apparatus 10 may be realized by cloud computing configured by, for example, one or more computers. Further, at least a part of the processing of the information processing apparatus 10 may be realized by, for example, another information processing apparatus 10. Such information processing apparatuses 10 are also included in an example of the "information processing apparatus" of the present disclosure.

[0075] Note that the present disclosure is not limited to the above-described embodiments, and can be appropriately changed without departing from the gist.

[0076] Some or all of the above embodiments may be described as follows in the following supplementary notes, but are not limited thereto. (Supplementary Note 1) Specific means for specifying a first image quality of a region of a specific part used for analysis in the image according to a shooting situation when an image is shot by a shooting device and an analysis item for the image distributed via a network; Control means for performing control to distribute the region of the specific part in the image as the first image quality; An information processing system having the above. (Supplementary Note 2) The shooting situation includes at least one of a state of the subject when the subject is shot and a situation around the subject. The information processing system according to Supplementary Note 1. (Supplementary Note 3) The specific means further specifies the first image quality according to a communication environment of the network through which the image is distributed. The information processing system according to Supplementary Note 1 or 2. (Supplementary Note 4) The specific means specifies at least one of a bit rate of encoding, a frame rate of encoding, a quantization parameter of encoding, a bit rate setting of a region of each layer of hierarchical encoding, and a setting of the shooting device as the first image quality. The information processing system according to any one of Supplementary Notes 1 to 3. (Appendix 5) Based on the region of the specific part of the image, information representing the state of the living body that is the subject of the image is analyzed. The information processing system according to any one of Appendices 1 to 4. (Appendix 6) The specific means specifies the first image quality using the learning result based on the dataset of the combination of the shooting situation, the item, the image quality of the region of the specific part, and the reliability of the analysis. The information processing system according to any one of Appendices 1 to 5. (Appendix 7) The specific means determines the region of the specific part to the first image quality and specifies the region other than the specific part to a second image quality lower than the first image quality. The control means performs control to cause the image in which the region of the specific part is the first image quality and the region other than the specific part is the second image quality to be distributed The information processing system according to any one of Appendices 1 to 6. (Appendix 8) A process of specifying a first image quality of a region of a specific part used for analysis in the image according to the shooting situation when an image is shot by a shooting device and an item for analysis of the image distributed via a network, and a process of controlling to cause the region of the specific part to be distributed as the first image quality in the image. An information processing method for executing. (Appendix 9) The shooting situation includes at least one of the state of the subject when the subject is shot and the situation around the subject. The information processing method according to Appendix 8. (Appendix 10) In the specifying process, the first image quality is further specified according to the communication environment of the network through which the image is distributed. The information processing method according to Appendix 8 or 9. (Appendix 11) In the specifying process, at least one of an encoding bit rate, an encoding frame rate, an encoding quantization parameter, a bit rate setting for each layer of hierarchical encoding, and a setting of the image capture device is specified as the first image quality. 11. An information processing method according to any one of appendices 8 to 10. (Appendix 12) information representing a state of a living body that is a subject of the image is analyzed based on the region of the specific part of the image; 12. An information processing method according to any one of appendices 8 to 11. (Appendix 13) In the identifying process, the first image quality is identified using a learning result based on a data set of the shooting situation, the item, the image quality of the region of the specific part, and an analysis reliability. 13. An information processing method according to any one of appendices 8 to 12. (Appendix 14) In the identifying process, determining the first image quality for the specific region and specifying a second image quality for the region other than the specific region, the second image quality being lower than the first image quality; In the control process, Control is performed to deliver an image in which the specific region has the first image quality and the region other than the specific region has the second image quality. 14. An information processing method according to any one of appendices 8 to 13. (Appendix 15) a specifying means for specifying a first image quality of a region of a specific part in the image to be used for analysis in accordance with a photographing condition when the image is photographed by the photographing device and an analysis item for the image distributed via a network; a control means for controlling delivery of the specific region of the image as the first image quality; An information processing device having the above. (Appendix 16) The photographing conditions include at least one of a state of the subject when the subject is photographed and a situation around the subject. 16. The information processing device according to claim 15. (Appendix 17) The specific means further specifies the first image quality according to the communication environment of the network through which the image is distributed. The information processing apparatus according to Appendix 15 or 16. (Appendix 18) The specific means specifies at least one of the encoding bit rate, the encoding frame rate, the encoding quantization parameter, the bit rate setting for the area of each layer of hierarchical encoding, and the setting of the imaging device as the first image quality. The information processing apparatus according to any one of Appendices 15 to 17. (Appendix 19) Based on the area of the specific part of the image, information representing the state of the living body that is the subject of the image is analyzed. The information processing apparatus according to any one of Appendices 15 to 18. (Appendix 20) The specific means specifies the first image quality by using the learning result based on the dataset of the combination of the shooting situation, the item, the image quality of the area of the specific part, and the reliability of the analysis. The information processing apparatus according to any one of Appendices 15 to 19. (Appendix 21) The specific means determines the area of the specific part as the first image quality, and specifies the area other than the specific part as a second image quality lower than the first image quality. The control means performs control to distribute an image in which the area of the specific part is the first image quality and the area other than the specific part is the second image quality. The information processing apparatus according to any one of Appendices 15 to 20.

Description of Signs

[0077] 1 Information processing system 10 Information processing apparatus 10A Information processing apparatus 10 Information processing apparatus 12 Specific part 13 Control part 20 Imaging device 30 Destination device for distribution N Network

Claims

1. Specific means for specifying a first image quality of a region of a specific part used for analysis in the image according to a shooting situation when an image is shot by a shooting device and an analysis item for the image distributed via a network; Control means for performing control to distribute the region of the specific part in the image as the first image quality; and has, The specific means is, Based on the shooting situation and the item, estimate one or more image qualities of the region of the specific part and the reliability of analysis corresponding to each of the one or more image qualities of the region of the specific part, Based on the communication amount when distributed at each of the estimated one or more image qualities and the reliability of analysis corresponding to each of the estimated one or more image qualities, specify the first image quality from among the estimated one or more image qualities, An information processing system.

2. The shooting situation includes at least one of a state of the subject when the subject is shot and a situation around the subject. The information processing system according to claim 1.

3. The specific means further specifies the first image quality according to the communication environment of the network through which the image is distributed. The information processing system according to claim 1 or 2.

4. The specific means specifies at least one of the encoding bit rate, the encoding frame rate, the encoding quantization parameter, the bit rate setting of the region of each layer of hierarchical encoding, and the setting of the shooting device as the first image quality. The information processing system according to any one of claims 1 to 3.

5. Based on the region of the specific part of the image, information representing the state of the living body that is the subject of the image is analyzed. The information processing system according to any one of claims 1 to 4.

6. The specific means is, Determine the region of the specific part as the first image quality, and specify the region other than the specific part as a second image quality lower than the first image quality, The control means is, Perform control to distribute an image in which the region of the specific part is the first image quality and the region other than the specific part is the second image quality. The information processing system according to any one of claims 1 to 5.

7. A process of specifying a first image quality of a region of a specific part used for analysis in the image according to a shooting situation when an image is shot by a shooting device and an analysis item for the image distributed via a network; In the image, execute a process of controlling to distribute the area of the specific part with the first image quality, and further, Based on the shooting situation and the item, estimate one or more image qualities of the area of the specific part and the reliability of the analysis corresponding to each of the one or more image qualities of the area of the specific part, Execute a process of specifying the first image quality from among the one or more estimated image qualities based on the communication volume when distributed with each of the one or more estimated image qualities and the reliability of the analysis corresponding to each of the one or more estimated image qualities, Information processing method.

8. The shooting situation includes at least one of the state of the subject when the subject is shot and the situation around the subject. The information processing method according to claim 7.

9. In the specifying process, further specify the first image quality according to the communication environment of the network through which the image is distributed. The information processing method according to claim 7 or 8.

10. In the specifying process, specify at least one of the encoding bit rate, the encoding frame rate, the encoding quantization parameter, the bit rate setting of the area of each layer of the hierarchical encoding, and the setting of the imaging device as the first image quality. The information processing method according to any one of claims 7 to 9.

11. Based on the area of the specific part of the image, information representing the state of the living body that is the subject of the image is analyzed. The information processing method according to any one of claims 7 to 10.

12. In the specifying process, Determine the area of the specific part as the first image quality, and specify the area other than the specific part as a second image quality lower than the first image quality. In the controlling process, Perform control to distribute an image in which the area of the specific part is the first image quality and the area other than the specific part is the second image quality. The information processing method according to any one of claims 7 to 11.

13. Specific means for specifying the first image quality of the area of the specific part used for analysis in the image according to the shooting situation when the image is shot by the imaging device and the item of analysis for the image distributed via the network, Control means for performing control to distribute the area of the specific part in the image with the first image quality, and having The specific means is Based on the shooting situation and the items, estimate the reliability of the analysis corresponding to each of one or more image qualities of the region of the specific part and one or more image qualities of the region of the specific part. Based on the communication volume when distributed at each of the one or more estimated image qualities and the reliability of the analysis corresponding to each of the one or more estimated image qualities, identify the first image quality from among the one or more estimated image qualities. An information processing apparatus.

14. The shooting situation includes at least one of the state of the subject when the subject is photographed and the situation around the subject. The information processing apparatus according to claim 13.

15. The identifying means further identifies the first image quality according to the communication environment of the network through which the image is distributed. The information processing apparatus according to claim 13 or 14.

16. The identifying means identifies at least one of the encoding bit rate, the encoding frame rate, the encoding quantization parameter, the bit rate setting of the region of each layer of hierarchical encoding, and the setting of the imaging device as the first image quality. The information processing apparatus according to any one of claims 13 to 15.

17. Based on the region of the specific part of the image, information representing the state of the living body that is the subject of the image is analyzed. The information processing apparatus according to any one of claims 13 to 16.

18. The identifying means For each of the one or more estimated image qualities, set a higher priority as the communication volume is smaller and the reliability of the analysis is higher. Identify the image quality with the highest priority among the one or more estimated image qualities as the first image quality. The information processing system according to claim 1.

19. For each of the one or more estimated image qualities, set a higher priority as the communication volume is smaller and the reliability of the analysis is higher. Identify the image quality with the highest priority among the one or more estimated image qualities as the first image quality. The information processing method according to claim 7.

20. The identifying means For each of the one or more estimated image qualities, set a higher priority as the communication volume is smaller and the reliability of the analysis is higher. Identify the image quality with the highest priority among the one or more estimated image qualities as the first image quality. The information processing apparatus according to claim 13.

Images