Image analysis method, image analysis device, and image analysis system
The image analysis method and system efficiently analyze compressed images with multiple wavelength information by selecting appropriate algorithms based on attached tags, reducing ID data volume and enhancing processing efficiency.
Patent Information
- Application Number
- JP2023116283
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-07-14
- Publication Date
- 2025-01-24
Smart Images
Figure 2025013024000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an image analysis method, an image analysis device, and an image analysis system. [Background technology]
[0002] Patent Document 1 discloses a personal cloud system including a smart device and a server including a virtual machine including a target text unit informationally linked to a header. In the personal cloud system, a header has a source code, and each image file including an identification (ID) corresponding to the source code is informationally linked, and the smart device reads and displays an application according to the source code corresponding to the header, and loads and executes the corresponding image file. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2017-162302 A Summary of the Invention [Problem to be solved by the invention]
[0004] However, Patent Document 1 requires an identification (ID) to be assigned to each image file, so when sending or receiving data of a large number of image files, there is a problem in that the amount of identification (ID) data increases in proportion to the number of image files.
[0005] The present disclosure has been devised in consideration of the above-mentioned conventional circumstances, and aims to provide an image analysis method, an image analysis device, and an image analysis system for analyzing a compressed image in which multiple pieces of wavelength information are compressed. [Means for solving the problem]
[0006] The present disclosure provides an image analysis method performed by an image analysis device that is a computer, which acquires a compressed image in which multiple wavelength information pieces are compressed, selects one of multiple algorithms to be used for image analysis processing of the compressed image based on the compressed image, and uses the selected processing algorithm to perform image analysis on at least one image of the compressed image that was captured with light of a wavelength corresponding to the processing algorithm, and outputs an image analysis result.
[0007] The present disclosure also provides an image analysis device including an acquisition unit that acquires a compressed image in which multiple pieces of wavelength information are compressed, a selection unit that selects, based on the compressed image, one of multiple algorithms to be used for image analysis processing of the compressed image, and an image analysis unit that uses the selected processing algorithm to perform image analysis on at least one image of the compressed image that is captured with light of a wavelength corresponding to the processing algorithm, and outputs a result of the image analysis.
[0008] The present disclosure also provides an image analysis system including a terminal device and an image analysis device capable of communicating with the terminal device, wherein the terminal device generates a compressed image in which multiple wavelength information items are compressed and transmits the image analysis device to the image analysis device, and the image analysis device selects, based on the compressed image, one of multiple algorithms to be used for image analysis processing of the compressed image, and uses the selected processing algorithm to perform image analysis on at least one image of the compressed image captured with light of a wavelength corresponding to the processing algorithm, and transmits the result of the image analysis to the terminal device. Effect of the Invention
[0009] According to the present disclosure, a compressed image in which multiple pieces of wavelength information are compressed can be subjected to image analysis. [Brief description of the drawings]
[0010] [Figure 1]FIG. 1 is a block diagram showing an example of the overall configuration of an image analysis system according to a modification of the first embodiment, the second embodiment, and the third embodiment; [Diagram 2] FIG. 1 is a sequence diagram showing an example of an operation procedure for advance preparation of an image analysis system according to the first embodiment, the second embodiment, and a modified example of the third embodiment. [Diagram 3] FIG. 1 is a sequence diagram showing an example of an operation procedure of an image analysis system according to the first embodiment, the second embodiment, and a modified example of the third embodiment. [Figure 4] FIG. 1 is a diagram for explaining an example of image analysis processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] Hereinafter, with reference to the drawings as appropriate, each embodiment that specifically discloses the configuration and operation of the image analysis method, image analysis device, and image analysis system according to the present disclosure will be described in detail. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters and duplicate explanation of substantially the same configuration may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the attached drawings and the following explanation are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.
[0012] (Embodiment 1) A use case of the image analysis system 100 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the overall configuration of the image analysis systems 100, 100A, and 100B according to the first embodiment, the second embodiment, and the modified examples of the third embodiment. Note that only the image analysis system 100 according to the first embodiment will be described here, and the image analysis systems 100A and 100B according to the second and third embodiments will be described later.
[0013] The image analysis system 100 according to the first embodiment assigns tags (information) related to image analysis to a compressed image captured by the terminal device P1 using the camera CM, which is a wideband hyperspectral camera, and in which information on a plurality of wavelengths is compressed. The image analysis system 100 according to the first embodiment selects an algorithm (hereinafter, referred to as a "processing algorithm") for executing an image analysis process corresponding to the tag assigned to the compressed image by the server S1, and performs image analysis on an image (so-called a spectral image, spectral data) included in the compressed image and captured with light of a predetermined wavelength, using the selected processing algorithm.
[0014] The image analysis system 100 includes a camera CM, a terminal device P1, an input unit INP, a monitor MN, a light LT, a server S1, and a network NW. The configuration of the image analysis system 100 is not limited to this. For example, the image analysis system 100 may include a plurality of terminal devices P1, a plurality of cameras CM, or a plurality of servers S1.
[0015] The camera CM is a hyperspectral camera capable of capturing images of light in a wide wavelength band (e.g., 400 nm to 1600 nm, etc.). The camera CM can accept control from the terminal device P1, and captures images in the wide wavelength band. The camera CM generates a compressed image (two-dimensional image data) in which information including the luminance of light having each of a plurality of wavelengths included in the wavelength band (i.e., 400 nm to 1600 nm) that the camera CM can capture is compressed (hereinafter, referred to as "wavelength information"), and transmits the compressed image to the terminal device P1. In the following description, the "wavelength information" may also be referred to as "spectral information".
[0016] The camera CM is configured to have at least a filter array (not shown), an image sensor (not shown), and a lens (not shown). The filter array is an optical element having a plurality of light-transmitting regions, and the light transmission spectrum, i.e., the wavelength dependency of the light transmittance, differs for each region. The image sensor is a solid-state imaging element including, for example, an InGaAs photodiode or a Si photodiode, and, for example, a Charged-Coupled Device (hereinafter, referred to as "CCD") or a Complementary Metal Oxide Semiconductor (hereinafter, referred to as "CMOS"), and converts an optical image formed on the imaging surface by light of a wide wavelength band into an electrical signal.
[0017] The terminal device P1 can accept operations by a user and is realized by, for example, a personal computer (hereinafter, referred to as "PC"), a notebook PC, a tablet terminal, a smartphone, etc. The terminal device P1 adds tags to compressed images transmitted from the camera CM, and outputs and displays image analysis results transmitted from the server S1 on the monitor MN. The terminal device P1 includes a communication unit 10, a processor 11, and a memory 12.
[0018] The communication unit 10 is connected to the server S1 via the network NW so as to be able to carry out wired or wireless communication, and is also connected to the input unit INP, the monitor MN, the camera CM, and the lighting LT so as to be able to carry out wired or wireless communication, respectively. The wireless communication here refers to communication via a wireless Local Area Network (LAN) such as Wi-Fi (registered trademark).
[0019] Processor 11 is configured using, for example, a Central Processing Unit (hereinafter referred to as "CPU") or a Field Programmable Gate Array (hereinafter referred to as "FPGA"), and performs various processes and controls in cooperation with memory 12. Specifically, processor 11 refers to programs and data stored in memory 12 and executes the programs to realize the respective functions of UI section 111, camera operation section 112, and lighting operation section 113.
[0020] The UI unit 111 realizes control of the input unit INP and the monitor MN, which are user interfaces. The UI unit 111 accepts user operations related to the camera parameters of the camera CM or the illumination intensity of the illumination LT, and accepts input operations of tags to be added to compressed images that are the subject of image analysis, via the input unit INP. The UI unit 111 outputs information on the camera parameters of the camera CM to the camera operation unit 112, and outputs information on the illumination intensity of the illumination LT to the illumination operation unit 113. The UI unit 111 adds tags to the compressed images and transmits them to the server S1.
[0021] Furthermore, the UI unit 111 acquires information on the image analysis results transmitted from the server S1, generates image data SC1, SC2 (see FIG. 4) indicating the image analysis results, and outputs them to the monitor MN for display.
[0022] Camera operation unit 112 can execute control of the camera CM, and changes camera parameters set in the camera CM and acquires compressed images transmitted from the camera CM based on the camera parameters of the camera CM output from UI unit 111. Camera operation unit 112 outputs the acquired compressed images to UI unit 111.
[0023] The lighting operation unit 113 is capable of executing control of the lighting LT, and changes the lighting intensity set in the lighting LT based on the lighting intensity of the lighting LT output from the UI unit 111.
[0024] The memory 12 has, for example, a random access memory (hereinafter referred to as "RAM") as a work memory used when the processor 11 executes each process, and a read only memory (hereinafter referred to as "ROM") for storing programs and data that define each operation of the processor 11. The RAM temporarily stores data or information generated or acquired by the processor 11. The ROM has written therein programs that define the operation of the processor 11.
[0025] The monitor MN is configured using, for example, a Liquid Crystal Display (LCD) or an organic electroluminescence (EL) display. The monitor MN displays an image analysis processing result screen (not shown) including image data SC1 and SC2 (see FIG. 4) output from the UI unit 111.
[0026] The input unit INP is a user interface capable of accepting input operations by a user, and is realized by, for example, a keyboard, a mouse, a touch panel, etc. The input unit INP converts the contents of the accepted input operation into an electric signal and transmits it to the UI unit 111. When the input unit INP is realized by a touch panel, the input unit INP may be configured integrally with the monitor MN.
[0027] The lighting LT is controlled by a lighting operation unit 113, and illuminates an imaging target imaged by the camera CM. The lighting LT includes one or more lighting elements, such as a Light Emittimg Diode (LED), a Laser Diode (LD), and an InfraRed (IR) lighting element.
[0028] The server S1 (an example of an image analysis device) is a so-called cloud server, and selects a processing algorithm to be used for image analysis processing based on a tag attached to a compressed image transmitted from the terminal device P1. The server S1 executes image analysis processing on the compressed image using the selected processing algorithm. The server S1 transmits the image processing result to the terminal device P1. The server S1 includes a communication unit 20, a processor 21, a memory 22, and an algorithm database DB. Note that the configuration of the server S1 is not limited to the example shown in FIG. 1.
[0029] The communication unit 20 (an example of an acquisition unit) is connected to the terminal device P1 via the network NW so as to be capable of wired or wireless communication with the terminal device P1. The communication unit 20 outputs the compressed image transmitted from the terminal device P1 to the processor 21, and transmits information on the image analysis result output from the processor 21 to the terminal device P1.
[0030] The processor 21 is configured using, for example, a CPU or an FPGA, and performs various processes and controls in cooperation with the memory 22. Specifically, the processor 21 refers to the programs and data stored in the memory 22 and executes the programs, thereby realizing the functions of a compressed image restoration unit 211 and a processing algorithm selection unit 212.
[0031] The compressed image restoration unit 211 restores the compressed images transmitted from the terminal device P1 and acquires images for each wavelength band. The compressed image restoration unit 211 outputs each of the restored images and information on tags assigned to the compressed images to the processing algorithm selection unit 212.
[0032] The processing algorithm selection unit 212 (an example of a selection unit) selects an algorithm to be used for image analysis processing from at least one algorithm registered in the algorithm database DB based on tag information output from the compressed image restoration unit 211. The processing algorithm selection unit 212 uses the selected algorithm to execute image analysis processing on at least one image captured with light of a wavelength that is the target of image analysis processing within the wavelength band captured by the camera CM. The processing algorithm selection unit 212 outputs information on the result of the image analysis processing to the communication unit 20, and causes it to be transmitted to the terminal device P1.
[0033] The memory 22 has, for example, a RAM as a work memory used when executing each process of the processor 21, and a ROM for storing programs and data that define each operation of the processor 21. The RAM temporarily stores data or information generated or acquired by the processor 21. The ROM has written therein a program that defines the operation of the processor 21.
[0034] The algorithm database DB is configured using a storage device such as a hard disk drive (HDD) or a solid state drive (SSD). The algorithm database DB stores at least one algorithm used for image analysis processing of a compressed image or an image captured by the camera CM. The algorithm database DB is an algorithm used for image analysis of a compressed image or an image captured by the camera CM, which is a hyperspectral camera, and is an algorithm that can realize image analysis processing such as object identification, abnormality detection inside packaging material, water wetness detection, high temperature detection, recognition of printed matter such as characters, material identification, and ingredient identification by being used in image analysis processing.
[0035] The network NW connects the terminal device P1 and the server S1 so as to enable data communication between them.
[0036] The image analysis system 100 according to the first embodiment may polarize the incident light incident on the image sensor of the camera CM using a polarizing plate. For example, the image analysis system 100 may further include a first polarizing plate that polarizes the illumination light of the illumination LT in a first direction, or a second polarizing plate that polarizes the incident light incident on the image sensor of the camera CM in a second direction different from the first direction. This allows the image analysis system 100 to capture an image more suitable for image analysis processing.
[0037] In addition, the configuration of the camera CM, the process of generating compressed images, and the process of restoring compressed images may use known configurations or technologies, for example, configurations or technologies disclosed in Patent Publication No. 2021-110869, Patent Publication No. 2022-000968, Patent Publication No. 2022-058424, Patent Publication No. 2023-082047, etc. may be used.
[0038] Next, an example of an operation procedure for advance preparation will be described with reference to Fig. 2. Fig. 2 is a sequence diagram showing an example of an operation procedure for advance preparation of the image analysis system 100 according to the first embodiment, the second embodiment, and the modified example of the third embodiment.
[0039] Here, an example of the operation procedure of the advance preparation performed by the image analysis system 100 according to the first embodiment will be described, but the explanation of the example of the operation procedure of the advance preparation performed by the image analysis systems 100A, 100B according to the second and third embodiments will be omitted since it is similar. Also, the advance preparation shown in Fig. 2 is a process that is performed every time the imaging environment (evaluation environment) of the imaging analysis target is changed, and is not essential and may be omitted.
[0040] The terminal device P1 accepts an input operation of information regarding the camera parameters of the camera CM through the input unit INP (St11). The camera parameters here are the exposure time, focal length, gain, etc. of the camera CM. The terminal device P1 acquires information regarding the camera parameters of the camera CM through the input unit INP, associates the information regarding the camera parameters with a control command requesting a change in the parameters of the camera CM, and transmits the information to the camera CM (St12).
[0041] The camera CM changes the current camera parameters to camera parameters corresponding to the information on the camera parameters transmitted from the terminal device P1 based on the control command transmitted from the terminal device P1 (St13). When the change of the camera parameters is completed, the camera CM generates a camera parameter change completion notification and transmits it to the terminal device P1 (St14).
[0042] The terminal device P1 receives an input operation related to the illumination intensity of the lighting LT through the input unit INP (St15). The terminal device P1 acquires the illumination intensity of the lighting LT through the input unit INP, associates the illumination intensity information with a control command requesting a change in the illumination intensity of the lighting LT, and transmits the information to the lighting LT (St16).
[0043] Based on the control command transmitted from the terminal device P1, the lighting LT changes the current lighting intensity to a lighting intensity corresponding to the lighting intensity information transmitted from the terminal device P1 (St17). When the lighting intensity change is completed, the lighting LT generates a lighting intensity change completion notification and transmits it to the terminal device P1 (St18).
[0044] The terminal device P1 generates a control command requesting the camera CM to capture a darkened image, and transmits the control command to the camera CM (St19). Note that the darkened image referred to here is an image that is a characteristic unique to the camera CM and is used to reduce noise that the camera CM has.
[0045] Based on a control command transmitted from the terminal device P1, the camera CM captures an image in a light-shielded state with the shutter closed (St20), and transmits the captured light-shielded image to the terminal device P1 (St21).
[0046] The terminal device P1 generates a control command requesting the camera CM to capture a whiteboard image, and transmits the control command to the camera CM (St22). Note that the whiteboard image here is an image used to remove the spectrum of the illumination light of the lamp LT.
[0047] The camera CM captures an image of the whiteboard based on the control command transmitted from the terminal device P1 (St23). The camera CM transmits the captured image of the whiteboard to the terminal device P1 (St24).
[0048] The terminal device P1 performs shading correction using the shading image on the acquired whiteboard image (St25). The terminal device P1 transmits the whiteboard image after the shading correction to the server S1 (St26).
[0049] The server S1 records the whiteboard image transmitted from the terminal device P1 in the memory 22 (St27).
[0050] Next, an example of an operation procedure of the image analysis system 100 will be described with reference to Fig. 3. Fig. 3 is a sequence diagram showing an example of an operation procedure of advance preparation of the image analysis system 100 according to the first embodiment, the second embodiment, and the modified example of the third embodiment.
[0051] Here, only an example of the operation procedure performed by the image analysis system 100 according to the first embodiment will be described, and examples of the operation procedure performed by the image analysis systems 100A and 100B according to the second and third embodiments will be described later.
[0052] The terminal device P1 accepts an input operation of information related to the camera parameters of the camera CM through the input unit INP (St31). The terminal device P1 acquires information related to the camera parameters of the camera CM through the input unit INP, associates the information related to the camera parameters with a control command requesting a change in the parameters of the camera CM, and transmits the information to the camera CM to cause the camera CM to change the camera parameters (St32).
[0053] The terminal device P1 receives an input operation related to the illumination intensity of the lighting LT through the input unit INP (St33). The terminal device P1 acquires the illumination intensity of the lighting LT through the input unit INP, associates the illumination intensity information with a control command requesting a change in the illumination intensity of the lighting LT, and transmits the information to the lighting LT to change the illumination intensity of the lighting LT (St34).
[0054] The terminal device P1 generates a control command requesting the camera CM to capture an image of a subject, and transmits the control command to the camera CM (St35).
[0055] The camera CM captures an image of a subject for each wavelength over the wavelength band that the camera CM can capture based on the control command transmitted from the terminal device P1 (St36). The camera CM generates a compressed image in which multiple pieces of wavelength information are compressed, and transmits the compressed image to the terminal device P1 (St37).
[0056] The terminal device P1 performs shading correction based on a shading image captured in advance on the compressed image transmitted from the camera CM (St38). Note that the process of step St38 may be performed only when a shading image is captured in advance, and is not essential and may be omitted.
[0057] The terminal device P1 accepts an input of a tag used for selecting a processing algorithm to be used in the image analysis process based on a user operation. The tag here refers to information including information on the subject to be image analyzed (e.g., the subject's name, model number, ID, etc.), information on the content of the analysis process (e.g., freshness, temperature, film thickness, water, etc.), etc. The terminal device P1 acquires the tag information transmitted from the input unit INP, assigns the acquired tag to the compressed image (St39), and transmits it to the server S1 (St40).
[0058] The server S1 restores the compressed image transmitted from the terminal device P1 and acquires an image for each wavelength band (St42). The server S1 performs whiteboard correction using a previously recorded whiteboard image for each of the acquired images (St43). Note that the process of step St43 may be executed only when a whiteboard image has been captured in advance, and is not essential and may be omitted.
[0059] The server S1 selects a processing algorithm to be used for image analysis processing of the restored images from among at least one processing algorithm registered in the algorithm database DB based on tag information attached to the compressed image (St44). The server S1 executes image analysis processing on at least one image corresponding to a predetermined wavelength band among the restored images using the selected processing algorithm (St44). The server S1 transmits the image analysis processing result to the terminal device P1 (St45).
[0060] The terminal device P1 generates image data SC1, SC2 (see FIG. 4) showing the image analysis processing result based on the image analysis result transmitted from the server S1 (St46). The terminal device P1 generates an image analysis processing result screen (not shown) including the generated image data SC1, SC2, and outputs it to the monitor MN for display (St47).
[0061] As described above, the image analysis system 100 according to the first embodiment can realize image analysis processing using a predetermined processing algorithm by the server S1 by adding a tag containing information for selecting a processing algorithm to be used for image analysis processing of the compressed image, into which multiple pieces of wavelength information have been compressed.
[0062] Next, an example of image analysis processing performed by the image analysis systems 100, 100A, and 100B will be described with reference to Fig. 4. Fig. 4 is a diagram for explaining the example of image analysis processing.
[0063] The image analysis processing example shown in Case No. (1) in Fig. 4 is an image analysis processing for estimating the freshness of an apple. The compressed image CIM in Case No. (1) is given tags including information on the subject "apple" and information on the content of the analysis processing "freshness" by the terminal device P1.
[0064] Based on the tag information attached to the compressed image CIM, the server S1 selects the processing algorithm "Apple freshness estimation" from among the processing algorithms registered in the algorithm database DB. Based on the selected processing algorithm "Apple freshness estimation", the server S1 extracts images captured with light having a wavelength corresponding to the wavelength band "450-750 (nm)" that is the target of image analysis from each of the restored images. The server S1 performs image analysis processing on each of the extracted images using the processing algorithm "Apple freshness estimation", and transmits information on the results of the image analysis processing to the terminal device P1.
[0065] Based on the information on the image analysis processing result transmitted from the server S1, the terminal device P1 generates image data SC1 showing the image analysis processing result (freshness of apples) in a heat map, and outputs and displays it on the monitor MN.
[0066] The image analysis processing example shown in Case No. (2) in Fig. 4 is an image analysis processing for estimating the variety of an apple. The compressed image CIM in Case No. (2) is given tags including information on the subject "apple" and information on the content of the analysis processing "variety" by the terminal device P1.
[0067] Based on the tag information added to the compressed image CIM, the server S1 selects the processing algorithm "apple variety estimation" from among the processing algorithms registered in the algorithm database DB. Based on the selected processing algorithm "apple variety estimation", the server S1 extracts images captured with light having a wavelength corresponding to the wavelength band "450-1600 (nm)" that is the target of image analysis from each of the restored images. The server S1 performs image analysis processing on each of the extracted images using the processing algorithm "apple variety estimation", and transmits information on the results of the image analysis processing to the terminal device P1.
[0068] Based on the information on the image analysis processing results transmitted from the server S1, the terminal device P1 generates image data SC2 in which the image analysis processing results (apple varieties) are indicated by classification such as color coding, and supplementary information regarding the image analysis processing results (classification results) of the image data SC2, "0: ●●, 1: XXX, 2: △△," and outputs it to the monitor MN for display.
[0069] The image analysis processing example shown in Case No. (3) in Fig. 4 is an image analysis processing for estimating the state (quality) of the paint. The compressed image in Case No. (3) is tagged by the terminal device P1 with tags including information on the subject, "paint", and information on the content of the analysis processing, "unevenness".
[0070] Based on the tag information added to the compressed image, the server S1 selects the processing algorithm "paint unevenness estimation" from among the processing algorithms registered in the algorithm database DB. Based on the selected processing algorithm "paint unevenness estimation", the server S1 extracts images captured with light having a wavelength corresponding to the wavelength band "450-850 (nm)" that is the target of image analysis from each of the restored images. The server S1 performs image analysis processing on each of the extracted images using the processing algorithm "paint unevenness estimation", and transmits information on the results of the image analysis processing to the terminal device P1.
[0071] The terminal device P1 generates image data showing the image analysis processing results (paint unevenness estimation) in a heat map based on the information on the image analysis processing results transmitted from the server S1, and outputs and displays the image data on the monitor MN.
[0072] The image analysis process example shown in Case No. (4) in Fig. 4 is an image analysis process for estimating the state (quality) of a coating. The compressed image in Case No. (4) is given tags by the terminal device P1, including information on the subject, "coating," and information on the content of the analysis process, "film thickness."
[0073] Based on the tag information attached to the compressed image, the server S1 selects the processing algorithm "film thickness estimation" from among the processing algorithms registered in the algorithm database DB. Based on the selected processing algorithm "film thickness estimation", the server S1 extracts images captured with light of a wavelength corresponding to the wavelength band "1600 (nm)" that is the target of image analysis from among the multiple restored images. Using the processing algorithm "film thickness estimation", the server S1 performs image analysis processing on each of the extracted images and generates supplementary information "(film thickness) units" related to the image analysis processing results (film thickness estimation) of this image data, and outputs and displays them on the monitor MN.
[0074] The terminal device P1 generates image data showing the paint film thickness estimate in a heat map based on the information of the image analysis processing result transmitted from the server S1, and outputs and displays it on the monitor MN.
[0075] The image analysis process example shown in Case No. (5) in Fig. 4 is an image analysis process for estimating the state (quality) of a paint coating. The compressed image in Case No. (5) is tagged by the terminal device P1 with tags including information on the subject "deterioration" and information on the content of the analysis process "deterioration".
[0076] Based on the tag information added to the compressed image, the server S1 selects the processing algorithm "paint deterioration estimation" from among the processing algorithms registered in the algorithm database DB. Based on the selected processing algorithm "paint deterioration estimation", the server S1 extracts, from each of the restored images, images captured with light having a wavelength corresponding to the wavelength band "750-1600 (nm)" that is the target of image analysis. The server S1 performs image analysis processing on each of the extracted images using the processing algorithm "paint deterioration estimation", and transmits information on the results of the image analysis processing to the terminal device P1.
[0077] Based on the information on the image analysis processing results transmitted from the server S1, the terminal device P1 generates image data showing the estimated results of paint deterioration in the form of a heat map, and outputs and displays the image data on the monitor MN.
[0078] (Embodiment 2) The image analysis system 100 according to the first embodiment has been described as an example in which a compressed image transmitted from the terminal device P1 is restored and image analysis processing is performed. The image analysis system 100A according to the second embodiment will be described as an example in which image analysis processing is performed without restoring the compressed image transmitted from the terminal device P1.
[0079] Since image analysis system 100A according to embodiment 2 has a configuration substantially similar to that of image analysis system 100 according to embodiment 1, in the following description, the same reference symbols are used for configurations similar to those of image analysis system 100 according to embodiment 1, and descriptions thereof will be omitted.
[0080] An image analysis system 100A according to the second embodiment will be described with reference to Fig. 1. The image analysis system 100A includes a camera CM, a terminal device P1, an input unit INP, a monitor MN, a light LT, a server S1A, and a network NW.
[0081] The server S1A (an example of an image analyzing device) includes a communication unit 20, a processor 21A, a memory 22, and an algorithm database DB.
[0082] The compressed image restoration unit 211A of the processor 21A determines whether or not the compressed image needs to be restored based on the tag information attached to the compressed image transmitted from the terminal device P1. For example, in the image analysis processing example shown in FIG. 4, when the tag attached to the compressed image is information on the subject "painted" and information on the content of the analysis processing "unevenness", the compressed image restoration unit 211A omits restoration of the compressed image based on the necessity "no" of restoring the compressed image. On the other hand, when the compressed image restoration unit 211A determines that the necessity "needs" of restoring the compressed image is based on the tag attached to the compressed image, it executes restoration of the compressed image. The compressed image restoration unit 211A outputs the compressed image or each of the multiple restored images and the information of the tag attached to the compressed image to the processing algorithm selection unit 212A.
[0083] The processing algorithm selection unit 212A (an example of a selection unit) selects an algorithm to be used for image analysis processing from at least one algorithm registered in the algorithm database DB, based on tag information output from the compressed image restoration unit 211. The processing algorithm selection unit 212A uses the selected algorithm to execute image analysis processing on at least one image corresponding to a wavelength or wavelength band that is the target of image analysis processing, from among the compressed image or the wavelength band captured by the camera CM.
[0084] An example of the operation procedure of the image analysis system 100A according to the embodiment 2 will be described with reference to Fig. 4. Note that, since the operation procedure of the image analysis system 100A is similar to the operation procedure described in the description of the operation procedure of the image analysis system 100 according to the embodiment 1 in steps St11 to St40 and steps St42 to St47, respectively, the description will be omitted.
[0085] The server S1A receives the compressed image transmitted from the terminal device P1, and determines whether or not the compressed image needs to be restored based on the tag information attached to the compressed image (St41A).
[0086] When the server S1A determines that the compressed image needs to be restored, it executes the processes of steps St42 and St43. Note that the process of step St43 is not essential and may be omitted.
[0087] On the other hand, when the server S1A determines that the decompression of the compressed image is not necessary, the server S1A skips the process of step St42 and executes the processes of step St43 and step St44. Note that the process of step St43 is not essential and may be omitted.
[0088] As described above, the image analysis system 100A according to the second embodiment can further reduce the time required for image analysis processing by omitting the restoration process of the compressed image when restoration of the compressed image is not required in the image analysis processing.
[0089] (Embodiment 3) The image analysis system 100 according to the first embodiment has been described as an example of executing image analysis processing based on a processing algorithm corresponding to a tag added to a compressed image transmitted from the terminal device P1. The image analysis system 100B according to the third embodiment will be described as an example of performing image analysis on a compressed image transmitted from the terminal device P1B, and executing image analysis processing based on a processing algorithm corresponding to the image analysis result.
[0090] Since image analysis system 100B of embodiment 3 has a configuration substantially similar to that of image analysis system 100 of embodiment 1, in the following description, the same reference symbols are used for configurations similar to those of image analysis system 100 of embodiment 1, and descriptions thereof will be omitted.
[0091] An image analysis system 100B according to the third embodiment will be described with reference to Fig. 1. The image analysis system 100B includes a camera CM, a terminal device P1B, an input unit INP, a monitor MN, a light LT, a server S1B, and a network NW.
[0092] The terminal device P1B includes a communication unit 10, a processor 11B, and a memory 12.
[0093] The processor 11B omits accepting tag information via the input unit INP and assigning tags to the compressed image. The processor 11B transmits a compressed image in which multiple pieces of wavelength information transmitted from the camera CM are compressed, to the server S1B.
[0094] The server S1B (an example of an image analyzing device) includes a communication unit 20, a processor 21B, a memory 22, and an algorithm database DB.
[0095] The processing algorithm selection unit 212B (an example of a selection unit) of the processor 21B performs image analysis on at least one image among the multiple images output from the compressed image restoration unit 211, and estimates information about the subject or information about the content of the analysis process. Note that the information about the content of the analysis process may be estimated based on the wavelength at which the image was captured.
[0096] The processing algorithm selection unit 212B selects a processing algorithm to be used for image analysis processing of the restored images from among at least one processing algorithm registered in the algorithm database DB, based on information about the estimated subject or information about the content of the analysis processing. The server S1B executes image analysis processing on each of the restored images using the selected processing algorithm.
[0097] An example of the operation procedure of the image analysis system 100B according to the embodiment 3 will be described with reference to Fig. 4. Note that, since the operation procedure of the image analysis system 100B is similar to the operation procedure described in the description of the operation procedure of the image analysis system 100 according to the embodiment 1 in steps St11 to St38 and steps St42 to St43, respectively, the description will be omitted.
[0098] The terminal device P1B accepts an input of a tag used in a selection process of a processing algorithm used in the image analysis process based on a user operation. The tag here is information including information on the subject that is the target of image analysis (e.g., the subject's name, model number, ID, etc.), information on the content of the analysis process (e.g., freshness, temperature, film thickness, water, etc.), etc. The terminal device P1B skips the process of step St39 (the process of acquiring tag information and the process of adding tags) and transmits the tag information to the server S1B (St40B).
[0099] The server S1B executes image analysis processing on the restored images and estimates information about the subject or information about the content of the analysis processing (St44B). Based on the estimated information about the subject or information about the content of the analysis processing, the server S1B selects a processing algorithm to be used for the image analysis processing of the restored images from among at least one processing algorithm registered in the algorithm database DB (St44B). The server S1B executes image analysis processing of each of the restored images using the selected processing algorithm (St44B).
[0100] This allows the server S1B in the second embodiment to select a processing algorithm based on the subject appearing in the restored image. This allows the server S1B to execute image analysis processing using the selected processing algorithm even if a tag is not added to the compressed image.
[0101] (Additional Note) The above description of each embodiment discloses the following techniques.
[0102] (Technology 1) An image analysis method performed by an image analysis device (server S1, S1A, S1B) which is a computer, Obtaining a compressed image in which multiple wavelength information is compressed; selecting, based on the compressed image, one of the plurality of algorithms to be used for image analysis processing of the compressed image; performing image analysis on at least one image captured with light of a wavelength corresponding to the selected processing algorithm among the compressed images using the selected processing algorithm, and outputting the image analysis result; Image analysis methods. With this configuration, the servers S1, S1A, and S1B can select a processing algorithm to be used for image analysis processing of the compressed image (image), and by using the selected processing algorithm, the compressed image in which multiple wavelength information is compressed can be image analyzed.
[0103] (Technology 2) selecting one of the plurality of algorithms to be used for image analysis processing of the compressed image based on tag information related to image analysis of the compressed image that is added to the compressed image; performing image analysis on at least one image captured with light of a wavelength corresponding to the selected processing algorithm among the compressed images using the selected processing algorithm, and outputting the image analysis result; The image analysis method described in (Technical 1). This configuration allows the servers S1 and S1A to select a processing algorithm suitable for image analysis processing of images captured at different wavelengths, based on the tag information.
[0104] (Technology 3) Determining whether or not the compressed image needs to be restored based on the tag information; When it is determined that restoration of the compressed image is necessary, the compressed image is restored, and an image captured with light of a wavelength corresponding to the processing algorithm is analyzed among a plurality of images corresponding to each of the restored plurality of wavelength information. The image analysis method described in (Technical 2). With this configuration, the server S1A can determine whether or not a compressed image needs to be restored based on the tag information.
[0105] (Technology 4) Determining whether or not the compressed image needs to be restored based on the tag information; If it is determined that restoration of the compressed image is unnecessary, the restoration of the compressed image is omitted, and the compressed image is subjected to image analysis using a processing algorithm. An image analysis method according to (Technology 2) or (Technology 3). With this configuration, when the server S1A determines based on the tag information that it is not necessary to restore the compressed image, the server S1A can omit the restoration process of the compressed image and execute the image analysis process, thereby reducing the processing time required for the restoration process of the compressed image.
[0106] (Technology 5) restoring the compressed image, and performing image processing on at least one image among the plurality of images corresponding to each of the plurality of restored wavelength information to detect a subject appearing in the image; selecting, based on the detected object, one of the plurality of algorithms to be used for image analysis processing of the image; The image analysis method described in (Technical 1). This configuration allows the server S1B to select a processing algorithm based on the subject in the image, and thus allows the server S1B to perform image analysis processing using the selected processing algorithm even if the compressed image is not tagged.
[0107] (Technology 6) In the compressed image, wavelength information from 400nm to 1600nm is compressed. The image analysis method according to any one of (Technology 1) to (Technology 5). This configuration allows the servers S1, S1A, and S1B to acquire a compressed image in which multiple pieces of wavelength information included in a wide wavelength band are compressed.
[0108] (Technology 7) An acquisition unit (communication unit 20) that acquires a compressed image in which a plurality of pieces of wavelength information are compressed; A selection unit (processing algorithm selection unit 212, 212A, 212B) that selects one of a plurality of algorithms to be used for image analysis processing of the compressed image based on the compressed image; and an image analysis unit (processing algorithm selection unit 212, 212A, 212B) that uses the selected processing algorithm to perform image analysis on at least one image captured with light of a wavelength corresponding to the processing algorithm among the compressed images, and outputs the result of the image analysis. Image analysis device (servers S1, S1A, S1B). With this configuration, the servers S1, S1A, and S1B can select a processing algorithm to be used for image analysis processing of the compressed image (image), and by using the selected processing algorithm, the compressed image in which multiple wavelength information is compressed can be image analyzed.
[0109] (Technology 8) Terminal devices P1 and P1B, Image analysis systems 100, 100A, and 100B each including a terminal device P1, P1B and an image analysis device (server S1, S1A, S1B) capable of communicating with each other, The terminal devices P1 and P1B are A compressed image in which multiple pieces of wavelength information are compressed is transmitted to an image analysis device (servers S1, S1A, S1B); The image analysis device (servers S1, S1A, S1B) selecting, based on the compressed image, one of the plurality of algorithms to be used for image analysis processing of the compressed image; Using the selected processing algorithm, perform image analysis on at least one image among the compressed images captured with light of a wavelength corresponding to the processing algorithm, and transmit the result of the image analysis to the terminal devices P1 and P1B. Image analysis systems 100, 100A, 100B. With this configuration, image analysis systems 100, 100A, and 100B can select a processing algorithm to be used for image analysis processing of a compressed image (image), and by using the selected processing algorithm, can perform image analysis of a compressed image in which multiple pieces of wavelength information are compressed.
[0110] Although various embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, corrections, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also belong to the technical scope of the present disclosure. In addition, the components in the various embodiments described above may be arbitrarily combined within the scope of the invention. [Industrial Applicability]
[0111] INDUSTRIAL APPLICABILITY The present disclosure is useful as providing an image analysis method, an image analysis device, and an image analysis system for analyzing a compressed image in which a plurality of pieces of wavelength information are compressed. [Explanation of symbols]
[0112] 10,20 Communications Department 11,11B,21,21A,21B Processors 12,22 Memory 100, 100A, 100B Image Analysis System 111 UI section 112 Camera operation section 113 Lighting control unit 211, 211A Compressed image restoration unit 212, 212A, 212B Processing algorithm selection section Commercial Camera DB Algorithm Database LT lighting MN Monitor NW Network P1, P1B terminal equipment S1, S1A, S1B servers
Claims
1. An image analysis method performed by an image analysis device that is a computer, comprising: Obtaining a compressed image in which multiple wavelength information is compressed; selecting, based on the compressed image, one of a plurality of algorithms to be used for image analysis processing of the compressed image; performing image analysis on at least one image captured with light of a wavelength corresponding to the processing algorithm among the compressed images using the selected processing algorithm, and outputting an image analysis result; Image analysis methods.
2. selecting one of the plurality of algorithms to be used for image analysis processing of the compressed image based on tag information related to image analysis of the compressed image that is added to the compressed image; using the selected processing algorithm, performing image analysis on the image captured with light of a wavelength corresponding to the processing algorithm among the compressed images, and outputting the image analysis result. The image analysis method according to claim 1 .
3. determining whether or not the compressed image needs to be restored based on the tag information; When it is determined that restoration of the compressed image is necessary, the compressed image is restored, and an image analysis is performed on the image captured with light of a wavelength corresponding to the processing algorithm among a plurality of images corresponding to each of the plurality of restored wavelength information. The image analysis method according to claim 2.
4. determining whether or not the compressed image needs to be restored based on the tag information; if it is determined that restoration of the compressed image is unnecessary, the restoration of the compressed image is omitted, and the compressed image is subjected to image analysis using the processing algorithm. The image analysis method according to claim 2.
5. restoring the compressed image, and performing image processing on at least one of a plurality of images corresponding to each of the plurality of wavelength information thus restored, to detect a subject appearing in the image; selecting, based on the detected subject, one of the plurality of algorithms to be used for image analysis processing of the image; The image analysis method according to claim 1 .
6. The compressed image compresses wavelength information from 400 nm to 1600 nm. The image analysis method according to claim 1 .
7. an acquisition unit that acquires a compressed image in which a plurality of pieces of wavelength information are compressed; a selection unit that selects, based on the compressed image, one of a plurality of algorithms to be used for image analysis processing of the compressed image; and an image analysis unit that performs image analysis on at least one image captured with light of a wavelength corresponding to the processing algorithm among the compressed images using the selected processing algorithm, and outputs a result of the image analysis. Image analysis device.
8. A terminal device; An image analysis system comprising: an image analysis device capable of communicating with the terminal device, The terminal device Sending a compressed image in which a plurality of pieces of wavelength information have been compressed to the image analysis device; The image analysis device includes: selecting, based on the compressed image, one of a plurality of algorithms to be used for image analysis processing of the compressed image; using the selected processing algorithm, performing image analysis on at least one image among the compressed images captured with light of a wavelength corresponding to the processing algorithm, and transmitting a result of the image analysis to the terminal device; Image analysis system.
Citation Information
Patent Citations
Biometric authentication device, biometric authentication method, and biometric authentication program
JP2017162302A