Image analysis method, image analysis device, and image analysis system

Abstract

This image analysis method is performed by an image analysis device, which is a computer, the method involving: acquiring a compressed image in which a plurality of pieces of wavelength information are compressed; selecting an algorithm to be used for image analysis processing of the compressed image among a plurality of algorithms on the basis of the compressed image; using the selected processing algorithm to perform image analysis of at least one image captured by light of a wavelength corresponding to the processing algorithm in the compressed image; and outputting image analysis results.

Description

Image analysis method, image analysis device, and image analysis system

[0001] The present disclosure relates to an image analysis method, an image analysis device, and an image analysis system.

[0002] Patent Literature 1 discloses a personal cloud system including a smart device and a server including a virtual machine including a target text unit information-linked to a header. In the personal cloud system, a header has source code, and each image file including an identification (ID) corresponding to the source code is information-linked. The smart device reads and displays an application according to the source code corresponding to the header, and loads and executes the corresponding image file.

[0003] Japanese Patent Application Publication No. 2017-162302

[0004] However, in Patent Document 1, since it is necessary to assign an identification (ID) to each image file, when transmitting and receiving data of a large number of image files, there is a problem that the amount of identification (ID) data increases in proportion to the number of image files.

[0005] The present disclosure has been devised in view of the above-described conventional circumstances, and aims to provide an image analysis method, an image analysis device, and an image analysis system for analyzing a compressed image in which multiple pieces of wavelength information are compressed.

[0006] The present disclosure provides an image analysis method performed by an image analysis device that is a computer, which acquires a compressed image in which multiple pieces of wavelength information have been compressed, selects one of multiple algorithms based on the compressed image to be used for image analysis processing of the compressed image, and uses the selected processing algorithm to perform image analysis on at least one image of the compressed image that has been captured with light of a wavelength corresponding to the processing algorithm, and outputs the image analysis results.

[0007] The present disclosure also provides an image analysis device including an acquisition unit that acquires a compressed image in which multiple pieces of wavelength information are compressed; a selection unit that selects, based on the compressed image, one of multiple algorithms to be used for image analysis processing of the compressed image; and an image analysis unit that uses the selected processing algorithm to perform image analysis on at least one image of the compressed image that has been captured with light of a wavelength corresponding to the processing algorithm, and outputs the results of the image analysis.

[0008] The present disclosure also provides an image analysis system including a terminal device and an image analysis device capable of communicating with the terminal device, wherein the terminal device generates a compressed image in which multiple pieces of wavelength information are compressed and transmits the image to the image analysis device, and the image analysis device selects, based on the compressed image, one of multiple algorithms to be used for image analysis processing of the compressed image, and uses the selected processing algorithm to perform image analysis on at least one image of the compressed image that was captured with light of a wavelength corresponding to the processing algorithm, and transmits the results of the image analysis to the terminal device.

[0009] According to the present disclosure, a compressed image in which multiple pieces of wavelength information are compressed can be subjected to image analysis.

[0010] FIG. 1 is a block diagram showing an example of the overall configuration of an image analysis system according to a modification of the first embodiment, the second embodiment, and the third embodiment; FIG. 2 is a sequence diagram showing an example of the operation procedure for advance preparation of the image analysis system according to a modification of the first embodiment, the second embodiment, and the third embodiment; FIG. 3 is a sequence diagram showing an example of the operation procedure of the image analysis system according to a modification of the first embodiment, the second embodiment, and the third embodiment; and FIG. 4 is a diagram explaining an example of image analysis processing.

[0011] Hereinafter, with reference to the drawings as appropriate, detailed descriptions of embodiments that specifically disclose the configurations and operations of an image analysis method, image analysis device, and image analysis system according to the present disclosure will be provided. However, more detailed descriptions than necessary may be omitted. For example, detailed descriptions of well-known matters or redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter recited in the claims.

[0012] (Embodiment 1) A use case of an image analysis system 100 according to embodiment 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the overall configuration of image analysis systems 100, 100A, and 100B according to embodiments 1, 2, and modifications of embodiment 3. Note that only the image analysis system 100 according to embodiment 1 will be described here, and image analysis systems 100A and 100B according to embodiments 2 and 3 will be described later.

[0013] The image analysis system 100 according to the first embodiment assigns tags (information) related to image analysis to a compressed image captured by a terminal device P1 using a camera CM, which is a wideband hyperspectral camera, and in which information about a plurality of wavelengths has been compressed. The image analysis system 100 according to the first embodiment selects an algorithm (hereinafter referred to as a "processing algorithm") for executing image analysis processing corresponding to the tag assigned to the compressed image by the server S1, and performs image analysis on an image (so-called spectral image or spectral data) included in the compressed image and captured with light of a predetermined wavelength, using the selected processing algorithm.

[0014] The image analysis system 100 includes a camera CM, a terminal device P1, an input unit INP, a monitor MN, a light LT, a server S1, and a network NW. Note that the configuration of the image analysis system 100 is not limited to this. For example, the image analysis system 100 may be configured to include multiple terminal devices P1, multiple cameras CM, or multiple servers S1.

[0015] The camera CM is a hyperspectral camera capable of capturing images of light with wavelengths in a wide wavelength band (e.g., 400 nm to 1600 nm, etc.). The camera CM can accept control from the terminal device P1 and captures images in the wide wavelength band. The camera CM generates a compressed image (two-dimensional image data) in which information including the brightness of light having each of multiple wavelengths included in the wavelength band (i.e., 400 nm to 1600 nm) that the camera CM can capture is compressed (hereinafter referred to as "wavelength information"), and transmits the compressed image to the terminal device P1. Note that in the following description, "wavelength information" may also be referred to as "spectral information."

[0016] The camera CM is configured with at least a filter array (not shown), an image sensor (not shown), and a lens (not shown). The filter array is an optical element having multiple light-transmitting regions, each of which has a different light transmission spectrum, i.e., wavelength dependency of light transmittance. The image sensor is a solid-state imaging element including, for example, an InGaAs photodiode or a Si photodiode and, for example, a charged-coupled device (hereinafter referred to as "CCD") or a complementary metal oxide semiconductor (hereinafter referred to as "CMOS"), and converts an optical image formed on the imaging surface by light of a wide wavelength band into an electrical signal.

[0017] The terminal device P1 can accept user operations and is realized by, for example, a personal computer (hereinafter referred to as "PC"), a notebook PC, a tablet terminal, a smartphone, etc. The terminal device P1 assigns tags to compressed images transmitted from the camera CM, and outputs and displays image analysis results transmitted from the server S1 on a monitor MN. The terminal device P1 includes a communication unit 10, a processor 11, and a memory 12.

[0018] The communication unit 10 is connected to the server S1 via the network NW so as to be able to communicate with them via wired or wireless communication, and is also connected to the input unit INP, the monitor MN, the camera CM, and the lighting LT so as to be able to communicate with them via wired or wireless communication. The wireless communication here refers to communication via a wireless local area network (LAN) such as Wi-Fi (registered trademark).

[0019] The processor 11 is configured using, for example, a Central Processing Unit (hereinafter referred to as "CPU") or a Field Programmable Gate Array (hereinafter referred to as "FPGA"), and performs various processes and controls in cooperation with the memory 12. Specifically, the processor 11 references the programs and data stored in the memory 12 and executes the programs to realize the functions of the UI unit 111, the camera operation unit 112, and the lighting operation unit 113.

[0020] The UI unit 111 controls the input unit INP and the monitor MN, which are user interfaces. The UI unit 111 accepts user operations related to the camera parameters of the camera CM or the illumination intensity of the illumination LT via the input unit INP, and accepts input operations for tags to be assigned to compressed images that are the subject of image analysis. The UI unit 111 outputs information on the camera parameters of the camera CM to the camera operation unit 112 and outputs information on the illumination intensity of the illumination LT to the illumination operation unit 113. The UI unit 111 assigns tags to the compressed images and transmits them to the server S1.

[0021] The UI unit 111 also acquires information on the image analysis results sent from the server S1, generates image data SC1 and SC2 (see FIG. 4) showing the image analysis results, and outputs and displays them on the monitor MN.

[0022] The camera operation unit 112 can execute control of the camera CM, and changes the camera parameters set in the camera CM and acquires compressed images transmitted from the camera CM based on the camera parameters of the camera CM output from the UI unit 111. The camera operation unit 112 outputs the acquired compressed images to the UI unit 111.

[0023] The lighting operation unit 113 is capable of controlling the lighting LT, and changes the lighting intensity set for the lighting LT based on the lighting intensity of the lighting LT output from the UI unit 111.

[0024] The memory 12 includes, for example, a random access memory (hereinafter referred to as "RAM") as a work memory used when executing each process of the processor 11, and a read only memory (hereinafter referred to as "ROM") that stores programs and data that define the operations of the processor 11. The RAM temporarily stores data or information generated or acquired by the processor 11. The ROM stores programs that define the operations of the processor 11.

[0025] The monitor MN is configured using, for example, a Liquid Crystal Display (LCD) or an organic electroluminescence (EL) display. The monitor MN displays an image analysis processing result screen (not shown) including the image data SC1 and SC2 (see FIG. 4) output from the UI unit 111.

[0026] The input unit INP is a user interface that can accept input operations by the user, and is realized by, for example, a keyboard, a mouse, a touch panel, etc. The input unit INP converts the content of the accepted input operation into an electrical signal and transmits it to the UI unit 111. When the input unit INP is realized by a touch panel, the input unit INP may be configured integrally with the monitor MN.

[0027] The lighting LT is controlled by the lighting operation unit 113 and illuminates the imaging target imaged by the camera CM. The lighting LT is configured to include one or more lighting elements such as a light-emitting diode (LED), a laser diode (LD), and an InfraRed (IR) light.

[0028] The server S1 (an example of an image analysis device) is a so-called cloud server that selects a processing algorithm to be used for image analysis processing based on a tag attached to a compressed image transmitted from the terminal device P1. The server S1 performs image analysis processing on the compressed image using the selected processing algorithm. The server S1 transmits the image processing results to the terminal device P1. The server S1 includes a communication unit 20, a processor 21, a memory 22, and an algorithm database DB. Note that the configuration of the server S1 is not limited to the example shown in FIG. 1 .

[0029] The communication unit 20 (an example of an acquisition unit) is connected to the terminal device P1 via the network NW so as to be able to communicate with the terminal device P1 via wired or wireless communication. The communication unit 20 outputs the compressed image transmitted from the terminal device P1 to the processor 21, and transmits information on the image analysis result output from the processor 21 to the terminal device P1.

[0030] The processor 21 is configured using, for example, a CPU or FPGA, and performs various processes and controls in cooperation with the memory 22. Specifically, the processor 21 references the programs and data stored in the memory 22 and executes the programs to realize the functions of a compressed image restoration unit 211 and a processing algorithm selection unit 212.

[0031] The compressed image restoration unit 211 restores the compressed images transmitted from the terminal device P1 and acquires images for each wavelength band. The compressed image restoration unit 211 outputs each of the restored images and information on tags assigned to the compressed images to the processing algorithm selection unit 212.

[0032] The processing algorithm selection unit 212 (an example of a selection unit) selects an algorithm to be used for image analysis processing from at least one algorithm registered in the algorithm database DB based on tag information output from the compressed image restoration unit 211. Using the selected algorithm, the processing algorithm selection unit 212 performs image analysis processing on at least one image captured by the camera CM using light of a wavelength that is the target of the image analysis processing within the wavelength band captured by the camera CM. The processing algorithm selection unit 212 outputs information on the results of the image analysis processing to the communication unit 20, which then transmits the information to the terminal device P1.

[0033] The memory 22 includes, for example, a RAM as a work memory used when the processor 21 executes each process, and a ROM for storing programs and data that define the operations of the processor 21. The RAM temporarily stores data or information generated or acquired by the processor 21. The ROM stores programs that define the operations of the processor 21.

[0034] The algorithm database DB is configured using a storage device such as a hard disk drive (HDD) or a solid state drive (SSD). The algorithm database DB stores at least one algorithm used for image analysis processing of compressed images or images captured by the camera CM. The algorithm database DB is an algorithm used for image analysis of compressed images or images captured by the camera CM, which is a hyperspectral camera, and is an algorithm that, when used in image analysis processing, can realize image analysis processing such as object identification, abnormality detection inside packaging material, detection of water damage, detection of high temperature areas, recognition of printed matter such as characters, material identification, and ingredient identification.

[0035] The network NW connects the terminal device P1 and the server S1 so that data communication is possible between them.

[0036] The image analysis system 100 according to the first embodiment may use a polarizing plate to polarize the incident light entering the image sensor of the camera CM. For example, the image analysis system 100 may further include a first polarizing plate that polarizes the illumination light from the illumination LT in a first direction, or a second polarizing plate that polarizes the incident light entering the image sensor of the camera CM in a second direction different from the first direction. This allows the image analysis system 100 to capture images that are more suitable for image analysis processing.

[0037] The configuration of the camera CM, the process of generating compressed images, and the process of restoring compressed images may use known configurations or technologies, such as those disclosed in Japanese Patent Application Laid-Open No. 2021-110869, Japanese Patent Application Laid-Open No. 2022-000968, Japanese Patent Application Laid-Open No. 2022-058424, Japanese Patent Application Laid-Open No. 2023-082047, etc.

[0038] Next, an example of an operation procedure for advance preparation will be described with reference to Fig. 2. Fig. 2 is a sequence diagram showing an example of an operation procedure for advance preparation of the image analysis system 100 according to the first embodiment, the second embodiment, and the modified example of the third embodiment.

[0039] Here, an example of the operational procedure of advance preparation performed by the image analysis system 100 according to embodiment 1 will be described, but the example of the operational procedure of advance preparation performed by the image analysis systems 100A and 100B according to embodiments 2 and 3 is similar, so description thereof will be omitted. Furthermore, the advance preparation shown in Fig. 2 is a process that is performed every time the imaging environment (evaluation environment) of the imaging analysis target is changed, and is not essential and may be omitted.

[0040] The terminal device P1 receives an input operation of information related to the camera parameters of the camera CM via the input unit INP (St11). The camera parameters here refer to the exposure time, focal length, gain, etc. of the camera CM. The terminal device P1 acquires the information related to the camera parameters of the camera CM via the input unit INP, associates the information related to the camera parameters with a control command requesting a change in the parameters of the camera CM, and transmits the associated information to the camera CM (St12).

[0041] Based on the control command transmitted from the terminal device P1, the camera CM changes the current camera parameters to camera parameters corresponding to the information on the camera parameters transmitted from the terminal device P1 (St13). When the camera parameter change is complete, the camera CM generates a camera parameter change completion notification and transmits it to the terminal device P1 (St14).

[0042] The terminal device P1 receives an input operation related to the illumination intensity of the lamp LT via the input unit INP (St15). The terminal device P1 acquires the illumination intensity of the lamp LT via the input unit INP, associates the illumination intensity information with a control command requesting a change in the illumination intensity of the lamp LT, and transmits the information to the lamp LT (St16).

[0043] Based on the control command transmitted from the terminal device P1, the lighting LT changes the current lighting intensity to a lighting intensity corresponding to the lighting intensity information transmitted from the terminal device P1 (St17). When the lighting intensity change is complete, the lighting LT generates a lighting intensity change completion notification and transmits it to the terminal device P1 (St18).

[0044] The terminal device P1 generates a control command requesting the camera CM to capture a dark image and transmits the control command to the camera CM (St19). Note that the dark image referred to here is an image that is specific to the camera CM and is used to reduce noise that the camera CM has.

[0045] Based on the control command transmitted from the terminal device P1, the camera CM captures an image in a light-blocking state with the shutter closed (St20), and transmits the captured light-blocking image to the terminal device P1 (St21).

[0046] The terminal device P1 generates a control command requesting the camera CM to capture a whiteboard image and transmits the control command to the camera CM (St22). Note that the whiteboard image here is an image used to remove the spectrum of the illumination light from the lamp LT.

[0047] The camera CM captures an image of the whiteboard based on the control command transmitted from the terminal device P1 (St23), and transmits the captured image of the whiteboard to the terminal device P1 (St24).

[0048] The terminal device P1 performs shading correction on the acquired whiteboard image using the shading image (St25), and transmits the shading-corrected whiteboard image to the server S1 (St26).

[0049] The server S1 records the whiteboard image transmitted from the terminal device P1 in the memory 22 (St27).

[0050] Next, an example of an operation procedure of the image analysis system 100 will be described with reference to Fig. 3. Fig. 3 is a sequence diagram showing an example of an operation procedure for advance preparation of the image analysis system 100 according to the first embodiment, the second embodiment, and the modified example of the third embodiment.

[0051] Here, only an example of the operation procedure performed by the image analysis system 100 according to the first embodiment will be described, and examples of the operation procedures performed by the image analysis systems 100A and 100B according to the second and third embodiments will be described later.

[0052] The terminal device P1 receives an input operation of information related to the camera parameters of the camera CM through the input unit INP (St31). The terminal device P1 acquires the information related to the camera parameters of the camera CM through the input unit INP, associates the information related to the camera parameters with a control command requesting a change of the camera CM's parameters, and transmits the information to the camera CM, causing the camera CM to change the camera parameters (St32).

[0053] The terminal device P1 receives an input operation related to the illumination intensity of the lamp LT via the input unit INP (St33). The terminal device P1 acquires the illumination intensity of the lamp LT via the input unit INP, associates the illumination intensity information with a control command requesting a change in the illumination intensity of the lamp LT, and transmits the control command to the lamp LT, thereby changing the illumination intensity of the lamp LT (St34).

[0054] The terminal device P1 generates a control command requesting the camera CM to capture an image of the subject, and transmits the control command to the camera CM (St35).

[0055] The camera CM captures an image of the subject for each wavelength across the wavelength band that the camera CM can capture based on the control command transmitted from the terminal device P1 (St36). The camera CM generates a compressed image in which information about multiple wavelengths is compressed, and transmits the compressed image to the terminal device P1 (St37).

[0056] The terminal device P1 performs shading correction on the compressed image transmitted from the camera CM based on the shading image captured in advance (St38). Note that the processing of step St38 may be performed only when the shading image has been captured in advance, and is not essential and may be omitted.

[0057] The terminal device P1 accepts, based on a user operation, an input of a tag used in the selection process of a processing algorithm to be used in the image analysis process. The tag here refers to information including information about the subject to be analyzed (e.g., the subject's name, model number, ID, etc.), information about the content of the analysis process (e.g., freshness, temperature, film thickness, water, etc.), etc. The terminal device P1 acquires the tag information transmitted from the input unit INP, assigns the acquired tag to the compressed image (St39), and transmits the image to the server S1 (St40).

[0058] The server S1 decompresses the compressed images transmitted from the terminal device P1 and acquires images for each wavelength band (St42). The server S1 performs whiteboard correction on each of the acquired images using a pre-recorded whiteboard image (St43). Note that the processing of step St43 may be executed only if a whiteboard image has been captured in advance, and is not essential and may be omitted.

[0059] Based on the tag information attached to the compressed image, the server S1 selects a processing algorithm to be used for image analysis processing of the restored images from at least one processing algorithm registered in the algorithm database DB (Step 44). The server S1 performs image analysis processing on at least one image corresponding to a predetermined wavelength band among the restored images using the selected processing algorithm (Step 44). The server S1 transmits the image analysis processing result to the terminal device P1 (Step 45).

[0060] The terminal device P1 generates image data SC1 and SC2 (see FIG. 4) showing the image analysis processing results based on the image analysis results transmitted from the server S1 (St46). The terminal device P1 generates an image analysis processing result screen (not shown) including the generated image data SC1 and SC2, and outputs it to the monitor MN for display (St47).

[0061] As described above, the image analysis system 100 according to the first embodiment can realize image analysis processing using a predetermined processing algorithm by the server S1 by assigning a single tag containing information for selecting a processing algorithm to be used in the image analysis processing of a compressed image in which multiple wavelength information pieces have been compressed.

[0062] Next, an example of image analysis processing performed by the image analysis systems 100, 100A, and 100B will be described with reference to Fig. 4. Fig. 4 is a diagram for explaining the example of image analysis processing.

[0063] The image analysis processing example shown in Case No. (1) in Fig. 4 is image analysis processing for estimating the freshness of an apple. The compressed image CIM in Case No. (1) is assigned tags by the terminal device P1, including information about the subject, "apple," and information about the content of the analysis processing, "freshness."

[0064] Based on the tag information attached to the compressed image CIM, the server S1 selects the processing algorithm "Apple Freshness Estimation" from among the processing algorithms registered in the algorithm database DB. Based on the selected processing algorithm "Apple Freshness Estimation," the server S1 extracts, from each of the restored images, images captured with light of a wavelength corresponding to the wavelength band "450-750 (nm)" that is the target of image analysis. The server S1 performs image analysis processing on each of the extracted images using the processing algorithm "Apple Freshness Estimation," and transmits information on the results of the image analysis processing to the terminal device P1.

[0065] Based on the information on the image analysis processing results sent from the server S1, the terminal device P1 generates image data SC1 that shows the image analysis processing results (freshness of the apples) in a heat map, and outputs and displays it on the monitor MN.

[0066] The image analysis processing example shown in Case No. (2) in Fig. 4 is image analysis processing for estimating the variety of an apple. In Case No. (2), the compressed image CIM is assigned tags by the terminal device P1, including information about the subject, "apple," and information about the content of the analysis processing, "variety."

[0067] Based on the tag information attached to the compressed image CIM, the server S1 selects the processing algorithm "apple variety estimation" from among the processing algorithms registered in the algorithm database DB. Based on the selected processing algorithm "apple variety estimation," the server S1 extracts, from each of the restored images, images captured with light having a wavelength corresponding to the wavelength band "450-1600 (nm)" that is the target of image analysis. The server S1 performs image analysis processing on each of the extracted images using the processing algorithm "apple variety estimation," and transmits information on the results of the image analysis processing to the terminal device P1.

[0068] Based on the information on the image analysis processing results sent from the server S1, the terminal device P1 generates image data SC2 that shows the image analysis processing results (apple varieties) using classification such as color coding, and supplementary information regarding the image analysis processing results (classification results) of this image data SC2, such as "0: ●●, 1: XXX, 2: △△", and outputs it to the monitor MN for display.

[0069] The image analysis processing example shown in Case No. (3) in Fig. 4 is an image analysis processing for estimating the state (quality) of a paint job. The compressed image in Case No. (3) is assigned tags by the terminal device P1, including information about the subject, "paint," and information about the content of the analysis processing, "unevenness."

[0070] Based on the tag information attached to the compressed image, server S1 selects the processing algorithm "paint unevenness estimation" from among the processing algorithms registered in the algorithm database DB. Based on the selected processing algorithm "paint unevenness estimation," server S1 extracts, from each of the restored images, images captured with light having a wavelength corresponding to the wavelength band "450-850 (nm)" that is the target of image analysis. Server S1 performs image analysis processing on each of the extracted images using the processing algorithm "paint unevenness estimation," and transmits information on the results of the image analysis processing to terminal device P1.

[0071] Based on the information on the image analysis processing results sent from the server S1, the terminal device P1 generates image data showing the image analysis processing results (paint unevenness estimation) in a heat map, and outputs and displays it on the monitor MN.

[0072] The image analysis process example shown in Case No. (4) in Fig. 4 is an image analysis process for estimating the state (quality) of a paint coating. The compressed image in Case No. (4) is assigned tags by the terminal device P1, including information about the subject, "paint," and information about the content of the analysis process, "film thickness."

[0073] Based on the tag information attached to the compressed image, the server S1 selects the processing algorithm "film thickness estimation" from among the processing algorithms registered in the algorithm database DB. Based on the selected processing algorithm "film thickness estimation," the server S1 extracts, from each of the restored images, images captured with light of a wavelength corresponding to the wavelength band "1600 (nm)" that is the target of image analysis. Using the processing algorithm "film thickness estimation," the server S1 performs image analysis processing on each of the extracted images and generates supplemental information "(film thickness) units" related to the image analysis processing results (film thickness estimation) of this image data, and outputs and displays the information on the monitor MN.

[0074] Based on the information on the image analysis processing results sent from the server S1, the terminal device P1 generates image data showing the estimated paint film thickness in a heat map, and outputs and displays it on the monitor MN.

[0075] The image analysis process example shown in Case No. (5) in Fig. 4 is an image analysis process for estimating the condition (quality) of a paint job. The compressed image in Case No. (5) is assigned tags by the terminal device P1, including information about the subject, "deterioration," and information about the content of the analysis process, "deterioration."

[0076] Based on the tag information attached to the compressed image, the server S1 selects the processing algorithm "paint deterioration estimation" from among the processing algorithms registered in the algorithm database DB. Based on the selected processing algorithm "paint deterioration estimation," the server S1 extracts, from each of the restored images, images captured with light having a wavelength corresponding to the wavelength band "750-1600 (nm)" that is the target of image analysis. The server S1 performs image analysis processing on each of the extracted images using the processing algorithm "paint deterioration estimation," and transmits information on the results of the image analysis processing to the terminal device P1.

[0077] Based on the information on the image analysis processing results sent from the server S1, the terminal device P1 generates image data showing the estimated results of paint deterioration in the form of a heat map, and outputs and displays the image data on the monitor MN.

[0078] (Embodiment 2) The image analysis system 100 according to embodiment 1 has been described as an example in which a compressed image transmitted from the terminal device P1 is restored and image analysis processing is performed. The image analysis system 100A according to embodiment 2 will be described as an example in which image analysis processing is performed without restoring the compressed image transmitted from the terminal device P1.

[0079] Note that the image analysis system 100A according to embodiment 2 has a configuration that is substantially the same as the configuration of the image analysis system 100 according to embodiment 1, and therefore in the following explanation, the same symbols will be used for the configuration that is the same as that of the image analysis system 100 according to embodiment 1, and explanations thereof will be omitted.

[0080] An image analysis system 100A according to Embodiment 2 will be described with reference to Fig. 1. The image analysis system 100A includes a camera CM, a terminal device P1, an input unit INP, a monitor MN, a light LT, a server S1A, and a network NW.

[0081] The server S1A (an example of an image analysis device) includes a communication unit 20, a processor 21A, a memory 22, and an algorithm database DB.

[0082] The compressed image restoration unit 211A of the processor 21A determines whether or not the compressed image needs to be restored based on tag information attached to the compressed image transmitted from the terminal device P1. For example, in the example of image analysis processing shown in FIG. 4 , if the tags attached to the compressed image are "painted" information about the subject and "uneven" information about the analysis processing content, the compressed image restoration unit 211A omits restoration of the compressed image based on whether or not restoration of the compressed image is necessary. On the other hand, if the compressed image restoration unit 211A determines that restoration of the compressed image is necessary based on the tags attached to the compressed image, the compressed image restoration unit 211A performs restoration of the compressed image. The compressed image restoration unit 211A outputs the compressed image or each of the multiple restored images, and information about the tags attached to the compressed images, to the processing algorithm selection unit 212A.

[0083] The processing algorithm selection unit 212A (an example of a selection unit) selects an algorithm to be used for image analysis processing from at least one algorithm registered in the algorithm database DB, based on tag information output from the compressed image restoration unit 211. Using the selected algorithm, the processing algorithm selection unit 212A performs image analysis processing on at least one image corresponding to a wavelength or wavelength band that is the target of image analysis processing, from among the compressed image or the wavelength band captured by the camera CM.

[0084] 4, an example of the operation procedure of the image analysis system 100A according to embodiment 2 will be described. Note that, since steps St11 to St40 and steps St42 to St47 of the operation procedure of the image analysis system 100A are the same as those described in the description of the operation procedure of the image analysis system 100 according to embodiment 1, their description will be omitted.

[0085] The server S1A receives the compressed image transmitted from the terminal device P1 and determines whether or not the compressed image needs to be decompressed based on the tag information attached to the compressed image (St41A).

[0086] If the server S1A determines that the compressed image needs to be decompressed, it executes the processes of steps St42 and St43. Note that the process of step St43 is not essential and may be omitted.

[0087] On the other hand, if the server S1A determines that decompression of the compressed image is not necessary, it skips the process of step St42 and executes the processes of step St43 and step St44. Note that the process of step St43 is not essential and may be omitted.

[0088] As described above, the image analysis system 100A according to the second embodiment can further reduce the time required for image analysis processing by omitting the restoration process of compressed images when restoration of compressed images is not required in the image analysis processing.

[0089] (Embodiment 3) The image analysis system 100 according to embodiment 1 has been described as an example of executing image analysis processing based on a processing algorithm corresponding to a tag attached to a compressed image transmitted from the terminal device P1. An image analysis system 100B according to embodiment 3 will be described as an example of performing image analysis on a compressed image transmitted from the terminal device P1B and executing image analysis processing based on a processing algorithm corresponding to the image analysis result.

[0090] Note that the image analysis system 100B according to embodiment 3 has a configuration that is substantially the same as that of the image analysis system 100 according to embodiment 1, and therefore in the following explanation, the same symbols will be used for the configuration that is the same as that of the image analysis system 100 according to embodiment 1, and explanations thereof will be omitted.

[0091] An image analysis system 100B according to Embodiment 3 will be described with reference to Fig. 1. The image analysis system 100B includes a camera CM, a terminal device P1B, an input unit INP, a monitor MN, a light LT, a server S1B, and a network NW.

[0092] The terminal device P1B includes a communication unit 10, a processor 11B, and a memory 12.

[0093] The processor 11B omits receiving tag information via the input unit INP and assigning tags to the compressed image. The processor 11B transmits a compressed image in which the plurality of wavelength information transmitted from the camera CM is compressed to the server S1B.

[0094] The server S1B (an example of an image analysis device) includes a communication unit 20, a processor 21B, a memory 22, and an algorithm database DB.

[0095] The processing algorithm selection unit 212B (an example of a selection unit) of the processor 21B performs image analysis on at least one of the multiple images output from the compressed image restoration unit 211, and estimates information about the subject or information about the content of the analysis processing. Note that the information about the content of the analysis processing may be estimated based on the wavelength at which the image was captured.

[0096] The processing algorithm selection unit 212B selects a processing algorithm to be used for image analysis processing of the restored images from at least one processing algorithm registered in the algorithm database DB, based on information about the estimated subject or information about the content of the analysis processing. The server S1B executes image analysis processing on each of the restored images using the selected processing algorithm.

[0097] 4, an example of the operation procedure of the image analysis system 100B according to embodiment 3 will be described. Note that, since steps St11 to St38 and steps St42 to St43 of the operation procedure of the image analysis system 100B are the same as those described in the description of the operation procedure of the image analysis system 100 according to embodiment 1, the description thereof will be omitted.

[0098] The terminal device P1B accepts, based on a user operation, an input of a tag to be used in the selection process of a processing algorithm to be used in the image analysis process. The tag here refers to information including information about the subject to be analyzed (e.g., the subject's name, model number, ID, etc.), information about the content of the analysis process (e.g., freshness, temperature, film thickness, water, etc.), etc. The terminal device P1B skips the process of step St39 (the process of acquiring tag information and the process of assigning tags) and transmits the tag information to the server S1B (St40B).

[0099] The server S1B performs image analysis processing on the restored images and estimates information about the subject or information about the content of the analysis processing (St44B). Based on the estimated information about the subject or information about the content of the analysis processing, the server S1B selects a processing algorithm to be used in the image analysis processing of the restored images from at least one processing algorithm registered in the algorithm database DB (St44B). The server S1B performs image analysis processing on each of the restored images using the selected processing algorithm (St44B).

[0100] This allows the server S1B in the second embodiment to select a processing algorithm based on the subject appearing in the restored image, thereby enabling the server S1B to perform image analysis processing using the selected processing algorithm even if a tag is not added to the compressed image.

[0101] (Additional Notes) The above description of each embodiment discloses the following techniques.

[0102] (Technology 1) An image analysis method performed by an image analysis device (server S1, S1A, S1B) that is a computer, comprising the steps of: acquiring a compressed image in which multiple pieces of wavelength information have been compressed; selecting one of multiple algorithms to be used for image analysis processing of the compressed image based on the compressed image; using the selected processing algorithm, performing image analysis on at least one image of the compressed image that was captured with light of a wavelength corresponding to the processing algorithm; and outputting the image analysis results. With this configuration, the servers S1, S1A, S1B can select a processing algorithm to be used for image analysis processing of the compressed image (image), and can perform image analysis of the compressed image in which multiple pieces of wavelength information have been compressed by using the selected processing algorithm.

[0103] (Technology 2) The image analysis method according to (Technology 1), comprising: selecting one of a plurality of algorithms to be used for image analysis processing of the compressed image based on tag information related to image analysis of the compressed image attached to the compressed image; using the selected processing algorithm, performing image analysis on at least one image among the compressed images captured with light of a wavelength corresponding to the processing algorithm; and outputting the image analysis results. With this configuration, the servers S1 and S1A can select processing algorithms suitable for image analysis processing of images captured with different wavelengths based on the tag information.

[0104] (Technology 3) The image analysis method according to (Technology 2), which determines whether a compressed image needs to be restored based on tag information, and if it is determined that the compressed image needs to be restored, restores the compressed image and performs image analysis on images captured with light of a wavelength corresponding to a processing algorithm among a plurality of images corresponding to each of the restored plurality of wavelength information. With this configuration, the server S1A can determine whether a compressed image needs to be restored based on the tag information.

[0105] (Technology 4) The image analysis method according to (Technology 2) or (Technology 3), in which the server S1A determines whether or not a compressed image needs to be restored based on tag information, and if it determines that the compressed image does not need to be restored, omits the restoration of the compressed image and performs image analysis on the compressed image using a processing algorithm. With this configuration, if the server S1A determines that the compressed image does not need to be restored based on the tag information, the server S1A can omit the restoration process of the compressed image and perform image analysis. This allows the server S1A to reduce the processing time required for the restoration process of the compressed image.

[0106] (Technology 5) The image analysis method according to (Technology 1) includes: restoring a compressed image; performing image processing on at least one of a plurality of images corresponding to each of the restored wavelength information to detect a subject appearing in the image; and selecting one of a plurality of algorithms to be used for image analysis processing of the image based on the detected subject. This configuration allows the server S1B to select a processing algorithm based on the subject appearing in the image. As a result, the server S1B can perform image analysis processing using the selected processing algorithm even if a tag is not assigned to the compressed image.

[0107] (Technology 6) The image analysis method according to any one of (Technology 1) to (Technology 5), wherein the compressed image is compressed with wavelength information from 400 nm to 1600 nm. With this configuration, the servers S1, S1A, and S1B can acquire a compressed image in which multiple wavelength information included in a wide wavelength band is compressed.

[0108] (Technology 7) An image analysis device (servers S1, S1A, S1B) comprising: an acquisition unit (communication unit 20) that acquires a compressed image in which multiple pieces of wavelength information have been compressed; a selection unit (processing algorithm selection units 212, 212A, 212B) that selects one of multiple algorithms to be used for image analysis processing of the compressed image based on the compressed image; and an image analysis unit (processing algorithm selection units 212, 212A, 212B) that uses the selected processing algorithm to perform image analysis on at least one image of the compressed image that has been captured with light of a wavelength corresponding to the processing algorithm, and outputs the results of the image analysis. With this configuration, the servers S1, S1A, S1B can select a processing algorithm to be used for image analysis processing of the compressed image (image), and can use the selected processing algorithm to perform image analysis of the compressed image in which multiple pieces of wavelength information have been compressed.

[0109] (Technology 8) Image analysis systems 100, 100A, 100B include terminal devices P1, P1B and image analysis devices (servers S1, S1A, S1B) that can communicate with the terminal devices P1, P1B, wherein the terminal devices P1, P1B transmit compressed images in which multiple pieces of wavelength information have been compressed to the image analysis devices (servers S1, S1A, S1B), and the image analysis devices (servers S1, S1A, S1B) select, based on the compressed images, one of multiple algorithms to be used for image analysis processing of the compressed images, and use the selected processing algorithm to perform image analysis on at least one image of the compressed images that was captured with light of a wavelength that corresponds to the processing algorithm, and transmit the results of the image analysis to the terminal devices P1, P1B. With this configuration, the image analysis systems 100, 100A, and 100B can select a processing algorithm to be used for image analysis processing of a compressed image (image), and by using the selected processing algorithm, can perform image analysis of a compressed image in which multiple wavelength information pieces are compressed.

[0110] Although various embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that those skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also fall within the technical scope of the present disclosure. Furthermore, the components of the various embodiments described above may be combined in any manner without departing from the spirit of the invention.

[0111] This application is based on a Japanese patent application (Patent Application No. 2023-116283) filed on July 14, 2023, the contents of which are incorporated herein by reference.

[0112] The present disclosure is useful as a proposal for an image analysis method, an image analysis device, and an image analysis system for analyzing a compressed image in which multiple pieces of wavelength information are compressed.

[0113] 10, 20 Communication unit 11, 11B, 21, 21A, 21B Processor 12, 22 Memory 100, 100A, 100B Image analysis system 111 UI unit 112 Camera operation unit 113 Lighting operation unit 211, 211A Compressed image restoration unit 212, 212A, 212B Processing algorithm selection unit CM Camera DB Algorithm database LT Lighting MN Monitor NW Network P1, P1B Terminal device S1, S1A, S1B Server

Claims

1. An image analysis method performed by an image analysis device which is a computer, comprising the steps of: acquiring a compressed image in which a plurality of wavelength information pieces are compressed; selecting, based on the compressed image, one of a plurality of algorithms to be used for image analysis processing of the compressed image; using the selected processing algorithm, performing image analysis on at least one image of the compressed image captured with light of a wavelength corresponding to the processing algorithm; and outputting the image analysis results.

2. The image analysis method according to claim 1, further comprising the steps of: selecting one of the plurality of algorithms to be used for image analysis processing of the compressed image based on tag information relating to image analysis of the compressed image added to the compressed image; performing image analysis of the compressed image captured with light of a wavelength corresponding to the processing algorithm using the selected processing algorithm; and outputting the image analysis result.

3. The image analysis method according to claim 2, further comprising determining whether or not the compressed image needs to be restored based on the tag information, and if it is determined that the compressed image needs to be restored, restoring the compressed image and performing image analysis on the image captured with light of a wavelength corresponding to the processing algorithm among a plurality of images corresponding to each of the restored plurality of wavelength information.

4. The image analysis method according to claim 2, further comprising: determining whether or not the compressed image needs to be restored based on the tag information; and, if it is determined that restoration of the compressed image is not necessary, omitting restoration of the compressed image and performing image analysis of the compressed image using the processing algorithm.

5. The image analysis method according to claim 1, further comprising the steps of: restoring the compressed image; executing image processing on at least one of a plurality of images corresponding to each of the restored plurality of wavelength information; detecting an object appearing in the image; and selecting, based on the detected object, one of the plurality of algorithms to be used for image analysis processing of the image.

6. The image analysis method according to claim 1, wherein the compressed image compresses wavelength information from 400 nm to 1600 nm.

7. An image analysis device comprising: an acquisition unit that acquires a compressed image in which multiple pieces of wavelength information are compressed; a selection unit that selects, based on the compressed image, one of multiple algorithms to be used for image analysis processing of the compressed image; and an image analysis unit that uses the selected processing algorithm to perform image analysis on at least one image of the compressed image that has been captured with light of a wavelength corresponding to the processing algorithm, and outputs the results of the image analysis.

8. An image analysis system comprising: a terminal device; and an image analysis device capable of communicating with the terminal device, wherein the terminal device transmits a compressed image in which a plurality of pieces of wavelength information are compressed to the image analysis device, and the image analysis device selects, based on the compressed image, one of a plurality of algorithms to be used for image analysis processing of the compressed image, and uses the selected processing algorithm to perform image analysis on at least one image of the compressed image captured with light of a wavelength corresponding to the processing algorithm, and transmits the result of the image analysis to the terminal device.

Images