Image analysis method, image analysis device, and image analysis system

Abstract

To perform image analysis of a plurality of different objects photographed in a compressed image.SOLUTION: An image analysis method acquires a compressed image obtained by compressing a plurality of pieces of wavelength information obtained by imaging a plurality of different subjects to detect a plurality of different subjects, individually specifies information on the subjects based on comparison between a restored image and a registered image containing predetermined wavelength information registered in advance for each subject, selects an algorithm used for image analysis of a plurality of different subjects on the basis of information on the specified subject, executes image analysis of the image by using the selected processing algorithm, and outputs an image analysis result.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to an image analysis method, an image analysis device, and an image analysis system. [Background technology]

[0002] Patent Document 1 discloses a processing device connected via a network to one or more terminals equipped with a hyperspectral sensor that generates compressed image data in which hyperspectral information of an object is compressed as two-dimensional image information. The processing device includes a storage device that stores a data set of multiple samples and a first restoration table for restoring the hyperspectral data from the compressed image data, where each sample data set includes hyperspectral data of the sample and data indicating a characteristic value of the sample, and a processing circuit that generates a statistical model for estimating the characteristic value from the hyperspectral data based on the data set of multiple samples and edits the first restoration table in accordance with the generated statistical model to generate the first restoration table. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2022 / 075094 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, in the analysis of objects (target objects) using hyperspectral data, there has been a demand for a technology that performs image analysis on a single compressed image captured of, for example, fresh food displayed in a supermarket, a plurality of different products being transported on a transport rail, etc. However, in Patent Document 1, the restoration of hyperspectral data for the same target object (e.g., a tomato) and the estimation of the characteristic values ​​of the target object are performed, and therefore the detection of a plurality of different targets, the restoration of hyperspectral data suitable for each of these detected targets, and the estimation of the characteristic values ​​of the target object are not assumed.

[0005] The present disclosure has been devised in view of the above-mentioned conventional circumstances, and aims to provide an image analysis method, an image analysis device, and an image analysis system for performing image analysis of a plurality of different objects appearing in a compressed image. [Means for solving the problem]

[0006] The present disclosure provides an image analysis method performed by an image analysis device that is a computer, which acquires and restores a compressed image in which multiple wavelength information of multiple different subjects captured by imaging the multiple different subjects is compressed, detects the multiple different subjects appearing in the restored image, individually identifies information about each of the subjects based on a comparison between the image and a registered image containing predetermined wavelength information that has been registered in advance for each subject, selects an algorithm to be used for image analysis of each of the subjects based on the identified information about the subjects, performs image analysis of the image using the selected processing algorithm, and outputs an image analysis result.

[0007] The present disclosure also provides an image analysis device including an acquisition unit that acquires a compressed image in which multiple wavelength information captured of multiple different subjects is compressed; a detection unit that restores the compressed image and detects the multiple different subjects appearing in the restored image; and an image analysis unit that individually identifies information about each of the subjects based on a comparison between the image and a registered image containing predetermined wavelength information that has been registered in advance for each subject, selects an algorithm to be used for image analysis of each of the subjects based on the identified information about the subjects, performs image analysis of the image using the selected processing algorithm, and outputs the image analysis results.

[0008] The present disclosure also provides an image analysis system including a terminal device and an image analysis device capable of communicating with the terminal device, wherein the terminal device transmits a compressed image in which multiple wavelength information of a plurality of different subjects captured is compressed to the image analysis device, and the image analysis device restores the compressed image and detects the plurality of different subjects appearing in the restored image, identifies information about each of the subjects individually based on a comparison between the image and a registered image including predetermined wavelength information registered in advance for each subject, selects an algorithm to be used for image analysis of each of the plurality of different subjects based on the identified information about the subjects, performs image analysis of the image using the selected processing algorithm, and outputs an image analysis result. Effect of the Invention

[0009] According to the present disclosure, image analysis can be performed on a plurality of different objects appearing in a compressed image. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing an example of the overall configuration of an image analysis system according to a first embodiment. [Diagram 2] A sequence diagram showing an example of an operation procedure for advance preparation of the image analysis system according to the first embodiment. [Diagram 3]FIG. 13 is a sequence diagram showing an example of an operation procedure for pre-registration in the image analysis system according to the first embodiment. [Figure 4] FIG. 1 is a sequence diagram showing a first example of an operation procedure of the image analysis system according to the first embodiment; [Diagram 5] FIG. 11 is a sequence diagram showing a second example of an operation procedure of the image analysis system according to the first embodiment. [Figure 6] FIG. 1 is a diagram for explaining an example of image analysis processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, with reference to the drawings as appropriate, each embodiment that specifically discloses the configuration and operation of the image analysis method, image analysis device, and image analysis system according to the present disclosure will be described in detail. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters and duplicate explanation of substantially the same configuration may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the attached drawings and the following explanation are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0012] (Embodiment 1) A use case of the image analysis system 100 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the overall configuration of the image analysis system 100 according to the first embodiment.

[0013] The image analysis system 100 according to the first embodiment restores a compressed image captured by a terminal device P1 using a camera CM, which is a wideband hyperspectral camera, and in which multiple pieces of wavelength information have been compressed. The image analysis system 100 detects multiple different objects captured in the restored hyperspectral data (so-called spectral image, spectral data, brightness image, etc.), and performs image analysis on the hyperspectral data captured at a predetermined wavelength (so-called spectral image, spectral data) using an algorithm capable of performing image analysis processing on each of the detected objects (hereinafter referred to as a "processing algorithm").

[0014] The compressed image referred to here is an image captured and generated by the camera CM, and is an image in which hyperspectral information of an object is compressed as two-dimensional image information.

[0015] The image analysis system 100 includes a camera CM, a terminal device P1, an input unit INP, a monitor MN, a light LT, a server S1, and a network NW. The configuration of the image analysis system 100 is not limited to this. For example, the image analysis system 100 may include a plurality of terminal devices P1, a plurality of cameras CM, or a plurality of servers S1.

[0016] The camera CM is a hyperspectral camera capable of capturing images of wavelengths in a wide wavelength band (e.g., 400 nm to 1600 nm, etc.). The camera CM can accept control from the terminal device P1, and captures images in the wide wavelength band. The camera CM generates a compressed image (two-dimensional image data) in which multiple wavelength information (i.e., hyperspectral information) included in the wavelength band that the camera CM can capture (i.e., 400 nm to 1600 nm) is compressed, and transmits the compressed image to the terminal device P1.

[0017] The camera CM is configured to have at least a filter array (not shown), an image sensor (not shown), and a lens (not shown). The filter array is an optical element having a plurality of light-transmitting regions, and the light transmission spectrum, i.e., the wavelength dependency of the light transmittance, differs for each region. The image sensor is a solid-state imaging element including, for example, an InGaAs photodiode or a Si photodiode, and, for example, a Charged-Coupled Device (hereinafter, referred to as "CCD") or a Complementary Metal Oxide Semiconductor (hereinafter, referred to as "CMOS"), and converts an optical image formed by visible light on the imaging surface into an electrical signal.

[0018] The terminal device P1 can accept operations by a user and is realized by, for example, a personal computer (hereinafter, referred to as "PC"), a notebook PC, a tablet terminal, a smartphone, etc. The terminal device P1 accepts registration of an object to be analyzed and displays the image analysis results. The terminal device P1 includes a communication unit 10, a processor 11, and a memory 12.

[0019] The communication unit 10 is connected to the server S1 via the network NW so as to be able to carry out wired or wireless communication, and is also connected to the input unit INP, the monitor MN, the camera CM, and the lighting LT so as to be able to carry out wired or wireless communication, respectively. The wireless communication here refers to communication via a wireless Local Area Network (LAN) such as Wi-Fi (registered trademark).

[0020] Processor 11 is configured using, for example, a Central Processing Unit (hereinafter referred to as "CPU") or a Field Programmable Gate Array (hereinafter referred to as "FPGA"), and performs various processes and controls in cooperation with memory 12. Specifically, processor 11 refers to programs and data stored in memory 12 and executes the programs to realize the respective functions of UI section 111, camera operation section 112, and lighting operation section 113.

[0021] The UI unit 111 realizes control of the input unit INP and the monitor MN, which are user interfaces. The UI unit 111 accepts user operations related to the camera parameters of the camera CM or the illumination intensity of the illumination LT, and accepts input operations of the registered name of an object that is the subject of image analysis, via the input unit INP. The UI unit 111 outputs information on the camera parameters of the camera CM to the camera operation unit 112, and outputs information on the illumination intensity of the illumination LT to the illumination operation unit 113. The UI unit 111 assigns the registered name of the object to the compressed image and transmits it to the server S1.

[0022] Furthermore, the UI unit 111 acquires information on the image analysis results transmitted from the server S1, generates image data SC1, SC2 (see FIG. 6) indicating the image analysis results, and outputs them to the monitor MN for display.

[0023] Camera operation unit 112 can execute control of the camera CM, and changes camera parameters set in the camera CM and acquires compressed images transmitted from the camera CM based on the camera parameters of the camera CM output from UI unit 111. Camera operation unit 112 outputs the acquired compressed images to UI unit 111.

[0024] The lighting operation unit 113 is capable of executing control of the lighting LT, and changes the lighting intensity set in the lighting LT based on the lighting intensity of the lighting LT output from the UI unit 111.

[0025] The memory 12 has, for example, a random access memory (hereinafter referred to as "RAM") as a work memory used when the processor 11 executes each process, and a read only memory (hereinafter referred to as "ROM") for storing programs and data that define each operation of the processor 11. The RAM temporarily stores data or information generated or acquired by the processor 11. The ROM has written therein programs that define the operation of the processor 11.

[0026] The monitor MN is configured using, for example, a Liquid Crystal Display (LCD) or an organic electroluminescence (EL) display. The monitor MN displays an image analysis processing result screen (not shown) including image data SC1 and SC2 (see FIG. 6) output from the UI unit 111.

[0027] The input unit INP is a user interface capable of accepting input operations by a user, and is realized by, for example, a keyboard, a mouse, a touch panel, etc. The input unit INP converts the contents of the accepted input operation into an electric signal and transmits it to the UI unit 111. When the input unit INP is realized by a touch panel, the input unit INP may be configured integrally with the monitor MN.

[0028] The lighting LT is controlled by a lighting operation unit 113, and illuminates an imaging target imaged by the camera CM. The lighting LT includes one or more lighting elements, such as a Light Emittimg Diode (LED), a Laser Diode (LD), and an InfraRed (IR) lighting element.

[0029] The server S1 is a so-called cloud server that detects a plurality of different objects appearing in a compressed image transmitted from the terminal device P1, identifies the registered names of the objects, and selects a processing algorithm corresponding to each of the identified registered names. The server S1 performs image analysis of each of the plurality of different objects using the selected processing algorithm, and transmits the image processing results to the terminal device P1. The server S1 includes a communication unit 20, a processor 21, a memory 22, an algorithm database DB1, and a registered name database DB2. Note that the configuration of the server S1 is not limited to the example shown in FIG. 1.

[0030] The communication unit 20 is connected to the terminal device P1 via the network NW so as to be able to perform wired or wireless communication with the terminal device P1. The communication unit 20 outputs data such as compressed images and registered names transmitted from the terminal device P1 to the processor 21, and transmits information on the image analysis results output from the processor 21 to the terminal device P1.

[0031] The processor 21 is configured using, for example, a CPU or an FPGA, and performs various processes and controls in cooperation with the memory 22. Specifically, the processor 21 refers to the programs and data stored in the memory 22 and executes the programs to realize the functions of an object registration unit 211, a compressed image restoration unit 212, an object detection unit 213, an object matching unit 214, and a processing algorithm selection unit 215.

[0032] The object registration unit 211 associates the registration name of the object transmitted from the terminal device P1 with the hyperspectral data in which the object appears, and registers the associated name in the registration name database DB2. Note that the object registration unit 211 generates a cut-out image by cutting out an area in which the object to be registered appears from the hyperspectral data, and registers the cut-out image in the registration name database DB2 in association with the registration name of the object.

[0033] The compressed image restoration unit 212 restores the compressed image transmitted from the terminal device P1 and acquires hyperspectral data for each wavelength band. The compressed image restoration unit 212 outputs each of the restored hyperspectral data to the object detection unit 213.

[0034] The object detection unit 213 detects a plurality of different objects appearing in each of the hyperspectral data output from the compressed image restoration unit 212. Note that the detected objects may be the same object multiple times. The object detection unit 213 sets a detection frame in the area where each of the detected objects is detected, and generates a cut-out image by cutting out each of the objects from the hyperspectral data according to the set detection frame. The object detection unit 213 outputs the cut-out image (hyperspectral data) generated for each object to the object matching unit 214.

[0035] The object matching unit 214 matches each of the objects appearing in the cutout image output from the object detection unit 213 with the hyperspectral data of each of the objects appearing in the cutout image stored in the registration name database DB2, and determines whether or not hyperspectral data identical or similar to the object appearing in the cutout image is registered. If the object matching unit 214 determines that hyperspectral data identical or similar to the object appearing in the cutout image is registered, it acquires information on the registration name of the object associated with the hyperspectral data, associates the information on the registration name of the object with the cutout image, and outputs it to the processing algorithm selection unit 215. If the object matching unit 214 determines that hyperspectral data identical or similar to the object appearing in the cutout image is not registered, it generates a control command (notification) requesting registration of the registration name of the object, and transmits it to the terminal device P1.

[0036] The processing algorithm selection unit 215 selects at least one algorithm associated with information of the same registered name from among the multiple algorithms registered in the algorithm database DB1, based on information of the registered name of each of the multiple different objects output from the object matching unit 214. The processing algorithm selection unit 215 executes image analysis processing on the cut-out image of the object using the algorithm selected for each object. The processing algorithm selection unit 215 outputs the results of the image analysis processing of each of the multiple different objects to the communication unit 20, and causes it to be transmitted to the terminal device P1.

[0037] The memory 22 has, for example, a RAM as a work memory used when executing each process of the processor 21, and a ROM for storing programs and data that define each operation of the processor 21. The RAM temporarily stores data or information generated or acquired by the processor 21. The ROM has written therein a program that defines the operation of the processor 21.

[0038] The algorithm database DB1 is configured using a storage device such as a hard disk drive (hereinafter, referred to as "HDD") or a solid state drive (hereinafter, referred to as "SSD"). The algorithm database DB1 stores an algorithm used in image analysis processing of hyperspectral data (cut-out image) in association with information on the registered name of the object. The algorithm is an algorithm used in image analysis processing of hyperspectral data (cut-out image). The information associated with each algorithm does not have to be limited to information on the registered name, and may be information that can be collated with information on the registered name registered in the registered name database DB2, and may include, for example, information on the subject that is the subject of image analysis (e.g., the name, model number, ID, etc. of the subject), information on the content of the analysis processing (e.g., sugar content, freshness, etc.), etc.

[0039] The registered name database DB2 is configured using a storage device such as a HDD or SSD, etc. The registered name database DB2 stores hyperspectral data in which an object is captured and the registered name of the object in association with each other.

[0040] The network NW connects the terminal device P1 and the server S1 so as to enable data communication between them.

[0041] The image analysis system 100 according to the first embodiment may polarize the incident light incident on the image sensor of the camera CM using a polarizing plate. For example, the image analysis system 100 may further include a first polarizing plate that polarizes the illumination light of the illumination LT in a first direction, or a second polarizing plate that polarizes the incident light incident on the image sensor of the camera CM in a second direction different from the first direction. This allows the image analysis system 100 to capture an image more suitable for image analysis processing.

[0042] In addition, the configuration of the camera CM, the process of generating compressed images, and the process of restoring compressed images may use known configurations or technologies, for example, configurations or technologies disclosed in Patent Publication No. 2021-110869, Patent Publication No. 2022-000968, Patent Publication No. 2022-058424, Patent Publication No. 2023-082047, etc. may be used.

[0043] Next, an example of an operation procedure of advance preparation will be described with reference to Fig. 2. Fig. 2 is a sequence diagram showing an example of an operation procedure of advance preparation of the image analysis system 100 according to the embodiment 1. Note that the advance preparation shown in Fig. 2 is a process that is performed every time the imaging environment (evaluation environment) of the imaging analysis target is changed, and is not essential and may be omitted.

[0044] The terminal device P1 accepts an input operation of information regarding the camera parameters of the camera CM through the input unit INP (St11). The camera parameters here are the exposure time, focal length, gain, etc. of the camera CM. The terminal device P1 acquires information regarding the camera parameters of the camera CM through the input unit INP, associates the information regarding the camera parameters with a control command requesting a change in the parameters of the camera CM, and transmits the information to the camera CM (St12).

[0045] The camera CM changes the current camera parameters to camera parameters corresponding to the information on the camera parameters transmitted from the terminal device P1 based on the control command transmitted from the terminal device P1 (St13). When the change of the camera parameters is completed, the camera CM generates a camera parameter change completion notification and transmits it to the terminal device P1 (St14).

[0046] The terminal device P1 receives an input operation related to the illumination intensity of the lighting LT through the input unit INP (St15). The terminal device P1 acquires the illumination intensity of the lighting LT through the input unit INP, associates the illumination intensity information with a control command requesting a change in the illumination intensity of the lighting LT, and transmits the information to the lighting LT (St16).

[0047] Based on the control command transmitted from the terminal device P1, the lighting LT changes the current lighting intensity to a lighting intensity corresponding to the lighting intensity information transmitted from the terminal device P1 (St17). When the lighting intensity change is completed, the lighting LT generates a lighting intensity change completion notification and transmits it to the terminal device P1 (St18).

[0048] The terminal device P1 generates a control command requesting the camera CM to capture a darkened image, and transmits the control command to the camera CM (St19). Note that the darkened image referred to here is an image that is a characteristic unique to the camera CM and is used to reduce noise that the camera CM has.

[0049] Based on a control command transmitted from the terminal device P1, the camera CM captures an image in a light-shielded state with the shutter closed (St20), and transmits the captured light-shielded image to the terminal device P1 (St21).

[0050] The terminal device P1 generates a control command requesting the camera CM to capture a whiteboard image, and transmits the control command to the camera CM (St22). Note that the whiteboard image here is an image used to remove the spectrum of the illumination light of the lamp LT.

[0051] The camera CM captures an image of the whiteboard based on the control command transmitted from the terminal device P1 (St23). The camera CM transmits the captured image of the whiteboard to the terminal device P1 (St24).

[0052] The terminal device P1 performs shading correction using the shading image on the acquired whiteboard image (St25). The terminal device P1 transmits the whiteboard image after the shading correction to the server S1 (St26).

[0053] The server S1 records the whiteboard image transmitted from the terminal device P1 in the memory 22 (St27).

[0054] Next, an example of an operation procedure of the image analysis system 100 will be described with reference to Fig. 3. Fig. 3 is a sequence diagram showing an example of an operation procedure of advance registration of the image analysis system 100 according to the first embodiment.

[0055] The terminal device P1 accepts an input operation of information related to the camera parameters of the camera CM through the input unit INP (St31). The terminal device P1 acquires information related to the camera parameters of the camera CM through the input unit INP, associates the information related to the camera parameters with a control command requesting a change in the parameters of the camera CM, and transmits the information to the camera CM to cause the camera CM to change the camera parameters (St32).

[0056] The terminal device P1 receives an input operation related to the illumination intensity of the lighting LT through the input unit INP (St33). The terminal device P1 acquires the illumination intensity of the lighting LT through the input unit INP, associates the illumination intensity information with a control command requesting a change in the illumination intensity of the lighting LT, and transmits the information to the lighting LT to change the illumination intensity of the lighting LT (St34).

[0057] The terminal device P1 generates a control command requesting the camera CM to capture an image of a subject, and transmits the control command to the camera CM (St35).

[0058] The camera CM captures an image of a subject for each wavelength over the wavelength band that the camera CM can capture based on the control command transmitted from the terminal device P1 (St36). The camera CM generates a compressed image in which multiple pieces of wavelength information are compressed, and transmits the compressed image to the terminal device P1 (St37).

[0059] The terminal device P1 performs shading correction based on a shading image captured in advance on the compressed image transmitted from the camera CM (St38). Note that the process of step St38 may be performed only when a shading image is captured in advance, and is not essential and may be omitted.

[0060] The terminal device P1 accepts an input of a registered name of an object appearing in a captured compressed image based on a user operation (St39). The terminal device P1 acquires information on the registered name transmitted from the input unit INP, associates the acquired registered name with the compressed image, and transmits the associated information to the server S1 (St40).

[0061] The server S1 restores the compressed image transmitted from the terminal device P1 and acquires hyperspectral data (St41). The server S1 performs whiteboard correction using a previously recorded whiteboard image for each of the acquired images (St42). Note that the process of step St43 may be executed only when a whiteboard image has been captured in advance, and is not essential and may be omitted.

[0062] The server S1 detects an object from the hyperspectral data (St43). The server S1 associates the hyperspectral data with the registered name and stores them in the registered name database DB2 (St44), and transmits the registration result to the terminal device P1 (St45).

[0063] The terminal device P1 outputs and displays the registration result transmitted from the server S1 on the monitor MN (St46).

[0064] As described above, the image analysis system 100 according to the first embodiment enables image analysis processing of multiple different objects appearing in a single compressed image by pre-registering the hyperspectral data in which each of the multiple different objects appears and the registered names of each of the multiple different objects.

[0065] Next, an operation procedure of the image analysis system 100 will be described with reference to Fig. 4. Fig. 4 is a sequence diagram showing a first operation procedure example of the image analysis system 100 according to the first embodiment. Here, in the first operation procedure shown in Fig. 4, an example will be described in which the registered names of all objects to be image analyzed are registered in advance in the registered name database DB2.

[0066] The process from step St31 to step St38 shown in FIG. 4 is similar to the process from step St31 to step St38 shown in FIG. 3, and therefore the description thereof will be omitted.

[0067] The terminal device P1 transmits the captured compressed image to the server S1 (St50).

[0068] The server S1 restores the compressed image transmitted from the terminal device P1 to obtain hyperspectral data (St51). The server S1 performs whiteboard correction using a previously recorded whiteboard image for each of the obtained images (St52). Note that the process of step St43 may be performed only when a whiteboard image has been captured in advance, and is not essential and may be omitted.

[0069] The server S1 detects a plurality of different objects from the restored hyperspectral data. The server S1 generates a cut-out image by cutting out each object from each of the hyperspectral data along the detection frame of the detected object. The server S1 compares the cut-out image for each object with the hyperspectral data registered in the registration name database DB2, and when it is determined that there is an object that is the same as or similar to the object shown in the cut-out image, it identifies the registration name of the object shown in the cut-out image based on the registration name of the object associated with the hyperspectral data of the object (St53A). As a result, the server S1 determines whether or not the processing algorithm used for the image analysis processing of the object can be identified.

[0070] In addition, when the server S1 detects two objects "A" and three objects "B," it generates two cut-out images each showing an object "A" and three cut-out images each showing an object "B." This allows the server S1 to perform image analysis processing not only on different types of objects but also on the same type of objects individually.

[0071] The server S1 refers to the algorithm database DB1 and selects an algorithm associated with the same registered name as the registered name of the identified object as a processing algorithm to be used for image analysis processing of the cut-out image of each object. The server S1 executes image analysis processing of the cut-out image of the corresponding object by each selected processing algorithm (St54). The server S1 transmits the image analysis processing result to the terminal device P1 (St55).

[0072] The terminal device P1 generates image data SC1, SC2 (see FIG. 6) showing the image analysis processing result based on the image analysis result transmitted from the server S1 (St56). The terminal device P1 generates an image analysis processing result screen (not shown) including the generated image data SC1, SC2, and outputs it to the monitor MN for display (St57).

[0073] As described above, the image analysis system 100 according to the first embodiment can perform image analysis on each of different types of objects captured in a compressed image, and obtain the image analysis processing results for each object.

[0074] Next, an operation procedure of the image analysis system 100 will be described with reference to Fig. 5. Fig. 5 is a sequence diagram showing a second operation procedure example of the image analysis system 100 according to the first embodiment. Here, in the second operation procedure shown in Fig. 5, an example will be described in which the registered name of at least one object among the registered names of all objects that are the targets of image analysis is not registered in the registered name database DB2.

[0075] Note that the process of steps St31 to St38 shown in Fig. 5 is similar to the process of steps St31 to St38 shown in Fig. 3, and therefore the description thereof will be omitted. Also, the process of steps St50 to St52 shown in Fig. 5 is similar to the process of steps St50 to St52 shown in Fig. 4, and therefore the description thereof will be omitted.

[0076] The server S1 compares the cut-out image for each object with the hyperspectral data registered in the registration name database DB2 (St53B). If the server S1 determines that there is no object that is identical or similar to the object shown in the cut-out image, the server S1 generates a control command requesting the setting of a registration name for the object whose registration name is not specified, and transmits the generated control command to the terminal device P1 in association with the cut-out image whose registration name is not specified (St60).

[0077] The terminal device P1 outputs the cut-out image transmitted from the server S1 to the monitor MN and requests the user to input the registered name of the object appearing in the cut-out image (St61). The terminal device P1 accepts the user's input of the registered name of the object (St62). The terminal device P1 transmits information on the entered registered name to the server S1 (St63).

[0078] The server S1 associates the hyperspectral data of the object with the registered name of the object and registers it in the registered name database DB2 (St64). Note that the hyperspectral data registered here may be a cut-out image.

[0079] The server S1 refers to the algorithm database DB1 and selects an algorithm associated with the same registered name as the registered name of the object transmitted from the terminal device P1 as a processing algorithm to be used for image analysis processing of the cut-out image of each object (St65). The server S1 executes image analysis processing of the cut-out image of the corresponding object by each selected processing algorithm (St66), and transmits the image analysis processing result to the terminal device P1 (St67).

[0080] The terminal device P1 generates image data SC1, SC2 (see FIG. 6) showing the image analysis processing result based on the image analysis result transmitted from the server S1 (St68). The terminal device P1 generates an image analysis processing result screen (not shown) including the generated image data SC1, SC2, and outputs it to the monitor MN for display (St69).

[0081] As described above, the image analysis system 100 according to the first embodiment can perform image analysis on each of different types of objects captured in a compressed image, and obtain the image analysis processing results for each object.

[0082] Next, an example of image analysis processing performed by the image analysis system 100 will be described with reference to Fig. 6. Fig. 6 is a diagram for explaining the image analysis processing example. The image analysis processing examples shown in Cases (1) to (5) in Fig. 6 will explain an example of image analysis processing of one captured image in which an apple and a banana, which are different objects, are captured.

[0083] Server S1 restores the compressed image CIM and detects one apple and one banana from the restored hyperspectral data. Server S1 determines whether the registered names of the detected objects, "apple" and "banana", are registered in a registered name database DB2. Server S1 generates an apple detection frame in an area where one apple is detected and a banana detection frame in an area where one banana is detected, and cuts out these detection frames to generate a cut-out image of the apple (not shown) and a cut-out image of the banana (not shown).

[0084] In cases No. (1) and (2), the server S1 performs image analysis processing on the cut-out image of the apple using each of the processing algorithms “Estimation of Apple Freshness” and “Determination of Apple Sugar Content” associated with information on the registered name “Apple” among the processing algorithms registered in the algorithm database DB1, and transmits information on the results of the image analysis processing by each processing algorithm to the terminal device P1.

[0085] Based on the information on the image analysis processing results transmitted from the server S1, the terminal device P1 outputs and displays on the monitor MN image data SC1 showing the image analysis processing results regarding the freshness of the apples in a heat map, supplementary information regarding the image analysis processing results of the image data SC1 (good: ●, bad: ◯), and image data SC2 showing the image analysis processing results regarding the sugar content of the apples in a heat map.

[0086] In cases (3), (4), and (5), the server S1 performs image analysis on the cut-out image of the banana using each of the processing algorithms “estimating banana freshness,” “determining sugar content of banana,” and “determining color of banana,” which are associated with information on the registered name “banana,” among the processing algorithms registered in the algorithm database DB1, and transmits information on the results of the image analysis processing by each processing algorithm to the terminal device P1.

[0087] Based on the information on the image analysis processing results transmitted from the server S1, the terminal device P1 outputs and displays on the monitor MN image data (not shown) showing the image analysis processing results regarding the freshness of the bananas in a heat map, image data (not shown) showing the image analysis processing results regarding the sugar content of the bananas in a heat map, and image data (not shown) showing the image analysis processing results regarding the color of the bananas in a heat map.

[0088] As described above, the image analysis system 100 according to embodiment 1 can perform multiple image analysis processes on a single object based on processing algorithms associated with the registered names of different types of objects appearing in a compressed image, and obtain the image analysis processing results.

[0089] In the image analysis system 100 according to the first embodiment, the image analysis process is executed by the server S1, but the image analysis system 100 is not limited to this. The image analysis system 100 may be configured by one computer, and this one computer may be capable of implementing the functions implemented by the terminal device P1 and the server S1 described above.

[0090] (Additional Note) The above description of each embodiment discloses the following techniques.

[0091] (Technology 1) An image analysis method performed by an image analysis device (server S1) which is a computer, Obtaining and restoring a compressed image in which multiple wavelength information pieces obtained by capturing images of multiple different subjects (examples of objects) are compressed; Detecting the plurality of different objects appearing in the restored image (hyperspectral data); Based on a comparison between the image (hyperspectral data) and a registered image (specifically, hyperspectral data registered in a registered name database DB2) including predetermined wavelength information previously registered for each subject, information about the subject (e.g., a registered name) is individually identified; Select an algorithm to be used for image analysis of each of the subjects based on the information on the identified subjects, perform image analysis of the image (hyperspectral data) using the selected processing algorithm, and output the image analysis result. Image analysis methods. With this configuration, the image analysis device (server S1) can identify the registered names (information about the subject) of different objects, and can identify the processing algorithms used in the image analysis process of each different object in the image analysis process using the hyperspectral data. Therefore, even if different objects are captured in the hyperspectral data, the image analysis device (server S1) can perform image analysis on each of the objects.

[0092] (Technology 2) If it is determined that information about the subject has not been specified, a notification requesting information about the subject is output, the information about the subject is acquired, an algorithm to be used for image analysis of the subject is selected based on the acquired information about the subject, image analysis of the image (hyperspectral data) is performed using the selected processing algorithm, and an image analysis result is output. The image analysis method described in (Technical 1). With this configuration, the image analysis device (server S1) can automate subsequent image analysis processing by registering the registered names of objects that cannot be identified in image analysis processing using hyperspectral data, and can perform image analysis of each of the objects even if multiple different objects are captured in the hyperspectral data.

[0093] (Technology 3) generating a cut-out image for each of the objects by cutting out an area in which the object is detected from the image; performing image analysis on the extracted image of the subject using the processing algorithm; An image analysis method according to (Technology 1) or (Technology 2). With this configuration, the image analysis device (server S1) can reduce the processing time for image analysis processing, and therefore can more efficiently execute image analysis processing for each object even if multiple objects are captured in the hyperspectral data. Also, the image analysis device (server S1) can generate and output individual image analysis results for objects of the same type even if multiple objects of the same type are captured in the hyperspectral data.

[0094] (Technology 4) generating a cut-out image for each of the objects by cutting out an area in which the object is detected from the image; registering the cut-out image of the subject as a registered image in association with information about the subject; An image analysis method according to any one of (Technology 1) or (Technology 3). With this configuration, the image analysis device (server S1) can accumulate cut-out images in which the area where the object was detected is partially cut out. Even if the hyperspectral data does not capture the entire object, i.e., only a portion of the object, the accuracy of matching the object can be improved, and image analysis processing of the object can be performed with higher accuracy.

[0095] (Technology 5) An acquisition unit (communication unit 20) that acquires a compressed image in which multiple pieces of wavelength information obtained by capturing images of multiple different subjects (examples of objects) are compressed; A detection unit (processor 21) that restores the compressed image and detects the plurality of different subjects appearing in the restored image (hyperspectral data); and an image analysis unit (processor 21) which individually identifies information about the subjects (e.g., a registered name) based on a comparison between the image (hyperspectral data) and a registered image (specifically, hyperspectral data registered in a registered name database DB2) containing predetermined wavelength information registered in advance for each subject, selects an algorithm to be used for image analysis of each of the subjects based on the identified information about the subjects, performs image analysis of the image (hyperspectral data) using the selected processing algorithm, and outputs the image analysis results. Image analysis device (server S1). With this configuration, the image analysis device (server S1) can identify the registered names (information about the subject) of different objects, and can identify the processing algorithms used in the image analysis process of each different object in the image analysis process using the hyperspectral data. Therefore, even if different objects are captured in the hyperspectral data, the image analysis device (server S1) can perform image analysis on each of the objects.

[0096] (Technology 6) A terminal device P1, An image analysis system 100 including an image analysis device (server S1) capable of communicating with the terminal device P1, The terminal device P1 A compressed image in which a plurality of wavelength information pieces obtained by capturing images of a plurality of different subjects (examples of objects) are compressed is transmitted to the image analysis device (server S1); The image analysis device (server S1) Decompressing the compressed image and detecting the different objects appearing in the decompressed image (hyperspectral data); Based on a comparison between the image (hyperspectral data) and a registered image (specifically, hyperspectral data registered in a registered name database DB2) including predetermined wavelength information previously registered for each subject, information about the subject (e.g., a registered name) is individually identified; selecting an algorithm to be used for image analysis of each of the different subjects based on the information on the identified subjects, performing image analysis of the images using the selected processing algorithm, and outputting a result of the image analysis; Image analysis system 100. With this configuration, the image analysis system 100 can identify the registered names (information about the subject) of different objects, and can identify the processing algorithms used in the image analysis processing of the different objects in the image analysis processing using the hyperspectral data. Therefore, even if different objects are captured in the hyperspectral data, the image analysis system 100 can perform image analysis of each of the objects.

[0097] Although various embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, corrections, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also belong to the technical scope of the present disclosure. In addition, the components in the various embodiments described above may be arbitrarily combined within the scope of the invention. [Industrial Applicability]

[0098] The present disclosure is useful as providing an image analysis method, an image analysis device, and an image analysis system for performing image analysis of a plurality of different objects appearing in a compressed image. [Explanation of symbols]

[0099] 10,20 Communications Department 11,21 Processor 12,22 Memory 100 Image Analysis System 111 UI section 211 Object Registration Unit 212 Compressed image restoration unit 213 Object detection unit 214 Object Matching Unit 215 Processing algorithm selection unit Commercial Camera DB1 Algorithm Database DB2 Registered Name Database P1 terminal equipment S1 Server SC1, SC2 image data

Claims

1. An image analysis method performed by an image analysis device that is a computer, comprising: Obtaining and restoring a compressed image in which multiple wavelength information pieces obtained by capturing images of multiple different subjects are compressed; Detecting the plurality of different objects appearing in the restored image; Identifying information about each of the subjects individually based on matching the image with a registered image including predetermined wavelength information registered in advance for each subject; selecting an algorithm to be used for image analysis of each of the subjects based on the information on the identified subjects, performing image analysis of the image using the selected processing algorithm, and outputting the image analysis result; Image analysis methods.

2. If it is determined that information regarding the subject has not been specified, a notification requesting information regarding the subject is output, the information regarding the subject is acquired, an algorithm to be used for image analysis of the subject is selected based on the acquired information regarding the subject, image analysis of the image is performed using the selected processing algorithm, and a result of the image analysis is output. The image analysis method according to claim 1 .

3. generating a cut-out image for each of the objects by cutting out an area in which the object is detected from the image; performing image analysis on the extracted image of the subject using the processing algorithm; The image analysis method according to claim 1 .

4. generating a cut-out image for each of the objects by cutting out an area in which the object is detected from the image; registering the cut-out image of the subject as a registered image in association with information about the subject; The image analysis method according to claim 1 .

5. an acquisition unit that acquires a compressed image in which a plurality of wavelength information pieces obtained by capturing images of a plurality of different subjects are compressed; a detection unit that restores the compressed image and detects the plurality of different subjects appearing in the restored image; and an image analysis unit that individually identifies information about each of the subjects based on a comparison between the image and a registered image including predetermined wavelength information registered in advance for each subject, selects an algorithm to be used for image analysis of each of the subjects based on the identified information about the subjects, performs image analysis of the image using the selected processing algorithm, and outputs a result of the image analysis. Image analysis device.

6. A terminal device; An image analysis system comprising: an image analysis device capable of communicating with the terminal device, The terminal device A compressed image in which a plurality of wavelength information pieces obtained by capturing images of a plurality of different subjects are compressed is transmitted to the image analysis device; The image analysis device includes: restoring the compressed image to detect the different objects appearing in the restored image; Identifying information about each of the subjects individually based on matching the image with a registered image including predetermined wavelength information registered in advance for each subject; selecting an algorithm to be used for image analysis of each of the different subjects based on the information on the identified subjects, performing image analysis of the images using the selected processing algorithm, and outputting a result of the image analysis; Image analysis system.

Images