Information processing system, information processing method, and program

Abstract

To develop technology related to an estimation model for detecting abnormalities.SOLUTION: The information processing system according to the present invention has storage means, learning means, acquisition means, and reception means. The storage means stores a first image showing an object in an abnormal state and a second image showing an object in a normal state. The learning means generates multiple first estimation models through machine learning using the first image and the second image. The acquisition means acquires a classification determined by using at least one of the first estimation models and determined for images showing an object among a plurality of classifications including both normal and abnormal states. The reception means receives the results of the user's correct or incorrect input for the classification determined for each image. The learning means further determines the normal state or the abnormal state for the image through machine learning using the images classified as the normal state or the abnormal state by the correct or incorrect input together with the first image and the second image.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a learning device, an estimation device, a learning method, and a program.

Background Art

[0002] Patent Document 1 discloses a technique for generating an estimation model that classifies an input image into a good image or a bad image by learning based on correct and incorrect teacher images. A good image is an image with a high degree of similarity to the correct teacher image, and a bad image is an image with a low degree of similarity to the correct teacher image. Patent Document 2 discloses a technique for defining abnormal behavior using a teacher image showing abnormal behavior and generating an estimation model for detecting the defined abnormal behavior.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the technique of generating an estimation model for detecting an abnormality, a technique for efficiently collecting teacher images is desired. Patent Document 1 does not disclose the problem and the solution means. In the case of the technique described in Patent Document 2, it is necessary to collect a large number of teacher images showing abnormal behavior. However, it is not easy to collect teacher images showing "abnormality". An object of the present invention is to provide a technique for efficiently collecting teacher images for generating an estimation model for detecting an abnormality.

Means for Solving the Problems

[0005] According to the present invention, an acquisition means for acquiring an image, Similarity calculation means for calculating the similarity between the acquired image and a first image indicating an abnormal state stored in advance; Registration means for registering the acquired image whose similarity is equal to or less than a first reference value as a second image indicating a normal state; Learning means for generating an estimation model for discriminating normal / abnormal by machine learning using the first image and the second image; There is provided a learning device having:

[0006] Also, according to the present invention, a computer acquires an image, calculates the similarity between the acquired image and a first image indicating an abnormal state stored in advance, registers the acquired image whose similarity is equal to or less than a first reference value as a second image indicating a normal state, and provides a learning method for generating an estimation model for discriminating normal / abnormal by machine learning using the first image and the second image.

[0007] Also, according to the present invention, a computer is caused to function as acquisition means for acquiring an image, similarity calculation means for calculating the similarity between the acquired image and a first image indicating an abnormal state stored in advance, registration means for registering the acquired image whose similarity is equal to or less than a first reference value as a second image indicating a normal state, learning means for generating an estimation model for discriminating normal / abnormal by machine learning using the first image and the second image, and there is provided a program for causing the computer to function as such.

[0008] Also, according to the present invention, there is provided an estimation device for discriminating normal / abnormal using the estimation model generated by the learning device.

Effects of the Invention

[0009] According to the present invention, it becomes possible to efficiently collect teacher images for generating an estimation model for detecting anomalies.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate.

[0012] <First Embodiment> The learning device of the present embodiment (hereinafter, may be simply referred to as "learning device") generates an estimation model for determining whether the state indicated by the input image is normal or abnormal.

[0013] The object of normal / anomaly discrimination is, for example, a location (such as a park, a station, a facility, etc.). The normal state observed most of the time is determined to be normal, and a state different from the normal state is determined to be abnormal. For example, a state where there is a person performing abnormal behavior, or a state where an object that always exists at that location has malfunctioned or moved, etc. is determined to be abnormal. Abnormal behavior is behavior different from the behavior performed by most people observed in the image. Note that the object of discrimination may be other facilities such as factories, stores, facilities, offices, etc., or may be other things. In any case, the normal state observed most of the time is determined to be normal, and a state different from the normal state is determined to be abnormal.

[0014] The learning device generates the above-mentioned estimation model by repeatedly executing the cycle shown in FIG. 1. As shown in FIG. 1, the learning device repeatedly executes the first image registration process S1, the image selection process S2, the learning process S3, the estimation process S4, the user confirmation process S5, and the second image registration process S6 in this order. Note that the order of the processes may be changed within the range where the same operational effects are achieved.

[0015] FIG. 2 shows an example of a functional block diagram of the learning device 10. As shown in the figure, the learning device 10 includes an acquisition unit 11, a similarity calculation unit 12, a registration unit 13, a learning unit 14, an estimation unit 15 during learning, a user confirmation unit 16, an image storage unit 17, and an estimation model storage unit 18. Each process shown in FIG. 1 is executed by these functional units.

[0016] FIG. 3 is a diagram showing the cycle of FIG. 1 in more detail. Using this figure, each process shown in FIG. 1 and the processes of each functional unit shown in FIG. 2 will be described.

[0017] "The first image registration process S1" The first image registration process S1 is a process of classifying and registering the image generated by the camera based on the similarity between the image generated by the camera and the image showing the abnormal state registered in advance.

[0018] The first to third image group databases DB17-1 to 17-3, camera D14, similarity calculation S10, and registration S11 in FIG. 3 are related to the process. And the acquisition unit 11, similarity calculation unit 12, registration unit 13, and image storage unit 17 in FIG. 2 are related to the process. The first to third image group databases DB17-1 to 17-3 are realized by the image storage unit 17 in FIG. 2.

[0019] First, as a pre-preparation for the process, labeled images with labels indicating abnormal states are stored in the first image group database (database) 17-1. The user prepares several images indicating abnormal states in advance, assigns labels for the abnormal states, and stores them in the first image group database 17-1. The images in the first image group database 17-1 accumulated in this way are highly reliable labeled images that have been confirmed by the user to indicate abnormal states. Note that the number of images initially stored in the first image group database 17-1 may be about several tens to several hundreds, and a large number of images are not necessary. With this number, the user burden required for collecting labeled images is not large. When defining an abnormal state in advance and generating an estimation model for detecting the abnormal state, generally, it is necessary to prepare several thousand to several tens of thousands or more teacher images indicating the abnormal state. The first image group database 17-1 corresponds to the image storage unit 17 in FIG. 2. Hereinafter, the images indicating abnormal states stored in the first image group database 17-1 are referred to as "first images".

[0020] The acquisition unit 11 acquires the images generated by the camera D14. The camera D14 may be a camera (such as a surveillance camera) that captures an object to be discriminated between normal / abnormal, or a camera that captures an object of the same type as the object to be discriminated. The camera D14 may capture a moving image, or may continuously capture still images at a frame interval longer than that of a moving image. In the figure, one camera D14 is shown, but a plurality of cameras D14 may be used.

[0021] The acquisition unit 11 may acquire the image generated by the camera D14 in real-time processing. In this case, the learning device 10 and the camera D14 are configured to be communicable with each other. Alternatively, the acquisition unit 11 may acquire the image generated by the camera D14 in batch processing. In this case, the image generated by the camera D14 is stored in the storage device of the camera D14 or any other arbitrary storage device, and the acquisition unit 11 acquires the stored image at an arbitrary timing.

[0022] In this specification, "acquisition" means, based on a user input or based on an instruction of a program, "the self-device goes to retrieve data stored in another device or storage medium (active acquisition)", for example, making a request or inquiry to another device and receiving it, accessing another device or storage medium and reading it out, etc., and, based on a user input or based on an instruction of a program, "inputting data output from another device to the self-device (passive acquisition)", for example, receiving data distributed (or transmitted, push-notified, etc.), and also, selecting and acquiring from the received data or information, and "generating new data by editing (textualizing, rearranging data, extracting some data, changing file format, etc.) the data and acquiring the new data", including at least any one of them.

[0023] The similarity calculation unit 12 calculates the similarity between the image acquired by the acquisition unit 11 (hereinafter referred to as "acquired image") and the first image indicating an abnormal state previously stored in the first image group DB17-1 (S10 in FIG. 3). The similarity calculation unit 12 may calculate the similarity between each of the plurality of first images stored in the first image group DB17-1 and each acquired image. Alternatively, the similarity calculation unit 12 may calculate the similarity between one image (e.g., average image) generated based on the plurality of first images stored in the first image group DB17-1 and each acquired image.

[0024] In addition, various methods have been proposed for calculating the similarity between images. In this embodiment, any method can be adopted. For example, the similarity calculation unit 12 may detect an object from within an image and calculate the similarity of the detection result (similarity of the number of detected objects, similarity of the appearance of the detected objects, etc.). Further, the similarity calculation unit 12 may input each image into an estimation model that performs image analysis generated by deep learning, and calculate the similarity of the obtained image analysis results (recognition results of the objects shown in the image, recognition results of the scenes shown in the image, etc.). Further, the similarity calculation unit 12 may calculate the similarity of colors or luminances appearing in the whole or local parts of the image.

[0025] The registration unit 13 registers the acquired image whose similarity is equal to or less than the first reference value as a second image (image with a normal state label) indicating a normal state in the second image group DB (database) 17-2 (S11). When the similarity calculation unit 12 calculates the similarity between each of the plurality of first images stored in the first image group DB17-1 and each acquired image, the registration unit 13 registers the acquired image whose similarity with all of the plurality of first images is equal to or less than the first reference value as the second image in the second image group DB17-2.

[0026] Further, the registration unit 13 registers the acquired image whose similarity is equal to or greater than the second reference value as a third image (image with an abnormal state label) indicating an abnormal state in the third image group DB (database) 17-3 (S11). When the similarity calculation unit 12 calculates the similarity between each of the plurality of first images stored in the first image group DB17-1 and each acquired image, the registration unit 13 registers the acquired image whose similarity with at least one of the plurality of first images is equal to or greater than the second reference value as the third image in the third image group DB17-3.

[0027] In the third image group DB17-3, images determined to be similar to the first image by a computer to a predetermined level or more in this way are registered as images indicating an abnormal state. In this regard, it is different from the first image group DB17-1 in which highly reliable first images confirmed by the user to indicate an abnormal state are stored.

[0028] The first reference value and the second reference value may be the same value or different values. However, by setting the first reference value and the second reference value to different values, making the first reference value a sufficiently small value and the second reference value a sufficiently large value, it is possible to suppress the inconvenience of registering an acquired image that exists in a gray zone (similarity is greater than the first reference value and less than the second reference value) and has neither a high nor a low similarity to the first image as the second image or the third image.

[0029] "Image Selection Process S2, Learning Process S3" The image selection process S2 is a process of selecting an image to be a teacher image from among the images stored in the first to third image groups DB17-1 to 17-3. The learning process S3 is a process of performing learning for each of the plurality of estimation models registered in the estimation model DB (database) 18-1 using the selected image as a teacher image.

[0030] The first to third image groups DB17-1 to 17-3, the estimation model DB18-1, the selection S12, and the learning S13 in FIG. 3 are related to the process. And the learning unit 14, the image storage unit 17, and the estimation model storage unit 18 in FIG. 2 are related to the process. The estimation model DB18-1 is realized by the estimation model storage unit 18 in FIG. 2.

[0031] First, information on a plurality of estimation models is stored in the estimation model DB18-1. All of the plurality of estimation models are models for discriminating whether the state indicated by the input image is normal or abnormal. The learning and estimation algorithms of the plurality of estimation models are different from each other. For example, the plurality of estimation models are generated by deep learning. In the present embodiment, for example, information on a plurality of estimation models learned and generated by a neural network, a Bayesian network, regression analysis, a support vector machine (SVM), a decision tree, a genetic algorithm, a nearest neighbor method classification, etc. is stored in the estimation model DB18-1.

[0032] The learning unit 14 selects at least a part from the images registered in the first to third image group databases 17-1 to 17-3 (S12 in FIG. 3), and generates an estimation model by machine learning using the selected images (S13 in FIG. 3).

[0033] There are various selection methods. For example, the learning unit 14 may randomly select a predetermined number of images from the entire first to third image group databases 17-1 to 17-3. Alternatively, the learning unit 14 may randomly select a first predetermined number of images from the first image group database 17-1, randomly select a second predetermined number of images from the second image group database 17-2, and randomly select a third predetermined number of images from the third image group database 17-3. The first to third predetermined numbers may be the same or different. That is, the ratio of the number of images selected from each of the first to third image group databases 17-1 to 17-3 (the ratio to the total number of selected images) may be the same or different.

[0034] Also, the learning unit 14 may select images for each estimation model. In this case, the first to third predetermined numbers and the above ratio may be different for each estimation model.

[0035] After selecting the images, the learning unit 14 uses the selected first to third images as teacher images and performs learning for each of the plurality of estimation models registered in the estimation model database (database) 18-1. That is, the learning unit 14 generates an estimation model for discriminating normal / abnormal by machine learning (including the concept of deep learning) using the first to third images.

[0036] "Estimation process S4" The estimation process S4 is a process of inputting the acquired image into each of the plurality of estimation models registered in the estimation model database (database) 18-1 and discriminating the state indicated by the acquired image.

[0037] The estimation model DB18-1, camera D14, and estimation S14 in FIG. 3 are related to the said process. And the acquisition unit 11, the estimation unit 15 during learning, and the estimation model storage unit 18 in FIG. 2 are related to the said process.

[0038] The estimation unit 15 during learning inputs the acquired image into each of the plurality of estimation models stored in the estimation model storage unit 18, and discriminates the state (normal / abnormal) indicated by the acquired image. Note that the acquired image input to the estimation model in the said process is an acquired image not used at that time for the generation (learning) of the estimation model. For example, the estimation unit 15 during learning can perform the said discrimination using the acquired image before being stored in the image storage unit 17.

[0039] Note that the discrimination results for each of the plurality of estimation models may be accumulated in the storage device within the learning device 10.

[0040] "User confirmation process S5" The user confirmation process S5 is a process of outputting the discrimination result of the estimation process S4 to the user and receiving the correct / incorrect input of the discrimination result from the user.

[0041] The display device D15, extraction S15, output S16, and correct / incorrect input S17 in FIG. 3 are related to the said process. And the user confirmation unit 16 in FIG. 2 is related to the said process.

[0042] The user confirmation unit 16 outputs the discrimination result by the estimation unit 15 during learning to the user (S16 in FIG. 3) and receives the correct / incorrect input of the discrimination result from the user (S17 in FIG. 3). For example, the user confirmation unit 16 outputs the acquired image and the discrimination result (normal state or abnormal state) and receives the correct / incorrect input of the discrimination result for the acquired image.

[0043] Executing the said process for all acquired images will increase the burden on the user. Therefore, the user confirmation unit 16 may extract some acquired images that satisfy a predetermined condition (S15 in FIG. 3), and perform only the output of the discrimination result (S16 in FIG. 3) and the reception of the correct / incorrect input (S17 in FIG. 3) for the extracted some acquired images.

[0044] Some of the acquired images for which the discrimination result is output and the correct / incorrect input is received may be, for example, any of the following.

[0045] · An acquired image determined to indicate an abnormal state in at least one estimation model. · An acquired image determined to indicate an abnormal state with a confidence level equal to or higher than a predetermined level in at least one estimation model. · An acquired image determined to indicate an abnormal state in a predetermined number or more of estimation models. · An acquired image determined to indicate an abnormal state with a confidence level equal to or higher than a predetermined level in a predetermined number or more of estimation models. · An acquired image determined to indicate an abnormal state in all estimation models. · An acquired image determined to indicate an abnormal state with a confidence level equal to or higher than a predetermined level in all estimation models.

[0046] Some of the acquired images for which the discrimination result is output and the correct / incorrect input is received may, in addition to any of the acquired images described above, include acquired images randomly picked from among the acquired images that do not satisfy the above conditions (acquired images presumed to indicate a normal state).

[0047] The user confirmation unit 16 may output the discrimination result via an arbitrary output device such as a display or a projection device, and may receive the correct / incorrect input via an arbitrary input device such as a keyboard, a mouse, a touch panel, a physical button, or a microphone. Alternatively, the user confirmation unit 16 may transmit the discrimination result to a predetermined mobile terminal and acquire the content of the correct / incorrect input made to the mobile terminal from the mobile terminal. Alternatively, the user confirmation unit 16 may store the discrimination result in a browsable state on an arbitrary server from an arbitrary device. Then, the user confirmation unit 16 may acquire the content of the correct / incorrect input input from an arbitrary device and stored in the above server. Note that the examples illustrated here are merely examples and are not limited thereto.

[0048] "Second Image Registration Process S6" The second image registration process S6 is a process of registering, as a first image, an acquired image in which an abnormal state is indicated in the user confirmation process S5 in the first image group DB17-1.

[0049] The first image group DB17-1 and the registration S18 in FIG. 3 are related to this process. And the registration unit 13 and the image storage unit 17 in FIG. 2 are related to this process.

[0050] The registration unit 13 registers, as a first image, an acquired image in which an abnormal state is indicated in the correct / incorrect input received by the user confirmation unit 16 in the first image group DB17-1.

[0051] The acquired images in which an abnormal state is indicated include acquired images in which the discrimination result is "abnormal state" and the correct / incorrect input is "correct", and acquired images in which the discrimination result is "normal state" and the correct / incorrect input is "incorrect", etc.

[0052] Here, a modified example of the learning device 10 of the present embodiment will be described. The learning device 10 may not have the third image group DB17-3. And the registration unit 13 may not execute a process of registering, as a second image, an acquired image whose similarity to the first image is equal to or less than a first reference value in the second image group DB17-2, and may not execute a process of registering, as a third image, an acquired image whose similarity to the first image is equal to or greater than a second reference value in the third image group DB17-3. In this case, images indicating a normal state will be accumulated by the process of the registration unit 13.

[0053] Next, an example of the hardware configuration of the learning device 10 will be described. Each functional unit of the learning device 10 is realized by an arbitrary combination of hardware and software centered around the CPU (Central Processing Unit) of a computer, memory, a program loaded into the memory, a storage unit such as a hard disk for storing the program (in addition to a program stored in advance at the stage of shipping the device, a program downloaded from a storage medium such as a CD (Compact Disc) or a server on the Internet can also be stored), and a network connection interface. And it is understood by those skilled in the art that there are various modifications to the realization method and device.

[0054] FIG. 4 is a block diagram illustrating the hardware configuration of the learning device 10. As shown in FIG. 4, the learning device 10 includes a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The learning device 10 may not have the peripheral circuit 4A. Note that the learning device 10 may be composed of a plurality of physically and / or logically separated devices, or may be composed of a single physically and / or logically integrated device. When the learning device 10 is composed of a plurality of physically and / or logically separated devices, each of the plurality of devices can have the above hardware configuration.

[0055] Bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to transmit and receive data from each other. The processor 1A is an arithmetic processing device such as a CPU or a GPU (Graphics Processing Unit). The memory 2A is a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The input / output interface 3A includes an interface for acquiring information from an input device, an external device, an external server, an external sensor, a camera, etc., and an interface for outputting information to an output device, an external device, an external server, etc. The input device is, for example, a keyboard, a mouse, a microphone, a physical button, a touch panel, etc. The output device is, for example, a display, a speaker, a printer, a mailer, etc. The processor 1A can issue commands to each module and perform operations based on their operation results.

[0056] Next, the operation and effect of the learning device 10 will be described.

[0057] The learning device 10 of this embodiment generates an estimation model for discriminating normal / abnormal by machine learning using an image indicating a normal state and an image indicating an abnormal state as teacher images. In the estimation model, the normal state observed most of the time is discriminated as normal, and a state different from the normal state is discriminated as abnormal.

[0058] In such a case, even if an abnormal state that has not been defined in advance occurs, as long as the state is different from the normal state, it can be discriminated as an abnormal state. Therefore, it is possible to detect abnormal states without omission.

[0059] Also, when defining an abnormal state in advance and generating an estimation model for detecting the abnormal state, it is necessary to prepare a large number of teacher images indicating each abnormal state. However, it is not easy to prepare teacher images indicating abnormal states. In the case of this embodiment, compared with the case of generating an estimation model for detecting an abnormal state defined in advance, the number of "images indicating abnormal states" to be prepared is reduced. As a result, the burden on the user is reduced.

[0060] In the case of this embodiment, a large number of "images indicating a normal state" are required. However, usually, many objects are in the "normal state", so it is possible to easily collect "images indicating a normal state" from the images of such objects being photographed.

[0061] Also, in the case of this embodiment, based on the result of similarity determination between a small number of "images indicating an abnormal state" prepared in advance (meaning a number smaller than the number of images indicating an abnormal state required to generate an estimation model for detecting a predefined abnormal state) and the images generated by a surveillance camera or the like, "images indicating a normal state" can be automatically accumulated. For this reason, the burden on the user is reduced.

[0062] Also, in the case of this embodiment, the second image registration process S6 can increase the "images indicating an abnormal state". Since the "images indicating an abnormal state" can be increased in this way, the estimation accuracy of the obtained estimation model is improved.

[0063] Also, in the case of this embodiment, the first image registration process S1 can also increase the "images indicating an abnormal state". In this case, by setting the above-described second reference value to a sufficiently high value, more reliable "images indicating an abnormal state" can be increased. And with the increase in the "images indicating an abnormal state", an improvement in the estimation accuracy of the obtained estimation model is expected.

[0064] Also, in the case of this embodiment, the images indicating an abnormal state can be divided and managed into "highly reliable first images confirmed by the user to indicate an abnormal state" and "third images determined by the computer to be similar to the first images at a predetermined level or higher". And only the first images can be the reference targets in the similarity calculation S10 in FIG. 3. In this way, by using only the highly reliable first images as the reference targets, the reliability of the process of classifying into a normal state / abnormal state based on the similarity between images (similarity calculation S10 and registration S11 in FIG. 3) is increased.

[0065] Also, in the case of this embodiment, a plurality of estimation models can be learned in parallel. Therefore, in an actual estimation scenario (estimation by the estimation device described in the following embodiments), it is possible to select and use an estimation model that can obtain a more preferable result from among them.

[0066] <Second Embodiment> FIG. 5 shows an example of a functional block diagram of the learning device 10 of this embodiment. Further, FIG. 6 shows a diagram that more specifically shows the cycle of FIG. 1. Comparing FIGS. 2 and 3 described in the first embodiment with FIGS. 5 and 6 showing the configuration of this embodiment, the learning device 10 of this embodiment is different in that it does not have the third image group DB17-3, and the image storage unit 17 does not store the third image group.

[0067] In the first embodiment, images indicating an abnormal state are separately managed as "high-reliability first images confirmed by the user to indicate an abnormal state" and "third images determined by the computer to be similar to the first images by a predetermined level or more". However, the learning device 10 of this embodiment does not perform such management. That is, "images with high reliability confirmed by the user to indicate an abnormal state" and "images determined by the computer to be similar to the high-reliability images by a predetermined level or more" are collectively managed as "first images indicating an abnormal state". The "first images" of this embodiment are images indicating an abnormal state, and are a concept including the first images and the third images described in the first embodiment.

[0068] The registration unit 13 registers an acquired image whose similarity to the first image registered in the first image group DB17-1 is equal to or higher than a second reference value as a first image in the first image group DB17-1.

[0069] Other configurations of the learning device 10 of this embodiment are the same as those of the first embodiment.

[0070] According to the learning device 10 of the present embodiment described above, the same operational effects as those of the learning device 10 of the first embodiment are achieved. In addition, images indicating an abnormal state can be efficiently collected. Note that the reliability (reliability of indicating an abnormal state) of "high-reliability images confirmed by the user to indicate an abnormal state" and "images determined by the computer to be similar to the high-reliability images above a predetermined level" may be different. Mixing and managing images with different reliabilities may have an adverse effect on learning accuracy, estimation accuracy, etc. However, if the second reference value described above is set to a sufficiently high value, such inconveniences can be reduced.

[0071] <Third Embodiment> The estimation device of the present embodiment discriminates the state (normal / abnormal) indicated by an image using the estimation model generated by the learning device 10 of the first or second embodiment.

[0072] Since the estimation device of the present embodiment can collect sufficient and highly accurate teacher images by a characteristic method as described above and use the estimation model generated by learning based on the teacher images, high estimation accuracy can be obtained.

[0073] As described above, the embodiments of the present invention have been described with reference to the drawings, but these are examples of the present invention, and various configurations other than the above can also be adopted.

[0074] In addition, in the plurality of flowcharts used in the above description, a plurality of steps (processes) are described in order, but the execution order of the steps executed in each embodiment is not limited to the described order. In each embodiment, the order of the illustrated steps can be changed within a range that does not substantially affect the content. Also, the above-described embodiments can be combined within a range where the contents do not conflict.

[0075] Some or all of the above embodiments can also be described as follows in the appended claims, but are not limited thereto. 1. Acquisition means for acquiring an image, Similarity calculation means for calculating the similarity between the acquired image and a first image indicating an abnormal state stored in advance; Registration means for registering the acquired image whose similarity is equal to or less than a first reference value as a second image indicating a normal state; Learning means for generating an estimation model for discriminating normal / abnormal by machine learning using the first image and the second image; A learning device having the above. 2. The registration means registers the acquired image whose similarity is equal to or more than a second reference value as a third image indicating an abnormal state, The learning means generates the estimation model by machine learning using the first image, the second image, and the third image. The learning device according to 1. 3. The registration means registers the acquired image whose similarity is equal to or more than a second reference value as the first image. The learning device according to 1. 4. The learning means selects a part from the registered images, and generates the estimation model by machine learning using the selected images. The learning device according to any one of 1 to 3. 5. Learning-time estimation means for discriminating the state indicated by the acquired image using the estimation model; User confirmation means for outputting the acquired image determined to indicate an abnormal state by the learning-time estimation means and receiving correct / incorrect input from the user; Further comprising: The registration means registers the acquired image input as indicating an abnormal state in the correct / incorrect input as the first image. The learning device according to any one of 1 to 4. 6. The learning means executes learning of each of a plurality of the estimation models that learn with different algorithms, The learning-time estimation means discriminates the state indicated by the acquired image using each of the plurality of the estimation models, and accumulates the discrimination results of each of the plurality of the estimation models. The learning device according to any one of 1 to 5. 7. The acquisition means acquires an image generated by a surveillance camera. The learning device according to any one of 1 to 6. 8. A computer, Obtain an image, calculate the similarity between the obtained image and a first image indicating an abnormal state stored in advance, register the obtained image with a similarity less than or equal to a first reference value as a second image indicating a normal state, A learning method for generating an estimation model for discriminating normal / abnormal by machine learning using the first image and the second image. 9. Cause a computer to an acquisition means for acquiring an image, a similarity calculation means for calculating the similarity between the obtained image and a first image indicating an abnormal state stored in advance, a registration means for registering the obtained image with a similarity less than or equal to a first reference value as a second image indicating a normal state, a learning means for generating an estimation model for discriminating normal / abnormal by machine learning using the first image and the second image, A program that functions as 10. An estimation device for discriminating normal / abnormal using the estimation model generated by the learning device according to any one of 1 to 7.

Explanation of Signs

[0076] 10 Learning device 11 Acquisition unit 12 Similarity calculation unit 13 Registration unit 14 Learning unit 15 Estimation unit during learning 16 User confirmation unit 17 Image storage unit 17-1 First image group DB 17-2 Second image group DB 17-3 Third image group DB 18 Estimation model storage unit 18-1 Estimation model DB D14 Camera D15 Display device

Claims

1. a storage means for storing a first image showing an object in an abnormal state and a second image showing an object in a normal state; A learning means for generating a plurality of first estimation models by machine learning using the first image and the second image; an acquisition means for acquiring a classification determined for a captured image showing an object from a plurality of classifications including a normal state and an abnormal state, the classification being determined using at least one of the plurality of first estimation models; a receiving means for receiving a result of a user's correct or incorrect input for the determined classification for each captured image; having The learning means further generates a second estimation model for determining whether an image is in a normal state or an abnormal state by machine learning using the captured image classified as being in a normal state or an abnormal state based on the correct / incorrect input, together with the first image and the second image. Information processing system.

2. the receiving means receives a result of the user inputting the correct or incorrect input for the captured image determined by the first estimation model to indicate an abnormal state with a reliability of a predetermined level or higher. The information processing system according to claim 1 .

3. the acquisition means acquires a classification determined for a captured image in which an object is captured, from among the plurality of classifications, using the second estimation model.

3. The information processing system according to claim 1 or 2.

4. The information processing system according to claim 1 , wherein the abnormal state indicates a breakdown or defect of the object.

5. 5. An information processing system according to claim 1, wherein the first image is a pre-stored image, and the second image is an image whose similarity to the first image is below a reference value and which is determined to indicate a normal state.

6. The information processing system according to claim 1 , wherein the plurality of first estimation models include estimation models generated based on mutually different algorithms.

7. The information processing system according to claim 1 , wherein the learning means generates a plurality of the second estimation models by machine learning using the captured images classified as being in a normal state or an abnormal state based on the correct / incorrect input, together with the first image and the second image.

8. The information processing system according to claim 1 , wherein the normal state and the abnormal state are classifications for objects of the same type.

9. One or more computers storing a first image showing an object in an abnormal state and a second image showing an object in a normal state; generating a plurality of first estimation models by machine learning using the first image and the second image; a classification determined for a captured image showing an object from a plurality of classifications including a normal state and an abnormal state, the classification being determined using at least one of the plurality of first estimation models; and receiving a result of a user's correct or incorrect input for the determined classification for each captured image; generating a second estimation model for determining whether an image is in a normal state or an abnormal state by machine learning using the captured image classified as being in a normal state or an abnormal state based on the correct / incorrect input, together with the first image and the second image; Information processing methods.

10. Computer, a storage means for storing a first image showing an object in an abnormal state and a second image showing an object in a normal state; a learning means for generating a plurality of first estimation models by machine learning using the first image and the second image; an acquisition means for acquiring a classification determined for a captured image showing an object from a plurality of classifications including a normal state and an abnormal state, the classification being determined using at least one of the plurality of first estimation models; a receiving means for receiving a result of a user's correct or incorrect input for the determined classification for each captured image; Functioning as a The learning means further generates a second estimation model for determining whether an image is in a normal state or an abnormal state by machine learning using the captured image classified as being in a normal state or an abnormal state based on the correct / incorrect input, together with the first image and the second image. program.

Images