Information processing system, information processing method and program

Abstract

To develop technology related to an estimation model for detecting abnormalities.SOLUTION: The information processing system according to the present invention has storage means, learning means, acquisition means, and reception means. The storage means stores a first image showing an object in an abnormal state and a second image showing an object in a normal state. The learning means generates multiple first estimation models through machine learning using the first image and the second image. The acquisition means acquires a classification determined by using at least one of the first estimation models and determined for images showing an object among a plurality of classifications including both normal and abnormal states. The reception means receives the results of the user's correct or incorrect input for the classification determined for each image. The learning means further determines the normal state or the abnormal state for the image through machine learning using the images classified as the normal state or the abnormal state by the correct or incorrect input together with the first image and the second image.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a learning device, an estimation device, a learning method, and a program. [Background technology]

[0002] Patent Document 1 discloses a technology for generating an estimation model that classifies input images into good or bad images through learning based on correct and incorrect teacher images. A good image is an image that has a high degree of similarity to the correct teacher image, and a bad image is an image that has a low degree of similarity to the correct teacher image. Patent Document 2 discloses a technology for defining abnormal behavior using teacher images that show abnormal behavior, and for generating an estimation model that detects the defined abnormal behavior. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2020-35097 [Patent Document 2] Japanese Patent Application Publication No. 2019-053384 Summary of the Invention [Problem to be solved by the invention]

[0004] In technology for generating estimation models for detecting anomalies, a technology for efficiently collecting teacher images is desired. Patent Document 1 does not disclose this problem or a means for solving it. The technology described in Patent Document 2 requires the collection of a large number of teacher images showing abnormal behavior. However, collecting teacher images showing "anomalies" is not easy. An object of the present invention is to provide a technology for efficiently collecting teacher images for generating estimation models for detecting anomalies. [Means for solving the problem]

[0005] According to the present invention, an acquisition means for acquiring an image; a similarity calculation means for calculating a similarity between the acquired image and a first image showing an abnormal state that has been stored in advance; a registration means for registering the acquired image having a similarity equal to or less than a first reference value as a second image indicating a normal state; a learning means for generating an estimation model for discriminating between normal and abnormal conditions by machine learning using the first image and the second image; A learning device is provided having:

[0006] Further, according to the present invention, The computer Acquire an image, Calculating a similarity between the acquired image and a first image showing an abnormal state that has been stored in advance; registering the acquired image whose similarity is equal to or less than a first reference value as a second image indicating a normal state; A learning method is provided that generates an estimation model for discriminating between normal and abnormal conditions by machine learning using the first image and the second image.

[0007] Further, according to the present invention, Computer, an acquisition means for acquiring an image; a similarity calculation means for calculating a similarity between the acquired image and a first image showing an abnormal state that has been stored in advance; a registration means for registering the acquired image having a similarity equal to or less than a first reference value as a second image indicating a normal state; a learning means for generating an estimation model for discriminating between normal and abnormal conditions by machine learning using the first image and the second image; A program is provided to function as a

[0008] Furthermore, according to the present invention, there is provided an estimation device that discriminates between normal and abnormal using the estimation model generated by the learning device. [Effects of the Invention]

[0009] According to the present invention, it is possible to efficiently collect training images for generating an estimation model for detecting anomalies. [Brief explanation of the drawings]

[0010] [Figure 1] 10 is a flowchart illustrating an example of a processing flow of the learning device of the present embodiment. [Figure 2] FIG. 2 is an example of a functional block diagram of the learning device according to the present embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of a processing flow of the learning device of the present embodiment in detail. [Figure 4] FIG. 2 is a diagram illustrating an example of the hardware configuration of a learning device according to the present embodiment. [Figure 5] FIG. 2 is an example of a functional block diagram of the learning device according to the present embodiment. [Figure 6] FIG. 2 is a diagram illustrating an example of a processing flow of the learning device of the present embodiment in detail. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, like components are designated by like reference numerals, and the description thereof will be omitted as appropriate.

[0012] First Embodiment The learning device of this embodiment (hereinafter, may be simply referred to as "learning device") generates an estimation model that determines whether the state shown by an input image is normal or abnormal.

[0013] The object to be judged as normal / abnormal is, for example, a place (a park, a station, a facility, etc.). The normal state observed most of the time is judged as normal, and a state that differs from the normal state is judged as abnormal. For example, a state in which there is a person behaving abnormally, or a state in which an object that is always present in the place breaks down or is moved, etc., are judged as abnormal. Abnormal behavior is behavior that differs from the behavior of the majority of people observed in images. Note that the object to be judged may also be equipment such as a factory, store, facility, office, or other object. In either case, the normal state observed most of the time is judged as normal, and a state that differs from the normal state is judged as abnormal.

[0014] The learning device generates the estimation model by repeatedly executing the cycle shown in Fig. 1. As shown in Fig. 1, the learning device repeatedly executes a first image registration process S1, an image selection process S2, a learning process S3, an estimation process S4, a user confirmation process S5, and a second image registration process S6 in this order. Note that the order of the processes may be changed as long as the same operational effect is achieved.

[0015] 2 shows an example of a functional block diagram of learning device 10. As shown in the figure, learning device 10 has an acquisition unit 11, a similarity calculation unit 12, a registration unit 13, a learning unit 14, a learning-time estimation unit 15, a user confirmation unit 16, an image storage unit 17, and an estimation model storage unit 18. These functional units execute the processes shown in FIG.

[0016] Fig. 3 is a diagram showing in more detail the cycle of Fig. 1. Using this diagram, the processes shown in Fig. 1 and the processes of the functional units shown in Fig. 2 will be explained.

[0017] "First image registration process S1" The first image registration process S1 is a process for classifying and registering an image generated by a camera based on the similarity between the image generated by the camera and an image showing an abnormal state that has been registered in advance.

[0018] The first to third image group DBs 17-1 to 17-3, camera D14, similarity calculation S10, and registration S11 in Fig. 3 are involved in this process. The acquisition unit 11, similarity calculation unit 12, registration unit 13, and image storage unit 17 in Fig. 2 are involved in this process. The first to third image group DBs 17-1 to 17-3 are realized by the image storage unit 17 in Fig. 2.

[0019] First, as a preparation for the process, labeled images labeled with abnormal states are stored in the first image group DB (database) 17-1. The user prepares several images showing abnormal states in advance, labels them with abnormal states, and stores them in the first image group DB 17-1. The images stored in the first image group DB 17-1 in this manner are highly reliable labeled images that have been confirmed by the user to show abnormal states. The number of images initially stored in the first image group DB 17-1 may be several tens to several hundred; a large number of images is not necessary. This number does not impose a large burden on the user to collect labeled images. When abnormal states are defined in advance and an estimation model is generated to detect the abnormal states, it is generally necessary to prepare thousands to tens of thousands of training images showing abnormal states. The first image group DB 17-1 corresponds to the image storage unit 17 in FIG. 2. Hereinafter, images showing abnormal states stored in the first image group DB 17-1 will be referred to as "first images."

[0020] The acquisition unit 11 acquires an image generated by the camera D14. The camera D14 may be a camera (such as a surveillance camera) that captures an object to be discriminated as normal / abnormal, or may be a camera that captures an object of the same type as the object to be discriminated. The camera D14 may capture moving images, or may continuously capture still images at frame intervals longer than those for moving images. Although one camera D14 is shown in the figure, multiple cameras D14 may be used.

[0021] The acquisition unit 11 may acquire images generated by the camera D14 by real-time processing. In this case, the learning device 10 and the camera D14 are configured to be able to communicate with each other. Alternatively, the acquisition unit 11 may acquire images generated by the camera D14 by batch processing. In this case, the images generated by the camera D14 are stored in a storage device possessed by the camera D14 or in any other storage device, and the acquisition unit 11 acquires the stored images at any timing.

[0022] In this specification, "acquisition" includes at least one of the following: "the device retrieves data stored in another device or storage medium (active acquisition)" based on user input or program instructions, such as receiving data by making a request or inquiry to another device, or accessing and reading out another device or storage medium; "the device inputs data output from another device (passive acquisition)" based on user input or program instructions, such as receiving data that is distributed (or transmitted, push notification, etc.), and selecting and acquiring data from received data or information; and "the device generates new data by editing data (converting it to text, rearranging data, extracting some data, changing the file format, etc.), and then acquires the new data."

[0023] The similarity calculation unit 12 calculates the similarity between an image acquired by the acquisition unit 11 (hereinafter referred to as an "acquired image") and a first image showing an abnormal state that has been stored in advance in a first image group DB17-1 (S10 in FIG. 3). The similarity calculation unit 12 may calculate the similarity between each of the acquired images and each of the multiple first images stored in the first image group DB17-1. Alternatively, the similarity calculation unit 12 may calculate the similarity between one image (e.g., an average image) generated based on the multiple first images stored in the first image group DB17-1 and each of the acquired images.

[0024] Various methods have been proposed for calculating the similarity between images. In this embodiment, any of these methods can be adopted. For example, the similarity calculation unit 12 may detect objects within an image and calculate the similarity of the detection results (such as the similarity of the number of detected objects and the similarity of the appearances of the detected objects). Alternatively, the similarity calculation unit 12 may input each image to an estimation model that performs image analysis generated by deep learning, and calculate the similarity of the analysis results of the obtained images (such as the recognition results of objects shown in the image and the recognition results of the scene shown in the image). Alternatively, the similarity calculation unit 12 may calculate the similarity of the color or brightness that appears in the entire image or a local part of the image.

[0025] The registration unit 13 registers the acquired image whose similarity is equal to or less than the first reference value in the second image group DB (database) 17-2 as a second image indicating a normal state (an image labeled with a normal state) (S11). When the similarity calculation unit 12 calculates the similarity between each of the plurality of first images stored in the first image group DB 17-1 and each acquired image, the registration unit 13 registers the acquired image whose similarity to all of the plurality of first images is equal to or less than the first reference value in the second image group DB 17-2 as a second image.

[0026] Furthermore, the registration unit 13 registers the acquired image whose similarity is equal to or greater than a second reference value in a third image group DB (database) 17-3 as a third image indicating an abnormal state (an image labeled with an abnormal state) (S11). When the similarity calculation unit 12 calculates the similarity between each of the multiple first images stored in the first image group DB17-1 and each acquired image, the registration unit 13 registers the acquired image whose similarity with at least one of the multiple first images is equal to or greater than a second reference value in the third image group DB17-3 as a third image.

[0027] The third image group DB17-3 stores images that are determined by the computer to be similar to the first image by a predetermined level or more as images that indicate an abnormal condition. In this respect, the third image group DB17-3 differs from the first image group DB17-1, which stores highly reliable first images that have been confirmed by the user to indicate an abnormal condition.

[0028] The first and second reference values ​​may be the same or different, but by setting the first and second reference values ​​to different values, making the first reference value sufficiently small, and making the second reference value sufficiently large, it is possible to prevent an acquired image that is in a gray zone where the similarity to the first image is neither high nor low (the similarity is greater than the first reference value and less than the second reference value) from being registered as the second image or the third image.

[0029] "Image selection process S2, learning process S3" The image selection process S2 is a process of selecting an image to be used as a teacher image from the images stored in the first to third image group DBs 17-1 to 17-3. The learning process S3 is a process of using the selected image as a teacher image to execute learning for each of the multiple estimation models registered in the estimation model DB (database) 18-1.

[0030] The first to third image group DBs 17-1 to 17-3, the estimation model DB 18-1, the selection S12, and the learning S13 in Fig. 3 are involved in this process. The learning unit 14, the image storage unit 17, and the estimation model storage unit 18 in Fig. 2 are also involved in this process. The estimation model DB 18-1 is realized by the estimation model storage unit 18 in Fig. 2.

[0031] First, information on a plurality of estimation models is stored in the estimation model DB18-1. Each of the plurality of estimation models is a model that determines whether the state shown by an input image is normal or abnormal. The plurality of estimation models have different learning and estimation algorithms. For example, the plurality of estimation models are generated by deep learning. In this embodiment, information on a plurality of estimation models that have been learned and generated by, for example, a neural network, a Bayesian network, regression analysis, a support vector machine (SVM), a decision tree, a genetic algorithm, nearest neighbor classification, etc. is stored in the estimation model DB18-1.

[0032] The learning unit 14 selects at least a portion of the images registered in the first to third image groups DB17-1 to DB17-3 (S12 in FIG. 3), and generates an estimation model through machine learning using the selected images (S13 in FIG. 3).

[0033] There are various selection methods. For example, the learning unit 14 may randomly select a predetermined number of images from the entire first to third image groups DB17-1 to DB17-3. Alternatively, the learning unit 14 may randomly select a predetermined first number of images from the first image group DB17-1, a predetermined second number of images from the second image group DB17-2, and a predetermined third number of images from the third image group DB17-3. The first to third numbers may be the same or different. In other words, the proportion of the number of images selected from each of the first to third image groups DB17-1 to DB17-3 (proportion to the entire number of selected images) may be the same or different.

[0034] Furthermore, the learning unit 14 may select images for each estimation model. In this case, the first to third predetermined numbers and the ratios may differ for each estimation model.

[0035] After selecting the images, the learning unit 14 uses the selected first to third images as teacher images to perform learning of each of the multiple estimation models registered in the estimation model DB (database) 18-1. That is, the learning unit 14 generates an estimation model that distinguishes between normal and abnormal by machine learning (a concept that includes deep learning) using the first to third images.

[0036] "Estimation process S4" The estimation process S4 is a process of inputting an acquired image into each of a plurality of estimation models registered in an estimation model DB (database) 18-1, and determining the state indicated by the acquired image.

[0037] The estimation model DB 18-1, camera D14, and estimation S14 in Fig. 3 are involved in this process, as are the acquisition unit 11, learning-time estimation unit 15, and estimation model storage unit 18 in Fig. 2.

[0038] The learning-time estimation unit 15 inputs an acquired image to each of a plurality of estimation models stored in the estimation model storage unit 18, and determines the state (normal / abnormal) indicated by the acquired image. Note that the acquired image input to the estimation model in this process is an acquired image that has not yet been used in generating (learning) the estimation model at that time. For example, the learning-time estimation unit 15 can make this determination using an acquired image before it is stored in the image storage unit 17.

[0039] The discrimination results of each of the multiple estimation models may be stored in a storage device within the learning device 10.

[0040] "User verification process S5" The user confirmation process S5 is a process of outputting the determination result of the estimation process S4 to the user and receiving an input from the user as to whether the determination result is correct or incorrect.

[0041] The display device D15, extraction S15, output S16, and correct / incorrect input S17 in Fig. 3 are involved in this process, as is the user confirmation unit 16 in Fig. 2.

[0042] The user confirmation unit 16 outputs the discrimination result by the learning-time estimation unit 15 to the user (S16 in FIG. 3), and receives an input of whether the discrimination result is correct or incorrect from the user (S17 in FIG. 3). For example, the user confirmation unit 16 outputs the acquired image and the discrimination result (normal state or abnormal state), and receives an input of whether the discrimination result for the acquired image is correct or incorrect.

[0043] Executing this process for all acquired images would impose a heavy burden on the user. Therefore, the user confirmation unit 16 may extract some acquired images that satisfy a predetermined condition (S15 in FIG. 3), and output the discrimination result (S16 in FIG. 3) and accept input of correctness (S17 in FIG. 3) only for the extracted part of acquired images.

[0044] The part of the acquired images for which the determination result is output and the correct / incorrect input is accepted may be, for example, any of the following.

[0045] Acquired images that are determined to exhibit an abnormal condition by at least one estimation model. Acquired images that are determined to exhibit an abnormal condition with a certain level of confidence or higher by at least one estimation model. - Acquired images that are determined to show an abnormal state in a predetermined number of estimation models or more. - Acquired images that are determined to show an abnormal state with a certain level of confidence or higher in a certain number of estimation models or more. Acquired images that were determined to show abnormal conditions in all estimation models. -Acquired images that are determined to show an abnormal state with a certain level of confidence or higher in all estimation models.

[0046] The part of the acquired images for which the discrimination result is output and correct / incorrect input is accepted may include, in addition to any of the acquired images listed above, acquired images randomly picked from acquired images that do not satisfy the above conditions (acquired images that are presumed to show a normal state).

[0047] The user confirmation unit 16 may output the determination result via any output device such as a display or a projection device, and may accept correct / incorrect input via any input device such as a keyboard, a mouse, a touch panel, a physical button, or a microphone. Alternatively, the user confirmation unit 16 may transmit the determination result to a predetermined mobile terminal and acquire the content of the correct / incorrect input made to the mobile terminal from the mobile terminal. Alternatively, the user confirmation unit 16 may store the determination result on any server in a state that can be viewed from any device. The user confirmation unit 16 may then acquire the content of the correct / incorrect input entered from any device and stored on the server. Note that the examples illustrated here are merely examples, and are not limiting.

[0048] "Second image registration process S6" The second image registration process S6 is a process for registering the acquired image, which has been input as indicating an abnormal state in the user confirmation process S5, as a first image in the first image group DB17-1.

[0049] The first image group DB17-1 and registration S18 in Fig. 3 are involved in this process, and the registration unit 13 and image storage unit 17 in Fig. 2 are involved in this process.

[0050] The registration unit 13 registers an acquired image for which an input indicating an abnormal state has been made in the correct / incorrect input received by the user confirmation unit 16 as a first image in the first image group DB17-1.

[0051] The acquired image that has been input to indicate an abnormal state corresponds to an acquired image where the discrimination result is "abnormal state" and the correct / incorrect input is "correct", or an acquired image where the discrimination result is "normal state" and the correct / incorrect input is "incorrect".

[0052] Here, a modified example of the learning device 10 of this embodiment will be described. The learning device 10 does not need to have the third image group DB17-3. The registration unit 13 does not need to execute the process of registering an acquired image whose similarity to the first image is equal to or less than a first reference value as a second image in the second image group DB17-2, and registering an acquired image whose similarity to the first image is equal to or greater than a second reference value as a third image in the third image group DB17-3. In this case, images indicating a normal state are accumulated by the process performed by the registration unit 13.

[0053] Next, we will explain an example of the hardware configuration of learning device 10. Each functional unit of learning device 10 is realized by any combination of hardware and software, centered around a CPU (Central Processing Unit) of any computer, memory, programs loaded into memory, a storage unit such as a hard disk that stores the programs (this can store programs pre-loaded at the time the device is shipped, as well as programs downloaded from storage media such as CDs (Compact Discs) or servers on the Internet), and a network connection interface. Those skilled in the art will understand that there are many variations in the implementation methods and devices.

[0054] FIG. 4 is a block diagram illustrating the hardware configuration of a learning device 10. As shown in FIG. 4, the learning device 10 has a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The learning device 10 does not necessarily have the peripheral circuit 4A. Note that the learning device 10 may be composed of multiple physically and / or logically separated devices, or may be composed of a single physically and / or logically integrated device. When the learning device 10 is composed of multiple physically and / or logically separated devices, each of the multiple devices can have the above hardware configuration.

[0055] The bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to transmit and receive data to and from each other. The processor 1A is an arithmetic processing device such as a CPU or a GPU (Graphics Processing Unit). The memory 2A is a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The input / output interface 3A includes an interface for acquiring information from an input device, an external device, an external server, an external sensor, a camera, etc., and an interface for outputting information to an output device, an external device, an external server, etc. Examples of input devices include a keyboard, a mouse, a microphone, physical buttons, a touch panel, etc. Examples of output devices include a display, a speaker, a printer, a mailer, etc. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

[0056] Next, the effects of the learning device 10 will be described.

[0057] The learning device 10 of this embodiment generates an estimation model for discriminating between normal and abnormal conditions through machine learning using images showing normal conditions and images showing abnormal conditions as teacher images. In the estimation model, the normal condition observed most of the time is discriminated as normal, and a condition different from the normal condition is discriminated as abnormal.

[0058] In this case, even if an abnormal state not defined in advance occurs, it can be determined as an abnormal state as long as the state is different from the normal state, making it possible to detect all abnormal states.

[0059] Furthermore, when abnormal conditions are defined in advance and an estimation model for detecting those abnormal conditions is generated, it is necessary to prepare a large number of training images representing each abnormal condition. However, preparing training images representing abnormal conditions is not easy. In this embodiment, the number of "images representing abnormal conditions" that need to be prepared is smaller than when generating an estimation model for detecting predefined abnormal conditions. As a result, the burden on the user is reduced.

[0060] In this embodiment, a large number of "images showing a normal state" are required. However, since many objects are usually in a "normal state," it is possible to easily collect "images showing a normal state" from images of such objects.

[0061] Furthermore, in this embodiment, "images showing normal states" can be automatically accumulated based on the results of similarity determination between a small number of "images showing abnormal states" (meaning a number smaller than the number of images showing abnormal states required to generate an estimation model for detecting a predefined abnormal state) prepared in advance and images generated by a surveillance camera, etc. This reduces the burden on the user.

[0062] Furthermore, in this embodiment, the second image registration process S6 can increase the number of "images showing abnormal conditions." In this way, the number of "images showing abnormal conditions" can be increased, thereby improving the estimation accuracy of the resulting estimation model.

[0063] Furthermore, in this embodiment, the first image registration process S1 can also increase the number of "images showing an abnormal state." In this case, by setting the second reference value described above to a sufficiently high value, it is possible to increase the number of "images showing an abnormal state" with higher reliability. Furthermore, the increase in the number of "images showing an abnormal state" is expected to improve the estimation accuracy of the resulting estimation model.

[0064] Furthermore, in this embodiment, images indicating an abnormal state can be managed by dividing them into "highly reliable first images that have been confirmed by the user to indicate an abnormal state" and "third images that have been determined by a computer to be more similar to the first images than a predetermined level." Only the first images can then be used as reference targets for similarity calculation S10 in FIG. 3. In this way, by using only the highly reliable first images as reference targets, the reliability of the process of classifying images into normal states / abnormal states based on the similarity between images (similarity calculation S10 and registration S11 in FIG. 3) is increased.

[0065] Furthermore, in this embodiment, multiple estimation models can be trained in parallel, which makes it possible to select and use an estimation model that will provide more favorable results in an actual estimation situation (estimation by an estimation device described in the following embodiments).

[0066] <Second embodiment> Fig. 5 shows an example of a functional block diagram of the learning device 10 of this embodiment. Fig. 6 shows a diagram illustrating the cycle of Fig. 1 in more detail. Comparing Figs. 2 and 3 described in the first embodiment with Figs. 5 and 6 showing the configuration of this embodiment, the learning device 10 of this embodiment differs in that it does not have a third image group DB17-3 and the image storage unit 17 does not store the third image group.

[0067] In the first embodiment, images indicating an abnormal state were managed separately as "highly reliable first images that have been confirmed by the user to indicate an abnormal state" and "third images that have been determined by a computer to be more similar to the first image than a predetermined level." However, the learning device 10 of this embodiment does not manage images in this manner. That is, "highly reliable images that have been confirmed by the user to indicate an abnormal state" and "images that have been determined by a computer to be more similar to the highly reliable images than a predetermined level" are managed together as "first images that indicate an abnormal state." The "first image" of this embodiment is an image that indicates an abnormal state, and is a concept that includes the first image and the third image described in the first embodiment.

[0068] The registration unit 13 registers an acquired image whose similarity to a first image registered in the first image group DB17-1 is equal to or greater than a second reference value as the first image in the first image group DB17-1.

[0069] Other configurations of the learning device 10 of this embodiment are the same as those of the first embodiment.

[0070] The learning device 10 of this embodiment described above achieves the same effects as the learning device 10 of the first embodiment. It also allows for efficient collection of images that indicate abnormal conditions. Note that the reliability (reliability of indicating an abnormal condition) of "highly reliable images that have been confirmed by a user to indicate an abnormal condition" and "images that have been determined by a computer to be more similar to the highly reliable images than a predetermined level" may differ. Managing images with different reliability levels may adversely affect the accuracy of learning and estimation. However, setting the second reference value described above to a sufficiently high value can mitigate such inconveniences.

[0071] <Third embodiment> The estimation device of this embodiment uses the estimation model generated by the learning device 10 of the first or second embodiment to determine the state (normal / abnormal) indicated by an image.

[0072] The estimation device of this embodiment can collect sufficient and highly accurate teacher images using the characteristic method described above, and can use an estimation model generated by learning based on the teacher images, thereby achieving high estimation accuracy.

[0073] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations can also be adopted.

[0074] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order, but the order of execution of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings can be changed to the extent that the content is not affected. Furthermore, the above-mentioned embodiments can be combined to the extent that the content is not contradictory.

[0075] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. 1. An acquisition means for acquiring an image; a similarity calculation means for calculating a similarity between the acquired image and a first image showing an abnormal state that has been stored in advance; a registration means for registering the acquired image having a similarity equal to or less than a first reference value as a second image indicating a normal state; a learning means for generating an estimation model for discriminating between normal and abnormal conditions by machine learning using the first image and the second image; A learning device having the above configuration. 2. The registration means registers the acquired image having the similarity equal to or greater than a second reference value as a third image indicating an abnormal state, The learning device according to claim 1, wherein the learning means generates the estimation model through machine learning using the first image, the second image, and the third image. 3. The learning device according to 1, wherein the registration means registers the acquired image whose similarity is equal to or greater than a second reference value as the first image. 4. A learning device according to any one of 1 to 3, wherein the learning means selects a portion of registered images and generates the estimation model through machine learning using the selected images. 5. A learning-time estimation means for determining a state indicated by the acquired image using the estimation model; a user confirmation means for outputting the acquired image determined by the learning time estimation means to be in an abnormal state and for receiving a correct or incorrect input from a user; and 5. A learning device according to any one of 1 to 4, wherein the registration means registers the acquired image in which the correct / incorrect input indicates an abnormal state as the first image. 6. The learning means executes learning of each of the plurality of estimation models that are learned using mutually different algorithms; 6. A learning device according to any one of 1 to 5, wherein the learning estimation means uses each of the plurality of estimation models to determine the state indicated by the acquired image and accumulates the determination results of each of the plurality of estimation models. 7. A learning device according to any one of 1 to 6, wherein the acquisition means acquires images generated by a surveillance camera. 8. The computer Acquire an image, Calculating a similarity between the acquired image and a first image showing an abnormal state that has been stored in advance; registering the acquired image whose similarity is equal to or less than a first reference value as a second image indicating a normal state; A learning method for generating an estimation model that discriminates between normal and abnormal conditions through machine learning using the first image and the second image. 9. Computer an acquisition means for acquiring an image; a similarity calculation means for calculating a similarity between the acquired image and a first image showing an abnormal state that has been stored in advance; a registration means for registering the acquired image having a similarity equal to or less than a first reference value as a second image indicating a normal state; a learning means for generating an estimation model for discriminating between normal and abnormal conditions by machine learning using the first image and the second image; A program that functions as a 10. An estimation device that discriminates between normal and abnormal using an estimation model generated by a learning device described in any one of 1 to 7. [Explanation of symbols]

[0076] 10 Learning Device 11 Acquisition Department 12 Similarity calculation unit 13 Registration Department 14 Learning Department 15 Learning estimation unit 16 User Verification Section 17 Image storage unit 17-1 First Image Group DB 17-2 Second Image Database 17-3 Third Image Group DB 18 Estimation model memory section 18-1 Estimation Model DB D14 camera D15 Display device

Claims

1. a storage means for storing a first image showing an object in an abnormal state and a second image showing an object in a normal state; a learning means for generating a plurality of first estimation models by machine learning using the first image and the second image; an acquisition means for acquiring a classification determined for a captured image showing an object from a plurality of classifications including a normal state and an abnormal state, the classification being determined using at least one of the plurality of first estimation models; a receiving means for receiving a result of a user's correct or incorrect input for the determined classification for each captured image; and The learning means further generates a second estimation model for determining whether an image is in a normal state or an abnormal state by machine learning using the captured image classified as being in a normal state or an abnormal state based on the correct / incorrect input, together with the first image and the second image. Information processing system.

2. the receiving means receives a result of the user inputting the correct or incorrect information for the captured image determined by the first estimation model to indicate an abnormal state with a reliability of a predetermined level or higher. The information processing system according to claim 1 .

3. the acquisition means acquires a classification determined for a captured image showing an object from among the plurality of classifications, the classification being determined using the second estimation model.

3. The information processing system according to claim 1 or 2.

4. The information processing system according to claim 1 , wherein the abnormal state indicates a failure or defect of the object.

5. 5. An information processing system according to claim 1, wherein the first image is a pre-stored image, and the second image is an image whose similarity to the first image is below a reference value and which is determined to indicate a normal state.

6. The information processing system according to claim 1 , wherein the plurality of first estimation models include estimation models generated based on mutually different algorithms.

7. 7. The information processing system according to claim 1, wherein the learning means generates a plurality of the second estimation models by machine learning using the captured images classified as normal or abnormal based on the correct / incorrect input, together with the first images and the second images.

8. The information processing system according to claim 1 , wherein the normal state and the abnormal state are classifications of objects of the same type.

9. One or more computers storing a first image showing an object in an abnormal state and a second image showing an object in a normal state; generating a plurality of first estimation models by machine learning using the first image and the second image; a classification determined for a captured image showing an object from a plurality of classifications including a normal state and an abnormal state, the classification being determined using at least one of the plurality of first estimation models; receiving a result of a user's correct or incorrect input for the determined classification for each of the captured images; generating a second estimation model that determines whether an image is in a normal state or an abnormal state by machine learning using the captured image classified as being in a normal state or an abnormal state based on the correct / incorrect input, together with the first image and the second image; Information processing methods.

10. Computer, a storage means for storing a first image showing an object in an abnormal state and a second image showing an object in a normal state; a learning means for generating a plurality of first estimation models by machine learning using the first image and the second image; an acquisition means for acquiring a classification determined for a captured image showing an object from a plurality of classifications including a normal state and an abnormal state, the classification being determined using at least one of the plurality of first estimation models; a receiving means for receiving a result of the user's correct or incorrect input for the determined classification for each of the captured images; It functions as The learning means further generates a second estimation model for determining whether an image is in a normal state or an abnormal state by machine learning using the captured image classified as being in a normal state or an abnormal state based on the correct / incorrect input, together with the first image and the second image. program.

Images