Learning system, inference system, learning method, and inference method

Abstract

To highly accurately determine the state of an inspection-target, even when it has complex shape.SOLUTION: A learning system performs: first acquisition processing acquiring a teacher signal which is a combination of the first image data of an inspection-target and the first state of the inspection-target; region designation processing designating a region in the first image data; first extraction processing extracting a first connotation region which connotes the designated region, from the first image data; first mask processing masking a first residual region of the first connotation region excluding the designated region; first calculation processing calculating first statistical information indicating the frequency of color information in the first mask connotation region which is composed of the designated region and a first mask region which is the masked first residual region; and learning processing learning a determination model on the basis of output data which has been output as a result of input of the first statistical information in the designated region to the determination model, and the first state.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a learning system, an inference system, a learning method, and an inference method. [Background technology]

[0002] As a background art of this technical field, Patent Document 1 discloses a texture evaluation device that extracts skin groove components from a skin image, acquires the multiple local image features using the extracted skin groove components, acquires a histogram by performing vector quantization on the multiple local image features, and evaluates the texture by inputting the histogram into a pre-prepared estimation model. This texture evaluation device includes a feature acquisition unit that acquires multiple local image features from a skin image, a histogram acquisition unit that acquires a histogram by performing vector quantization on the multiple local image features, and an evaluation unit that evaluates the texture by inputting the histogram into a pre-prepared estimation model, and the feature acquisition unit has a skin groove component extraction processing unit that extracts skin groove components from the skin image, and evaluates the texture by acquiring the multiple local image features from an image processed by the skin groove component extraction processing unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-092694 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the above-mentioned Patent Document 1, the condition for extracting the skin groove component from the skin image is that the center point of the local image feature may be either the skin ridge or the skin groove, and therefore, no consideration is given to improving the evaluation accuracy in the evaluation of the shape.

[0005] An object of the present invention is to improve the accuracy of state determination for an inspection object by eliminating the shape from the inspection object. [Means for solving the problem]

[0006] A learning system which is a first disclosed technology of the present application is a learning system having a processor which executes a program and a storage device which stores the program, wherein the processor executes the following processes: a first acquisition process which acquires a teacher signal which is a combination of first image data of an inspection target and a first state of the inspection target; a region designation process which designates a region within the first image data acquired by the first acquisition process; a first extraction process which extracts from the first image data a first included region which contains the designated region designated by the region designation process; a first masking process which masks a first remaining region excluding the designated region from the first included region extracted by the first extraction process; a first calculation process which calculates first statistical information which indicates the frequency of color information in a first mask included region which consists of the designated region and a first mask region in which the first remaining region is masked by the first masking process; and a learning process which learns the judgment model based on output data which is output as a result of inputting the first statistical information of the designated region, among the first statistical information calculated by the first calculation process, into a judgment model, and the first state.

[0007] An inference system, which is a second disclosed technology of the present application, is an inference system having a processor that executes a program and a storage device that stores the program, and is capable of accessing a judgment model that is learned using first statistical information of a designated area in first image data of an inspection object as an explanatory variable and a first state of the inspection object as a target variable, and the processor executes the following processes: an acquisition process that acquires second image data of the inspection object; an extraction process that extracts a second encapsulated area that contains the designated area from the second image data acquired by the acquisition process; a masking process that masks a remaining area excluding the designated area from the encapsulated area extracted by the extraction process; a calculation process that calculates second statistical information indicating the frequency of color information in a mask encapsulated area consisting of the designated area and a mask area in which the remaining area is masked by the masking process; an inference process that infers the second state of the inspection object by inputting the second statistical information of the designated area from the second statistical information calculated by the calculation process into the judgment model; and an output process that outputs the inference result by the inference process. Effect of the Invention

[0008] According to a representative embodiment of the present invention, it is possible to eliminate the shape from the inspection object and to improve the accuracy of the state judgment for the inspection object. Problems, configurations and effects other than those described above will become apparent from the description of the following examples. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of a system configuration of a state determination system according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of the hardware configuration of a computer. [Diagram 3] FIG. 3 is a block diagram of an example of a functional configuration of the state determination system according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a method for storing learning data according to the first embodiment. [Diagram 5]FIG. 5 is a diagram illustrating an example of a pre-learning process and an example of an inference pre-process according to the first embodiment. [Figure 6] FIG. 6 is a flowchart of an example of a learning process according to the first embodiment. [Figure 7] FIG. 7 is a flowchart of an example of a processing procedure of the state determination process according to the first embodiment. [Figure 8] FIG. 8 is a flowchart of an example of a processing procedure of the status notification processing according to the first embodiment. [Figure 9] FIG. 9 is a block diagram of an example of a functional configuration of the state determination system according to the second embodiment. As illustrated in FIG. [Figure 10] FIG. 10 is a block diagram of an example of a functional configuration of the state determination system according to the third embodiment. As illustrated in FIG. [Figure 11] FIG. 11 is a flowchart of an example of a processing procedure of the state determination processing according to the third embodiment. [Figure 12] FIG. 12 is a block diagram of an example of a functional configuration of the state determination system according to the fourth embodiment. As illustrated in FIG. [Figure 13] FIG. 13 is a diagram illustrating an example of a pre-learning process and an example of an inference pre-process according to the fourth embodiment. [Figure 14] FIG. 14 is a flowchart of an example of a learning process according to the fourth embodiment. [Figure 15] FIG. 15 is a flowchart of an example of a processing procedure of the state determination process according to the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In all the drawings for explaining the embodiment, the same members are generally given the same reference numerals, and the repeated description thereof will be omitted. In addition, in the following embodiment, the components (including element steps, etc.) are not necessarily essential, unless otherwise specified or considered to be obviously essential in principle. In addition, when it is said that "consists of A", "consists of A", "has A", or "includes A", it is needless to say that other elements are not excluded, except when it is specifically specified that only the element is included. Similarly, in the following embodiment, when referring to the shape, positional relationship, etc. of the components, etc., it is intended to include those that are substantially similar or similar to the shape, etc., except when it is specifically specified or considered to be obviously not essential in principle.

[0011] The designations "first," "second," "third," and the like in this specification are given to identify components, and do not necessarily limit the number, order, or content. Furthermore, numbers for identifying components are used in different contexts, and a number used in one context does not necessarily indicate the same configuration in another context. Furthermore, a component identified by a certain number does not prevent the component from also having the function of a component identified by another number.

[0012] In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings, etc. may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings, etc. EXAMPLES

[0013] <Figure 1 Status Judgment System> FIG. 1 is an explanatory diagram showing an example of a system configuration of a state determination system according to a first embodiment. The state determination system 100 includes a server 101, one or more clients 102, and a patrol lamp 106. The server, the clients, and the patrol lamp are communicatively connected via a network 105 such as the Internet, a local area network (LAN), or a wide area network (WAN). The server 101 is a computer that manages the clients 102. The client 102 is a computer that is connected to a sensor 103 and acquires data from the sensor 103. The patrol lamp 106 turns on and off according to a signal from the client 102.

[0014] The sensor 103 detects analysis target data from the analysis environment. The sensor 103 is, for example, a camera that captures still images or videos. The sensor 103 captures images of the analysis environment at fixed points. Therefore, for example, there are objects whose shape or size changes between time-series frames output from the sensor 103, and there are also congruent objects whose shape and size do not change between frames. Furthermore, even congruent objects between frames may have different colors.

[0015] The teacher signal DB 104 is a database that holds combinations of learning data (image data) of the inspection target and the state of the inspection target (for example, correct answer data representing two or more states such as "normal" and "abnormal") as teacher signals.

[0016] The inspection object is an area whose state is judged by fixed-point photography, for example, an area such as a drainage facility where the outline shape does not change but the state needs to be judged by a change in color. Therefore, the drainage facility is only an example, and the inspection object can be applied to other facilities having such an area. The teacher signal DB 104 may be stored in the server 101, or may be connected to a computer capable of communicating with the server 101 or the client 102 via the network 105.

[0017] The state determination system 100 has a learning function using a teacher signal DB 104 and a state determination function using a state determination model obtained by the learning function. The state determination model is a learning model for determining the state of an object shown in an image. The learning function and the state determination function may be implemented in either the server 101 or the client 102 as long as they are implemented in the state determination system 100. For example, the server 101 may implement the learning function, and the client 102 may implement the state determination function. Alternatively, the server 101 may implement the learning function and the state determination function, and the client 102 may transmit data from the sensor 103 to the server 101 and accept a state inference result by the state determination function from the server 101.

[0018] Also, the client 102 may implement a learning function and a state determination function, and the server 101 may manage a state determination model and a state inference result from the client 102. A computer that implements the learning function is referred to as a learning device, and a computer that implements at least the state determination function of the learning function and the state determination function is referred to as a state determination device. Also, while a client-server type state determination system 100 is given as an example in FIG. 1, a stand-alone type state determination device may also be used. In the first embodiment, for convenience of explanation, a state determination system 100 in which the server 101 implements a learning function (learning device) and the client 102 implements a state determination function will be described as an example.

[0019] <Figure 2 Example of computer hardware configuration> FIG. 2 is a block diagram showing an example of a hardware configuration of a computer (server 101, client 102). The computer 200 has a processor 201, a storage device 202, an input device 203, an output device 204, and a communication interface (communication IF) 205. The processor 201, the storage device 202, the input device 203, the output device 204, and the communication IF 205 are connected by a bus 206. The processor 201 controls the computer 200. The storage device 202 is a working area for the processor 201. The storage device 202 is a non-transient or temporary recording medium that stores various programs and data. Examples of the storage device 202 include a ROM (Read Only Memory), a RAM (Random Access Memory), a HDD (Hard Disk Drive), and a flash memory. The input device 203 inputs data. Examples of the input device 203 include a keyboard, a mouse, a touch panel, a numeric keypad, and a scanner. The output device 204 outputs data. The output device 204 may be, for example, a display, a printer, or a speaker. The communication IF 205 is connected to the network 105 and transmits and receives data.

[0020] <Figure 3: Example of functional configuration of state determination system 100> 3 is a block diagram showing an example of a functional configuration of the state determination system 100 according to the first embodiment. The server 101 includes a teacher signal acquisition unit 300, a region designation unit 301, an inclusive region extraction unit 302, a mask processing unit 303, a statistical information calculation unit 304, and a determination model learning unit 305. The client 102 includes an analysis target data acquisition unit 310, a designated region reproduction unit 311, an inclusive region extraction unit 312, a mask processing unit 313, a statistical information calculation unit 314, a determination model inference unit 315, and an inference result output unit 316. The patrol lamp 106 includes an inference result reception unit 320.

[0021] The client 102 includes a teacher signal acquisition unit 300, a region designation unit 301, an inclusive region extraction unit 302, a mask processing unit 303, a statistical information calculation unit 304, a judgment model learning unit 305, and a judgment model learning unit 305. The client 102 includes an analysis target data acquisition unit 310, a designated region reproduction unit 311, an inclusive region extraction unit 312, a mask processing unit 313, a statistical information calculation unit 314, a judgment model inference unit 315, and an inference result output unit 316, which are specifically realized by, for example, causing the processor 201 to execute a program stored in the storage device 202 shown in FIG.

[0022] 3, the teacher signal DB 104 exists outside the server 101, but it may be realized by the storage device 202 in the server 101 or the client 102. First, an example of the functional configuration on the server 101 side will be described.

[0023] The teacher signal acquisition unit 300 acquires one or more teacher signals to be used for learning the teacher signal acquired from the teacher signal DB 104, and outputs the selected teacher signal to the region designation unit 301. The teacher signal is composed of learning data and a state. Details of the teacher signal will be described with reference to FIG. 4.

[0024] The area designation unit 301 designates an area for determining a state for the learning data among the acquired teacher signals. Specifically, for example, the area designation unit 301 designates an area by a polygon by designating a plurality of vertices for the learning data. Here, the designated area is referred to as a designated area. The area designation unit 301 outputs the designated area to the encapsulated area extraction unit 302 and the designated area reproduction unit 311. Note that the method of designating the area is not limited to a polygon, and area designation may be performed by a curve such as a circle or an ellipse, or area designation may be performed by edge detection or segmentation, and the method of designating the area is not limited.

[0025] The inclusive region extraction unit 302 extracts a rectangle that includes the acquired designated region from the learning data in the teacher signal, and outputs the extracted rectangle as the inclusive region to the mask processing unit 303. Here, since the inclusive region is a rectangle, it can be processed as a matrix variable. Note that although the inclusive region is a rectangle, for example, the inclusive region may be handled as a figure that includes the designated region, together with information indicating the positions of the inclusive region and the designated region in pixel units.

[0026] The mask processing unit 303 performs processing to mask the difference region obtained by excluding the designated region from the acquired inclusive region with a fixed brightness, and outputs the masked inclusive region to the statistical information calculation unit 304 .

[0027] The statistical information calculation unit 304 calculates a histogram of luminance for the acquired mask inclusion region, and outputs the histogram to the judgment model learning unit 305. In calculating the histogram, the histogram may be calculated for each primary color (R, G, B) for the mask inclusion region, or the histogram may be calculated after correcting to grayscale, or the histogram may be calculated after converting to HSV color space. In addition, the histogram may be calculated by combining (R, G, B), grayscale, and HSV color space, and the method of calculating the histogram is not limited.

[0028] The judgment model learning unit 305 performs learning using the acquired histogram as an explanatory variable and the state constituting the teacher signal as a target variable, and generates a judgment model that judges the state using the histogram by machine learning. The judgment model learning unit 305 outputs the judgment model generated by learning to the judgment model inference unit 315.

[0029] Next, a description will be given of an example of a functional configuration on the side of the client 102. The analysis target data acquisition unit 310 acquires analysis target data from the sensor 103, and outputs the data to the designated area reproduction unit 311.

[0030] The designated region reproduction unit 311 assigns the designated region acquired from the region designation unit 301 to the analysis target data acquired from the analysis target data acquisition unit 310 , and outputs the data to the inclusive region extraction unit 312 .

[0031] The encapsulating region extraction unit 312 extracts a rectangle that contains the acquired specified region from the acquired analysis target data, and outputs the extracted rectangle as the encapsulating region to the mask processing unit 313. Here, since the encapsulating region is a rectangle, it can be processed as a matrix variable.

[0032] The mask processing unit 313 performs a process of masking the difference region obtained by excluding the specified region from the acquired encapsulation region with a fixed brightness to obtain a mask region, and outputs the mask encapsulation region consisting of the specified region and the mask region to the statistical information calculation unit 314.

[0033] The statistical information calculation unit 314 calculates a histogram of luminance for the acquired mask inclusion region, and outputs it to the judgment model inference unit 315. In calculating the histogram, a histogram may be calculated for each primary color of (R, G, B) for the mask inclusion region, or the histogram may be calculated after correcting to grayscale, or the histogram may be calculated after converting to HSV color space. In addition, the histogram may be calculated by combining (R, G, B), grayscale, and HSV color space, and the method of calculating the histogram is not limited.

[0034] The judgment model inference unit 315 judges the state of the specified region in the analysis target data by using the acquired histogram as an input of the judgment model acquired from the judgment model learning unit 305. The judgment model inference unit 315 outputs the inference result to the inference result output unit 316.

[0035] If the inference result acquired from the judgment model inference unit 315 is a predetermined inference result, the inference result output unit 316 transmits the inference result to the inference result receiving unit 320 in the patrol lamp 106. The predetermined inference result indicates, for example, a state in which additional control is required based on the results of the data to be analyzed, and the necessary control can be performed by transmitting the predetermined inference result. As an example of the additional control, if the inference result indicates an abnormal state, the patrol lamp 106 can be turned on to prompt an operator to take action. Also, in the case of an abnormal state, an operator can add chemicals to the drainage equipment or remove the abnormal object.

[0036] Next, a description will be given of an example of a functional configuration on the side of the patrol lamp 106. The inference result receiving unit 320 turns on the patrol lamp in accordance with the inference result received from the inference result output unit 316, to indicate that additional control is required.

[0037] Incidentally, the inclusive region extraction unit 302 and the inclusive region extraction unit 312 perform the same process, the mask processing unit 303 and the mask processing unit 313 perform the same process, and the statistical information calculation unit 304 and the statistical information calculation unit 314 perform the same process.

[0038] <Figure 4 Learning data> 4 is an explanatory diagram showing an example of a method for saving learning data. The learning data is saved in a teacher signal saving folder 400 of the teacher signal DB 104. The teacher signal saving folder 400 has a first subfolder 410, a second subfolder 420, and a third subfolder 430.

[0039] The first subfolder 410 stores image data of 000.jpg411 and 001.jpg412. The second subfolder 420 stores image data of 010.jpg421, 011.jpg422, and 012.avi423. The third subfolder 430 stores image data of 020.avi431, 021.avi432, and 012.avi433.

[0040] The image data are grouped into a first subfolder 410, a second subfolder 420, and a third subfolder 430 according to their status. The status is the name of the subfolder. Specifically, for example, the status of the first subfolder 410 is "class A", the status of the second subfolder 420 is "class B", and the status of the third subfolder 430 is "class C". For example, class A means "normal", class B means "abnormal", and class C means "observation required". In addition, class A, class B, and class C may indicate the additional control described above. For example, class A is "normal", class B is "abnormal 1" (addition of medicine), and class C is "abnormal 2" (operation stopped).

[0041] 4, the number of subfolders is three, but it may be two or more. Furthermore, the number of image data in the first subfolder 410, the second subfolder 420, and the third subfolder 430 is not limited to that in FIG.

[0042] The teacher signal acquisition unit 300 acquires image data organized by subfolder together with the subfolder name indicating the state, and uses it for learning. For example, when learning a judgment model on a class-by-class basis, by specifying the target subfolder, the server 101 acquires the image data in the subfolder and inputs it to the judgment model, compares the output result with the subfolder name to calculate the value of the loss function, and performs error backpropagation, thereby making it possible to learn the judgment model.

[0043] Furthermore, when adding a new teacher signal to the teacher signal storage folder 400, the server 101 searches the teacher signal storage folder 400 in the state of the teacher signal to be added, identifies a subfolder to be used as a storage destination, and stores the image data to be added. In this way, the generation and management of teacher signals is simplified.

[0044] <Figure 5 Example of pre-learning processing and pre-inference processing> 5 is an explanatory diagram showing an example of pre-learning processing and pre-inference processing according to the first embodiment. The pre-learning processing is processing executed by the area designation unit 301, the designated area reproduction unit 311, the inclusive area extraction unit 302, the mask processing unit 303, and the statistical information calculation unit 304. The pre-inference processing is processing executed by the analysis target data acquisition unit 310, the designated area reproduction unit 311, the inclusive area extraction unit 312, the mask processing unit 313, and the statistical information calculation unit 314. The learning data and the analysis target data, which are image data, are denoted by a reference symbol 500. When there is no distinction between the learning data and the analysis target data, they are written as image data 500.

[0045] The area designation unit 301 generates a designated area 501 by setting multiple vertices of an inspection target area in the learning data 500 by operating a mouse, and outputs the designated area 501 to the designated area reproduction unit 311 and the encapsulated area extraction unit 302. The designated area 501 may be generated by curve designation, area designation using segmentation, or area designation using edge detection, and the method of generating the designated area 501 is not limited.

[0046] The designated region reproduction unit 311 assigns the designated region 501 to the analysis target data 500 acquired from the analysis target data acquisition unit 310 , and outputs the analysis target data 500 to the inclusive region extraction unit 312 .

[0047] The inclusive region extraction units 302 and 312 generate an inclusive region 502 that contains the specified region 501 , and output the generated inclusive region 502 to the mask processing units 303 and 313 .

[0048] The mask processing units 303, 313 perform a process of masking the acquired encapsulated region 502, i.e., the differential region 520 obtained by excluding the specified region 501 from the encapsulated region 502, with a fixed brightness value 550 to generate a mask region 530 (masked with black having a brightness value of 0 as an example in Figure 5), generate a mask encapsulated region 503 consisting of the specified region 501 and the mask region 530, and output the mask encapsulated region 503 to the statistical information calculation units 304, 314.

[0049] The statistical information calculation units 304 and 314 calculate a histogram 504 relating to the luminance values ​​and output it to the judgment model learning unit 305 and the judgment model inference unit 315, respectively. The horizontal axis of the histogram 504 represents the luminance value, and the vertical axis represents the frequency, i.e., the number of pixels. Here, since the image data 500 is a time-series of frames captured at a fixed point, the number of pixels in the mask area 530 masked with the luminance value 550 fixed by the mask processing units 303 and 313 is fixed.

[0050] Therefore, in the histogram 504, a fixed frequency 505 is calculated for a fixed luminance value 550, regardless of changes over time in the image data 500. In a time-series series of image data 500, the fixed frequency 505 does not change depending on each piece of image data 500, so the determination model learning unit 305 treats the mask region 530 as luminance data with no information volume, and generates a determination model using the residual histogram 506 excluding the fixed frequency 505. The residual histogram 506 is a histogram calculated using the specified region 501.

[0051] The judgment model inference unit 315 uses the judgment model generated from the residual histogram 506 excluding the fixed frequency 505 to perform inference on the specified region 501 by judging the state of the mask inclusion region 503 of the analysis target data 500. Therefore, the state of only the specified region 501 is judged with high accuracy.

[0052] The region designation unit 301 may generate a plurality of designated regions 501, and the number of designated regions 501 is not limited. When a plurality of designated regions 501 are generated, the inclusive region extraction units 302, 312 generate one inclusive region 502 that contains the plurality of designated regions 501, and output the one inclusive region 502 to the mask processing units 303, 313. The mask processing units 303, 313 perform a process of masking the difference region 520, which is obtained by removing the plurality of designated regions 501 from the inclusive region 502, with a fixed brightness value 550 to generate a mask region 530, thereby generating a mask inclusive region 503 consisting of the plurality of designated regions 501 and the mask region 530.

[0053] Furthermore, when a plurality of designated regions 501 are generated, the state determination system 100 may generate an inclusive region 502 and a mask inclusive region 503 for each designated region 501 .

[0054] Furthermore, the fixed luminance value 550 is not limited to "0." If it overlaps with a pixel in the specified region 501, the variation in luminance for each image data becomes the amount of information and contributes to the determination model, so even if the fixed luminance value 550 overlaps with a luminance value in the histogram 504, the fixed luminance value 550 will be removed as an offset in the learning process.

[0055] <Figure 6 Learning process> 6 is a flowchart illustrating a detailed example of a procedure of a learning process by the server 101 (learning device) according to the embodiment 1. The server 101, by the teacher signal acquiring unit 300, acquires one or more teacher signals to be used for learning from the teacher signal DB 104 (step S600).

[0056] Next, the server 101 causes the region designation unit 301 to designate a region for determining a state for the learning data 500 from the acquired teacher signal, and acquires it as a designated region 501 (step S601).

[0057] Next, the server 101 causes the inclusive region extraction unit 302 to extract, from the teacher signal, a rectangle that includes the acquired specified region 501 for the learning data 500, as an inclusive region 502 (step S602).

[0058] Next, the server 101 performs a process of masking the difference region 520, which is the encapsulated region 502 minus the specified region 501, with a fixed brightness value 550 to generate a mask region 530 using the mask processing unit 303, and generates a mask encapsulated region 503 consisting of the specified region 501 and the mask region 530 (step S603).

[0059] Next, the server 101 calculates a luminance histogram 504 for the masked region 503 by the statistical information calculation unit 304, and outputs a residual histogram 506 excluding the fixed frequency 505 (step S604).

[0060] Next, the server 101 uses the acquired histogram 506 as an explanatory variable and the state constituting the teacher signal as a target variable to perform learning using the judgment model learning unit 305, and generates a judgment model by machine learning that judges the state using the histogram 506 (step S605).

[0061] <Fig. 7 Status determination device> 7 is a flowchart illustrating an example of a state determination process procedure by the client 102 (state determination device) according to the embodiment 1. The client 102 acquires the analysis target data 500 from the sensor 103 by the analysis target data acquisition unit 310 (step S700).

[0062] Next, the client 102 assigns the designated area 501 acquired from the area designation unit 301 to the analysis object data 500 acquired from the analysis object data acquisition unit 310 by the designated area reproduction unit 311 (step S701).

[0063] Next, the client 102 causes the included region extraction unit 312 to extract, from the acquired analysis target data 500, a rectangular region that includes the acquired specified region 501 as an included region 502 (step S702).

[0064] Next, the client 102 uses the mask processing unit 313 to mask the difference region 520, which is the encapsulated region 502 minus the specified region 501, with a fixed brightness value 550 to produce a mask region 530, and generates a mask encapsulated region 503 consisting of the specified region 501 and the mask region 530 (step S703).

[0065] Next, the client 102 calculates a luminance histogram 504 for the masked region 503 by the statistical information calculation unit 314, and outputs a residual histogram 506 excluding the fixed frequency 505 (step S704).

[0066] Next, the client 102 uses the histogram 506 as an input of the determination model generated in step S605 by the determination model inference unit 315 to determine the state of the specified region 501 in the analysis target data 500 (step S705).

[0067] Next, if the inference result of step S705 is a predetermined inference result, the client 102 transmits the inference result to the patrol lamp 106 through the inference result output unit 316 (step S706). The predetermined inference result is, for example, a specific classification class (e.g., class C in FIG. 5) that is in a state in which additional control is required. Note that the client 102 may transmit the inference result to the patrol lamp 106 even if the inference result is not a specific classification class.

[0068] Next, the client 102 determines whether to end the acquisition of the analysis target data from the sensor 103 (step S707), and if the acquisition is not to be ended (step S707: No), the process returns to step S700. On the other hand, if the acquisition is to be ended (step S707: Yes), the state determination process of the client 102 is ended.

[0069] <Fig. 8 Status notification device> 8 is a flowchart showing an example of a status notification process procedure by the patrol lamp 106 (status notification device) according to the first embodiment. The patrol lamp 106 receives the inference result of step S705 from the client 102 by the inference result receiving unit 320. The patrol lamp 106 turns on the patrol lamp 106 according to the received inference result to indicate that additional control is required (step S801). The patrol lamp 106 turns on differently for each inference result.

[0070] As described above, according to the first embodiment, in state judgment for an inspection target having a complex shape, a specified region 501 to be inspected and a differential region 520 obtained by excluding the specified region 501 from an included region 502 that includes the specified region 501 are masked, a histogram 504 is calculated for a mask included region 503 consisting of the specified region 501 and the mask region 530, and a judgment model is generated in which the remaining histogram 506 excluding a fixed frequency 505 is used as a feature.

[0071] This makes it possible to exclude the differential region 520 as a fixed frequency 505, and therefore makes it possible to accurately determine the state of the specified region 501 even if it has a complex shape.

[0072] The client 102 transmits the inference result of step S705 to the patrol lamp 106 (step S706), but may transmit the result to a communication terminal such as a tablet communicatively connected to the network 105. In this case, the communication terminal can notify the user of the communication terminal of the status of the analysis target data by executing a specific screen display process or audio output process according to the received inference result. EXAMPLES

[0073] The second embodiment will be described with a focus on the differences from the first embodiment. The same reference numerals are used to designate the same parts as the first embodiment, and the description thereof will be omitted.

[0074] <Figure 9: Example of functional configuration of state determination system 100> 9 is a block diagram showing an example of a functional configuration of a state determination system 100 according to Example 2. The learning process performed by the server 101 (learning device) in Example 1 is performed by the client 102 in Example 2. As a result, the inclusive region extraction unit 302 is integrated into the inclusive region extraction unit 312, the mask processing unit 303 is integrated into the mask processing unit 313, and the statistical information calculation unit 304 is integrated into the statistical information calculation unit 314.

[0075] When outputting the designated region to the inclusive region extraction unit 312, the region designation unit 301 adds current processing information indicating that the process is a learning process.

[0076] When outputting the designated region to the inclusive region extraction unit 312, the designated region reproduction unit 311 adds current processing information indicating that the designated region is an inference process.

[0077] The encapsulated region extraction unit 312 and the mask processing unit 313 perform the same processes as those in the first embodiment.

[0078] The statistical information calculation unit 314 calculates a luminance histogram for the acquired mask-included region, and if the current processing information is a learning process, outputs the calculated histogram to the judgment model learning unit 305, and if the current processing information is an inference process, outputs the calculated histogram to the judgment model inference unit 315.

[0079] In this way, according to the second embodiment, by performing the learning process at the client 102, it becomes possible to consolidate similar processes that were previously performed at the server 101 and the client 102, thereby simplifying the system and enabling stand-alone status determination. EXAMPLES

[0080] Example 3 will be described focusing on the differences between Example 1 and Example 2. When the analysis target data is time-series data output at regular time intervals, the state of the continuous analysis target data is unlikely to change to a certain extent. In this case, if the inference result switches in a short time, it is determined to be an erroneous determination. For this reason, the state determination system 100 according to Example 3 can notify the inference result with higher accuracy by suppressing such erroneous determination. Note that the same reference numerals are used for the points common to Examples 1 and 2, and the description thereof will be omitted.

[0081] <Figure 10 Example of functional configuration of state determination system 100> 10 is a block diagram showing an example of a functional configuration of a state determination system 100 according to a third embodiment. In the third embodiment, an inference result consecutive value counting unit 1000 is newly added to the client 102. The determination model inference unit 315 outputs the inference result to the inference result consecutive value counting unit 1000 regardless of the content of the inference result (for example, any of class A to class C in FIG. 5).

[0082] The inference result consecutive value counting unit 1000 has a counter. When the inference result output from the determination model inference unit 315 for each analysis target data 500 is the same as the inference result output immediately before, the inference result consecutive value counting unit 1000 counts up the counter by 1. When the value of the counter (count value) becomes equal to or greater than a predetermined count value, the inference result consecutive value counting unit 1000 outputs the inference result to the inference result output unit 316.

[0083] The inference result consecutive value counting unit 1000 resets the counter value to 0 when it acquires an inference result whose count value is different from the inference result output immediately before.

[0084] The predetermined count value is a fixed value set in advance, but may be changeable depending on the frame rate of the analysis target data 500 set in the sensor 103. For example, the predetermined count value is a value obtained by multiplying a predetermined unit period by the frame rate. The client 102 may change the predetermined count value so that the higher the frame rate, the larger it becomes.

[0085] The inference result output unit 316 receives the inference result from the inference result consecutive value count unit 1000 and transmits the received inference result to the inference result receiving unit 320 in the patrol lamp 106.

[0086] When the judgment model inference unit 315 outputs an inference result that switches below a predetermined count value, the inference result consecutive value count unit 1000 resets the counter value at the timing of switching of the inference result. Therefore, the inference result consecutive value count unit 1000 does not output the inference result to the inference result output unit 316, and the inference result output unit 316 does not receive the inference result from the inference result consecutive value count unit 1000. This suggests the possibility that the learned judgment model is not able to correctly judge the state.

[0087] Therefore, when the inference result output unit 316 does not obtain an inference result for a predetermined time or longer, it outputs a judgment model anomaly signal indicating an anomaly in the judgment model to the inference result receiving unit 320 to notify the anomaly of the judgment model. Here, the predetermined time is a time set in advance.

[0088] The inference result output unit 316 starts measuring the time during which an inference result has not been obtained from the inference result consecutive value count unit 1000 when the first analysis target data 500 is obtained. The initial value of the unobtained time is 0. After the start of measuring the unobtained time, when an inference result is obtained from the inference result consecutive value count unit 1000, the unobtained time is reset to 0 and the measurement of the unobtained time is restarted from 0.

[0089] Next, an example of the functional configuration of the patrol lamp 106 will be described. The patrol lamp 106 receives the inference result from the inference result output unit 316 by the inference result receiving unit 320. The patrol lamp 106 turns on the patrol lamp according to the received inference result. The patrol lamp 106 turns on differently for each inference result.

[0090] <Fig. 11 Status determination device> 11 is a flowchart illustrating an example of a state determination process procedure by the client 102 (state determination device) according to the embodiment 3. After step S705, the client 102 executes an inference result consecutive value count control by the inference result consecutive value count unit 1000 (step S1100).

[0091] Specifically, for example, if the inference result output from the judgment model inference unit 315 is the same inference result as the inference result obtained last time, the client 102 increments the counter value by 1 using the inference result consecutive value counting unit 1000, and if the inference result is different, resets the counter value to 0.

[0092] In step S1101, if the count value of the inference result consecutive value count unit 1000 is greater than or equal to a predetermined count value (step S1101: Yes), the client 102 proceeds to processing of step S1103, and if the count value is less than the predetermined count value (step S1101: No), the client 102 proceeds to processing of step S1102.

[0093] Next, the client 102 transmits the inference result output from the inference result consecutive value count unit 1000 to the patrol lamp 106 via the inference result output unit 316 (step S1103).

[0094] In step S1102, the client 102 determines whether the unobtained time is equal to or longer than a predetermined time through the inference result output unit 316. If the unobtained time is equal to or longer than the predetermined time (step S1102: Yes), the process proceeds to step S1104, and if the unobtained time is not equal to or longer than the predetermined time (step S1102: No), the process proceeds to step S707.

[0095] Next, the client 102 transmits a determination model anomaly signal indicating the determination model anomaly to the patrol lamp 106 by the inference result output unit 316 (step S1104).

[0096] In this way, according to the third embodiment, in the condition judgment of an inspection target having a complex shape, it is possible to eliminate erroneous inference results output within a short period of time, and also to perform a condition judgment regarding the accuracy of the judgment model. EXAMPLES

[0097] The fourth embodiment will be described with a focus on the differences between the first embodiment, the second embodiment, and the third embodiment. The same reference numerals are used to designate the same parts as the first embodiment to the third embodiment, and the description thereof will be omitted.

[0098] <Figure 12 Example of functional configuration of state determination system 100> 12 is a block diagram showing an example of a functional configuration of a state determination system 100 according to a fourth embodiment. In the fourth embodiment, margin inclusive area generating units 1200 and 1201 are newly added. The margin inclusive area generating unit 1200 generates a margin area that is the outer edge of the specified area 501. Specifically, the margin inclusive area generating unit 1200 shrinks the specified area 501 inward from the boundary thereof, and generates a margin area that is the difference between the specified area 501 before and after the shrinking.

[0099] The shape of the margin area is, for example, similar to the shape of the specified area 501. If the specified area 501 is the surface of a body of water, the margin area corresponds to the edge of that surface of the water. Therefore, the width of the margin area toward the center of the specified area 501 is less than half the distance from the outer edge of the specified area 501 to the center of the specified area 501.

[0100] Moreover, the margin inclusive area generating unit 1200 outputs the margin area to the statistical information calculating unit 304 , and the margin inclusive area generating unit 1201 outputs the margin area to the statistical information calculating unit 314 .

[0101] <Figure 13 Example of pre-learning processing and pre-inference processing> 13 is an explanatory diagram showing an example of a learning pre-processing and an example of an inference pre-processing according to the fourth embodiment. The learning pre-processing is a process executed by the area designation unit 301, the designated area reproduction unit 311, the inclusive area extraction unit 302, the mask processing unit 303, the margin inclusive area generation unit 1200, and the statistical information calculation unit 304. The inference pre-processing is a process executed by the analysis target data acquisition unit 310, the designated area reproduction unit 311, the inclusive area extraction unit 312, the mask processing unit 313, the margin inclusive area generation unit 1201, and the statistical information calculation unit 314.

[0102] The margin inclusive region generating units 1200 and 1201 copy the mask inclusive region 503. Here, for convenience of explanation, the mask inclusive region 503 is copied, but a margin region may be generated in the mask inclusive region 503 without copying the mask inclusive region 503.

[0103] The margin inclusive region generating units 1200 and 1201 generate a margin specified region 1301 by moving each vertex of the specified region 501 toward the center of the destination mask included region 1303. The margin inclusive region generating units 1200 and 1201 generate a region between the specified region 501 and the margin specified region 1301 as a margin region 1302.

[0104] Margin area 1302 is the result of subtracting margin specified area 1301 from specified area 501. Therefore, the luminance value inside margin specified area 1301 is 0. If fixed luminance value 550 of mask area 530 is 0, the luminance value of margin specified area 1301 will also be fixed luminance value 550. If fixed luminance value 550 is not 0, it does not match the luminance value of margin specified area 1301, so margin inclusive area generating units 1200 and 1201 may perform mask processing on margin specified area 1301 so that it has the same luminance value as fixed luminance value 550.

[0105] The margin inclusive area generation units 1200 and 1201 output the mask inclusive area 503 to the statistical information calculation units 304 and 314. The margin inclusive area generation units 1200 and 1201 also output the destination mask inclusive area 1303 including the margin area 1302 as the margin inclusive area 1303 to the statistical information calculation units 304 and 314. The remaining area of ​​the margin inclusive area 1303 excluding the margin area 1302 is the margin specified area 1301 and the mask area 530.

[0106] As shown in the first embodiment, the statistical information calculation units 304 and 314 calculate a histogram 504 from the mask encapsulated region 503, and output a residual histogram 506 excluding a fixed frequency 505 to the judgment model learning unit 305 and the judgment model inference unit 315, respectively.

[0107] Moreover, the statistical information calculation units 304 and 314 calculate a histogram 1304 from the margin inclusive area 1303, and output a residual histogram 1306 excluding the fixed frequency 1305 to the judgment model learning unit 305 and the judgment model inference unit 315, respectively. The fixed frequency 1305 is the number of pixels in the mask area 530 and the margin specified area 1301.

[0108] The judgment model learning unit 305 performs learning using the histograms 506, 1306 acquired for each identical training data 500 as explanatory variables and a state constituting a teacher signal together with the training data 500 as a target variable, and generates, by machine learning, a judgment model that judges the state based on the histograms 506, 1306. The judgment model learning unit 305 outputs the judgment model generated by learning to the judgment model inference unit 315.

[0109] The judgment model inference unit 315 uses the histograms 506, 1306 acquired for each identical analysis target data 500 as input to the judgment model acquired from the judgment model learning unit 305, and judges the state of the specified region 501 in the analysis target data 500. The judgment model inference unit 315 outputs the inference result to the inference result output unit 316.

[0110] There are cases where state changes near the boundary of the specified region 501 are greater than those in the center of the specified region 501. In such cases, by generating a determination model using histogram 1306 near the boundary as information extracted from histogram 506, it is possible to accurately determine the state by taking into account state changes near the boundary where the changes are greater.

[0111] <Figure 14 Learning process> 14 is a flowchart showing a detailed example of a procedure of a learning process by the server 101 (learning device) according to the embodiment 4. The server 101 reduces the designated area 501 inward by the margin inclusion area generating unit 1200, and generates a margin area 1302 that is the difference between the designated area 501 before and after the reduction (step S1400).

[0112] The server 101 calculates the histograms 506 and 1306 by the statistical information calculation unit 304 (step S1404).

[0113] Then, the server 101 performs learning by the judgment model learning unit 305 using the histograms 506 and 1306 as explanatory variables and the state as a target variable, and generates a judgment model that judges the state using the histograms 506 and 1306 by machine learning (step S1405).

[0114] <Fig. 15 Status determination device> 15 is a flowchart showing an example of a state determination process procedure by the client 102 (state determination device) according to the embodiment 4. The client 102 reduces the designated area 501 inward by the margin inclusion area generating unit 1201, and generates a margin area 1302 which is the difference between the designated area 501 before and after the reduction (step S1500).

[0115] The server 101 calculates the histograms 506 and 1306 by the statistical information calculation unit 314 (step S1504).

[0116] Then, the server 101 uses the histograms 506 and 1306 as inputs to a judgment model by the judgment model inference unit 315, judges the state of the specified region 501 in the analysis target data 500, and outputs the inference result.

[0117] In this way, according to the fourth embodiment, the state can be determined with high accuracy by taking into consideration information near the boundary where the state change is large.

[0118] In the fourth embodiment, the margin inclusive area generating units 1200 and 1201 shrink the specified area 501 inward and generate the margin area 1302 which is the difference between the specified area 501 before and after the shrinking, but the generation of the margin area 1302 is not limited to this processing content. For example, the margin inclusive area generating units 1200 and 1201 may expand the specified area 501 outward from its boundary, set the area that includes the expanded specified area 501 as the inclusive area 502, and generate the area that is the difference between the specified area 501 before and after the shrinking as the margin area 1302.

[0119] In this way, according to the above-mentioned first to fourth embodiments, the state of the inspection target can be determined with high accuracy regardless of the complexity of the shape of the area to be inspected.

[0120] 4, the data structure of the teacher signal storage folder 400 in which image data serving as explanatory variables are stored in subfolders with their states as folder names has been described, but the problem of complicated generation and management of teacher signals exists not only in the techniques of Examples 1 to 4 but also in other machine learning, so the data structure shown in Fig. 4 can be applied to other machine learning not only in the techniques of Examples 1 to 4. In this case, the explanatory variables stored in the subfolders are not limited to image data, and may be text data or audio data.

[0121] As described above, the condition determination systems according to the first to fourth embodiments can determine the condition of an inspection object with high accuracy even if the inspection object has a complex shape.

[0122] Moreover, the state determination systems of the above-mentioned first to fourth embodiments can also be configured as follows (1) to (21).

[0123] (1) In a learning system having a processor that executes a program and a storage device that stores the program, the processor executes the following processes: a first acquisition process that acquires a teacher signal which is a combination of first image data (learning data) of an inspection target and a first state of the inspection target; a region designation process that designates a region within the first image data acquired by the first acquisition process; a first extraction process that extracts from the first image data a first included region that contains the designated region designated by the region designation process; a first mask process that masks a first remaining region excluding the designated region from the first included region extracted by the first extraction process; a first calculation process that calculates first statistical information indicating a frequency of color information in a first mask included region consisting of the designated region and a first mask region in which the first remaining region is masked by the first mask process; and a learning process that learns the judgment model based on output data output as a result of inputting the first statistical information of the designated region, among the first statistical information calculated by the first calculation process, into a judgment model and the first state.

[0124] This makes it possible to generate a judgment model that can determine state changes in color information of the inspection object with high accuracy by excluding information related to the shape of the inspection object.

[0125] (2) In the learning system of (1) above, in the area designation process, the processor receives designation of a plurality of vertices and generates the designated area as a polygon connecting the plurality of vertices.

[0126] This makes it possible to improve the degree of freedom in area designation.

[0127] (3) In the learning system of (1) above, in the area designation process, the processor performs area designation by segmenting the first image data.

[0128] This makes it possible to automate the area designation.

[0129] (4) In the learning system of (1) above, in the area designation process, the processor designates multiple areas within the first image data, in the first extraction process, the processor extracts a first included area from the first image data that contains the multiple designated areas, and in the first masking process, the processor masks a first remaining area of ​​the first included area excluding the multiple designated areas.

[0130] This makes it possible to apply the method to cases where the region of interest within the object of inspection is separated into multiple locations.

[0131] (5) A learning system according to (1) above, wherein the processor executes a second acquisition process for acquiring second image data of the inspection object, a second extraction process for extracting a second encapsulated area containing the designated area from the second image data acquired by the second acquisition process, a second mask process for masking a second residual area excluding the designated area from the second encapsulated area extracted by the second extraction process, a second calculation process for calculating second statistical information indicating the frequency of color information in a second mask encapsulated area consisting of the designated area and a second mask area in which the second residual area is masked by the second mask process, an inference process for inferring a second state of the inspection object by inputting the second statistical information of the designated area among the second statistical information calculated by the second calculation process into the judgment model, and an output process for outputting the inference result by the inference process.

[0132] This makes it possible to judge with high accuracy the state change of color information in the inspection object using a judgment model that excludes information related to the shape of the inspection object.

[0133] (6) In the learning system of (6) above, the processor executes the second acquisition process, the second extraction process, the second mask process, the second calculation process, the inference process, and the output process for each of the second image data input in time series according to the time series.

[0134] This makes it possible to repeatedly perform highly accurate judgments using a judgment model that excludes information related to the shape of the inspection object.

[0135] (7) In the learning system of (6) above, in the output process, when the inference process outputs an inference result in which the second state is the same a predetermined number of times in succession, the processor outputs the inference result.

[0136] This makes it possible to improve the inference accuracy of the judgment model.

[0137] (8) In the learning system of (7) above, in the output process, if the inference result is not output for a predetermined time, the processor outputs an abnormality signal indicating an abnormality in the judgment model.

[0138] This makes it possible to detect erroneous inferences made in a short period of time by a judgment model that outputs an inference result in which the second state is the same for less than a predetermined number of consecutive times, and to encourage re-learning of the judgment model.

[0139] (9) In the learning system of (1) above, the processor executes a first generation process to generate a first margin containing area in which a third residual area of ​​the first mask containing area other than a margin area that is the outer edge of the specified area is masked, and in the first calculation process, the processor calculates first margin statistical information indicating the frequency of color information in the first margin containing area generated by the first generation process, and in the learning process, the processor learns the judgment model based on output data output as a result of inputting the first statistical information of the specified area and the first margin statistical information of the margin area among the first margin statistical information into a judgment model, and based on the first state.

[0140] This makes it possible to learn with high accuracy a judgment model that infers changes in the state of color information in the object to be inspected, taking into account changes in the state near the boundary that forms the edge of the specified area.

[0141] (10) In the learning system of (9) above, the processor includes a second acquisition process for acquiring second image data of the inspection target, a second extraction process for extracting a second inclusion region containing the designated region from the second image data acquired by the second acquisition process, a second mask process for masking a second remaining region excluding the designated region from the second inclusion region extracted by the second extraction process, and a second margin inclusion region in which a fourth remaining region other than the margin region is masked from a second mask inclusion region consisting of the designated region and a second mask region in which the second remaining region is masked by the second mask process. a second generation process that calculates second statistical information indicating the frequency of color information in the second mask inclusion region and second margin statistical information indicating the frequency of color information in the second margin inclusion region generated by the second generation process; an inference process that infers a second state of the inspection object by inputting the second statistical information of the specified region among the second statistical information calculated by the second calculation process and the second statistical information of the margin region among the second margin statistical information calculated by the second calculation process into a judgment model; and an output process that outputs the inference result by the inference process.

[0142] This makes it possible to learn with high accuracy a judgment model that infers changes in the state of color information in the object to be inspected, while excluding information related to the shape of the object to be inspected and taking into account changes in state near the boundaries that form the edges of the specified area.

[0143] (11) In the learning system of (10) above, the processor executes the second acquisition process, the second extraction process, the second mask process, the second generation process, the second calculation process, the inference process, and the output process for each of the second image data input in time series according to the time series.

[0144] This makes it possible to repeatedly perform highly accurate judgments using a judgment model that eliminates information about the shape of the inspection object and takes into account state changes near the boundary that forms the edge of the specified region.

[0145] (12) In the learning system of (11) above, in the output process, when the inference process outputs an inference result in which the second state is the same a predetermined number of times in succession, the processor outputs the inference result.

[0146] This makes it possible to improve the inference accuracy of the judgment model.

[0147] (13) In the learning system of (12) above, in the output process, if the inference result is not output for a predetermined time, the processor outputs an abnormality signal indicating an abnormality in the judgment model.

[0148] This makes it possible to detect erroneous inferences made in a short period of time by a judgment model that outputs an inference result in which the second state is the same for less than a predetermined number of consecutive times, and to encourage re-learning of the judgment model.

[0149] (14) An inference system having a processor that executes a program and a storage device that stores the program, the inference system being capable of accessing a judgment model that has been trained using first statistical information of a designated area in first image data of an inspection target as an explanatory variable and a first state of the inspection target as a target variable, and the processor executes an acquisition process that acquires second image data (data to be analyzed) of the inspection target, an extraction process that extracts a second encapsulated area that contains the designated area from the second image data acquired by the acquisition process, a masking process that masks a remaining area excluding the designated area from the encapsulated area extracted by the extraction process, a calculation process that calculates second statistical information that indicates the frequency of color information in a mask encapsulated area consisting of the designated area and a mask area in which the remaining area is masked by the masking process, an inference process that infers a second state of the inspection target by inputting the second statistical information of the designated area from the second statistical information calculated by the calculation process into the judgment model, and an output process that outputs an inference result by the inference process.

[0150] This makes it possible to judge with high accuracy the state change of color information in the inspection object using a judgment model that excludes information related to the shape of the inspection object.

[0151] (15) In the inference system of (14) above, the processor executes the acquisition process, the extraction process, the mask process, the calculation process, the inference process, and the output process for each of the second image data input in time series according to the time series.

[0152] This makes it possible to repeatedly perform highly accurate judgments using a judgment model that excludes information related to the shape of the inspection object.

[0153] (16) In the inference system of (15) above, in the output process, when the inference process outputs an inference result in which the second state is the same a predetermined number of times consecutively, the processor outputs the inference result.

[0154] This makes it possible to improve the inference accuracy of the judgment model.

[0155] (17) In the inference system of (16) above, in the output process, if the inference result is not output for a predetermined time, the processor outputs an abnormality signal indicating an abnormality in the judgment model.

[0156] This makes it possible to detect erroneous inferences made in a short period of time by a judgment model that outputs an inference result in which the second state is the same for less than a predetermined number of consecutive times, and to encourage re-learning of the judgment model.

[0157] (18) In the inference system of (14) above, the judgment model is trained using first statistical information of the designated area and first margin statistical information of a margin area that is the outer edge of the designated area as explanatory variables and a first state of the inspection object as a target variable, and the processor executes a generation process to generate a margin inclusion area in which a remaining area other than the margin area of ​​the mask inclusion area is masked, and in the calculation process, the processor calculates second statistical information indicating the frequency of color information in the mask inclusion area and second margin statistical information indicating the frequency of color information in the margin inclusion area generated by the generation process, and in the inference process, the processor infers a second state of the inspection object by inputting the second statistical information of the designated area among the second statistical information calculated by the calculation process and the second statistical information of the margin area among the second margin statistical information calculated by the calculation process into a judgment model.

[0158] This makes it possible to learn with high accuracy a judgment model that infers changes in the state of color information in the object to be inspected, while excluding information related to the shape of the object to be inspected and taking into account changes in state near the boundaries that form the edges of the specified area.

[0159] (19) In the inference system of (18) above, the processor executes the acquisition process, the extraction process, the mask process, the generation process, the calculation process, the inference process, and the output process for each of the second image data input in time series, according to the time series.

[0160] This makes it possible to repeatedly perform highly accurate judgments using a judgment model that eliminates information about the shape of the inspection object and takes into account state changes near the boundary that forms the edge of the specified region.

[0161] (20) In the inference system of (19) above, in the output process, when the inference process outputs an inference result in which the second state is the same a predetermined number of times consecutively, the processor outputs the inference result.

[0162] This makes it possible to improve the inference accuracy of the judgment model.

[0163] (21) In the inference system of (20) above, in the output process, the processor outputs an abnormality signal indicating an abnormality in the judgment model if the inference result is not output for a predetermined time.

[0164] This makes it possible to detect erroneous inferences made in a short period of time by a judgment model that outputs an inference result in which the second state is the same for less than a predetermined number of consecutive times, and to encourage re-learning of the judgment model.

[0165] The present invention is not limited to the above-described embodiments, and includes various modified examples and equivalent configurations within the spirit of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the configurations described. Also, a part of the configuration of one embodiment may be replaced with a configuration of another embodiment. Also, a configuration of another embodiment may be added to a configuration of one embodiment. Also, a part of the configuration of each embodiment may be added to, deleted from, or replaced with another configuration.

[0166] In addition, each of the configurations, functions, processing units, processing means, etc. described above may be realized in hardware, for example by designing some or all of them as an integrated circuit, or may be realized in software by processor 201 interpreting and executing a program that realizes each function.

[0167] Information such as programs, tables, files, etc. that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC (Integrated Circuit) card, an SD card, or a DVD (Digital Versatile Disc).

[0168] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines that are necessary for implementation. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0169] 100 Status Judgment System 101 Server 102 Clients 103 Sensors 104 Teacher signal DB 106 Patrol Lamp 201 Processor 202 Storage Devices 203 Input Devices 204 Output Device 205 Communication Interface 300 Teacher signal acquisition section 301 Area specification section 310 Sensor information acquisition unit 311 Specified area playback section 302,312 Inclusion region extraction unit 303,313 Mask processing section 304,314 Statistical information calculation section 305 Decision Model Learning Unit 315 Decision Model Inference Unit 316 Inference result output part 320 Inference result receiving unit 1000 Inference result consecutive value count section 1200,1201 Margin inclusion area generation section

Claims

1. A learning system having a processor that executes a program and a storage device that stores the program, The processor, a first acquisition process for acquiring a teacher signal which is a combination of first image data of an object to be inspected and a first state of the object to be inspected; an area designation process for designating an area within the first image data acquired by the first acquisition process; a first extraction process for extracting, from the first image data, a first inclusive region that includes the designated region designated by the region designation process; a first masking process for masking a first remaining region excluding the designated region from the first encapsulated region extracted by the first extraction process; a first calculation process for calculating first statistical information indicating a frequency of color information in a first mask included region including the designated region and a first mask region obtained by masking the first residual region by the first mask process; a learning process for learning the judgment model based on output data output as a result of inputting the first statistical information of the specified region, among the first statistical information calculated by the first calculation process, into a judgment model, and based on the first state; A learning system comprising:

2. The learning system according to claim 1 , In the area designation process, the processor receives designation of a plurality of vertices and generates the designated area as a polygon by connecting the plurality of vertices. A learning system characterized by:

3. The learning system according to claim 1 , In the region designation process, the processor performs region designation by segmenting the first image data. A learning system characterized by:

4. The learning system according to claim 1 , In the region designation process, the processor performs a plurality of region designations within the first image data; In the first extraction process, the processor extracts a first including region that includes the plurality of designated regions from the first image data; In the first masking process, the processor masks a first remaining region of the first contained region excluding the plurality of designated regions. A learning system characterized by:

5. The learning system according to claim 1 , The processor, A second acquisition process for acquiring second image data of the inspection object; a second extraction process for extracting a second including region that includes the designated region from the second image data acquired by the second acquisition process; a second masking process for masking a second remaining region excluding the designated region from the second encapsulated region extracted by the second extraction process; a second calculation process for calculating second statistical information indicating a frequency of color information in a second mask included region including the designated region and a second mask region obtained by masking the second remaining region by the second mask process; an inference process of inferring a second state of the inspection object by inputting second statistical information of the specified region, among second statistical information calculated by the second calculation process, into the judgment model; an output process for outputting an inference result obtained by the inference process; A learning system comprising:

6. The learning system according to claim 5, the processor executes, for each of the second image data input in time series, the second acquisition process, the second extraction process, the second mask process, the second calculation process, the inference process, and the output process in accordance with the time series; A learning system characterized by:

7. A learning system according to claim 6, In the output process, when an inference result in which the second state is the same is output a predetermined number of times consecutively by the inference process, the processor outputs the inference result. A learning system characterized by:

8. A learning system according to claim 7, In the output process, when the inference result is not output for a predetermined time, the processor outputs an abnormality signal indicating an abnormality in the judgment model. A learning system characterized by:

9. The learning system according to claim 1 , The processor, A first generation process is executed to generate a first margin inclusion region in which a third remaining region other than a margin region that is an outer edge of the specified region is masked from the first mask inclusion region; In the first calculation process, the processor calculates first margin statistical information indicating a frequency of color information in the first margin inclusion area generated by the first generation process; In the learning process, the processor learns the judgment model based on output data output as a result of inputting first statistical information of the specified region and the first margin statistical information of the margin region out of the first margin statistical information into a judgment model, and based on the first state. A learning system characterized by:

10. A learning system according to claim 9, The processor, A second acquisition process for acquiring second image data of the inspection object; a second extraction process for extracting a second including region that includes the designated region from the second image data acquired by the second acquisition process; a second masking process for masking a second remaining region excluding the designated region from the second encapsulated region extracted by the second extraction process; a second generation process for generating a second margin inclusive region in which a fourth remaining region other than the margin region is masked out of a second mask inclusive region consisting of the designated region and a second mask region in which the second remaining region is masked out by the second mask process; a second calculation process for calculating second statistical information indicating a frequency of color information in the second mask inclusion region and second margin statistical information indicating a frequency of color information in the second margin inclusion region generated by the second generation process; an inference process for inferring a second state of the inspection object by inputting, into a judgment model, second statistical information of the designated region out of the second statistical information calculated by the second calculation process and second statistical information of the margin region out of the second margin statistical information calculated by the second calculation process; an output process for outputting an inference result obtained by the inference process; A learning system comprising:

11. The learning system according to claim 10, the processor executes, for each of the second image data input in time series, the second acquisition process, the second extraction process, the second mask process, the second generation process, the second calculation process, the inference process, and the output process in accordance with the time series; A learning system characterized by:

12. The learning system according to claim 11, In the output process, when an inference result in which the second state is the same is output a predetermined number of times consecutively by the inference process, the processor outputs the inference result. A learning system characterized by:

13. A learning system according to claim 12, In the output process, when the inference result is not output for a predetermined time, the processor outputs an abnormality signal indicating an abnormality in the judgment model. A learning system characterized by:

14. A reasoning system having a processor for executing a program and a storage device for storing the program, a judgment model that is trained using first statistical information of a specified region in first image data of an object to be inspected as an explanatory variable and a first state of the object to be inspected as a response variable is accessible; The processor, an acquisition process for acquiring second image data of the inspection object; an extraction process of extracting a second inclusive region that includes the specified region from the second image data acquired by the acquisition process; a masking process for masking a remaining area, excluding the designated area, from the encapsulation area extracted by the extraction process; a calculation process for calculating second statistical information indicating a frequency of color information in a mask included region that is composed of the designated region and a mask region in which the residual region is masked by the mask process; an inference process of inferring a second state of the inspection object by inputting second statistical information of the specified region, among second statistical information calculated by the calculation process, into the judgment model; an output process for outputting an inference result obtained by the inference process; 1. An inference system comprising:

15. 15. A reasoning system as claimed in claim 14, comprising: the processor executes the acquisition process, the extraction process, the mask process, the calculation process, the inference process, and the output process for each of the second image data input in time series, in accordance with the time series; 1. An inference system comprising:

16. 16. A reasoning system as claimed in claim 15, comprising: In the output process, when an inference result in which the second state is the same is output a predetermined number of times consecutively by the inference process, the processor outputs the inference result.

1. An inference system comprising:

17. 17. A reasoning system as claimed in claim 16, comprising: In the output process, when the inference result is not output for a predetermined time, the processor outputs an abnormality signal indicating an abnormality in the judgment model.

1. An inference system comprising:

18. 15. A reasoning system as claimed in claim 14, comprising: the judgment model is trained using first statistical information of the designated area and first margin statistical information of a margin area that is an outer edge of the designated area as explanatory variables and using a first state of the inspection object as a response variable; The processor, A generation process is performed to generate a margin inclusion region in which a remaining region other than the margin region is masked from the mask inclusion region; In the calculation process, the processor calculates second statistical information indicating a frequency of color information in the mask inclusion region and second margin statistical information indicating a frequency of color information in the margin inclusion region generated by the generation process; In the inference process, the processor infers a second state of the inspection object by inputting, into a judgment model, second statistical information of the designated region among the second statistical information calculated by the calculation process and second statistical information of the margin region among the second margin statistical information calculated by the calculation process.

1. An inference system comprising:

19. 20. A reasoning system as claimed in claim 18, comprising: the processor executes the acquisition process, the extraction process, the mask process, the generation process, the calculation process, the inference process, and the output process for each of the second image data input in time series, in accordance with the time series; 1. An inference system comprising:

20. 20. A reasoning system as claimed in claim 19, comprising: In the output process, when an inference result in which the second state is the same is output a predetermined number of times consecutively by the inference process, the processor outputs the inference result.

1. An inference system comprising:

21. 21. A reasoning system as claimed in claim 20, comprising: In the output process, when the inference result is not output for a predetermined time, the processor outputs an abnormality signal indicating an abnormality in the judgment model.

1. An inference system comprising:

22. A learning method executed by a learning system having a processor that executes a program and a storage device that stores the program, comprising: The processor, a first acquisition process for acquiring a teacher signal which is a combination of first image data of an object to be inspected and a first state of the object to be inspected; an area designation process for designating an area within the first image data acquired by the first acquisition process; a first extraction process for extracting, from the first image data, a first inclusive region that includes the designated region designated by the region designation process; a first masking process for masking a first remaining region excluding the designated region from the first encapsulated region extracted by the first extraction process; a first calculation process for calculating first statistical information indicating a frequency of color information in a first mask included region including the designated region and a first mask region obtained by masking the first residual region by the first mask process; a learning process for learning the judgment model based on output data output as a result of inputting the first statistical information of the specified region, among the first statistical information calculated by the first calculation process, into a judgment model, and based on the first state; A learning method comprising:

23. An inference method executed by an inference system having a processor that executes a program and a storage device that stores the program, comprising: a judgment model that is trained using first statistical information of a specified region in first image data of an object to be inspected as an explanatory variable and a first state of the object to be inspected as a response variable is accessible; The processor, an acquisition process for acquiring second image data of the inspection object; an extraction process of extracting a second inclusive region that includes the specified region from the second image data acquired by the acquisition process; a masking process for masking a remaining area, excluding the designated area, from the encapsulation area extracted by the extraction process; a calculation process for calculating second statistical information indicating a frequency of color information in the designated region and a mask included region formed by masking the residual region by the mask process; an inference process of inferring a second state of the inspection object by inputting second statistical information of the specified region, among second statistical information calculated by the calculation process, into the judgment model; an output process for outputting an inference result obtained by the inference process; 2. An inference method comprising:

Images