Information processing device and information processing program

Abstract

This invention provides an information processing device and an information processing program that optimize the image processing model of the information processing device to suit the actual environment in which the camera is installed. [Solution] In an information processing system in which a server device and multiple cameras are connected by a predetermined network, the camera 50, which functions as an information processing device, comprises a generation unit 543 and a learning unit 544. The generation unit 543 generates teaching data concerning the state of the object to be detected detected from the image using a teaching model. The learning unit 544 uses the teaching data generated by the generation unit 543 to train the image processing model to be trained.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and an information processing program.

Background Art

[0002] Conventionally, there is an image processing system that analyzes an image captured by a camera. For example, there is known a technique for determining whether or not an object shown in an image is in a specific state by using an AI technique called VQA (Visual Question Answering) (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art, there is room for improvement in optimizing the image processing model according to the actual environment where the camera is installed.

[0005] The present invention has been made in view of the above, and an object thereof is to provide an information processing apparatus and an information processing program capable of optimizing an image processing model according to the actual environment where a camera is installed.

Means for Solving the Problems

[0006] In order to solve the above-described problems and achieve the object, an information processing apparatus according to the present invention includes a generation unit and a learning unit. The generation unit generates teaching data regarding the state of a detection target detected from an image using a teaching model. The learning unit performs learning of an image processing model to be learned using the teaching data generated by the generation unit.

Effects of the Invention

[0007] According to the present invention, the image processing model can be optimized to suit the actual environment in which the camera is installed. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 shows an example of the configuration of an information processing system according to an embodiment. [Figure 2] Figure 2 is a block diagram of the camera according to the embodiment. [Figure 3] Figure 3 shows an example of information stored in the image data storage unit according to this embodiment. [Figure 4] Figure 4 is a conceptual diagram of the teaching data according to the embodiment. [Figure 5] Figure 5 is a flowchart showing the processing procedure performed by the camera according to this embodiment. [Modes for carrying out the invention]

[0009] The camera 50 (an example of an information processing device) according to the embodiment described below comprises a generation unit 543 that generates teaching data relating to the state of a detection target detected from an image using a teaching model, and a learning unit 544 that performs learning of an image processing model to be learned using the teaching data generated by the generation unit 543.

[0010] The image processing model according to the embodiment described below is a model with a lower processing load than the teaching model.

[0011] In the embodiment described below, the learning unit 544 uses images with different detection results for the teaching model and the image processing model as teaching data to train the image processing model.

[0012] The camera 50 according to the embodiment described below includes a tracking unit 542 that tracks the detected target based on the detection result of the image processing model.

[0013] The teaching model and image processing model according to the embodiments described below are models that generate natural language answers to natural language questions about images.

[0014] The information processing program according to the embodiment described below causes a computer to execute a generation procedure that generates teaching data relating to the state of the object to be detected detected from an image using a teaching model, and a learning procedure that uses the teaching data generated by the generation procedure to train the image processing model to be trained.

[0015] (Embodiment) Embodiments of the present invention will be described below with reference to the drawings. Note that the embodiments described below are not intended to limit the technology disclosed by the present invention. Furthermore, the same parts are denoted by the same reference numerals in each embodiment, and redundant descriptions are omitted.

[0016] First, the information processing system will be explained using Figure 1. Figure 1 is a diagram showing an example configuration of the information processing system according to the embodiment. The information processing system S according to the embodiment can be introduced into various facilities, such as factories.

[0017] For example, the information processing system S provides various services based on the analysis results of images captured by each camera 50. For instance, if the information processing system S is introduced into a factory, it can determine whether workers are in a dangerous condition (for example, whether they are wearing equipment that violates the safety manual) and provide services such as notifying factory managers of the determination results. The information processing system S can also provide various services such as pedestrian flow analysis based on images captured by each camera 50.

[0018] As shown in Figure 1, the information processing system S comprises a server device 1 and a plurality of cameras 50. Also, as shown in Figure 1, the server device 1 and the plurality of cameras 50 are connected by a predetermined network N.

[0019] The server device 1 controls the entire information processing system S. Further, the server device 1 provides various services based on the images captured by each camera 50.

[0020] The camera 50 is installed in various facilities and captures images. Further, the camera 50 has an image processing model and performs image processing using the image processing model. For example, the image processing model is a model (AI) that is machine-learned to detect detection targets such as people, and is, for example, VQA (Visual Question Answering).

[0021] Note that the camera 50 corresponds to an example of the information processing device according to the embodiment. Further, the camera 50 may be configured integrally with, for example, a lighting device installed on the ceiling. For example, in such a case, the camera 50 can capture an image looking down from the ceiling.

[0022] By the way, the image processing model is required to be optimized for each actual environment where the camera 50 is installed. As a method for optimizing the image processing model, there is a method of re-learning the image processing model using the images captured in the actual environment as learning data.

[0023] In re-learning the image processing model, it is necessary to prepare appropriate teaching data. Based on such a background, in the present disclosure, the process for appropriately generating the teaching data will be described using the drawings after FIG. 2.

[0024] Next, a configuration example of the camera 50 according to the embodiment will be described using FIG. 2. FIG. 2 is a block diagram of the camera 50 according to the embodiment. As shown in FIG. 2, the camera 50 includes a communication unit 51, an imaging unit 52, a storage unit 53, and a control unit 54.

[0025] The communication unit 51 is implemented by, for example, a predetermined communication circuit such as a NIC (Network Interface Card), and performs data communication with the server device 1 and the like via a communication network such as Ethernet (registered trademark) or LAN.

[0026] The imaging unit 52 is implemented using a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor. The imaging unit 52 captures an image area, generates an image, and outputs it to the control unit 54.

[0027] The memory unit 53 is implemented using, for example, semiconductor memory elements such as RAM (Random Access Memory) or flash memory, or storage devices such as hard disks or optical discs. In the example shown in Figure 2, the memory unit 53 includes an image data storage unit 531, an image processing model storage unit 532, a teaching model storage unit 533, and a follow-up model storage unit 534.

[0028] The image data storage unit 531 stores image data. The image data is data of images captured by the imaging unit 52. Figure 3 is a diagram showing an example of the information stored in the image data storage unit 531 according to this embodiment.

[0029] As shown in Figure 3, the image data storage unit 531 stores information such as "imaging date and time," "image data," and "analysis results" in a manner that associates them with each other. The "imaging date and time" item stores the imaging date and time of the corresponding image data.

[0030] The "Image Data" field stores the data of the corresponding image. The "Analysis Results" field stores the analysis results of the image processing applied to the corresponding image.

[0031] Returning to the explanation of Figure 2, let's describe the image processing model storage unit 532. The image processing model storage unit 532 stores the image processing model. The image processing model is a so-called VQA, which is a model that generates natural language answers to natural language questions about an image.

[0032] The teaching model storage unit 533 stores the teaching model. The teaching model is a VQA, similar to the image processing model. In this disclosure, the teaching model is a model with higher accuracy than the image processing model. In other words, the image processing model has a lower processing load than the teaching model, and the teaching model has higher processing accuracy than the image processing model.

[0033] In other words, while image processing models are inferior to training models in terms of image processing accuracy, they have the advantage of being able to perform image processing more quickly than training models.

[0034] The tracking model storage unit 534 stores tracking models. A tracking model is a model (AI) that has been trained to track objects in an image. For example, the tracking model determines whether the objects detected in preceding and succeeding frames are the same, based on the similarity of the color information of the objects in each image.

[0035] Next, the control unit 54 will be described. The control unit 54 is a controller that controls the entire camera 50. For example, the control unit 54 can be implemented by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). Alternatively, the control unit 54 may be implemented by an integrated circuit such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0036] In the example shown in Figure 2, the control unit 54 includes an analysis unit 541, a tracking unit 542, a generation unit 543, and a learning unit 544. The analysis unit 541 performs various analysis processes using an image processing model on the image captured by the imaging unit 52.

[0037] For example, the analysis unit 541 analyzes whether an object (e.g., a person) in the image captured by the imaging unit 52 is in a pre-set state. More specifically, the analysis unit 541 inputs the image and question text to an image processing model, which is a VQA, and obtains the answer from the image processing model.

[0038] For example, the question text might be something like, "Is the face visible in the picture?" or "Is that person wearing glasses?", which are questions about the state of the object being detected (for example, a person such as a worker).

[0039] For such a question, the VQA image processing model will output a "Yes" or "No" response. The analysis unit 541 performs this process for each image, thereby analyzing the state of each object depicted in each image.

[0040] The tracking unit 542 tracks the detected object based on the detection result of the image processing model. For example, the tracking unit 542 uses a tracking model to track each detected object (e.g., a person) based on the position of the detected object detected by the analysis unit 541.

[0041] For example, the tracking results of each detected target by the tracking unit 542 are provided to administrators, etc., as a pedestrian flow analysis.

[0042] The generation unit 543 generates teaching data relating to the state of the object to be detected from the image using a teaching model. In this disclosure, the generation of teaching data by the generation unit 543 is performed in parallel with the analysis by the analysis unit 541.

[0043] In other words, in this disclosure, the retraining of the image processing model is performed in parallel with the analysis processing by the analysis unit 541. This makes it possible to improve the accuracy of the analysis by the image processing model while providing services based on the analysis results of the analysis unit 541.

[0044] Here, we will explain the teaching data generated by the generation unit 543 using Figure 4. Figure 4 is a conceptual diagram of the teaching data according to this embodiment. For example, as shown in Figure 5, the same image and the same question text are input to the image processing model and the teaching model.

[0045] The image processing model and the teaching model will each output "Yes" or "No" as their answer to the question. If the answer from the teaching model is considered correct, then if the image processing model and the teaching model output different answers, the answer from the image processing model will be considered incorrect.

[0046] The generation unit 543 generates images as teaching data that result in different answers from the image processing model and the teaching model. In other words, the generation unit 543 generates images that have been misdetected or misrecognized by the image processing model as teaching data.

[0047] This allows the generation unit 543 to efficiently generate a set of teaching data for images that are prone to being misdetected by the image processing model.

[0048] Returning to the explanation of Figure 2, let's describe the learning unit 544. The learning unit 544 performs training of the image processing model using the training data generated by the generation unit 543. For example, the learning unit 544 retrains the image processing model when the amount of training data reaches a predetermined value. Various known methods can be used to retrain the image processing model.

[0049] Thus, in this disclosure, the image processing model is retrained using images that have been misdetected or otherwise misdetected by the image processing model as training data in the environment in which the camera 50 is actually installed. As a result, according to the camera 50 (an example of an information processing device) of the embodiment, the image processing model can be optimized to match the actual environment in which the camera 50 is installed.

[0050] Next, the processing procedure performed by the camera 50 according to the embodiment will be described using Figure 5. Figure 5 is a flowchart showing the processing procedure performed by the camera 50 according to the embodiment.

[0051] As shown in Figure 5, the camera 50 first acquires an image captured by the imaging unit 52 (step S101). Next, the camera 50 performs image processing using an image processing model (step S102).

[0052] Next, camera 50 performs image processing using the teaching model (step S103) and determines whether the answers from both the image processing model and the teaching model match (step S104).

[0053] If camera 50 determines that the answers from both sides do not match (step S104; No), it adds the corresponding image to the teaching data (step S105). If camera 50 determines in step S104 that the answers from both sides do match (step S104; Yes), it proceeds to the process in step S101.

[0054] Next, camera 50 determines whether or not the administrator has performed an operation to end the learning process (step S106). If the administrator has performed an operation to end the process (step S106; Yes), camera 50 terminates the process. If, in the determination in step S106, the administrator has not performed an operation to end the process (step S106; No), camera 50 proceeds to the process in step S101.

[0055] As described above, the camera 50 (an example of an information processing device) according to the embodiment comprises a generation unit 543 that generates teaching data relating to the state of a detection target detected from an image using a teaching model, and a learning unit 544 that performs learning of an image processing model to be learned using the teaching data generated by the generation unit 543.

[0056] Therefore, according to the camera 50 of this embodiment, the image processing model can be optimized to suit the actual environment in which the camera 50 is installed.

[0057] While embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be carried out in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0058] 1 Server device 50 Cameras 51 Communications Department 52 Imaging Department 53 Memory section 54 Control Unit 531 Image data storage unit 532 Image Processing Model Storage Unit 533 Teaching Model Memory Unit 534 Follow-up Model Memory Unit 541 Analysis Department 542 Follow-up section 543 Generation part 544 Learning Department S Information Processing System

Claims

1. A generation unit that generates teaching data about the state of the target detected from the image using a teaching model; A learning unit that performs training on an image processing model to be trained using the teaching data generated by the generation unit; An information processing device equipped with the following:

2. The aforementioned image processing model is This model has a lower processing load than the aforementioned teaching model. The information processing apparatus according to claim 1.

3. The aforementioned learning unit, The image processing model is trained using the images, for which the detection results differ for the teaching model and the image processing model, respectively, as teaching data. The information processing apparatus according to claim 1.

4. A tracking unit that tracks the detected target based on the detection result of the image processing model; The information processing apparatus according to claim 1, comprising the following:

5. The teaching model and the image processing model are This is a model that generates natural language answers to natural language questions about images. The information processing apparatus according to claim 1.

6. A generation procedure for generating teaching data about the state of the target detected from an image using a teaching model; A learning procedure which involves training an image processing model to be trained using the teaching data generated by the generation procedure; An information processing program that causes a computer to execute an action.

Images