Information processing device and information processing program

Abstract

To facilitate optimization of image processing models. [Solution] An information processing device according to an embodiment includes an annotation unit, a generation unit, and a learning unit. The annotation unit annotations images captured by a camera using an image processing model. The generation unit generates, as a training dataset, a group of images in which the ratio of annotations added by the annotation unit satisfies a predetermined variance condition. The learning unit performs training of the image processing model using the training dataset generated by the generation unit.

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an information processing program. [Background technology]

[0002] Conventionally, there are image processing systems that analyze images captured by a camera. For example, in the image processing system, various types of image processing are performed using an image processing model that has been machine-learned to distinguish a target from an image (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2024-98450 Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional techniques have room for improvement in the detection accuracy of image processing models. For example, it is necessary to optimize the image processing model depending on the environment in which the camera is installed, but it is not easy to set up an appropriate training dataset.

[0005] The present invention has been made in view of the above, and has an object to provide an information processing device and an information processing program that can facilitate optimization of an image processing model. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, an information processing device according to the present invention includes an annotation unit, a generation unit, and a learning unit. The annotation unit annotations images captured by a camera using an image processing model. The generation unit generates, as a training dataset, a group of images in which the ratio of annotations added by the annotation unit satisfies a predetermined variance condition. The learning unit performs training of the image processing model using the training dataset generated by the generation unit. [Effects of the Invention]

[0007] According to the present invention, it is possible to easily optimize an image processing model. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to an embodiment. [Figure 2] FIG. 2 is a block diagram of the camera according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of information stored in the image data storage unit according to the embodiment. [Figure 4] FIG. 4 is a flowchart showing a processing procedure executed by the camera according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] The camera 50 (an example of an information processing device) according to the embodiment described below includes an annotation unit 542 that adds annotations to images captured by the camera 50 using an image processing model, a generation unit 543 that generates, as a training dataset, a group of images in which the ratio of annotations added by the annotation unit 542 satisfies a predetermined variance condition, and a learning unit 544 that performs training of the image processing model using the training dataset generated by the generation unit 543.

[0010] Furthermore, the generating unit 543 according to the embodiment described below generates a training data set based on the distribution conditions set by the administrator.

[0011] Furthermore, the generating unit 543 according to the embodiment described below generates a training dataset based on other annotations added by other models that are image processing models different from the image processing model.

[0012] Furthermore, the image processing model according to the embodiment described below is a model that annotates the type of skeleton of a person appearing in an image and whether or not the person has a skeleton, and the generation unit 543 generates a group of images as a training dataset, in which the combination of the type of skeleton and the presence or absence of a skeleton satisfies the dispersion condition.

[0013] In addition, the information processing program according to the embodiment described below includes an annotation procedure for annotating images captured by a camera 50 using an image processing model, a generation procedure for generating a group of images as a training dataset in which the ratio of annotations added by the annotation procedure satisfies a predetermined variance condition, and training of the image processing model using the training dataset generated by the generation procedure.

[0014] (Embodiment) Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the following embodiments do not limit the technology disclosed by the present invention. Furthermore, the same components in each embodiment are designated by the same reference numerals, and redundant explanations will be omitted.

[0015] First, an information processing system will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an information processing system according to an embodiment. The information processing system S according to the embodiment is introduced into various facilities, including public facilities and private facilities, and provides various services based on images captured by each camera 50.

[0016] For example, the information processing system S analyzes images captured by each camera 50, analyzes the flow of people, etc., and provides the analysis results to clients (for example, public facilities, private facilities, etc.).

[0017] 1, the information processing system S includes a server device 1 and a plurality of cameras 50. Also, as shown in FIG. 1, the server device 1 and the plurality of cameras 50 are connected via a predetermined network N. The camera 50 corresponds to an example of an information processing device according to the embodiment.

[0018] The server device 1 controls the entire information processing system S. Furthermore, the server device 1 provides various services such as people flow analysis to clients based on images captured by each camera 50.

[0019] The camera 50 is installed in various facilities and captures images. The camera 50 also has an image processing model and performs image processing using the image processing model. For example, the image processing model is a machine-learned model (AI) that detects targets such as people.

[0020] Incidentally, it is preferable that the image processing model uses part of the analysis results as learning data and updates the learning data periodically. However, if the learning data is updated one after another, the balance of the learning data will be lost, and the detection accuracy may decrease for events that are unlikely to occur in the installation environment of the camera 50 due to the low learning frequency.

[0021] More specifically, for example, a camera 50 in a busy corridor can collect sufficient learning data, so the accuracy of person detection can be maintained constant, whereas a camera 50 in a less busy corridor cannot collect sufficient learning data, so the accuracy of person detection may decrease.

[0022] As described above, when performing continuous learning of an image processing model, the selection of learning data is important. In the present disclosure, a process for optimizing the selection of learning data is performed. As a result, the present disclosure can facilitate the optimization of the image processing model.

[0023] Next, an example of the configuration of a camera 50 according to an embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram of the camera 50 according to the embodiment. As shown in Fig. 2, the camera 50 has a communication unit 51, an imaging unit 52, a storage unit 53, and a control unit 54.

[0024] The communication unit 51 is implemented by a predetermined communication circuit such as a NIC (Network Interface Card), and performs data communication with the server device 1 and the like via a communication network such as Ethernet (registered trademark) or LAN.

[0025] The imaging unit 52 is implemented by a CMOS (Complementary Metal Oxide Semiconductor) image sensor, a CCD (Charge Coupled Device) image sensor, etc. The imaging unit 52 captures an image of an imaging area to generate an image, and outputs the image to the control unit .

[0026] The storage unit 53 is implemented by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. In the example shown in FIG. 2, the storage unit 53 has an image data storage unit 531 and an image processing model storage unit 532.

[0027] The image data storage unit 531 stores image data. The image data is data of an image captured by the imaging unit 52. Fig. 3 is a diagram showing an example of information stored in the image data storage unit 531 according to the embodiment.

[0028] 3, the image data storage unit 531 stores information such as "imaging date and time," "image data," and "annotation information" in association with each other. The "imaging date and time" item stores the imaging date and time of the corresponding image data.

[0029] The "image data" field stores the data of the corresponding image. The "annotation information" field stores the annotations added to the corresponding image. Details of annotations will be described later.

[0030] Returning to the explanation of Fig. 2, the image processing model storage unit 532 will be explained. The image processing model storage unit 532 stores an image processing model. The image processing model is a model that has been machine-learned to detect a detection target (for example, a person, etc.). Note that the image processing model storage unit 532 may be configured to store a plurality of image processing models.

[0031] Next, the control unit 54 will be described. The control unit 54 is a controller that controls the entire camera 50. For example, the control unit 54 can be implemented by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). Alternatively, the control unit 54 may be implemented by an integrated circuit such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0032] 2, the control unit 54 includes an acquisition unit 541, an assignment unit 542, a generation unit 543, and a learning unit 544. The acquisition unit 541 acquires an image captured by the imaging unit 52. The acquisition unit 541 also stores the acquired image in the image data storage unit 531.

[0033] The adding unit 542 adds annotations to the image captured by the imaging unit 52. The adding unit 542 inputs the image to the image processing model, and adds the output result of the image processing model to the image as an annotation.

[0034] For example, the annotation unit 542 uses, as an image processing model, a skeleton detection model that detects the skeleton of a person appearing in an image to add annotations. The skeleton detection model adds coordinate information at which the skeleton is detected and an identification number that indicates the type of skeleton. Note that, since an image usually shows some skeleton parts missing, the annotation unit 542 may add information indicating the missing skeleton parts to the image as an annotation.

[0035] The annotation unit 542 may also be configured to add annotations using a model other than the skeleton detection model. For example, the annotation unit 542 may add annotations using an image processing model such as VQA (Visual Question Answering). In such a case, for example, the annotation unit 542 may add annotations for items such as the person's clothing, standing position, posture, and physique.

[0036] The generating unit 543 generates, as a training dataset, a group of images in which the ratio of annotations assigned by the assigning unit 542 satisfies a predetermined distribution condition. The distribution condition can be set, for example, by an administrator, and the generating unit 543 generates, as a training dataset, a group of images that satisfy the distribution condition set by the administrator.

[0037] For example, when generating a learning dataset based on annotations of skeletal detection information, the generation unit 543 selects images that are missing for each skeletal point, such as 100 images where the right wrist is missing and 100 images where the right elbow is missing, and selects the selected group of images as the learning dataset.

[0038] By generating such a training dataset, it is possible to generate a balanced training dataset. Note that, for example, if an administrator desires to avoid false positives in a particular situation, the administrator can set the system to always add images of that situation to the training dataset.

[0039] In such a case, the generation unit 543 generates a training dataset including images of annotations corresponding to the situation specified by the administrator. In this way, the generation unit 543 generates a group of images that satisfy the distribution conditions specified by the administrator as a training dataset, thereby enabling training of an image processing model that meets the administrator's purpose.

[0040] Regarding the generation of the training dataset, the generation unit 543 may generate the training dataset including images that have been used as a training dataset in the past, or may generate the training dataset using only new images. When generating a training dataset using new images, the generation unit 543 may preferentially use the new images.

[0041] The learning unit 544 performs learning of the image processing model using the learning dataset generated by the generation unit 543. For example, the learning unit 544 performs learning of the image processing model at a predetermined timing, such as when the learning dataset is generated. Note that the learning by the learning unit 544 can use a known method. The camera 50 can use the latest image processing model that has undergone continuous learning.

[0042] Next, a processing procedure executed by the camera 50 according to the embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the processing procedure executed by the camera 50 according to the embodiment. Note that the processing shown below is executed by the control unit 54 of the camera 50 each time an image is acquired.

[0043] 4, the camera 50 first acquires an image (step S101) and then adds an annotation to the acquired image (step S102). Then, the camera 50 determines whether the annotation added to the image satisfies a distribution condition (step S103).

[0044] If the camera 50 determines that the distribution condition is satisfied (step S103; Yes), it adds the corresponding image to the training data set (step S104). Subsequently, the camera 50 determines whether the training data set is complete (step S105).

[0045] When it is determined that the learning data set is complete (step S105; Yes), the camera 50 performs a learning process for the image processing model using the learning data set (step S106), and ends the process.

[0046] Furthermore, if the camera 50 determines in step S103 that the dispersion condition is not satisfied (step S103; No), or if the camera 50 determines in step S105 that the learning data set is not complete (step S105; No), it skips the subsequent processing and terminates the processing.

[0047] As described above, the camera 50 (an example of an information processing device) according to the embodiment includes an annotation unit 542 that adds annotations to images captured by the camera 50 using an image processing model, a generation unit 543 that generates, as a training dataset, a group of images in which the ratio of annotations added by the annotation unit 542 satisfies a predetermined variance condition, and a learning unit 544 that performs training of the image processing model using the training dataset generated by the generation unit 543.

[0048] Therefore, the camera 50 according to the embodiment can facilitate optimization of the image processing model.

[0049] Although an embodiment of the present invention has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. This embodiment can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as the inventions described in the claims and their equivalents. [Explanation of symbols]

[0050] 1. Server device 50 cameras 51 Communications Department 52 Imaging unit 53 Storage section 54 Control Unit 531 Image data storage unit 532 Image processing model memory unit 541 Acquisition Department 542 Granting Department 543 Generation part 544 Learning Department S Information Processing System

Claims

1. an annotation unit that uses an image processing model to an image captured by a camera; a generation unit that generates, as a training dataset, a group of images in which the ratio of the annotations assigned by the assignment unit satisfies a predetermined variance condition; a learning unit that uses the learning data set generated by the generation unit to learn the image processing model; An information processing device comprising:

2. The generation unit generating the learning dataset based on the distribution conditions set by an administrator; The information processing device according to claim 1 .

3. The generation unit generating the learning dataset based on other annotations provided by another model that is an image processing model different from the image processing model; The information processing device according to claim 1 .

4. The image processing model is a model to which the annotations are added indicating the type of skeleton of a person appearing in the image and whether or not the skeleton is present, The generation unit generating, as the training dataset, the group of images in which the combination of the skeleton type and the presence or absence of the skeleton satisfies the distribution condition; The information processing device according to claim 1 .

5. An annotation procedure for an image captured by a camera using an image processing model; a generation step of generating, as a training dataset, a group of images in which the ratio of the annotations added by the annotation adding step satisfies a predetermined distribution condition; a learning procedure for performing learning of the image processing model using the learning dataset generated by the generation procedure; An information processing program that causes a computer to execute the following.

Images