Information processing device and information processing program

Abstract

To facilitate optimization of image processing models. [Solution] An information processing device according to an embodiment includes a map generation unit, a data generation unit, and a learning unit. The map generation unit generates, for each camera, a feature map that indicates the distribution of features of images captured by the cameras. The data generation unit generates, for each camera, a training data set for an image processing model based on the feature map generated by the generation unit. The learning unit uses the training data set generated by the data generation unit to train the image processing model for each camera.

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an information processing program. [Background technology]

[0002] Conventionally, there are image processing systems that analyze images captured by a camera. For example, in the image processing system, various types of image processing are performed using an image processing model that has been machine-learned to distinguish a target from an image (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2024-98450 Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional techniques have room for improvement in the detection accuracy of image processing models. For example, it is necessary to optimize the image processing model depending on the environment in which the camera is installed, but it is not easy to set up an appropriate training dataset.

[0005] The present invention has been made in view of the above, and has an object to provide an information processing device and an information processing program that can facilitate optimization of an image processing model. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, an information processing device according to the present invention includes a map generation unit, a data generation unit, and a learning unit. The map generation unit generates, for each camera, a feature map that indicates the distribution of features of images captured by the cameras. The data generation unit generates, for each camera, a training data set for an image processing model to be installed in the camera based on the feature map generated by the map generation unit. The learning unit uses the training data set generated by the data generation unit to train the image processing model for each camera. [Effects of the Invention]

[0007] According to the present invention, it is possible to easily optimize an image processing model. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to an embodiment. [Figure 2] FIG. 2 is a block diagram of the information processing apparatus according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of information stored in the image data storage unit according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of information stored in a camera information storage unit according to the embodiment. [Figure 5] FIG. 5 is a flowchart showing a processing procedure executed by the information processing apparatus according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] The information processing device 1 according to the embodiment described below includes a map generation unit 42 that generates, for each camera 50, a feature map that indicates the distribution of features of images captured by the camera 50; a data generation unit 43 that generates, for each camera 50, a training data set for an image processing model to be installed in the camera 50 based on the feature map generated by the map generation unit 42; and a learning unit 44 that performs training of the image processing model for each camera 50 using the training data set generated by the data generation unit 43.

[0010] Furthermore, the map generating unit 42 according to the embodiment described below generates a feature amount map based on annotations assigned to each image.

[0011] Furthermore, the map generating unit 42 according to the embodiment described below generates a feature amount map using skeleton detection information of a person appearing in an image as an annotation.

[0012] Furthermore, the data generating unit 43 according to the embodiment described below generates a learning data set with the amount of data adjusted according to the distribution state of the feature amount map.

[0013] Furthermore, the data generation unit 43 according to the embodiment described below generates a training dataset using images captured by a plurality of cameras 50.

[0014] Furthermore, the information processing program according to the embodiment described below causes a computer to execute a map generation procedure for generating, for each camera 50, a feature map showing the distribution of features of images captured by the camera 50; a data generation procedure for generating, for each camera 50, a training data set for an image processing model to be installed in the camera 50 based on the feature map generated by the map generation procedure; and a learning procedure for performing training of the image processing model for each camera 50 using the training data set generated by the data generation procedure.

[0015] (Embodiment) Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the following embodiments do not limit the technology disclosed by the present invention. Furthermore, the same components in each embodiment are designated by the same reference numerals, and redundant explanations will be omitted.

[0016] First, an information processing system will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an information processing system according to an embodiment. The information processing system S according to the embodiment is introduced into various facilities, including public facilities and private facilities, and provides various services based on images captured by each camera 50.

[0017] For example, the information processing system S analyzes images captured by each camera 50, analyzes the flow of people, etc., and provides the analysis results to clients (for example, public facilities, private facilities, etc.).

[0018] 1, the information processing system S includes an information processing device 1 and a plurality of cameras 50. Also, as shown in FIG. 1, the information processing device 1 and the plurality of cameras 50 are connected by a predetermined network N.

[0019] The information processing device 1 controls the entire information processing system S. Furthermore, the information processing device 1 provides various services such as people flow analysis to clients based on images captured by each camera 50.

[0020] In addition, in the present disclosure, the information processing device 1 executes various processes related to continuous learning of the image processing model installed in each camera 50.

[0021] Cameras 50 are installed in various facilities and capture images. Cameras 50 also have an image processing model and perform image processing using the image processing model. For example, the image processing model is a machine-learned model (AI) that detects targets such as people.

[0022] Incidentally, it is preferable that the image processing model uses part of the analysis results as learning data and updates the learning data periodically. However, if the learning data is updated one after another, the balance of the learning data will be lost, and the detection accuracy may decrease for events that are unlikely to occur in the installation environment of the camera 50 due to the low learning frequency.

[0023] More specifically, for example, a camera 50 installed in a busy corridor can acquire sufficient learning data, so the accuracy of person detection can be maintained constant, whereas a camera 50 installed in a less busy corridor cannot acquire sufficient learning data, so the accuracy of person detection may decrease.

[0024] As described above, when performing continuous learning of an image processing model, the selection of learning data is important. In the present disclosure, a process for optimizing the selection of learning data is performed. This makes it possible to easily optimize the image processing model.

[0025] Next, a configuration example of the information processing device 1 according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram of the information processing device 1 according to the embodiment. As shown in Fig. 2, the information processing device 1 includes a communication unit 2, a storage unit 3, and a control unit 4.

[0026] The communication unit 2 is implemented by a predetermined communication circuit such as a NIC (Network Interface Card), and performs data communication with the camera 50 and the like via a communication network such as Ethernet (registered trademark) or LAN.

[0027] The storage unit 3 is implemented by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. In the example shown in FIG. 2, the storage unit 3 has an image data storage unit 31 and a camera information storage unit 32.

[0028] The image data storage unit 31 stores image data. The image data is data of an image captured by the camera 50. Fig. 3 is a diagram showing an example of information stored in the image data storage unit 31 according to the embodiment.

[0029] 3, the image data storage unit 31 stores information such as "camera ID," "capture date and time," "image data," and "annotation information" in association with each other. The "camera ID" item stores an identifier for identifying each camera 50.

[0030] The "imaging date and time" field stores the imaging date and time of the corresponding image data. The "image data" field stores the data of the corresponding image. The "annotation information" field stores the annotations added to the corresponding image. Details of annotations will be described later.

[0031] Returning to the explanation of FIG. 2, the camera information storage unit 32 will be described. The camera information storage unit 32 stores camera information. The camera information includes various information related to each camera 50. FIG. 4 is a diagram showing an example of information stored in the camera information storage unit 32 according to the embodiment. As shown in FIG. 4, the camera information storage unit 32 stores information on items such as "camera ID," "feature map," and "image processing model" in association with each other.

[0032] The "camera ID" item stores an identifier for identifying each camera 50. The "feature map" item stores a feature map of the camera 50 identified by the corresponding camera ID.

[0033] The "image processing model" item stores the image processing model installed in the camera 50 identified by the corresponding camera ID.

[0034] Returning to the explanation of Fig. 2, the control unit 4 will be described. The control unit 4 is a controller that controls the entire information processing device 1. For example, the control unit 4 can be implemented by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). Alternatively, the control unit 4 may be implemented by an integrated circuit such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0035] 2, the control unit 4 includes an image processing unit 41, a map generating unit 42, a data generating unit 43, and a learning unit 44. The image processing unit 41 performs various image processing on the images captured by each camera 50.

[0036] The image processing unit 41 adds annotations to images captured by each camera 50. The image processing unit 41 adds annotations to images using an image processing model. Specifically, the image processing unit 41 inputs an image to the image processing model and adds annotations output from the image processing model to the image. Note that the annotations here may be, for example, feature vectors indicating the features of the image.

[0037] At this time, the image processing unit 41 may add annotations specialized for the learning data (hereinafter referred to as specialized annotations). The specialized annotations are added by skeleton detection.

[0038] For example, the image processing unit 41 inputs an image into an image processing model that detects a person's skeleton from the image, detects the skeleton of the person appearing in the image, and assigns specialized annotations to the image by associating the type of skeleton detected and the presence or absence of a skeleton with each coordinate in the image.

[0039] This makes it possible to accumulate information such as, for example, where people are cut off in the image, or where each person is in a particular pose. The specialized annotation may include, for example, the person's orientation, the angle at which the image was taken, their clothing, etc. In such a case, for example, the image processing unit 41 can add specialized annotations using an image processing model such as VQA (Visual Question Answering).

[0040] The map generation unit 42 generates a feature map for each camera 50, which indicates the distribution of features of images captured by the cameras 50. For example, the map generation unit 42 generates a feature map for each camera 50 by embedding feature vectors indicating the features of images captured by the cameras 50 into a distributed representation space. For example, the map generation unit 42 converts images including specialized annotations into feature vectors, and embeds the feature vectors into the distributed representation space to generate a feature map for each camera 50.

[0041] Then, the map generation unit 42 uses the generated feature amount map to update the previous feature amount map registered in the camera information storage unit 32. When updating the feature amount map, the map generation unit 42 may update the feature amount map by adding a new feature amount to the previous feature amounts, or may delete the previous feature amounts and update the feature amount map using only the new feature amounts.

[0042] The data generation unit 43 generates a training data set for the image processing model for each camera 50 based on the feature map generated by the map generation unit 42. The data generation unit 43 generates the training data set by weighting each image according to the distribution state of the feature map.

[0043] Specifically, the data generating unit 43 assigns a higher weight to an image with dense feature amounts in the feature map and a lower weight to an image with sparse feature amounts. The data generating unit 43 sets a larger amount of data for an image with a heavy weighted feature amount and a smaller amount of data for an image with a light weighted feature amount.

[0044] Then, the data generating unit 43 generates a training data set by extracting images with data amounts corresponding to the respective weightings from the image data storage unit 31. In this case, the data generating unit 43 may generate the training data set using images captured by a plurality of cameras 50.

[0045] That is, the data generation unit 43 can efficiently generate a training data set by using images captured by multiple cameras 50. Furthermore, when weighting, the data generation unit 43 may perform correction according to the purpose of image processing. Note that such correction may be performed by an administrator.

[0046] The learning unit 44 uses the learning data set generated by the data generation unit 43 to perform learning of the image processing model for each camera 50. For example, the learning unit 44 performs learning of the image processing model installed in each camera 50 at a predetermined timing, such as when the learning data set is generated.

[0047] Note that a known method can be used for learning by the learning unit 44. Then, the learning unit 44 installs the image processing model that has completed learning into the corresponding camera 50. This allows each camera 50 to use the latest image processing model that has undergone continuous learning.

[0048] Next, a processing procedure executed by the information processing device 1 according to the embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the processing procedure executed by the information processing device 1 according to the embodiment. Note that the continuous learning process for one camera 50 will be described below.

[0049] 5, the information processing device 1 first generates a feature amount map indicating the distribution of feature amounts of an image captured by the camera 50 (step S101). Next, the information processing device 1 generates a learning dataset based on the feature amount map (step S102).

[0050] Next, the information processing device 1 executes a learning process for the image processing model using the learning data set (step S103), and ends the process.

[0051] As described above, the information processing device 1 according to the embodiment includes a map generation unit 42 that generates, for each camera 50, a feature map that indicates the distribution of features of images captured by the camera 50; a data generation unit 43 that generates, for each camera 50, a training data set for an image processing model to be installed in the camera 50 based on the feature map generated by the map generation unit 42; and a learning unit 44 that performs training of the image processing model for each camera 50 using the training data set generated by the data generation unit 43.

[0052] Therefore, the information processing device 1 according to the embodiment can easily optimize the image processing model.

[0053] Although an embodiment of the present invention has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. This embodiment can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as the inventions described in the claims and their equivalents. [Explanation of symbols]

[0054] 1. Information processing equipment 2. Communications Department 3 Storage section 4. Control Unit 31 Image data storage unit 32 Camera information storage unit 41 Image processing section 42 Map generation unit 43 Data Generation Unit 44 Learning Department 50 cameras S Information Processing System

Claims

1. a map generation unit that generates, for each camera, a feature amount map that indicates a distribution of feature amounts of images captured by the cameras; a data generation unit that generates, for each camera, a training data set for an image processing model to be mounted on the camera based on the feature map generated by the map generation unit; a learning unit that uses the learning data set generated by the data generation unit to learn the image processing model for each of the cameras; An information processing device comprising:

2. The map generation unit generating the feature map based on the annotations assigned to each of the images; The information processing device according to claim 1 .

3. The map generation unit generating the feature map using skeleton detection information of the person appearing in the image as the annotation; The information processing device according to claim 2 .

4. The data generation unit generating the learning dataset with the amount of data adjusted according to the distribution state of the feature map; The information processing device according to claim 1 .

5. The data generation unit generating the training dataset using the images captured by the plurality of cameras; The information processing device according to claim 1 .

6. a map generation step of generating, for each camera, a feature map indicating a distribution of feature values ​​of images captured by the cameras; a data generation step of generating, for each camera, a training data set for an image processing model to be mounted on the camera, based on the feature map generated by the map generation step; a learning step of learning the image processing model for each of the cameras using the learning data set generated by the data generation step; An information processing program that causes a computer to execute the following.

Images