Image classification device, image classification system, and image classification program

Abstract

This invention provides an image classification device that automatically sets classification criteria from recorded images to improve the efficiency of sorting photographs. [Solution] An image classification device for classifying and storing multiple input images, comprising: an image semantic identification means for identifying semantic information that can be estimated from an image and associating it with the image; a classification generation means for generating image classifications for grouping images by performing cluster analysis on semantic information corresponding to multiple images; a classification storage area setting means for setting a storage area corresponding to each image classification; and an image storage means for storing the images in the storage area.

Description

Technical Field

[0001] The present invention relates to an image classification device, an image classification system, and an image classification program.

Background Art

[0002] A system for appropriately sorting and storing images taken for recording is known. For example, a large number of images are recorded for the purpose of recording during construction work or at the completion of construction work. Using the large number of recorded photos, a completed drawing is created by classifying (sorting) them into a form suitable for a predetermined format. A system used for these series of operations is disclosed (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The technique disclosed in Patent Document 1 uses a blank ledger for classifying photos. The blank ledger is set and prepared in advance, and a large number of photos are classified by fitting the recorded images (photos) to this blank ledger. That is, in the prior art, it is necessary to prepare in advance the image sorting rules such as the prior setting of the blank ledger. That is, prior setting of sorting rules by the user is required, and there is a problem in automatically setting classification conditions from the images for recording and improving the efficiency of photo sorting.

[0005] An object of the present invention is to provide an image classification device that automatically sets classification conditions from images for recording and improves the efficiency of photo sorting.

Means for Solving the Problems

[0006] To solve the above problems, the present invention relates to an image classification device in one aspect, an image classification device that classifies and stores a plurality of input images, The aforementioned Semantic information that can be inferred from the image of The painting In the statue Make it work to identify Means for determining the meaning of an image, Corresponding to multiple aforementioned images The aforementioned semantic information Used Cluster analysis reveals that The aforementioned The system comprises: a division generation means for generating image divisions for grouping images; a classification storage area setting means for setting a storage area corresponding to each of the image divisions; and an image storage means for storing the images in the storage areas. The classification generation means generates the image classification and classification name of the image classification from the semantic information identified for the image based on a learning model generated using a machine learning process, and if there is classification history information in which the image semantic identification means has previously set the image classification based on semantic information generated for images relating to the same user, it corrects the generation results of the image classification and classification name by referring to the classification history information. It is characterized by the following. [Effects of the Invention]

[0007] According to the present invention, classification conditions can be automatically set from the recorded images to improve the efficiency of sorting photographs. [Brief explanation of the drawing]

[0008] [Figure 1] A block diagram showing an overview of an image classification system relating to one embodiment of the present invention. [Figure 2] Hardware configuration diagram of the image classification device according to this embodiment. [Figure 3] Hardware configuration diagram of the information processing device according to this embodiment. [Figure 4] A functional configuration diagram of the image classification device according to this embodiment. [Figure 5] Detailed diagram of the functional configuration of the image classification device according to this embodiment. [Figure 6] Detailed diagram of the functional configuration of the image classification device according to this embodiment. [Figure 7] A flowchart illustrating the image classification processing flow according to this embodiment. [Figure 8] This figure shows an example of a screen in an electronic book format that can display the results of the image classification processing according to this embodiment. [Modes for carrying out the invention]

[0009] The following examples will be explained with reference to the attached drawings. In the following explanation, the reference numerals in the drawings refer to the same elements. Furthermore, the embodiments are not limited to the following examples, and embodiments may include elements other than those shown in the drawings.

[0010] [First Embodiment] Figure 1 is a system configuration diagram showing an embodiment of the image classification system according to the present invention. As shown in Figure 1, the photo sorting system 1 is a system that utilizes cloud computing.

[0011] The photo sorting system 1 is configured to enable communication between a photo sorting device 10 (which functions as an image classification device), an image input device 20, and an image output device 30 via a communication network 40. The image input device 20 and the image output device 30 can also be hardware with similar functions housed in the same enclosure.

[0012] The photo sorting system 1 is constructed using the hardware resources of computers located on the Internet, a type of communication network 40, and its predetermined functions are realized by software technology executed on those hardware resources. The functional configuration of the photo sorting device 10 will be described later. The photo sorting device 10 identifies the classification of an image based on the image input device 20, such as a portable information terminal or a digital imaging device, and is equipped with a learning model that has learned sorting rules based on the identified classification through a machine learning process.

[0013] In the photo sorting system 1, a learning model installed in the photo sorting device 10 extracts the meaning of images from subjects contained in construction site images and identifies them as semantic information using natural language processing. In addition, it extracts items and content such as construction details as strings from images of construction blackboards containing construction-related information included in the construction site images. The photo sorting device 10 is equipped with a machine learning model that assigns classifications to construction photos based on this semantic information and strings and automatically generates sorting conditions.

[0014] The image input device 20 has a function of realizing image input to the photo sorting device 10. In the following description, the image input device 20 exemplifies a so-called tablet-type information terminal. Also, it has an information output function (including image display). Note that, in the following description, the image output device 30 exemplifies an information processing device represented by a so-called personal computer.

[0015] The hardware resources responsible for the input and output of information to the photo sorting device 10 according to the present embodiment are not limited to the image input device 20 and the image output device 30. Any form or the like is acceptable as long as it can realize the information processing functions described below.

[0016] The images input to the photo sorting device 10 are, for example, images used for recording the situation of a construction site, recording the materials used at the construction site, recording the construction content, etc. In this specification, the said images are referred to as "construction photos". To repeat, in the photo sorting device 10, information derivable from the subjects included in the construction photos is extracted using a learning model generated by machine learning. Then, by assigning a meaning in natural language to the extracted information and analyzing this meaning information, an appropriate classification for storing each construction photo is specified, and a storage area corresponding to the classification is automatically generated. Also, the photo sorting device 10 specifies an appropriate classification name for associating each construction image with the storage area based on the analysis result of the meaning information. Then, based on the correlation between the specified classification name and each storage area, the sorting process of the construction photos can be automatically performed.

[0017] The information contained in construction photographs includes, for example, notes on construction blackboards, construction locations, construction objects, construction materials, scales used to indicate construction methods, and qualification certificates of construction personnel, and refers to information that can be included in images recorded as construction photographs. In other words, the photo sorting system 1 uses a learning model in the photo sorting device 10 to extract and analyze semantic information contained in construction photographs, automatically generates classification rules (sometimes called classification conditions or sorting conditions) based on the semantic information related to the construction photographs that can be identified from the analysis results, and automatically sorts the construction photographs based on the classification rules. This makes it possible to classify and sort construction photographs flexibly and with high accuracy.

[0018] Furthermore, the photo sorting device 10 has a function to output the sorted photos in an electronic book format generated based on the request of the image output device 30.

[0019] [Example hardware configuration of photo sorting device 10] Figure 2 shows an example of the hardware configuration of the information processing device that constitutes the photo sorting device 10, which utilizes cloud computing technology. For example, the photo sorting device 10 has the following hardware resources.

[0020] As shown in Figure 2, the photo sorting device 10 has a processing unit 110 consisting of a CPU (Central Processing Unit). It also has a storage device 120 consisting of RAM (Random Access Memory), ROM (Read Only Memory), and SSD (Solid State Drive).

[0021] Furthermore, the photo sorting device 10 also includes an input device 130 that inputs data (such as construction images) received via a communication network into the arithmetic unit 110, and an output device 140 that outputs the results processed by the arithmetic unit 110.

[0022] Furthermore, the photo sorting device 10 has an interface (hereinafter referred to as "I / F150"), etc. A communication module 151 is connected to the I / F150. The communication module 151 is responsible for the function of enabling the photo sorting device 10 to communicate with a communication network 40 such as the Internet.

[0023] Furthermore, the photo sorting device 10 may have auxiliary devices in addition to the arithmetic unit 110, storage device 120, input device 130, and output device 140. Specifically, the information processing device may have auxiliary devices such as a GPU (Graphics Processing Unit) either externally or internally.

[0024] In Figure 2, the photo sorting device 10 is represented as a hardware resource housed in a single enclosure. However, the configuration in which the photo sorting device 10 is implemented is not limited to this configuration. For example, in this embodiment, the photo sorting device 10 may be configured to realize the same hardware resources as described above by combining multiple information processing devices. In this case, the number of hardware resources that realize the functions and the configuration of connection and combination are not limited, and it is sufficient if they can be configured as hardware resources capable of realizing the processing functions described later.

[0025] [Example hardware configuration of image input device 20] Figure 3(a) shows an example of the hardware configuration of an image input device 20, which serves as an information processing device for inputting construction images to a photo sorting device 10. As shown in Figure 3(a), the image input device 20 has the same hardware resources as the photo sorting device 10. Specifically, it has a CPU 210 as an arithmetic processing unit, a ROM 220 as a non-volatile memory unit, a RAM 230 as a volatile memory unit, and an SSD 240 for storing and saving input information. It also has an I / F 250 as an external interface, and the I / F 250 has a communication module 251, a camera 252, and a monitor 253.

[0026] The communication module 251 has the function of transmitting the read data to the photo sorting device 10. The data transmitted by the image input device 20 to the photo sorting device 10 (construction images as input data) is not limited to images captured by the camera 252, but may also be image files captured in advance by other imaging devices.

[0027] Camera 252 is an imaging means for photographing the construction blackboard, and takes photographs in response to the operation of the shutter on the monitor 253. The captured construction photographs are then transmitted to the photo sorting device 10 by the communication module 251.

[0028] In this embodiment, the image input device 20 is shown as an example of transmitting construction images as data files to the photo sorting device 10. As mentioned above, the construction images that become input data transmitted to the photo sorting device 10 are expected to contain a variety of subjects.

[0029] [Example of hardware configuration for image output device 30] Figure 3(b) shows an example of the hardware configuration of the image output device 30. The image output device 30 is an information processing device for displaying and outputting an electronic book format that is automatically generated by the photo sorting device 10. Since the hardware resources of the image output device 30 have many similarities to those already described, we will explain only the parts that are unique to the image output device 30.

[0030] As shown in Figure 3(b), the I / F350 includes a communication module 351, an image reading module 352, and a monitor 353. The image reading module 352 is configured to read images captured by a digital camera or the like.

[0031] The monitor 353 displays images of construction documents generated by the photo sorting device 10, as well as image albums acquired for construction record-keeping, classified according to their respective classification criteria, and compiled into e-book format.

[0032] [Image classification program embodiment] Next, an example of a functional block realized by executing the image classification program according to the present invention in the photo sorting system 1, which includes the photo sorting device 10, will be described.

[0033] As illustrated in Figure 4, the photo sorting device 10 has a arithmetic unit 110 which includes a communication unit 111, an image acquisition unit 112, a semantic information identification unit 113, a classification generation unit 114, a classification storage area setting unit 115, a photo sorting unit 116, an image storage unit 117, and an e-book generation unit 118.

[0034] The communication unit 111, acting as an image receiving means, receives construction photographs from the image input device 20 via the communication network 40 and notifies the image acquisition unit 112. The communication unit 111, acting as an image transmission means, also transmits the image file in ebook format generated by the ebook generation unit 118 to the image output device 30 via the communication network 40.

[0035] The image acquisition unit 112 notifies the semantic information identification unit 113 of multiple construction photos acquired from the image input device 20, and the image acquisition unit 112 also temporarily stores multiple construction photos in the storage device 120 via the image storage unit 117.

[0036] The semantic information identification unit 113, which serves as a means of determining the meaning of an image, performs processing corresponding to the image semantic identification step. The semantic information identification unit 113 uses a learning model to extract string information that correlates with the meaning of each subject included in the construction photograph. The semantic information identification unit 113 also extracts strings written on construction blackboards included as subjects. These strings may be referred to as semantic information below. The semantic information identification unit 113 then classifies the extracted strings into predetermined information groups using clustering technology.

[0037] As a classification generation means, the classification generation unit 114 performs a classification generation step in which it extracts string information that correlates with the meaning of each information group from the information group classified using the learning model in the semantic information identification unit 113, and summarizes the extracted string information. Then, it performs a classification generation step in which it generates a classification name based on the summarized string information. This classification name is notified to the classification storage area setting unit 115.

[0038] The classification storage area setting unit 115, acting as a classification storage area setting means, executes a classification storage area setting step to set a classification storage area in the storage device 120 based on the classification name generated by the classification generation unit 114. A classification storage area corresponds to a file folder, known as a structure for recording image files, for example. The classification name corresponds to a folder name. In other words, the classification storage area setting unit 115 sets a folder related to the classification name in the storage device 120.

[0039] The photo sorting unit 116, which serves as an image storage means, executes the processing related to the image storage step. The photo sorting unit 116 executes the process of sorting each construction image into the file folder set in the classification storage area setting unit 115. That is, based on the semantic information extracted by the semantic information identification unit 113 and the classification corresponding to this semantic information, each image is stored in the file folder formed in the storage device 120.

[0040] The image storage unit 117 stores each construction image sorted by the photo sorting unit 116 into the corresponding file folder.

[0041] The electronic book generation unit 118, acting as a means for outputting classified images, generates a display file in electronic book format from each image file stored in the image storage unit 117, based on the file folder set in the image storage unit 117, and notifies the communication unit 111.

[0042] [Details of the semantic information identification unit 113] Next, we will explain the details of the semantic information identification unit 113, which is one of the functional blocks implemented in the image formation program, using Figure 5. As shown in Figure 5, the semantic information identification unit 113 includes a character information extraction unit 1131, a subject semantic information extraction unit 1132, a semantic embedding processing unit 1133, and a clustering processing unit 1134.

[0043] The character information extraction unit 1131 uses OCR technology to extract characters shown on construction blackboards and other materials included in the construction images notified by the image acquisition unit 112.

[0044] The subject semantic information extraction unit 1132 estimates and extracts semantic information of subjects included in the construction image notified by the image acquisition unit 112.

[0045] The semantic embedding processing unit 1133 embeds the semantic information extracted by the subject semantic information extraction unit 1132 into the corresponding construction image and notifies the clustering processing unit 1134.

[0046] The clustering processing unit 1134 performs a grouping process on the group of construction images notified by the semantic embedding processing unit 1133, using similarity based on the features of each construction image as an indicator, and grouping similar construction images into the same group. The clustering processing unit 1134 performs the grouping of construction images by cluster analysis.

[0047] Furthermore, the clustering processing unit 1134 divides the entire group of construction photos according to a natural structure by separating (classifying) construction images with different characteristics into different clusters, and notifies the classification generation unit 114 of each group of construction photos. The clustering processing unit 1134 is executed by unsupervised learning in machine learning, and without providing prior labels for sorting (classification) of the construction images or the number of sortings (number of classifications), it classifies the group of construction images based on the intrinsic patterns (semantic information) of the construction images themselves and the relationships estimated from the semantic information. The clustering processing unit 1134 notifies the classification group of construction images and the text information of each construction image of the classification generation unit 114.

[0048] [Details of the classification generation unit 114] Next, we will describe in detail the section generation unit 114, which is one of the functional blocks implemented in the image formation program. As shown in Figure 5, the section generation unit 114 includes an image group semantic information extraction unit 1141, a summarization processing unit 1142, and a section name generation unit 1143.

[0049] The image group semantic information extraction unit 1141 identifies the character information associated with the construction photo group notified by the semantic information identification unit 113 as semantic information for each construction image group and notifies the summarization processing unit 1142.

[0050] The summarization processing unit 1142 summarizes the semantic information for each construction image group identified by the image group semantic information extraction unit 1141 and notifies the classification name generation unit 1143.

[0051] The classification name generation unit 1143 generates and assigns classification names to each group of construction photographs based on the summaries generated by the summarization processing unit 1142. These classification names become the folder names of the file folders where the corresponding groups of construction photographs are stored.

[0052] Furthermore, if classification history information set by the same user in past processing exists, the classification generation unit 114 may correct the information notified to the classification storage area setting unit 115 using that past classification and classification name. In other words, by referring to the storage structure of classification history information based on previously used classifications and classification names and correcting the configuration of the classification generation result, consistent image classification is made possible for the same user.

[0053] [Processing flow by image classification program] Next, the processing flow of the image classification program executed in the photo sorting device 10 will be explained using the flowchart in Figure 7. First, construction images are input to the photo sorting device 10 from the image input device 20 (S701). The image acquisition unit 112 acquires the construction images.

[0054] Next, the semantic information identification unit 113 extracts semantic information from the subjects included in the construction image and converts it into a string. If the subjects contain text, it also identifies that string and notifies the classification generation unit 114 of it (S702).

[0055] Next, the classification generation unit 114 uses the string notified by the semantic information identification unit 113 to determine whether there is classification history information, which is classification information generated in the past, in order to generate classifications for classifying and storing construction images (S703). If there is no classification history information (S703: No), the image classification generation process is executed based only on the image semantic information identified in step S702 (S704).

[0056] If classification history information is available (S703: Yes), the image classification generation process is executed while referring to the classification history information, which is the classification of past classifications (S705) (S704).

[0057] Next, based on the image classification categories and category names generated by the category generation unit 114, the classification storage area setting unit 115 sets the image categories, which are the storage categories for construction images, in the storage device 120 (S706). Specifically, it sets a file folder using the category name generated by the category generation unit 114.

[0058] Next, the classification storage area setting unit 115 performs a process to classify and store the construction images in the file folders set up, based on the semantic information of each construction image identified by the semantic information identification unit 113 (S707).

[0059] Next, the electronic book generation unit 118 converts the construction images stored in the storage device 120 in a specific file structure into an electronic book format based on that file structure, and notifies the image output device 30 of this (S708).

[0060] [Example of displaying construction images in e-book format] Figure 8 shows an example of an electronic book-format construction image displayed on the image output device 30. As shown in Figure 8, the electronic book-format construction image display screen 301 has a folder area FG that displays a folder tree of file folders to which classification names have been assigned, and an image area G that displays a list of construction photos that have been classified and stored in the currently open folder.

[0061] On the construction image display screen 301, when a file folder corresponding to the divided storage area is selected, thumbnail images of the stored construction images are output in the image area G. These thumbnail images may also be output in a hierarchical display format.

[0062] Furthermore, the collection of photographs displayed on the construction image display screen 301, compiled in an e-book format, can also be treated as the construction completion documents.

[0063] According to the embodiment described above, for example, for a large number of photographs taken at a construction site (construction photographs), a learning model can be used to analyze the visual elements contained in each image, such as construction blackboards, scales indicating the depth and length of the construction area, structures used at the construction area (such as columns), and qualification certificates as information indicating the attributes of the construction personnel. Based on the semantic content, clustering, classification, and labeling can be automatically performed.

[0064] In other words, the photo sorting device 10 according to this embodiment can significantly reduce the labor and automate the classification of construction photographs, which was previously a cumbersome manual process, thus preventing classification errors and omissions. Furthermore, by visually organizing and displaying the classification results in an electronic book format, users can view the data intuitively and easily, greatly improving the efficiency of document creation and reporting tasks.

[0065] Furthermore, by incorporating a mechanism to correct classification results by referring to past classification history, highly accurate classification tailored to each user's classification tendencies can be achieved, and continuous use can be expected to lead to system learning and improvement.

[0066] Thus, the present invention enables the automation and intelligent organization of construction photograph classification tasks, significantly contributing to the improvement of the reliability of construction records and operational efficiency.

[0067] [Other embodiments] Furthermore, each device does not necessarily have to be a single device. In other words, each device may be a combination of multiple devices.

[0068] The present invention may be implemented by a process for realizing the vehicle management method exemplified above, or by a program (including firmware and similar components; hereinafter simply referred to as "program") that performs a process equivalent to the process described above.

[0069] In other words, the present invention may be implemented by a program written in a programming language or the like, which issues commands to a computer to obtain a predetermined result. The program may also be configured so that a part of the processing is executed by hardware such as an IC (integrated circuit).

[0070] A program causes the computer to perform the above-mentioned processes by having its arithmetic unit, control unit, and memory device work together. In other words, a program is loaded into main memory, issues commands to the arithmetic unit to perform calculations, and operates the computer.

[0071] Furthermore, the program may be provided via a computer-readable storage medium or via a telecommunications line such as a network.

[0072] The present invention may be implemented in a system composed of multiple devices. That is, an information processing system consisting of multiple computers may execute the above-described processes in a redundant, parallel, distributed, or combination thereof. Therefore, the present invention may be implemented in devices other than those described above, and in systems other than those described above.

[0073] The processing in this invention (including data storage, or a part of the processing) may be performed on an information processing device installed overseas.

[0074] It should be noted that the present invention is not limited to the embodiments exemplified above. Therefore, the present invention can be modified by adding or changing components without departing from the technical spirit. Thus, all technical matters included in the technical concept described in the claims are covered by the present invention. The embodiments exemplified above are specific examples that are suitable for implementation. Furthermore, those skilled in the art can implement various modifications from the disclosed content, and such modifications are included in the technical scope described in the claims.

[0075] [Aspects of the present invention] The contents of this invention are, for example, as follows: <1> An image classification device that classifies and stores multiple input images, Image semantic identification means for identifying semantic information that can be estimated from the aforementioned image and relating it to the image, A segmentation generation means for generating image segments for grouping the images by performing cluster analysis on the semantic information corresponding to multiple images, A classification storage area setting means for setting a storage area corresponding to each of the aforementioned image divisions, Image storage means for storing the aforementioned image in the memory area, This is an image classification device characterized by having the following features: <2> The system includes a classified image output means that outputs each image stored in the memory area in a format corresponding to the image category. The aforementioned <1> This is the image classification device described in [reference]. <3> The classified image output means outputs the image in an electronic book format corresponding to the structure of the storage area. The electronic book format outputs a hierarchical display with thumbnail images of the images stored in the memory area. <2> This is the image classification device described in [reference]. <4> The aforementioned storage area is a file folder set on the storage medium, and the folder name of the file folder corresponds to the image section. The aforementioned <1> or the above <3> It is an image classification device described in any one of the following. <5> The image semantic identification means identifies subjects included in the image based on a learning model generated using a machine learning process, analyzes semantic information corresponding to each subject, and extracts the semantic information. The aforementioned <1> or the above <4> It is an image classification device described in any one of the following. <6> The classification generation means generates the image classification and the classification name of the image classification from the semantic information identified for the image based on a learning model generated using a machine learning process. The aforementioned <1> or the above <5> It is an image classification device described in any one of the following. <7> When the classification generation means has classification history information in which the image classification has been set based on semantic information previously generated by the image semantic identification means for images relating to the same user, the classification history information The generation results of the image divisions and division names are corrected by referring to the above. The aforementioned <6> This is the image classification device described in [reference]. <8> The aforementioned image is a construction photograph used in the construction completion documents. The aforementioned <1> or the above <7> It is an image classification device described in any one of the following. <9> The aforementioned construction photographs include, at a minimum, the construction site, the construction site and its contents, or materials used to clearly indicate the construction site and its contents, the construction method, or the state of the construction, as well as information indicating the attributes of the person in charge of construction. The aforementioned <8> This is the image classification device described in [reference]. <10> The component used to clearly indicate the manner or state of the aforementioned construction work is a construction blackboard. The aforementioned <9> This is the image classification device described in [reference]. <11> An image classification system comprising: an imaging device for capturing images of a subject; and an image classification device connected to the imaging device via a communication network for classifying and storing multiple images captured by the imaging device, The imaging device includes imaging means for imaging the subject, Image transmission means for transmitting the captured image to the image classification device via the aforementioned communication network, It has, The aforementioned image classification device is An image receiving means connected to the aforementioned communication network and receiving the aforementioned image, Image semantic identification means for identifying semantic information that can be estimated from the received image and associating it with the image, A segmentation generation means for generating image segments for grouping the images by performing cluster analysis on the semantic information corresponding to multiple images, A classification storage area setting means for setting a storage area corresponding to each of the aforementioned image divisions, Image storage means for storing the aforementioned image in the memory area, Having, This is an image classification system characterized by the following: <12> to the computer Image semantic identification step: In which semantic information that can be estimated from multiple input images is identified in association with the image in question. A segmentation generation step that generates image segments for grouping the images by performing cluster analysis on the semantic information corresponding to multiple images, A classification storage area setting step in which a storage area corresponding to each of the aforementioned image segments is set, Image storage step of storing the image in the memory area, This is an image classification program characterized by executing [a specific action]. [Explanation of Symbols]

[0076] 1: Photo sorting system 10: Photo sorting device 20: Image input device 30: Image output device 40: Communication Networks 110: Arithmetic device 111: Communications Department 112: Image acquisition unit 113:Semantic information identification part 114:Classification generator 115: Classification storage area setting section 116: Photo sorting department 117: Image storage unit 118: Electronic Book Generation Department 301: Construction image display screen 1131: Character information extraction unit 1132: Subject semantic information extraction unit 1133: Semantic embedding processing unit 1134: Clustering Processing Unit 1141: Image group semantic information extraction unit 1142: Summarization Processing Unit 1143: Category name generation part

Claims

1. An image classification device that classifies and stores multiple input images, Image semantic identification means for identifying semantic information that can be estimated from the aforementioned image and relating it to the image, A segmentation generation means for generating image segments for grouping the images by performing cluster analysis on the semantic information corresponding to multiple images, A classification storage area setting means for setting a storage area corresponding to each of the aforementioned image divisions, The system includes an image storage means for storing the aforementioned image in the storage area, The classification generation means generates image classifications and classification names for the image classifications from the semantic information identified for the image based on a learning model generated using a machine learning process, and when there is classification history information in which the image semantic identification means has previously set image classifications for images relating to the same user based on semantic information, it corrects the generation results of the image classifications and classification names by referring to the classification history information. An image classification device characterized by the following features.

2. The image classification device according to claim 1, further comprising a classified image output means for outputting each image stored in the memory area in a format corresponding to the image category.

3. The classified image output means outputs the image in an electronic book format corresponding to the structure of the storage area. The image classification device according to claim 2, wherein the electronic book format outputs a hierarchical display with thumbnail images of the images stored in the memory area.

4. The image classification device according to any one of claims 1 to 3, wherein the storage area is a file folder set on a storage medium, and the folder name of the file folder corresponds to the image category.

5. The image classification apparatus according to any one of claims 1 to 3, wherein the image semantic identification means identifies subjects included in the image based on a learning model generated using a machine learning process, analyzes the semantic information corresponding to each subject, and extracts the semantic information.

6. The image classification device according to any one of claims 1 to 3, wherein the aforementioned image is a construction photograph used in the construction completion documents.

7. The image classification device according to claim 6, wherein the aforementioned construction photograph includes, at a minimum, a construction site, the construction site and its contents, or a component for clearly indicating the construction site and its contents, the construction method or the state of the construction, and information for clearly indicating the attributes of the person in charge of construction, as the subject of the photograph.

8. The image classification device according to claim 7, wherein the member for clearly indicating the manner or state of the aforementioned construction is a construction blackboard.

9. An image classification system comprising: an imaging device for capturing images of a subject; and an image classification device connected to the imaging device via a communication network for classifying and storing multiple images captured by the imaging device, The imaging device includes imaging means for imaging the subject, The system includes an image transmission means for transmitting the captured image to the image classification device via the aforementioned communication network, The aforementioned image classification device is An image receiving means connected to the aforementioned communication network and receiving the aforementioned image, Image semantic identification means for identifying semantic information that can be estimated from the received image and associating it with the image, A segmentation generation means for generating image segments for grouping the images by performing cluster analysis on the semantic information corresponding to multiple images, A classification storage area setting means for setting a storage area corresponding to each of the aforementioned image divisions, The system includes an image storage means for storing the aforementioned image in the memory area, The classification generation means generates image classifications and classification names for the image classifications from the semantic information identified for the image based on a learning model generated using a machine learning process, and when there is classification history information in which the image semantic identification means has previously set image classifications for images relating to the same user based on semantic information, it corrects the generation results of the image classifications and classification names by referring to the classification history information. An image classification system characterized by the following features.

10. to the computer Image semantic identification step: In which semantic information that can be estimated from multiple input images is identified in association with the image in question. A segmentation generation step is performed to generate image segments for grouping the images by cluster analysis used on the semantic information, A classification storage area setting step in which a storage area corresponding to each of the aforementioned image segments is set, The image storage step of storing the image in the memory area is executed, In the aforementioned classification generation step, the image classification and classification name of the image classification are generated from the semantic information identified for the image based on a learning model generated using a machine learning process, and if there is classification history information in which the image semantic identification means has previously set the image classification based on semantic information generated for images relating to the same user, the generation results of the image classification and classification name are corrected by referring to the classification history information. An image classification program characterized by [this feature].

Images