Information processing device

Abstract

An information processing device (S) includes: a first training unit (1) that performs training for identifying at least a facility region and a local authority region in a captured image by using a data set relating to the captured image; and a detection unit (3) that, using the training result obtained by the first training unit (1), identifies a facility region, a local authority region, and a private property region from the inputted captured image, and generates a facility region extraction image, a local authority region extraction image, and a private property region extraction image; a processing unit (4) that integrates the facility region extraction image, the local authority region extraction image, and the private property region extraction image to generate a tensor; a second training unit (5) that executes training of location classification of the facility by using the tensor and a label for the correct location classification of the facility; a determination unit (6) that, adopting the tensor as input, executes determination of location classification of the facility by using the training result of the second training unit (5); and an output unit (7) that outputs the result of the determination.

Description

Information processing device

[0001] The present disclosure relates to an information processing device.

[0002] For the repair or maintenance of numerous pieces of equipment, such as utility poles, located in various locations, it is important for facility managers to determine whether the land is public land, such as government land, or private land. A known method for determining this is to surround objects with bounding boxes and recognize the relative positions of the objects (see, for example, Non-Patent Document 1).

[0003] Rowan Zellers et al., “Neural Motifs: Scene Graph Parsing with Global Context”, Preprint, Computer Vision and Pattern Recognition 2018, pp.1-12, [Online], 2018, [October 6, 2023], Internet <URL: https: / / arxiv.org / abs / 1711.06640>

[0004] It is difficult to estimate the location classification of a facility, which requires three-dimensional spatial information including the depth direction, from an image that only has two-dimensional information. In order to automatically determine the location classification of a facility from an image captured by an in-vehicle camera, etc., it is necessary to recognize the relationship between the facility and the ground area around the facility. The method proposed in Non-Patent Document 1 uses a bounding box, making it difficult to accurately grasp the complex ground area.

[0005] The purpose of the present disclosure, made in consideration of such circumstances, is to improve the accuracy of determining the location classification of facilities.

[0006] An information processing device according to one embodiment includes: a first learning unit that uses a dataset related to the captured image to perform learning to identify at least equipment areas and government land areas in the captured image; a detection unit that uses the learning results obtained by the first learning unit to identify equipment areas, government land areas, and private land areas from the input captured image and generate an equipment area extraction image, a government land area extraction image, and a private land area extraction image; a processing unit that integrates the equipment area extraction image, the government land area extraction image, and the private land area extraction image to generate a tensor; a second learning unit that uses the tensor and a label of the correct location category of the equipment to perform learning of the location category of the equipment; a determination unit that uses the learning results of the second learning unit to input the tensor and determine the location category of the equipment; and an output unit that outputs the results of the determination.

[0007] According to the present disclosure, it is possible to improve the accuracy of determining the location classification of equipment.

[0008] 1 is a block diagram showing a configuration of an information processing device according to an embodiment; FIG. 2 is a diagram showing detection using a multi-class model; FIG. 3 is a diagram showing detection using a composite single-class model; FIG. 4 is a diagram showing detection using a deep learning technique; FIG. 5 is a block diagram showing a first learning unit in an embodiment using a multi-class model; FIG. 6 is a block diagram showing an input unit and a detection unit in an embodiment using a multi-class model; FIG. 7 is a block diagram showing a first learning unit in an embodiment using a composite single-class model; FIG. 8 is a block diagram showing an input unit and a detection unit in an embodiment using a composite single-class model; FIG. 9 is a block diagram showing a detection unit in an embodiment using a composite single-class model; FIG. 10 is a block diagram showing a processing unit; FIG. 11 is a block diagram showing a second learning unit, a determination unit, and an output unit;

[0009] An embodiment will be described below with reference to the drawings. In each drawing, identical or corresponding parts are designated by the same reference numerals. In the description of this embodiment, the description of identical or corresponding parts will be omitted or simplified as appropriate.

[0010] The information processing device S shown in FIG. 1 can communicate with one or more terminals via a network. The network includes, for example, the Internet, at least one WAN, at least one MAN, or any combination thereof. "WAN" is an abbreviation for wide area network. "MAN" is an abbreviation for metropolitan area network. The network may include at least one wireless network, at least one optical network, or any combination thereof. The wireless network may be, for example, an ad hoc network, a cellular network, a wireless LAN, a satellite communication network, or a terrestrial microwave network. "LAN" is an abbreviation for local area network.

[0011] 1 shows one information processing device S, but alternatively, there may be any number of information processing devices S. The processing executed by the information processing device S may be executed by a plurality of information processing devices S in a distributed arrangement.

[0012] The information processing device S is a general-purpose computer such as a PC, a server computer such as a cloud server, or a dedicated computer. "PC" is an abbreviation for personal computer.

[0013] The configuration of an information processing device S according to this embodiment will be described. The information processing device S includes a first learning unit 1, an input unit 2, a detection unit 3, a processing unit 4, a second learning unit 5, a determination unit 6, and an output unit 7. Arrows shown in Fig. 1 indicate the flow of information.

[0014] The functions of the first learning unit 1, the detection unit 3, the processing unit 4, the second learning unit 5, and the determination unit 6 are performed by a control unit. The control unit includes, for example, one or more general-purpose processors including a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control unit may include one or more dedicated processors specialized for specific processing. Instead of including a processor, the control unit may include one or more dedicated circuits. The dedicated circuits may be, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit may include an ECU (Electronic Control Unit). The control unit controls the communication unit to send and receive any information.

[0015] The input unit 2 and the output unit 7 are interfaces for receiving input from a network or a user.

[0016] The information processing device S may further include a storage unit and a communication unit.

[0017] The storage unit includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, a RAM, a ROM, or a flash memory. "RAM" is an abbreviation for random access memory. "ROM" is an abbreviation for read only memory. RAM is, for example, an SRAM or a DRAM. "SRAM" is an abbreviation for static random access memory. "DRAM" is an abbreviation for dynamic random access memory. ROM is, for example, an EEPROM. "EEPROM" is an abbreviation for electrically erasable programmable read only memory. Flash memory is, for example, an SSD. "SSD" is an abbreviation for solid-state drive. Magnetic memory is, for example, an HDD. "HDD" is an abbreviation for hard disk drive. The storage unit functions, for example, as a main storage device, an auxiliary storage device, or a cache memory.

[0018] The communication unit includes at least one communication module. The communication module is, for example, a module compatible with a wired LAN communication standard such as Ethernet (registered trademark), a wireless LAN communication standard such as IEEE 802.11, or a mobile communication standard such as LTE, 4G standard, or 5G standard. "IEEE" is an abbreviation for Institute of Electrical and Electronics Engineers. "LTE" is an abbreviation for Long Term Evolution. "4G" is an abbreviation for 4th generation. "5G" is an abbreviation for 5th generation. The communication unit receives data used in the operation of the information processing device S and transmits data obtained by the operation of the information processing device S.

[0019] An overview of this embodiment will be described. The detection unit 3 can use either the multi-class model shown in FIG. 2 or the composite single-class model shown in FIG. 3. In the multi-class model shown in FIG. 2, the detection unit 3 simultaneously detects government land areas and private land areas. In the composite single-class model shown in FIG. 3, the detection unit 3 detects the private land area shown in C by removing the government land area shown in B from the ground area shown in A. As shown in FIG. 4, the determination unit 6 uses a deep learning technique on the detection results to determine the location classification of the utility pole based on the positional relationship between the ground and the utility pole. Utility poles are an example of equipment.

[0020] The details of this embodiment are described below.

[0021] 1-1. Pixel Region Detection Using Multi-Class Model Referring to FIG. 5, the process executed in the first learning unit 1 will be described.

[0022] The captured image input unit 11, utility pole mask image input unit 12, official land mask image input unit 13, and private land mask image input unit 14A each accept input of: a captured image (RGB) with a width of w pixels in the x direction and a height of h pixels in the y direction (e.g., height 5120 pixels, width 5120 pixels); a utility pole mask image (binary) generated by labeling utility pole areas in the captured image; an official land mask image (binary) generated by labeling official land areas in the captured image; and a private land mask image (binary) generated by labeling private land areas in the captured image. The input is performed by another terminal or a user. The utility pole mask image is an example of a facility mask image. The input image is stored in the memory unit of the information processing device S.

[0023] The photographed image, the utility pole mask image, the official land mask image, and the private land mask image are stored in association with each other in groups of n. The image set creation unit 15 creates n sets of data sets related to the photographed image, each set including the photographed image, the utility pole mask image, the official land mask image, and the ground mask image. The image set creation unit 15 stores the created data sets in the storage unit.

[0024] The image cropping unit 16 crops the top of each stored image so that it is horizontally long with a width of i pixels in the x direction and a height of j pixels in the y direction (e.g., a height of 3840 pixels, a width of 5120 pixels, and an aspect ratio of 4:3). In this case, j is preferably the same size as the width of the captured image.

[0025] The image compression unit 17 reduces each stored image to a width of k pixels in the x direction and a height of l pixels in the y direction (e.g., a height of 288 pixels and a width of 384 pixels). The image compression unit 17 transmits the reduced captured image and utility pole mask image to the utility pole area learning unit 18A. The image compression unit 17 transmits the reduced captured image, official land mask image, and private land mask image to the official land / private land area learning unit 19A.

[0026] With the above configuration, the first learning unit 1 can delete the empty space that is not involved in determining the location of the utility pole by cropping the top of the image when converting the captured square image into a landscape format. By compressing the image, the first learning unit 1 can adjust the amount of learning data according to the specifications of the device that performs the learning.

[0027] The utility pole area learning unit 18A and the public land / private land area learning unit 19A may have an FCN (full-layer convolutional neural network) having an encoder-decoder structure as a model structure.

[0028] The utility pole area learning unit 18A performs learning to identify utility pole areas using the captured image and the utility pole mask image. The utility pole area learning unit 18A transmits the learning result to the detection unit 3.

[0029] The government / private land area learning unit 19A performs learning to identify government land areas and private land areas using the captured image, the government land mask image, and the private land mask image. The government / private land area learning unit 19A transmits the learning result to the detection unit 3.

[0030] With the above configuration, the first learning unit 1 can perform semantic segmentation and detect each region, such as a utility pole region, on a pixel-by-pixel basis. Since utility pole regions have fewer target pixels than public land / private land regions, it is difficult to simultaneously detect utility pole regions and ground regions. By performing the detection of utility pole regions separately, the first learning unit 1 can more accurately detect utility pole regions.

[0031] 6 , the captured image input unit 21 of the input unit 2 accepts input of a captured image (e.g., an RGB image) having a width of w pixels in the x direction and a height of h pixels in the y direction (e.g., a height of 5120 pixels and a width of 5120 pixels). The input unit 2 transmits the captured image to the detection unit 3.

[0032] The image cropping unit 31 of the detection unit 3 crops the top of the captured image received from the input unit 2 so that the image is horizontally long with a width of i pixels in the x direction and a height of j pixels in the y direction (e.g., a height of 3840 pixels and a width of 5120 pixels, with an aspect ratio of 4:3). In this case, j is preferably the same size as the width of the captured image. The image compression unit 32 reduces the size of the captured image using, for example, a method similar to that used by the image compression unit 17. The image compression unit 32 stores the reduced image in a storage unit.

[0033] The utility pole area extraction unit 33A uses the learning results obtained by the utility pole area learning unit 18A to identify a utility pole area from the image stored in the image compression unit 32, and generates an extracted utility pole area image.

[0034] The government land / private land area extraction unit 34A uses the learning results obtained by the government land / private land area learning unit 19A to identify government land areas and private land areas from the images stored in the image compression unit 32 of the detection unit 3. The government land / private land area extraction unit 34A generates a government land area extraction image and a private land area extraction image.

[0035] The utility pole area result output unit 35A and the government land / private land area result output unit 36A output the utility pole area extraction image, government land area extraction image, and private land area extraction image to the processing unit 4.

[0036] [1-2. Pixel Region Detection Using a Composite Single-Class Model] An alternative example to the above "1-1. Pixel Region Detection Using a Multi-Class Model" will now be described.

[0037] In FIG. 7, the process executed in the first learning unit 1 will be described.

[0038] The captured image input unit 11, utility pole mask image input unit 12, official ground mask image input unit 13, and ground mask image input unit 14B each accept input of: a captured image (RGB) with a width of w pixels in the x direction and a height of h pixels in the y direction (e.g., height 5120 pixels, width 5120 pixels), a utility pole mask image (binary) generated by labeling the utility pole region in the captured image, a official ground mask image (binary) generated by labeling the ground region in the captured image, and a ground mask image (binary) generated by labeling the ground region in the captured image. The input images are stored in the storage unit.

[0039] The captured image, utility pole mask image, official land mask image, and ground mask image are stored in a number of n pieces. The image set creation unit 15 creates n sets of data sets related to the captured image, each set including the captured image, utility pole mask image, official land mask image, and ground mask image. The image set creation unit 15 stores the created data sets in the storage unit.

[0040] The image cropping unit 16 crops the top of each stored image so that it is horizontally long with a width of i pixels in the x direction and a height of j pixels in the y direction (e.g., a height of 3840 pixels, a width of 5120 pixels, and an aspect ratio of 4:3). In this case, j is preferably the same size as the width of the captured image.

[0041] The image compression unit 17 reduces each stored image to a width of k pixels in the x direction and a height of l pixels in the y direction (e.g., a height of 288 pixels and a width of 384 pixels). The image compression unit 17 outputs each reduced image.

[0042] The utility pole area learning unit 18B, the government land area learning unit 19B, and the ground area learning unit 191B may have an FCN (full-layer convolutional neural network) having an encoder-decoder structure as a model structure.

[0043] The utility pole area learning unit 18B performs learning to identify utility pole areas using the captured image and the utility pole mask image from the data set. The result of this learning is used by the detection unit 3.

[0044] The image area learning unit 19B uses the captured image and the image area mask image from the data set to perform learning to identify image areas. The result of this learning is used by the detection unit 3.

[0045] The ground area learning unit 191B performs learning to identify the ground area using the captured image and the ground mask image from the data set. The result of this learning is used by the detection unit 3.

[0046] With the above configuration, the first learning unit 1 can perform semantic segmentation to detect utility pole areas, ground areas, and designated land areas pixel by pixel. Private land areas have many features, so detection accuracy is expected to be low. In the composite single-class model, the first learning unit 1 detects ground areas and designated land areas, and identifies areas of the ground area that are not designated land areas as private land areas, thereby improving accuracy. Although the ground area and designated land areas partially overlap, semantic segmentation cannot detect overlapping areas as two classes (i.e., ground areas and designated land areas). Therefore, the ground area and designated land area are trained separately, and target areas are detected for each, which can then be used for subsequent detection of private land areas.

[0047] 8, the processes executed by the captured image input unit 21, the image cropping unit 31, and the image compression unit 32 of the input unit 2 are as described above, and therefore will not be described again here.

[0048] The utility pole area extraction unit 33B uses the learning result obtained by the utility pole area learning unit 18B to identify a utility pole area from the image stored in the image compression unit 32, and generates an extracted utility pole area image.

[0049] The land area extraction unit 34B uses the learning results obtained by the land area learning unit 19B to identify land areas from the image stored in the image compression unit 32, and generates an extracted land area image.

[0050] The ground area extraction unit 35B uses the learning result obtained by the ground area learning unit 191B to identify the ground area from the image stored in the image compression unit 32, and generates a ground area extraction image.

[0051] The utility pole area result output unit 36B and the official land area result output unit 37B transmit the utility pole area extraction image, the official land area extraction image, and the ground area extraction image to the processing unit 4 in image format.

[0052] The ground area result output unit 37B and the ground area result output unit 38B output the extraction results of the ground area extraction unit 34B and the ground area extraction unit 35B in image format to the expansion processing unit 391B. The processing of the expansion processing unit 391B is explained with reference to FIG.

[0053] 9, the expansion processing unit 391B performs expansion processing on the output ground region extraction image and ground region extraction image. Specifically, the expansion processing unit 391B focuses on a specific pixel and its surrounding pixels (e.g., a 3×3 area) and sets the pixel value of the specific pixel to the maximum value of the pixels in the 3×3 area. The pixel filling unit 392B performs filling processing on the expanded image and stores the image in a storage unit.

[0054] With the above configuration, the detection unit 3 can perform expansion processing to correct holes or cracks that appear as noise within the region and fill in the region. By performing hole-filling processing, the detection unit 3 can remove small imperfections that form within the region when semantic segmentation is performed. As a result, the detection unit 3 can make the detection results closer to the actual ground and ground, allowing for more accurate understanding of ground information.

[0055] The civil land pixel derivation unit 393B derives the civil land area using the ground area extraction image and the official land area extraction image stored by the pixel filling unit 392B. The civil land pixel derivation unit 393B identifies, as a civil land area, pixels that were not detected as official land areas among the pixels extracted as the ground area and performs labeling. The civil land pixel derivation unit 393B outputs the labeled civil land area result to the processing unit 4 as a civil land area extraction image via the civil land area result output unit 394B.

[0056] With the above configuration, the detection unit 3 derives private land areas by excluding public land areas from the detected ground area, thereby improving the identification accuracy compared with the detection results of segmentation that has learned private land areas.

[0057] 10 illustrates the processing of the processing unit 4. The processing unit 4 includes a selection unit 41, a division unit 42, and an integration unit 43.

[0058] The selection unit 41 counts the pixels in the x direction of one utility pole from the top to the bottom of the pole, and when it determines that the average number of pixels in the x direction is smaller than m pixels (e.g., 12 pixels), it excludes the utility pole from targets for determining location classification.As an additional example or alternative example, the selection unit 41 may exclude a utility pole from targets for determining location classification when the total number of pixels of one utility pole is smaller than d pixels (e.g., 100 pixels).

[0059] If the width of a utility pole is smaller than a certain value, it is considered to be located far away. This makes it difficult to grasp the ground area required to perform location determination. Furthermore, if the width of a utility pole is larger than a predetermined value (located nearby) but the total number of pixels is smaller than a predetermined value, it is assumed that part of the utility pole is obstructed by a roadside tree, a car, or the like. This makes it difficult to accurately grasp the utility pole area. By excluding these utility poles from the targets for determining location classification, the processing unit 4 can perform more accurate location determination.

[0060] When the dividing unit 42 determines that a single image contains multiple utility pole regions in the utility pole region extraction image output from the detection unit 3, the dividing unit 42 generates an image in which only the respective utility pole regions are displayed. For example, when three utility pole regions, utility pole A, utility pole B, and utility pole C, are detected in a single image, the dividing unit 42 generates three images: an image in which only the utility pole A region remains, an image in which only the utility pole B region remains, and an image in which only the utility pole C region remains.

[0061] With the above configuration, the processing unit 4 can divide the image into individual utility poles and establish a one-to-one correspondence between the utility poles and the correct location classification determination results in order to perform learning to recognize the relationship between utility poles and ground areas. This allows the processing unit 4 to efficiently perform learning regardless of the number of utility poles in the image.

[0062] The integrating unit 43 integrates the government land area extraction image output from the detection unit 3, the private land area extraction image, and the utility pole area extraction image excluding utility poles that are not subject to the determination, to generate a three-layer tensor. The number of layers can be set as desired. For example, in a certain image, the integrating unit 43 labels utility pole area pixels as 0, government land area pixels as 1, private land area pixels as 2, and pixels that do not correspond to any area as 3. In this case, if an area extracted as a utility pole area is also extracted as another area, the result of extraction as a utility pole extraction area takes priority, and the pixel is determined to be the utility pole area.

[0063] With the above configuration, the processing unit 4 prioritizes integration of utility pole areas, correctly acquires information on the ground area where the utility pole and the ground contact, and can maintain the accuracy of the location classification determination. If priority is given to ground information when the ground and the utility pole contact each other around the bottom end of the utility pole, the ground area around the utility pole may change.

[0064] 3. Location Determination Using Deep Learning Method The location determination method will be described with reference to FIG.

[0065] The integrated image input unit 51 of the second learning unit 5 reads the three-layer tensor received from the processing unit 4.

[0066] The data set creation unit 53 classifies the location classification in which the utility pole is located into one of the following three labels, and reads the correct label of each utility pole from the correct label input unit 52. 0: Utility pole stands on public land 1: Utility pole stands on private land 2: Utility pole location classification needs to be confirmed

[0067] If the ground area obtained by extending the bottom end of the utility pole downward does not belong to either public or private land, or if the bottom end of the utility pole is cut off below the bottom edge of the image, the dataset creation unit 53 sets the location classification of the utility pole to the above-mentioned "2. Utility pole location classification needs to be confirmed." The reason for this is that if the ground area under the utility pole does not belong to either public or private land, the bottom end of the utility pole may be obstructed by a vehicle or the like, and the ground area may not be correctly identified. Another reason is that if the bottom end of the utility pole is cut off below the bottom edge of the image, ground information around the bottom end of the utility pole has not been acquired.

[0068] The n' number of integrated images are stored, and n' labels indicating the correct locations of the utility poles are also stored. Therefore, the data set creation unit 53 creates n' sets of data sets and stores them in the storage unit.

[0069] The utility pole location learning unit 54 uses a three-layer tensor that integrates utility pole areas, private land areas, and government land areas, and labels of correct utility pole location categories to learn utility pole location categories. The learning results are used by the utility pole location determination unit 62.

[0070] The utility pole location learning unit 54 may have an FCN (full-layer convolutional neural network) having an encoder-decoder structure as a model structure.

[0071] With the above configuration, the second learning unit 5 detects utility pole areas, public land areas, and private land areas at the pixel level, and then performs location determination using a relationship recognition model between utility poles and ground areas. Therefore, the second learning unit 5 can perform more accurate utility pole location classification determination compared to object relationship recognition using bounding boxes.

[0072] The integrated image input unit 61 of the determination unit 6 reads the three-layer tensor received from the processing unit 4. The utility pole location determination unit 62 uses the learning results of the utility pole location learning unit 54 to input the three-layer tensor and executes determination of the utility pole location classification. The result of the location determination is one of the following three classes: - Utility pole is standing on public land - Utility pole is standing on private land - Utility pole location classification needs to be confirmed

[0073] As described above, the determination unit 6 performs determination of the location classification of utility poles using deep learning.

[0074] The label output unit 71 of the output unit 7 outputs the determination result of the utility pole location determination unit 62 for each utility pole, for example in the form of a table. The determination result may be displayed on the display unit of the information processing device S, or may be transmitted to another terminal via a network.

[0075] The present disclosure is not limited to the above-described embodiments. For example, two or more blocks shown in the block diagram may be integrated, or one block may be divided. Two or more steps shown in the flowchart may be executed in parallel or in a different order, instead of being executed in chronological order as described, depending on the processing capabilities of the device executing each step, or as needed. Other modifications are possible within the scope of the present disclosure.

[0076] The information processing device S of this embodiment can also be realized by a computer and a program. Furthermore, in the above embodiment, a program that executes all or part of the functions or processes of the information processing device S can be recorded on a computer-readable recording medium. Computer-readable recording media include non-transitory computer-readable media, such as magnetic recording devices, optical discs, magneto-optical recording media, or semiconductor memories. The program can be distributed, for example, by selling, transferring, or lending portable recording media, such as SD (Secure Digital) cards, DVDs (Digital Versatile Discs), or CD-ROMs (Compact Disc Read Only Memory) on which the program is recorded. The program can also be distributed by storing the program in the storage of a server and transmitting the program from the server to another computer via a network. The program may also be provided as a program product. The present disclosure can also be realized as a program executable by a processor. Some or all of the functions of the information processing device S may be realized by a programmable circuit or a dedicated circuit. Some or all of the functions of the information processing device S may be realized by hardware.

[0077] The following additional notes are provided regarding the above-described embodiments.

[0078] (Supplementary Item 1) An information processing device including: a first learning unit that uses a dataset related to a captured image to perform learning to identify at least equipment areas and government land areas in the captured image; a detection unit that uses the learning results obtained by the first learning unit to identify equipment areas, government land areas, and private land areas from the input captured image and generate an equipment area extraction image, a government land area extraction image, and a private land area extraction image; a processing unit that integrates the equipment area extraction image, the government land area extraction image, and the private land area extraction image to generate a tensor; a second learning unit that uses the tensor and a label of the correct location classification of the equipment to perform learning of the location classification of the equipment; a determination unit that uses the learning results of the second learning unit to input the tensor and perform a determination of the location classification of the equipment; and an output unit that outputs the result of the determination. (Supplementary Item 2) In the information processing device described in Supplementary Item 1, the first learning unit performs learning to identify equipment areas using captured images included in the dataset and equipment mask images in which equipment areas are labeled in the captured images, and performs learning to identify government land areas and private land areas using captured images included in the dataset, official land mask images in which government land areas are labeled in the captured images, and private land mask images in which private land areas are labeled in the captured images, and the detection unit uses the learning results of the first learning unit to identify the equipment areas, the government land areas, and the private land areas from the input captured images.(Supplementary Item 3) In the information processing device described in Supplementary Item 1, the first learning unit performs learning to identify equipment areas using captured images included in the dataset and equipment mask images in which equipment areas are labeled in the captured images, performs learning to identify official land areas using captured images included in the dataset and official land mask images in which official land areas are labeled in the captured images, and performs learning to identify ground areas using captured images included in the dataset and ground mask images in which ground areas are labeled in the captured images, and the detection unit uses learning results from the first learning unit to identify the equipment areas, the official land areas, and the ground areas from the input captured images, and identifies areas identified as ground areas that are not identified as official land areas as private land areas. (Supplementary Item 4) In the information processing device described in any one of Supplementary Items 1 to 3, at least one of the first learning unit and the second learning unit has a full-layer convolutional neural network having an encoder-decoder structure as a model structure.

[0079] S Information processing device

Claims

1. An information processing device including: a first learning unit that uses a dataset related to a captured image to perform learning to identify at least equipment areas and government land areas in the captured image; a detection unit that uses the learning results obtained by the first learning unit to identify equipment areas, government land areas, and private land areas from the input captured image and generate an equipment area extracted image, a government land area extracted image, and a private land area extracted image; a processing unit that integrates the equipment area extracted image, the government land area extracted image, and the private land area extracted image to generate a tensor; a second learning unit that uses the tensor and a label of a correct location classification of the equipment to perform learning of the location classification of the equipment; a determination unit that uses the learning results of the second learning unit to determine the location classification of the equipment by inputting the tensor; and an output unit that outputs the result of the determination.

2. An information processing device as described in claim 1, wherein the first learning unit performs learning to identify equipment areas using captured images included in the dataset and equipment mask images in which equipment areas are labeled in the captured images, and performs learning to identify public land areas and private land areas using captured images included in the dataset, official land mask images in which official land areas are labeled in the captured images, and private land mask images in which private land areas are labeled in the captured images, and the detection unit uses the learning results of the first learning unit to identify the equipment areas, the official land areas, and the private land areas from the input captured images.

3. An information processing device as described in claim 1, wherein the first learning unit: performs learning to identify equipment areas using captured images included in the dataset and equipment mask images in which equipment areas are labeled in the captured images; performs learning to identify official land areas using captured images included in the dataset and official land mask images in which official land areas are labeled in the captured images; performs learning to identify ground areas using captured images included in the dataset and ground mask images in which ground areas are labeled in the captured images; and the detection unit: uses the learning results of the first learning unit to identify the equipment areas, the official land areas, and the ground areas from the input captured images; and identifies areas identified as ground areas that are not identified as official land areas as private land areas.

4. An information processing device according to claim 1, wherein at least one of the first learning unit and the second learning unit has a full-layer convolutional neural network having an encoder-decoder structure as a model structure.