Information processing device, object recognition method, and program

Abstract

The purpose of the present invention is to provide an information processing device (10) capable of improving recognition accuracy of an object. The information processing device (10) includes a recognition unit (11) that recognizes a plurality of objects present in a predetermined space included in image data by using a first recognition model and a second recognition model, a selection unit (12) that selects which of a recognition result in the first recognition model and a recognition result in the second recognition model is to be used for each of the objects, and a generation unit (13) that generates recognition data including the selected recognition result of each of the objects, and the first recognition model recognizes the object on the basis of a positional relationship among the objects, while the second recognition model recognizes the object on the basis of a shape of the object.

Description

Information processing device, object recognition method, and non-transitory computer-readable medium

[0001] The present disclosure relates to an information processing device, an object recognition method, and a non-transitory computer-readable medium.

[0002] In order to perform their work efficiently, workers working within a facility such as a substation need information about what objects are located within the facility, as well as information about the placement of objects within the facility.

[0003] Patent Literature 1 discloses the configuration of an object recognition device that uses three-dimensional shape data measured within infrastructure facilities to enable accurate confirmation of the shape and orientation of owned facilities on a computer. Specifically, Patent Literature 1 discloses the configuration of an object recognition device that recognizes the shape of an object using a measurement point cloud and a model point cloud. The object recognition device recognizes an object by matching a measurement point cloud and a model point cloud that have the same shape attributes, such as planes and lines.

[0004] International Publication No. 2014 / 155715

[0005] However, because the object recognition device in Patent Document 1 performs matching using shape attributes, it is unable to recognize objects that have the same shape in the measurement point cloud and the model point cloud but have different uses. As a result, the object recognition device in Patent Document 1 has a problem in that matching accuracy may be reduced.

[0006] In view of the above-mentioned problems, an object of the present disclosure is to provide an information processing device, an object recognition method, and a non-transitory computer-readable medium that can improve the accuracy of object recognition.

[0007] An information processing device according to a first aspect of the present disclosure comprises a recognition unit that uses a first recognition model and a second recognition model to recognize at least one object existing within a predetermined space contained in image data, a selection unit that selects for each of the objects whether to use the recognition result from the first recognition model or the recognition result from the second recognition model, and a generation unit that generates recognition data including the recognition result for each of the selected objects, wherein the first recognition model recognizes the objects based on the positional relationship between the objects, and the second recognition model recognizes the objects based on the shape of the objects.

[0008] An object recognition method according to a second aspect of the present disclosure uses a first recognition model and a second recognition model to recognize at least one object existing within a predetermined space contained in image data, selects for each of the objects whether to use the recognition result from the first recognition model or the recognition result from the second recognition model, generates recognition data including the recognition result for each of the selected objects, and the first recognition model recognizes the objects based on the positional relationship between the objects, and the second recognition model recognizes the objects based on the shape of the objects.

[0009] A non-transitory computer-readable medium according to a third aspect of the present disclosure stores a program that causes a computer to execute the following steps: recognize at least one object that exists within a specified space contained in image data using a first recognition model and a second recognition model; select, for each of the objects, whether to use the recognition result from the first recognition model or the recognition result from the second recognition model; and generate recognition data that includes the recognition result for each of the selected objects; wherein the first recognition model recognizes the objects based on the positional relationship between the objects, and the second recognition model recognizes the objects based on the shape of the objects.

[0010] The present disclosure makes it possible to provide an information processing device, an object recognition method, and a non-transitory computer-readable medium that can improve the accuracy of object recognition.

[0011] FIG. 1 is a configuration diagram of an information processing device according to the present disclosure. FIG. 2 is a diagram showing a flow of object identification processing executed in an information processing device according to the present disclosure. FIG. 3 is a diagram showing a configuration diagram of an information processing device according to the present disclosure. FIG. 4 is a diagram showing an example of recognition processing in a recognition model according to the present disclosure. FIG. 5 is a diagram showing a flow of object identification processing executed in an information processing device according to the present disclosure. FIG. 6 is a diagram showing a flow of object identification processing executed in an information processing device according to the present disclosure.

[0012] (Embodiment 1) Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. An example configuration of an information processing device 10 will be described using FIG. 1. The information processing device 10 may be a computer device that operates when a processor executes a program stored in a memory. Furthermore, the components such as functional blocks that make up the information processing device 10 may be distributed and arranged in multiple information processing devices. Each information processing device may be connected via a network.

[0013] The information processing device 10 includes a recognition unit 11, a selection unit 12, and a generation unit 13. The recognition unit 11, the selection unit 12, and the generation unit 13 may be software or modules that are executed by a processor executing a program stored in a memory, or may be hardware such as a circuit or a chip.

[0014] The recognition unit 11 uses the first recognition model and the second recognition model to recognize at least one object existing in a predetermined space contained in the image data. Recognizing an object means determining which object each of the smallest elements (pixels in two-dimensional image data, points in three-dimensional point cloud data, and voxels in three-dimensional voxel data, hereinafter referred to as "elements") that make up the image data is a part of (identifying the name and category of the object for each element). This recognition method generally uses a technology called semantic segmentation. The predetermined space may be, for example, a closed space within a building or an open outdoor space. In an open outdoor space, for example, the area to be recognized as an object may be predetermined using location information. The location information may be, for example, information indicating latitude and longitude, information such as an address, or information indicating the distance and direction from a predetermined reference point.

[0015] The object may be, for example, an object fixedly installed on the ground or floor, or may be a movable object. The object may also include a human, a non-human animal, a plant, etc. Furthermore, the object may be a device, a component, etc. located within a substation.

[0016] The image data may be data generated by an imaging device or data generated by a sensor that measures distance. The image data may be two-dimensional data or three-dimensional data. The data generated by the sensor that measures distance may be, for example, point cloud data including points having three-dimensional information. Alternatively, the data generated by the sensor that measures distance may be a depth image or a depth map that indicates the depth of an object.

[0017] The first recognition model is a model that recognizes objects based on the positional relationship between the objects. The second recognition model is a model that recognizes objects based on their shapes. The recognition model may be referred to as a machine learning model or a trained model. Recognizing an object may mean recognizing the name of the object, the category to which the object belongs, the properties of the object, etc. Furthermore, recognizing an object may also be rephrased as identifying the object, determining the object, etc.

[0018] The first recognition model uses the positional relationships between objects in the learning process, and therefore learns data for, for example, an entire facility including multiple objects. Therefore, the data size is reduced by processes such as thinning the data so that it can be stored in computer memory. As a result, learning is performed using data that retains information about the positional relationships of objects but loses detailed shape features. By learning data for the entire facility including multiple objects, the first recognition model improves object recognition accuracy for larger objects included in the data. The reason why object recognition accuracy improves for larger objects is that the larger the object size, the more areas or points are used for object matching, and the learning results of this majority of data are more likely to be reflected in updating the learning parameters. In other words, the first recognition model tends to have lower recognition accuracy for smaller objects than larger objects.

[0019] On the other hand, the second recognition model may divide the image data into small regions to include the object to be recognized in order to recognize objects based on the shape of each object. By using image data divided into small regions, the second recognition model improves the object recognition accuracy as the object size becomes smaller. In other words, the second recognition model reduces the object recognition accuracy as the object size becomes larger. The shape of the object may be, for example, a straight line, a curved line, a flat surface, a curved surface, etc.

[0020] The selection unit 12 selects whether to use the recognition results of the first recognition model or the second recognition model for each object. The first recognition model and the second recognition model analyze the same image data. Therefore, the first recognition model and the second recognition model recognize the same object included in the image data. In other words, the recognition results of each object included in the image data are output from the first recognition model and the second recognition model, respectively. Alternatively, the second recognition model may analyze image data excluding the object recognized by the first recognition model.

[0021] The selection unit 12 selects either the recognition result of the first recognition model or the recognition result of the second recognition model for each object included in the image data. The selection unit 12 may select either the recognition result of the first recognition model or the recognition result of the second recognition model according to a predetermined criterion. The criterion may be, for example, the size of the object. Specifically, the selection unit 12 may select the recognition result of the first recognition model if the size of the object is larger than a threshold, and may select the recognition result of the second recognition model if the size of the object is smaller than the threshold. The criterion is not limited to the size of the object.

[0022] The generation unit 13 generates recognition data including the recognition results of each selected object. The recognition data may be data in which the name of each object, the category to which the object belongs, the properties of the object, etc. are specified. Including the recognition results of each object may mean combining the recognition results of each object. The recognition data may be generated as image data. Alternatively, the recognition data may be data in which the position of an object in the image data is associated with the name of the object, etc.

[0023] Next, an object recognition method executed by the information processing device 10 according to the first embodiment will be described with reference to Fig. 2. First, the recognition unit 11 recognizes a plurality of objects existing in a predetermined space included in image data using a first recognition model and a second recognition model (S11). The first recognition model recognizes objects based on the positional relationships between the objects. The second recognition model recognizes objects based on the shapes of the objects.

[0024] Next, the selection unit 12 selects whether to use the recognition results of the first recognition model or the second recognition model for each object (S12). Here, each element has obtained the results of each recognition model. The following methods can be considered for selecting the results of a recognition model. First, the selection unit 12 groups elements determined to be elements of the same object by a clustering process based on the recognition results of the first recognition model. A criterion for grouping is that the distance between elements be lower than a threshold. Next, the selection unit 12 calculates the spatial extent of each group, and if the calculated spatial extent is equal to or greater than a threshold value that indicates a large object, the selection unit 12 adopts the results of the first recognition model. For other elements, the selection unit 12 adopts the results of the second recognition model. Next, the generation unit 13 generates recognition data including the recognition results of each selected object (S13).

[0025] As described above, the information processing device 10 has multiple recognition models with different characteristics. Furthermore, the information processing device 10 selects a recognition model to use for object recognition depending on the object included in the image data. As a result, the information processing device 10 can improve the recognition accuracy of multiple objects included in the image data by using a recognition model with higher recognition accuracy for each object.

[0026] For example, as the size of the facility to be recognized increases, the first recognition model needs to thin out data in order to learn the positional relationships between objects, and the first recognition model is trained using data in which the shape characteristics of each object have been lost. Therefore, when recognizing each object included in the facility, the selection unit 12 may determine to use the second recognition model, which improves recognition accuracy as the size of the object decreases. In this way, the information processing device 10 can improve the recognition accuracy of the entire facility to be recognized by using another recognition model to perform recognition processing on areas where the recognition accuracy of each recognition model is poor.

[0027] (Embodiment 2) Next, an example configuration of an information processing device 20 will be described with reference to Fig. 3. The information processing device 20 has a configuration in which a data input unit 24, a parameter input unit 25, and an output unit 26 are added to the information processing device 10. The data input unit 24, the parameter input unit 25, and the output unit 26 may be software or modules that perform processing when a processor executes a program stored in a memory. Alternatively, the data input unit 24, the parameter input unit 25, and the output unit 26 may be hardware such as a circuit or a chip.

[0028] The recognition unit 21 corresponds to the recognition unit 11 in the information processing device 10. The selection unit 22 corresponds to the selection unit 12 in the information processing device 10. The generation unit 23 corresponds to the generation unit 13 in the information processing device.

[0029] The data input unit 24 inputs, for example, point cloud data as image data to the recognition unit 21. In other words, the data input unit 24 transfers the point cloud data to the recognition unit 21. The point cloud data is a collection of points having three-dimensional information. The point cloud data may be generated, for example, by a sensor that measures distance or an imaging device. The sensor that measures distance may be, for example, a sensor that measures the distance from the sensor to an object using LiDAR (Light Detection and Ranging). The point cloud data may be generated using distance information measured by a sensor that measures distance and position information measured using a GPS (Global Positioning System).

[0030] The various sensors that generate the point cloud data may be mounted on the information processing device 20, or may be connected to the information processing device 20 via a network. The data input unit 24 acquires the point cloud data generated by the various sensors.

[0031] Alternatively, the user may input point cloud data generated by various sensors as offline data to the information processing device 20. Furthermore, the data input unit 24 may generate point cloud data using data measured by various sensors.

[0032] For example, the point cloud data may be generated using software that generates three-dimensional information using multiple two-dimensional image data.

[0033] The recognition unit 21 has recognition models 1 to N (N is an integer equal to or greater than 2). 1 to N are identification information for identifying the recognition models. Each recognition model may have different features. For example, recognition model 1 may be a model that recognizes objects based on the positional relationship between the objects. Recognition model 2 may be a model that recognizes objects based on their shape.

[0034] Each recognition model recognizes the shape of an object, including, for example, straight lines, curves, planes, curved surfaces, etc. Shape recognition may also mean extracting shape features. Shape features may be, for example, information indicating whether an object is linear, planar, or vertical. Specifically, shape features may be information indicating the length, area, volume, curvature, etc. of an object. Furthermore, when point cloud data is used, shape features may be information indicating the spatial distribution of points, such as whether they are distributed linearly, on a plane, or vertically. Furthermore, when data generated by an imaging device is used, shape features may include color information, such as RGB (Red Green Blue) information, of each pixel.

[0035] Each recognition model may extract shape features from the positional relationship between a reference point and points surrounding the reference point. Furthermore, each recognition model may determine which shape each point included in the point cloud data constitutes. In other words, each recognition model may determine whether each point included in the point cloud data is included in a shape such as a straight line, a curve, a plane, or a curved surface.

[0036] Recognition model_2 may determine, for example, based on shape features, which points included in the point cloud data belong to which object. Specifically, recognition model_2 may determine that points showing shape features with high flatness belong to the ground or floor. Recognition model_2 may divide the point cloud data into smaller parts, taking into account the memory size of the information processing device 20 and ensuring that shape features are not lost, and then recognize objects using the point cloud data. Recognition model_2 may perform recognition processing using PointNet, for example.

[0037] Recognition model_1 may recognize which object each group of points showing a specific feature is a part of based on the positional relationship between the groups of points. A group of points showing a specific feature may be, for example, a set of points showing the same shape. A group may also be called a cluster. Recognition model_1 may perform recognition processing using, for example, a Super Point Graph.

[0038] An example of recognition processing in recognition model_1 will be described using Figure 4. Figure 4 shows point cloud data relating to a table and a chair. S1 to S6 represent groups of points. The point cloud included in S2 represents the table top. The point clouds included in S1 and S3 represent the table legs. The point cloud included in S4 represents the chair legs. The point cloud included in S5 represents the chair seat. The point cloud included in S6 represents the chair back and legs.

[0039] Recognition model_1 first recognizes the shapes of S1 to S6. For example, recognition model_1 recognizes that S1, S3, S4, and S6 are point clouds with points distributed vertically. Furthermore, recognition model_1 recognizes that S2 and S5 are point clouds with points distributed horizontally. In other words, recognition model_1 extracts feature quantities of the shapes of S1 to S6.

[0040] Next, recognition model_1 identifies the objects in each group based on the positional relationships between the groups. Recognition model_1 learns the positional relationships between the groups during a learning process. The learning of the positional relationships between the groups may be performed, for example, by a learning device having a GRU (Gated Recurrent Unit) or an LSTM (Long Short-Term Memory). The learning of the positional relationships between the groups may be, for example, recognizing that a group of horizontal shapes that is in contact with multiple groups of vertical shapes is a tabletop. Furthermore, the learning of the positional relationships between the groups may be recognizing that a group of vertical shapes that is in contact with a group of horizontal shapes is a leg.

[0041] 4 is input, recognition model_1 identifies that S1 and S3 are table legs and S2 is the table top. Also, when recognition model_1 receives the point cloud data shown in Fig. 4, it identifies that S4 is a chair leg, S5 is the chair seat, and S6 is the backrest and legs.

[0042] Recognition of a table and a chair by recognition model_1 is an example of a recognition process. For example, when a group showing a cylindrical shape and a cluster showing a linear shape are in a positional relationship where they are adjacent to each other, recognition model_1 may identify the respective groups as a utility pole and a power line. Furthermore, recognition model_1 may recognize a pipe or the like showing a similar shape in a substation as a pipe for carrying electricity or a pipe used for cooling, depending on the position of the pipe relative to other objects arranged around the pipe.

[0043] In order to learn the positional relationships between groups, recognition model_1 needs to use, for example, point cloud data of the entire facility as input data. Recognition model_1 may also thin out the point cloud data so that the input data fits into the memory of the information processing device 20. Thinning out the point cloud data may impair the shape characteristics of each group, but it improves the recognition accuracy of objects that combine each group in the entire facility.

[0044] The recognition unit 21 may further include another recognition model_N having characteristics different from the recognition model_1 and the recognition model_2. Alternatively, the recognition unit 21 may further include a recognition model_N having characteristics similar to the recognition model_1 or the recognition model_2, but with a different learning scale or recognition accuracy, for example.

[0045] Returning to FIG. 3 , recognition model_1 to recognition model_N included in the recognition unit 21 each execute a process of recognizing objects included in the point cloud data using the point cloud data received from the data input unit 24. In other words, recognition model_1 to recognition model_N included in the recognition unit 21 execute parallel processing to recognize objects included in the point cloud data. For example, the data input unit 24 may duplicate the point cloud data by the number of recognition models. The data input unit 24 may input each of the duplicated point cloud data to recognition model_1 to recognition model_N.

[0046] Recognition model_1 to recognition model_N output the object recognition results to the selection unit 22. Recognition model_1 to recognition model_N perform recognition processing using the same point cloud data. Therefore, the selection unit 22 receives up to N recognition results for the same object.

[0047] The selection unit 22 selects one recognition result for each object from a maximum of N recognition results in accordance with a selection criterion received from the parameter input unit 25.

[0048] The selection criterion may be, for example, the size of the object. Specifically, the selection criterion may be a size threshold. The selection unit 22 receives the size threshold from the parameter input unit 25. For objects larger than the threshold, the selection unit 22 may select the recognition result received from recognition model_1. For objects equal to or smaller than the threshold, the selection unit 22 may select the recognition result received from recognition model_2. Recognition model_1 has a characteristic of having higher recognition accuracy for large objects than for small objects. Recognition model_2 has a characteristic of having higher recognition accuracy for small objects than for large objects.

[0049] Assume that the point cloud data input by the data input unit 24 to the recognition unit 21 is data indicating facilities in a substation, for example. The selection unit 22 may select the recognition result of recognition model_1 by determining that transformers, circuit breakers, disconnecting switches, lightning arresters, etc. in the substation are facilities larger than the threshold. The selection unit 22 may select the recognition result of recognition model_2 by determining that insulators, transformer radiators, pipes for carrying cooling oil, etc. in the substation are facilities smaller than the threshold.

[0050] The generation unit 23 generates recognition data by combining the objects selected by the selection unit 22. The generation unit 23 may generate, for example, image data as the recognition data. In other words, the generation unit 23 may generate image data in which objects included in the point cloud data input to the recognition unit 21 by the data input unit 24 are identified.

[0051] The recognition data may be image data in which a different color is assigned to each object. The recognition data may also be image data in which the name of each object is displayed in the area of ​​each object. The recognition data may also be image data in which the same color is assigned to identical or similar shapes. The identical or similar shapes may be shapes in which the difference in feature amount between each shape is within a predetermined range. The recognition data may also be three-dimensional data so that the user can rotate or move the recognition data displayed on the display device.

[0052] The output unit 26 displays the recognition data generated by the generation unit 23 on a display unit such as a display. The display unit such as a display may be used integrally with the information processing device 20, or may be a display device connected to the information processing device 20 via a cable, a network, or the like.

[0053] Next, the flow of object recognition processing in the information processing device 20 will be described with reference to Fig. 5. First, the data input unit 24 inputs point cloud data to the recognition unit 21 (S21). The data input unit 24 may, for example, duplicate the point cloud data by the number of recognition models included in the recognition unit 21. The data input unit 24 may input the duplicated point cloud data to the recognition unit 21.

[0054] Next, each recognition model included in the recognition unit 21 uses the point cloud data to perform a recognition process for each object included in the point cloud data (S22). Each recognition model outputs the recognition result for the object included in the point cloud data to the selection unit 22.

[0055] Next, the selection unit 22 selects a recognition result for each object using the parameters output from the parameter input unit 25 (S23). Each object is recognized by its own recognition model. That is, there are a maximum of as many recognition results for each object as there are recognition models. The selection unit 22 selects one recognition result from among the multiple recognition results for each object. The parameters may also be referred to as selection criteria.

[0056] Next, the generation unit 23 generates recognition data by combining the selected recognition results (S24). For example, assume that the selection unit 22 selects the recognition result of recognition model_1 for object_A and the recognition result of recognition model_2 for object_B. The generation unit 23 combines the recognition result of recognition model_1 for object_A with the recognition result of recognition model_2 for object_B. Combining the recognition results may mean, for example, including multiple recognition results in the same image data. In other words, the image data generated as recognition data may include objects recognized by multiple recognition models.

[0057] Next, the output unit 26 outputs the recognition data to a display unit such as a display (S25). The user can analyze the recognition data by rotating or moving the recognition data displayed on the display.

[0058] As described above, the information processing device 20 selects one recognition result from multiple recognition results output from multiple recognition models according to the size of the object included in the point cloud data. As a result, the information processing device 20 can generate recognition data for each object that combines recognition results with higher recognition accuracy.

[0059] In addition, in the second embodiment, the selection unit 22 uses the size of the object as the selection criterion, but the selection criterion is not limited to the size of the object. The selection unit 22 may select a recognition model based on a feature of the object other than the size.

[0060] For example, the selection unit 22 may separate recognition models for equipment whose size increases according to the size of the facility and equipment whose size does not change regardless of the size of the facility. Specifically, the selection unit 22 may be configured to select the recognition result of recognition model 1 for elements recognized by recognition model 1 as equipment whose size increases according to the size of the facility, and to select the recognition result of recognition model 2 for other elements, which are equipment whose size does not change regardless of the size of the facility.

[0061] The parameter input unit 25 may output information specifying equipment whose size increases according to the size of the facility and equipment whose size does not change regardless of the size of the facility to the selection unit 22. The equipment whose size increases according to the size of the facility may be, for example, steel structures and electric wires in a substation.

[0062] The selection unit 22 may also separate recognition models for equipment that has the same shape but is used for different purposes from equipment that has the same shape but is used for the same purpose. Specifically, the selection unit 22 may be configured to select the recognition result of recognition model 1 for elements recognized by recognition model 1 as equipment that has the same shape but is used for different purposes, and to select the recognition result of recognition model 2 for other elements that are equipment that has the same shape but is used for the same purpose.

[0063] The parameter input unit 25 may output information specifying facilities having the same shape but used for different purposes and facilities having the same shape but used for the same purpose to the selection unit 22. Facilities having the same shape but used for different purposes may be, for example, pipes in a substation.

[0064] The selection unit 22 may be configured to select the recognition result of recognition model 1 for elements recognized by recognition model 1 as equipment with a simple shape, and to select the recognition result of recognition model 2 for other elements, which are equipment with a complex shape. Alternatively, the selection unit 22 may be configured to calculate the complexity of the shape in advance, and to select the result of recognition model 2 for areas with high complexity, and the result of recognition model 1 for areas with low complexity. As a measure of complexity, it is possible to use the change in the direction of the normal vector if the image data is point cloud data, based on the relationship with neighboring elements.

[0065] The parameter input unit 25 may output information specifying the equipment having a simple shape and the equipment having a complex shape to the selection unit 22. The equipment having a simple shape may be, for example, a pipe for flowing cooling oil.

[0066] (Embodiment 3) Next, an example configuration of the information processing device 30 will be described with reference to Fig. 6. The information processing device 30 is configured such that the data input unit 24 inputs point cloud data to one recognition model included in the recognition unit 21, for example, recognition model_1, rather than inputting point cloud data to all recognition models included in the recognition unit 21. Alternatively, the data input unit 24 may input the point cloud data to the selection unit 22. In this case, the selection unit 22 may input the point cloud data to recognition model_1 to recognition model_N in order.

[0067] The parameter input unit 25 outputs information or parameters indicating which object is to be recognized using which recognition model to the selection unit 22. For example, the parameter input unit 25 may use recognition model_1 to recognize transformers, circuit breakers, and disconnectors present in a substation. The parameter input unit 25 may use recognition model_2 to output parameters indicating that radiators and pipes present in the substation are to be recognized to the selection unit 22. Furthermore, the parameter input unit 25 may use recognition model_N (N is an integer of 3 or greater) to output parameters indicating that facilities other than transformers, circuit breakers, disconnectors, radiators, and pipes are to be recognized to the selection unit 22.

[0068] The selection unit 22 selects the recognition result of each object from among the objects recognized in each recognition model based on the parameters. The selection unit 22 selects the recognition result of a specific object from among the objects recognized in recognition model_1. The selection unit 22 may output point cloud data of an object for which the recognition result was not selected to recognition model_2. The selection unit 22 selects the recognition result of a specific object from among the objects recognized in recognition model_2. The selection unit 22 may output point cloud data of an object for which the recognition result was not selected to recognition model_N. In other words, recognition model_1 to recognition model_N included in the recognition unit 21 perform serial processing to recognize objects included in the point cloud data.

[0069] The selection unit 22 outputs a recognition result selected from the recognition results of each recognition model to the generation unit 23. The processes in the generation unit 23 and the output unit 26 are similar to those in the information processing device 20, and therefore detailed description thereof will be omitted.

[0070] Next, the flow of the object recognition process executed in the information processing device 30 will be described with reference to Fig. 7. The point cloud data input to the recognition unit 21 by the data input unit 24 is assumed to be subjected to the recognition process repeatedly in order from recognition model_1 to recognition model_N. The selection unit 22 is also assumed to have a counter that indicates the number of times the recognition process has been executed. The selection unit 22 increments the counter each time the recognition process is executed.

[0071] First, the selection unit 22 determines whether the counter value i exceeds N (S31). If the selection unit 22 determines that the counter value i does not exceed N, the recognition model i performs object recognition processing (S32). If the selection unit 22 determines that the counter value i does not exceed N, the selection unit 22 may output point cloud data of an object not selected from the recognition results of the recognition model_i-1 to the recognition model_i. The initial value of the counter value i may be 1.

[0072] Next, the selection unit 22 selects a recognition result of a specific object from the recognition results of the object in the recognition model_i based on the parameters (S33). The selection unit 22 outputs the selected recognition result to the generation unit 23.

[0073] Next, the selection unit 22 extracts point cloud data of the unselected object from the recognition results of the recognition model_i (S34). In other words, the selection unit 22 may generate point cloud data from the recognition results of the recognition model_i by deleting the point clouds related to the selected object.

[0074] Next, the selection unit 22 increments the counter value i to i+1 (S35). Next, the selection unit 22 returns to step S31 and determines whether the new counter value i exceeds N.

[0075] If the selection unit 22 determines in step S31 that the counter value i exceeds N, the generation unit 23 integrates the selected recognition results (S36). "Integrating" can also be rephrased as "combining." The generation unit 23 integrates the selected recognition results to generate recognition data. Next, the output unit 26 outputs the recognition data to a display unit such as a monitor (S37).

[0076] As described above, recognition model_1 to recognition model_N included in information processing device 30 perform object recognition processing using point cloud data excluding point cloud data of already selected objects. Each time recognition processing is performed, the amount of point cloud data decreases. As a result, the processing load related to the recognition processing performed by the recognition models decreases each time recognition processing is performed.

[0077] FIG. 8 is a block diagram showing an example configuration of the information processing devices 10 to 30 (hereinafter referred to as information processing devices 10, etc.) described in the above-described embodiments. Referring to FIG. 8, the information processing device 10, etc. includes a network interface 1201, a processor 1202, and a memory 1203. The network interface 1201 may be used to communicate with a network node. The network interface 1201 may include, for example, a network interface card (NIC) conforming to the IEEE 802.3 series. IEEE stands for Institute of Electrical and Electronics Engineers.

[0078] The processor 1202 reads and executes software (computer programs) from the memory 1203 to perform the processing of the measurement device 20 described using the flowcharts in the above-described embodiment. The processor 1202 may be, for example, a microprocessor, an MPU, or a CPU. The processor 1202 may include multiple processors.

[0079] The memory 1203 is configured by a combination of volatile memory and non-volatile memory. The memory 1203 may include storage located remotely from the processor 1202. In this case, the processor 1202 may access the memory 1203 via an I / O (Input / Output) interface (not shown).

[0080] 8, the memory 1203 is used to store a group of software modules. The processor 1202 reads and executes these software modules from the memory 1203, thereby performing the processing of the information processing device 10 and the like described in the above-described embodiment.

[0081] As explained using Figure 8, each of the processors possessed by the information processing device 10 etc. in the above-mentioned embodiment executes one or more programs including a group of instructions for causing a computer to perform the algorithm explained using the drawings.

[0082] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0083] The technical ideas of the present disclosure are not limited to the above-described embodiments, and can be modified as appropriate within the scope of the gist of the present disclosure.

[0084] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0085] (Supplementary Note 1) An information processing device comprising: recognition means for recognizing at least one object existing in a predetermined space included in image data using a first recognition model and a second recognition model; selection means for selecting, for each of the objects, whether to use the recognition result of the first recognition model or the recognition result of the second recognition model; and generation means for generating recognition data including the recognition result of each of the selected objects, wherein the first recognition model recognizes the object based on a positional relationship between the objects, and the second recognition model recognizes the object based on a shape of the object. (Supplementary Note 2) The information processing device according to Supplementary Note 1, wherein the selection means selects either the first recognition model or the second recognition model based on features of the object to be recognized. (Supplementary Note 3) The information processing device according to Supplementary Note 1 or 2, wherein the selection means selects either the first recognition model or the second recognition model based on a size of the object to be recognized. (Supplementary Note 4) The information processing device according to Supplementary Note 1 or 2, wherein the selection means selects either the first recognition model or the second recognition model based on a use of the object to be recognized. (Supplementary Note 5) The information processing device according to Supplementary Note 1 or 2, wherein the selection means selects either the first recognition model or the second recognition model based on a ratio of the size of the object to the size of facilities included in the specified space. (Supplementary Note 6) The information processing device according to any one of Supplements 1 to 5, wherein the recognition means processes the image data in parallel using the first recognition model and the second recognition model to recognize the at least one object. (Supplementary Note 7) The information processing device according to Supplementary Note 6, further comprising data input means for inputting point cloud data as the image data to the first recognition model and the second recognition model. (Supplementary Note 8) The information processing device according to any one of Supplements 1 to 5, wherein the recognition means processes the image data in series using the first recognition model and the second recognition model to recognize the at least one object.(Supplementary Note 9) The information processing device according to Supplementary Note 8, further comprising data input means for inputting point cloud data as the image data to the first recognition model or the second recognition model, wherein the recognition means recognizes the at least one object present in the point cloud data using one of the first recognition model and the second recognition model, and then recognizes the at least one object present in the point cloud data using the other of the first recognition model and the second recognition model. (Supplementary Note 10) An object recognition method, comprising: recognizing at least one object present in a predetermined space included in image data using a first recognition model and a second recognition model; selecting, for each of the objects, whether to use the recognition result of the first recognition model or the recognition result of the second recognition model; generating recognition data including the recognition result of each of the selected objects, wherein the first recognition model recognizes the objects based on a positional relationship between the objects, and the second recognition model recognizes the objects based on a shape of the object. (Supplementary Note 11) The object recognition method according to Supplementary Note 10, wherein, when selecting the recognition result, one of the first recognition model and the second recognition model is selected based on features of the object to be recognized. (Supplementary Note 12) The object recognition method according to Supplementary Note 10 or 11, wherein, when selecting the recognition result, one of the first recognition model and the second recognition model is selected based on the size of the object to be recognized. (Supplementary Note 13) The object recognition method according to Supplementary Note 10 or 11, wherein, when selecting the recognition result, one of the first recognition model and the second recognition model is selected based on the use of the object to be recognized. (Supplementary Note 14) The object recognition method according to Supplementary Note 10 or 11, wherein, when selecting the recognition result, one of the first recognition model and the second recognition model is selected based on the ratio of the size of the object to the size of a facility included in the specified space. (Supplementary Note 15) The object recognition method according to any one of Supplementary Notes 10 to 14, wherein when selecting the recognition result, the image data is processed in parallel using the first recognition model and the second recognition model to recognize the at least one object.(Supplementary Note 16) The object recognition method according to Supplementary Note 15, wherein, before recognizing the at least one object, point cloud data is input as the image data to the first recognition model and the second recognition model. (Supplementary Note 17) The object recognition method according to any one of Supplements 10 to 14, wherein, when selecting the recognition result, the image data is serially processed using the first recognition model and the second recognition model to recognize the at least one object. (Supplementary Note 18) The object recognition method according to Supplementary Note 17, wherein, before recognizing the at least one object, point cloud data is input as the image data to the first recognition model or the second recognition model, and after recognizing the at least one object present in the point cloud data using one of the first recognition model or the second recognition model, the at least one object present in the point cloud data is recognized using the other of the first recognition model and the second recognition model. (Supplementary Note 19) A non-transitory computer-readable medium storing a program that causes a computer to execute the following steps: recognize at least one object that exists in a predetermined space included in image data using a first recognition model and a second recognition model; select, for each of the objects, whether to use the recognition result in the first recognition model or the recognition result in the second recognition model; and generate recognition data that includes the recognition result for each of the selected objects, wherein the first recognition model recognizes the objects based on the positional relationship between the objects, and the second recognition model recognizes the objects based on the shape of the objects.

[0086] REFERENCE SIGNS LIST 10 Information processing device 11 Recognition unit 12 Selection unit 13 Generation unit 20 Information processing device 21 Recognition unit 22 Selection unit 23 Generation unit 24 Data input unit 25 Parameter input unit 26 Output unit 30 Information processing device

Claims

1. a recognition means for recognizing at least one object existing in a predetermined space included in image data by using the first recognition model and the second recognition model; a selection means for selecting whether to use a recognition result in the first recognition model or a recognition result in the second recognition model for each of the objects; and a generation means for generating recognition data including a recognition result for each of the selected objects, wherein the first recognition model recognizes the objects based on a positional relationship between the objects, and the second recognition model recognizes the objects based on a shape of the objects.

2. The selection means The information processing apparatus according to claim 1 , wherein one of the first recognition model and the second recognition model is selected based on features of the object to be recognized.

3. The selection means The information processing apparatus according to claim 1 , wherein either the first recognition model or the second recognition model is selected based on a size of the object to be recognized.

4. The selection means The information processing apparatus according to claim 1 , wherein either the first recognition model or the second recognition model is selected based on a use of the object to be recognized.

5. The selection means The information processing apparatus according to claim 1 , further comprising: selecting one of the first recognition model and the second recognition model based on a ratio of a size of the object to a size of a facility included in the predetermined space.

6. The recognition means The information processing apparatus according to claim 1 , wherein the image data is processed in parallel using the first recognition model and the second recognition model to recognize the at least one object.

7. The information processing apparatus according to claim 6 , further comprising a data input unit for inputting point cloud data as the image data into the first recognition model and the second recognition model.

8. The recognition means The information processing apparatus according to claim 1 , wherein the image data is serially processed using the first recognition model and the second recognition model to recognize the at least one object.

9. Recognizing at least one object present in a predetermined space included in the image data using the first recognition model and the second recognition model; selecting, for each of the objects, whether to use a recognition result in the first recognition model or a recognition result in the second recognition model; An object recognition method comprising: generating recognition data including recognition results for each of the selected objects; the first recognition model recognizing the objects based on a positional relationship between the objects; and the second recognition model recognizing the objects based on a shape of the objects.

10. Recognizing at least one object present in a predetermined space included in the image data using the first recognition model and the second recognition model; selecting, for each of the objects, whether to use a recognition result in the first recognition model or a recognition result in the second recognition model; A program for causing a computer to generate recognition data including a recognition result of each of the selected objects, The first recognition model recognizes the objects based on a positional relationship between the objects, and the second recognition model recognizes the objects based on a shape of the objects.