Image processing device and program

Abstract

An image processing device according to one embodiment of the present invention comprises: a selection unit that positions a virtual camera at a predetermined position; a projection unit that receives point cloud data having three-dimensional coordinate information and projects the point cloud data from the virtual camera onto an image; a first recognition unit that recognizes a specific object from the image and generates a recognition label as a recognition result and the reliability of the recognition label; and an optimization unit that, when the object is recognized, searches for a virtual camera position such that the reliability is equal to or greater than a threshold value. The selection unit repositions the virtual camera on the basis of the searched virtual camera position.

Description

Image processing apparatus and program

[0001] One aspect of the present invention relates to an image processing apparatus and a program.

[0002] Using depth cameras, LiDAR (Light Detection and Ranging) and the like, depth information in a three-dimensional space can be measured as point cloud data. Point cloud data is expected to be utilized in creation of high-precision three-dimensional maps used in autonomous driving vehicles, autonomous mobile robots and the like, urban planning, simulations and other applications. In addition, research and development on object recognition from measured point cloud data is widely conducted.

[0003] Many methods for estimating semantic labels of point cloud data through deep learning have been proposed. Compared with image and text information, point cloud data is more difficult to obtain in diverse varieties. For this reason, there has been proposed a method of performing segmentation of point cloud data based on two-dimensional knowledge, by recognizing an image obtained by projecting point cloud data from a virtual camera using a pre-trained general-purpose image recognizer.

[0004] Takmaz, Ayca et al., "OpenMask3D: Open-Vocabulary 3D Instance Segmentation", Advances in Neural Information Processing Systems (NeurIPS), 2023

[0005] In the conventional art, virtual cameras are arranged at predetermined positions. However, since the size of objects varies depending on the type of object, the appropriate arrangement of the image recognizer differs according to the object to be recognized. When imaging is performed from the photographer's viewpoint, relatively large objects such as high-rise buildings, tunnels, and bridges may not fit within the field of view of the virtual camera, and thus may not be recognized with sufficient accuracy.

[0006] The present invention has been made in view of the above circumstances, and an object thereof is to provide a technique for improving the accuracy of object recognition.

[0007] An image processing apparatus according to one aspect of the present invention comprises: a selection unit that places a virtual camera at a predetermined position; a projection unit that receives point cloud data having three-dimensional coordinate information and projects the point cloud data onto an image from the virtual camera; a first recognition unit that recognizes a specific object from the image and generates a recognition label as a recognition result and a confidence level of the recognition label; and an optimization unit that, when the object is recognized, searches for a virtual camera position such that the confidence level is equal to or greater than a threshold. The selection unit then places the virtual camera again based on the searched virtual camera position.

[0008] According to one aspect of the present invention, a technique for improving the accuracy of object recognition can be provided.

[0009] Figure 1 is a block diagram of an image processing device according to the first embodiment of the present invention. Figure 2 is a block diagram showing an example of the hardware configuration of the image processing device. Figure 3 is a flowchart illustrating the operation of the image processing device. Figure 4 is a schematic diagram illustrating the operation of positioning a virtual camera. Figure 5 is a schematic diagram illustrating the operation of searching for the virtual camera position. Figure 6 is a flowchart illustrating the operation of learning the virtual camera position in the image processing device according to the second embodiment of the present invention. Figure 7 is a flowchart illustrating the image recognition operation in the image processing device. Figure 8 is a block diagram of an image processing device according to the third embodiment of the present invention. Figure 9 is a flowchart illustrating the operation of the image processing device.

[0010] Embodiments will be described below with reference to the drawings. The embodiments shown below are illustrative examples of devices and methods for realizing the technical concept of the present invention, and the technical concept of the present invention is not defined by the shape, structure, arrangement, etc. of the components. Each functional block can be realized as hardware, software, or a combination of both. It is not essential that each functional block is distinguished as in the following example. For example, some functions may be performed by functional blocks other than the illustrative functional blocks. Furthermore, the illustrative functional blocks may be further divided into finer functional subblocks. In the following description, elements having the same function and configuration are denoted by the same reference numerals, and redundant descriptions are omitted.

[0011] [1] First Embodiment [1-1] The configuration diagram 1 of the image processing apparatus 1 is a block diagram of the image processing apparatus 1 according to the first embodiment of the present invention. The image processing apparatus 1 comprises a control unit 10, a storage unit 20, a creator designation interface 30, and an output unit 31.

[0012] The memory unit 20 includes a machine learning model storage unit 21 and a virtual camera position information storage unit 22.

[0013] The machine learning model storage unit 21 stores machine learning models related to image recognition. These machine learning models are pre-trained models. The machine learning models are pre-trained models for recognizing specific objects from images. The control unit 10 can read the pre-trained models from the machine learning model storage unit 21.

[0014] The virtual camera position information storage unit 22 stores information regarding the location where the virtual camera is placed (referred to as virtual camera position information). The control unit 10 can store the virtual camera position information in the virtual camera position information storage unit 22 and read the virtual camera position information from the virtual camera position information storage unit 22.

[0015] The CRIE designation interface 30 receives CRIEs from an external device. A CRIE is a command to specify the object to be recognized. The CRIE designation interface 30 transmits the received CRIE to the control unit 10.

[0016] The control unit 10 comprehensively controls the operation of the image processing device 1. The control unit 10 includes a virtual camera selection unit 11, a point cloud projection unit 12, an image recognition unit 13, a virtual camera optimization unit 14, and a point cloud recognition unit 15.

[0017] The virtual camera selection unit 11 positions virtual cameras based on information received from an external device. The virtual camera selection unit 11 also positions virtual cameras based on information received from the virtual camera optimization unit 14. A virtual camera is a virtual camera positioned at a virtual viewpoint and virtually captures images of a three-dimensional space defined by point cloud data.

[0018] The point cloud projection unit 12 receives point cloud data from an external device. It projects the point cloud data onto a two-dimensional image using a virtual camera.

[0019] The image recognition unit 13 recognizes the object specified by the creator from the image. The image recognition unit 13 performs image recognition processing using the machine learning model stored in the machine learning model storage unit 21. The image recognition unit 13 generates a recognition label and the confidence level (also called the confidence score) of this recognition label. The recognition label is the recognition result obtained by the image recognition processing. The confidence level is a numerical value (for example, a number between 0 and 1) that represents the degree of statistical certainty that the recognition label has been correctly detected.

[0020] The virtual camera optimization unit 14 sets a search range for the virtual camera position. The virtual camera optimization unit 14 searches for the virtual camera position based on the search range. Furthermore, the virtual camera optimization unit 14 searches for the virtual camera position in a way that optimizes the virtual camera position.

[0021] The point cloud recognition unit 15 integrates multiple recognition results obtained using multiple virtual cameras. Based on this integrated data, the point cloud recognition unit 15 generates the final recognition result for the point cloud data.

[0022] The output unit 31 outputs the recognition results generated by the point cloud recognition unit 15 to an external device.

[0023] (Hardware configuration of the image processing device 1) Figure 2 is a block diagram showing an example of the hardware configuration of the image processing device 1.

[0024] The image processing device 1 can be configured using a computer. The image processing device 1 includes a processor 40, a program storage unit 41, a data storage unit 42, a communication interface unit (communication I / F unit) 43, an input / output interface unit (input / output I / F unit) 44, an input unit 46, and an output unit 31. The data storage unit 42, the program storage unit 41, the communication interface unit 43, and the input / output interface unit 44 are connected to the processor 40 via a bus 45.

[0025] The processor 40 consists of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and the like. The processor 40 executes the operations of each of the processing units described above.

[0026] The program storage unit 41 includes, for example, a non-volatile memory that can be written to and read at any time, such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), and a non-volatile memory such as ROM (Read Only Memory). The program storage unit 41 stores the programs necessary for the processor 40 to perform various processes according to this embodiment. In other words, the operation of each processing unit described above is realized by having the processor 40 execute the programs stored in the program storage unit 41.

[0027] The data storage unit 42 includes, for example, non-volatile memory such as an HDD or SSD, and volatile memory such as RAM (Random Access Memory). The data storage unit 42 temporarily stores various data acquired and generated by the processor 40 during the process of executing various processes. The data storage unit 42 is also used as a workspace for the processor 40.

[0028] The communication interface unit 43 includes a wired communication module and / or a wireless communication module. The wireless communication module includes a wireless LAN (Local Area Network). The communication interface unit 43 performs interface processing with an external device according to a predetermined communication standard. The communication interface unit 43 is capable of receiving information from an external device and transmitting information to an external device.

[0029] The input / output interface unit 44 is connected to the input unit 46 and the output unit 31, among others. The input / output interface unit 44 performs interface processing with the input unit 46 and the output unit 31, respectively, according to a predetermined communication standard.

[0030] The input unit 46 receives information entered by the user. The input unit 46 includes, for example, a keyboard and a mouse.

[0031] The output unit 31 outputs data generated by the processor 40. The output unit 31 includes, for example, a liquid crystal display device or an organic EL (Electro-Luminescence) display device.

[0032] [1-2] Operation Next, the operation of the image processing device 1 configured as described above will be explained. Figure 3 is a flowchart illustrating the operation of the image processing device 1.

[0033] The point cloud projection unit 12 receives point cloud data from an external device (step S100). The external device includes a device operated by the user. Point cloud data is a collection of points that have three-dimensional position information (three-dimensional coordinate information) and color information. Point cloud data with three-dimensional position information is also called three-dimensional point cloud data. Point cloud data consists of N point data P １ , P ２ , P ３ , ..., P Ｎ It consists of the following. Point cloud data can be measured using depth cameras and LiDAR (Light Detection and Ranging), etc.

[0034] Next, the virtual camera selection unit 11 receives information regarding the virtual camera trajectory from an external device (step S101). The virtual camera trajectory is the path by which the virtual camera moves. Information regarding the virtual camera trajectory is referred to as virtual camera trajectory information. The virtual camera trajectory information is information for positioning the virtual camera and is predetermined by the user.

[0035] Next, the virtual camera selection unit 11 places a virtual camera at a predetermined position based on the virtual camera trajectory information (step S102). For example, the virtual camera selection unit 11 selects one of several positions on the virtual camera trajectory as the virtual camera position.

[0036] Figure 4 is a schematic diagram illustrating the operation of positioning a virtual camera. Figure 4 shows an object 50 placed on the ground and a virtual camera 51. The field of view of the virtual camera 51 is shown by two solid lines. In this embodiment, the virtual camera trajectory is a reference line extending horizontally perpendicular to the vertical direction. The virtual camera selection unit 11 positions the virtual camera 51 at any position on the virtual camera trajectory.

[0037] Next, the point cloud projection unit 12 projects the point cloud data from the virtual camera onto a two-dimensional image (step S103). The point cloud projection unit 12 then generates a two-dimensional image. The point cloud projection unit 12 transmits the image to the image recognition unit 13.

[0038] Next, the image recognition unit 13 receives a crie from an external device via the crie designation interface 30, which specifies the object to be recognized (step S104). For example, the object to be recognized includes objects whose overall appearance cannot be grasped unless viewed from above, and specifically includes tall buildings, bridges, and tunnels.

[0039] Next, the image recognition unit 13 recognizes the object specified by Crea from the image (step S105). The image recognition unit 13 performs image recognition processing using the machine learning model stored in the machine learning model storage unit 21. The image recognition unit 13 generates a recognition label and a confidence level for this recognition label. The image recognition unit 13 transmits the recognition label and confidence level to the virtual camera optimization unit 14.

[0040] Subsequently, the virtual camera optimization unit 14 determines whether a specified object has been recognized based on the recognition label (step S106). If the specified object is not recognized (step S106=No), the process proceeds to step S111.

[0041] If the specified object is recognized (step S106=Yes), the virtual camera optimization unit 14 sets a search range for the virtual camera position (step S107). The search range for the virtual camera position is set in the vertical direction with reference to the virtual camera arranged in step S102.

[0042] Subsequently, the virtual camera optimization unit 14 searches for a virtual camera position based on the search range (step S108). The search for the virtual camera position is performed based on the reliability of the recognition label. When the entire object is included in the image of the virtual camera, the reliability of the recognition label increases. The virtual camera optimization unit 14 searches for the virtual camera position such that the entire object is included in the image of the virtual camera.

[0043] Subsequently, the virtual camera optimization unit 14 recognizes the specified object from the searched virtual camera position (step S109). Specifically, the virtual camera optimization unit 14 transmits the searched virtual camera position information to the virtual camera selection unit 11. The virtual camera selection unit 11 arranges the virtual camera based on the virtual camera position information transmitted from the virtual camera optimization unit 14. Then, the point cloud projection unit 12, the image recognition unit 13, and the virtual camera optimization unit 14 perform image projection processing and image recognition processing.

[0044] FIG. 5 is a schematic diagram for explaining the operation of searching for a virtual camera position. FIG. 5 shows three virtual cameras 51-1 to 51-3 in order from the bottom. The virtual camera 51-1 is a virtual camera arranged on the virtual camera trajectory. A point where the object 50 intersects the virtual camera trajectory is defined as a reference point. The angle of view of each of the virtual cameras 51-1 to 51-3 is indicated by two solid lines.

[0045] The virtual camera optimization unit 14 sets the vertical direction passing through the virtual camera 51-1 as the search range for the virtual camera position. The image captured by the virtual camera 51-1 includes only a part of the object 50. Therefore, the reliability of the recognition label by the virtual camera 51-1 is low.

[0046] The virtual camera 51-2 is arranged on the upper side in the vertical direction of the virtual camera 51-1. The orientation of the virtual camera 51-2 is set such that its optical axis passes through the reference point. The image captured by the virtual camera 51-2 also includes only a part of the object 50. Therefore, the reliability of the recognition label by the virtual camera 51-2 is low. Note that the reliability of the virtual camera 51-2 is higher than that of the virtual camera 51-1.

[0047] The virtual camera 51-3 is arranged on the upper side in the vertical direction of the virtual camera 51-2. The orientation of the virtual camera 51-3 is set such that its optical axis passes through the reference point. The image captured by the virtual camera 51-3 includes the entire object 50. Therefore, the reliability of the recognition label by the virtual camera 51-3 is maximized.

[0048] The virtual camera optimization unit 14 searches for the virtual camera position until the virtual camera position is optimized (step S110). Whether the virtual camera position is optimized is determined based on whether the reliability of the recognition label is maximized. For example, the virtual camera optimization unit 14 determines that the reliability of the recognition label is maximized when the reliability of the recognition label is equal to or greater than a predetermined threshold. The virtual camera optimization unit 14 transmits the recognition label for each virtual camera to the point cloud recognition unit 15.

[0049] The point cloud recognition unit 15 receives the recognition label for each virtual camera from the virtual camera optimization unit 14. When the virtual camera position is optimized (step S110=Yes), the point cloud recognition unit 15 integrates the recognition labels of the plurality of virtual cameras, and generates the final recognition result of the point cloud data based on the integrated data (step S111). The recognition result by the point cloud recognition unit 15 is transmitted to the output unit 31. The output unit 31 outputs the received recognition result to an external device.

[0050] Furthermore, if the virtual camera position is optimized, the virtual camera optimization unit 14 stores the optimized virtual camera position information in the virtual camera position information storage unit 22.

[0051] [1-3] Effects of the First Embodiment According to the first embodiment, for objects that do not fit within the field of view of the virtual camera and have low recognition accuracy (reliability) (for example, large structures such as tall buildings, bridges, and tunnels), image recognition can be performed from an overhead position. This improves the accuracy of object recognition.

[0052] Furthermore, when changing the position of the virtual camera, the search range of the virtual camera is limited to the vertical direction from the reference position (the position of the initially placed virtual camera). This reduces the time required to optimize the virtual camera position. It also reduces the processing load on the control unit 10.

[0053] Furthermore, it becomes possible to recognize objects in point cloud data using existing image recognition systems without having to train the system on objects in point cloud data measured by MMS (Mobile Mapping System) or drones. In addition, even for objects with low recognition accuracy, the recognition accuracy can be improved by using information on objects specified by the user in the system.

[0054] [2] Second Embodiment The second embodiment uses point cloud data for training to pre-train an optimized virtual camera position, and then performs image recognition based on the trained information.

[0055] Figure 6 is a flowchart illustrating the operation of learning the virtual camera position in the image processing device 1 according to the second embodiment of the present invention. The block diagram of the image processing device 1 is the same as that of the first embodiment.

[0056] The point cloud projection unit 12 receives training point cloud data from an external device (step S200). The training point cloud data is point cloud data for learning labels for specific objects. Specific objects include, for example, tall buildings, bridges, and tunnels.

[0057] Next, the virtual camera selection unit 11 places a virtual camera at a predetermined position (step S201). For example, the virtual camera selection unit 11 selects one of a plurality of pre-set positions as the virtual camera position.

[0058] Next, the point cloud projection unit 12 projects the point cloud data from the virtual camera onto a two-dimensional image (step S202). The point cloud projection unit 12 then generates a two-dimensional image.

[0059] Next, the image recognition unit 13 recognizes a designated label from the image (step S203). The designated label is pre-set correct data and contains information about a specific object. The designated label may be stored in the storage unit 20 or received from an external device. The image recognition unit 13 generates a confidence score for the designated label.

[0060] Next, the virtual camera optimization unit 14 sets the search range for the virtual camera position (step S204). The search range for the virtual camera position is set vertically with reference to the virtual camera positioned in step S201.

[0061] Next, the virtual camera optimization unit 14 searches for a virtual camera position based on the search range (step S205). The search for the virtual camera position is performed based on the confidence level of the specified label.

[0062] Next, the virtual camera optimization unit 14 recognizes the specified label from the searched virtual camera position (step S206). The process in step S206 is the same as step S109 in Figure 3.

[0063] Next, the virtual camera optimization unit 14 searches for a virtual camera position until the virtual camera position is optimized (step S207). Whether or not the virtual camera position has been optimized is determined by whether or not the confidence level of the specified label has reached its maximum.

[0064] If the virtual camera position is optimized (step S207 = Yes), the virtual camera optimization unit 14 stores the optimized virtual camera position information in the virtual camera position information storage unit 22 (step S208).

[0065] Figure 7 is a flowchart illustrating the image recognition operation in the image processing device 1. The processes in steps S300 and S301 in Figure 7 are the same as steps S100 and S101 in Figure 3.

[0066] Next, the virtual camera selection unit 11 determines whether the specified label is a target for recognition (step S302). If the specified label is a target for recognition (step S302 = Yes), the virtual camera selection unit 11 reads the virtual camera position information from the virtual camera position information storage unit 22 (step S303). This virtual camera position information is information about the virtual camera position that has been optimized through pre-training.

[0067] Next, the point cloud projection unit 12 projects the point cloud data from the virtual camera onto a two-dimensional image (step S305). The point cloud projection unit 12 then generates a two-dimensional image.

[0068] On the other hand, if the specified label is not the target of recognition (step S302 = No), the virtual camera selection unit 11 places the virtual camera at a predetermined position based on the virtual camera trajectory information (step S304). Then, the process proceeds to step S305.

[0069] Next, the image recognition unit 13 recognizes an object from the image (step S306). The image recognition unit 13 generates a recognition label and a confidence level for this recognition label.

[0070] Next, the point cloud recognition unit 15 generates the final recognition result of the point cloud data based on the recognition labels of the virtual camera (step S307). The recognition result from the point cloud recognition unit 15 is transmitted to the output unit 31. The output unit 31 outputs the received recognition result to an external device.

[0071] According to the second embodiment, optimized virtual camera position information can be obtained by pre-learning designated labels for specific objects. Then, object recognition can be performed on point cloud data using the optimized virtual camera position information. This improves the accuracy of object recognition.

[0072] [3] Third Embodiment The third embodiment further optimizes the parameters of the virtual camera.

[0073] Figure 8 is a block diagram of an image processing apparatus 1 according to a third embodiment of the present invention. The storage unit 20 further comprises a parameter storage unit 23.

[0074] The parameter storage unit 23 stores the parameters of the virtual camera (referred to as virtual camera parameters). The control unit 10 can store the virtual camera parameters in the parameter storage unit 23 and read the virtual camera parameters from the parameter storage unit 23.

[0075] The image recognition unit 13 further uses the virtual camera parameters stored in the parameter storage unit 23 to recognize the object specified in the CRIE from the image. The virtual camera optimization unit 14 sets the virtual camera parameters.

[0076] Figure 9 is a flowchart illustrating the operation of the image processing device 1. The processes in steps S400 to S407 in Figure 9 are the same as the processes in steps S100 to S107 in Figure 3.

[0077] Next, the virtual camera optimization unit 14 searches for a virtual camera position based on the search range (step S408). The search for the virtual camera position is performed based on the confidence level of the recognition label.

[0078] Next, the virtual camera optimization unit 14 sets the virtual camera parameters (step S409). The virtual camera parameters include focal length, rendering information, and image processing information. The rendering information includes shading information, as well as a rotation matrix and translation vector for converting point cloud data to the camera coordinate system. The image processing information includes noise reduction, sharpening, image size, number of pixels, resolution, and brightness.

[0079] Next, the virtual camera optimization unit 14 recognizes a specified object from the searched virtual camera positions (step S410). Specifically, the virtual camera optimization unit 14 transmits information about the searched virtual camera positions to the virtual camera selection unit 11. The virtual camera selection unit 11 positions the virtual cameras based on the information transmitted from the virtual camera optimization unit 14. Then, the point cloud projection unit 12, the image recognition unit 13, and the virtual camera optimization unit 14 perform image recognition processing based on the virtual camera parameters.

[0080] The virtual camera optimization unit 14 searches for the virtual camera position until the virtual camera position is optimized (step S411). The virtual camera optimization unit 14 transmits a recognition label for each virtual camera to the point cloud recognition unit 15. The virtual camera optimization unit 14 also stores the virtual camera parameters related to the optimized virtual camera in the parameter storage unit 23.

[0081] The point cloud recognition unit 15 receives recognition labels for each virtual camera from the virtual camera optimization unit 14. If the virtual camera positions are optimized (step S411 = Yes), the point cloud recognition unit 15 integrates the recognition labels of the multiple virtual cameras and generates the final recognition result of the point cloud data based on this integrated data (step S412). The recognition result from the point cloud recognition unit 15 is transmitted to the output unit 31. The output unit 31 outputs the received recognition result to an external device.

[0082] According to the third embodiment, virtual camera parameters related to the optimized virtual camera can be set and saved. Then, image recognition processing can be performed using the virtual camera parameters. This improves the accuracy of object recognition.

[0083] [Other Embodiments] The functions of the elements disclosed herein may be implemented using circuits or processing circuits that include general-purpose processors, special-purpose processors, integrated circuits, ASICs (Application Specific Integrated Circuits), FPGAs (Field Programmable Gate Arrays), other conventional general-purpose circuits, and / or combinations thereof that are programmed using one or more programs stored in one or more memories, or otherwise configured to perform the disclosed functions. A processor is considered a processing circuit or circuit because it includes transistors and other circuits. A processor may be a programmed processor that executes programs stored in memory.

[0084] In this disclosure, a circuit, unit, or means is hardware that performs the enumerated functions, or hardware programmed to perform the enumerated functions. Hardware may be any hardware disclosed herein that is programmed or configured to perform the enumerated functions.

[0085] Furthermore, the computer instructions contained within the computer program provide logic and routines that enable hardware (e.g., processing circuits or circuits) to perform the methods disclosed herein. This computer program can be implemented in commonly known forms, such as computer-readable storage media, computer program products, memory devices, recording media such as CD-ROMs and DVDs, and / or memory for FPGAs and ASICs.

[0086] The present invention is not limited to the embodiments described above, and can be modified in various ways during implementation without departing from its essence. Furthermore, each embodiment may be combined as appropriate, and in that case, the combined effects can be obtained. Moreover, the above embodiments include various inventions, and various inventions can be extracted by selecting combinations from the multiple constituent elements disclosed. For example, if the problem can be solved and effects obtained even if some constituent elements are deleted from all the constituent elements shown in the embodiment, then the configuration with these deleted constituent elements can be extracted as an invention.

[0087] 1...Image processing unit 10...Control unit 11...Virtual camera selection unit 12...Point cloud projection unit 13...Image recognition unit 14...Virtual camera optimization unit 15...Point cloud recognition unit 20...Storage unit 21...Machine learning model storage unit 22...Virtual camera position information storage unit 23...Parameter storage unit 30...Creator specification interface 31...Output unit 40...Processor 41...Program storage unit 42...Data storage unit 43...Communication interface unit 44...Input / output interface unit 45...Bus 46...Input unit

Claims

1. An image processing device comprising: a selection unit that places a virtual camera at a predetermined position; a projection unit that receives point cloud data having three-dimensional coordinate information and projects the point cloud data onto an image from the virtual camera; a first recognition unit that recognizes a specific object from the image and generates a recognition label as a recognition result and a confidence level of the recognition label; and an optimization unit that, when the object is recognized, searches for a virtual camera position such that the confidence level is equal to or greater than a threshold, wherein the selection unit places the virtual camera again based on the searched virtual camera position.

2. The image processing apparatus according to claim 1, wherein the optimization unit searches for the virtual camera position along the vertical direction with respect to the virtual camera.

3. The image processing apparatus according to claim 1, further comprising a second recognition unit that generates recognition results for the point cloud data based on multiple recognition levels based on multiple virtual cameras.

4. A program for causing a computer to function as each part of the image processing apparatus according to any one of claims 1 to 3.

Images