Data processing device and data processing method

Abstract

Efficiently collect large amounts of point cloud data by sensing objects from various positions. [Solution] The data processing device includes a virtual space management unit that manages a virtual space that simulates real space, an object placed in the virtual space, a first moving body having a first point cloud sensor capable of moving in the virtual space, and a second moving body having a second point cloud sensor capable of moving in the virtual space, and an information acquisition unit that stores in a measurement information table first point cloud data obtained by the first point cloud sensor of the first moving body sensing the object from a first position, and second point cloud data obtained by the second point cloud sensor of the second moving body sensing the object from a second position different from the first position.

Description

[Technical field]

[0001] The present disclosure relates to a data processing device and a data processing method. [Background technology]

[0002] A large amount of training data is required to train artificial intelligence (AI) using deep learning etc. In the case of AI that recognizes point cloud data, it is necessary to prepare a large amount of data with correct labels for each region (segment) of the point cloud data, but doing this manually takes time and effort.

[0003] Therefore, it has been proposed to use a virtual space that mimics the real world to acquire point cloud data and automatically assign correct labels to it. Since the virtual space is created by combining known models, it is known what each part of it is. Therefore, if point cloud data is acquired in the virtual space, it is possible to automatically assign correct labels that indicate what the area is, and it is possible to create a large amount of learning data without spending a lot of time and effort. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2023-165453 A Summary of the Invention [Problem to be solved by the invention]

[0005] Sensors (e.g., LiDAR (Light Detection And Ranging)) that acquire point cloud data in real space are equipped and used on various types of moving objects (e.g., automobiles, drones, etc.), and the characteristics of the point cloud data acquired differ depending on the type of moving object. Therefore, for example, point cloud data collected by a smartphone held by a person in a virtual space as disclosed in Patent Document 1 is insufficient for learning artificial intelligence.

[0006] An object of the present disclosure is to provide a technology that can efficiently collect a large amount of point cloud data obtained by sensing an object from various positions. [Means for solving the problem]

[0007] One aspect of the present disclosure provides a data processing device comprising: a virtual space that mimics real space, an object placed in the virtual space, a first moving body having a first point cloud sensor capable of moving in the virtual space, and a second moving body having a second point cloud sensor capable of moving in the virtual space; and an information acquisition unit that stores in a measurement information table first point cloud data obtained by the first point cloud sensor of the first moving body sensing the object from a first position, and second point cloud data obtained by the second point cloud sensor of the second moving body sensing the object from a second position different from the first position.

[0008] One aspect of the present disclosure provides a data processing method that manages a virtual space that mimics real space, an object placed in the virtual space, a first moving body having a first point cloud sensor capable of moving in the virtual space, and a second moving body having a second point cloud sensor capable of moving in the virtual space, and stores in a measurement information table first point cloud data obtained by the first point cloud sensor of the first moving body sensing the object from a first position, and second point cloud data obtained by the second point cloud sensor of the second moving body sensing the object from a second position different from the first position.

[0009] Furthermore, these comprehensive or specific aspects may be realized by a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized by any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. Effect of the Invention

[0010] According to the present disclosure, it is possible to efficiently collect a large amount of point cloud data by sensing an object from various positions. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example of the configuration of a data processing device according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a schematic diagram showing an example of a virtual space according to the present embodiment. [Diagram 3] FIG. 4 is a diagram showing an example of a measurement information table according to the present embodiment. [Figure 4] 10 is a flowchart showing an example of a method for collecting measurement information in a virtual space according to the present embodiment. [Diagram 5] 3 is a flowchart showing an example of a learning method for the segmentation AI according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed explanations than necessary may be omitted. For example, detailed explanations of already well-known matters and duplicate explanations of substantially identical configurations may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the attached drawings and the following explanation are provided for those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims. In addition, the function of one configuration shown in this embodiment may be realized by two or more physical configurations, or the function of two or more configurations may be realized by, for example, one physical configuration.

[0013] (Present embodiment) <System configuration> FIG. 1 is a block diagram showing an example of the configuration of a data processing device 100 according to the present embodiment.

[0014] The data processing device 100 includes a processor 101, a memory 102, a storage 103, a communication unit 104, an input unit 105, and a display unit 106.

[0015] The processor 101 realizes the functions of the data processing device 100 according to this embodiment by reading and processing programs and data from the memory 102 or the storage 103. Details of the functions will be described later. The processor 101 may be interpreted as a CPU (Central Processing Unit), a control device, a controller, or the like. The processor 101 may also include a GPU (Graphics Processing Unit) and / or an NPU (Neural network Processing Unit).

[0016] The memory 102 is configured with a volatile storage medium and / or a non-volatile storage medium, and stores programs and data. The memory 102 may be interpreted as a RAM (Random Access Memory).

[0017] The storage 103 is configured by a non-volatile storage medium (for example, a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc.), and stores programs and data.

[0018] The communication unit 104 transmits and receives data to and from other devices (for example, the terminal 60) through the communication network 50. Examples of the communication network 50 include a wired LAN (Local Area Network), a wireless LAN, the Internet, and a mobile communication network.

[0019] The input unit 105 is a device for a user to input instructions, and is, for example, a keyboard, a mouse, a touch pad, a touch panel, a microphone, or the like.

[0020] The display unit 106 is a device for displaying images or information, and is, for example, a liquid crystal display or an organic EL display.

[0021] The data processing device 100 has, as functions, a virtual space management unit 111, an information acquisition unit 112, an AI learning unit 113, a segmentation AI 120, and a measurement information table 200. Next, the virtual space management unit 111 will be described with reference to FIG.

[0022] In this embodiment, an example in which the user directly operates the data processing device 100 will be described, but the user may operate the data processing device 100 via a terminal 60 connected to the communication network 50. Examples of the terminal 60 include a PC, a tablet terminal, a smartphone, and the like.

[0023] FIG. 2 is a schematic diagram showing an example of a virtual space 10 according to the present embodiment.

[0024] The virtual space management unit 111 manages and controls the virtual space 10 that imitates the real space. As illustrated in FIG. 2, the virtual space management unit 111 generates the virtual space 10 in which virtual objects (3D objects) that imitate objects in the real space are arranged. For example, the virtual space 10 includes stationary objects 11 that do not move, such as buildings, roads, guardrails, sidewalks, bridges, traffic lights, and utility poles, and movable objects 12 that can move, such as automobiles 13, drones 14, bicycles (not shown), smartphones 16 carried by people 15, flying objects (not shown), and motorcycles (not shown). The virtual space management unit 111 also controls the behavior of objects that imitate the physical laws of the real space in the virtual space 10. That is, the virtual space management unit 111 emulates the real space in the virtual space 10.

[0025] The virtual space management unit 111 may display on the display unit 106 the state of the virtual space 10 as exemplified in FIG.

[0026] Furthermore, the virtual space management unit 111 controls the movement of the moving object 12 in the virtual space 10. The movement of the moving object 12 may be performed manually by a user via the input unit 105, or may be performed automatically by the virtual space management unit 111 based on a route or rules set in advance.

[0027] The mobile body 12 is provided with a point cloud sensor 20 capable of sensing an object in the virtual space 10 and acquiring three-dimensional point cloud data. The point cloud sensor 20 may be a sensor simulating a LiDAR or the like in the real space. In other words, the point cloud sensor 20 irradiates a laser in the virtual space 10 and generates point cloud data based on the light reflected by the laser from the object.

[0028] Furthermore, the point cloud sensor 20 provided on the moving body 12 in the virtual space 10 may mimic the position and performance of a point cloud sensor provided on an actual moving body in real space. For example, if the type of moving body 12 is an automobile, the point cloud sensor 20 provided on the automobile 13 in the virtual space 10 may mimic the position and performance of a LiDAR provided on an actual automobile. This makes it possible to obtain point cloud data in the virtual space 10 that is closer to that of the real space.

[0029] 2, a point cloud sensor 20 is provided on each of an automobile 13, a drone 14, and a smartphone 16 carried by a person 15 in a virtual space 10. The point cloud sensor 20 may be provided on a stationary body 11, not limited to a moving body 12. For example, the point cloud sensor 20 may be provided on a utility pole, a wall, a traffic light, or the like in the virtual space 10.

[0030] The virtual space management unit 111 manages the position, shape, etc. of each stationary body 11 in the virtual space 10. The virtual space management unit 111 also manages a moving body ID for uniquely identifying each moving body 12 in the virtual space 10, the shape of each moving body 12, the type of each moving body 12, and the performance of the point cloud sensor 20 equipped to each moving body 12. The virtual space management unit 111 also manages the position, attitude, speed, and moving direction of each moving body 12 in the virtual space 10 at each time. The virtual space management unit 111 also manages point cloud data obtained by sensing by the point cloud sensor 20 of each moving body 12 in the virtual space 10 at each time. That is, the virtual space management unit 111 manages all information related to the virtual space 10.

[0031] Returning to the explanation of Figure 1.

[0032] The information acquisition unit 112 cooperates with the virtual space management unit 111 to acquire information on each stationary body 11 in the virtual space 10. The information acquisition unit 112 also cooperates with the virtual space management unit 111 to acquire information on each moving body 12 in the virtual space 10. The information acquisition unit 112 also cooperates with the virtual space management unit 111 to acquire point cloud data obtained by sensing with the point cloud sensor 20 of each stationary body 11 and / or each moving body 12 in the virtual space 10. The information acquisition unit 112 also identifies the type of object 30 included in the acquired point cloud data and the area of ​​the object 30 in the point cloud data. The information acquisition unit 112 stores this information in the measurement information table 200. Details of the process performed by the information acquisition unit 112 will be described later (see FIG. 4).

[0033] The measurement information table 200 stores information and the like acquired by the information acquisition unit 112 from the virtual space 10. Details of the measurement information table 200 will be described later (see FIG. 3).

[0034] The AI ​​learning unit 113 uses information stored in the measurement information table 200 to train the segmentation AI 120. The segmentation AI 120 is an AI for performing semantic segmentation on point cloud data. For example, when point cloud data is input, the segmentation AI 120 outputs the type (semantic) of an object included in the point cloud data and the area (segment) of the object in the point cloud data. Details of the AI ​​learning unit 113 and the segmentation AI 120 will be described later (see FIG. 5).

[0035] <Measurement information table> FIG. 3 is a diagram showing an example of the measurement information table 200 according to the present embodiment.

[0036] 3, the measurement information table 200 has, as items, a time 201, a moving object ID 202, a moving object type 203, a moving object position 204, a moving object attitude 205, a moving object speed 206, a moving object movement direction 207, a point cloud sensor performance 208, point cloud data 209, and object segment information 210. However, the measurement information table 200 may have a configuration including only a part of these items.

[0037] Time 201 indicates the time in the virtual space 10. Note that instead of the time, the elapsed time from a predetermined time in the virtual space 10 may be used.

[0038] The moving body ID 202 is an ID for uniquely identifying the moving body 12 in the virtual space 10.

[0039] The moving object type 203 indicates the type of the moving object 12 of the moving object ID 202. Examples of the moving object type 203 include an automobile, a drone, a bicycle, a smartphone carried by a person, an airborne object, and a motorcycle.

[0040] The moving object position 204 indicates the position of the moving object 12 having the moving object ID 202 at the time 201. The moving object position 204 may be indicated by three-dimensional coordinates (x, y, z) in the virtual space 10.

[0041] The moving body attitude 205 indicates the attitude of the moving body 12 of the moving body ID 202 at the time 201. The moving body attitude 205 may be indicated by a roll angle (θ), a pitch angle (φ), and a yaw angle (ψ) with respect to the three-dimensional coordinates of the virtual space 10.

[0042] The moving object speed 206 indicates the moving speed of the moving object 12 with the moving object ID 202 at the time 201 .

[0043] The moving object moving direction 207 indicates the moving direction of the moving object 12 of the moving object ID 202 at the time 201. The moving object moving direction 207 may be indicated by a three-dimensional vector (a, b, c) in the virtual space 10.

[0044] The point cloud sensor performance 208 indicates the performance of the point cloud sensor 20 provided in the moving body 12 having the moving body ID 202. The point cloud sensor performance 208 is, for example, a sensing method of the point cloud sensor 20, a sensing cycle of the point cloud sensor 20, a distance and a range that the point cloud sensor 20 can sense, etc. Information on the performance of the point cloud sensor 20 provided in each moving body 12 may be held in advance by the virtual space management unit 111.

[0045] The point cloud data 209 indicates point cloud data measured at a time 201 by the point cloud sensor 20 provided on the moving object 12 having the moving object ID 202 .

[0046] The object segment information 210 is information indicating a segment of the object 30 included in the point cloud data 209. Here, the object 30 is one of the objects existing in the virtual space 10. In this embodiment, one object of interest among the objects existing in the virtual space 10 is referred to as the object 30. In other words, the object 30 may be read as an object. The object 30 may be either a stationary object 11 or a moving object 12. The object 30 may be selected by the user from among the objects existing in the virtual space 10. The object segment information 210 may include information indicating the type (semantic) of an object included in the point cloud data 209 and information indicating the area (segment) of the object in the point cloud data 209.

[0047] 3 indicates that an automobile 13, which is a moving object 12 with a moving object ID of "1" existing in a virtual space 10, is located at (x1, y1, z1) at time "t1", has an attitude of (θ1, φ1, ψ1), moves in a direction of (a1, b1, c1) at a speed of "v1", is equipped with a point cloud sensor 20 with "performance 1", and "first point cloud data" has been measured by the point cloud sensor 20. Furthermore, the object segment information in the first line indicates that the first point cloud data includes an object A (for example, the object 30 in the case of FIG. 2 is a "bridge"), and indicates the area (segment) of the object A in the first point cloud data.

[0048] For example, the second line in FIG. 3 indicates that the drone 14, which is a moving object 12 with a moving object ID of "2" existing in the virtual space 10, is located at (x2, y2, z2) at time "t1", has an attitude of (θ2, φ2, ψ2), moves in the direction of (a2, b2, c2) at a speed of "v2", is equipped with a point cloud sensor 20 with "performance 2", and second point cloud data has been measured by the point cloud sensor 20. Furthermore, the object segment information in the second line indicates that the second point cloud data includes object A (for example, object 30 in the case of FIG. 2 is a "bridge"), and indicates the area (segment) of object A in the second point cloud data.

[0049] The measurement information table 200 may store information acquired by the point cloud sensor 20 of the stationary body 11, not limited to the moving body 12. In this case, the ID of the stationary body 11 may be stored in the moving body ID 202, the type of the stationary body 11 may be stored in the moving body type 203, the position of the stationary body 11 may be stored in the moving body position 204, and the performance of the point cloud sensor 20 of the stationary body 11 may be stored in the point cloud sensor performance 208. In addition, the point cloud data acquired by the point cloud sensor 20 of the stationary body 11 may be stored in the point cloud data 209, and segment information indicating the type and area of ​​the object in the point cloud data may be stored in the object segment information 210. Furthermore, zero or NULL may be stored in the moving body attitude 205, the moving body speed 206, and the moving body moving direction.

[0050] <Collecting information in virtual space> 4 is a flowchart showing an example of a method for collecting measurement information in virtual space 10 according to this embodiment. Next, a method (processing) for data processing device 100 to collect measurement information from virtual space 10 will be described with reference to FIG.

[0051] The virtual space management unit 111 moves each moving object 12 in the virtual space 10 (S101). The movement of the moving object 12 may be performed automatically based on a predetermined route or rule, or may be performed manually by the user.

[0052] The information acquisition unit 112 cooperates with the virtual space management unit 111 to identify multiple moving objects 12 that are sensing the target object 30 at the same time in the virtual space 10 (S102). The target object 30 may be designated in advance by a user, or may be automatically designated by the information acquisition unit 112 from among objects present in the virtual space 10 based on predetermined conditions. Also, one target object 30 may be designated, or multiple targets 30 may be designated.

[0053] The information acquisition unit 112, in cooperation with the virtual space management unit 111, acquires the sensing time, moving body ID, moving body type, moving body position, moving body posture, moving body speed, moving body movement direction, point cloud sensor performance, and point cloud data for each moving body 12 sensing the target object 30 identified in step S102 (S103).

[0054] The information acquisition unit 112, in cooperation with the virtual space management unit 111, identifies segment information of the object 30 included in the point cloud data obtained in step S103 (S104). Since the relative positions and attitudes of the object 30 and the moving body 12 are all known in the virtual space 10, the information acquisition unit 112 can identify which areas of the point cloud data are segments of the object 30 and the type of the object 30 based on this information.

[0055] The information acquisition unit 112 stores the information and point cloud data of each moving object 12 acquired in step S103, and the segment information of the target object 30 identified in step S104 in each item (201 to 210) of the measurement information table 200 (S105). Then, the process returns to step S101.

[0056] 3, the measurement information table 200 stores point cloud data 209 obtained by sensing the object 30 from various positions and directions by various moving bodies 12, and segment information of the object 30 included in the point cloud data 209. Furthermore, the measurement information table 200 stores the position, attitude, speed, etc. of the moving body 12 at the time of sensing.

[0057] For example, in the case of the virtual space 10 shown in Figure 2, at the same time t1, first point cloud data obtained by the point cloud sensor 20 of the automobile 13 sensing the object 30, a "bridge", from the road, segment information of the object "bridge" in the first point cloud data, second point cloud data obtained by the drone 14 sensing the same object 30, a "bridge", from the sky, and segment information of the object "bridge" in the second point cloud data are stored in the measurement information table 200.

[0058] In this manner, according to the present embodiment, it is possible to simultaneously collect point cloud data obtained by sensing the target object 30 by various moving bodies 12 in the virtual space 10. Therefore, it is possible to efficiently collect a large amount of point cloud data to be used for training the segmentation AI 120, and it is also possible to efficiently collect point cloud data specific to various moving bodies 12.

[0059] <Segmentation AI learning> 5 is a flowchart showing an example of a learning method for the segmentation AI 120 according to the present embodiment. Next, a method (processing) for the data processing device 100 to learn the segmentation AI 120 using information in the measurement information table 200 will be described with reference to FIG.

[0060] The AI ​​learning unit 113 selects an object 30 for learning (S201). The object 30 may be specified by the user, or may be specified by the AI ​​learning unit 113 from among objects present in the virtual space 10 based on predetermined conditions. In addition, one object 30 may be specified, or multiple objects 30 may be specified.

[0061] The AI ​​learning unit 113 extracts a row having object segment information corresponding to the object 30 selected in step S201 from the measurement information table 200 (S202). For example, when a specific "bridge" is specified as the object 30, the AI ​​learning unit 113 extracts a row (measurement information) having object segment information 210 of the "bridge" from the measurement information table 200. The extracted row (measurement information) may be interpreted as teacher data.

[0062] The AI ​​learning unit 113 selects one unselected row from the rows extracted in step S202 (S203).

[0063] The AI ​​learning unit 113 uses the moving object type 203 and point cloud data 209 of the row selected in step S203 as input data, and the object segment information 210 of the selected row as correct answer data, to train the segmentation AI 120 (S204).

[0064] The AI ​​learning unit 113 determines whether or not all of the rows extracted in step S202 have been selected (S205). If there are any unselected rows remaining among the rows extracted in step S202 (S205: NO), the AI ​​learning unit 113 returns the process to step S203. If all of the rows extracted in step S202 have been selected (S205: YES), the AI ​​learning unit 113 ends this process. Note that the AI ​​learning unit 113 may also perform the process shown in FIG. 5 for other objects.

[0065] This makes it possible to generate a segmentation AI 120 that can input point cloud data obtained by sensing with a point cloud sensor equipped on a moving object in real space and the type of the moving object, and can accurately output a segmentation of the object area for the point cloud data and the type (label) of the object.

[0066] If the type of moving body is different, the obtained point cloud data may be different even when the same object is sensed. For example, the area (segment) of the bridge in the point cloud data obtained by sensing a bridge (an example of a target object 30) from a road by an automobile 13 (an example of a moving body 12) is different from the area (segment) of the bridge in the point cloud data obtained by sensing the same bridge from the sky by a drone 14.

[0067] In contrast, the segmentation AI 120 according to the present embodiment has been trained using point cloud data obtained from different types of moving objects 12 in the virtual space 10. Therefore, the segmentation AI 120 can perform object segmentation with high accuracy even for point cloud data obtained by different types of moving objects in the real space.

[0068] Furthermore, even when the same object is sensed, the characteristics (e.g., noise, etc.) of the obtained point cloud data may differ if the attitude and speed of the moving body are different. For example, the characteristics of point cloud data obtained by sensing a bridge while the drone 14 is flying at an attitude (θ2a, φ2a, ψ2a) and a speed v2a may differ from those of point cloud data obtained by sensing a bridge while the same drone 14 is flying at an attitude (θ2b, φ2b, ψ2b) and a speed v2b.

[0069] Therefore, in step S204, the AI ​​learning unit 113 may use the moving body posture 205 and moving body speed 206 as input data in addition to the moving body type 203 and point cloud data 209 of the row selected in step S203, and may train the segmentation AI 120 using the object segment information 210 of the selected row as correct answer data.

[0070] This makes it possible to generate a segmentation AI 120 that can input point cloud data obtained by sensing a moving object in real space, as well as the type, attitude, and speed of the moving object, and accurately output a segmentation of the object area for the point cloud data and the type (label) of the object.

[0071] Furthermore, even when the same object is sensed, the characteristics (e.g., resolution, noise, etc.) of the point cloud data obtained may differ depending on the performance of the point cloud sensor 20. For example, the characteristics of point cloud data obtained by sensing a bridge with a point cloud sensor 20 having a first performance may differ from those of point cloud data obtained by sensing a bridge with a point cloud sensor 20 having a second performance.

[0072] Therefore, in step S204, the AI ​​learning unit 113 may learn the segmentation AI 120 using the point cloud sensor performance 208 as input data in addition to the moving object type 203 and point cloud data 209 of the row selected in step S203, and the object segment information 210 of the selected row as correct answer data.

[0073] This makes it possible to generate a segmentation AI 120 that can input point cloud data obtained by sensing a moving object in real space and the performance of the point cloud sensor of the moving object, and can accurately output segmentation of object areas in the point cloud data and the type (label) of the object.

[0074] That is, the AI ​​learning unit 113 may learn the segmentation AI 120 using at least one of the moving body type 203, moving body position 204, moving body posture 205, moving body speed 206, moving body movement direction 207, and point cloud sensor performance 208 in the measurement information table 200 and the point cloud data 209 as input data, and the object segment information 210 as correct answer data.

[0075] (Summary of this embodiment) The above description of the present embodiment discloses the following techniques.

[0076] <Technology 1> A data processing device (100) according to this embodiment includes a virtual space (10) simulating a real space, an object (30) arranged in the virtual space, a first moving body (12) having a first point cloud sensor (20) capable of moving in the virtual space, and a second moving body (12) having a second point cloud sensor (20) capable of moving in the virtual space; and an information acquisition unit (112) that stores in a measurement information table (200) first point cloud data obtained by the first point cloud sensor of the first moving body sensing the object from a first position and second point cloud data obtained by the second point cloud sensor of the second moving body sensing the object from a second position different from the first position. This allows multiple point cloud data obtained by sensing the same object from different positions in the virtual space to be stored in the measurement information table. In other words, it is possible to efficiently collect a large amount of point cloud data obtained by sensing the object from various positions.

[0077] <Technology 2> In the data processing device according to Technique 1, the first moving object and the second moving object are different in type from each other. This allows multiple point cloud data obtained by sensing different types of moving objects to be stored in the measurement information table, making it possible to efficiently collect a large amount of point cloud data obtained by sensing targets from various types of moving objects.

[0078] <Technology 3> In the data processing device according to Technique 1 or 2, the first moving body and the second moving body have different moving speeds. This allows multiple point cloud data obtained by sensing from moving objects with different moving speeds to be stored in the measurement information table, which means that a large amount of point cloud data obtained by sensing objects from moving objects with various moving speeds can be efficiently collected.

[0079] <Technology 4> In the data processing device according to Technique 2 or 3, the type of the first moving object or the second moving object is a car, a motorcycle, a bicycle, a person, an aircraft, or a drone. This allows point cloud data obtained from cars, motorcycles, bicycles, people, flying objects, or drones in virtual space to be stored in the measurement information table.

[0080] <Technology 5> In the data processing device according to any one of claims 1 to 4, the first point cloud sensor and the second point cloud sensor have different performances. This allows multiple point cloud data obtained by sensing from point cloud sensors with different performance to be stored in the measurement information table. In other words, it is possible to efficiently collect a large amount of point cloud data obtained by sensing an object from point cloud sensors with various performances.

[0081] <Technology 6> In a data processing device described in any one of Techniques 2 to 5, the information acquisition unit associates the first point cloud data with a type of the first moving body, and associates the second point cloud data with a type of the second moving body, and stores them in the measurement information table. This allows the measurement information table to store the type of moving body from which point cloud data was obtained in association with the point cloud data.

[0082] <Technology 7> In the data processing device described in Technology 5, the information acquisition unit associates the first point cloud data with the type of the first moving body and the performance of the first point cloud sensor, and associates the second point cloud data with the type of the second moving body and the performance of the second point cloud sensor, and stores them in the measurement information table. This allows the measurement information table to store the type of moving body from which the point cloud data was obtained and the performance of the point cloud sensor in association with the point cloud data.

[0083] <Technology 8> In the data processing device described in any one of Techniques 1 to 7, a stationary third point cloud sensor arranged in the virtual space is further managed, and the information acquisition unit acquires third point cloud data obtained by the third point cloud sensor sensing the object, and stores the third point cloud data in the measurement information table. This makes it possible to store a plurality of third point cloud data obtained by sensing the object with the stationary point cloud sensor in the measurement information table.

[0084] <Technology 9> In a data processing device described in any one of Techniques 1 to 8, the information acquisition unit associates the first point cloud data with object segment information indicating the area and type of the object in the first point cloud data, and associates the second point cloud data with object segment information indicating the area and type of the object in the second point cloud data, and stores them in the measurement information table. This allows the measurement information table to store object segment information indicating the area and type of object in point cloud data obtained by various moving bodies in association with the point cloud data.

[0085] <Technology 10> The data processing device described in Technology 9 further includes an AI learning unit (113) that uses the information stored in the measurement information table to train an AI (e.g., segmentation AI 120) that performs segmentation on point cloud data obtained by a point cloud sensor provided on a mobile body capable of moving in the real space. The measurement information table stores a large amount of point cloud data obtained by sensing various moving objects in virtual space and the corresponding object segment information, which enables the AI ​​learning unit to efficiently train the AI ​​to perform object segmentation using point cloud data obtained by sensing various moving objects in real space.

[0086] <Technology 11> The data processing method manages a virtual space (10) that mimics real space, an object (30) placed in the virtual space, a first moving body (12) having a first point cloud sensor (20) capable of moving in the virtual space, and a second moving body (12) having a second point cloud sensor (20) capable of moving in the virtual space, and stores in a measurement information table (200) first point cloud data obtained by the first point cloud sensor of the first moving body sensing the object from a first position, and second point cloud data obtained by the second point cloud sensor of the second moving body sensing the object from a second position different from the first position. This allows multiple point cloud data obtained by sensing the same object from different positions in the virtual space to be stored in the measurement information table. In other words, it is possible to efficiently collect a large amount of point cloud data obtained by sensing the object from various positions.

[0087] Although the embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, corrections, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also belong to the technical scope of the present disclosure. In addition, the components in the above-mentioned embodiments may be arbitrarily combined within the scope of the invention. [Industrial Applicability]

[0088] The technology disclosed herein is useful for collecting point cloud data with different characteristics, training artificial intelligence on point cloud data, and utilizing the artificial intelligence. [Explanation of symbols]

[0089] 10 Virtual Space 11 Stationary object 12 Mobile 13. Automobiles 14. Drone 15 people 16 Smartphone 20 Point Cloud Sensor 30 Objects 50 Communication Network 60 terminals 100 Data processing device 101 Processor 102 Memory 103 Storage 104 Communications Department 105 Input section 106 Display section 111 Virtual Space Management Department 112 Information Acquisition Department 113 AI Learning Department 120 Segmentation AI 200 Measurement Information Table 201 Time 202 Mobile ID 203 Mobile Type 204 Mobile object position 205 Mobile body posture 206 Moving object speed 207 Moving object movement direction 208 Point Cloud Sensor Performance 209 Point Cloud Data 210 Object Segment Information

Claims

1. a virtual space management unit that manages a virtual space simulating a real space, an object placed in the virtual space, a first moving body having a first point cloud sensor capable of moving in the virtual space, and a second moving body of a different type from the first moving body having a second point cloud sensor capable of moving in the virtual space; an information acquisition unit that associates first point cloud data obtained by the first point cloud sensor of the first moving body sensing the object from a first position, the type of the first moving body, and first object segment information indicating the area and type of the object in the first point cloud data in a measurement information table, and that associates second point cloud data obtained by the second point cloud sensor of the second moving body sensing the object from a second position different from the first position, the type of the second moving body, and second object segment information indicating the area and type of the object in the second point cloud data in the measurement information table; An AI learning unit that uses the information stored in the measurement information table to learn an AI (Artificial Intelligence) that performs segmentation on point cloud data obtained by a point cloud sensor provided on a mobile object that can move in the real space, the AI ​​learning unit uses the first moving body type and the first point cloud data stored in the measurement information table as input data, and the first object segment information as correct answer data to learn the AI; and uses the second moving body type and the second point cloud data stored in the measurement information table as input data, and the second object segment information as correct answer data to learn the AI. Data processing device.

2. The first moving body and the second moving body have different moving speeds.

2. A data processing apparatus according to claim 1.

3. The type of the first moving object or the second moving object is a car, a motorcycle, a bicycle, a person, an aircraft, or a drone.

2. A data processing apparatus according to claim 1.

4. The first point cloud sensor and the second point cloud sensor have different performances.

2. A data processing apparatus according to claim 1.

5. The information acquisition unit the first point cloud data is associated with the type of the first moving body and the performance of the first point cloud sensor, and the second point cloud data is associated with the type of the second moving body and the performance of the second point cloud sensor, and the data are stored in the measurement information table; 5. A data processing apparatus according to claim 4.

6. The virtual space management unit further manages a third point cloud sensor that is not moved and is arranged in the virtual space; The information acquisition unit The third point cloud sensor senses the object to obtain third point cloud data; storing the third point cloud data in the measurement information table; 2. A data processing apparatus according to claim 1.

7. managing a virtual space simulating a real space, an object placed in the virtual space, a first moving body having a first point cloud sensor capable of moving in the virtual space, and a second moving body of a different type from the first moving body having a second point cloud sensor capable of moving in the virtual space; first point cloud data obtained by the first point cloud sensor of the first moving body sensing the object from a first position, the type of the first moving body, and first object segment information indicating the area and type of the object in the first point cloud data are associated with each other and stored in a measurement information table; second point cloud data obtained by the second point cloud sensor of the second moving body sensing the object from a second position different from the first position, the type of the second moving body, and second object segment information indicating the area and type of the object in the second point cloud data are associated with each other and stored in the measurement information table; Using the information stored in the measurement information table, learning is performed on an AI (Artificial Intelligence) that performs segmentation on point cloud data obtained by a point cloud sensor provided on a mobile object that can move in the real space; In the learning of the AI, the first moving body type and the first point cloud data stored in the measurement information table are used as input data, and the first object segment information is used as correct answer data to learn the AI; the second moving body type and the second point cloud data stored in the measurement information table are used as input data, and the second object segment information is used as correct answer data to learn the AI. Data processing methods.

Images