Data processing device and data processing method

Abstract

Efficiently collect large amounts of point cloud data by sensing objects from various positions. [Solution] The data processing device includes a virtual space management unit that manages a virtual space that simulates real space, an object placed in the virtual space, a first mobile body having a first point cloud sensor that can move in the virtual space, and a second mobile body having a second point cloud sensor that can move in the virtual space, and an information acquisition unit that stores in a measurement information table first point cloud data obtained by the first point cloud sensor of the first mobile body sensing the object from a first position, and second point cloud data obtained by the second point cloud sensor of the second mobile body sensing the object from a second position different from the first position.

Description

[Technical Field]

[0001] The present disclosure relates to a data processing device and a data processing method. [Background technology]

[0002] A large amount of training data is required to train artificial intelligence (AI) using deep learning etc. In the case of AI that recognizes point cloud data, a large amount of data with correct labels attached to each region (segment) of the point cloud data must be prepared, but doing this manually takes time and effort.

[0003] Therefore, it has been considered to use a virtual space that mimics real space to acquire point cloud data and automatically assign correct labels to it. Since virtual spaces are created by combining known models, the identity of each component is known. Therefore, if point cloud data is acquired in a virtual space, it is possible to automatically assign correct labels indicating the identity of each region, making it possible to create large amounts of training data without spending a lot of time and effort. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2023-165453 Summary of the Invention [Problem to be solved by the invention]

[0005] Sensors (e.g., LiDAR (Light Detection and Ranging)) that acquire point cloud data in real space are equipped and used on various types of moving objects (e.g., automobiles, drones, etc.), and the characteristics of the point cloud data acquired vary depending on the type of moving object. Therefore, for example, point cloud data collected by a smartphone held by a person in a virtual space, as disclosed in Patent Document 1, is insufficient for learning artificial intelligence.

[0006] An object of the present disclosure is to provide a technology that can efficiently collect a large amount of point cloud data obtained by sensing an object from various positions. [Means for solving the problem]

[0007] One aspect of the present disclosure provides a data processing device comprising: a virtual space management unit that manages a virtual space that simulates real space, an object placed in the virtual space, a first mobile body having a first point cloud sensor that can move in the virtual space, and a second mobile body having a second point cloud sensor that can move in the virtual space; and an information acquisition unit that stores, in a measurement information table, first point cloud data obtained by the first point cloud sensor of the first mobile body sensing the object from a first position, and second point cloud data obtained by the second point cloud sensor of the second mobile body sensing the object from a second position different from the first position.

[0008] One aspect of the present disclosure provides a data processing method that manages a virtual space that simulates real space, an object placed in the virtual space, a first mobile body having a first point cloud sensor that can move in the virtual space, and a second mobile body having a second point cloud sensor that can move in the virtual space, and stores first point cloud data obtained by the first point cloud sensor of the first mobile body sensing the object from a first position and second point cloud data obtained by the second point cloud sensor of the second mobile body sensing the object from a second position different from the first position in a measurement information table.

[0009] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0010] According to the present disclosure, it is possible to efficiently collect a large amount of point cloud data obtained by sensing an object from various positions. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example of the configuration of a data processing device according to an embodiment of the present invention; [Figure 2] FIG. 1 is a schematic diagram illustrating an example of a virtual space according to an embodiment of the present invention. [Figure 3] FIG. 10 is a diagram showing an example of a measurement information table according to the present embodiment. [Figure 4] 10 is a flowchart illustrating an example of a method for collecting measurement information in a virtual space according to the present embodiment. [Figure 5] 1 is a flowchart showing an example of a learning method for a segmentation AI according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure will be described in detail with appropriate reference to the drawings. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure and are not intended to limit the subject matter described in the claims. Furthermore, the function of one configuration shown in the present embodiment may be realized by two or more physical configurations, or the functions of two or more configurations may be realized by, for example, one physical configuration.

[0013] (Embodiment 1) <System configuration> FIG. 1 is a block diagram showing an example of the configuration of a data processing device 100 according to this embodiment.

[0014] The data processing device 100 includes a processor 101, a memory 102, a storage 103, a communication unit 104, an input unit 105, and a display unit 106.

[0015] The processor 101 reads and processes programs and data from the memory 102 or the storage 103, thereby realizing the functions of the data processing device 100 according to this embodiment. Details of the functions will be described later. The processor 101 may be interpreted as a CPU (Central Processing Unit), a control device, a controller, or the like. The processor 101 may also include a GPU (Graphics Processing Unit) and / or an NPU (Neural network Processing Unit).

[0016] The memory 102 is configured by a volatile storage medium and / or a non-volatile storage medium, and stores programs and data. The memory 102 may be interpreted as a RAM (Random Access Memory).

[0017] The storage 103 is configured by a non-volatile storage medium (for example, a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc.), and stores programs and data.

[0018] The communication unit 104 transmits and receives data to and from other devices (for example, the terminal 60) through the communication network 50. Examples of the communication network 50 include a wired LAN (Local Area Network), a wireless LAN, the Internet, and a mobile communication network.

[0019] The input unit 105 is a device for a user to input instructions, and is, for example, a keyboard, a mouse, a touchpad, a touch panel, a microphone, or the like.

[0020] The display unit 106 is a device for displaying images or information, and is, for example, a liquid crystal display, an organic EL display, or the like.

[0021] The data processing device 100 has, as its functions, a virtual space management unit 111, an information acquisition unit 112, an AI learning unit 113, a segmentation AI 120, and a measurement information table 200. Next, the virtual space management unit 111 will be described with reference to FIG.

[0022] Although the present embodiment describes an example in which the user directly operates the data processing device 100, the user may also operate the data processing device 100 via a terminal 60 connected to the communication network 50. Examples of the terminal 60 include a PC, a tablet terminal, a smartphone, etc.

[0023] FIG. 2 is a schematic diagram showing an example of a virtual space 10 according to the present embodiment.

[0024] The virtual space management unit 111 manages and controls a virtual space 10 that mimics real space. As illustrated in FIG. 2 , the virtual space management unit 111 generates a virtual space 10 in which virtual objects (3D objects) that mimic objects in real space are arranged. For example, the virtual space 10 includes stationary objects 11 that do not move, such as buildings, roads, guardrails, sidewalks, bridges, traffic lights, and utility poles, and mobile objects 12 that can move, such as automobiles 13, drones 14, bicycles (not shown), smartphones 16 carried by people 15, flying objects (not shown), and motorcycles (not shown). The virtual space management unit 111 also controls the behavior of objects in the virtual space 10 that mimic the physical laws of real space. In other words, the virtual space management unit 111 emulates real space in the virtual space 10.

[0025] The virtual space management unit 111 may display the state of the virtual space 10 as exemplified in FIG. 2 on the display unit 106.

[0026] The virtual space management unit 111 also controls the movement of the moving object 12 in the virtual space 10. The movement of the moving object 12 may be performed manually by the user via the input unit 105, or may be performed automatically by the virtual space management unit 111 based on a route or rules set in advance.

[0027] The mobile body 12 is provided with a point cloud sensor 20 that can sense objects in the virtual space 10 and acquire three-dimensional point cloud data. The point cloud sensor 20 may be a sensor that simulates a LiDAR or the like in the real space. In other words, the point cloud sensor 20 irradiates a laser in the virtual space 10 and generates point cloud data based on the light reflected from the laser by the object.

[0028] Furthermore, the point cloud sensor 20 provided on the moving body 12 in the virtual space 10 may mimic the position and performance of a point cloud sensor provided on an actual moving body in real space. For example, if the moving body 12 is an automobile, the point cloud sensor 20 provided on the automobile 13 in the virtual space 10 may mimic the position and performance of a LiDAR provided on an actual automobile. This makes it possible to acquire point cloud data in the virtual space 10 that is closer to that of the real space.

[0029] 2, a point cloud sensor 20 is provided on each of an automobile 13, a drone 14, and a smartphone 16 carried by a person 15 in a virtual space 10. The point cloud sensor 20 may be provided on a stationary body 11, not limited to a moving body 12. For example, the point cloud sensor 20 may be provided on a utility pole, a wall, a traffic light, or the like in the virtual space 10.

[0030] The virtual space management unit 111 manages the position, shape, etc. of each stationary body 11 in the virtual space 10. The virtual space management unit 111 also manages a moving body ID for uniquely identifying each moving body 12 in the virtual space 10, the shape of each moving body 12, the type of each moving body 12, and the performance of the point cloud sensor 20 provided in each moving body 12. The virtual space management unit 111 also manages the position, posture, speed, and movement direction of each moving body 12 in the virtual space 10 at each time. The virtual space management unit 111 also manages point cloud data obtained by sensing by the point cloud sensor 20 of each moving body 12 in the virtual space 10 at each time. In other words, the virtual space management unit 111 manages all information related to the virtual space 10.

[0031] Returning to the explanation of Figure 1.

[0032] The information acquisition unit 112 cooperates with the virtual space management unit 111 to acquire information on each stationary object 11 in the virtual space 10. The information acquisition unit 112 also cooperates with the virtual space management unit 111 to acquire information on each moving object 12 in the virtual space 10. The information acquisition unit 112 also cooperates with the virtual space management unit 111 to acquire point cloud data obtained by sensing using the point cloud sensor 20 of each stationary object 11 and / or each moving object 12 in the virtual space 10. The information acquisition unit 112 also identifies the type of object 30 included in the acquired point cloud data and the area of ​​the object 30 in the point cloud data. The information acquisition unit 112 stores this information in the measurement information table 200. Details of the processing performed by the information acquisition unit 112 will be described later (see FIG. 4).

[0033] The measurement information table 200 stores information etc. acquired by the information acquisition unit 112 from the virtual space 10. Details of the measurement information table 200 will be described later (see FIG. 3).

[0034] The AI ​​learning unit 113 uses information stored in the measurement information table 200 to train the segmentation AI 120. The segmentation AI 120 is an AI for performing semantic segmentation on point cloud data. For example, when point cloud data is input, the segmentation AI 120 outputs the type (semantic) of an object included in the point cloud data and the area (segment) of the object in the point cloud data. Details of the AI ​​learning unit 113 and the segmentation AI 120 will be described later (see FIG. 5).

[0035] <Measurement information table> FIG. 3 is a diagram showing an example of the measurement information table 200 according to this embodiment.

[0036] 3, the measurement information table 200 has, as items, a time 201, a moving object ID 202, a moving object type 203, a moving object position 204, a moving object attitude 205, a moving object speed 206, a moving object movement direction 207, a point cloud sensor performance 208, point cloud data 209, and object segment information 210. However, the measurement information table 200 may have a configuration that includes only some of these items.

[0037] Time 201 indicates the time in virtual space 10. Note that instead of the time, the elapsed time from a predetermined time in virtual space 10 may be used.

[0038] The moving body ID 202 is an ID for uniquely identifying the moving body 12 in the virtual space 10.

[0039] The moving object type 203 indicates the type of the moving object 12 of the moving object ID 202. Examples of the moving object type 203 include an automobile, a drone, a bicycle, a smartphone carried by a person, a flying object, and a motorcycle.

[0040] The moving object position 204 indicates the position of the moving object 12 with the moving object ID 202 at the time 201. The moving object position 204 may be indicated by three-dimensional coordinates (x, y, z) in the virtual space 10.

[0041] The moving body attitude 205 indicates the attitude of the moving body 12 of the moving body ID 202 at the time 201. The moving body attitude 205 may be indicated by a roll angle (θ), a pitch angle (φ), and a yaw angle (ψ) relative to the three-dimensional coordinates of the virtual space 10.

[0042] The moving object speed 206 indicates the moving speed of the moving object 12 with the moving object ID 202 at the time 201 .

[0043] The moving object moving direction 207 indicates the moving direction of the moving object 12 with the moving object ID 202 at the time 201. The moving object moving direction 207 may be indicated by a three-dimensional vector (a, b, c) in the virtual space 10.

[0044] The point cloud sensor performance 208 indicates the performance of the point cloud sensor 20 provided to the mobile body 12 having the mobile body ID 202. The point cloud sensor performance 208 is, for example, the sensing method of the point cloud sensor 20, the sensing cycle of the point cloud sensor 20, the distance and range that the point cloud sensor 20 can sense, etc. Information on the performance of the point cloud sensor 20 provided to each mobile body 12 may be stored in advance by the virtual space management unit 111.

[0045] The point cloud data 209 indicates point cloud data measured at time 201 by the point cloud sensor 20 provided on the mobile object 12 with the mobile object ID 202 .

[0046] The object segment information 210 is information indicating a segment of the object 30 included in the point cloud data 209. Here, the object 30 is one of the objects existing in the virtual space 10. In the present embodiment, one object of interest among the objects existing in the virtual space 10 is referred to as the object 30. In other words, the object 30 may be read as an object. The object 30 may be either a stationary object 11 or a moving object 12. The object 30 may be selected by the user from among the objects existing in the virtual space 10. The object segment information 210 may include information indicating the type (semantic) of an object included in the point cloud data 209 and information indicating the area (segment) of the object in the point cloud data 209.

[0047] 3 indicates that an automobile 13, which is a moving object 12 with a moving object ID of "1" existing in a virtual space 10, is located at (x1, y1, z1) at time "t1," has an attitude of (θ1, φ1, ψ1), is moving in a direction of (a1, b1, c1) at a speed of "v1," is equipped with a point cloud sensor 20 with "performance 1," and has measured "first point cloud data" by the point cloud sensor 20. Furthermore, the object segment information in the first line indicates that the first point cloud data includes an object A (for example, the object 30 in FIG. 2 is a "bridge"), and indicates the area (segment) of the object A in the first point cloud data.

[0048] For example, the second line in FIG. 3 indicates that a drone 14, which is a moving object 12 with moving object ID "2" existing in virtual space 10, is located at (x2, y2, z2) at time "t1," has an attitude of (θ2, φ2, ψ2), is moving in the direction of (a2, b2, c2) at a speed of "v2," is equipped with a point cloud sensor 20 with "performance 2," and that second point cloud data was measured by the point cloud sensor 20. Furthermore, the object segment information in the second line indicates that the second point cloud data includes an object A (for example, the object 30 in FIG. 2 is a "bridge"), and indicates the area (segment) of object A in the second point cloud data.

[0049] Note that the measurement information table 200 may store information acquired by the point cloud sensor 20 of the stationary body 11, not limited to the mobile body 12. In this case, the ID of the stationary body 11 may be stored in the mobile body ID 202, the type of the stationary body 11 in the mobile body type 203, the position of the stationary body 11 in the mobile body position 204, and the performance of the point cloud sensor 20 of the stationary body 11 in the point cloud sensor performance 208. In addition, point cloud data acquired by the point cloud sensor 20 of the stationary body 11 may be stored in the point cloud data 209, and segment information indicating the type and area of ​​the object in the point cloud data may be stored in the object segment information 210. Furthermore, zero or NULL may be stored in the mobile body attitude 205, the mobile body speed 206, and the mobile body movement direction.

[0050] <Collecting information in virtual space> 4 is a flowchart showing an example of a method for collecting measurement information in virtual space 10 according to this embodiment. Next, a method (processing) for data processing device 100 to collect measurement information from virtual space 10 will be described with reference to FIG.

[0051] The virtual space management unit 111 moves each moving object 12 in the virtual space 10 (S101). The movement of the moving object 12 may be performed automatically based on a predetermined route or rule, or may be performed manually by the user.

[0052] The information acquisition unit 112, in cooperation with the virtual space management unit 111, identifies multiple moving bodies 12 that are sensing the target object 30 at the same time in the virtual space 10 (S102). The target object 30 may be designated in advance by the user, or may be automatically designated by the information acquisition unit 112 from among objects existing in the virtual space 10 based on predetermined conditions. Furthermore, one target object 30 may be designated, or multiple targets 30 may be designated.

[0053] The information acquisition unit 112, in cooperation with the virtual space management unit 111, acquires the sensing time, moving body ID, moving body type, moving body position, moving body posture, moving body speed, moving body direction, point cloud sensor performance, and point cloud data for each moving body 12 that is sensing the target object 30 identified in step S102 (S103).

[0054] The information acquisition unit 112, in cooperation with the virtual space management unit 111, identifies segment information of the object 30 included in the point cloud data obtained in step S103 (S104). Since the relative positions and postures of the object 30 and the moving body 12 are all known in the virtual space 10, the information acquisition unit 112 can identify which areas of the point cloud data are segments of the object 30 and the type of object 30 based on this information.

[0055] The information acquisition unit 112 stores the information and point cloud data of each moving body 12 acquired in step S103 and the segment information of the target object 30 identified in step S104 in each item (201 to 210) of the measurement information table 200 (S105). Then, the process returns to step S101.

[0056] 3, the measurement information table 200 stores point cloud data 209 obtained by sensing the object 30 from various positions and directions by various moving bodies 12, and segment information of the object 30 included in the point cloud data 209. Furthermore, the measurement information table 200 stores the position, attitude, speed, etc. of the moving body 12 at the time of sensing.

[0057] For example, in the case of the virtual space 10 shown in Figure 2, at the same time t1, the first point cloud data obtained by the point cloud sensor 20 of the automobile 13 sensing the object 30, a "bridge", from the road, the segment information of the object "bridge" in the first point cloud data, the second point cloud data obtained by the drone 14 sensing the same object 30, a "bridge", from the sky, and the segment information of the object "bridge" in the second point cloud data are stored in the measurement information table 200.

[0058] As described above, according to this embodiment, it is possible to simultaneously collect point cloud data of the target object 30 sensed by various moving bodies 12 in the virtual space 10. Therefore, it is possible to efficiently collect a large amount of point cloud data to be used for training the segmentation AI 120, and also to efficiently collect point cloud data specific to various moving bodies 12.

[0059] <Segmentation AI learning> 5 is a flowchart showing an example of a training method for segmentation AI 120 according to this embodiment. Next, a method (processing) for data processing device 100 to train segmentation AI 120 using information in measurement information table 200 will be described with reference to FIG.

[0060] The AI ​​learning unit 113 selects a learning object 30 (S201). The object 30 may be designated by the user, or the AI ​​learning unit 113 may designate an object 30 based on predetermined conditions from among objects present in the virtual space 10. Furthermore, one or more objects 30 may be designated.

[0061] The AI ​​learning unit 113 extracts a row having object segment information corresponding to the object 30 selected in step S201 from the measurement information table 200 (S202). For example, if a specific "bridge" is specified as the object 30, the AI ​​learning unit 113 extracts a row (measurement information) having object segment information 210 of the "bridge" from the measurement information table 200. The extracted row (measurement information) may be interpreted as training data.

[0062] The AI ​​learning unit 113 selects one unselected row from the rows extracted in step S202 (S203).

[0063] The AI ​​learning unit 113 uses the moving object type 203 and point cloud data 209 of the row selected in step S203 as input data and the object segment information 210 of the selected row as correct answer data to train the segmentation AI 120 (S204).

[0064] The AI ​​learning unit 113 determines whether all of the rows extracted in step S202 have been selected (S205). If there are any unselected rows remaining among the rows extracted in step S202 (S205: NO), the AI ​​learning unit 113 returns the process to step S203. If all of the rows extracted in step S202 have been selected (S205: YES), the AI ​​learning unit 113 terminates this process. Note that the AI ​​learning unit 113 may also perform the process shown in FIG. 5 for other objects.

[0065] This makes it possible to generate a segmentation AI120 that can accurately output the segmentation of object regions for the point cloud data and the type (label) of the object by inputting point cloud data obtained by sensing with a point cloud sensor equipped on a moving object in real space and the type of the moving object.

[0066] If the type of moving body is different, the obtained point cloud data may be different even when the same object is sensed. For example, the area (segment) of the bridge in the point cloud data obtained by sensing a bridge (an example of a target object 30) from a road by an automobile 13 (an example of a moving body 12) will be different from the area (segment) of the bridge in the point cloud data obtained by sensing the same bridge from the sky by a drone 14.

[0067] In contrast, the segmentation AI 120 according to this embodiment has been trained using point cloud data obtained from different types of moving objects 12 in the virtual space 10. Therefore, the segmentation AI 120 can accurately perform object segmentation even for point cloud data obtained by different types of moving objects in the real space.

[0068] Furthermore, even when the same object is sensed, the characteristics (e.g., noise, etc.) of the obtained point cloud data may differ if the attitude, speed, etc. of the moving object differs. For example, point cloud data obtained by sensing a bridge while the drone 14 is flying at an attitude (θ2a, φ2a, ψ2a) and a speed v2a may have different characteristics from point cloud data obtained by sensing a bridge while the same drone 14 is flying at an attitude (θ2b, φ2b, ψ2b) and a speed v2b.

[0069] Therefore, in step S204, the AI ​​learning unit 113 may use the moving body type 203 and point cloud data 209 of the row selected in step S203 as input data, as well as the moving body posture 205 and moving body speed 206, and may use the object segment information 210 of the selected row as correct answer data to train the segmentation AI 120.

[0070] This makes it possible to generate a segmentation AI120 that can accurately output the segmentation of object regions for the point cloud data and the type (label) of the object by inputting point cloud data obtained by sensing a moving object in real space, as well as the type, posture, and speed of the moving object.

[0071] Furthermore, even when the same object is sensed, the characteristics (e.g., resolution, noise, etc.) of the point cloud data obtained may differ depending on the performance of the point cloud sensor 20. For example, the characteristics of point cloud data obtained by sensing a bridge with a point cloud sensor 20 having a first performance may differ from those of point cloud data obtained by sensing a bridge with a point cloud sensor 20 having a second performance.

[0072] Therefore, in step S204, the AI ​​learning unit 113 may use the point cloud sensor performance 208 as input data in addition to the moving object type 203 and point cloud data 209 of the row selected in step S203, and may train the segmentation AI 120 using the object segment information 210 of the selected row as correct answer data.

[0073] This makes it possible to generate a segmentation AI120 that can accurately output the segmentation of object regions for the point cloud data and the type (label) of the object by inputting point cloud data obtained by sensing a moving object in real space and the performance of the point cloud sensor of the moving object.

[0074] That is, the AI ​​learning unit 113 may use at least one of the moving body type 203, moving body position 204, moving body posture 205, moving body speed 206, moving body movement direction 207, and point cloud sensor performance 208 in the measurement information table 200 and the point cloud data 209 as input data, and the object segment information 210 as correct answer data to train the segmentation AI 120.

[0075] (Summary of this embodiment) The above description of the present embodiment discloses the following techniques.

[0076] <Technology 1> A data processing device (100) according to this embodiment includes a virtual space management unit (111) that manages a virtual space (10) that mimics a real space, an object (30) placed in the virtual space, a first mobile body (12) having a first point cloud sensor (20) that can move in the virtual space, and a second mobile body (12) having a second point cloud sensor (20) that can move in the virtual space, and an information acquisition unit (112) that stores, in a measurement information table (200), first point cloud data obtained by the first point cloud sensor of the first mobile body sensing the object from a first position and second point cloud data obtained by the second point cloud sensor of the second mobile body sensing the object from a second position different from the first position. This allows multiple point cloud data obtained by sensing the same object from different positions in virtual space to be stored in the measurement information table, making it possible to efficiently collect a large amount of point cloud data obtained by sensing the object from various positions.

[0077] <Technology 2> In the data processing device described in Technique 1, the first moving body and the second moving body are of different types. This allows multiple point cloud data obtained by sensing from different types of moving objects to be stored in the measurement information table, making it possible to efficiently collect a large amount of point cloud data obtained by sensing objects from various types of moving objects.

[0078] <Technology 3> In the data processing device according to Technique 1 or 2, the first moving body and the second moving body have different moving speeds. This allows multiple point cloud data obtained by sensing from moving objects with different movement speeds to be stored in the measurement information table, which means that a large amount of point cloud data obtained by sensing objects from moving objects with various movement speeds can be efficiently collected.

[0079] <Technology 4> In the data processing device according to Technique 2 or 3, the type of the first moving object or the second moving object is a car, a motorbike, a bicycle, a person, an aircraft, or a drone. This allows point cloud data acquired from cars, motorcycles, bicycles, people, flying objects, or drones in virtual space to be stored in the measurement information table.

[0080] <Technology 5> 5. The data processing device according to claim 1, wherein the first point cloud sensor and the second point cloud sensor have different performances. This allows multiple point cloud data obtained by sensing from point cloud sensors with different performance to be stored in the measurement information table, which means that a large amount of point cloud data obtained by sensing an object from point cloud sensors with various performance capabilities can be efficiently collected.

[0081] <Technology 6> In the data processing device described in any one of Techniques 2 to 5, the information acquisition unit associates the first point cloud data with the type of the first moving body, and associates the second point cloud data with the type of the second moving body, and stores them in the measurement information table. This allows the measurement information table to store the type of moving body from which point cloud data has been obtained in association with the point cloud data.

[0082] <Technology 7> In the data processing device described in Technology 5, the information acquisition unit associates the first point cloud data with the type of the first moving body and the performance of the first point cloud sensor, and associates the second point cloud data with the type of the second moving body and the performance of the second point cloud sensor, and stores them in the measurement information table. This allows the measurement information table to store the type of moving body from which the point cloud data was obtained and the performance of the point cloud sensor in association with the point cloud data.

[0083] <Technology 8> In the data processing device described in any one of Techniques 1 to 7, a third point cloud sensor that is not moving and is placed in the virtual space is further managed, and the information acquisition unit acquires third point cloud data obtained by the third point cloud sensor sensing the object, and stores the third point cloud data in the measurement information table. This allows a plurality of third point cloud data obtained by sensing the object with the stationary point cloud sensor to be stored in the measurement information table.

[0084] <Technology 9> In the data processing device described in any one of Techniques 1 to 8, the information acquisition unit associates the first point cloud data with object segment information indicating the area and type of the object in the first point cloud data, and associates the second point cloud data with object segment information indicating the area and type of the object in the second point cloud data, and stores the associated data in the measurement information table. This allows the measurement information table to store object segment information indicating the area and type of object in point cloud data obtained by various moving bodies in association with the point cloud data.

[0085] <Technology 10> The data processing device described in Technology 9 further includes an AI learning unit (113) that uses the information stored in the measurement information table to train an AI (e.g., segmentation AI 120) that performs segmentation on point cloud data obtained by a point cloud sensor equipped on a mobile body that can move in the real space. The measurement information table stores a large amount of point cloud data obtained by sensing various moving objects in virtual space and the corresponding object segment information, which allows the AI ​​learning unit to efficiently train the AI ​​to segment objects using point cloud data obtained by sensing various moving objects in real space.

[0086] <Technology 11> The data processing method manages a virtual space (10) that mimics real space, an object (30) placed in the virtual space, a first mobile body (12) having a first point cloud sensor (20) that can move in the virtual space, and a second mobile body (12) having a second point cloud sensor (20) that can move in the virtual space, and stores in a measurement information table (200) first point cloud data obtained by the first point cloud sensor of the first mobile body sensing the object from a first position, and second point cloud data obtained by the second point cloud sensor of the second mobile body sensing the object from a second position different from the first position. This allows multiple point cloud data obtained by sensing the same object from different positions in virtual space to be stored in the measurement information table, making it possible to efficiently collect a large amount of point cloud data obtained by sensing the object from various positions.

[0087] Although the embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also fall within the technical scope of the present disclosure. Furthermore, the components in the above-described embodiments may be combined in any manner without departing from the spirit of the invention. [Industrial Applicability]

[0088] The techniques disclosed herein are useful for collecting point cloud data with different characteristics, training artificial intelligence on point cloud data, and utilizing the artificial intelligence. [Explanation of symbols]

[0089] 10 Virtual Space 11 Stationary Objects 12 Mobile 13. Automobiles 14 Drone 15 people 16. Smartphones 20 Point Cloud Sensor 30 Objects 50 Communication Network 60 devices 100 Data processing device 101 processors 102 memory 103 Storage 104 Communications Department 105 Input section 106 Display section 111 Virtual Space Management Department 112 Information Acquisition Department 113 AI Learning Department 120 Segmentation AI 200 Measurement Information Table 201 Time 202 Mobile ID 203 Mobile Type 204 Mobile object position 205 Moving body posture 206 Moving object speed 207 Moving direction of moving object 208 Point Cloud Sensor Performance 209 Point Cloud Data 210 Object Segment Information

Claims

1. a virtual space management unit that manages a virtual space that mimics a real space, an object placed in the virtual space, a first moving body having a first point cloud sensor that can move in the virtual space, and a second moving body having a second point cloud sensor that can move in the virtual space; an information acquisition unit that stores, in a measurement information table, first point cloud data obtained by the first point cloud sensor of the first moving body sensing the object from a first position and second point cloud data obtained by the second point cloud sensor of the second moving body sensing the object from a second position different from the first position; Data processing device.

2. The first moving body and the second moving body are of different types.

2. The data processing device according to claim 1.

3. The first moving body and the second moving body have different moving speeds.

2. The data processing device according to claim 1.

4. The type of the first moving object or the second moving object is a car, a motorcycle, a bicycle, a person, an aircraft, or a drone.

3. The data processing device according to claim 2.

5. the first point cloud sensor and the second point cloud sensor have different performances; 3. The data processing device according to claim 2.

6. The information acquisition unit the first point cloud data is associated with the type of the first moving body, and the second point cloud data is associated with the type of the second moving body, and the data are stored in the measurement information table; 3. The data processing device according to claim 2.

7. The information acquisition unit the first point cloud data is associated with the type of the first moving body and the performance of the first point cloud sensor, and the second point cloud data is associated with the type of the second moving body and the performance of the second point cloud sensor, and the data are stored in the measurement information table; 6. A data processing device according to claim 5.

8. the virtual space management unit further manages a third point cloud sensor that is not moving and is arranged in the virtual space; The information acquisition unit acquiring third point cloud data obtained by sensing the object with the third point cloud sensor; storing the third point cloud data in the measurement information table; 2. The data processing device according to claim 1.

9. The information acquisition unit associates the first point cloud data with object segment information indicating the area and type of the object in the first point cloud data, associates the second point cloud data with object segment information indicating the area and type of the object in the second point cloud data, and stores the associated object segment information in the measurement information table; 9. A data processing device according to any one of claims 1 to 8.

10. An AI learning unit that uses the information stored in the measurement information table to learn an AI that performs segmentation on point cloud data obtained by a point cloud sensor provided on a mobile object that can move in the real space, 10. The data processing device according to claim 9.

11. managing a virtual space simulating a real space, an object placed in the virtual space, a first moving body having a first point cloud sensor capable of moving in the virtual space, and a second moving body having a second point cloud sensor capable of moving in the virtual space; storing, in a measurement information table, first point cloud data obtained by the first point cloud sensor of the first mobile body sensing the object from a first position, and second point cloud data obtained by the second point cloud sensor of the second mobile body sensing the object from a second position different from the first position; Data processing methods.

Images