Data processing system, data processing method, and program

Abstract

Provided is a data processing system comprising: a transmission unit that transmits first sensor data measured by a sensor at a first timing; a construction unit that constructs a first three-dimensional model on the basis of the transmitted first sensor data; a prediction unit that predicts, on the basis of the constructed first three-dimensional model, the state of a second three-dimensional model at a second timing which is later than the first timing; and an identification unit that identifies, on the basis of a difference between the predicted state of the second three-dimensional model and second sensor data measured by the sensor at the second timing, a transmission region to be transmitted by the transmission unit of the second sensor data.

Description

Data processing system, data processing method, and program

[0001] The present disclosure relates to a data processing system, a data processing method, and a program.

[0002] Attention has been paid to a three-dimensional digital twin (also referred to as a 3D digital twin) that integrates data measured by a plurality of sensors and maps information of the real three-dimensional space to a virtual three-dimensional space. For example, research on safely controlling the autonomous driving of vehicles is underway by using a three-dimensional digital twin constructed from data measured by sensors on the road side (also referred to as the roadside) and sensors on the vehicle side. Patent Document 1 is known as a technology related to a road-vehicle cooperative system using roadside sensors and vehicle-side sensors. In addition, Non-Patent Document 1 is known as a technology related to the prediction of a three-dimensional digital twin.

[0003] Japanese Unexamined Patent Application Publication No. 2022-010903

[0004] Sriram N N, Buyu Liu, Francesco Pittaluga, Manmohan Chandraker, "SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction", ECCV 2020, pp.463-479, European Conference on Computer Vision, November 2020, Internet <URL: https: / / www.ecva.net / papers / eccv_2020 / papers_ECCV / papers / 123720460.pdf>

[0005] In related technologies such as Patent Document 1, when sending an image from a sensor to a server, it is possible to suppress the communication volume of the sensor by sending the target area at a high resolution and sending areas other than the target area at a low resolution. Since more sensor data may be required to construct a three-dimensional model such as a three-dimensional digital twin, it is desirable to further suppress the communication volume.

[0006] In light of these challenges, one of the objectives of this disclosure is to provide a data processing system, data processing method, and program that can reduce the amount of data transmitted.

[0007] A data processing system according to one aspect of the present disclosure includes: a transmission unit that transmits first sensor data measured by a sensor at a first timing; a construction unit that constructs a first three-dimensional model based on the transmitted first sensor data; a prediction unit that predicts the state of a second three-dimensional model at a second timing after the first timing based on the constructed first three-dimensional model; and a specification unit that identifies a transmission region of the second sensor data to be transmitted by the transmission unit based on the difference between the predicted state of the second three-dimensional model and the second sensor data measured by the sensor at the second timing.

[0008] A data processing method according to one aspect of the present disclosure includes: transmitting first sensor data measured by a sensor at a first timing; constructing a first three-dimensional model based on the transmitted first sensor data; predicting the state of a second three-dimensional model at a second timing after the first timing based on the constructed first three-dimensional model; and identifying a transmission region of the second sensor data to be transmitted based on the difference between the predicted state of the second three-dimensional model and the second sensor data measured by the sensor at the second timing.

[0009] A program according to one aspect of the present disclosure is a program for causing a computer to perform the following processes: transmitting first sensor data measured by a sensor at a first timing; constructing a first three-dimensional model based on the transmitted first sensor data; predicting the state of a second three-dimensional model at a second timing after the first timing based on the constructed first three-dimensional model; and identifying a transmission region to be transmitted from the second sensor data based on the difference between the predicted state of the second three-dimensional model and the second sensor data measured by the sensor at the second timing.

[0010] According to this disclosure, the amount of data transmitted can be reduced.

[0011] This is a configuration diagram showing an example configuration of a data processing system according to several embodiments. This is a configuration diagram showing an example configuration of a data processing device according to several embodiments. This is a flowchart showing an example of a data processing method according to several embodiments. This is a configuration diagram showing an example configuration of a vehicle-infrastructure cooperation system according to several embodiments. This is a configuration diagram showing an example configuration of a terminal and a vehicle-infrastructure cooperation server according to several embodiments. This is a diagram for explaining an example of operation of a vehicle-infrastructure cooperation system according to several embodiments. This is a flowchart showing an example of operation of important area extraction processing according to several embodiments. This is a configuration diagram showing an example configuration of a terminal and a vehicle-infrastructure cooperation server according to several embodiments. This is a configuration diagram showing an example configuration of a terminal and a vehicle-infrastructure cooperation server according to several embodiments. This is a configuration diagram showing an example configuration of a terminal and a vehicle-infrastructure cooperation server according to several embodiments. This is a configuration diagram showing an example configuration of a terminal and a vehicle-infrastructure cooperation server according to several embodiments. This is a configuration diagram showing an example configuration of computer hardware according to several embodiments.

[0012] The embodiments will be described below with reference to the drawings. In each drawing, the same elements are denoted by the same reference numerals, and redundant explanations will be omitted where necessary.

[0013] (Embodiment 1) First, Embodiment 1 will be described. In this embodiment, the outlines of several embodiments will be described.

[0014] Figure 1 shows an example configuration of a data processing system 10 according to several embodiments. For example, the data processing system 10 is a road-vehicle-to-road cooperative system that processes data from roadside sensors and data from vehicle-side sensors.

[0015] In the example shown in Figure 1, the data processing system 10 includes a transmission unit 11, a construction unit 12, a prediction unit 13, and a identification unit 14.

[0016] The transmitting unit 11 transmits sensor data measured by the sensor. The sensor may be a camera, a LiDAR (Light Detection and Ranging), or any other type of sensor. If the sensor is a camera, the sensor data is video data; if the sensor is a LiDAR, the sensor data is point cloud data. For example, the transmitting unit 11 may transmit the first sensor data measured by the sensor at a first timing to a server including the construction unit 12. Alternatively, the transmitting unit 11 may transmit a transmission area of ​​the sensor data that has been identified by the identification unit 14.

[0017] The construction unit 12 constructs a three-dimensional model based on the sensor data transmitted by the transmission unit 11. A three-dimensional model refers to a three-dimensional object or event constructed on a computer. The three-dimensional model may also be described as a three-dimensional digital twin. For example, the construction unit 12 constructs a first three-dimensional model at a first timing based on a plurality of first sensor data at a first timing.

[0018] The prediction unit 13 predicts the future state of the 3D model based on the 3D model constructed by the construction unit 12. For example, the prediction unit 13 predicts the state of the second 3D model at a second timing after the first timing, based on the constructed first 3D model. Here, predicting the state of the second 3D model may refer to, for example, predicting the temporal changes of an object or event constructed based on sensor data.

[0019] The identification unit 14 identifies a transmission region from the sensor data to be transmitted by the transmission unit 11, based on the difference between the state of the second 3D model at the second timing predicted by the prediction unit 13 and the second sensor data measured by the sensor at the second timing. The identification unit 14 may also identify a region as the transmission region where the difference between the predicted state of the second 3D model and the measured second sensor data is greater than a predetermined threshold. The transmission region is the region transmitted by the transmission unit 11 and is a part of the sensor data. For example, the transmission region may be a part of the image region in the image data, or a part of the point cloud region in the point cloud data.

[0020] The data processing system 10 may include a conversion unit that converts the predicted state of the second three-dimensional model into third sensor data measured from the sensor. Sensor data measured from the sensor is sensor viewpoint data measured from the sensor's viewpoint. For example, if the sensor is a camera, it is video data obtained by taking a picture from the camera's viewpoint, and if the sensor is a LiDAR, it is point cloud data obtained by scanning from the LiDAR's viewpoint. The identification unit 14 may find the difference between the converted third sensor data and the second sensor data measured by the sensor. For example, the identification unit 14 may find the difference between objects included in the converted third sensor data and objects included in the second sensor data measured by the sensor.

[0021] The data processing system 10 may be composed of any number of devices. Figure 2 shows an example configuration in which the functions of the data processing system 10 are arranged in two data processing devices 21 and 22. In the example in Figure 2, the data processing device 21 includes a transmission unit 11 and a specification unit 14, and the data processing device 22 includes a construction unit 12 and a prediction unit 13. For example, the data processing device 21 may be a sensor or a terminal connected to a sensor, and the data processing device 22 may be a cloud or edge server. The example in Figure 2 is not limited to this case, and the transmission unit 11, construction unit 12, prediction unit 13, and specification unit 14 may be distributed among multiple devices.

[0022] Figure 3 shows examples of data processing methods according to several embodiments. For example, the data processing methods according to some embodiments may be performed by the data processing system 10 in Figure 1 and the data processing devices 21 and 22 in Figure 2.

[0023] In the example shown in Figure 3, the transmission unit 11 transmits the first sensor data measured by the sensor to the server at the first timing (S11).

[0024] Next, the construction unit 12 constructs a first three-dimensional model at a first timing based on the transmitted first sensor data (S12). Next, the prediction unit 13 predicts the state of the second three-dimensional model at a second timing after the first timing, based on the constructed first three-dimensional model (S13).

[0025] Next, the identification unit 14 identifies a transmission region from the second sensor data to be transmitted by the transmission unit 11, based on the difference between the predicted state of the second three-dimensional model and the second sensor data measured by the sensor at the second timing (S14). For example, the identification unit 14 identifies a region as the transmission region where the difference between the state of the second three-dimensional model and the second sensor data is greater than a predetermined threshold. Furthermore, the transmission unit 11 transmits the data of the identified transmission region to the server.

[0026] In this embodiment, the future state of a 3D model is predicted from a 3D model constructed from sensor data, and the transmission region to be transmitted next from the sensor is identified based on the difference between the predicted state of the 3D model and the sensor data actually measured by the sensor. As a result, for example, only data from regions with a large difference from the predicted state of the 3D model can be transmitted, thereby reducing the amount of data transmitted from the sensor to the server.

[0027] The following embodiments will describe specific examples of Embodiment 1.

[0028] (Embodiment 2) Next, Embodiment 2 will be described. In this embodiment, an example of reducing the amount of communication between a terminal and a vehicle-infrastructure cooperation server in a vehicle-infrastructure cooperation system will be described.

[0029] Figure 4 shows some configuration examples of the vehicle-infrastructure cooperation system 1 according to several embodiments. For example, the vehicle-infrastructure cooperation system 1 is a system that supports the safety of vehicle operation by combining and analyzing information collected from roadside sensors and information collected from vehicle sensors. The vehicle-infrastructure cooperation system 1 may be a remote monitoring system such as an ITS system (Intelligence Transport System) that monitors roads and vehicles, or a remote control system that controls vehicles according to the monitoring results. The vehicle may be an automobile, motorcycle, heavy machinery such as a forklift, a train, a robot, a drone, etc.

[0030] In the example shown in Figure 4, the vehicle-infrastructure cooperation system 1 includes multiple terminals 100, a cloud server 200a, a MEC 200b, and a base station 300. For example, either the cloud server 200a or the MEC 200b constitutes the vehicle-infrastructure cooperation server 200.

[0031] Terminal 100, MEC 200b, and base station 300 are located on the road side and vehicle side (also called the road-vehicle side), while the cloud server 200a is located on the cloud side. For example, the cloud server 200a is located in a data center or the like, at a location far from the road-vehicle side. For example, the road-vehicle side is the edge side relative to the cloud.

[0032] Terminal 100 and base station 300 are connected via network NW1 for communication. Network NW1 is a wireless network such as 4G, LTE (Long Term Evolution), local 5G / 5G, other generations of mobile communication, or wireless LAN. For example, network NW1 may be a DSRC (Dedicated Short Range Communication) network for ITS systems or a V2X (Vehicle to Everything) network that connects vehicles to everything. V2X may be LTE-V2X (Long Term Evolution-V2X), NR-V2X (New Radio V2X), C-V2X (Cellular V2X), etc. Note that network NW1 is not limited to a wireless network but may also be a wired network.

[0033] The base station 300 and the MEC 200b are connected in a way that enables communication using any communication method. It can also be said that the terminal 100 and the MEC 200b are connected in a way that enables communication via the base station 300. The base station 300 and the MEC 200b may be a single device. For example, the base station 300 may have the functions of the MEC 200b.

[0034] The base station 300 and the cloud server 200a are connected via network NW2, enabling communication. Network NW2 includes, for example, core networks such as 5GC (5th Generation Core network) and EPC (Evolved Packet Core), as well as the internet. Note that network NW2 is not limited to a wired network; it may also be a wireless network. It can also be said that the terminal 100 and the cloud server 200a are connected via the base station 300 and network NW2, enabling communication.

[0035] Terminal 100 is a terminal device connected to the network NW1. Terminal 100 is also a transmitting device that transmits sensor data measured by the roadside sensor 101. Terminal 100 and sensor 101 may be a single device. That is, terminal 100 may include sensor 101, and sensor 101 may include terminal 100. For example, terminal 100 may be a roadside unit (RSU) installed on the roadside or an on-board unit (OBU) mounted on a vehicle. For example, the roadside sensor 101 may include a 3D sensor such as LiDAR that generates point cloud data, and a camera that takes images (video). Sensor 101 is not limited to LiDAR or cameras, but may also include other sensors such as infrared cameras or millimeter-wave radar.

[0036] Terminal 100 compresses the sensor data from sensor 101 using a predetermined compression method and transmits the compressed data to MEC 200b or cloud server 200a. In other words, terminal 100 includes an encoder that encodes the sensor data using a predetermined encoding method. For example, terminal 100 extracts important regions of the sensor data and transmits the extracted important regions to reduce the amount of data transmitted (data rate).

[0037] Base station 300 is a base station device for network NW1 and also a relay device that relays communication between terminal 100 and MEC200b or cloud server 200a. For example, base station 300 may be a local 5G base station, a 5G gNB (next generation node B), an LTE eNB (evolved node B), a wireless LAN access point, or any other relay device.

[0038] The MEC (Multi-access Edge Computing) 200b is an edge server installed on the edge side of the system. The MEC 200b may be one or more physical computers, or a virtual computer built on any virtualization platform. The MEC 200b may process sensor data received from terminal 100 and send the processed data to cloud server 200a, or it may control terminal 100 as needed.

[0039] The cloud server 200a is a server located on the cloud side. The cloud server 200a may be one or more physical servers, or it may be a virtualized server built on any virtualization platform. The cloud server 200a may process sensor data received from the terminal 100 and control the terminal 100 as needed.

[0040] The vehicle-infrastructure cooperative server 200, which consists of a cloud server 200a or MEC 200b, monitors the situation around the vehicle by analyzing and recognizing sensor data, including point cloud data and video data from the vehicle-infrastructure side, and controls the vehicle as needed. The vehicle-infrastructure cooperative server 200 integrates point cloud data from multiple LiDARs and video data from multiple cameras to construct a 3D digital twin (3D integrated data). By inputting information about objects recognized on the 3D digital twin into a predictive AI engine, the vehicle-infrastructure cooperative server 200 predicts the movement of the vehicle and the situation of pedestrians, fallen objects, animals, etc., around the moving vehicle.

[0041] The road-vehicle cooperation server 200 may feedback the predicted result to the vehicle terminal 100. The road-vehicle cooperation server 200 may transmit the predicted information around the vehicle, or may transmit control information for controlling the running of the vehicle, etc. For example, it can provide information such as obstacles in areas that are blind spots and cannot be recognized on the vehicle side. Even when the vehicle cannot recognize the surrounding situation at night or in bad weather, it can safely control the automatic driving. It can surely grasp dangerous events that cannot be recognized by a single sensor in the 3D space of the 3D digital twin, and prevent serious accidents by providing information to the vehicle side.

[0042] FIG. 5 shows a configuration example of the terminal 100 and the road-vehicle cooperation server 200 according to some embodiments. Note that FIG. 5 is an example of the configuration of each device, and other configurations may be used as long as the operations according to some embodiments are possible. For example, some functions of the terminal 100 may be arranged in the road-vehicle cooperation server 200 or other devices, or some functions of the road-vehicle cooperation server 200 may be arranged in the terminal 100 or other devices.

[0043] In the example of FIG. 5, the terminal 100 includes an acquisition unit 110, an important area extraction unit 120, an encoder 130, a transmission unit 140, and a reception unit 150.

[0044] The acquisition unit 110 acquires the sensor data measured by the sensor 101. When the sensor 101 is a LiDAR, the sensor data is point cloud data. The point cloud data includes the coordinate information of the 3D space obtained by the reflected light from the object at each point in the measurement range measured by the LiDAR. The coordinate information indicates the depth or 3D position of the point in the 3D space. The coordinate information is not limited to the coordinates at each point, and may include the reflectivity of light at each point, etc. Also, when the sensor 101 is a camera, the sensor data is video data. The video data includes a plurality of images in time series, that is, frames.

[0045] The important area extraction unit 120 extracts important areas from the sensor data acquired from the sensor 101. The important area extraction unit 120 also acts as a identification unit that identifies important areas. The identified important areas are the transmission areas to be sent to the vehicle-infrastructure cooperation server 200. In other words, areas in the sensor data other than the important areas are non-transmission areas that are not sent to the vehicle-infrastructure cooperation server 200. Alternatively, information from areas other than the important areas may be transmitted with a smaller amount of data than that from the important areas. The important area extraction unit 120 extracts important areas based on the sensor data acquired from the sensor 101 and the prediction results (sensor viewpoint data described later) received from the vehicle-infrastructure cooperation server 200. The sensor data acquired from the sensor 101 and the prediction results received from the vehicle-infrastructure cooperation server 200 are compared, and areas where the difference is greater than a predetermined value are determined to be important areas.

[0046] The encoder 130 encodes the sensor data acquired from the sensor 101 using a predetermined encoding method. The encoder 130 is a compression unit that compresses the sensor data using a predetermined compression method. When the important region extraction unit 120 extracts an important region, the encoder 130 encodes the extracted important region from the sensor data. For example, if the sensor data is video data, it is encoded using a video compression method such as H.264 or H.265. If the sensor data is point cloud data, it is encoded using a point cloud compression method such as V-PCC or G-PCC.

[0047] The transmitting unit 140 transmits the encoded data encoded by the encoder 130, i.e., the compressed data, to the vehicle-infrastructure cooperation server 200. The transmitting unit 140 transmits the compressed data to the cloud server 200a or MEC 200b via the base station 300. The receiving unit 150 receives the prediction results from the vehicle-infrastructure cooperation server 200. The receiving unit 150 receives the prediction results from the cloud server 200a or MEC 200b via the base station 300. The transmitting unit 140 and the receiving unit 150 have an interface that can communicate with the base station 300, which is a wireless interface such as 4G, local 5G / 5G, LTE, or wireless LAN, but may also be a wireless or wired interface of any other communication method.

[0048] In the example of FIG. 5, the road-vehicle cooperation server 200 includes a receiving unit 210, a transmitting unit 220, a decoder 230, a 3D digital twin construction unit 240, a prediction unit 250, and a sensor viewpoint data generation unit 260.

[0049] The receiving unit 210 receives the encoded data transmitted from the terminal 100, that is, the compressed data obtained by compressing the sensor data. The receiving unit 210 receives the compressed data from the terminal 100 via the base station 300.

[0050] The transmitting unit 220 transmits the prediction results predicted by the prediction unit 250 and generated by the sensor viewpoint data generation unit 260 to the terminal 100. The receiving unit 210 transmits the prediction results to the terminal 100 via the base station 300. The receiving unit 210 and the transmitting unit 220 are interfaces capable of communicating with the base station 300, the Internet, or the core network. For example, they are wired interfaces for IP communication, but may also be wired or wireless interfaces of any other communication method.

[0051] The decoder 230 decodes the encoded data received from the terminal 100 using a predetermined decoding method. The decoder 230 is an expansion unit that expands the compressed data using a predetermined expansion method and restores the sensor data. The decoder 230 corresponds to the encoding method of the terminal 100. For example, in the case of video, it decodes using a video compression method such as H.264 or H.265, and in the case of point cloud data, it decodes using a point cloud compression method such as V-PCC or G-PCC.

[0052] The 3D digital twin construction unit 240 constructs a 3D digital twin based on the sensor data received from the terminal 100. The 3D digital twin construction unit 240 includes a data integration unit 241 and an object recognition unit 242.

[0053] The data integration unit 241 integrates multiple sensor data, such as point cloud data and video data, received from multiple terminals 100 to construct a three-dimensional digital twin (three-dimensional integrated data). For example, three-dimensional map information including the position and orientation of each sensor may be stored in the storage unit in advance, and the position and orientation of each sensor may be obtained from the three-dimensional map information, or the position and orientation of each sensor may be obtained from the sensor data from each sensor. Based on the acquired position and orientation of each sensor, the data integration unit 241 maps the point cloud and images of the sensor data into three-dimensional space. For example, a three-dimensional model of an object may be generated from the point cloud and images using a three-dimensional model generation engine that uses machine learning such as deep learning, and the generated three-dimensional model of the object may be mapped into three-dimensional space.

[0054] The object recognition unit 242 recognizes objects from the constructed 3D digital twin. The object recognition unit 242 extracts rectangular regions (bounding boxes, object regions) corresponding to objects in the 3D digital twin and recognizes the object type of the objects within the extracted rectangular regions. The object recognition unit 242 calculates the feature quantities of the objects contained in the rectangular regions and recognizes the objects based on the calculated feature quantities. For example, the object recognition unit 242 recognizes objects in the 3D digital twin using an object recognition engine that employs machine learning such as deep learning. Objects can be recognized by machine learning the features of the object's 3D model and the object type. The object recognition result includes the object type, location information of the rectangular region containing the object, and an object type score. The object type score is the likelihood of the detected object type, i.e., the confidence level or certainty level. The object recognition unit 242 integrates the object recognition result into the 3D digital twin. That is, the constructed 3D digital twin contains information about the objects recognized within the 3D digital twin.

[0055] The prediction unit 250 predicts future 3D digital twins based on the constructed 3D digital twins. The prediction unit 250 predicts the 3D digital twin for the next step (timing), or for any n steps ahead. Specifically, the prediction unit 250 predicts the position (movement) of each object in the constructed 3D digital twin and generates a 3D digital twin that includes the predicted objects. It can also be said that the prediction unit 250 predicts the state of future 3D digital twins. For example, the prediction unit 250 may predict the position of an object in the next step by linear prediction from 3D digital twins at multiple steps, including past 3D digital twins. Alternatively, the position of each object in the 3D digital twin may be predicted using the technology described in Non-Patent Document 1 (Simultaneous Multi-Agent Recurrent Trajectory Prediction). For example, the position of each object in the 3D digital twin may be predicted using a prediction engine that uses machine learning such as deep learning.

[0056] The prediction unit 250 may acquire information other than the 3D digital twin and predict the position of an object. For example, the prediction unit 250 may acquire control information related to the movement of an autonomous vehicle, or it may acquire route information from a navigation application running on a terminal such as a car, pedestrian, or bicycle. The prediction unit 250 may acquire information such as right turns / left turns and lane changes from the control information of the autonomous vehicle or the route information of the navigation application, and predict the position of the vehicle, etc., based on the acquired information. This can improve the accuracy of predicting the position of an object.

[0057] Furthermore, the prediction unit 250 may acquire information about traffic lights. The prediction unit 250 may predict that a vehicle will stop when the traffic light turns red, and that the vehicle will start moving when the light turns green. The prediction unit 250 may also acquire information about road congestion. For example, the prediction unit 250 may predict the movement of a vehicle based on changes in speed when the vehicle is stuck in a traffic jam, when the vehicle joins the end of a traffic jam, or when the vehicle leaves a traffic jam. This can improve the accuracy of predicting the position of an object.

[0058] The sensor viewpoint data generation unit 260 generates sensor viewpoint data for each sensor from the predicted 3D digital twin. The sensor viewpoint data generation unit 260 also acts as a conversion unit that converts the 3D digital twin into sensor viewpoint data for each sensor. Sensor viewpoint data is sensor data acquired when a sensor measures the 3D digital twin. For example, if the sensor is a camera, it is image data obtained by photographing the 3D digital twin from the camera's position, orientation, and field of view. If the sensor is a LiDAR, it is point cloud data obtained by measuring the 3D digital twin from the LiDAR's position, orientation, and measurement range. For example, sensor viewpoint data for each sensor may be generated from the 3D digital twin using a simulator that can observe a virtual 3D model from various viewpoints.

[0059] Figure 6 shows an example of the operation of a vehicle-infrastructure cooperative system 1 according to several embodiments. In the example in Figure 6, terminals 100-1 to 100-4 are connected to sensors 101-1 to 101-4, respectively. Sensors 101-1 to 101-4 may be cameras or LiDARs, respectively.

[0060] First, sensors 101-1 to 101-4 perform capture (measurement) in step i and generate sensor data including video data or point cloud data (S101). Terminals 100-1 to 100-4 acquire the sensor data captured by sensors 101-1 to 101-4 in step i and transmit the acquired sensor data to the vehicle-infrastructure cooperation server 200 (S102). Specifically, encoder 130 encodes the sensor data acquired from sensor 101. Transmission unit 140 transmits the encoded data to the vehicle-infrastructure cooperation server 200 via base station 300.

[0061] Next, the vehicle-infrastructure cooperation server 200 receives sensor data 101-1 to 101-4 from terminals 100-1 to 100-4, and uses the received sensor data from sensors 101-1 to 101-4 to construct a three-dimensional digital twin of step i (S103). Specifically, the receiving unit 210 receives encoded data from multiple terminals 100 via the base station 300. The decoder 230 decodes the received encoded data and restores the sensor data. The data integration unit 241 integrates the sensor data from the multiple sensors of step i to construct a three-dimensional digital twin of step i. The object recognition unit 242 recognizes an object from the constructed three-dimensional digital twin and integrates the object recognition results (object position, type, etc.) into the three-dimensional digital twin.

[0062] Next, the vehicle-infrastructure cooperation server 200 predicts the movement of each object in the next step i+1, for example, from the three-dimensional digital twin of step i that it has constructed (S104). The prediction unit 250 predicts the position of the objects in step i+1 from the three-dimensional digital twin of step i and generates a three-dimensional digital twin of step i+1 that includes the predicted objects.

[0063] Next, the vehicle-infrastructure cooperation server 200 generates sensor viewpoint data for sensors 101-1 to 101-4 in step i+1 from the predicted 3D digital twin (S105). The sensor viewpoint data generation unit 260 converts the predicted 3D digital twin in step i+1, which includes the object, into sensor viewpoint data from sensors 101-1 to 101-4. Based on the type, position, orientation, etc., of sensors 101-1 to 101-4, the sensor viewpoint data generation unit 260 generates video data or point cloud data from each viewpoint.

[0064] Next, the vehicle-infrastructure cooperation server 200 notifies terminals 100-1 to 100-4 of the prediction results of sensors 101-1 to 101-4 in step i+1 (S106). The transmission unit 220 may transmit the sensor viewpoint data of each sensor predicted in step i+1 to terminals 100-1 to 100-4 as a prediction result, or it may transmit the position information and object type of each object in each sensor viewpoint data to terminals 100-1 to 100-4. The position information of each object indicates the position (2D coordinates or 3D coordinates) in the video data or point cloud data which is the sensor viewpoint data.

[0065] Furthermore, sensors 101-1 to 101-4 capture (measure) in step i+1 and generate sensor data (S107). Terminals 100-1 to 100-4 acquire the sensor data captured by sensors 101-1 to 101-4 in step i+1, and also receive the prediction results for sensors 101-1 to 101-4 in step i+1 from the vehicle-infrastructure cooperation server 200. Based on the acquired sensor data and the prediction results, they extract important regions of the sensor data (S108). The important region extraction unit 120 identifies regions where the difference between the sensor data in step i+1 and the predicted prediction results for step i+1 is large as important regions.

[0066] Figure 7 shows a specific example of the important region extraction process according to several embodiments. In the example in Figure 7, the important region extraction unit 120 performs object matching between sensor data and prediction results, or compares data between sensor data and prediction results (S201).

[0067] For example, if the prediction result includes information about an object (location and type), the important region extraction unit 120 uses an object recognition engine or the like to perform object recognition processing on the sensor data to recognize objects included in the sensor data, and matches the location and type of the object included in the object recognition result of the sensor data with the location and type of the object included in the prediction result.

[0068] Furthermore, if the prediction result includes sensor viewpoint data, the important region extraction unit 120 compares the sensor data with the sensor viewpoint data. That is, it compares the predicted video data with the measured video data, or compares the predicted point cloud data with the measured point cloud data. In addition, even when comparing sensor data with sensor viewpoint data, object recognition may be performed on the sensor data and sensor viewpoint data, and the recognized objects may be matched.

[0069] Next, the important region extraction unit 120 determines whether the difference between the sensor data and the prediction result is greater than a threshold (S202), and identifies the region where the difference is greater than the threshold as an important region (S203).

[0070] For example, when object matching is performed, the important region extraction unit 120 identifies the region containing the object whose difference is greater than the threshold as an important region if the difference between the object's position in the sensor data and the object's position in the prediction result is greater than the threshold for the matched object.

[0071] Furthermore, when comparing sensor data and sensor viewpoint data, the important region extraction unit 120 identifies areas where the difference between the sensor data and sensor viewpoint data is greater than a threshold as important regions. For example, in the case of video data, areas where the difference in the value of each pixel is large may be identified as important regions, and in the case of point cloud data, areas where the difference in the coordinates of each point is large may be identified as important regions.

[0072] Returning to Figure 6, terminals 100-1 to 100-4 then transmit the important region of the sensor data from step i+1 of sensors 101-1 to 101-4 to the vehicle-infrastructure cooperation server 200 (S109). The encoder 130 encodes only the important region of the sensor data, and the transmission unit 140 transmits the encoded data to the vehicle-infrastructure cooperation server 200 via the base station 300. The transmission unit 140 may transmit only the data in which the important region of the sensor data has been encoded (compressed), or it may transmit data in which the difference between the sensor data and the prediction result (sensor viewpoint data or information on each object) has been encoded.

[0073] Next, the vehicle-infrastructure cooperation server 200 receives important regions of sensor data from sensors 101-1 to 101-4 from terminals 100-1 to 100-4, and uses the received important regions of sensor data from sensors 101-1 to 101-4 to construct a step i+1 three-dimensional digital twin (S110). The receiving unit 210 receives encoded data (important regions or differences) from multiple terminals 100 via the base station 300, and the decoder 230 decodes the received encoded data. The data integration unit 241 integrates the received important region or difference data into the predicted step i+1 three-dimensional digital twin to construct a step i+1 three-dimensional digital twin.

[0074] As described above, in this embodiment, in the vehicle-infrastructure cooperative system, future sensing results for each sensor are predicted based on a three-dimensional digital twin constructed from sensor data, and only the area where the difference between the predicted result and the actual sensing result exceeds a specified amount is distributed from the terminal. This significantly reduces the amount of data transmitted from the sensor terminal to the server. Therefore, the amount of communication between the terminal and the base station for each sensor can be reduced, and even if the number of sensors increases, it becomes possible to construct a three-dimensional digital twin with sufficient accuracy without a shortage of communication resources.

[0075] (Modification 1 of Embodiment 2) As a modification 1 of Embodiment 2, the transmission of critical areas may be controlled according to the communication quality between the terminal and the base station. By changing whether or not to transmit critical areas and the compression ratio according to the communication quality, critical areas can be transmitted appropriately.

[0076] Figure 8 shows an example configuration of a terminal 100 and a vehicle-infrastructure cooperative server 200 according to several embodiments, illustrating another example of the configuration in Figure 5. In the example in Figure 8, the vehicle-infrastructure cooperative server 200 further includes a communication quality acquisition unit 270 in addition to the configuration in Figure 5. The communication quality acquisition unit 270 acquires information regarding the communication quality between the terminal 100 and the base station 300. The information regarding communication quality may be, for example, the communication bandwidth that each terminal 100 can use with the base station 300, or a combination of the base station 300's total communication bandwidth and the number of sensors (number of terminals). For example, the communication quality acquisition unit 270 may acquire information regarding communication quality from the base station 300.

[0077] In the example shown in Figure 8, when the transmission unit 220 of the vehicle-infrastructure cooperation server 200 notifies the terminal 100 of the prediction result, it also notifies the terminal 100 of information regarding communication quality along with the prediction result. The receiving unit 150 of the terminal 100 acquires the prediction result and the information regarding communication quality.

[0078] Furthermore, in addition to the configuration shown in Figure 5, terminal 100 also includes a transmission determination unit 160. The transmission determination unit 160 determines whether or not to transmit the important regions extracted by the important region extraction unit 120 and the compression ratio based on information regarding communication quality. For example, if the communication bandwidth available to terminal 100 is insufficient to transmit the important regions, it may decide not to transmit the important regions, or it may increase the compression ratio of the important regions. When adjusting the compression ratio, parameters related to the compression of video or point clouds are changed. In this case, parameters such as frame rate, resolution, RoI (region of interest) or other areas to be made high quality, color or monochrome, and the number of bits per pixel may be changed. For example, the number of bits per pixel is the number of bits for RGB or YUV respectively in the case of video, and the number of bits for each point in the case of point clouds.

[0079] (Modification 2 of Embodiment 2) As a modification 2 of Embodiment 2, important regions may be identified based on priority according to their location. By prioritizing regions within the 3D digital twin, regions that are more important than other regions can be reliably transmitted.

[0080] Figure 9 shows an example configuration of terminal 100 and vehicle-infrastructure cooperation server 200 according to several embodiments, illustrating another example of the configuration in Figure 5. In the example in Figure 9, the vehicle-infrastructure cooperation server 200 further includes a priority determination unit 271 in addition to the configuration in Figure 5.

[0081] The priority determination unit 271 determines the priority of each area based on the constructed 3D digital twin. For example, the priority determination unit 271 may, based on the object recognition results in the 3D digital twin, assign high priority to areas where the number of people or vehicles is greater than a predetermined value, and low priority to areas where the number of people or vehicles is less than a predetermined value. In other words, the priority determination unit 271 may determine the priority according to the type and number of objects recognized in the 3D digital twin. Furthermore, areas that are blind spots from the perspective of the sensor in the 3D digital twin may be assigned high priority, and areas that are visible may be assigned low priority. Here, a blind spot refers to an area where it is difficult for the sensor to acquire data, for example, an area that is hidden by the shadow of an object when viewed from the sensor and cannot be photographed or measured. An area that is visible refers to an area where it is easy for the sensor to acquire data, for example, an area where there are no obstructing objects when viewed from the sensor and it is possible to photograph or measure. Furthermore, 3D map information (a heat map may also be used) indicating the degree of traffic risk based on accident history may be stored in advance, and areas with a higher-than-specified number of accidents may be given high priority, while areas with a lower-than-specified number of accidents may be given low priority.

[0082] In the example shown in Figure 9, when the transmission unit 220 of the vehicle-infrastructure cooperation server 200 notifies the terminal 100 of the prediction results, it also notifies the priority of each area along with the prediction results. The receiving unit 150 of the terminal 100 obtains the prediction results and the priority of each area.

[0083] The important region extraction unit 120 identifies important regions of the sensor data based on the acquired prediction results and the priority of each region. For example, if there are multiple regions with a large difference between the sensor data and the prediction results, only the regions with high priority may be considered important regions. Alternatively, regions with a small difference between the sensor data and the prediction results may also be considered important regions if they have a high priority. Furthermore, the terminal 100 may perform object recognition from the sensor data measured by the sensor 101 and determine the priority of each region in the sensor data based on the object recognition results.

[0084] (Modification 3 of Embodiment 2) As a modification 3 of Embodiment 2, critical regions and whether or not to transmit critical regions may be determined based on the required accuracy of the 3D digital twin. Since the data required by the server differs depending on the required accuracy, by identifying critical regions according to the required accuracy, the appropriate critical regions that are needed can be transmitted.

[0085] Figure 10 shows an example configuration of a terminal 100 and a vehicle-infrastructure cooperative server 200 according to several embodiments, and illustrates another example of the configuration in Figure 5. In the example in Figure 11, the vehicle-infrastructure cooperative server 200 further includes a required accuracy setting unit 272 in addition to the configuration in Figure 5. The required accuracy setting unit 272 pre-sets the required accuracy necessary for constructing a three-dimensional digital twin. The required accuracy is also a condition for transmitting sensor data from the terminal 100 to the vehicle-infrastructure cooperative server 200. For example, the required accuracy may be a threshold (the threshold in Figure 7) used by the terminal 100 to determine the difference between the sensor data and the prediction result.

[0086] In the example shown in Figure 10, the transmission unit 220 of the vehicle-infrastructure cooperation server 200 notifies the terminal 100 of the prediction result, along with the requested accuracy. The receiving unit 150 of the terminal 100 acquires the prediction result and the requested accuracy.

[0087] The critical region extraction unit 120 identifies critical regions of the sensor data based on the acquired prediction results and required accuracy. For example, as shown in Figure 7, the difference between the sensor data and the prediction results may be compared with the required accuracy (threshold), and regions where the difference is greater than the required accuracy (threshold) may be determined to be critical regions. Furthermore, even if the difference between the sensor data and the prediction results is large, if the required accuracy is met, the critical regions do not need to be transmitted.

[0088] (Embodiment 3) Next, Embodiment 3 will be described. In this embodiment, an example will be described of transmitting an area of ​​unknown objects or events that cannot be recognized by the vehicle-infrastructure cooperation server.

[0089] Figure 11 shows an example configuration of a terminal 100 and a vehicle-infrastructure cooperative server 200 according to several embodiments. In the example of Figure 11, the terminal 100 is further equipped with an unknown area detection unit 170 in addition to the configuration of Figure 5.

[0090] The unknown area detection unit 170 detects an area of ​​an unknown object or event based on the prediction results notified by the vehicle-infrastructure cooperation server 200. The transmission unit 140 transmits the area containing the detected unknown object or event to the vehicle-infrastructure cooperation server 200.

[0091] The unknown area is the area other than the area containing the object predicted by the road-vehicle cooperation server 200, that is, the area where no object is recognized. For example, the unknown area detection unit 170 may use the difference between frames in the sensor data to select an object approaching from a distance on a straight road, etc., as an unknown object. Alternatively, the unit may identify the location of a road that extends outside the sensor's field of view in the sensor data, monitor changes in the area of ​​that road (such as an object moving from the outside to the inside), and select that area as an unknown area. The location of the road that extends outside the field of view may be identified from 3D map information, or it may be identified by online learning of the location where an object enters from outside the field of view. This narrows the monitoring area and reduces the processing load.

[0092] In this way, areas of unknown objects or events that cannot be recognized by the vehicle-infrastructure cooperation server may be transmitted from the terminal to the vehicle-infrastructure cooperation server. This allows areas not included in the prediction results of the vehicle-infrastructure cooperation server to be transmitted from the terminal to the vehicle-infrastructure cooperation server, enabling the construction of a highly accurate 3D digital twin.

[0093] (Embodiment 4) Next, Embodiment 4 will be described. In this embodiment, an example of filtering information transmitted from a terminal will be described.

[0094] Figure 12 shows an example configuration of a terminal 100 and a vehicle-infrastructure cooperative server 200 according to several embodiments. In the example of Figure 12, the vehicle-infrastructure cooperative server 200 is further equipped with a region identification unit 280 in addition to the configuration of Figure 5.

[0095] The area identification unit 280 identifies areas in the 3D digital twin where events such as accidents, congestion, traffic restrictions, and road construction are occurring. For example, a dynamic map containing quasi-dynamic information indicating areas where accidents, congestion, traffic restrictions, and road construction are occurring may be stored in advance, or quasi-dynamic information may be acquired from an external source. The quasi-dynamic information may include accident information, congestion information, traffic restriction information, road construction information, weather information, etc. The area identification unit 280 may identify areas where events such as accidents, congestion, traffic restrictions, and road construction are occurring based on the quasi-dynamic information of the dynamic map. The dynamic map may include static information, quasi-static information, and dynamic information. The static information may include 3D map information, road surface information, lane information, 3D structures, etc. The quasi-static information may include traffic restriction schedule information, road construction schedule information, weather forecast information, etc. The dynamic information may include vehicle information, pedestrian information, signal information, etc.

[0096] In the example shown in Figure 12, when the transmission unit 220 of the road-vehicle coordination server 200 notifies the terminal 100 of the prediction results, it also notifies event occurrence information indicating the area where an accident, traffic congestion, traffic restrictions, road construction, or other event is occurring. The receiving unit 150 of the terminal 100 acquires the prediction results and the event occurrence information.

[0097] Terminal 100, in addition to the configuration shown in Figure 5, further includes a transmission determination unit 180. The transmission determination unit 180 determines whether or not to transmit data in a critical area based on the acquired event occurrence information. For example, even if an area is determined to be a critical area, if it is an area where an event such as an accident, traffic congestion, traffic restrictions, or road construction is occurring, it may be determined not to transmit data in the critical area. The transmission determination unit 180 may also determine whether or not to transmit data in a critical area based on traffic restriction schedules or road construction schedules identified by quasi-static information.

[0098] In this way, information to be transmitted may be filtered based on the difference in the real-time nature of the events. In areas where road construction or accident response is being carried out, even if there is movement within that area, it is not necessary to grasp the detailed movement (depending on the application), so this information is not transmitted from the terminal to the vehicle-infrastructure cooperation server. This can further reduce the amount of data transmitted from the terminal.

[0099] This disclosure is not limited to the embodiments described above, and may be modified as appropriate without departing from its intent.

[0100] Each configuration in the above-described embodiment is composed of hardware, software, or both, and may consist of one piece of hardware or software, or multiple pieces of hardware or software. Functions (processing) of terminals, road-vehicle cooperation servers, etc., may be realized by a computer 30 having a processor 31 such as a CPU (Central Processing Unit) and a memory 32 as a storage device, as shown in Figure 13. For example, a program for performing the method in the embodiment may be stored in the memory 32, and each function may be realized by executing the program stored in the memory 32 with the processor 31.

[0101] The above program, when loaded into a computer, includes a set of instructions (or software code) for causing the computer to perform one or more of the functions described in the embodiments. The program may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrically, optically, acoustically or otherwise propagating signals.

[0102] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure can be made as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0103] Each drawing is merely illustrative to illustrate one or more embodiments. Each drawing may be associated with one or more other embodiments, rather than being associated with only one specific embodiment. As those skilled in the art will understand, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings, for example, to create embodiments not explicitly shown or described. Not all features or steps shown in any one drawing to illustrate an exemplary embodiment are necessarily required, and some features or steps may be omitted. The order of steps described in any of the drawings may be changed as appropriate.

[0104] Some or all of the above embodiments may also be described as follows, but are not limited to the following:

[0105] (Note 1) A data processing system comprising: a transmission unit that transmits first sensor data measured by a sensor at a first timing; a construction unit that constructs a first three-dimensional model based on the transmitted first sensor data; a prediction unit that predicts the state of a second three-dimensional model at a second timing after the first timing based on the constructed first three-dimensional model; and a specification unit that identifies a transmission region of the second sensor data to be transmitted by the transmission unit based on the difference between the predicted state of the second three-dimensional model and the second sensor data measured by the sensor at the second timing. (Note 2) The data processing system according to Note 1, wherein the specification unit identifies a region in which the difference is greater than a predetermined threshold as the transmission region. (Note 3) The data processing system according to Note 1 or 2, further comprising: a conversion unit that converts the state of the second three-dimensional model into third sensor data when measured from the sensor, wherein the specification unit determines the difference between the converted third sensor data and the second sensor data measured by the sensor. (Note 4) The data processing system according to Note 3, further comprising a notification unit that notifies the sensor of information about an object included in the third sensor data. (Note 5) The data processing system according to Note 4, wherein the identification unit determines the difference between information about an object included in the third sensor data and information about an object included in the second sensor data. (Note 6) The data processing system according to Note 1 or 2, wherein the identification unit determines the presence or absence of the transmission area or the compression ratio of the transmission area based on the communication quality of the network transmitted by the transmission unit. (Note 7) The data processing system according to Note 1 or 2, wherein the identification unit identifies the transmission area based on priority according to the position in the three-dimensional model. (Note 8) The data processing system according to Note 7, wherein the priority is determined according to the type and number of objects recognized in the three-dimensional model. (Note 9) The data processing system according to Note 1 or 2, wherein the identification unit identifies the transmission area based on the required accuracy of the three-dimensional model.(Note 10) The data processing system according to Note 1 or 2, wherein the identification unit identifies a region containing an object not included in the three-dimensional model as the transmission region. (Note 11) The data processing system according to Note 1 or 2, wherein the identification unit determines whether or not to transmit the transmission region depending on whether or not the transmission region is a region where a predetermined event is occurring. (Note 12) The data processing system according to Note 3, wherein the transmission unit transmits the difference between the transmission region and the third sensor data. (Note 13) The data processing system according to Note 5, wherein the transmission unit transmits the difference between the object information included in the third sensor data and the object information included in the second sensor data. (Note 14) A data processing method comprising: transmitting first sensor data measured by a sensor at a first timing; constructing a first three-dimensional model based on the transmitted first sensor data; predicting the state of a second three-dimensional model at a second timing after the first timing based on the constructed first three-dimensional model; and identifying the transmission region to be transmitted from the second sensor data based on the difference between the predicted state of the second three-dimensional model and the second sensor data measured by the sensor at the second timing. (Note 15) A program for causing a computer to perform a process that includes: transmitting first sensor data measured by a sensor at a first timing; constructing a first three-dimensional model based on the transmitted first sensor data; predicting the state of a second three-dimensional model at a second timing after the first timing based on the constructed first three-dimensional model; and identifying the transmission region to be transmitted from the second sensor data based on the difference between the predicted state of the second three-dimensional model and the second sensor data measured by the sensor at the second timing.

[0106] Some or all of the elements (e.g., configuration and function) described in Appendices 2 to 13 that are subordinate to Appendice 1 (Data Processing System) may also be subordinate to Appendice 14 (Data Processing Method) and Appendice 15 (Program) in the same way as those described in Appendices 2 to 13. Some or all of the elements described in any appendice may be applied to various hardware, software, recording means, systems, and methods for recording software.

[0107] This application claims priority based on Japanese Patent Application No. 2024-170282, filed on 30 September 2024, and incorporates all of its disclosures herein.

[0108] 1. Infrastructure-vehicle cooperation system 10. Data processing system 11. Transmission unit 12. Construction unit 13. Prediction unit 14. Identification unit 21, 22. Data processing device 30. Computer 31. Processor 32. Memory 100. Terminal 101. Sensor 110. Acquisition unit 120. Important area extraction unit 130. Encoder 140. Transmission unit 150. Receiving unit 160. Transmission determination unit 170. Unknown area detection unit 180. Transmission determination unit 200. Infrastructure-vehicle cooperation server 200a. Cloud server 200b. MEC 210. Receiving unit 220. Transmission unit 230. Decoder 240. 3D digital twin construction unit 241. Data integration unit 242. Object recognition unit 250. Prediction unit 260. Sensor viewpoint data generation unit 270. Communication quality acquisition unit 271. Priority determination unit 272. Request accuracy setting section 280 Area specification section

Claims

1. A data processing system comprising: a transmission unit that transmits first sensor data measured by a sensor at a first timing; a construction unit that constructs a first three-dimensional model based on the transmitted first sensor data; a prediction unit that predicts the state of a second three-dimensional model at a second timing after the first timing based on the constructed first three-dimensional model; and a specification unit that identifies a transmission region of the second sensor data to be transmitted by the transmission unit based on the difference between the predicted state of the second three-dimensional model and the second sensor data measured by the sensor at the second timing.

2. The data processing system according to claim 1, wherein the identifying unit identifies a region in which the difference is greater than a predetermined threshold as the transmission region.

3. A data processing system according to claim 1 or 2, comprising a conversion unit that converts the state of the second three-dimensional model into third sensor data when measured from the sensor, wherein the identification unit calculates the difference between the converted third sensor data and the second sensor data measured by the sensor.

4. The data processing system according to claim 3, further comprising a notification unit that notifies the sensor of information about an object included in the third sensor data.

5. The data processing system according to claim 4, wherein the identifying unit determines the difference between the information of an object included in the third sensor data and the information of an object included in the second sensor data.

6. The data processing system according to claim 1 or 2, wherein the identifying unit determines the presence or absence of the transmission area, or the compression ratio of the transmission area, based on the communication quality of the network transmitted by the transmitting unit.

7. The data processing system according to claim 1 or 2, wherein the identifying unit identifies the transmission area based on priority according to its position in the three-dimensional model.

8. The data processing system according to claim 7, wherein the priority order is determined according to the type and number of objects recognized in the three-dimensional model.

9. The data processing system according to claim 1 or 2, wherein the identifying unit identifies the transmission area based on the required accuracy of the three-dimensional model.

10. The data processing system according to claim 1 or 2, wherein the identifying unit identifies a region containing an object not included in the three-dimensional model as the transmission region.

11. The data processing system according to claim 1 or 2, wherein the identifying unit determines whether or not to transmit data in the transmission area depending on whether or not the transmission area is an area where a predetermined event is occurring.

12. The data processing system according to claim 3, wherein the transmitting unit transmits the difference between the transmission area and the third sensor data.

13. The data processing system according to claim 5, wherein the transmitting unit transmits the difference between the information of an object included in the third sensor data and the information of an object included in the second sensor data.

14. A data processing method comprising: transmitting first sensor data measured by a sensor at a first timing; constructing a first three-dimensional model based on the transmitted first sensor data; predicting the state of a second three-dimensional model at a second timing after the first timing based on the constructed first three-dimensional model; and identifying the transmission region to be transmitted from the second sensor data based on the difference between the predicted state of the second three-dimensional model and the second sensor data measured by the sensor at the second timing.

15. A program for causing a computer to perform a process that includes: transmitting first sensor data measured by a sensor at a first timing; constructing a first three-dimensional model based on the transmitted first sensor data; predicting the state of a second three-dimensional model at a second timing after the first timing based on the constructed first three-dimensional model; and identifying the transmission region of the second sensor data to be transmitted based on the difference between the predicted state of the second three-dimensional model and the second sensor data measured by the sensor at the second timing.

Images