Map update device, map update method, and program

Abstract

To provide a platform that lets multiple users cooperate to update urban space maps representing three-dimensional urban spaces, the platform having never existed so far.SOLUTION: A map update device 1 includes: an image receiving unit 121 that receives images obtained by photographing an urban space and associated with location information from each of two or more users' terminal devices 2; an image acquiring unit 131 that acquires three-dimensional images of all or part of the urban space using all or some of the images; and an updating unit 135 that performs an update process to update, by using the three-dimensional images acquired by the image acquiring unit 131, a three-dimensional image in an urban space map in a map storage unit 111, which is a three-dimensional image in an urban space map corresponding to location information of an area that matches an area indicated by the location information associated with the images that are origin of the three-dimensional images acquired by the image acquiring unit 131. The map update device can provide a platform where multiple users cooperate to update the urban space map.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a map updating device that updates a three-dimensional urban space map using received images. [Background technology]

[0002] In the prior art, there is a 3D map correction device that claims to be capable of correcting low-precision 3D maps (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2010-191066 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the prior art, there was no platform where multiple users could cooperate to update an urban space map that represents an urban space. [Means for solving the problem]

[0005] The map updating device of the first invention is a map updating device comprising: an image receiving unit that receives images, which are photographed of urban space and correspond to location information, from each of two or more users' terminal devices; an image acquiring unit that acquires a three-dimensional image of all or part of the urban space using all or part of the images received by the image receiving unit; and an update unit that performs an update process to update, with the three-dimensional image acquired by the image acquiring unit, the three-dimensional image in the urban space map in a map storage unit in which an urban space map, which is a collection of three-dimensional images corresponding to location information and which is three-dimensional information representing urban space, is stored, and which corresponds to location information of an area that matches the area indicated by the location information corresponding to the image that was the basis of the three-dimensional image acquired by the image acquiring unit.

[0006] This configuration makes it possible to provide a platform where multiple users can work together to update urban space maps.

[0007] In addition, compared to the first invention, the map update device of the second invention is a map update device in which the image receiving unit receives images corresponding to a user identifier that identifies a user and location information from each of two or more terminal devices, and the update unit stores the three-dimensional images acquired by the image acquisition unit in correspondence with the user identifier.

[0008] With this configuration, it is possible to manage users who have contributed to updating the urban space map.

[0009] In addition, the map updating device of the third invention is a map updating device in which, compared to the first or second invention, the image acquisition unit acquires a three-dimensional image excluding dynamic objects contained in the image received by the image receiving unit.

[0010] With this configuration, an appropriate urban space map can be constructed.

[0011] In addition, the map updating device of the fourth invention is a map updating device in which, compared to any one of the first to third inventions, the image acquisition unit acquires a three-dimensional image that does not include the static object when the three-dimensional image acquired by the image acquisition unit includes only a portion of the static object.

[0012] With this configuration, an appropriate urban space map can be constructed.

[0013] Furthermore, the map updating device of the fifth invention, compared to the fourth invention, further includes a partial storage unit that stores partial objects, which are information about a portion of a static object, when the three-dimensional image acquired by the image acquisition unit includes only a portion of the static object, and an object determination unit that determines whether the entire static object can be constructed using the partial objects stored by the partial storage unit and the partial objects, which are information about a portion of the static object in the information based on the image received by the image receiving unit after the partial objects have been stored.When the object determination unit determines that the entire static object can be constructed, the image acquisition unit acquires a three-dimensional image of the static object using the partial objects stored by the partial storage unit and the partial objects in the information based on the image received by the image receiving unit, and the update unit performs an update process to store the three-dimensional image of the static object acquired by the image acquisition unit in the map storage unit.

[0014] With this configuration, a more appropriate urban space map can be constructed.

[0015] In addition, the map updating device of the sixth invention is a map updating device according to any one of the first to fifth inventions, in which the image receiving unit receives images having two or more different data structures, and the image acquiring unit performs different conversion processing according to the data structure of the image received by the image receiving unit, thereby acquiring a three-dimensional image.

[0016] With this configuration, it is possible to obtain the cooperation of many users for updating the urban space map.

[0017] Furthermore, the map updating device of the seventh invention is a map updating device in which, compared to any one of the first to sixth inventions, the urban space map is point cloud data which is a collection of point data which is data representing points that make up a three-dimensional urban space and has position information and color information, and the image acquisition unit acquires a three-dimensional image having the point cloud data using all or part of the image received by the image receiving unit.

[0018] This configuration makes it possible to provide a platform where multiple users can work together to update urban space maps.

[0019] In addition, the map update device of the eighth invention is a map update device according to any one of the first to seventh inventions, further comprising a condition judgment unit which, when the image receiving unit receives an image, judges whether or not an update condition for updating the urban space map using the image is satisfied, and the update unit performs the update process only when the condition judgment unit judges that the update condition is satisfied.

[0020] This configuration makes it possible to provide a platform where multiple users can work together to update urban space maps.

[0021] Furthermore, the map updating device of the ninth invention is a map updating device in which, compared to the eighth invention, the update conditions include any of the following conditions: image conditions related to one or more image attribute values ​​which are attribute values ​​of the image received by the image receiving unit, user conditions related to one or more user attribute values ​​of the user who sent the image, or environmental conditions related to one or more environmental attribute values ​​related to the environment in which the image was acquired.

[0022] This configuration makes it possible to provide a platform where multiple users can work together to update urban space maps.

[0023] In addition, the map update device of the tenth invention is a map update device according to the eighth or ninth invention, further comprising a result transmitting unit that transmits result information regarding whether or not the update unit has performed an update to the terminal device that transmitted the image received by the image receiving unit.

[0024] This configuration makes it possible to provide a platform where multiple users can work together to update urban space maps.

[0025] Furthermore, the map update device of the eleventh invention is a map update device in which, compared to any one of the first to tenth inventions, the image acquisition unit acquires one or more object data based on an image received by the image receiving unit, and the update unit performs an update process in which the one or more object data acquired by the image acquisition unit are stored in the map storage unit.

[0026] This configuration makes it possible to provide a platform where multiple users can work together to update urban space maps.

[0027] Furthermore, the map update device of the twelfth invention is a map update device in which, compared to any one of the first to tenth inventions, the image acquisition unit acquires one or more unit data which are data based on an image received by the image receiving unit and are data within the range of a unit to be updated, and the update unit performs an update process in which the one or more unit data acquired by the image acquisition unit are stored in the map storage unit.

[0028] This configuration makes it possible to provide a platform where multiple users can work together to update urban space maps. Effect of the Invention

[0029] The map updating device according to the present invention can provide a platform where multiple users can cooperate to update an urban space map. [Brief description of the drawings]

[0030] [Figure 1] Conceptual diagram of map system A in embodiment 1. [Diagram 2] Block diagram of map system A [Diagram 3] A flowchart illustrating an example of the operation of the map update device 1. [Figure 4] A flowchart illustrating an example of the update process. [Diagram 5] A flowchart illustrating an example of the condition determination process. [Figure 6] 11 is a flowchart illustrating a first example of the image acquisition process. [Figure 7] Flowchart for explaining an example of a point cloud data acquisition process [Figure 8] A flowchart illustrating an example of the dynamic object removal process. [Figure 9] A flowchart illustrating an example of the partial object process. [Figure 10] A flowchart illustrating an example of the object determination process. [Figure 11] 1 is a flowchart illustrating a first example of the image storage process. [Figure 12] 11 is a flowchart illustrating a second example of the image storage process. [Figure 13] 11 is a flowchart illustrating a second example of the image acquisition process. [Figure 14] 11 is a flowchart illustrating a third example of the image storage process. [Figure 15] 11 is a flowchart illustrating a first example of the output image acquisition process. [Figure 16] 11 is a flowchart illustrating a second example of the output image acquisition process. [Figure 17] A flowchart for explaining an example of the operation of the terminal device 2 [Figure 18] Overview of the computer system [Figure 19] Block diagram of the computer system DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] Hereinafter, an embodiment of a map update device and the like will be described with reference to the drawings. In the embodiments, components with the same reference numerals perform similar operations, so that repeated description may be omitted.

[0032] (Embodiment 1) In this embodiment, a map updating device that updates a part of a three-dimensional urban space map using images received from terminal devices of two or more users will be described.

[0033] In addition, in this embodiment, a map updating device that associates an updated three-dimensional image with the user identifier of the user who transmitted the original image will be described.

[0034] In addition, in this embodiment, a map updating device is described which updates the urban space map by excluding dynamic objects in the received images.

[0035] In addition, in this embodiment, a map updating device will be described in which, when a received image contains only a part of an object, information on the object is not used to update the urban space map.

[0036] In addition, in this embodiment, a map update device is described that, when a received image contains only a portion of an object, stores information about that portion of the object, and stores information about the object once data for the entire object is available.

[0037] Also, in this embodiment, a map updating device will be described that can update a part of a three-dimensional urban space map using images received with two or more different data structures.

[0038] In addition, in this embodiment, a map updating device is described which converts a received image into an intermediate data structure (e.g., point cloud data), partially updates the urban space map of the intermediate data structure, and constructs and accumulates an object-based urban space map from the updated intermediate data.

[0039] Furthermore, in this embodiment, a map update device is described which, when an image is received, determines whether or not the image satisfies an update condition, and uses the image to update a portion of the urban space map only if the image satisfies the update condition.

[0040] In this specification, information X being associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is not important. Information X and information Y may be linked, may exist in the same buffer, information X may be included in information Y, or information Y may be included in information X, etc.

[0041] 1 is a conceptual diagram of a map system A according to the present embodiment. The map system A includes a map update device 1 and two or more terminal devices 2.

[0042] The map updating device 1 is a device that updates a three-dimensional urban space map using an image received from the terminal device 2. The map updating device 1 is, for example, a cloud server or an ASP server, but the type is not important.

[0043] The terminal device 2 is a terminal used by a user. The terminal device 2 may be, for example, a personal computer, a smartphone, or a tablet terminal, but the type of the terminal device 2 is not important.

[0044] The map updating device 1 and two or more terminal devices 2 are capable of communicating with each other via a network such as the Internet.

[0045] 2 is a block diagram of the map system A in this embodiment. The map update device 1 includes a storage unit 11, a receiving unit 12, a processing unit 13, and a transmitting unit .

[0046] The storage unit 11 includes a map storage unit 111 and a user management unit 112. The reception unit 12 includes an image reception unit 121 and an instruction reception unit 122. The processing unit 13 includes an image acquisition unit 131, a partial accumulation unit 132, an object determination unit 133, a condition determination unit 134, an update unit 135, and an output image acquisition unit 136. The transmission unit 14 includes a result transmission unit 141 and an output image transmission unit 142.

[0047] The terminal device 2 includes a terminal storage unit 21, a terminal reception unit 22, a terminal processing unit 23, a terminal transmission unit 24, a terminal reception unit 25, and a terminal output unit .

[0048] Various types of information are stored in the storage unit 11 constituting the map updating device 1. The various types of information are, for example, an urban space map (to be described later), user information (to be described later), one or more update conditions, or an object management table.

[0049] The update condition is a condition for determining whether or not to update the urban space map using the received image. The update condition may be a condition for determining not to update the urban space map, or a condition for determining to update the urban space map. The update condition is, for example, one or a combination of two or more of an image condition, a user condition, and an environmental condition. It goes without saying that the update condition of the storage unit 11 may be embedded in the program.

[0050] The image condition is a condition related to one or more image attribute values, which are attribute values ​​of the image received by the image receiving unit 121. The image attribute values ​​are, for example, time information, image size, area information specifying the area covered by the image, and the size of the area covered by the image. The time information is information specifying the time when the image was taken. The time information is, for example, the shooting date and time, and the shooting date. The image condition is, for example, "the time when the image was taken is newer than the time when the target area in the urban space map was taken," "the size of the image is equal to or larger than the threshold," and "the size of the area covered by the image is equal to or larger than the threshold." Note that the target area is the same area as the area covered by the image received by the image receiving unit 121.

[0051] The user condition is a condition related to one or more user attribute values ​​of the user who transmitted the image. For example, the user condition is that "the rank of the user is equal to or higher than a threshold value."

[0052] The environmental condition is a condition related to one or more environmental attribute values ​​related to the environment in which the image was captured. The environmental attribute values ​​are, for example, information indicating the weather, temperature, humidity, and whether the image is captured outdoors or indoors.

[0053] An object management table is a table that manages one or two types of objects, dynamic objects and static objects. An object management table is, for example, a table that manages the correspondence between "object name" and "object type." Dynamic objects are, for example, people and moving objects. Static objects are, for example, buildings, bridges and roads. Records in the object management table are, for example, "people: dynamic object," "moving object: dynamic object," "buildings: static object," "bridges: static object," and "roads: static object."

[0054] An urban space map is stored in the map storage unit 111. Urban space maps of two or more types of data structures may be stored in the map storage unit 111. The two types of data structures are, for example, point cloud data (described later) and object sets (described later).

[0055] An urban space map is three-dimensional information that represents urban space. It can be said that an urban space map is a collection of three-dimensional images that correspond to location information. A three-dimensional image is one or more pieces of point cloud data, or a collection of one or more objects. An urban space map is, for example, point cloud data or a collection of objects. The collection of three-dimensional images can be a single file, and the data structure, etc., does not matter.

[0056] Point cloud data is a collection of point data. The point data has position information. The point data preferably has color information. The point data preferably has both position information and color information.

[0057] Position information is information that specifies a three-dimensional position in space. Position information can be expressed as (x, y, h). For example, position information is (latitude, longitude, altitude). It is preferable that position information is information that specifies an absolute position. However, position information may also be information that specifies a relative position from a reference point.

[0058] Color information is information that specifies a color. Usually, it is (R,G,B), but it can also be (C,M,Y,K), etc.

[0059] The point data has, for example, a structure of (x,y,h,R,G,B), where "x" is the latitude, "y" is the longitude, "h" is the altitude, "R" is the red value, "G" is the green value, and "B" is the blue value.

[0060] One or more point data in the point cloud data constituting the urban space map may be managed for each object. In other words, each of the two or more point data may be associated with an object identifier. Note that an object identifier is information for identifying an object. An object identifier is, for example, an object ID or an object name.

[0061] An object set is a set of two or more object data. Object data is information that represents an object. The object data may be, for example, data that represents an object in a graph or point cloud data, but the structure is not important. The object data may correspond to, for example, an object identifier. Data that represents an object in a graph may be, for example, a "3D Scene Graph" (see web page URL "https: / / github.com / StanfordVL / 3DSceneGraph").

[0062] An object is an object that may exist in an urban space. An object is usually a physical object. An object can be a static object or a dynamic object. A static object is usually an object that does not move autonomously. Static objects are, for example, buildings and roads. A dynamic object is usually an elephant that moves autonomously. Dynamic objects are, for example, moving objects, people and animals. Moving objects are, for example, cars, motorcycles, trucks, trains and airplanes.

[0063] The user management unit 112 stores one or more pieces of user information. The user information is information related to a user. The user information usually includes a user identifier. The user information includes, for example, one or more user attribute values.

[0064] A user identifier is information that identifies a user. The user identifier is, for example, a user ID, a telephone number, or an email address. The user attribute values ​​are, for example, age, gender, the number of image transmissions, and rank. The number of image transmissions is the number of times the user has transmitted images to the map update device 1. The rank is the rank of the user. For example, the rank is higher the more images are transmitted. For example, the rank is higher the more times the urban space map is updated using images transmitted by the user. For example, the rank is lower the more times the user has transmitted images but not updated the urban space map using those images.

[0065] The receiving unit 12 receives various types of information and instructions from the terminal device 2. The various types of information and instructions are, for example, images, which will be described later, and viewing instructions, which will be described later.

[0066] The image receiving unit 121 receives images from two or more users' terminal devices 2. The images received by the image receiving unit 121 are information obtained by sensing urban space. It can also be said that the images received by the image receiving unit 121 are information obtained by scanning urban space. The images may also be called sensing information.

[0067] It is preferable that the image receiving unit 121 receives an image associated with a user identifier and location information from each of two or more terminal devices. It is preferable that the image receiving unit 121 receives an image having two or more different data structures. In other words, it is preferable that the data structures of the images transmitted from each of two or more terminal devices 2 may be different. It is preferable that one or more image attribute values ​​are associated with an image. The one or more image attribute values ​​are, for example, time information. It is preferable that the time information is information that specifies the time, and in this case, for example, the shooting date and time, and the shooting date. It is preferable that one or more environmental attribute values ​​are associated with an image. The one or more environmental attribute values ​​are, for example, information indicating the weather when the image was taken, the temperature when the image was taken, and whether the location where the image was taken is indoors or outdoors.

[0068] The position information associated with the image received by the image receiving unit 121 is absolute position information, for example, (latitude, longitude, altitude). Moreover, such position information is absolute position information of a reference point in the image. The user identifier associated with the image received by the image receiving unit 121 is usually a user identifier that identifies the user who transmitted the image.

[0069] The image here is an image captured of an urban space. The image is, for example, a distance image, a LiDAR image, point cloud data, or graph data (for example, "3D Scene Graph"). A LiDAR image is an image acquired by a LiDAR. However, the data structure of the image received by the image receiving unit 121 does not matter. Note that an image associated with location information may be an image in which location information is included in the information constituting the image.

[0070] The number of images received by the image receiving unit 121 may be two or more. In other words, the image received by the image receiving unit 121 may be a moving image having two or more still images.

[0071] The instruction receiving unit 122 receives various instructions from the terminal device 2. The various instructions are, for example, a viewing instruction.

[0072] The viewing instruction is an instruction to view the urban space map. The viewing instruction includes, for example, area information that specifies an area of ​​the urban space. The area information is, for example, one or more pieces of location information.

[0073] The processing unit 13 performs various types of processing, such as processing performed by an image acquisition unit 131, a partial storage unit 132, an object determination unit 133, a condition determination unit 134, an update unit 135, and an output image acquisition unit 136.

[0074] The image acquisition unit 131 uses all or part of the images received by the image reception unit 121 to acquire a three-dimensional image of all or part of the urban space.

[0075] A three-dimensional image is three-dimensional information that represents an urban space, and is, for example, point cloud data or an object-based image.

[0076] The image acquiring unit 131 acquires a three-dimensional image by, for example, converting all or a part of the image received by the image receiving unit 121. However, the three-dimensional image may be the image received by the image receiving unit 121.

[0077] The image acquisition unit 131 converts, for example, a distance image, a LiDAR image, or graph data received by the image receiving unit 121, and acquires a three-dimensional image that is point cloud data. Note that the process of converting a distance image, a LiDAR image, or graph data and acquiring point cloud data is a publicly known technique. The image acquisition unit 131 acquires, for example, position information (absolute position information) of each point constituting the point cloud data from absolute position information corresponding to the distance image received by the image receiving unit 121 and relative position information of each point in the distance image, acquires color information of each point in the distance image, and acquires point cloud data that is a collection of point data having the position information and color information of each point.

[0078] The image acquisition unit 131 acquires, for example, a three-dimensional image excluding dynamic objects included in the image received by the image receiving unit 121. The image acquisition unit 131 performs object recognition processing on all or part of the image received by the image receiving unit 121 or all or part of an image (for example, a three-dimensional image) obtained by converting all or part of the image, and detects one or more objects. Then, the image acquisition unit 131 acquires, for example, type information indicating the type of each of the one or more objects. Next, the image acquisition unit 131 refers to, for example, an object management table, determines object data of one or more objects corresponding to the type information that are dynamic objects, and acquires a three-dimensional image excluding the determined one or more object data. Note that the image acquisition unit 131 may provide data of one or more objects acquired by the object recognition processing and a learning model to a machine learning prediction module, execute the prediction module, acquire a classification result of whether the object is a dynamic object or a static object, and acquire a three-dimensional image excluding one or more object data that are dynamic objects. In such a case, the learning model is information acquired by performing a machine learning learning process using two or more pieces of teacher data having object data and classification information as to whether the object is a dynamic object or a static object. The learning model may be called a learner, a classifier, a classification model, or the like. The machine learning algorithm here may be deep learning, random forest, decision tree, SVM, or the like. For the machine learning, various machine learning functions such as the TensorFlow (registered trademark) library and the random forest module of the R language, or various existing libraries, may be used.

[0079] For example, when only a part of a static object is included in an image received by the image receiving unit 121, the image acquiring unit 131 acquires a three-dimensional image that does not include the static object. That is, for example, the image acquiring unit 131 performs an object recognition process on all or a part of the image received by the image receiving unit 121 or all or a part of an image (for example, a three-dimensional image) obtained by converting all or a part of the image, and detects one or more objects. Then, the image acquiring unit 131 acquires, for example, type information indicating the type of each of the one or more objects. Next, the image acquiring unit 131 refers to, for example, an object management table, and acquires a three-dimensional image that leaves only object data of one or more objects that correspond to the type information that is a static object. Next, the image acquiring unit 131 determines whether or not the one or more static objects are missing a part. Then, the image acquiring unit 131 acquires a three-dimensional image that does not include a static object that is missing a part.

[0080] The image acquisition unit 131, for example, provides object data of a static object and a learning model to a machine learning prediction module, and obtains a judgment result as to whether the object data is data of a complete object or data of an object missing a part. The learning model is information obtained by providing two or more teacher data having object data and information indicating whether the object data is data of a complete object or data of an object missing a part to a machine learning learning module. The machine learning algorithm here may be deep learning, random forest, decision tree, SVM, or the like.

[0081] Note that the image acquisition unit 131 may acquire, for example, by the above-mentioned processing, an image excluding dynamic objects or a three-dimensional image which is a static object in a three-dimensional image and does not include a static object with a missing portion.

[0082] Furthermore, it is preferable that the partial storage unit 132, which will be described later, temporarily stores object data of a static object having a missing portion in the storage unit 11 or a buffer (not shown).

[0083] If the object determination unit 133 described later determines that a whole can be constructed using the partial object stored by the partial storage unit 132 and a part of a static object in the image-based information received by the image receiving unit 121 after the partial object was stored, the image acquisition unit 131 may acquire a three-dimensional image of the static object using the partial object stored by the partial storage unit 132 and the partial object in the image-based information received by the image receiving unit 121.

[0084] In addition, the information based on the image received by the image receiving unit 121 may be an image received by the image receiving unit 121, a three-dimensional image acquired by the image acquisition unit 131, or intermediate data constructed from the image received by the image receiving unit 121.

[0085] It is preferable that the image acquiring unit 131 acquires three-dimensional images of the same data structure by performing different conversion processes according to the data structures of two or more images received by the image receiving unit 121. In other words, it is preferable that the image acquiring unit 131 can acquire a three-dimensional image from images having two or more different types of data structures.

[0086] The image acquisition unit 131 acquires a three-dimensional image having point cloud data, for example, by using all or part of the image received by the image receiving unit 121.

[0087] For example, the image acquisition unit 131 acquires one or more object data based on the image received by the image receiving unit 121. For example, the image acquisition unit 131 performs an object recognition process on the image received by the image receiving unit 121 to detect one or more objects, and acquires object data for each of the one or more objects. Note that the object data is, for example, point cloud data or graph data.

[0088] The image acquisition unit 131 acquires one or more unit data, which is, for example, data based on an image received by the image receiving unit 121 and is data of a unit range to be updated. The unit data is information of a range of a certain size in urban space. The unit data is, for example, information of an area with a fixed width (w), depth (d), and height (h). The unit data is, for example, information of a cubic area with a fixed width (w), depth (d), and height (h). The unit data is a three-dimensional image.

[0089] The partial accumulation unit 132 accumulates a partial object, which is information on a portion of the static object, when only a portion of the static object is included in the three-dimensional image acquired by the image acquisition unit 131. The partial accumulation unit 132 accumulates the partial object, for example, in the storage unit 11 or a buffer not shown.

[0090] The object determination unit 133 determines whether or not the entire static object can be constructed using the partial object stored by the partial storage unit 132 and a part of the static object in the image-based information received by the image receiving unit 121 after the partial object was stored. The image-based information received by the image receiving unit 121 is, for example, an image received by the image receiving unit 121 or a three-dimensional image acquired by the image acquiring unit 131.

[0091] The object determination unit 133 acquires object data having, for example, partial objects stored by the partial storage unit 132 and partial objects based on images received by the image receiving unit 121. At this time, the object determination unit 133 combines two or more pieces of point data having matching position information among two or more pieces of partial object data into one piece of point data. In such a case, the object determination unit 133 prioritizes, for example, the point data of the partial object data associated with the latest shooting date and time. Next, the object determination unit 133 provides the object data acquired using two or more pieces of partial object data to a machine learning prediction module, and acquires a determination result as to whether the object data is data of a complete object or data of an object missing a part.

[0092] When the image receiving unit 121 receives an image, the condition determining unit 134 determines whether or not the update condition is satisfied.

[0093] The condition determination unit 134, for example, obtains one or more image attribute values ​​corresponding to an image received by the image receiving unit 121, and determines whether or not the one or more image attribute values ​​match an image condition. The condition determination unit 134, for example, obtains one or more image attribute values ​​from an image received by the image receiving unit 121, and determines whether or not the one or more image attribute values ​​match an image condition.

[0094] The condition determination unit 134, for example, acquires one or more user attribute values ​​associated with an image received by the image receiving unit 121, and determines whether or not the one or more user attribute values ​​match a user condition. The condition determination unit 134, for example, acquires a user identifier associated with an image received by the image receiving unit 121, acquires one or more user attribute values ​​paired with the user identifier from the storage unit 11, and determines whether or not the one or more user attribute values ​​match a user condition.

[0095] The condition determination unit 134, for example, acquires one or more environmental attribute values ​​associated with an image received by the image receiving unit 121, and determines whether or not the one or more environmental attribute values ​​match environmental conditions. The condition determination unit 134, for example, accesses a server (not shown) using location information associated with an image received by the image receiving unit 121 as a key, acquires one or more environmental attribute values ​​(for example, weather) from the server, and determines whether or not the one or more environmental attribute values ​​match environmental conditions.

[0096] The update unit 135 updates the three-dimensional image in the urban space map corresponding to the position information of an area that matches the area indicated by the position information corresponding to the image that is the basis of the three-dimensional image acquired by the image acquisition unit 131, with the three-dimensional image acquired by the image acquisition unit 131. Such processing is called an update processing. It is preferable that the update unit 135 stores the three-dimensional image acquired by the image acquisition unit 131 in the map storage unit 111 in association with the time information corresponding to the image. The time information corresponding to the image is, for example, the shooting date and time, or the shooting date.

[0097] The update unit 135 is, for example, a three-dimensional image of an area corresponding to the three-dimensional image acquired by the image acquisition unit 131, and overwrites the three-dimensional image in the map storage unit 111 with the three-dimensional image acquired by the image acquisition unit 131. Such overwriting is an example of an update process.

[0098] For example, the update unit 135 simply accumulates the three-dimensional image acquired by the image acquisition unit 131 in the map storage unit 111. For example, the update unit 135 accumulates the three-dimensional image acquired by the image acquisition unit 131 in the map storage unit 111 in association with time information. In this case, the old three-dimensional image of the area corresponding to the three-dimensional image acquired by the image acquisition unit 131 also remains in the map storage unit 111, but this may be considered to be an update process. In other words, the process by the update unit 135 to simply accumulate the three-dimensional image acquired by the image acquisition unit 131 in the map storage unit 111 may also be considered to be an update process. This is because, even in such a case, the information used to display the urban space is a three-dimensional image associated with the latest time information.

[0099] The update unit 135 performs an update process of, for example, storing in the map storage unit 111 the three-dimensional images of one or more static objects acquired by the image acquisition unit 131 .

[0100] It is preferable that the update unit 135 performs the update process only when the condition determination unit 134 determines that the update condition is satisfied.

[0101] The update unit 135 performs an update process of, for example, accumulating one or more pieces of object data acquired by the image acquisition unit 131 in the map storage unit 111. The update unit 135 performs an update process of, for example, accumulating one or more pieces of unit data acquired by the image acquisition unit 131 in the map storage unit 111. In other words, the update unit 135 may update any unit of information at one time. The update unit 135 may update each piece of object data, each piece of unit data, or the entire three-dimensional image acquired by the image acquisition unit 131.

[0102] The update unit 135 acquires result information. The result information is information on whether or not an update has been performed. For example, when an update process has been performed, the update unit 135 acquires result information indicating that "update has been performed." For example, when an update process has not been performed even though an image has been received, the update unit 135 acquires result information indicating that "update has not been performed."

[0103] The update unit 135 typically stores the acquired three-dimensional image in association with a user identifier associated with the received image.

[0104] The output image acquisition unit 136 constructs an urban space to be output using the urban space map in the map storage unit 111. The output image acquisition unit 136, for example, acquires area information included in the viewing instruction received by the instruction receiving unit 122, acquires a three-dimensional image of the area specified by the area information, and constructs an output image using the three-dimensional image. The output image acquisition unit 136, for example, acquires area information included in the viewing instruction received by the instruction receiving unit 122, and constructs an output image by arranging points of color information held by each point data of the three-dimensional image of the area specified by the area information at positions in virtual space that are positions of position information held by each point data.

[0105] The transmission unit 14 transmits various types of information, such as result information and an output image.

[0106] The result transmission unit 141 transmits the result information acquired by the update unit 135 to the terminal device 2 that transmitted the image received by the image reception unit 121 .

[0107] The output image transmission unit 142 transmits the output image acquired by the output image acquisition unit 136 to the terminal device 2. The terminal device 2 is usually the device that has sent the viewing instruction.

[0108] Various types of information are stored in the terminal storage unit 21 constituting the terminal device 2. The various types of information are, for example, a user identifier. Note that the user identifier in the terminal storage unit 21 may be information input by a user and temporarily stored.

[0109] The terminal reception unit 22 receives various information, instructions, etc. The various information, instructions, etc. are, for example, a viewing instruction, a photographing instruction, a transmission instruction, a user identifier, and one or more user attribute values. Note that a photographing instruction is an instruction to photograph an image. A transmission instruction is an instruction to transmit the photographed image.

[0110] The means for inputting various information and instructions may be anything, such as a touch panel, a keyboard, a mouse, a menu screen, a microphone, etc.

[0111] The terminal processing unit 23 performs various types of processing. For example, the various types of processing are processing for converting received information, instructions, etc. into information or instructions with a structure to be transmitted. For example, the various types of processing are processing for converting received information into information with a structure to be output.

[0112] The device processing unit 23 captures images and acquires images. That is, the device processing unit 23 includes, for example, a camera or LiDAR. Such images are, for example, distance images, LiDAR images, point cloud data, and graph data (for example, "3D Scene Graph").

[0113] The device processing unit 23 acquires, for example, one or more image attribute values. The device processing unit 23 acquires, for example, time information that is the shooting date and time.

[0114] The device processing unit 23 acquires, for example, one or more environmental attribute values. The device processing unit 23 acquires, for example, information indicating the weather, the temperature, and whether the device is outdoors or indoors.

[0115] The terminal transmission unit 24 transmits various information, instructions, etc. to the map update device 1. The various information, instructions, etc. are, for example, a viewing instruction, an image, a user identifier, one or more image attribute values, one or more user attribute values, and one or more environmental attribute values.

[0116] The terminal transmission unit 24 transmits the image acquired by the device processing unit 23 to the map update device 1, for example, in association with a user identifier. The terminal transmission unit 24 transmits the image acquired by the device processing unit 23 to the map update device 1, for example, in association with one or more types of attribute values ​​among one or more image attribute values, one or more user attribute values, and one or more environmental attribute values.

[0117] The terminal receiving unit 25 receives various types of information, such as an output image and result information.

[0118] The terminal output unit 26 outputs various types of information, such as an output image and result information.

[0119] Here, output is a concept that includes display on a display, projection using a projector, printing on a printer, sound output, transmission to an external device, storage on a recording medium, and handing over the processing results to other processing devices or other programs, etc.

[0120] The storage unit 11, the map storage unit 111, the user management unit 112, and the terminal storage unit 21 are preferably non-volatile recording media, but may also be realized as volatile recording media.

[0121] There is no restriction on the process by which information is stored in the storage unit 11, etc. For example, information may be stored in the storage unit 11, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 11, etc., or information inputted via an input device may be stored in the storage unit 11, etc.

[0122] The receiving unit 12, the image receiving unit 121, the instruction receiving unit 122, and the terminal receiving unit 25 are usually realized by wireless or wired communication means, but may be realized by means for receiving broadcasts.

[0123] The processing unit 13, image acquisition unit 131, partial storage unit 132, object determination unit 133, condition determination unit 134, update unit 135, output image acquisition unit 136, and device processing unit 23 can usually be realized by a processor, memory, etc. The processing procedure of the processing unit 13, etc. is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit). The processor may be a CPU, MPU, GPU, etc., and the type is not important.

[0124] The transmission unit 14, the result transmission unit 141, and the output image transmission unit 142 are usually realized by wireless or wired communication means, but may also be realized by broadcasting means.

[0125] The terminal reception unit 22 can be realized by a device driver for an input means such as a touch panel or a keyboard, or control software for a menu screen.

[0126] The terminal output unit 26 may be considered to include, or may not include, an output device such as a display, a speaker, etc. The terminal output unit 26 may be realized by driver software for an output device, or a combination of driver software for an output device and an output device, etc.

[0127] Next, an example of the operation of the map updating device 1 will be described with reference to the flowchart of FIG.

[0128] (Step S301) The image receiving unit 121 judges whether or not an image has been received from the terminal device 2. If an image has been received, the process proceeds to step S302, and if an image has not been received, the process proceeds to step S311. Note that a user identifier is associated with the image.

[0129] (Step S302) The processing unit 13 acquires a user identifier associated with the image received in step S301.

[0130] (Step S303) The processing unit 13 acquires the image received in step S301.

[0131] (Step S304) The processing unit 13 performs an update process using the image acquired in step S303. An example of the update process will be described with reference to the flowchart of FIG.

[0132] (Step S305) The processing unit 13 judges whether the urban space map has been updated as a result of the processing in step S304. If the urban space map has been updated, the process proceeds to step S306, and if the urban space map has not been updated, the process proceeds to step S307.

[0133] (Step S306) The processing unit 13 acquires result information indicating that the update has been performed. Then, the process proceeds to step S308.

[0134] (Step S307) The processing unit 13 acquires result information indicating that "no update was performed."

[0135] (Step S308) The result transmitting unit 141 transmits the result information acquired in step S307 to the terminal device 2 that transmitted the image.

[0136] (Step S309) The processing unit 13 uses the result information to change the rank paired with the user identifier. Then, the process returns to step S301.

[0137] For example, when the processing unit 13 obtains result information indicating that the information has been updated, the processing unit 13 increases the rank, and when the processing unit 13 obtains result information indicating that the information has not been updated, the processing unit 13 does not change the rank or decreases the rank.

[0138] (Step S310) The instruction receiving unit 122 judges whether or not a viewing instruction has been received from the terminal device 2. If a viewing instruction has been received, the process proceeds to step S311, and if not, the process proceeds to step S313.

[0139] (Step S311) The output image acquisition unit 136 acquires an output image. An example of such output image acquisition processing will be described with reference to the flowcharts of FIGS.

[0140] (Step S312) The output image transmission unit 142 transmits the output image acquired in step S311 to the terminal device 2 that has transmitted the viewing instruction. Return to step S301.

[0141] (Step S313) The receiving unit 12 judges whether or not user information has been received from the terminal device 2. If user information has been received, the process proceeds to step S314, and if not, the process returns to step S301. Note that the user information received here does not include a user identifier, but includes one or more user attribute values.

[0142] (Step S314) The processing unit 13 generates a unique user identifier. The processing unit 13 configures user information including the user identifier and the received one or more user attribute values. The processing unit 13 accumulates the user information in the user management unit 112.

[0143] (Step S314) The transmission unit 14 transmits the user identifier to the terminal device 2. The process returns to step S301.

[0144] In the flowchart of FIG. 3, the process ends when the power is turned off or an interrupt occurs to end the process.

[0145] Next, an example of the update process in step S304 will be described with reference to the flowchart in FIG.

[0146] (Step S401) The condition decision unit 134 acquires one or more image attribute values ​​of the image received in step S301.

[0147] (Step S402) The condition decision unit 134 acquires from the user management unit 112 one or more user attribute values ​​that are paired with the user identifier associated with the image received in step S301.

[0148] (Step S403) The condition decision unit 134 acquires one or more environmental attribute values ​​associated with the image received in step S301.

[0149] (Step S404) The condition determination unit 134 determines whether to update the urban space map using the received image using one or more types of information from one or more image attribute values, one or more user attribute values, and one or more environment attribute values. An example of such a condition determination process will be described with reference to the flowchart in FIG. 5.

[0150] (Step S405) If the result of the decision in step S404 is "update", the process proceeds to step S406, and if the result is "do not update", the process returns to the upper process.

[0151] (Step S406) The image acquisition unit 131 performs a process of acquiring a three-dimensional image using the received image. An example of such an image acquisition process will be described with reference to the flowcharts of FIGS.

[0152] (Step S407) The update unit 135 performs a storage process of the three-dimensional image using the three-dimensional image acquired in step S406. The process returns to the upper level process. An example of such an image storage process will be described with reference to the flowcharts of FIGS. 11 and 12.

[0153] Next, an example of the condition determination process in step S404 will be described with reference to the flowchart in FIG.

[0154] (Step S501) The condition decision unit 134 assigns 1 to a counter i.

[0155] (Step S502) The condition decision unit 134 decides whether or not the i-th update condition exists in the storage unit 11. If the i-th update condition exists, the process proceeds to step S503, and if not, the process proceeds to step S508.

[0156] (Step S503) The condition decision unit 134 acquires the i-th update condition.

[0157] (Step S504) The condition decision unit 134 acquires one or more attribute values ​​to be used for deciding whether the i-th update condition is satisfied. Each of the one or more attribute values ​​is an image attribute value, a user attribute value, or an environment attribute value.

[0158] (Step S505) The condition determination unit 134 determines whether the i-th update condition is met by using one or more attribute values ​​acquired in step S504. If the i-th update condition is met, the process proceeds to step S506, and if the i-th update condition is not met, the process proceeds to step S507.

[0159] (Step S506) The condition decision unit 134 increments the counter i by 1. The process returns to step S502.

[0160] (Step S507) The condition decision unit 134 assigns "do not update" to the variable "decision result." Then, the process returns to the upper level process.

[0161] (Step S508) The condition decision unit 134 assigns "update" to the variable "decision result." Then, the process returns to the upper level process.

[0162] In the flowchart of Fig. 5, the condition determination unit 134 determines not to update when the update condition is not met. However, the update condition may be a condition that determines to update when the update condition is not met, or a condition that determines to update when the update condition is met. The update condition may be a condition related to one attribute value, or a condition related to two or more attribute values. The update condition may be a condition in which two or more conditions are linked by AND, or a condition in which two or more conditions are linked by OR.

[0163] Next, a first example of the image acquisition process in step S406 will be described with reference to the flowchart in FIG.

[0164] (Step S601) The image acquisition unit 131 acquires point cloud data based on a received image. An example of such point cloud data acquisition processing will be described with reference to the flowchart in FIG.

[0165] (Step S602) The image acquisition unit 131 performs object recognition processing on the received image or the point cloud data acquired in step S601 to detect one or more objects. Note that the processing of recognizing objects from images of various data structures is a well-known technique, and therefore detailed description thereof will be omitted.

[0166] (Step S603) The image acquisition unit 131 performs a process of removing one or more dynamic objects among the one or more objects detected in step S602 from the point cloud data acquired in step S601. An example of such dynamic object removal process will be described with reference to the flowchart in FIG.

[0167] (Step S604) The image acquisition unit 131 performs processing on a partial object, which is an object in the point cloud data acquired in step S601 and is a part of the object. The process returns to the upper level processing. An example of such partial object processing will be described with reference to the flowchart in FIG. 9.

[0168] A partial object is an object that is included in the point cloud data, but is not entirely contained in the point cloud data.

[0169] Next, an example of the point cloud data acquisition process in step S601 will be described with reference to the flowchart in FIG.

[0170] (Step S701) The image acquisition unit 131 acquires a data structure identifier that identifies the data structure of the received image. The data structure identifier is, for example, a "range image," "LiDAR," or "point cloud data." The process of acquiring the data structure identifier of an image is a known technique.

[0171] (Step S702) The image acquisition unit 131 judges whether or not the data structure identifier acquired in step S701 is “point cloud data.” If it is “point cloud data,” the process proceeds to step S711, and if it is not “point cloud data,” the process proceeds to step S703.

[0172] (Step S703) The image acquisition unit 131 acquires reference position information, which is position information paired with the received image. The reference position information corresponds to, for example, the received image. The reference position information is absolute position information (usually (latitude, longitude, altitude)).

[0173] (Step S704) The image acquiring unit 131 assigns 1 to a counter i.

[0174] (Step S705) The image acquisition unit 131 determines whether the area of ​​the i-th point exists, which is the area of ​​points in the received image. If the i-th point exists, the process proceeds to step S706, and if the i-th point does not exist, the process returns to the upper process. The i-th point may be considered to be the area of ​​the i-th point.

[0175] (Step S706) The image obtaining unit 131 obtains position information of the i-th point in the received image, which is relative position information with respect to reference position information.

[0176] (Step S707) The image acquisition unit 131 acquires position information of the i-th point using the reference position information acquired in step S703 and the relative position information acquired in step S706. The position information of the i-th point is absolute position information, for example, (x, y, h).

[0177] (Step S708) The image acquisition unit 131 acquires color information (for example, (R, GB)) of the i-th point in the received image.

[0178] (Step S709) The image acquisition unit 131 acquires point data (e.g., (x, y, h, R, GB)) having the position information acquired in step S707 and the color information acquired in step S708, and accumulates it in a buffer not shown.

[0179] (Step S710) The image acquiring unit 131 increments the counter i by 1. The process returns to step S705.

[0180] (Step S711) The image acquisition unit 131 acquires the point cloud data received in step S301, and accumulates it in a buffer (not shown). Then, the process returns to the upper level process.

[0181] In the flowchart of FIG. 7, point cloud data corresponding to the received image is stored in a buffer (not shown).

[0182] Next, an example of the dynamic object removal process in step S603 will be described with reference to the flowchart in FIG.

[0183] (Step S801) The image acquiring unit 131 assigns 1 to a counter i.

[0184] (Step S802) The image acquisition unit 131 judges whether or not the i-th object is present among one or more objects resulting from the object recognition process performed in step S602. If the i-th object is present, the process proceeds to step S803; if not, the process returns to the upper process.

[0185] (Step S803) The image acquisition unit 131 acquires object data of the i-th object. The object data here is usually point cloud data.

[0186] (Step S804) The image acquisition unit 131 acquires an object name, which is the type of object specified by the object data acquired in step S803. The object name is, for example, "car", "building", or "person".

[0187] Note that the image acquisition unit 131, for example, provides the object data and the learning model to a machine learning prediction processing module, executes the module, and acquires the object name.

[0188] Also, the image acquisition unit 131 may acquire the object name from the object data by, for example, a known image recognition process. The process for acquiring the object name is not limited.

[0189] (Step S805) The image acquisition unit 131 refers to the object management table of the storage unit 11, and acquires the object type from the object name acquired in step S804. The object type is a "dynamic object" or a "static object." Note that the image acquisition unit 131 may acquire the object type from the object data by the above-mentioned machine learning prediction process.

[0190] (Step S806) The image acquisition unit 131 judges whether or not the object type acquired in step S805 is a “dynamic object.” If it is a “dynamic object,” the process proceeds to step S807, and if it is not a “dynamic object,” the process proceeds to step S808.

[0191] (Step S807) The image acquiring unit 131 deletes the object data of the i-th object from the point cloud data acquired in step S601.

[0192] Here, deleting the object data of the i-th object means, for example, rewriting the color information included in each point data constituting the object data of the i-th object to the color information of the data of the area adjacent to the i-th object. For example, if a car running on a road is deleted, the color of the car's area becomes the color of the road.

[0193] (Step S808) The image acquiring unit 131 increments the counter i by 1. The process returns to step S802.

[0194] Note that the image acquisition unit 131 may perform the dynamic object removal process by machine learning instead of the process in Fig. 8. That is, the image acquisition unit 131 may provide a three-dimensional image (e.g., point cloud data) of an area including a dynamic object to a machine learning prediction module, execute the prediction module, and obtain a three-dimensional image from which the dynamic object has been removed.

[0195] Next, an example of the partial object processing in step S604 will be described with reference to the flowchart in FIG.

[0196] (Step S901) The image acquiring unit 131 assigns 1 to a counter i.

[0197] (Step S902) The image acquisition unit 131 judges whether or not the i-th static object exists in the point cloud data after the processing of step S603. If the i-th static object exists, the process proceeds to step S903, and if not, the process proceeds to step S907.

[0198] (Step S903) The image acquisition unit 131 judges whether the object data of the i-th static object is data of the entire object or data of a part of the object. If it is data of the entire object, the process proceeds to step S906, and if it is data of a part of the object, the process proceeds to step S904.

[0199] (Step S904) The image acquisition unit 131 acquires partial data, which is object data of the i-th static object, from the point cloud data after the processing of step S603, and accumulates the partial data in the storage unit 11 in combination with the user identifier.

[0200] (Step S905) The image acquisition unit 131 deletes partial data, which is object data of the i-th static object, from the point cloud data after the process of step S603.

[0201] (Step S906) The image acquiring unit 131 increments the counter i by 1. The process returns to step S902.

[0202] (Step S907) In response to the reception of the image in this step S301, the image acquisition unit 131 determines whether or not the partial object was stored in the storage unit 11 in step S904. If the partial object was stored, the process proceeds to step S908, and if not, the process returns to the upper level process.

[0203] (Step S908) The image acquisition unit 131 judges whether or not the partial objects accumulated in step S904 can constitute the entire data of the object (whether or not the entire data has been collected). Such a judgment is called an object judgment. An example of the object judgment process will be described with reference to the flowchart in FIG. 10.

[0204] (Step S909) If the determination result in step S908 is that the entire data of the object can be constructed, the image obtaining unit 131 proceeds to step S910, and if the determination result is that the entire data cannot be constructed, the image obtaining unit 131 returns to the upper process.

[0205] (Step S910) The image acquisition unit 131 acquires object data, which is the entire data of the object, and accumulates the object data in a buffer (not shown). Then, the process returns to the upper level process.

[0206] In the flowchart of Fig. 9, the processes from steps S907 to S910 are preferably performed for each partial object accumulated in step S904. That is, in the flowchart of Fig. 9, the processes from steps S907 to S910 may be performed for two or more partial objects.

[0207] Next, an example of the object determination process in step S908 will be described with reference to the flowchart in FIG.

[0208] (Step S1001) The image acquisition unit 131 acquires the most recently acquired partial object in the partial object processing, and temporarily stores it in a buffer (not shown).

[0209] (Step S1002) Image acquisition unit 131 uses position information of one or more partial objects stored in storage unit 11 and position information of the partial objects acquired in step S1001 to sort one or more partial objects stored in storage unit 11 in order of proximity to the partial object acquired in step S1001.

[0210] (Step S1003) The image acquiring unit 131 assigns 1 to a counter i.

[0211] (Step S1004) The image acquisition unit 131 judges whether or not the i-th partial object exists in the storage unit 11. If the i-th partial object exists, the process proceeds to step S1005, and if not, the process proceeds to step S1013.

[0212] (Step S1005) The image acquisition unit 131 judges whether or not the area indicated by the position information of the point cloud data temporarily stored in a buffer (not shown) overlaps or touches the area indicated by the position information of the point cloud data of the i-th partial object. If there is overlap or they touch, the process proceeds to step S1006, and if there is no overlap or they do not touch (they are separated), the process proceeds to step S1013.

[0213] (Step S1006) The image acquiring unit 131 assigns 1 to a counter j.

[0214] (Step S1007) The image acquisition unit 131 judges whether or not the j-th point data exists in the i-th partial object data. If the j-th point data exists, the process proceeds to step S1008, and if not, the process proceeds to step S1012.

[0215] (Step S1008) The image obtaining unit 131 obtains position information of the j-th point data in the i-th partial object data.

[0216] (Step S1009) The image acquisition unit 131 determines whether or not point data having position information that matches the position information acquired in step S1008 is temporarily stored in a buffer (not shown). If the point data is temporarily stored in a buffer (not shown), the process proceeds to step S1011. If the point data is not temporarily stored, the process proceeds to step S1010.

[0217] (Step S1010) The image acquisition unit 131 temporarily stores the j-th point data in the i-th partial object data in a buffer (not shown).

[0218] (Step S1011) The image acquiring unit 131 increments the counter j by 1. The process returns to step S1007.

[0219] (Step S1012) The image acquiring unit 131 increments the counter i by 1. The process returns to step S1004.

[0220] (Step S1013) The image acquisition unit 131 judges whether the object data of the point cloud data temporarily stored in a buffer (not shown) is data of a completed object. If it is data of a completed object, the process proceeds to step S1014, and if it is not data of a completed object, the process proceeds to step S1015.

[0221] The image acquisition unit 131, for example, provides object data of point cloud data temporarily stored in a buffer (not shown) or an image acquired from the object data and a learning model to a machine learning prediction module, executes the module, and acquires information indicating whether the data is of a completed object. Such a learning model is data acquired by providing two or more teacher data having object data or an image of point cloud data and information indicating whether the data is of a completed object to a machine learning learning module and executing the module.

[0222] The image acquisition unit 131 may, for example, extract a contour from object data of point cloud data temporarily stored in a buffer not shown, and determine whether or not the data is of a completed object based on the contour data of the object.

[0223] (Step S1014) The image obtaining unit 131 assigns "Complete" to the variable "determination result", and returns to the upper process.

[0224] (Step S1015) The image acquiring unit 131 assigns "incomplete" to the variable "determination result", and returns to the upper process.

[0225] Next, a first example of the image storage process in step S407 will be described with reference to the flowchart in FIG.

[0226] (Step S1101) The update unit 135 acquires a user identifier corresponding to a received image.

[0227] (Step S1102) The update unit 135 accumulates the three-dimensional image (usually point cloud data) acquired in step S406 in the map storage unit 111 in association with the user identifier acquired in step S1101.

[0228] (Step S1103) The update unit 135 determines whether or not object data of a completed object exists in the partial object process of step S604. If object data of a completed object exists, the process proceeds to step S1104, and if not, the process returns to the upper level process.

[0229] (Step S1104) The update unit 135 acquires a user identifier paired with the data of each of the two or more partial objects that are the basis of the object data of the completed object. Next, the update unit 135 accumulates the object data of the completed object in the map storage unit 111 in association with the two or more acquired user identifiers.

[0230] (Step S1105) Processing unit 13 deletes the data of each of the two or more partial objects that formed the basis of the object data of the completed object from storage unit 11. Then, the process returns to the upper level process.

[0231] Next, a second example of the image storage process in step S407 will be described with reference to the flowchart in FIG.

[0232] (Step S1201) The update unit 135 acquires a user identifier corresponding to a received image.

[0233] (Step S1202) The update unit 135 assigns 1 to the counter i.

[0234] (Step S1203) The update unit 135 judges whether or not the i-th point data exists in a buffer (not shown). If the i-th point data exists, the process proceeds to step S1204, and if not, the process returns to the upper process.

[0235] (Step S1204) The update unit 135 acquires the position information of the i-th point data.

[0236] (Step S1205) The update unit 135 searches the urban space map in the map storage unit 111 using the position information acquired in step S1204 as a key.

[0237] (Step S1206) In step S1205, the update unit 135 determines whether point data having the same position information as the position information acquired in step S1204 has been found. If the point data has been found, the process proceeds to step S1207. If the point data has not been found, the process proceeds to step S1208.

[0238] (Step S1207) The update unit 135 overwrites the point data searched for in step S1205 with the i-th point data in association with the user identifier acquired in step S1201.

[0239] (Step S1208) The update unit 135 adds the i-th point data to the map storage unit 111 in association with the user identifier acquired in step S1201.

[0240] (Step S1209) The update unit 135 increments the counter i by 1. The process returns to step S1203.

[0241] Next, a second example of the image acquisition process in step S406 will be described with reference to the flowchart in Fig. 13. The second example of the image acquisition process is a case where data is acquired for each object.

[0242] (Step S1301) The image acquisition unit 131 performs object recognition processing on a received image to detect one or more objects. Note that the processing of recognizing an object from an image is a known technique.

[0243] (Step S1302) The image acquisition unit 131 performs a process of removing one or more dynamic objects from among the one or more objects detected in step S1301 from the received image. An example of such a dynamic object removal process has been described with reference to the flowchart of FIG.

[0244] (Step S1303) The image acquiring unit 131 assigns 1 to a counter i.

[0245] (Step S1304) The image acquisition unit 131 judges whether or not the i-th object exists among the objects detected in step S1301. If the i-th object exists, the process proceeds to step S1305, and if not, the process returns to the upper process.

[0246] (Step S1305) The image obtaining unit 131 obtains object data which is data of the i-th object, and temporarily stores the object data in a buffer (not shown).

[0247] (Step S1306) The image acquiring unit 131 searches the map storage unit 111 for the object data acquired in step S1305.

[0248] (Step S1307) As a result of the search process in step S1306, the image acquisition unit 131 determines whether or not the same data as the object data acquired in step S1305 exists in the map storage unit 111. If the same data exists, the process proceeds to step S1308, and if not, the process proceeds to step S1309.

[0249] Note that even if two pieces of object data have the same position information, if the two pieces of object data have different color information, the two object data are not the same data.

[0250] (Step S1308) The image obtaining unit 131 deletes the object data obtained in step S1305 from a buffer (not shown).

[0251] (Step S1309) The image acquiring unit 131 increments the counter i by 1. The process returns to step S1304.

[0252] Next, a third example of the image storage process in step S407, which corresponds to the second example of the image acquisition process in Fig. 13, will be described with reference to the flowchart in Fig. 14. The third example of the image storage process is a case where storage is performed for each object data.

[0253] (Step S1401) The update unit 135 acquires a user identifier corresponding to a received image.

[0254] (Step S1402) The update unit 135 assigns 1 to the counter i.

[0255] (Step S1403) Update unit 135 judges whether or not the i-th object data exists. If the i-th object data exists, the process proceeds to step S1404, and if not, the process returns to the upper level process.

[0256] (Step S1404) The update unit 135 accumulates the object data in the map storage unit 111 in association with the user identifier acquired in step S1401.

[0257] (Step S1405) The update unit 135 increments the counter i by 1. The process returns to step S1403.

[0258] Next, a first example of the output image acquisition process in step S311 will be described with reference to the flowchart in Fig. 15. The first example of the output image acquisition process is a case in which the urban space map is made up of a set of point cloud data.

[0259] (Step S1501) The output image obtaining unit 136 obtains area information contained in the received viewing instruction.

[0260] (Step S1502) The output image acquisition unit 136 acquires, from the point data stored in the map storage unit 111, two or more point data items each having position information within the area specified by the area information.

[0261] (Step S1503) The output image obtaining unit 136 assigns 1 to a counter i.

[0262] (Step S1504) The output image acquisition unit 136 determines whether or not the i-th point data exists among the point data acquired in step S1502. If the i-th point data exists, the process proceeds to step S1505, and if not, the process returns to the upper process.

[0263] (Step S1505) The output image obtaining unit 136 places a point of the color indicated by the color information contained in the i-th point data at the position indicated by the position information contained in the i-th point data in a virtual three-dimensional space.

[0264] (Step S1506) The output image obtaining unit 136 increments the counter i by 1. The process returns to step S1504.

[0265] Next, a second example of the output image acquisition process in step S311 will be described with reference to the flowchart in Fig. 16. The second example of the output image acquisition process is a case where the urban space map is made up of a set of object data. Note that the object data may be graph data or point cloud data, and the data structure is not important.

[0266] (Step S1601) The output image obtaining unit 136 obtains area information contained in the received viewing instruction.

[0267] (Step S1602) The output image obtaining unit 136 assigns 1 to a counter i.

[0268] (Step S1603) The output image acquisition unit 136 judges whether or not the i-th object data exists in the map storage unit 111. If the i-th object data exists, the process proceeds to step S1604, and if not, the process returns to the upper level process.

[0269] (Step S1604) The output image obtaining unit 136 obtains one or more pieces of position information corresponding to the i-th object data.

[0270] (Step S1605) Using the area information acquired in step S1601 and the one or more pieces of position information acquired in step S1604, output image acquisition unit 136 determines whether or not the i-th object data, whose position is identified by the one or more pieces of position information acquired in step S1604, exists in the area identified by the area information acquired in step S1601. If the i-th object data exists in the area, the process proceeds to step S1606, and if not, the process proceeds to step S1608.

[0271] (Step S1606) The output image acquisition unit 136 acquires the i-th object data.

[0272] (Step S1607) The output image acquisition unit 136 places the i-th object data in a virtual three-dimensional space at a position specified by one or more pieces of position information paired with the object data.

[0273] (Step S1608) The output image obtaining unit 136 increments the counter i by 1. The process returns to step S1603.

[0274] Next, an example of the operation of the terminal device 2 will be described with reference to the flowchart of FIG.

[0275] (Step S1701) The terminal reception unit 22 judges whether or not a shooting instruction has been received. If a shooting instruction has been received, the process proceeds to step S1702, and if a shooting instruction has not been received, the process proceeds to step S1703.

[0276] (Step S1702) The terminal processing unit 23 senses the urban space, acquires an image, and temporarily stores it in the terminal storage unit 21. The process returns to step S1701.

[0277] (Step S1703) The device processing unit 23 judges whether or not to transmit an image. If an image is to be transmitted, the process proceeds to step S1704, and if an image is not to be transmitted, the process proceeds to step S1711. Note that an image is transmitted when, for example, a transmission instruction is accepted or an image is stored in the terminal storage unit 21.

[0278] (Step S1704) The device processing unit 23 acquires the user identifier from the device storage unit 21.

[0279] (Step S1705) The device processing unit 23 acquires one or more image attribute values. The one or more image attribute values ​​are, for example, time information that is the current time. Note that it is preferable that the current time is acquired immediately before or immediately after step S1702.

[0280] (Step S1706) The device processing unit 23 acquires one or more environmental attribute values. The one or more environmental attribute values ​​are, for example, the weather and temperature at the shooting location. Note that it is preferable that the one or more environmental attribute values ​​are acquired immediately before or after step S1702.

[0281] (Step S1707) The device processing unit 23 composes information to be transmitted. This information includes, for example, an image temporarily stored in the device storage unit 21, a user identifier, one or more image attribute values, and one or more environment attribute values. This information may also include one or more user attribute values.

[0282] (Step S1708) The terminal transmitting unit 24 transmits to the map updating device 1 the information configured in step S1707, which includes the image.

[0283] (Step S1709) Terminal receiving unit 25 determines whether or not result information has been received from map update device 1. If result information has been received, the process proceeds to step S1710, and if not, the process returns to step S1709.

[0284] (Step S1710) The device processing unit 23 composes result information to be output using the information received in step S1709. The terminal output unit 26 outputs the result information. Return to step S1701.

[0285] (Step S1711) The terminal reception unit 22 judges whether or not a viewing instruction has been received. If a viewing instruction has been received, the process proceeds to step S1712, and if a shooting instruction has not been received, the process proceeds to step S1715.

[0286] (Step S1712) The terminal processing unit 23 composes a viewing instruction to be transmitted. The viewing instruction includes, for example, area information and a user identifier. The terminal transmitting unit 24 transmits the viewing instruction to the map updating device 1.

[0287] (Step S1713) The terminal receiving unit 25 judges whether or not an output image has been received from the map updating device 1. If an output image has been received, the process proceeds to step S1714, and if not, the process returns to step S1713.

[0288] (Step S1714) The device processing unit 23 composes an output image to be output using the information received in step S1711. The terminal output unit 26 outputs the output image. The process returns to step S1701.

[0289] (Step S1715) The terminal reception unit 22 judges whether or not the user information has been received. If the user information has been received, the process proceeds to step S1716, and if not, the process returns to step S1701.

[0290] (Step S1716) The terminal processing unit 23 composes user information to be transmitted. The terminal transmitting unit 24 transmits the user information to the map update device 1.

[0291] (Step S1717) The terminal receiving unit 25 judges whether or not a user identifier has been received from the map update device 1. If a user identifier has been received, the process proceeds to step S1718, and if not, the process returns to step S1717.

[0292] (Step S1718) The terminal output unit 26 outputs the user identifier received in step S1717. Return to step S1701. Note that the terminal output unit 26 accumulates the user identifier in the terminal storage unit 21, for example.

[0293] In the flowchart of FIG. 17, the process ends when the power is turned off or an interrupt occurs to end the process.

[0294] A specific example of the operation of the map system A in this embodiment will now be described.

[0295] It is assumed that an image condition constituting an update condition, "size of area covered by image >= first threshold value," a user condition constituting an update condition, "being a registered user AND rank >= 1," and an environmental condition constituting an update condition, "weather = sunny OR weather = cloudy," are stored in the storage unit 11 of the map update device 1. It is assumed that the condition determination unit 134 determines to accumulate the received image when all the update conditions in the storage unit 11 are satisfied.

[0296] It is also assumed that user information of a large number of users is registered in the user management unit 112. The user information here includes a user identifier and a rank.

[0297] In this situation, it is assumed that a registered user A takes a distance image in an urban space with the terminal device 2 and transmits it to the map update device 1. It is assumed that the distance image is associated with an image attribute value of "photographed date and time = 10:38:19 on July 25, 2023", a user identifier of "A", and an environmental attribute value of "weather = sunny".

[0298] Next, the image receiving unit 121 of the map updating device 1 receives the distance image, etc. Next, the processing unit 13 stores the three-dimensional image in association with the user identifier "A" by the process described using the flowchart in Fig. 3. Here, it is assumed that the three-dimensional image is point cloud data.

[0299] That is, first, the condition determination unit 134 obtains the size of the area covered by the distance image (W1), and determines that the image condition "W1>=first threshold" is satisfied. The condition determination unit 134 also obtains the rank "1" paired with the user identifier "A" from the user management unit 112, and determines that the user information satisfies the user condition "registered user AND rank>=1". Furthermore, the condition determination unit 134 obtains the environmental attribute value "weather=sunny" paired with the distance image, and determines that the environmental condition "weather=sunny OR weather=cloudy" is satisfied.

[0300] Next, the image acquisition unit 131 acquires point cloud data from the received distance image. Next, the image acquisition unit 131 performs object recognition processing on the point cloud data to detect one or more object data that are point cloud data. Next, the image acquisition unit 131 detects a dynamic object from the one or more object data by the above-mentioned algorithm. Next, the image acquisition unit 131 deletes one or more dynamic objects from the point cloud data. Also, the image acquisition unit 131 detects a partial object from the point cloud data and deletes the partial object from the point cloud data. Note that here, the image acquisition unit 131 may accumulate partial objects by the above-mentioned algorithm, and acquire object data of one object when one object is completed using two or more partial objects.

[0301] Next, the update unit 135 stores the three-dimensional image, which is the point cloud data, in association with the user identifier "A". Next, the processing unit 13 acquires result information indicating that the update has been performed. The result transmission unit 141 transmits the result information to the terminal device 2 of the user A.

[0302] Next, the terminal device 2 of the user A receives and outputs the result information.

[0303] Also, a registered user B senses an urban space using a terminal device 2 having a LiDAR and acquires a LiDAR image. Then, the terminal device 2 of the user B transmits the image to the map update device 1. The image is associated with an image attribute value of "photographed date and time = 12:11:34 on July 27, 2023", a user identifier "B", and an environmental attribute value of "weather = cloudy".

[0304] Next, the image receiving unit 121 of the map updating device 1 receives the image, etc. Next, the processing unit 13 stores the three-dimensional image in association with the user identifier "B" by the process described using the flowchart in Fig. 3. Here, it is assumed that the three-dimensional image is point cloud data.

[0305] That is, the condition determination unit 134 acquires the size of the area covered by the image (W2) and determines that the image condition is satisfied. The condition determination unit 134 also acquires the rank "2" paired with the user identifier "B" from the user management unit 112 and determines that the user information satisfies the user condition. Furthermore, the condition determination unit 134 acquires the environmental attribute value "weather=cloudy" paired with the image and determines that the environmental condition is satisfied.

[0306] Next, the image acquisition unit 131 acquires point cloud data from the received LiDAR image. Next, the image acquisition unit 131 performs object recognition processing on the point cloud data to detect one or more object data that are point cloud data. Next, the image acquisition unit 131 detects a dynamic object from the one or more object data by the above-mentioned algorithm. Next, the image acquisition unit 131 deletes the one or more dynamic objects from the point cloud data. In addition, the image acquisition unit 131 detects a partial object from the point cloud data, and deletes the partial object from the point cloud data.

[0307] Next, the update unit 135 stores the three-dimensional image, which is the point cloud data, in association with the user identifier "B". Next, the processing unit 13 acquires result information indicating "updated". The result transmission unit 141 transmits the result information to the terminal device 2 of the user B.

[0308] Next, the terminal device 2 of user B receives and outputs the result information.

[0309] Next, assume that user A takes a distance image in an urban space with the terminal device 2 and transmits it to the map update device 1. Assume that the distance image is associated with an image attribute value of "photographed date and time = 15:11:20 on July 31, 2023", a user identifier "A", and an environmental attribute value of "weather = sunny".

[0310] Next, the image receiving unit 121 of the map updating device 1 receives the distance image, etc. Next, the processing unit 13 stores the three-dimensional image in association with the user identifier "A" by the process described using the flowchart in Fig. 3. Here, it is assumed that the three-dimensional image is point cloud data.

[0311] That is, the condition determination unit 134 obtains the size of the area covered by the distance image (W3) and determines that the image condition "W3>=first threshold" is satisfied. The condition determination unit 134 also obtains the rank "2" paired with the user identifier "A" from the user management unit 112 and determines that the user information satisfies the user condition. Furthermore, the condition determination unit 134 obtains the environmental attribute value "weather=sunny" paired with the distance image and determines that the environmental condition is satisfied.

[0312] Next, the image acquisition unit 131 acquires point cloud data from the received distance image. Next, the image acquisition unit 131 performs object recognition processing on the point cloud data to detect one or more object data that are point cloud data. Next, the image acquisition unit 131 detects a dynamic object from among the one or more object data by the above-mentioned algorithm. Next, the image acquisition unit 131 deletes the one or more dynamic objects from the point cloud data. Furthermore, the image acquisition unit 131 detects a partial object from the point cloud data, and deletes the partial object from the point cloud data.

[0313] Next, the update unit 135 stores the three-dimensional image, which is the point cloud data, in association with the user identifier "A". At this time, the update unit 135 determines that the area of ​​the LiDAR image transmitted by user B is included in the area of ​​the distance image transmitted by user A on July 31, 2023, and that the point cloud data acquired from the LiDAR image and the point cloud data acquired from the distance image of the same area are not identical, and overwrites the point cloud data based on the LiDAR image transmitted by user B with the point cloud data based on the distance image transmitted by user A on July 31, 2023. Next, the processing unit 13 acquires result information indicating that "updated". The result transmission unit 141 transmits the result information to the terminal device 2 of user A.

[0314] Next, the terminal device 2 of the user A receives and outputs the result information.

[0315] Also, a registered user C senses an urban space using a terminal device 2 having a LiDAR and acquires a LiDAR image. Then, the terminal device 2 of the user C transmits the image to the map update device 1. The image is associated with an image attribute value of "shooting date and time = 14:05:31 on August 1, 2023", a user identifier "C", and an environmental attribute value of "weather = rain".

[0316] Next, the image receiving unit 121 of the map update device 1 receives the image and the like. Next, the condition determining unit 134 obtains the size (W4) of the area covered by the image, and determines that the image condition is satisfied. Also, the condition determining unit 134 obtains the rank "1" paired with the user identifier "C" from the user management unit 112, and determines that the user information satisfies the user condition. Furthermore, the condition determining unit 134 obtains the environmental attribute value "weather=rain" paired with the image, and determines that the environmental condition is not satisfied. In other words, the condition determining unit 134 determines not to update the urban space map based on the image transmitted by user C. Next, the processing unit 13 obtains result information indicating that "no update was performed." The result transmitting unit 141 transmits the result information to the terminal device 2 of user C.

[0317] Next, the terminal device 2 of the user C receives and outputs the result information.

[0318] As described above, according to this embodiment, the map update device 1 updates the urban space map using images received from the terminal devices 2 of multiple users, thereby providing a platform where multiple users cooperate to update the urban space map.

[0319] Furthermore, according to this embodiment, by storing three-dimensional images in association with user identifiers, it is possible to manage users who have contributed to updating the urban space map.

[0320] Furthermore, according to this embodiment, an appropriate urban space map can be constructed by updating the urban space map using only images that meet the update conditions.

[0321] Furthermore, according to this embodiment, even if images with different data structures are received, three-dimensional images with a unified data structure can be accumulated to update the urban space map, thereby providing a platform that makes it easy to update the urban space map with the cooperation of a large number of users.

[0322] The process in this embodiment may be realized by software. This software may be distributed by software download or the like. This software may be recorded on a recording medium such as a CD-ROM and distributed. This also applies to other embodiments in this specification. The software for realizing the map update device 1 in this embodiment is a program as follows. That is, this program is a program for making a computer that can access a map storage unit in which an urban space map, which is a collection of three-dimensional images corresponding to location information, which represents an urban space, is stored, function as an update unit that performs an update process to update, using the three-dimensional image acquired by the image acquisition unit, a three-dimensional image in the urban space map in the map storage unit, which corresponds to the location information of an area that matches the area indicated by the location information corresponding to the image that was the basis of the three-dimensional image acquired by the image acquisition unit.

[0323] 18 shows the appearance of a computer that executes the programs described in this specification to realize the map update device 1 and the like of the various embodiments described above. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 18 is an overview of this computer system 300, and FIG. 19 is a block diagram of the system 300.

[0324] In FIG. 18, a computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0325] 19, computer 301 includes, in addition to CD-ROM drive 3012, MPU 3013, bus 3014 connected to CD-ROM drive 3012 etc., ROM 3015 for storing programs such as a boot-up program, RAM 3016 connected to MPU 3013 for temporarily storing instructions of application programs and providing temporary storage space, and hard disk 3017 for storing application programs, system programs, and data. Although not shown here, computer 301 may further include a network card for providing connection to a LAN.

[0326] A program for causing computer system 300 to execute functions such as those of the map updating device 1 of the above-described embodiment may be stored on CD-ROM 3101, inserted into CD-ROM drive 3012, and then transferred to hard disk 3017. Alternatively, the program may be transmitted to computer 301 via a network (not shown) and stored on hard disk 3017. The program is loaded into RAM 3016 when executed. The program may also be loaded directly from CD-ROM 3101 or the network.

[0327] The program does not necessarily include an operating system (OS) or a third party program that causes the computer 301 to execute the functions of the map updating device 1 of the above-described embodiment. The program only needs to include instructions for calling appropriate functions (modules) in a controlled manner to achieve a desired result. How the computer system 300 operates is well known, and a detailed description will be omitted.

[0328] In addition, in the above program, the steps of transmitting information and receiving information do not include processing performed by hardware, such as processing performed by a modem or interface card in the transmitting step (processing that is performed only by hardware).

[0329] The program may be executed by a single computer or a plurality of computers. That is, the program may be executed by a centralized processing or a distributed processing.

[0330] Furthermore, in each of the above embodiments, it goes without saying that two or more communication means present in one device may be physically realized by one medium.

[0331] In each of the above embodiments, each process may be realized by centralized processing in a single device, or may be realized by distributed processing in a plurality of devices.

[0332] The present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these modifications are also included within the scope of the present invention. [Industrial Applicability]

[0333] As described above, the map updating device 1 according to the present invention has an effect of being a platform whereby a plurality of users cooperate to update an urban space map, and is useful as a server or the like that manages an urban space map. [Explanation of symbols]

[0334] A Map System 1 Map update device 2 Terminal Device 11 Storage area 12 Receiving section 13 Processing section 14 Transmitter 21 Terminal storage section 22 Terminal Reception 23 Terminal Processing Section 24 Terminal transmitter 25 Terminal receiving section 26 Terminal Output Section 111 Map storage area 112 User Management Department 121 Image receiving unit 122 Instruction receiving unit 131 Image acquisition unit 132 Partial storage section 133 Object Judgment Department 134 Condition judgment section 135 Update Department 136 Output image acquisition unit 141 Result transmission unit 142 Output image transmission unit

Claims

1. an image receiving unit that receives images of an urban space, the images corresponding to location information, from the terminal devices of two or more users; an image acquisition unit that acquires a three-dimensional image of all or a part of the urban space by using all or a part of the image received by the image receiving unit; a map updating device comprising: an update unit that performs an update process to update, with the three-dimensional image acquired by the image acquisition unit, a three-dimensional image in an urban space map in a map storage unit in which an urban space map is stored, the urban space map being a collection of three-dimensional images corresponding to location information, the three-dimensional image in the urban space map corresponding to location information of an area that matches an area indicated by the location information corresponding to the image that is the basis of the three-dimensional image acquired by the image acquisition unit.

2. The image receiving unit includes: receiving the image associated with a user identifier that identifies a user and location information from each of the two or more terminal devices; The update unit is 2. The map updating device according to claim 1, wherein the three-dimensional image acquired by the image acquisition unit is stored in association with the user identifier.

3. The image acquisition unit includes: The map updating device according to claim 1 , wherein the image receiving unit acquires the three-dimensional image excluding dynamic objects contained in the image received.

4. The image acquisition unit includes: The map updating device according to claim 1 , wherein, when the three-dimensional image acquired by the image acquisition unit includes only a portion of a static object, the three-dimensional image that does not include the static object is acquired.

5. a partial storage unit that stores a partial object, which is information on a portion of a static object, when only a portion of the static object is included in the three-dimensional image acquired by the image acquisition unit; an object determination unit that determines whether or not the entire static object can be configured using a partial object stored by the partial storage unit and a partial object that is information of a part of a static object in information based on the image received by the image receiving unit after the partial object is stored; The image acquisition unit includes: When the object determination unit determines that the entire image can be constructed, a three-dimensional image of the static object is obtained using the partial objects stored by the partial storage unit and the partial objects in the information based on the image received by the image receiving unit; The update unit is 5. The map updating device according to claim 4, wherein an updating process is performed in which the three-dimensional image of the static object acquired by the image acquisition unit is stored in the map storage unit.

6. The image receiving unit includes: receiving an image having two or more different data structures; The image acquisition unit includes: The map updating device according to claim 1 , further comprising: a conversion unit configured to convert the image received by the image receiving unit into a three-dimensional image according to the data structure of the image.

7. The urban space map is data expressing points constituting a three-dimensional urban space, and is point cloud data which is a collection of point data having position information and color information; The image acquisition unit includes: The map updating device according to claim 1 , wherein the three-dimensional image having point cloud data is acquired using all or a part of the image received by the image receiving unit.

8. a condition determining unit that determines, when the image receiving unit receives the image, whether or not an update condition for updating the urban space map using the image is satisfied; The update unit is The map updating device according to claim 1 , wherein the update process is performed only when the condition determining unit determines that the update condition is satisfied.

9. 9. The map update device according to claim 8, wherein the update conditions include any one of an image condition related to one or more image attribute values ​​that are attribute values ​​of the image received by the image receiving unit, a user condition related to one or more user attribute values ​​of a user who transmitted the image, or an environmental condition related to one or more environmental attribute values ​​related to an environment in which the image was acquired.

10. 9. The map updating device according to claim 8, further comprising a result transmitting unit configured to transmit, to the terminal device that transmitted the image received by the image receiving unit, result information regarding whether or not the updating unit has performed an update.

11. The image acquisition unit includes: acquiring one or more object data based on the image received by the image receiving unit; The update unit is 2. The map updating device according to claim 1, wherein the updating process is performed by storing the one or more object data acquired by the image acquisition unit in the map storage unit.

12. The image acquisition unit includes: acquiring one or more unit data, which is data based on the image received by the image receiving unit and is data within a range of units to be updated; The update unit is 2 . The map updating device according to claim 1 , wherein the updating process is performed by storing the one or more unit data acquired by the image acquisition unit in the map storage unit.

13. A map updating method implemented by a map storage unit in which an urban space map, which is three-dimensional information expressing an urban space and is a collection of three-dimensional images corresponding to location information, is stored, an image receiving unit, an image acquiring unit, and an updating unit, an image receiving step in which the image receiving unit receives images captured of an urban space and associated with location information from the terminal devices of two or more users; an image acquisition step in which the image acquisition unit acquires a three-dimensional image of all or a part of the urban space using all or a part of the image received by the image receiving unit; and an updating step of performing an updating process to update, with the three-dimensional image acquired by the image acquisition unit, the three-dimensional image in the urban space map in the map storage unit, which corresponds to location information of an area that matches the area indicated by the location information corresponding to the image that is the basis of the three-dimensional image acquired by the image acquisition unit.

14. A computer that can access a map storage unit that stores an urban space map, which is three-dimensional information representing an urban space and is a collection of three-dimensional images corresponding to location information, an image receiving unit that receives images of an urban space, the images corresponding to location information, from the terminal devices of two or more users; an image acquisition unit that acquires a three-dimensional image of all or a part of the urban space by using all or a part of the image received by the image receiving unit; A program for causing the map storage unit to function as an update unit that performs an update process to update a three-dimensional image in the urban space map that corresponds to location information of an area that matches the area indicated by the location information corresponding to the image that was the basis of the three-dimensional image acquired by the image acquisition unit, using the three-dimensional image acquired by the image acquisition unit.

Images