Map update device, map update method, and program

Abstract

To provide a platform that provides reward to rights holders who have provided images when multiple people cooperate to update urban space maps representing three-dimensional urban spaces, the platform having never existed so far.SOLUTION: A map update device 3 includes: an image receiving unit 121 that receives images obtained by photographing an urban space from a terminal device 4; an image acquiring unit 131 that acquires three-dimensional images of all or part of the urban space using all or some of the received images; an update unit 135 that performs an update process of accumulating the three-dimensional images acquired by the image acquiring unit 131 in association with user identifiers; and a reward processing unit 332 that performs a reward process, which is a process for giving rewards to users. The map update device can provide a platform for rewarding right holders who have provided the images when multiple people cooperate to update the urban space map.SELECTED DRAWING: Figure 18

Description

[Technical field]

[0001] The present invention relates to a map updating device that provides a reward to a user who transmits an image for updating a three-dimensional urban space map. [Background technology]

[0002] In the prior art, there is a 3D map correction device that claims to be capable of correcting low-precision 3D maps (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2010-191066 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, when multiple people work together to update an urban space map that represents a three-dimensional urban space, there was no platform that could provide a reward to the rights holder who provided the image. [Means for solving the problem]

[0005] The map updating device of the first invention is a map updating device comprising: an image receiving unit that receives from a terminal device an image taken of an urban space, the image corresponding to a user identifier and location information; an image acquisition unit that acquires a three-dimensional image of all or part of the urban space using all or part of the image received by the image receiving unit; an update unit that performs an update process in which the three-dimensional image in the urban space map in the map storage unit in which the urban space map, which is a collection of three-dimensional images corresponding to a user identifier and location information, is stored in association with a user identifier corresponding to the image received by the image receiving unit, and the three-dimensional image corresponds to location information of an area that matches an area indicated by location information corresponding to an image that is the basis of the three-dimensional image acquired by the image acquisition unit; and a reward processing unit that performs a reward process that is a process of giving a reward to a user identified by the user identifier.

[0006] With this configuration, when multiple people work together to update an urban space map, a platform can be provided that rewards the rights holders who provide images.

[0007] In addition, compared to the first invention, the map update device of the second invention further includes a reward determination unit that determines whether the image received by the image receiving unit, or the three-dimensional image stored by the update unit, or the user identified by the user identifier satisfies the reward conditions, and the reward processing unit performs reward processing only when the reward determination unit determines that the reward conditions are satisfied.

[0008] With this configuration, when a three-dimensional image of a portion of an urban space map that is updated collaboratively by multiple people is used, a platform can be provided that rewards the rights holder who provided the image that is the basis for the three-dimensional image.

[0009] In addition, the map updating device of the third invention, compared to the first or second invention, further includes an instruction receiving unit that receives an output instruction for the urban space map from a terminal device, an output image acquisition unit that acquires a three-dimensional image of the area corresponding to the output instruction from the map storage unit and acquires an output image using the three-dimensional image, and an output image sending unit that transmits the output image acquired by the output image acquisition unit to the terminal device that sent the output instruction, and the reward processing unit is a map updating device that performs a usage reward processing, which is a process of giving a reward to a rights holder identified by a user identifier corresponding to the three-dimensional image that is the basis of the output image.

[0010] With this configuration, a platform can be provided that rewards the rights holder who provided the image that is the basis for a three-dimensional image for the use of the three-dimensional image.

[0011] In addition, the map update device of the fourth invention is a map update device in which, compared to the first invention, the reward processing unit performs an update reward processing which, when the update processing is performed, gives a reward to a rights holder identified by a user identifier corresponding to the image received by the image receiving unit.

[0012] With this configuration, when a part of an urban space map that is updated collaboratively by multiple people is updated, a platform can be provided that rewards the rights holder who provided the image for the update.

[0013] In addition, the map update device of the fifth invention is a map update device in which, compared to any one of the first to fourth inventions, the reward processing unit performs a transmission reward processing to give a reward to a user identified by a user identifier corresponding to an image received by the image receiving unit, even if the update processing is not performed.

[0014] With this configuration, it is possible to provide a platform that rewards users who send images even if the urban space map is not updated using the images they send, which can be expected to encourage users to send images actively.

[0015] In addition, the map update device of the sixth invention is a map update device according to any one of the first to fifth inventions, in which the reward processing unit performs reward processing to award rewards to rights holders identified by each of the two or more user identifiers when two or more user identifiers correspond to a three-dimensional image.

[0016] With this configuration, when two or more user identifiers correspond to one three-dimensional image, a reward can be given to each of the right holders identified by the two or more user identifiers.

[0017] In addition, the map update device of the seventh invention is a map update device in which, with respect to any one of the first to sixth inventions, an old user identifier, which is a user identifier corresponding to the three-dimensional image before the update process is performed, is also associated with the three-dimensional image to be subjected to the update process.

[0018] With this configuration, the history of the rights holders of the three-dimensional images can be managed.

[0019] In addition, the map update device of the eighth invention is a map update device in which, compared to any one of the first to seventh inventions, the reward processing unit acquires one or more image attribute values ​​of an image received by the image receiving unit or a three-dimensional image used in the update processing of the update unit, determines a reward amount according to the one or more image attribute values, and performs reward processing according to the reward amount.

[0020] With this configuration, it is possible to give the right holder an appropriate remuneration according to the image attribute value.

[0021] Furthermore, the map updating device of the ninth invention is a map updating device according to any one of the first to eighth inventions, in which the reward processing unit determines different reward amounts when the update unit performs an append process to accumulate a three-dimensional image of a new area and when the update unit performs a change process to accumulate a new three-dimensional image of an area where a three-dimensional image already exists, and performs reward processing according to the reward amounts.

[0022] With this configuration, it is possible to give the right holder an appropriate remuneration according to the content of the update process of the urban space map.

[0023] In addition, the map update device of the tenth invention is a map update device according to any one of the first to ninth inventions, further comprising a result transmission unit that, when the update unit has performed the update process, transmits result information indicating that the update process has been performed to the terminal device that transmitted the image.

[0024] With this configuration, the user who transmitted the image can be notified that he or she has contributed to updating the urban space map.

[0025] In addition, the map update device of the eleventh invention is a map update device according to any one of the first to tenth inventions, further comprising a result sending unit which, when the update unit has performed an update process, sends result information to a user identified by an old user identifier corresponding to the updated three-dimensional image, indicating that the user is no longer a rights holder as a result of the update process being performed.

[0026] With this configuration, a user who is no longer a right holder eligible to receive a reward can be notified of the fact that he or she is no longer a right holder. Effect of the Invention

[0027] According to the map updating device of the present invention, when a plurality of people cooperate to update an urban space map, a platform can be provided for giving a reward to a right holder who provided an image. [Brief description of the drawings]

[0028] [Figure 1] Conceptual diagram of map system A in embodiment 1. [Diagram 2] Block diagram of map system A [Diagram 3] A flowchart illustrating an example of the operation of the map update device 1. [Figure 4] A flowchart illustrating an example of the update process. [Diagram 5] A flowchart illustrating an example of the condition determination process. [Figure 6]11 is a flowchart illustrating a first example of the image acquisition process. [Figure 7] Flowchart for explaining an example of a point cloud data acquisition process [Figure 8] A flowchart illustrating an example of the dynamic object removal process. [Figure 9] A flowchart illustrating an example of the partial object process. [Figure 10] A flowchart illustrating an example of the object determination process. [Figure 11] 1 is a flowchart illustrating a first example of the image storage process. [Figure 12] 11 is a flowchart illustrating a second example of the image storage process. [Figure 13] 11 is a flowchart illustrating a second example of the image acquisition process. [Figure 14] 11 is a flowchart illustrating a third example of the image storage process. [Figure 15] 11 is a flowchart illustrating a first example of the output image acquisition process. [Figure 16] 11 is a flowchart illustrating a second example of the output image acquisition process. [Figure 17] A flowchart for explaining an example of the operation of the terminal device 2 [Figure 18] Block diagram of map system B in embodiment 2 [Figure 19] A flowchart for explaining an example of the operation of the map update device 3. [Figure 20] A flowchart illustrating an example of a process for notifying a former right holder [Figure 21] A flowchart illustrating an example of the remuneration determination process [Figure 22] A flowchart illustrating an example of the compensation process. [Figure 23] A flowchart illustrating an example of the reward amount determination process [Figure 24] FIG. 13 shows a first example of the map management table. [Diagram 25]FIG. 2 is a diagram showing a second example of the map management table. [Figure 26] FIG. 11 is a diagram showing a third example of the map management table. [Figure 27] The user management table [Figure 28] A diagram showing the reward amount management table [Figure 29] Overview of the computer system according to the above embodiment. [Diagram 30] Block diagram of the computer system DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] Hereinafter, an embodiment of a map update device and the like will be described with reference to the drawings. In the embodiments, components with the same reference numerals perform similar operations, so that repeated description may be omitted.

[0030] (Embodiment 1) In this embodiment, a map updating device that updates a part of a three-dimensional urban space map using images received from terminal devices of two or more users will be described.

[0031] In addition, in this embodiment, a map updating device that associates an updated three-dimensional image with the user identifier of the user who transmitted the original image will be described.

[0032] In addition, in this embodiment, a map updating device is described which updates the urban space map by excluding dynamic objects in the received images.

[0033] In addition, in this embodiment, a map updating device will be described in which, when a received image contains only a part of an object, information on the object is not used to update the urban space map.

[0034] In addition, in this embodiment, a map update device is described that, when a received image contains only a portion of an object, stores information about that portion of the object, and stores information about the object once data for the entire object is available.

[0035] Also, in this embodiment, a map updating device will be described that can update a part of a three-dimensional urban space map using images received with two or more different data structures.

[0036] In addition, in this embodiment, a map updating device is described which converts a received image into an intermediate data structure (e.g., point cloud data), partially updates the urban space map of the intermediate data structure, and constructs and accumulates an object-based urban space map from the updated intermediate data.

[0037] Furthermore, in this embodiment, a map update device is described which, when an image is received, determines whether or not the image satisfies an update condition, and uses the image to update a portion of the urban space map only if the image satisfies the update condition.

[0038] In this specification, information X being associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is not important. Information X and information Y may be linked, may exist in the same buffer, information X may be included in information Y, or information Y may be included in information X, etc.

[0039] 1 is a conceptual diagram of a map system A according to the present embodiment. The map system A includes a map update device 1 and two or more terminal devices 2.

[0040] The map updating device 1 is a device that updates a three-dimensional urban space map using an image received from the terminal device 2. The map updating device 1 is, for example, a cloud server or an ASP server, but the type is not important.

[0041] The terminal device 2 is a terminal used by a user. The terminal device 2 may be, for example, a personal computer, a smartphone, or a tablet terminal, but the type of the terminal device 2 is not important.

[0042] The map updating device 1 and two or more terminal devices 2 are capable of communicating with each other via a network such as the Internet.

[0043] 2 is a block diagram of the map system A in this embodiment. The map update device 1 includes a storage unit 11, a receiving unit 12, a processing unit 13, and a transmitting unit .

[0044] The storage unit 11 includes a map storage unit 111 and a user management unit 112. The reception unit 12 includes an image reception unit 121 and an instruction reception unit 122. The processing unit 13 includes an image acquisition unit 131, a partial accumulation unit 132, an object determination unit 133, a condition determination unit 134, an update unit 135, and an output image acquisition unit 136. The transmission unit 14 includes a result transmission unit 141 and an output image transmission unit 142.

[0045] The terminal device 2 includes a terminal storage unit 21, a terminal reception unit 22, a terminal processing unit 23, a terminal transmission unit 24, a terminal reception unit 25, and a terminal output unit .

[0046] Various types of information are stored in the storage unit 11 constituting the map updating device 1. The various types of information are, for example, an urban space map (to be described later), user information (to be described later), one or more update conditions, or an object management table.

[0047] The update condition is a condition for determining whether or not to update the urban space map using the received image. The update condition may be a condition for determining not to update the urban space map, or a condition for determining to update the urban space map. The update condition is, for example, one or a combination of two or more of an image condition, a user condition, and an environmental condition. It goes without saying that the update condition of the storage unit 11 may be embedded in the program.

[0048] The image condition is a condition related to one or more image attribute values, which are attribute values ​​of the image received by the image receiving unit 121. The image attribute values ​​are, for example, time information, image size, area information specifying the area covered by the image, and the size of the area covered by the image. The time information is information specifying the time when the image was taken. The time information is, for example, the shooting date and time, and the shooting date. The image condition is, for example, "the time when the image was taken is newer than the time when the target area in the urban space map was taken," "the size of the image is equal to or larger than the threshold," and "the size of the area covered by the image is equal to or larger than the threshold." Note that the target area is the same area as the area covered by the image received by the image receiving unit 121.

[0049] The user condition is a condition related to one or more user attribute values ​​of the user who transmitted the image. For example, the user condition is that "the rank of the user is equal to or higher than a threshold value."

[0050] The environmental condition is a condition related to one or more environmental attribute values ​​related to the environment in which the image was captured. The environmental attribute values ​​are, for example, information indicating the weather, temperature, humidity, and whether the image is captured outdoors or indoors.

[0051] An object management table is a table that manages one or two types of objects, dynamic objects and static objects. An object management table is, for example, a table that manages the correspondence between "object name" and "object type." Dynamic objects are, for example, people and moving objects. Static objects are, for example, buildings, bridges and roads. Records in the object management table are, for example, "people: dynamic object," "moving object: dynamic object," "buildings: static object," "bridges: static object," and "roads: static object."

[0052] An urban space map is stored in the map storage unit 111. Urban space maps of two or more types of data structures may be stored in the map storage unit 111. The two types of data structures are, for example, point cloud data (described later) and object sets (described later).

[0053] An urban space map is three-dimensional information that represents urban space. It can be said that an urban space map is a collection of three-dimensional images that correspond to location information. A three-dimensional image is one or more pieces of point cloud data, or a collection of one or more objects. An urban space map is, for example, point cloud data or a collection of objects. The collection of three-dimensional images can be a single file, and the data structure, etc., does not matter.

[0054] Point cloud data is a collection of point data. The point data has position information. The point data preferably has color information. The point data preferably has both position information and color information.

[0055] Position information is information that specifies a three-dimensional position in space. Position information can be expressed as (x, y, h). For example, position information is (latitude, longitude, altitude). It is preferable that position information is information that specifies an absolute position. However, position information may also be information that specifies a relative position from a reference point.

[0056] Color information is information that specifies a color. Usually, it is (R,G,B), but it can also be (C,M,Y,K), etc.

[0057] The point data has, for example, a structure of (x,y,h,R,G,B), where "x" is the latitude, "y" is the longitude, "h" is the altitude, "R" is the red value, "G" is the green value, and "B" is the blue value.

[0058] One or more point data in the point cloud data constituting the urban space map may be managed for each object. In other words, each of the two or more point data may be associated with an object identifier. Note that an object identifier is information for identifying an object. An object identifier is, for example, an object ID or an object name.

[0059] An object set is a set of two or more object data. Object data is information that represents an object. The object data may be, for example, data that represents an object in a graph or point cloud data, but the structure is not important. The object data may correspond to, for example, an object identifier. Data that represents an object in a graph may be, for example, a "3D Scene Graph" (see web page URL "https: / / github.com / StanfordVL / 3DSceneGraph").

[0060] An object is an object that may exist in an urban space. An object is usually a physical object. An object can be a static object or a dynamic object. A static object is usually an object that does not move autonomously. Static objects are, for example, buildings and roads. A dynamic object is usually an elephant that moves autonomously. Dynamic objects are, for example, moving objects, people and animals. Moving objects are, for example, cars, motorcycles, trucks, trains and airplanes.

[0061] The user management unit 112 stores one or more pieces of user information. The user information is information related to a user. The user information usually includes a user identifier. The user information includes, for example, one or more user attribute values.

[0062] A user identifier is information that identifies a user. The user identifier is, for example, a user ID, a telephone number, or an email address. The user attribute values ​​are, for example, age, gender, the number of image transmissions, and rank. The number of image transmissions is the number of times the user has transmitted images to the map update device 1. The rank is the rank of the user. For example, the rank is higher the more images are transmitted. For example, the rank is higher the more times the urban space map is updated using images transmitted by the user. For example, the rank is lower the more times the user has transmitted images but not updated the urban space map using those images.

[0063] The receiving unit 12 receives various types of information and instructions from the terminal device 2. The various types of information and instructions are, for example, images, which will be described later, and viewing instructions, which will be described later.

[0064] The image receiving unit 121 receives images from two or more users' terminal devices 2. The images received by the image receiving unit 121 are information obtained by sensing urban space. It can also be said that the images received by the image receiving unit 121 are information obtained by scanning urban space. The images may also be called sensing information.

[0065] It is preferable that the image receiving unit 121 receives an image associated with a user identifier and location information from each of two or more terminal devices. It is preferable that the image receiving unit 121 receives an image having two or more different data structures. In other words, it is preferable that the data structures of the images transmitted from each of two or more terminal devices 2 may be different. It is preferable that one or more image attribute values ​​are associated with an image. The one or more image attribute values ​​are, for example, time information. It is preferable that the time information is information that specifies the time, and in this case, for example, the shooting date and time, and the shooting date. It is preferable that one or more environmental attribute values ​​are associated with an image. The one or more environmental attribute values ​​are, for example, information indicating the weather when the image was taken, the temperature when the image was taken, and whether the location where the image was taken is indoors or outdoors.

[0066] The position information associated with the image received by the image receiving unit 121 is absolute position information, for example, (latitude, longitude, altitude). Moreover, such position information is absolute position information of a reference point in the image. The user identifier associated with the image received by the image receiving unit 121 is usually a user identifier that identifies the user who transmitted the image.

[0067] The image here is an image captured of an urban space. The image is, for example, a distance image, a LiDAR image, point cloud data, or graph data (for example, "3D Scene Graph"). A LiDAR image is an image acquired by a LiDAR. However, the data structure of the image received by the image receiving unit 121 does not matter. Note that an image associated with location information may be an image in which location information is included in the information constituting the image.

[0068] The number of images received by the image receiving unit 121 may be two or more. In other words, the image received by the image receiving unit 121 may be a moving image having two or more still images.

[0069] The instruction receiving unit 122 receives various instructions from the terminal device 2. The various instructions are, for example, a viewing instruction.

[0070] The viewing instruction is an instruction to view the urban space map. The viewing instruction includes, for example, area information that specifies an area of ​​the urban space. The area information is, for example, one or more pieces of location information.

[0071] The processing unit 13 performs various types of processing, such as processing performed by an image acquisition unit 131, a partial storage unit 132, an object determination unit 133, a condition determination unit 134, an update unit 135, and an output image acquisition unit 136.

[0072] The image acquisition unit 131 uses all or part of the images received by the image reception unit 121 to acquire a three-dimensional image of all or part of the urban space.

[0073] A three-dimensional image is three-dimensional information that represents an urban space, and is, for example, point cloud data or an object-based image.

[0074] The image acquiring unit 131 acquires a three-dimensional image by, for example, converting all or a part of the image received by the image receiving unit 121. However, the three-dimensional image may be the image received by the image receiving unit 121.

[0075] The image acquisition unit 131 converts, for example, a distance image, a LiDAR image, or graph data received by the image receiving unit 121, and acquires a three-dimensional image that is point cloud data. Note that the process of converting a distance image, a LiDAR image, or graph data and acquiring point cloud data is a publicly known technique. The image acquisition unit 131 acquires, for example, position information (absolute position information) of each point constituting the point cloud data from absolute position information corresponding to the distance image received by the image receiving unit 121 and relative position information of each point in the distance image, acquires color information of each point in the distance image, and acquires point cloud data that is a collection of point data having the position information and color information of each point.

[0076] The image acquisition unit 131 acquires, for example, a three-dimensional image excluding dynamic objects included in the image received by the image receiving unit 121. The image acquisition unit 131 performs object recognition processing on all or part of the image received by the image receiving unit 121 or all or part of an image (for example, a three-dimensional image) obtained by converting all or part of the image, and detects one or more objects. Then, the image acquisition unit 131 acquires, for example, type information indicating the type of each of the one or more objects. Next, the image acquisition unit 131 refers to, for example, an object management table, determines object data of one or more objects corresponding to the type information that are dynamic objects, and acquires a three-dimensional image excluding the determined one or more object data. Note that the image acquisition unit 131 may provide data of one or more objects acquired by the object recognition processing and a learning model to a machine learning prediction module, execute the prediction module, acquire a classification result of whether the object is a dynamic object or a static object, and acquire a three-dimensional image excluding one or more object data that are dynamic objects. In such a case, the learning model is information acquired by performing a machine learning learning process using two or more pieces of teacher data having object data and classification information as to whether the object is a dynamic object or a static object. The learning model may be called a learner, a classifier, a classification model, or the like. The machine learning algorithm here may be deep learning, random forest, decision tree, SVM, or the like. For the machine learning, various machine learning functions such as the TensorFlow (registered trademark) library and the random forest module of the R language, or various existing libraries, may be used.

[0077] For example, when only a part of a static object is included in an image received by the image receiving unit 121, the image acquiring unit 131 acquires a three-dimensional image that does not include the static object. That is, for example, the image acquiring unit 131 performs an object recognition process on all or a part of the image received by the image receiving unit 121 or all or a part of an image (for example, a three-dimensional image) obtained by converting all or a part of the image, and detects one or more objects. Then, the image acquiring unit 131 acquires, for example, type information indicating the type of each of the one or more objects. Next, the image acquiring unit 131 refers to, for example, an object management table, and acquires a three-dimensional image that leaves only object data of one or more objects that correspond to the type information that is a static object. Next, the image acquiring unit 131 determines whether or not the one or more static objects are missing a part. Then, the image acquiring unit 131 acquires a three-dimensional image that does not include a static object that is missing a part.

[0078] The image acquisition unit 131, for example, provides object data of a static object and a learning model to a machine learning prediction module, and obtains a judgment result as to whether the object data is data of a complete object or data of an object missing a part. The learning model is information obtained by providing two or more teacher data having object data and information indicating whether the object data is data of a complete object or data of an object missing a part to a machine learning learning module. The machine learning algorithm here may be deep learning, random forest, decision tree, SVM, or the like.

[0079] Note that the image acquisition unit 131 may acquire, for example, by the above-mentioned processing, an image excluding dynamic objects or a three-dimensional image which is a static object in a three-dimensional image and does not include a static object with a portion missing.

[0080] Furthermore, it is preferable that the partial storage unit 132, which will be described later, temporarily stores object data of a static object having a missing portion in the storage unit 11 or a buffer (not shown).

[0081] If the object determination unit 133 described later determines that a whole can be constructed using the partial object stored by the partial storage unit 132 and a part of a static object in the image-based information received by the image receiving unit 121 after the partial object was stored, the image acquisition unit 131 may acquire a three-dimensional image of the static object using the partial object stored by the partial storage unit 132 and the partial object in the image-based information received by the image receiving unit 121.

[0082] In addition, the information based on the image received by the image receiving unit 121 may be an image received by the image receiving unit 121, a three-dimensional image acquired by the image acquisition unit 131, or intermediate data constructed from the image received by the image receiving unit 121.

[0083] It is preferable that the image acquiring unit 131 acquires three-dimensional images of the same data structure by performing different conversion processes according to the data structures of two or more images received by the image receiving unit 121. In other words, it is preferable that the image acquiring unit 131 can acquire a three-dimensional image from images having two or more different types of data structures.

[0084] The image acquisition unit 131 acquires a three-dimensional image having point cloud data, for example, by using all or part of the image received by the image receiving unit 121.

[0085] For example, the image acquisition unit 131 acquires one or more object data based on the image received by the image receiving unit 121. For example, the image acquisition unit 131 performs an object recognition process on the image received by the image receiving unit 121 to detect one or more objects, and acquires object data for each of the one or more objects. Note that the object data is, for example, point cloud data or graph data.

[0086] The image acquisition unit 131 acquires one or more unit data, which is, for example, data based on an image received by the image receiving unit 121 and is data of a unit range to be updated. The unit data is information of a range of a certain size in urban space. The unit data is, for example, information of an area with a fixed width (w), depth (d), and height (h). The unit data is, for example, information of a cubic area with a fixed width (w), depth (d), and height (h). The unit data is a three-dimensional image.

[0087] The partial accumulation unit 132 accumulates a partial object, which is information on a portion of the static object, when only a portion of the static object is included in the three-dimensional image acquired by the image acquisition unit 131. The partial accumulation unit 132 accumulates the partial object, for example, in the storage unit 11 or a buffer not shown.

[0088] The object determination unit 133 determines whether or not the entire static object can be constructed using the partial object stored by the partial storage unit 132 and a part of the static object in the image-based information received by the image receiving unit 121 after the partial object was stored. The image-based information received by the image receiving unit 121 is, for example, an image received by the image receiving unit 121 or a three-dimensional image acquired by the image acquiring unit 131.

[0089] The object determination unit 133 acquires object data having, for example, partial objects stored by the partial storage unit 132 and partial objects based on images received by the image receiving unit 121. At this time, the object determination unit 133 combines two or more pieces of point data having matching position information among two or more pieces of partial object data into one piece of point data. In such a case, the object determination unit 133 prioritizes, for example, the point data of the partial object data associated with the latest shooting date and time. Next, the object determination unit 133 provides the object data acquired using two or more pieces of partial object data to a machine learning prediction module, and acquires a determination result as to whether the object data is data of a complete object or data of an object missing a part.

[0090] When the image receiving unit 121 receives an image, the condition determining unit 134 determines whether or not the update condition is satisfied.

[0091] The condition determination unit 134, for example, obtains one or more image attribute values ​​corresponding to an image received by the image receiving unit 121, and determines whether or not the one or more image attribute values ​​match an image condition. The condition determination unit 134, for example, obtains one or more image attribute values ​​from an image received by the image receiving unit 121, and determines whether or not the one or more image attribute values ​​match an image condition.

[0092] The condition determination unit 134, for example, acquires one or more user attribute values ​​associated with an image received by the image receiving unit 121, and determines whether or not the one or more user attribute values ​​match a user condition. The condition determination unit 134, for example, acquires a user identifier associated with an image received by the image receiving unit 121, acquires one or more user attribute values ​​paired with the user identifier from the storage unit 11, and determines whether or not the one or more user attribute values ​​match a user condition.

[0093] The condition determination unit 134, for example, acquires one or more environmental attribute values ​​associated with an image received by the image receiving unit 121, and determines whether or not the one or more environmental attribute values ​​match environmental conditions. The condition determination unit 134, for example, accesses a server (not shown) using location information associated with an image received by the image receiving unit 121 as a key, acquires one or more environmental attribute values ​​(for example, weather) from the server, and determines whether or not the one or more environmental attribute values ​​match environmental conditions.

[0094] The update unit 135 updates the three-dimensional image in the urban space map corresponding to the position information of an area that matches the area indicated by the position information corresponding to the image that is the basis of the three-dimensional image acquired by the image acquisition unit 131, with the three-dimensional image acquired by the image acquisition unit 131. Such processing is called an update processing. It is preferable that the update unit 135 stores the three-dimensional image acquired by the image acquisition unit 131 in the map storage unit 111 in association with the time information corresponding to the image. The time information corresponding to the image is, for example, the shooting date and time, or the shooting date.

[0095] The update unit 135 is, for example, a three-dimensional image of an area corresponding to the three-dimensional image acquired by the image acquisition unit 131, and overwrites the three-dimensional image in the map storage unit 111 with the three-dimensional image acquired by the image acquisition unit 131. Such overwriting is an example of an update process.

[0096] For example, the update unit 135 simply accumulates the three-dimensional image acquired by the image acquisition unit 131 in the map storage unit 111. For example, the update unit 135 accumulates the three-dimensional image acquired by the image acquisition unit 131 in the map storage unit 111 in association with time information. In this case, the old three-dimensional image of the area corresponding to the three-dimensional image acquired by the image acquisition unit 131 also remains in the map storage unit 111, but this may be considered to be an update process. In other words, the process by the update unit 135 to simply accumulate the three-dimensional image acquired by the image acquisition unit 131 in the map storage unit 111 may also be considered to be an update process. This is because, even in such a case, the information used to display the urban space is a three-dimensional image associated with the latest time information.

[0097] The update unit 135 performs an update process of, for example, storing in the map storage unit 111 the three-dimensional images of one or more static objects acquired by the image acquisition unit 131 .

[0098] It is preferable that the update unit 135 performs the update process only when the condition determination unit 134 determines that the update condition is satisfied.

[0099] The update unit 135 performs an update process of, for example, accumulating one or more pieces of object data acquired by the image acquisition unit 131 in the map storage unit 111. The update unit 135 performs an update process of, for example, accumulating one or more pieces of unit data acquired by the image acquisition unit 131 in the map storage unit 111. In other words, the update unit 135 may update any unit of information at one time. The update unit 135 may update each piece of object data, each piece of unit data, or the entire three-dimensional image acquired by the image acquisition unit 131.

[0100] The update unit 135 acquires result information. The result information is information on whether or not an update has been performed. For example, when an update process has been performed, the update unit 135 acquires result information indicating that "update has been performed." For example, when an update process has not been performed even though an image has been received, the update unit 135 acquires result information indicating that "update has not been performed."

[0101] The update unit 135 typically stores the acquired three-dimensional image in association with a user identifier associated with the received image.

[0102] The output image acquisition unit 136 constructs an urban space to be output using the urban space map in the map storage unit 111. The output image acquisition unit 136, for example, acquires area information included in the viewing instruction received by the instruction receiving unit 122, acquires a three-dimensional image of the area specified by the area information, and constructs an output image using the three-dimensional image. The output image acquisition unit 136, for example, acquires area information included in the viewing instruction received by the instruction receiving unit 122, and constructs an output image by arranging points of color information held by each point data of the three-dimensional image of the area specified by the area information at positions in virtual space that are positions of position information held by each point data.

[0103] The transmission unit 14 transmits various types of information, such as result information and an output image.

[0104] The result transmission unit 141 transmits the result information acquired by the update unit 135 to the terminal device 2 that transmitted the image received by the image reception unit 121 .

[0105] The output image transmission unit 142 transmits the output image acquired by the output image acquisition unit 136 to the terminal device 2. The terminal device 2 is usually the device that has sent the viewing instruction.

[0106] Various types of information are stored in the terminal storage unit 21 constituting the terminal device 2. The various types of information are, for example, a user identifier. Note that the user identifier in the terminal storage unit 21 may be information input by a user and temporarily stored.

[0107] The terminal reception unit 22 receives various information, instructions, etc. The various information, instructions, etc. are, for example, a viewing instruction, a photographing instruction, a transmission instruction, a user identifier, and one or more user attribute values. Note that a photographing instruction is an instruction to photograph an image. A transmission instruction is an instruction to transmit the photographed image.

[0108] The means for inputting various information and instructions may be anything, such as a touch panel, a keyboard, a mouse, a menu screen, a microphone, etc.

[0109] The terminal processing unit 23 performs various types of processing. For example, the various types of processing are processing for converting received information, instructions, etc. into information or instructions with a structure to be transmitted. For example, the various types of processing are processing for converting received information into information with a structure to be output.

[0110] The device processing unit 23 captures images and acquires images. That is, the device processing unit 23 includes, for example, a camera or LiDAR. Such images are, for example, distance images, LiDAR images, point cloud data, and graph data (for example, "3D Scene Graph").

[0111] The device processing unit 23 acquires, for example, one or more image attribute values. The device processing unit 23 acquires, for example, time information that is the shooting date and time.

[0112] The device processing unit 23 acquires, for example, one or more environmental attribute values. The device processing unit 23 acquires, for example, information indicating the weather, the temperature, and whether the device is outdoors or indoors.

[0113] The terminal transmission unit 24 transmits various information, instructions, etc. to the map update device 1. The various information, instructions, etc. are, for example, a viewing instruction, an image, a user identifier, one or more image attribute values, one or more user attribute values, and one or more environmental attribute values.

[0114] The terminal transmission unit 24 transmits the image acquired by the device processing unit 23 to the map update device 1, for example, in association with a user identifier. The terminal transmission unit 24 transmits the image acquired by the device processing unit 23 to the map update device 1, for example, in association with one or more types of attribute values ​​among one or more image attribute values, one or more user attribute values, and one or more environmental attribute values.

[0115] The terminal receiving unit 25 receives various types of information, such as an output image and result information.

[0116] The terminal output unit 26 outputs various types of information, such as an output image and result information.

[0117] Here, output is a concept that includes display on a display, projection using a projector, printing on a printer, sound output, transmission to an external device, storage on a recording medium, and handing over the processing results to other processing devices or other programs, etc.

[0118] The storage unit 11, the map storage unit 111, the user management unit 112, and the terminal storage unit 21 are preferably non-volatile recording media, but may also be realized as volatile recording media.

[0119] There is no restriction on the process by which information is stored in the storage unit 11, etc. For example, information may be stored in the storage unit 11, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 11, etc., or information inputted via an input device may be stored in the storage unit 11, etc.

[0120] The receiving unit 12, the image receiving unit 121, the instruction receiving unit 122, and the terminal receiving unit 25 are usually realized by wireless or wired communication means, but may be realized by means for receiving broadcasts.

[0121] The processing unit 13, image acquisition unit 131, partial storage unit 132, object determination unit 133, condition determination unit 134, update unit 135, output image acquisition unit 136, and device processing unit 23 can usually be realized by a processor, memory, etc. The processing procedure of the processing unit 13, etc. is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit). The processor may be a CPU, MPU, GPU, etc., and the type is not important.

[0122] The transmission unit 14, the result transmission unit 141, and the output image transmission unit 142 are usually realized by wireless or wired communication means, but may also be realized by broadcasting means.

[0123] The terminal reception unit 22 can be realized by a device driver for an input means such as a touch panel or a keyboard, or control software for a menu screen.

[0124] The terminal output unit 26 may be considered to include, or may not include, an output device such as a display, a speaker, etc. The terminal output unit 26 may be realized by driver software for an output device, or a combination of driver software for an output device and an output device, etc.

[0125] Next, an example of the operation of the map updating device 1 will be described with reference to the flowchart of FIG.

[0126] (Step S301) The image receiving unit 121 judges whether or not an image has been received from the terminal device 2. If an image has been received, the process proceeds to step S302, and if an image has not been received, the process proceeds to step S311. Note that a user identifier is associated with the image.

[0127] (Step S302) The processing unit 13 acquires a user identifier associated with the image received in step S301.

[0128] (Step S303) The processing unit 13 acquires the image received in step S301.

[0129] (Step S304) The processing unit 13 performs an update process using the image acquired in step S303. An example of the update process will be described with reference to the flowchart of FIG.

[0130] (Step S305) The processing unit 13 judges whether the urban space map has been updated as a result of the processing in step S304. If the urban space map has been updated, the process proceeds to step S306, and if the urban space map has not been updated, the process proceeds to step S307.

[0131] (Step S306) The processing unit 13 acquires result information indicating that the update has been performed. Then, the process proceeds to step S308.

[0132] (Step S307) The processing unit 13 acquires result information indicating that "no update was performed."

[0133] (Step S308) The result transmitting unit 141 transmits the result information acquired in step S307 to the terminal device 2 that transmitted the image.

[0134] (Step S309) The processing unit 13 uses the result information to change the rank paired with the user identifier. Then, the process returns to step S301.

[0135] For example, when the processing unit 13 obtains result information indicating that the information has been updated, the processing unit 13 increases the rank, and when the processing unit 13 obtains result information indicating that the information has not been updated, the processing unit 13 does not change the rank or decreases the rank.

[0136] (Step S310) The instruction receiving unit 122 judges whether or not a viewing instruction has been received from the terminal device 2. If a viewing instruction has been received, the process proceeds to step S311, and if not, the process proceeds to step S313.

[0137] (Step S311) The output image acquisition unit 136 acquires an output image. An example of such output image acquisition processing will be described with reference to the flowcharts of FIGS.

[0138] (Step S312) The output image transmission unit 142 transmits the output image acquired in step S311 to the terminal device 2 that has transmitted the viewing instruction. Return to step S301.

[0139] (Step S313) The receiving unit 12 judges whether or not user information has been received from the terminal device 2. If user information has been received, the process proceeds to step S314, and if not, the process returns to step S301. Note that the user information received here does not include a user identifier, but includes one or more user attribute values.

[0140] (Step S314) The processing unit 13 generates a unique user identifier. The processing unit 13 configures user information including the user identifier and the received one or more user attribute values. The processing unit 13 accumulates the user information in the user management unit 112.

[0141] (Step S314) The transmission unit 14 transmits the user identifier to the terminal device 2. The process returns to step S301.

[0142] In the flowchart of FIG. 3, the process ends when the power is turned off or an interrupt occurs to end the process.

[0143] Next, an example of the update process in step S304 will be described with reference to the flowchart in FIG.

[0144] (Step S401) The condition decision unit 134 acquires one or more image attribute values ​​of the image received in step S301.

[0145] (Step S402) The condition decision unit 134 acquires from the user management unit 112 one or more user attribute values ​​that are paired with the user identifier associated with the image received in step S301.

[0146] (Step S403) The condition decision unit 134 acquires one or more environmental attribute values ​​associated with the image received in step S301.

[0147] (Step S404) The condition determination unit 134 determines whether to update the urban space map using the received image using one or more types of information from one or more image attribute values, one or more user attribute values, and one or more environment attribute values. An example of such a condition determination process will be described with reference to the flowchart in FIG. 5.

[0148] (Step S405) If the result of the decision in step S404 is "update", the process proceeds to step S406, and if the result is "do not update", the process returns to the upper process.

[0149] (Step S406) The image acquisition unit 131 performs a process of acquiring a three-dimensional image using the received image. An example of such an image acquisition process will be described with reference to the flowcharts of FIGS.

[0150] (Step S407) The update unit 135 performs a storage process of the three-dimensional image using the three-dimensional image acquired in step S406. The process returns to the upper level process. An example of such an image storage process will be described with reference to the flowcharts of FIGS. 11 and 12.

[0151] Next, an example of the condition determination process in step S404 will be described with reference to the flowchart in FIG.

[0152] (Step S501) The condition decision unit 134 assigns 1 to a counter i.

[0153] (Step S502) The condition decision unit 134 decides whether or not the i-th update condition exists in the storage unit 11. If the i-th update condition exists, the process proceeds to step S503, and if not, the process proceeds to step S508.

[0154] (Step S503) The condition decision unit 134 acquires the i-th update condition.

[0155] (Step S504) The condition decision unit 134 acquires one or more attribute values ​​to be used for deciding whether the i-th update condition is satisfied. Each of the one or more attribute values ​​is an image attribute value, a user attribute value, or an environment attribute value.

[0156] (Step S505) The condition determination unit 134 determines whether the i-th update condition is met by using one or more attribute values ​​acquired in step S504. If the i-th update condition is met, the process proceeds to step S506, and if the i-th update condition is not met, the process proceeds to step S507.

[0157] (Step S506) The condition decision unit 134 increments the counter i by 1. The process returns to step S502.

[0158] (Step S507) The condition decision unit 134 assigns "do not update" to the variable "decision result." Then, the process returns to the upper level process.

[0159] (Step S508) The condition decision unit 134 assigns "update" to the variable "decision result." Then, the process returns to the upper level process.

[0160] In the flowchart of Fig. 5, the condition determination unit 134 determines not to update when the update condition is not met. However, the update condition may be a condition that determines to update when the update condition is not met, or a condition that determines to update when the update condition is met. The update condition may be a condition related to one attribute value, or a condition related to two or more attribute values. The update condition may be a condition in which two or more conditions are linked by AND, or a condition in which two or more conditions are linked by OR.

[0161] Next, a first example of the image acquisition process in step S406 will be described with reference to the flowchart in FIG.

[0162] (Step S601) The image acquisition unit 131 acquires point cloud data based on a received image. An example of such point cloud data acquisition processing will be described with reference to the flowchart in FIG.

[0163] (Step S602) The image acquisition unit 131 performs object recognition processing on the received image or the point cloud data acquired in step S601 to detect one or more objects. Note that the processing of recognizing objects from images of various data structures is a well-known technique, and therefore detailed description thereof will be omitted.

[0164] (Step S603) The image acquisition unit 131 performs a process of removing one or more dynamic objects among the one or more objects detected in step S602 from the point cloud data acquired in step S601. An example of such dynamic object removal process will be described with reference to the flowchart in FIG.

[0165] (Step S604) The image acquisition unit 131 performs processing on a partial object, which is an object in the point cloud data acquired in step S601 and is a part of the object. The process returns to the upper level processing. An example of such partial object processing will be described with reference to the flowchart in FIG. 9.

[0166] A partial object is an object that is included in the point cloud data, but is not entirely contained in the point cloud data.

[0167] Next, an example of the point cloud data acquisition process in step S601 will be described with reference to the flowchart in FIG.

[0168] (Step S701) The image acquisition unit 131 acquires a data structure identifier that identifies the data structure of the received image. The data structure identifier is, for example, a "range image," "LiDAR," or "point cloud data." The process of acquiring the data structure identifier of an image is a known technique.

[0169] (Step S702) The image acquisition unit 131 judges whether or not the data structure identifier acquired in step S701 is “point cloud data.” If it is “point cloud data,” the process proceeds to step S711, and if it is not “point cloud data,” the process proceeds to step S703.

[0170] (Step S703) The image acquisition unit 131 acquires reference position information, which is position information paired with the received image. The reference position information corresponds to, for example, the received image. The reference position information is absolute position information (usually (latitude, longitude, altitude)).

[0171] (Step S704) The image acquiring unit 131 assigns 1 to a counter i.

[0172] (Step S705) The image acquisition unit 131 determines whether the area of ​​the i-th point exists, which is the area of ​​points in the received image. If the i-th point exists, the process proceeds to step S706, and if the i-th point does not exist, the process returns to the upper process. The i-th point may be considered to be the area of ​​the i-th point.

[0173] (Step S706) The image obtaining unit 131 obtains position information of the i-th point in the received image, which is relative position information with respect to reference position information.

[0174] (Step S707) The image acquisition unit 131 acquires position information of the i-th point using the reference position information acquired in step S703 and the relative position information acquired in step S706. The position information of the i-th point is absolute position information, for example, (x, y, h).

[0175] (Step S708) The image acquisition unit 131 acquires color information (for example, (R, GB)) of the i-th point in the received image.

[0176] (Step S709) The image acquisition unit 131 acquires point data (e.g., (x, y, h, R, GB)) having the position information acquired in step S707 and the color information acquired in step S708, and accumulates it in a buffer not shown.

[0177] (Step S710) The image acquiring unit 131 increments the counter i by 1. The process returns to step S705.

[0178] (Step S711) The image acquisition unit 131 acquires the point cloud data received in step S301, and accumulates it in a buffer (not shown). Then, the process returns to the upper level process.

[0179] In the flowchart of FIG. 7, point cloud data corresponding to the received image is stored in a buffer (not shown).

[0180] Next, an example of the dynamic object removal process in step S603 will be described with reference to the flowchart in FIG.

[0181] (Step S801) The image acquiring unit 131 assigns 1 to a counter i.

[0182] (Step S802) The image acquisition unit 131 judges whether or not the i-th object is present among one or more objects resulting from the object recognition process performed in step S602. If the i-th object is present, the process proceeds to step S803; if not, the process returns to the upper process.

[0183] (Step S803) The image acquisition unit 131 acquires object data of the i-th object. The object data here is usually point cloud data.

[0184] (Step S804) The image acquisition unit 131 acquires an object name, which is the type of object specified by the object data acquired in step S803. The object name is, for example, "car", "building", or "person".

[0185] Note that the image acquisition unit 131, for example, provides the object data and the learning model to a machine learning prediction processing module, executes the module, and acquires the object name.

[0186] Also, the image acquisition unit 131 may acquire the object name from the object data by, for example, a known image recognition process. The process for acquiring the object name is not limited.

[0187] (Step S805) The image acquisition unit 131 refers to the object management table of the storage unit 11, and acquires the object type from the object name acquired in step S804. The object type is a "dynamic object" or a "static object." Note that the image acquisition unit 131 may acquire the object type from the object data by the above-mentioned machine learning prediction process.

[0188] (Step S806) The image acquisition unit 131 judges whether or not the object type acquired in step S805 is a “dynamic object.” If it is a “dynamic object,” the process proceeds to step S807, and if it is not a “dynamic object,” the process proceeds to step S808.

[0189] (Step S807) The image acquiring unit 131 deletes the object data of the i-th object from the point cloud data acquired in step S601.

[0190] Here, deleting the object data of the i-th object means, for example, rewriting the color information included in each point data constituting the object data of the i-th object to the color information of the data of the area adjacent to the i-th object. For example, if a car running on a road is deleted, the color of the car's area becomes the color of the road.

[0191] (Step S808) The image acquiring unit 131 increments the counter i by 1. The process returns to step S802.

[0192] Note that the image acquisition unit 131 may perform the dynamic object removal process by machine learning instead of the process in Fig. 8. That is, the image acquisition unit 131 may provide a three-dimensional image (e.g., point cloud data) of an area including a dynamic object to a machine learning prediction module, execute the prediction module, and obtain a three-dimensional image from which the dynamic object has been removed.

[0193] Next, an example of the partial object processing in step S604 will be described with reference to the flowchart in FIG.

[0194] (Step S901) The image acquiring unit 131 assigns 1 to a counter i.

[0195] (Step S902) The image acquisition unit 131 judges whether or not the i-th static object exists in the point cloud data after the processing of step S603. If the i-th static object exists, the process proceeds to step S903, and if not, the process proceeds to step S907.

[0196] (Step S903) The image acquisition unit 131 judges whether the object data of the i-th static object is data of the entire object or data of a part of the object. If it is data of the entire object, the process proceeds to step S906, and if it is data of a part of the object, the process proceeds to step S904.

[0197] (Step S904) The image acquisition unit 131 acquires partial data, which is object data of the i-th static object, from the point cloud data after the processing of step S603, and accumulates the partial data in the storage unit 11 in combination with the user identifier.

[0198] (Step S905) The image acquisition unit 131 deletes partial data, which is object data of the i-th static object, from the point cloud data after the process of step S603.

[0199] (Step S906) The image acquiring unit 131 increments the counter i by 1. The process returns to step S902.

[0200] (Step S907) In response to the reception of the image in this step S301, the image acquisition unit 131 determines whether or not the partial object was stored in the storage unit 11 in step S904. If the partial object was stored, the process proceeds to step S908, and if not, the process returns to the upper level process.

[0201] (Step S908) The image acquisition unit 131 judges whether or not the partial objects accumulated in step S904 can constitute the entire data of the object (whether or not the entire data has been collected). Such a judgment is called an object judgment. An example of the object judgment process will be described with reference to the flowchart in FIG. 10.

[0202] (Step S909) If the determination result in step S908 is that the entire data of the object can be constructed, the image obtaining unit 131 proceeds to step S910, and if the determination result is that the entire data cannot be constructed, the image obtaining unit 131 returns to the upper process.

[0203] (Step S910) The image acquisition unit 131 acquires object data, which is the entire data of the object, and accumulates the object data in a buffer (not shown). Then, the process returns to the upper level process.

[0204] In the flowchart of Fig. 9, the processes from steps S907 to S910 are preferably performed for each partial object accumulated in step S904. That is, in the flowchart of Fig. 9, the processes from steps S907 to S910 may be performed for two or more partial objects.

[0205] Next, an example of the object determination process in step S908 will be described with reference to the flowchart in FIG.

[0206] (Step S1001) The image acquisition unit 131 acquires the most recently acquired partial object in the partial object processing, and temporarily stores it in a buffer (not shown).

[0207] (Step S1002) Image acquisition unit 131 uses position information of one or more partial objects stored in storage unit 11 and position information of the partial objects acquired in step S1001 to sort one or more partial objects stored in storage unit 11 in order of proximity to the partial object acquired in step S1001.

[0208] (Step S1003) The image acquiring unit 131 assigns 1 to a counter i.

[0209] (Step S1004) The image acquisition unit 131 judges whether or not the i-th partial object exists in the storage unit 11. If the i-th partial object exists, the process proceeds to step S1005, and if not, the process proceeds to step S1013.

[0210] (Step S1005) The image acquisition unit 131 judges whether or not the area indicated by the position information of the point cloud data temporarily stored in a buffer (not shown) overlaps or touches the area indicated by the position information of the point cloud data of the i-th partial object. If there is overlap or they touch, the process proceeds to step S1006, and if there is no overlap or they do not touch (they are separated), the process proceeds to step S1013.

[0211] (Step S1006) The image acquiring unit 131 assigns 1 to a counter j.

[0212] (Step S1007) The image acquisition unit 131 judges whether or not the j-th point data exists in the i-th partial object data. If the j-th point data exists, the process proceeds to step S1008, and if not, the process proceeds to step S1012.

[0213] (Step S1008) The image obtaining unit 131 obtains position information of the j-th point data in the i-th partial object data.

[0214] (Step S1009) The image acquisition unit 131 determines whether or not point data having position information that matches the position information acquired in step S1008 is temporarily stored in a buffer (not shown). If the point data is temporarily stored in a buffer (not shown), the process proceeds to step S1011. If the point data is not temporarily stored, the process proceeds to step S1010.

[0215] (Step S1010) The image acquisition unit 131 temporarily stores the j-th point data in the i-th partial object data in a buffer (not shown).

[0216] (Step S1011) The image acquiring unit 131 increments the counter j by 1. The process returns to step S1007.

[0217] (Step S1012) The image acquiring unit 131 increments the counter i by 1. The process returns to step S1004.

[0218] (Step S1013) The image acquisition unit 131 judges whether the object data of the point cloud data temporarily stored in a buffer (not shown) is data of a completed object. If it is data of a completed object, the process proceeds to step S1014, and if it is not data of a completed object, the process proceeds to step S1015.

[0219] The image acquisition unit 131, for example, provides object data of point cloud data temporarily stored in a buffer (not shown) or an image acquired from the object data and a learning model to a machine learning prediction module, executes the module, and acquires information indicating whether the data is of a completed object. Such a learning model is data acquired by providing two or more teacher data having object data or an image of point cloud data and information indicating whether the data is of a completed object to a machine learning learning module and executing the module.

[0220] The image acquisition unit 131 may, for example, extract a contour from object data of point cloud data temporarily stored in a buffer not shown, and determine whether or not the data is of a completed object based on the contour data of the object.

[0221] (Step S1014) The image obtaining unit 131 assigns "Complete" to the variable "determination result", and returns to the upper process.

[0222] (Step S1015) The image acquiring unit 131 assigns "incomplete" to the variable "determination result", and returns to the upper process.

[0223] Next, a first example of the image storage process in step S407 will be described with reference to the flowchart in FIG.

[0224] (Step S1101) The update unit 135 acquires a user identifier corresponding to a received image.

[0225] (Step S1102) The update unit 135 accumulates the three-dimensional image (usually point cloud data) acquired in step S406 in the map storage unit 111 in association with the user identifier acquired in step S1101.

[0226] (Step S1103) The update unit 135 determines whether or not object data of a completed object exists in the partial object process of step S604. If object data of a completed object exists, the process proceeds to step S1104, and if not, the process returns to the upper level process.

[0227] (Step S1104) The update unit 135 acquires a user identifier paired with the data of each of the two or more partial objects that are the basis of the object data of the completed object. Next, the update unit 135 accumulates the object data of the completed object in the map storage unit 111 in association with the two or more acquired user identifiers.

[0228] (Step S1105) Processing unit 13 deletes the data of each of the two or more partial objects that formed the basis of the object data of the completed object from storage unit 11. Then, the process returns to the upper level process.

[0229] Next, a second example of the image storage process in step S407 will be described with reference to the flowchart in FIG.

[0230] (Step S1201) The update unit 135 acquires a user identifier corresponding to a received image.

[0231] (Step S1202) The update unit 135 assigns 1 to the counter i.

[0232] (Step S1203) The update unit 135 judges whether or not the i-th point data exists in a buffer (not shown). If the i-th point data exists, the process proceeds to step S1204, and if not, the process returns to the upper process.

[0233] (Step S1204) The update unit 135 acquires the position information of the i-th point data.

[0234] (Step S1205) The update unit 135 searches the urban space map in the map storage unit 111 using the position information acquired in step S1204 as a key.

[0235] (Step S1206) In step S1205, the update unit 135 determines whether point data having the same position information as the position information acquired in step S1204 has been found. If the point data has been found, the process proceeds to step S1207. If the point data has not been found, the process proceeds to step S1208.

[0236] (Step S1207) The update unit 135 overwrites the point data searched for in step S1205 with the i-th point data in association with the user identifier acquired in step S1201.

[0237] (Step S1208) The update unit 135 adds the i-th point data to the map storage unit 111 in association with the user identifier acquired in step S1201.

[0238] (Step S1209) The update unit 135 increments the counter i by 1. The process returns to step S1203.

[0239] Next, a second example of the image acquisition process in step S406 will be described with reference to the flowchart in Fig. 13. The second example of the image acquisition process is a case where data is acquired for each object.

[0240] (Step S1301) The image acquisition unit 131 performs object recognition processing on a received image to detect one or more objects. Note that the processing of recognizing an object from an image is a known technique.

[0241] (Step S1302) The image acquisition unit 131 performs a process of removing one or more dynamic objects from among the one or more objects detected in step S1301 from the received image. An example of such a dynamic object removal process has been described with reference to the flowchart of FIG.

[0242] (Step S1303) The image acquiring unit 131 assigns 1 to a counter i.

[0243] (Step S1304) The image acquisition unit 131 judges whether or not the i-th object exists among the objects detected in step S1301. If the i-th object exists, the process proceeds to step S1305, and if not, the process returns to the upper process.

[0244] (Step S1305) The image obtaining unit 131 obtains object data which is data of the i-th object, and temporarily stores the object data in a buffer (not shown).

[0245] (Step S1306) The image acquiring unit 131 searches the map storage unit 111 for the object data acquired in step S1305.

[0246] (Step S1307) As a result of the search process in step S1306, the image acquisition unit 131 determines whether or not the same data as the object data acquired in step S1305 exists in the map storage unit 111. If the same data exists, the process proceeds to step S1308, and if not, the process proceeds to step S1309.

[0247] Note that even if two pieces of object data have the same position information, if the two pieces of object data have different color information, the two object data are not the same data.

[0248] (Step S1308) The image obtaining unit 131 deletes the object data obtained in step S1305 from a buffer (not shown).

[0249] (Step S1309) The image acquiring unit 131 increments the counter i by 1. The process returns to step S1304.

[0250] Next, a third example of the image storage process in step S407, which corresponds to the second example of the image acquisition process in Fig. 13, will be described with reference to the flowchart in Fig. 14. The third example of the image storage process is a case where storage is performed for each object data.

[0251] (Step S1401) The update unit 135 acquires a user identifier corresponding to a received image.

[0252] (Step S1402) The update unit 135 assigns 1 to the counter i.

[0253] (Step S1403) Update unit 135 judges whether or not the i-th object data exists. If the i-th object data exists, the process proceeds to step S1404, and if not, the process returns to the upper level process.

[0254] (Step S1404) The update unit 135 accumulates the object data in the map storage unit 111 in association with the user identifier acquired in step S1401.

[0255] (Step S1405) The update unit 135 increments the counter i by 1. The process returns to step S1403.

[0256] Next, a first example of the output image acquisition process in step S311 will be described with reference to the flowchart in Fig. 15. The first example of the output image acquisition process is a case in which the urban space map is made up of a set of point cloud data.

[0257] (Step S1501) The output image obtaining unit 136 obtains area information contained in the received viewing instruction.

[0258] (Step S1502) The output image acquisition unit 136 acquires, from the point data stored in the map storage unit 111, two or more point data items each having position information within the area specified by the area information.

[0259] (Step S1503) The output image obtaining unit 136 assigns 1 to a counter i.

[0260] (Step S1504) The output image acquisition unit 136 determines whether or not the i-th point data exists among the point data acquired in step S1502. If the i-th point data exists, the process proceeds to step S1505, and if not, the process returns to the upper process.

[0261] (Step S1505) The output image obtaining unit 136 places a point of the color indicated by the color information contained in the i-th point data at the position indicated by the position information contained in the i-th point data in a virtual three-dimensional space.

[0262] (Step S1506) The output image obtaining unit 136 increments the counter i by 1. The process returns to step S1504.

[0263] Next, a second example of the output image acquisition process in step S311 will be described with reference to the flowchart in Fig. 16. The second example of the output image acquisition process is a case where the urban space map is made up of a set of object data. Note that the object data may be graph data or point cloud data, and the data structure is not important.

[0264] (Step S1601) The output image obtaining unit 136 obtains area information contained in the received viewing instruction.

[0265] (Step S1602) The output image obtaining unit 136 assigns 1 to a counter i.

[0266] (Step S1603) The output image acquisition unit 136 judges whether or not the i-th object data exists in the map storage unit 111. If the i-th object data exists, the process proceeds to step S1604, and if not, the process returns to the upper level process.

[0267] (Step S1604) The output image obtaining unit 136 obtains one or more pieces of position information corresponding to the i-th object data.

[0268] (Step S1605) Using the area information acquired in step S1601 and the one or more pieces of position information acquired in step S1604, output image acquisition unit 136 determines whether or not the i-th object data, whose position is identified by the one or more pieces of position information acquired in step S1604, exists in the area identified by the area information acquired in step S1601. If the i-th object data exists in the area, the process proceeds to step S1606, and if not, the process proceeds to step S1608.

[0269] (Step S1606) The output image acquisition unit 136 acquires the i-th object data.

[0270] (Step S1607) The output image acquisition unit 136 places the i-th object data in a virtual three-dimensional space at a position specified by one or more pieces of position information paired with the object data.

[0271] (Step S1608) The output image obtaining unit 136 increments the counter i by 1. The process returns to step S1603.

[0272] Next, an example of the operation of the terminal device 2 will be described with reference to the flowchart of FIG.

[0273] (Step S1701) The terminal reception unit 22 judges whether or not a shooting instruction has been received. If a shooting instruction has been received, the process proceeds to step S1702, and if a shooting instruction has not been received, the process proceeds to step S1703.

[0274] (Step S1702) The terminal processing unit 23 senses the urban space, acquires an image, and temporarily stores it in the terminal storage unit 21. The process returns to step S1701.

[0275] (Step S1703) The device processing unit 23 judges whether or not to transmit an image. If an image is to be transmitted, the process proceeds to step S1704, and if an image is not to be transmitted, the process proceeds to step S1711. Note that an image is transmitted when, for example, a transmission instruction is accepted or an image is stored in the terminal storage unit 21.

[0276] (Step S1704) The device processing unit 23 acquires the user identifier from the device storage unit 21.

[0277] (Step S1705) The device processing unit 23 acquires one or more image attribute values. The one or more image attribute values ​​are, for example, time information that is the current time. Note that it is preferable that the current time is acquired immediately before or immediately after step S1702.

[0278] (Step S1706) The device processing unit 23 acquires one or more environmental attribute values. The one or more environmental attribute values ​​are, for example, the weather and temperature at the shooting location. It is preferable that the one or more environmental attribute values ​​are acquired immediately before or after step S1702.

[0279] (Step S1707) The device processing unit 23 composes information to be transmitted. This information includes, for example, an image temporarily stored in the device storage unit 21, a user identifier, one or more image attribute values, and one or more environment attribute values. This information may also include one or more user attribute values.

[0280] (Step S1708) The terminal transmitting unit 24 transmits to the map updating device 1 the information configured in step S1707, which includes the image.

[0281] (Step S1709) Terminal receiving unit 25 determines whether or not result information has been received from map update device 1. If result information has been received, the process proceeds to step S1710, and if not, the process returns to step S1709.

[0282] (Step S1710) The device processing unit 23 composes result information to be output using the information received in step S1709. The terminal output unit 26 outputs the result information. Return to step S1701.

[0283] (Step S1711) The terminal reception unit 22 judges whether or not a viewing instruction has been received. If a viewing instruction has been received, the process proceeds to step S1712, and if a shooting instruction has not been received, the process proceeds to step S1715.

[0284] (Step S1712) The terminal processing unit 23 composes a viewing instruction to be transmitted. The viewing instruction includes, for example, area information and a user identifier. The terminal transmitting unit 24 transmits the viewing instruction to the map updating device 1.

[0285] (Step S1713) The terminal receiving unit 25 judges whether or not an output image has been received from the map updating device 1. If an output image has been received, the process proceeds to step S1714, and if not, the process returns to step S1713.

[0286] (Step S1714) The device processing unit 23 composes an output image to be output using the information received in step S1711. The terminal output unit 26 outputs the output image. The process returns to step S1701.

[0287] (Step S1715) The terminal reception unit 22 judges whether or not the user information has been received. If the user information has been received, the process proceeds to step S1716, and if not, the process returns to step S1701.

[0288] (Step S1716) The terminal processing unit 23 composes user information to be transmitted. The terminal transmitting unit 24 transmits the user information to the map update device 1.

[0289] (Step S1717) The terminal receiving unit 25 judges whether or not a user identifier has been received from the map update device 1. If a user identifier has been received, the process proceeds to step S1718, and if not, the process returns to step S1717.

[0290] (Step S1718) The terminal output unit 26 outputs the user identifier received in step S1717. Return to step S1701. Note that the terminal output unit 26 accumulates the user identifier in the terminal storage unit 21, for example.

[0291] In the flowchart of FIG. 17, the process ends when the power is turned off or an interrupt occurs to end the process.

[0292] A specific example of the operation of the map system A in this embodiment will now be described.

[0293] It is assumed that an image condition constituting an update condition, "size of area covered by image >= first threshold value," a user condition constituting an update condition, "being a registered user AND rank >= 1," and an environmental condition constituting an update condition, "weather = sunny OR weather = cloudy," are stored in the storage unit 11 of the map update device 1. It is assumed that the condition determination unit 134 determines to accumulate the received image when all the update conditions in the storage unit 11 are satisfied.

[0294] It is also assumed that user information of a large number of users is registered in the user management unit 112. The user information here includes a user identifier and a rank.

[0295] In this situation, it is assumed that a registered user A takes a distance image in an urban space with the terminal device 2 and transmits it to the map update device 1. It is assumed that the distance image is associated with an image attribute value of "photographed date and time = 10:38:19 on July 25, 2023", a user identifier of "A", and an environmental attribute value of "weather = sunny".

[0296] Next, the image receiving unit 121 of the map updating device 1 receives the distance image, etc. Next, the processing unit 13 stores the three-dimensional image in association with the user identifier "A" by the process described using the flowchart in Fig. 3. Here, it is assumed that the three-dimensional image is point cloud data.

[0297] That is, first, the condition determination unit 134 obtains the size of the area covered by the distance image (W1), and determines that the image condition "W1>=first threshold" is satisfied. The condition determination unit 134 also obtains the rank "1" paired with the user identifier "A" from the user management unit 112, and determines that the user information satisfies the user condition "registered user AND rank>=1". Furthermore, the condition determination unit 134 obtains the environmental attribute value "weather=sunny" paired with the distance image, and determines that the environmental condition "weather=sunny OR weather=cloudy" is satisfied.

[0298] Next, the image acquisition unit 131 acquires point cloud data from the received distance image. Next, the image acquisition unit 131 performs object recognition processing on the point cloud data to detect one or more object data that are point cloud data. Next, the image acquisition unit 131 detects a dynamic object from the one or more object data by the above-mentioned algorithm. Next, the image acquisition unit 131 deletes one or more dynamic objects from the point cloud data. Also, the image acquisition unit 131 detects a partial object from the point cloud data and deletes the partial object from the point cloud data. Note that here, the image acquisition unit 131 may accumulate partial objects by the above-mentioned algorithm, and acquire object data of one object when one object is completed using two or more partial objects.

[0299] Next, the update unit 135 stores the three-dimensional image, which is the point cloud data, in association with the user identifier "A". Next, the processing unit 13 acquires result information indicating that the update has been performed. The result transmission unit 141 transmits the result information to the terminal device 2 of the user A.

[0300] Next, the terminal device 2 of the user A receives and outputs the result information.

[0301] Also, a registered user B senses an urban space using a terminal device 2 having a LiDAR and acquires a LiDAR image. Then, the terminal device 2 of the user B transmits the image to the map update device 1. The image is associated with an image attribute value of "photographed date and time = 12:11:34 on July 27, 2023", a user identifier "B", and an environmental attribute value of "weather = cloudy".

[0302] Next, the image receiving unit 121 of the map updating device 1 receives the image, etc. Next, the processing unit 13 stores the three-dimensional image in association with the user identifier "B" by the process described using the flowchart in Fig. 3. Here, it is assumed that the three-dimensional image is point cloud data.

[0303] That is, the condition determination unit 134 acquires the size of the area covered by the image (W2) and determines that the image condition is satisfied. The condition determination unit 134 also acquires the rank "2" paired with the user identifier "B" from the user management unit 112 and determines that the user information satisfies the user condition. Furthermore, the condition determination unit 134 acquires the environmental attribute value "weather=cloudy" paired with the image and determines that the environmental condition is satisfied.

[0304] Next, the image acquisition unit 131 acquires point cloud data from the received LiDAR image. Next, the image acquisition unit 131 performs object recognition processing on the point cloud data to detect one or more object data that are point cloud data. Next, the image acquisition unit 131 detects a dynamic object from the one or more object data by the above-mentioned algorithm. Next, the image acquisition unit 131 deletes the one or more dynamic objects from the point cloud data. In addition, the image acquisition unit 131 detects a partial object from the point cloud data, and deletes the partial object from the point cloud data.

[0305] Next, the update unit 135 stores the three-dimensional image, which is the point cloud data, in association with the user identifier "B". Next, the processing unit 13 acquires result information indicating "updated". The result transmission unit 141 transmits the result information to the terminal device 2 of the user B.

[0306] Next, the terminal device 2 of user B receives and outputs the result information.

[0307] Next, assume that user A takes a distance image in an urban space with the terminal device 2 and transmits it to the map update device 1. Assume that the distance image is associated with an image attribute value of "photographed date and time = 15:11:20 on July 31, 2023", a user identifier "A", and an environmental attribute value of "weather = sunny".

[0308] Next, the image receiving unit 121 of the map updating device 1 receives the distance image, etc. Next, the processing unit 13 stores the three-dimensional image in association with the user identifier "A" by the process described using the flowchart in Fig. 3. Here, it is assumed that the three-dimensional image is point cloud data.

[0309] That is, the condition determination unit 134 obtains the size of the area covered by the distance image (W3) and determines that the image condition "W3>=first threshold" is satisfied. The condition determination unit 134 also obtains the rank "2" paired with the user identifier "A" from the user management unit 112 and determines that the user information satisfies the user condition. Furthermore, the condition determination unit 134 obtains the environmental attribute value "weather=sunny" paired with the distance image and determines that the environmental condition is satisfied.

[0310] Next, the image acquisition unit 131 acquires point cloud data from the received distance image. Next, the image acquisition unit 131 performs object recognition processing on the point cloud data to detect one or more object data that are point cloud data. Next, the image acquisition unit 131 detects a dynamic object from among the one or more object data by the above-mentioned algorithm. Next, the image acquisition unit 131 deletes the one or more dynamic objects from the point cloud data. Furthermore, the image acquisition unit 131 detects a partial object from the point cloud data, and deletes the partial object from the point cloud data.

[0311] Next, the update unit 135 stores the three-dimensional image, which is the point cloud data, in association with the user identifier "A". At this time, the update unit 135 determines that the area of ​​the LiDAR image transmitted by user B is included in the area of ​​the distance image transmitted by user A on July 31, 2023, and that the point cloud data acquired from the LiDAR image and the point cloud data acquired from the distance image of the same area are not identical, and overwrites the point cloud data based on the LiDAR image transmitted by user B with the point cloud data based on the distance image transmitted by user A on July 31, 2023. Next, the processing unit 13 acquires result information indicating that "updated". The result transmission unit 141 transmits the result information to the terminal device 2 of user A.

[0312] Next, the terminal device 2 of the user A receives and outputs the result information.

[0313] Also, a registered user C senses an urban space using a terminal device 2 having a LiDAR and acquires a LiDAR image. Then, the terminal device 2 of the user C transmits the image to the map update device 1. The image is associated with an image attribute value of "shooting date and time = 14:05:31 on August 1, 2023", a user identifier "C", and an environmental attribute value of "weather = rain".

[0314] Next, the image receiving unit 121 of the map update device 1 receives the image and the like. Next, the condition determining unit 134 obtains the size (W4) of the area covered by the image, and determines that the image condition is satisfied. Also, the condition determining unit 134 obtains the rank "1" paired with the user identifier "C" from the user management unit 112, and determines that the user information satisfies the user condition. Furthermore, the condition determining unit 134 obtains the environmental attribute value "weather=rain" paired with the image, and determines that the environmental condition is not satisfied. In other words, the condition determining unit 134 determines not to update the urban space map based on the image transmitted by user C. Next, the processing unit 13 obtains result information indicating that "no update was performed." The result transmitting unit 141 transmits the result information to the terminal device 2 of user C.

[0315] Next, the terminal device 2 of the user C receives and outputs the result information.

[0316] As described above, according to this embodiment, the map update device 1 updates the urban space map using images received from the terminal devices 2 of multiple users, thereby providing a platform where multiple users cooperate to update the urban space map.

[0317] Furthermore, according to this embodiment, by storing three-dimensional images in association with user identifiers, it is possible to manage users who have contributed to updating the urban space map.

[0318] Furthermore, according to this embodiment, an appropriate urban space map can be constructed by updating the urban space map using only images that meet the update conditions.

[0319] Furthermore, according to this embodiment, even if images with different data structures are received, three-dimensional images with a unified data structure can be accumulated to update the urban space map, thereby providing a platform that makes it easy to update the urban space map with the cooperation of a large number of users.

[0320] The process in this embodiment may be realized by software. This software may be distributed by software download or the like. This software may be recorded on a recording medium such as a CD-ROM and distributed. This also applies to other embodiments in this specification. The software for realizing the map update device 1 in this embodiment is a program as follows. That is, this program is a program for making a computer that can access a map storage unit in which an urban space map, which is a collection of three-dimensional images corresponding to location information, which represents an urban space, is stored, function as an update unit that performs an update process to update, using the three-dimensional image acquired by the image acquisition unit, a three-dimensional image in the urban space map in the map storage unit, which corresponds to the location information of an area that matches the area indicated by the location information corresponding to the image that was the basis of the three-dimensional image acquired by the image acquisition unit.

[0321] (Embodiment 2) In this embodiment, a map updating device that performs a reward process to give a reward to a user who has submitted an image for updating an urban space model will be described.

[0322] In this embodiment, a map update device that performs reward processing when a reward condition is satisfied will be described.

[0323] In addition, in this embodiment, a map update device will be described that performs a reward process to give a reward to a user who has sent an image even if the image has not been used for updating.

[0324] In addition, in this embodiment, when two or more user identifiers correspond to a three-dimensional image that is part of an urban space model, a map update device is described that gives rewards to rights holders identified by each of the two or more user identifiers.

[0325] In this embodiment, a map update device that manages the history of user identifiers for three-dimensional images that constitute an urban space model will be described.

[0326] Further, in this embodiment, a map updating device that appropriately determines the amount of reward to be given to the user based on image attribute values ​​and the like will be described.

[0327] Fig. 18 is a block diagram of a map system B in this embodiment. The map system B includes a map update device 3 and two or more terminal devices 4. The conceptual diagram of the map system B is the same as Fig. 1 except for the reference numerals of the devices.

[0328] The map update device 3 is a device that updates a three-dimensional urban space map using images received from the terminal device 4, and is a device that performs processing to give rewards to users. The map update device 3 is, for example, a cloud server or an ASP server, but the type is not important.

[0329] The terminal device 4 is a terminal used by a user. The terminal device 4 may be, for example, a personal computer, a smartphone, or a tablet terminal, but the type of the terminal device 4 is not important.

[0330] The map update device 3 includes a storage unit 31, a receiving unit 12, a processing unit 33, and a transmitting unit 34. The storage unit 31 includes a map storage unit 311, and a user management unit 112. The receiving unit 12 includes an image receiving unit 121, and an instruction receiving unit 122. The processing unit 33 includes an image acquiring unit 131, a partial accumulation unit 132, an object determining unit 133, a condition determining unit 134, an updating unit 135, an output image acquiring unit 136, a reward determining unit 331, and a reward processing unit 332. The transmitting unit 34 includes a result transmitting unit 341 and an output image transmitting unit 142.

[0331] The terminal device 4 includes a terminal storage unit 21, a terminal reception unit 42, a terminal processing unit 23, a terminal transmission unit 24, a terminal reception unit 45, and a terminal output unit 46.

[0332] Various types of information are stored in the storage unit 31 constituting the map update device 3. The various types of information are, for example, an urban space map described later, user information described later, one or more update conditions, an object management table, or one or more reward conditions.

[0333] The reward condition is a condition regarding whether or not to give a reward to a user who provides an image. The reward condition is, for example, one or a combination of two or more of an image condition, a user condition, and an environmental condition.

[0334] The map storage unit 311 stores an urban space map. The urban space map has two or more three-dimensional images. The three-dimensional images are associated with one or more user identifiers. It is preferable that time information is stored in association with each of the three-dimensional images and one or more user identifiers. The time information is information that specifies the time when the image that is the basis of the three-dimensional image was taken or the time when the three-dimensional image was accumulated. The time when the three-dimensional image was accumulated may be the time when the image was received. The time information is, for example, a date and time, but may also be a day, and the granularity does not matter.

[0335] The processing unit 33 performs various types of processing. The various types of processing are, for example, processing performed by the image acquisition unit 131, the partial accumulation unit 132, the object determination unit 133, the condition determination unit 134, the update unit 135, the output image acquisition unit 136, or the reward processing unit 332.

[0336] The update unit 135 performs the above-mentioned update process. The update process is a process of updating the urban space map. The update process is, for example, an append process or a change process. The append process is a process of accumulating a new three-dimensional image in an area where a three-dimensional image has not existed until now. The change process is a process of accumulating a new three-dimensional image in an area where a three-dimensional image has existed until now. The change process may overwrite an old three-dimensional image with a new three-dimensional image, or may append a new three-dimensional image while leaving the old three-dimensional image.

[0337] The update unit 135 performs an update process of storing the three-dimensional image acquired by the image acquisition unit 131 in association with a user identifier associated with the image received by the image receiving unit 121 for a three-dimensional image in the urban space map in the map storage unit 311, the three-dimensional image corresponding to area information of an area that matches an area indicated by area information associated with an image that is the basis of the three-dimensional image acquired by the image acquisition unit 131. Here, storing may be by overwriting or appending.

[0338] When the update unit 135 performs an update process, the update unit 135 may obtain result information to notify a user identified by the old user identifier corresponding to the updated three-dimensional image, and the result information may indicate that the user is no longer the rights holder as a result of the update process being performed.

[0339] The reward determination unit 331 determines whether or not the image received by the image receiving unit 121, or the three-dimensional image stored by the update unit 135, or the user identified by the user identifier satisfies the reward condition.

[0340] The reward processing unit 332 performs reward processing. The reward processing is processing for giving rewards to users. The user here is, for example, a right holder who contributed to updating the urban space map, or a person who transmitted an image. The reward processing is, for example, processing for giving points to a user, processing for giving money to a user, processing for giving tickets to a user, and processing for giving some kind of right to a user. Note that the reward given to a user is not limited.

[0341] The process of giving points to a user is, for example, a process of increasing points managed in pair with a user identifier. The process of giving money to a user is, for example, a process of transferring money to a user using bank account information managed in pair with a user identifier, and a process of sending money to a user using an ID of an electronic money application managed in pair with a user identifier. The process of giving tickets to a user is, for example, a process of giving an electronic ticket to play a game to a user, and a process of giving an electronic ticket to participate in an event to a user. The process of giving rights to a user is, for example, a process of giving a user the right to read an electronic book.

[0342] It is preferable that the reward processing unit 332 performs reward processing only when the reward determination unit 331 determines that the reward condition is satisfied.

[0343] The reward process is, for example, a usage reward process, an update reward process, and a transmission reward process. The usage reward process is a process of giving a reward to a right holder identified by one or more user identifiers paired with a three-dimensional image when the three-dimensional image is used. The use is, for example, viewing or downloading by a user. The update reward process is a process of giving a reward to a right holder identified by a user identifier paired with an image when an image is sent and an update process of a city map is performed using the image. The transmission reward process is a process of giving a reward to a user who sent an image when the image is sent.

[0344] That is, it is preferable that the reward processing unit 332 performs a usage reward process that is a process of giving a reward to a right holder identified by a user identifier corresponding to the three-dimensional image that is the basis of the output image. Also, in addition to or instead of the usage reward process, it is preferable that the reward processing unit 332 performs an update reward process that gives a reward to a right holder identified by a user identifier corresponding to the image received by the image receiving unit 121 when an update process is performed. Furthermore, it is preferable that the reward processing unit 332 performs a transmission reward process that gives a reward to a right holder identified by a user identifier corresponding to the image received by the image receiving unit 121 even when an update process is not performed.

[0345] When two or more user identifiers are associated with a three-dimensional image, the reward processing unit 332 preferably performs reward processing to give rewards to right holders identified by each of the two or more user identifiers. Each of the two or more user identifiers associated with a three-dimensional image is an identifier of a user who transmitted each of the two or more images that are the basis of the three-dimensional image, or a history of user identifiers of two or more users who transmitted an image of an area covered by the three-dimensional image.

[0346] It is preferable that the reward processing unit 332 determines the amount of reward and performs reward processing according to the amount of reward. The reward processing according to the amount of reward is processing for giving points, money, rights, etc. according to the amount of reward to the user.

[0347] The reward processing unit 332 acquires, for example, one or more image attribute values ​​of the image received by the image receiving unit 121 or the three-dimensional image used in the update process by the update unit 135. Next, the reward processing unit 332 determines, for example, a reward amount according to the one or more image attribute values. The image attribute values ​​are, for example, data amount, data quality, area information corresponding to the image, number of views, or viewing frequency. In such a case, for example, the reward amount differs depending on the area (for example, urban area and mountain area) supported by the image received by the image receiving unit 121. For example, the reward amount corresponding to the area information or the calculation method of the reward amount (for example, an arithmetic formula) is managed, and the reward processing unit 332 acquires the reward amount using the managed information and the area information corresponding to the image received by the image receiving unit 121. The reward processing unit 332 acquires, for example, the number of views or viewing frequency of the three-dimensional image, and acquires the reward amount according to the number of views or viewing frequency. For example, the reward processing unit 332 acquires a larger reward amount per view as the number of views or viewing frequency increases.

[0348] The reward processing unit 332 determines different reward amounts for, for example, an append process and a change process. The append process is a process in which the update unit 135 accumulates a three-dimensional image of a new area. The change process is a process in which the update unit 135 accumulates a new three-dimensional image of an area where a three-dimensional image already exists. It is preferable that the reward processing unit 332 determines a larger reward amount when the append process is performed than when the change process is performed.

[0349] The transmission unit 34 transmits various types of information, such as result information and an output image.

[0350] When the update unit 135 has performed the update process, the result transmission unit 341 transmits result information indicating that the update process has been performed to the terminal device 4 that transmitted the image.

[0351] For example, when the update unit 135 performs an update process, the result sending unit 341 sends result information to a user identified by the old user identifier corresponding to the updated three-dimensional image, indicating that the user is no longer the rights holder as a result of the update process.

[0352] The terminal reception unit 42 included in the terminal device 4 receives various information, instructions, etc. Examples of the various information, instructions, etc. include a viewing instruction, a photographing instruction, a transmission instruction, a user identifier, one or more user attribute values, and a reward output instruction.

[0353] A reward output instruction is an instruction to output the amount of reward held by a user.

[0354] The means for inputting various information and instructions may be anything, such as a touch panel, a keyboard, a mouse, a menu screen, a microphone, etc.

[0355] The terminal receiving unit 45 receives various types of information, such as an output image, result information, and a reward amount.

[0356] The terminal output unit 46 outputs various types of information, such as an output image, result information, and a reward amount.

[0357] Here, output is a concept that includes display on a display, projection using a projector, printing on a printer, sound output, transmission to an external device, storage on a recording medium, and handing over the processing results to other processing devices or other programs, etc.

[0358] The storage unit 31 and the map storage unit 311 are preferably non-volatile recording media, but may also be realized as volatile recording media.

[0359] There is no restriction on the process by which information is stored in the storage unit 31, etc. For example, information may be stored in the storage unit 31, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 31, etc., or information inputted via an input device may be stored in the storage unit 31, etc.

[0360] The processing unit 33, the reward determination unit 331, and the reward processing unit 332 can usually be realized by an MPU, a memory, etc. The processing procedure of the processing unit 33 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may also be realized by hardware (dedicated circuit).

[0361] The transmission unit 34 and the result transmission unit 341 are usually realized by wireless or wired communication means, but may also be realized by broadcasting means.

[0362] The terminal reception unit 42 can be realized by a device driver for an input means such as a numeric keypad or keyboard, or control software for a menu screen.

[0363] The terminal receiving unit 45 is usually realized by a wireless or wired communication means, but may be realized by a means for receiving a broadcast.

[0364] The terminal output unit 46 may or may not include an output device such as a display, a speaker, etc. The terminal output unit 46 may be realized by driver software for an output device, or a combination of driver software for an output device and an output device, etc.

[0365] Next, an example of the operation of the map updating device 3 will be described with reference to the flowchart of Fig. 19. In the flowchart of Fig. 19, the description of the same steps as those in the flowchart of Fig. 3 will be omitted.

[0366] (Step S1901) When there is a user who is no longer a right holder due to an update of the urban space map, the processing unit 33 etc. notifies the user. An example of such a former right holder notification process will be described with reference to the flowchart of FIG.

[0367] (Step S1902) The remuneration determination unit 331 determines whether or not the remuneration condition is satisfied. An example of such remuneration determination processing will be described with reference to the flowchart of FIG.

[0368] (Step S1903) If the determination result obtained in step S1902 is "a reward should be given", the process proceeds to step S1904, and if the determination result is "no reward should be given", the process returns to step S301.

[0369] (Step S1904) The reward processing unit 332 performs a process of giving a reward to the user. An example of such reward processing will be described with reference to the flowchart of FIG.

[0370] In the flowchart of FIG. 19, the process ends when the power is turned off or an interrupt occurs to end the process.

[0371] Next, an example of the previous rights holder notification process in step S1901 will be described with reference to the flowchart in FIG.

[0372] (Step S2001) The processing unit 13 judges whether or not the urban space map has been updated as a result of the processing in step S304. If the urban space map has been updated, the process proceeds to step S2002, and if the urban space map has not been updated, the process returns to the upper process.

[0373] (Step S2002) The update unit 135 judges whether or not there is an old three-dimensional image corresponding to the area of ​​the updated three-dimensional image. If there is an old three-dimensional image, the process proceeds to step S2003, and if there is not, the process returns to the upper process.

[0374] (Step S2003) The update unit 135 acquires one or more user identifiers that were paired with the old three-dimensional image.

[0375] (Step S2004) The update unit 135 assigns 1 to the counter i.

[0376] (Step S2005) The update unit 135 judges whether or not the i-th user identifier exists among the user identifiers acquired in step S2003. If the i-th user identifier exists, the process proceeds to step S2006, and if not, the process returns to the upper process.

[0377] (Step S2006) The update unit 135 acquires area information paired with the i-th user identifier. The update unit 135 determines whether or not the three-dimensional images of all areas specified by the area information have been updated. If the three-dimensional images of all areas have been updated, the update unit 135 acquires result information indicating that the user is "no longer the rights holder." If the three-dimensional images of some areas remain unupdated, the update unit 135 acquires result information indicating that the user is "no longer the rights holder in some areas."

[0378] (Step S2007) The result transmission unit 341 acquires notification destination information paired with the i-th user identifier from the user management unit 112. The result transmission unit 341 transmits the result information acquired in step S2006 to the notification destination indicated by the notification destination information. Note that the notification destination information is, for example, an email address or a user ID of a communication application (for example, Line (registered trademark)).

[0379] (Step S2008) The update unit 135 increments the counter i by 1. The process returns to step S2005.

[0380] Next, an example of the remuneration determination process in step S1902 will be described with reference to the flowchart in FIG.

[0381] (Step S2101) The reward determining unit 331 assigns 1 to a counter i.

[0382] (Step S2102) The reward determination unit 331 determines whether or not the i-th reward condition exists in the storage unit 31. If the i-th reward condition exists, the process proceeds to step S2103, and if not, the process proceeds to step S2107.

[0383] (Step S2103) The reward determination unit 331 acquires the i-th reward condition from the storage unit 31.

[0384] (Step S2104) The reward determination unit 331 acquires one or more pieces of information necessary for determining the i-th reward condition. The reward determination unit 331 determines whether or not the one or more pieces of information satisfy the i-th reward condition. If the i-th reward condition is satisfied, the process proceeds to step S2105. If the i-th reward condition is not satisfied, the process proceeds to step S2106.

[0385] (Step S2105) The reward determination unit 331 assigns information indicating "a reward will be given" to the variable "result." Then, the process returns to the upper level process.

[0386] (Step S2106) The reward determination unit 331 increments the counter i by 1. The process returns to step S2102.

[0387] (Step S2107) The reward determination unit 331 assigns information indicating that "reward will not be given" to the variable "result." Then, the process returns to the upper level process.

[0388] In the flowchart of Fig. 21, when two or more reward conditions exist, it is determined that a reward is to be given even if only one of the reward conditions is satisfied. However, in the flowchart of Fig. 21, when two or more reward conditions exist, it may be determined that a reward is to be given only when all of the reward conditions are satisfied.

[0389] Next, an example of the reward process in step S1904 will be described with reference to the flowchart in FIG.

[0390] (Step S2201) The reward processing unit 332 determines the amount of reward. An example of such reward amount determination processing will be described with reference to the flowchart in Fig. 23. Note that the determined amount of reward may be constant.

[0391] (Step S2202) The reward processing unit 332 assigns 1 to the counter i.

[0392] (Step S2203) The reward processing unit 332 judges whether or not the i-th user identifier stored in the buffer (not shown) in step S2201 exists. If the i-th user identifier exists, the process proceeds to step S2204, and if not, the process returns to the upper process.

[0393] (Step S2204) The reward processing unit 332 obtains the reward amount paired with the i-th user identifier from a buffer (not shown).

[0394] (Step S2205) The reward processing unit 332 adds points based on the reward amount acquired in step S2204 to the i-th user identifier.

[0395] (Step S2206) The reward processing unit 332 increments the counter i by 1. The process returns to step S2203.

[0396] Next, an example of the reward amount determination process in step S2201 will be described with reference to the flowchart in FIG.

[0397] (Step S2301) Before determining the reward, the reward processing unit 332 determines whether an image has been received in step S301 or whether a viewing instruction has been received in step S310. If an image has been received, the process proceeds to step S2302, and if a viewing instruction has been received, the process proceeds to step S2309.

[0398] (Step S2302) The reward processing unit 332 acquires a user identifier.

[0399] (Step S2303) The reward processing unit 332 uses the received image to determine whether the urban space map has been updated. If the urban space map has been updated, the process proceeds to step S2304. If the urban space map has not been updated, the process proceeds to step S2308.

[0400] (Step S2304) The reward processing unit 332 obtains one or more image attribute values ​​of the received image or a three-dimensional image based on the received image.

[0401] (Step S2305) The reward processing unit 332 judges whether an additional process or a change process has been performed using a three-dimensional image based on the received image. If an additional process has been performed, the process proceeds to step S2306, and if a change process has been performed, the process proceeds to step S2307.

[0402] (Step S2306) The reward processing unit 332 acquires the amount of reward when the additional processing is performed, and stores the amount of reward in a buffer (not shown) in a pair with the user identifier acquired in step S2302. The reward processing unit 332 returns to the upper process. It is preferable that the reward processing unit 332 acquires the amount of reward when the additional processing is performed using one or more image attribute values ​​acquired in step S2303.

[0403] (Step S2307) The reward processing unit 332 acquires the amount of reward when the change processing is performed, and stores the amount of reward in a buffer (not shown) in a pair with the user identifier acquired in step S2302. The reward processing unit 332 returns to the upper process. Note that it is preferable that the reward processing unit 332 acquires the amount of reward when the change processing is performed using one or more image attribute values ​​acquired in step S2303.

[0404] (Step S2308) The reward processing unit 332 acquires the reward amount in the case where the user did not contribute to the update, and stores the reward amount in a buffer (not shown) in combination with the user identifier acquired in step S2302. The process returns to the upper level process. Note that the reward amount in the case where the user did not contribute to the update may be "0".

[0405] (Step S2309) The reward processing unit 332 assigns 1 to the counter i.

[0406] (Step S2310) The reward processing unit 332 judges whether or not the i-th 3D image that was the source of the output image exists. If the i-th 3D image exists, the process proceeds to step S2311, and if not, the process returns to the upper process.

[0407] (Step S2311) The reward processing unit 332 acquires one or more image attribute values ​​of the i-th three-dimensional image.

[0408] (Step S2312) The reward processing unit 332 acquires a user identifier paired with the i-th three-dimensional image.

[0409] (Step S2313) The reward processor 332 acquires the reward amount in the case of browsing, and stores the reward amount in a buffer (not shown) in combination with the user identifier acquired in step S2312. Note that it is preferable for the reward processor 332 to acquire the reward amount in the case of browsing by using one or more image attribute values ​​acquired in step S2311.

[0410] (Step S2314) The reward processing unit 332 increments the counter i by 1. The process returns to step S2310.

[0411] A specific example of the operation of the map system B in this embodiment will now be described.

[0412] The map storage unit 311 of the map updating device 3 stores a map management table shown in Fig. 24, Fig. 25, or Fig. 26. The map management table is a table for managing the urban space map.

[0413] The map management table in FIG. 24 manages one or more records having "ID", "area information", and "update information". The "update information" has a "right holder identifier", "date and time", and "file identifier". The "ID" is information for identifying a record. The "area information" is information for identifying the area of ​​a file identified by a file identifier. The "area information" is information for identifying the area supported by a three-dimensional image, and is two pieces of position information in this case. The "area information" is a rectangular or cubic area in this case. The "right holder identifier" is a user identifier, and is the ID of the right holder of the file identified by the file identifier. The "date and time" is the shooting date and time of the image that is the source of the file identified by the file identifier. The "file identifier" is a file name. The file is a three-dimensional image file, and is a file in which point cloud data is stored in this case. The file is stored in the map storage unit 311 in this case. In the map management table in FIG. 24, the size of the area supported by the area information of each record may not be the same, or may be the same. In other words, the three-dimensional image corresponding to the record here is not unit data.

[0414] The map management table in FIG. 25 manages one or more records having "ID", "area information", and "update information". Here, two or more pieces of "update information" can exist for one record. In other words, in FIG. 25, the update history of the "area information" is clearly managed. Here, it is preferable that the "area information" is an area of ​​a fixed size. The area specified by the "area information" is usually a cubic area, but it may also be a rectangular parallelepiped area. The three-dimensional image corresponding to the record here is unit data.

[0415] The map management table in FIG. 26 manages one or more records having "ID", "area information", "object identifier", and "update information". The record may have type information that identifies the type of object. The map management table in FIG. 26 is a table for managing three-dimensional images for each object. The objects are, for example, buildings, roads, and bridges.

[0416] The user management unit 112 stores a user management table shown in FIG. 27. The user management table is a table for managing information related to users. The user management table manages one or more records having an "ID", "user identifier", "user attribute value", and "points". The "user attribute value" has "age", "gender", "number of transmissions", and "rank". The "number of transmissions" is the number of times a user has transmitted images to the map update device 3. The "rank" is the rank of the user. The "points" are points held by the user. The points can be used, for example, to purchase products on an EC site not shown. However, there is no restriction on how the points are used.

[0417] It is also assumed that the update condition "date of shooting >= X days before today" is stored in the storage unit 31. It is also assumed that the reward conditions "update: size of area covered by image >= first threshold" and "update" are stored in the storage unit 31. The reward condition "update: size of area covered by image >= first threshold" is when an update process (append process or change process) is performed, and indicates that the reward condition is met when the condition "size of area covered by image >= first threshold" is met. The reward condition "update" indicates that the reward condition is met when the image is viewed.

[0418] Furthermore, it is assumed that the storage unit 31 stores a reward amount management table shown in FIG. 28. The reward amount management table is a table for managing the amount of reward given to a user. The reward amount management table manages one or more records having an "ID", "processing condition", and "reward amount". The "processing condition" is a condition related to processing. When an append process is performed using an image sent by a user, the record with "ID=1" stores the size of the area of ​​the appended three-dimensional image as "R 1 If the number of users is more than 10, the user is given 10 points and the size of the area is increased to R 1", the point "7" is added to the user. The record "ID=2" indicates that if a change process is performed using an image sent by the user, the point "5" is added to the user. The record "ID=3" indicates that if an image is sent from the user but the image is not used to update the urban space map, the point "1" is added to the user. The record "ID=4" indicates that the point "3" is given to a user identified by one or more right holder identifiers that are paired with the three-dimensional image (file) used to view the urban space map. Note that even if an image is sent from a user but not used to update the urban space map, the user can be given a reward to encourage the user to actively send images.

[0419] In such a situation, the following five specific examples will be described. Specific Example 1 is a case where the map management table of FIG. 24 is used. Specific Example 2 is a case where the map management table of FIG. 25 is used. Specific Example 3 is a case where the map management table of FIG. 26 is used. Specific Example 4 is a case where the urban space map is updated using an image sent by the user, but the reward condition is not satisfied and the user is not given a reward. Specific Example 5 is a case where the urban space map is not updated using an image sent by the user, but the user is given a reward.

[0420] (Example 1) It is assumed that a registered user A (user identifier = U001) has taken a range image in an urban space using a terminal device 4 and transmitted it to the map update device 3. The range image is associated with an image attribute value "date and time of image capture = 10:38:19 on July 25, 2023", a user identifier "U001", and absolute position information (latitude, longitude, altitude).

[0421] Next, the image receiving unit 121 of the map update device 3 receives the distance image, etc. Next, the condition determination unit 134 obtains today's date from a clock (not shown), and determines that the update condition "photographed date >= X days before today" is satisfied using today's date and the date "July 25, 2023" contained in the photographed date and time, which is an image attribute value.

[0422] Next, the image acquisition unit 131 acquires point cloud data from the received distance image based on the operation of the flowchart in FIG. 6, and writes the point cloud data to a file “F x " is stored in the map storage unit 311.

[0423] Next, the update unit 135 updates the area information “(x m ,y m ,h m )(x n ,y n ,h n The update unit 135 obtains the area information, the user identifier "U001", the shooting date and time "July 25, 2023, 10:38:19", the file identifier "F x " and add the record to the map management table (FIG. 24).

[0424] Next, the update unit 135 increments the number of transmissions in the user management table (FIG. 27), which is the number of transmissions paired with the user identifier "U001", by "1". Also, the update unit 135 increments the rank in the user management table (FIG. 27), which is the rank paired with the user identifier "U001", by "1".

[0425] Next, the reward determination unit 331 calculates the area information “(x m ,y m ,h m )(x n ,y n ,h n) to obtain the size of the area, and determine that the size satisfies the reward condition "size of area covered by image >= first threshold value."

[0426] Next, the reward processing unit 332 converts the area information “(x m ,y m ,h m )(x n ,y n ,h n In other words, the reward processing unit 332 determines that an append process has been performed. Next, the reward processing unit 332 obtains the reward amount that is paired with the processing condition "append process" in the reward amount management table (FIG. 28). Next, the reward processing unit 332 obtains the reward amount that is paired with the processing condition "append process" in the reward amount management table (FIG. 28). Next, the reward processing unit 332 obtains the reward amount that is paired with the processing condition "append process" in the area information "(x m ,y m ,h m )(x n ,y n ,h n )" specifies the area whose size is "R 1 " or more, and obtains points "10" from the reward amount management table (FIG. 28). Next, the reward processing unit 332 adds "10" to the points in the user management table (FIG. 27) that are paired with the user identifier "U001".

[0427] It is assumed that a viewing instruction is then transmitted from the terminal device 4 of the user X.

[0428] Next, the instruction receiving unit 122 of the map update device 3 receives a viewing instruction from the terminal device 4. Next, the output image acquiring unit 136 acquires area information included in the received viewing instruction. Next, the output image acquiring unit 136 refers to the map management table (FIG. 24), acquires a file identifier that is paired with area information indicating an area within the area indicated by the acquired area information, and sorts the file identifiers in order of most recent date and time. Next, the output image acquiring unit 136 places the color information (R, G, B) of each of the two or more point data at the position indicated by the position information (x, y, z) of each of the two or more point data in the sorted order only if color information has not already been placed at the position indicated by the position information (x, y, z) of each of the two or more point data of the three-dimensional image (point cloud data) identified by the file identifier. As a result, the output image acquiring unit 136 acquires an output image in a virtual three-dimensional space. Next, the output image transmitting unit 142 transmits the acquired output image to the terminal device 4 of the user X. As a result, the output image is output to the terminal device 4 of the user X.

[0429] Next, the reward determination unit 331 refers to the reward condition "viewing" in the storage unit 31 and determines that the reward condition is met. Next, the reward processing unit 332 obtains the reward amount "3" that is paired with the processing condition "viewing" from the reward amount management table (FIG. 28). Next, the reward processing unit 332 obtains one or more user identifiers that are paired with the file identifiers of the files used to compose the output image from the map management table (FIG. 24).

[0430] Next, the reward processing unit 332 increases the points in the user management table (FIG. 27), which are paired with each of the one or more acquired user identifiers, by the acquired points "3".

[0431] As described above, in this specific example, a user can be appropriately rewarded for his / her contribution to updating the urban space map.

[0432] (Example 2) It is assumed that a registered user A (user identifier=U001) takes a LiDAR image in an urban space with the terminal device 4 and transmits it to the map update device 3. It is assumed that the LiDAR image is associated with an image attribute value of “photographed date and time=July 25, 2023, 10:38:19”, a user identifier “U001”, and absolute position information.

[0433] Next, the image receiving unit 121 of the map updating device 3 receives the LiDAR image, etc. Next, it is assumed that the updating unit 135 determines that the update condition is satisfied, as described above.

[0434] Next, the image acquisition unit 131 acquires point cloud data from the received LiDAR image based on the operation of the flowchart in FIG. 6, and stores the point cloud data in a file “F y " is stored in the map storage unit 311.

[0435] Next, the update unit 135 updates the area information “(X 31 ,Y 31 ,H 31 )(X 32 ,Y 32 ,H 32 The update unit 135 obtains the area information "(X 31 ,Y 31 ,H 31 )(X 32 ,Y 32 ,H 32 Next, the update unit 135 determines that a record (record with "ID=3") containing the user identifier "U001", the shooting date and time "July 25, 2023, 10:38:19", the file identifier "F y " is added to the map management table (FIG. 25) as update information for the record "ID=3". In other words, the update information for the record "ID=3" in the map management table (FIG. 25) is the already existing "<Rights holder identifier>U003 <Date and time>T 31 <file identifier>F 31" In addition to "<Rights holder identifier>U001 <Date and time>2023 / 07 / 25 10:38:19 <File identifier>F y " is stored.

[0436] Next, the update unit 135 increments the number of transmissions in the user management table (FIG. 27), which is the number of transmissions paired with the user identifier "U001", by "1". Also, the update unit 135 increments the rank in the user management table (FIG. 27), which is the rank paired with the user identifier "U001", by "1".

[0437] Next, the reward determination unit 331 calculates the area information "(X 31 ,Y 31 ,H 31 )(X 32 ,Y 32 ,H 32 ) to obtain the size of the area, and determine that the size satisfies the reward condition.

[0438] Next, the reward processing unit 332 receives the area information “(X 31 ,Y 31 ,H 31 )(X 32 ,Y 32 ,H 32 ) already exists in the map management table (Fig. 25). In other words, the reward processing unit 332 determines that a change process has been performed. Next, the reward processing unit 332 obtains the reward amount "5" that is paired with the processing condition "change process" in the reward amount management table (Fig. 28). Next, the reward processing unit 332 adds "5", which is the point paired with the user identifier "U001", to the point in the user management table (Fig. 27).

[0439] It is assumed that a viewing instruction is then transmitted from the terminal device 4 of the user X.

[0440] Next, the instruction receiving unit 122 of the map update device 3 receives a viewing instruction from the terminal device 4. Next, the output image acquisition unit 136 acquires area information contained in the received viewing instruction. Next, the output image acquisition unit 136 refers to the map management table (FIG. 25) and determines a record including area information indicating an area within the area indicated by the acquired area information. Here, it is assumed that the output image acquisition unit 136 has determined records "ID=1", "ID=2", and "ID=3", for example.

[0441] Next, the output image acquisition unit 136 searches for the file identifier "F" that is paired with the latest (most recent) date and time for each of the records "ID=1", "ID=2", and "ID=3". 13 " "F 22 " "F 31 " is obtained from the map management table (FIG. 25). Next, the output image obtaining unit 136 obtains the files identified by the three file identifiers, and constructs an output image using the point cloud data of the three files. Next, the output image transmitting unit 142 transmits the output image to the terminal device 4 of the user X.

[0442] Next, the reward determination unit 331 refers to the reward condition "viewing" in the storage unit 31 and determines that the reward condition is met. Next, the reward processing unit 332 obtains the reward amount "3" that is paired with the processing condition "viewing" from the reward amount management table (Fig. 28). Next, the reward processing unit 332 obtains the user identifiers "U003", "U004", and "U003" that are paired with the file identifiers of the three files used to compose the output image from the map management table (Fig. 24).

[0443] Next, the reward processing unit 332 increases the points in the user management table (FIG. 27) that are paired with each of the three acquired user identifiers by the acquired point "3." Here, the reward processing unit 332 increases the points paired with the user identifier "U003" by "6," and increases the points paired with the user identifier "U004" by "3."

[0444] As described above, in this specific example, a user can be appropriately rewarded for his / her contribution to updating the urban space map.

[0445] (Example 3) It is assumed that a registered user A (user identifier=U001) has taken a range image in an urban space with the terminal device 4 and transmitted it to the map update device 3. The range image is associated with the image attribute value "photographed date and time=July 25, 2023, 10:38:19", the user identifier "U001", and absolute position information.

[0446] Next, the image receiving unit 121 of the map updating device 3 receives the distance image etc. Next, it is assumed that the updating unit 135 determines that the distance image satisfies the update condition.

[0447] Next, it is assumed that image acquisition unit 131 acquires point cloud data from the received distance image, recognizes one or more objects from the point cloud data, and acquires object data, which is point cloud data, for each of the one or more objects based on the operation of the flowchart in Fig. 6. Note that since dynamic objects have been removed by the above-mentioned process, the one or more object data acquired by image acquisition unit 131 does not include object data of dynamic objects.

[0448] Next, the update unit 135 extracts area information (x b ,y b ,h b ) is acquired. The area information is the position information of the reference point of the object. Here, for example, this position information is the position information with the minimum values ​​for all of (latitude, longitude, altitude) among the position information possessed by the object data.

[0449] Next, the update unit 135 updates the object data in the file “F z The update unit 135 then stores the area information (x b ,y b ,h b ) and the user identifier "U001", the shooting date and time "July 25, 2023 10:38:19", and the file identifier "Fz " is added to the map management table (FIG. 26) as update information for the record with "ID=2".

[0450] Next, the update unit 135 increments the number of transmissions in the user management table (FIG. 27), which is the number of transmissions paired with the user identifier "U001", by "1". Also, the update unit 135 increments the rank in the user management table (FIG. 27), which is the rank paired with the user identifier "U001", by "1".

[0451] Next, the reward determination unit 331 acquires the size of the object data, and determines that the size satisfies the reward condition "size of area covered by image>=first threshold value".

[0452] Next, the reward processing unit 332 converts the area information “(x b ,y b ,h b ) already exists in the map management table (Fig. 26). In other words, the reward processing unit 332 determines that a change process has been performed. Next, the reward processing unit 332 obtains the reward amount "5" that is paired with the processing condition "change process" in the reward amount management table (Fig. 28). Next, the reward processing unit 332 adds "5", which is the point paired with the user identifier "U001", to the point in the user management table (Fig. 27).

[0453] It is assumed that a viewing instruction is then transmitted from the terminal device 4 of the user X.

[0454] Next, the instruction receiving unit 122 of the map update device 3 receives a viewing instruction from the terminal device 4. Next, the output image acquisition unit 136 acquires area information contained in the received viewing instruction. Next, the output image acquisition unit 136 refers to the map management table (FIG. 26) and determines a record including area information indicating an area within the area indicated by the acquired area information. Here, it is assumed that the output image acquisition unit 136 has determined records "ID=1", "ID=2", and "ID=3", for example.

[0455] Next, the output image acquisition unit 136 searches for the file identifier "F" that is paired with the most recent date and time for each of the records "ID=1", "ID=2", and "ID=3". 11 " "F 22 " "F 33 " is obtained from the map management table (FIG. 26). Next, the output image obtaining unit 136 obtains the files identified by the three file identifiers, and uses the object data (point cloud data) of the three files to construct an output image. Next, the output image transmitting unit 142 transmits the output image to the terminal device 4 of the user X.

[0456] Next, the reward determination unit 331 refers to the reward condition "when viewing" in the storage unit 31 and determines that the reward condition is met. Next, the reward processing unit 332 obtains the reward amount "3" that is paired with the processing condition "view" from the reward amount management table (Fig. 28). Next, the reward processing unit 332 obtains the user identifiers "U001", "U003", and "U001" that are paired with the file identifiers of the three files used to compose the output image from the map management table (Fig. 24).

[0457] Next, the reward processing unit 332 increases the points in the user management table (FIG. 27) that are paired with each of the three acquired user identifiers by the acquired point "3." Here, the reward processing unit 332 increases the points paired with the user identifier "U001" by "6," and increases the points paired with the user identifier "U003" by "3."

[0458] As described above, in this specific example, a user can be appropriately rewarded for his / her contribution to updating the urban space map.

[0459] (Example 4) In the specific example 3, it is assumed that the reward determination unit 331 acquires the size of the object data and determines that the size does not satisfy the reward condition "size of the area covered by the image >= first threshold value." In such a case, the reward determination unit 331 does not perform reward processing.

[0460] As described above, in this embodiment, a user can be rewarded for his / her contribution to updating the urban space map only when appropriate.

[0461] (Example 5) It is assumed that a registered user B (user identifier=U002) has taken a range image in an urban space with the terminal device 4 and transmitted it to the map update device 3. It is assumed that the range image is associated with the image attribute value "taken date and time=March 21, 2023, 8:08:09" and the user identifier "U002".

[0462] Next, the image receiving unit 121 of the map update device 3 receives the distance image, etc. Next, the update unit 135 acquires today's date from a clock (not shown), and determines that the update condition "photographed date >= X days before today" is not satisfied based on the acquired date and the date "March 21, 2023" included in the photographed date and time.

[0463] Next, the update unit 135 obtains the result information indicating that "update was not performed." Next, the result transmission unit 341 transmits the result information to the terminal device 4 of the user B.

[0464] Next, the reward processing unit 332 judges that the processing condition "send only" is met based on the result information of "not updating", and obtains the point "1" that corresponds to the processing condition from the reward amount management table (FIG. 28). Next, the reward processing unit 332 adds "1" to the point in the user management table (FIG. 27) that corresponds to the user identifier "U002".

[0465] Furthermore, the terminal device 4 of user B receives and outputs the result information indicating that the update was not performed.

[0466] As described above, in this specific example, the urban space map was not updated using the images submitted by the user, but by giving rewards to the user, it is possible to increase the motivation of the user to submit images.

[0467] As described above, according to this embodiment, when a plurality of people cooperate to update an urban space map, a platform can be provided that rewards the rights holder who provided the image.

[0468] Furthermore, according to this embodiment, when a three-dimensional image of a portion of an urban space map that is updated cooperatively by multiple people is used, a platform can be provided that rewards the rights holder who provided the image that is the basis of the three-dimensional image.

[0469] Furthermore, according to this embodiment, when a part of an urban space map that is updated through the cooperation of multiple people is updated, a platform can be provided that rewards the rights holder who provided the image for the update.

[0470] Furthermore, according to the present embodiment, even if the urban space map is not updated using the image sent by the user, a platform that rewards the user who sent the image can be provided. As a result, it is expected that users will actively send images.

[0471] Furthermore, according to this embodiment, when two or more user identifiers correspond to one three-dimensional image, a reward can be given to each of the right holders identified by the two or more user identifiers.

[0472] Furthermore, according to this embodiment, the history of the rights holders of the three-dimensional images can be managed.

[0473] Furthermore, according to this embodiment, it is possible to give appropriate remuneration to the rights holder according to the image attribute value.

[0474] Furthermore, according to this embodiment, it is possible to give the right holder an appropriate remuneration according to the content of the update process of the urban space map.

[0475] The software for realizing the map system B in this embodiment is the following program. That is, this program is a program for making a computer that can access a map storage unit in which an urban space map, which is a collection of three-dimensional images corresponding to a user identifier and location information, function as an image receiving unit that receives from a terminal device an image of an urban space that corresponds to a user identifier and location information, an image acquiring unit that acquires a three-dimensional image of all or a part of the urban space using all or a part of the image received by the image receiving unit, an update unit that performs an update process for storing the three-dimensional image acquired by the image acquiring unit in association with the user identifier that corresponds to the image received by the image receiving unit, which is a three-dimensional image in the urban space map in the map storage unit and corresponds to location information of an area that matches an area indicated by the location information that corresponds to the image that is the basis of the three-dimensional image acquired by the image acquiring unit, and a reward processing unit that performs a reward process that is a process of giving a reward to a user identified by the user identifier.

[0476] Moreover, Fig. 29 shows the appearance of a computer that executes the programs described in this specification to realize the map update device 3 and the like of the various embodiments described above. The above-mentioned embodiments can be realized by computer hardware and a computer program executed thereon. Fig. 29 is an overview of this computer system 300, and Fig. 30 is a block diagram of the system 300.

[0477] In FIG. 29, a computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0478] 30, computer 301 includes, in addition to CD-ROM drive 3012, MPU 3013, bus 3014 connected to CD-ROM drive 3012 etc., ROM 3015 for storing programs such as a boot-up program, RAM 3016 connected to MPU 3013 for temporarily storing instructions of application programs and providing temporary storage space, and hard disk 3017 for storing application programs, system programs, and data. Although not shown here, computer 301 may further include a network card for providing connection to a LAN.

[0479] A program for causing computer system 300 to execute functions such as those of map updating device 3 of the above-described embodiment may be stored on CD-ROM 3101, inserted into CD-ROM drive 3012, and then transferred to hard disk 3017. Alternatively, the program may be transmitted to computer 301 via a network (not shown) and stored on hard disk 3017. The program is loaded into RAM 3016 when executed. The program may also be loaded directly from CD-ROM 3101 or the network.

[0480] The program does not necessarily include an operating system (OS) or a third party program that causes the computer 301 to execute the functions of the map update device 3 of the above-mentioned embodiment. The program only needs to include an instruction portion that calls appropriate functions (modules) in a controlled manner to achieve a desired result. How the computer system 300 operates is well known, and a detailed description will be omitted.

[0481] In addition, in the above program, the steps of transmitting information and receiving information do not include processing performed by hardware, such as processing performed by a modem or interface card in the transmitting step (processing that is performed only by hardware).

[0482] The program may be executed by a single computer or a plurality of computers. That is, the program may be executed by a centralized processing or a distributed processing.

[0483] Furthermore, in each of the above embodiments, it goes without saying that two or more communication means present in one device may be physically realized by one medium.

[0484] In each of the above embodiments, each process may be realized by centralized processing in a single device, or may be realized by distributed processing in a plurality of devices.

[0485] The present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these modifications are also included within the scope of the present invention. [Industrial Applicability]

[0486] As described above, the map update device 3 of the present invention has the effect of providing a platform for rewarding rights holders who provide images when multiple people cooperate to update an urban space map, and is useful as a server for managing urban space maps. [Explanation of symbols]

[0487] B Map System 3 Map update device 4 Terminal Equipment 12 Receiving section 31 Storage area 33 Processing section 34 Transmitter 311 Map Storage Unit 112 User Management Department 121 Image receiving unit 122 Instruction receiving unit 131 Image acquisition unit 132 Partial storage section 133 Object Judgment Department 134 Condition judgment section 135 Update Department 136 Output image acquisition unit 341 Result transmission unit 142 Output image transmission unit 331 Remuneration Judgment Department 332 Reward Processing Unit

Claims

1. an image receiving unit that receives from a terminal device an image of an urban space, the image corresponding to a user identifier and location information; an image acquisition unit that acquires a three-dimensional image of all or a part of the urban space by using all or a part of the image received by the image receiving unit; an update unit that performs an update process for storing a three-dimensional image in an urban space map in a map storage unit, the three-dimensional image being a collection of three-dimensional images corresponding to a user identifier and location information, the three-dimensional image acquired by the image acquisition unit being associated with the user identifier corresponding to the image received by the image receiving unit, the three-dimensional image being associated with location information of an area that matches an area indicated by the location information corresponding to the image that was the basis of the three-dimensional image acquired by the image acquisition unit; a reward processing unit that performs a reward process that is a process of giving a reward to a user identified by the user identifier.

2. Further comprising a reward determination unit that determines whether the image received by the image receiving unit, the three-dimensional image stored by the update unit, or the user identified by the user identifier satisfies a reward condition; The reward processing unit: The map updating device according to claim 1 , wherein the reward process is performed only when the reward determination unit determines that the reward condition is satisfied.

3. an instruction receiving unit that receives an instruction to output the urban space map from a terminal device; an output image acquisition unit that acquires a three-dimensional image of an area corresponding to the output instruction from the map storage unit and acquires an output image using the three-dimensional image; an output image transmission unit that transmits the output image acquired by the output image acquisition unit to the terminal device that transmitted the output instruction, The reward processing unit:

3. The map update device according to claim 1, further comprising a usage reward process for giving a reward to a right holder identified by the user identifier associated with the three-dimensional image on which the output image is based.

4. The reward processing unit:

2. The map update device according to claim 1, further comprising an update reward process for giving a reward to a right holder identified by the user identifier corresponding to the image received by the image receiving unit when the update process is performed.

5. The reward processing unit:

2. The map update device according to claim 1, further comprising a transmission reward process for giving a reward to a user identified by the user identifier corresponding to the image received by the image receiving unit even if the update process is not performed.

6. The reward processing unit:

2. The map update device according to claim 1, wherein, when two or more user identifiers correspond to the three-dimensional image, a reward process is performed to give a reward to each of the right holders identified by the two or more user identifiers.

7. 2. The map updating device according to claim 1, further comprising: a previous user identifier, which is a user identifier associated with the three-dimensional image before the update processing is performed, associated with the three-dimensional image for which the update processing is performed.

8. The reward processing unit: The map update device according to claim 1, further comprising: acquiring one or more image attribute values ​​of the image received by the image receiving unit or the three-dimensional image used in the update processing of the update unit; determining a reward amount according to the one or more image attribute values; and performing reward processing according to the reward amount.

9. The reward processing unit: The map update device according to claim 1, wherein different reward amounts are determined when the update unit performs an append process to accumulate a three-dimensional image of a new area and when the update unit performs a change process to accumulate a new three-dimensional image of an area where a three-dimensional image already exists, and reward processing is performed according to the determined reward amounts.

10. 2. The map updating device according to claim 1, further comprising a result transmitting unit that transmits, when the updating unit has performed the update process, result information indicating that the update process has been performed to the terminal device that transmitted the image.

11. The map update device according to claim 1, further comprising a result sending unit that, when the update unit performs the update process, sends result information to a user identified by a previous user identifier corresponding to the updated three-dimensional image, indicating that the user is no longer a rights holder as a result of the update process.

12. A map update method implemented by a map storage unit in which an urban space map, which is a collection of three-dimensional images corresponding to a user identifier and location information, is stored, an image receiving unit, an image acquiring unit, an update unit, and a reward processing unit, an image receiving step in which the image receiving unit receives from a terminal device an image of an urban space, the image corresponding to a user identifier and location information; an image acquisition step in which the image acquisition unit acquires a three-dimensional image of all or a part of the urban space using all or a part of the image received by the image receiving unit; an updating step in which the updating unit performs an updating process in which the three-dimensional image acquired by the image acquiring unit is stored in a three-dimensional image in an urban space map in a map storage unit in which an urban space map, which is a collection of three-dimensional images corresponding to a user identifier and location information, is stored in a three-dimensional image corresponding to location information of an area that matches an area indicated by the location information corresponding to the image that is the basis of the three-dimensional image acquired by the image acquiring unit, in association with the user identifier corresponding to the image received by the image receiving unit; The map updating method further comprises a reward processing step in which the reward processing unit performs a reward processing for giving a reward to the user identified by the user identifier.

13. A computer that can access a map storage unit in which an urban space map, which is a collection of three-dimensional images corresponding to a user identifier and location information, is stored, an image receiving unit that receives from a terminal device an image of an urban space, the image corresponding to a user identifier and location information; an image acquisition unit that acquires a three-dimensional image of all or a part of the urban space by using all or a part of the image received by the image receiving unit; an update unit that performs an update process for storing the three-dimensional image acquired by the image acquisition unit in association with the user identifier associated with the image received by the image receiving unit, the three-dimensional image being a three-dimensional image in the urban space map in the map storage unit and corresponding to location information of an area that matches an area indicated by the location information associated with the image that is the basis of the three-dimensional image acquired by the image acquisition unit; A program for functioning as a reward processing unit that performs reward processing, which is a process of giving a reward to a user identified by the user identifier.

Images