Program, information processing device, system, and information processing method

Abstract

To provide a program, an information processing device, a system and an information processing method that are capable of controlling devices by suppressing communication traffic volumes and arithmetic loads.SOLUTION: According to one aspect of the present invention, a program causes a computer to implement: a feature point information acquisition function 102 for acquiring a first feature point and a second feature point generated on the basis of map information to be reference and first photographic information acquired by a first terminal at a reference position corresponding to the map information; a change information acquisition function 104 for acquiring information related to changes at positions detected on the basis of the first feature point and the second feature point; and an execution function 105 for causing at least one of the first terminal and a second terminal different from the first terminal to execute prescribed operation including coping with the changes. The prescribed operation excludes the acquisition or transmission of second photographic information at the reference position.SELECTED DRAWING: Figure 1

Description

Technical Field

[0006] , , ,

[0005] , ,

[0001] The present invention relates to a program, an information processing apparatus, a system, and an information processing method.

Background Art

[0002] Patent Document 1 discloses a technique for updating map data without compressing the communication band based on the output of sensors installed in a moving body.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, although the invention described in Patent Document 1 performs various processes for updating map data in a device inside a vehicle, there is a problem that the calculation load on the device increases due to the concentration of the processes in one device.

[0005] Therefore, in view of the above problems, an object of the present invention is to provide a program, an information processing apparatus, a system, and an information processing method that can control the operation of a data bus by suppressing communication volume and calculation load.

Means for Solving the Problems

[0006] A program according to one aspect of the present invention provides a computer with a feature point information acquisition function that acquires information about first and second feature points generated based on reference map information and first photographic information acquired by a first terminal at a reference position corresponding to the map information; a change information acquisition function that acquires information about changes in a position detected based on the first and second feature points; and an execution function that, when information about a change is detected, causes at least one of the first terminal and a second terminal different from the first terminal to perform a predetermined operation including responding to the change, wherein the predetermined operation is limited to acquiring or transmitting second photographic information at the reference position.

[0007] An information processing device according to one aspect of the present invention includes: a feature point information acquisition unit that acquires information about first and second feature points generated based on reference map information and first photographic information acquired by a first terminal at a reference position corresponding to the map information; a change information acquisition unit that acquires information about changes in a position detected based on the first and second feature points; and an execution unit that, when information about a change is detected, causes at least one of the first terminal and a second terminal different from the first terminal to perform a predetermined operation including responding to the change, wherein the predetermined operation is limited to acquiring or transmitting second photographic information at the reference position.

[0008] An information processing system according to one aspect of the present invention includes: a feature point information acquisition unit that acquires information about first and second feature points generated based on reference map information and first photographic information acquired by a first terminal at a reference position corresponding to the map information; a change information acquisition unit that acquires information about changes in a position detected based on the first and second feature points; and an execution unit that, when information about a change is detected, causes at least one of the first terminal and a second terminal different from the first terminal to perform a predetermined operation including responding to the change, wherein the predetermined operation is limited to acquiring or transmitting second photographic information at the reference position.

[0009] An information processing method according to one aspect of the present invention provides a computer with the following steps: a feature point information acquisition step of acquiring information about first and second feature points generated based on reference map information and first photographic information acquired by a first terminal at a reference location corresponding to the map information; a change information acquisition step of acquiring information about changes at a location detected based on the first and second feature points; and an execution step of causing at least one of the first terminal and a second terminal different from the first terminal to perform a predetermined operation including responding to the change when information about a change is detected, wherein the predetermined operation is limited to acquiring or transmitting second photographic information at the reference location. [Effects of the Invention]

[0010] According to the present invention, it is possible to control the operation of a device by suppressing the amount of data transmitted and the computational load. [Brief explanation of the drawing]

[0011] [Figure 1] This is a diagram illustrating the overview of the system according to this embodiment. [Figure 2] This diagram illustrates an example of the functional configuration of the system according to this embodiment. [Figure 3] This diagram illustrates an example of the operation of the system according to this embodiment. [Figure 4] This diagram illustrates an example of the operation of the system according to this embodiment. [Figure 5] This diagram illustrates an example of the operation of the system according to this embodiment. [Figure 6] This diagram illustrates an example of the operation of the system according to this embodiment. [Figure 7] This diagram illustrates an example of the operation of the system according to this embodiment. [Figure 8] This diagram illustrates an example of the operation of the system according to this embodiment. [Figure 9] This diagram illustrates an example of the operation of the system according to this embodiment. [Figure 10] This is a diagram for explaining an example of the operation of the system according to this embodiment. [Figure 11] This is a diagram for explaining an example of the operation of the system according to this embodiment. [Figure 12] This is a diagram for explaining an example of the hardware configuration of the system according to this embodiment. [Figure 13] This is a diagram for explaining a modification example of the system according to this embodiment. **Modes for Carrying Out the Invention**

[0012] Preferred embodiments of the present invention (hereinafter referred to as "this embodiment") will be described with reference to the accompanying drawings. In each figure, those denoted by the same reference numerals have the same or similar configurations.

[0013] In the present invention, "part", "means", "device", and "system" do not simply mean physical means, but also include cases where the functions of the "part", "means", "device", and "system" are realized by software. Also, even if the functions of one "part", "means", "device", or "system" are realized by two or more physical means, devices, or software, or the functions of two or more "parts", "means", "devices", or "systems" are realized by one physical means, device, or software, it is acceptable.

[0014] FIG. 1 shows a schematic diagram of a device control system 1 according to this embodiment. The device control system 1 is a system that controls the terminal 14 based on imaging information acquired by an imaging device such as a camera provided in the terminal 14. Controlling the terminal 14 includes changing the moving state of the movable terminal 14 (for example, changing the moving speed and moving route), notifying the user of the terminal 14, and causing the terminal 14 to transmit the acquired imaging information.

[0015] <1. System Configuration> Referring to FIG. 1, an example of the system configuration of the device control system 1 will be described.

[0016] As shown in FIG. 1, the device control system 1 includes a device control server 10, a map information server 12, a terminal 14, and a communication network 16. The number of terminals is not limited to, for example, three, and may be two or less or four or more. The device control server 10, the map information server 12, and the terminal 14 are communicably connected to each other via the communication network 16.

[0017] The device control server 10 is an information processing device capable of communicating with the map information server 12 and the terminal 14. By executing a predetermined program, the device control server 10 realizes a function of controlling the terminal 14. Controlling the terminal 14 includes causing the terminal 14 to execute a predetermined operation (excluding transmission and acquisition of shooting information) and causing the terminal 14 to transmit shooting information.

[0018] The map information server 12 is an information processing device used by a person who manages map information and is an information processing device capable of communicating with the device control server 10 and the terminal 14. By executing a predetermined program, the map information server 12 realizes a function of updating map information.

[0019] The terminal 14 is a device capable of communicating with the device control server 10 and the map information server 12. By executing a predetermined program, the terminal 14 realizes a function of acquiring map information and shooting information and generating information related to feature points based on these information. The terminal 14 is, for example, an automobile, a fixed-point camera, or a smartphone.

[0020] The communication network 16 consists of wireless and wired networks. Examples of networks include mobile phone networks, PHS (Personal Handy-phone System) networks, wireless LAN (Local Area Network, including communication compliant with IEEE 802.11 (so-called Wi-Fi®)), 3G (3rd Generation), LTE (Long Term Evolution), 4G (4th Generation), 5G (5th Generation), WiMax®, infrared communication, visible light communication, Bluetooth®, wired LAN, telephone lines, power line communication networks, and networks compliant with IEEE 1394, etc.

[0021] Here, map information refers to information about all or part of the features on the Earth's surface. Specifically, map information includes the shape and name of a feature at any given coordinate. Furthermore, map information may pertain to multiple features within a certain range, or to a single feature located at a specific coordinate. For example, map information may pertain to all buildings in Tokyo, or to a single building at a specific location. Map information may also include design drawings, design models, and 3D data of completed products.

[0022] Map information may be based on either two-dimensional or three-dimensional maps. A two-dimensional map is a map that represents features in a two-dimensional plane on a two-dimensional coordinate system. In two-dimensional maps, the three-dimensional shape of terrain is typically represented by contour lines, etc., while the three-dimensional shape of buildings is not represented. On the other hand, a three-dimensional map is a map that represents features in three dimensions on a three-dimensional coordinate system. Three-dimensional maps are also called 3D maps and are used for terrain data in virtual spaces such as the metaverse. Map information also includes other types of information, such as bird's-eye views that combine two-dimensional and three-dimensional representations.

[0023] Map information may include location and information about features associated with them. For example, in addition to the three-dimensional shape of a feature, its name, latitude, and longitude may also be associated with it. Specific examples of map information will be described later using Figure 5.

[0024] Furthermore, imaging information is typically information generated by an imaging device. Imaging devices include cameras, LiDAR (Light Detection and Ranging), infrared cameras, and sonar, which are devices that can determine the shape or pattern of an object by performing their functions.

[0025] The captured information is typically image data or video data. Furthermore, the image data and video data may be either 2D or 3D data. For example, the captured information may be 3D shape data in OBJ (object) format generated using LiDAR, or 2D image data in JPEG (Joint Photographic Experts Group) format generated using a general optical camera.

[0026] The shooting information may be associated with additional information such as location information, information about the photographer, and information about the time of shooting.

[0027] The image information includes first image information for generating information about feature points and second image information for updating map information stored in the map information server 12. Specific examples of the image information will be described later using Figure 7.

[0028] Furthermore, information regarding feature points refers to predetermined information generated based on map information or photographic information. Examples of information regarding feature points include the height, aspect ratio, type of entrance, number of windows, overall shape, unevenness of specific locations, and distance from other features of a building included in the map information.

[0029] Information about feature points can also be called features. In this case, information about feature points may be generated using algorithms that calculate features based on image data, such as SIFT (Scale-Invariant Feature Transform) and HOG (Histograms of Oriented Gradients).

[0030] Information regarding feature points includes information regarding first feature points generated based on map information and information regarding second feature points generated based on first image information. Specific examples of generating information regarding first feature points will be described later using Figure 6, and specific examples of generating information regarding second feature points will be described later using Figure 8.

[0031] <2. Functional Configuration of Each Device> Referring to the functional block diagram in Figure 2, an example of the function of each device included in the device control system 1 will be explained.

[0032] As shown in Figure 2, the device control system 1 includes a device control server 10, a map information server 12, a terminal 14, and a communication network 16.

[0033] —(1) Device control server 10— The device control server 10 includes a control unit 100, a storage unit 120, and a network interface unit 140.

[0034] The memory unit 120 includes a control mode information DB (DataBase) 121, a feature point information DB 122, a change information DB 124, a terminal information DB 126, and an image capture information DB 128. Below are some examples of the information stored in each DB of the memory unit 120.

[0035] —Control Mode Information DB121— The control mode information DB121 stores information regarding the control mode of the device control server 10. The control mode is a setting value used to determine whether to activate the execution unit 105 or the image acquisition unit 106 when the change information acquisition unit 104 detects information regarding a change. The device control server 10 activates the execution unit 105 when the control mode is set to "predetermined operation mode," and activates the image acquisition unit 106 when the control mode is set to "image acquisition mode." An example of the operation of the device control server 10 according to the control mode will be described later using Figure 3.

[0036] The control mode may be pre-configured (for example, from the time of factory shipment of the device control server 10), or it may be set retrospectively. The control mode may be fixed and unchangeable once set, or it may be changeable.

[0037] —Feature Point Information DB122— The feature point information DB122 stores information about feature points acquired by the feature point information acquisition unit 102 from the terminal 14. The stored information about feature points is used by the change information acquisition unit 104 to acquire information about changes.

[0038] —Change Information DB124— The change information DB124 stores information about changes at a location corresponding to the reference map information (hereinafter referred to as the "reference location") acquired by the change information acquisition unit 104. The stored change information is used by the update unit 108 to determine whether or not it is necessary to update the map information in the map information server 12.

[0039] Changes at the reference location include changes in geographical features at that location. Changes in geographical features include, for example, changes in buildings and topography. Changes in buildings include, for example, additions, reductions, rebuilding, demolition, and new construction of buildings, as well as changes in tenants, changes in design, changes in interior, and deterioration or malfunction of equipment. Changes in topography include, for example, man-made changes such as construction work on rivers and slopes, as well as natural changes such as landslides and earthquakes.

[0040] —Terminal Information DB126— The terminal information DB126 stores identification information and status information for each terminal. Terminal identification information is information that can uniquely identify each terminal, and can also be called a terminal ID. Status information includes information indicating the terminal's location, speed of movement, and direction it is facing. Typically, information on all terminals 14 of the device control system 1 is registered in the terminal information DB126.

[0041] —Filming Information DB128— The shooting information DB128 stores the second shooting information acquired by the shooting information acquisition unit 106 from the terminal 14. The stored second shooting information is used to update the map information stored in the map information server 12.

[0042] The control unit 100 includes a feature point information acquisition unit 102, a change information acquisition unit 104, an execution unit 105, a shooting information acquisition unit 106, an update unit 108, and a communication unit 110. The following shows an example of the processing in each functional unit of the control unit 100.

[0043] —Feature Point Information Acquisition Unit 102— The feature point information acquisition unit 102 acquires information regarding the first and second feature points, which are generated based on the map information at the reference location and the first photographic information acquired by the terminal 14 at the reference location.

[0044] The feature point information acquisition unit 102 acquires at least a portion of the information regarding feature points stored in the feature point information DB 326 of the terminal 14. The acquired information regarding feature points is stored in the feature point information DB 122.

[0045] —Change Information Acquisition Unit 104— The change information acquisition unit 104 acquires information regarding changes at the reference position, which are detected based on the first and second feature points.

[0046] Detecting a change based on the first and second feature points includes, for example, the existence of a difference between the first and second feature points. It also includes, for example, the difference between the first and second feature points being greater than or equal to a predetermined threshold. Specific examples of acquiring information about changes will be described later using Figure 9.

[0047] The change information acquisition unit 104 acquires information about changes based on the information about the first and second feature points stored in the feature point information DB 122. The acquired information about changes is stored in the change information DB 124.

[0048] —Execution Section 105— The execution unit 105, upon detecting information regarding a change in the reference position and confirming that the control mode stored in the control mode information DB 121 is a "predetermined operation mode," instructs the terminal 14 to perform a predetermined operation, including responding to the change. The predetermined operation includes, for example, changing the movement speed, changing the movement route, and notifying the terminal user, excluding acquiring or transmitting shooting information.

[0049] —Shooting Information Acquisition Unit 106— The imaging information acquisition unit 106 acquires second imaging information at the reference position when it detects information regarding a change at the reference position and the control mode stored in the control mode information DB 121 is "imaging information acquisition mode".

[0050] Typically, the first and second photographic information pertains to the same subject. As mentioned above, if the difference between the first and second feature points exceeds a predetermined threshold (i.e., if a change at the reference location is detected), it is highly likely that the features contained in the map information and the first photographic information are different. In such cases, the second photographic information pertaining to the same subject as the first photographic information is used to update the map information.

[0051] Furthermore, differences in the land features include both differences in the land features themselves (for example, when a building is constructed on vacant land, or when a building is demolished and the land becomes vacant) and differences in the state of the land features (for example, when the design of a building or the tenants change).

[0052] The shooting information acquisition unit 106 acquires the second shooting information stored in the shooting information DB 324 of the terminal 14. The acquired second shooting information is stored in the shooting information DB 128.

[0053] ―Update section 108― When the shooting information acquisition unit 106 acquires the second shooting information, the update unit 108 updates the map information based on the second shooting information.

[0054] Updating map information means replacing the existing map information with new information by analyzing and processing the second set of photographic data. The analysis includes recognizing features included in the second set of photographic data and extracting their shape (including three-dimensional shape), color, pattern, etc. Processing includes enlargement, reduction, cropping, interpolation, and modification of shape, etc.

[0055] Specifically, the recognition of a geographic feature involves determining its type and name from the second set of photographic information. For example, if the second set of photographic information is image data of "○○ Elementary School," this includes associating the type "school" and the name "○○ Elementary School" with that second set of photographic information.

[0056] Extracting the shape of geographic features, etc., specifically involves understanding the shape of geographic features, etc., from the second set of photographic information. For example, this includes creating a six-view drawing or 3D model of a building from image data of the building captured by the second set of photographic information.

[0057] One example of updating map information is to generate a 3D model of a building by analyzing second-party image data (image data) related to the building, and then use this model to update the map information, which is a 3D map. A specific example of updating map information will be described later using Figure 11.

[0058] Updating map information includes updating map information stored in other devices. In this embodiment, the map information server 12 will be described as updating the map information DB 222 based on the second photographic information stored in the photographic information DB 128.

[0059] —Communications Department 110— The communication unit 110 communicates with the map information server 12 and the terminal 14 via the network interface unit 140. Specific examples of the information transmitted and received will be described later using Figure 4, etc.

[0060] The network interface unit 140 communicates with the map information server 12 and the terminal 14 via the communication network 16.

[0061] ―(2) Map information server 12―

[0062] The map information server 12 includes a control unit 200, a storage unit 220, and a network interface unit 240.

[0063] The memory unit 220 includes a map information database 222. Map information is stored in the map information database 222. The stored map information is acquired by the map information acquisition unit 302 of the terminal 14 and updated by the map information update unit 202.

[0064] The control unit 200 includes a map information update unit 202 and a communication unit 204.

[0065] —Map Information Update Department 202— The map information update unit 202 updates the map information stored in the map information DB 222 based on the second image information in response to a request from the update unit 108. A specific example of map information updating will be described later with reference to Figure 11.

[0066] —Communications Department 204— The communication unit 204 communicates with the device control server 10 and the terminal 14 via the network interface unit 240. Specific examples of the information transmitted and received will be described later using Figure 4, etc.

[0067] The network interface unit 240 communicates with the device control server 10 and the terminal 14 via the communication network 16.

[0068] —(3) Terminal 14— Terminal 14 includes a control unit 300, a storage unit 320, a network interface unit 340, an input unit 360, and an output unit 380. In this embodiment, terminal 14 is assumed to be a car equipped with a computer.

[0069] The memory unit 320 includes a map information DB 322, a shooting information DB 324, and a feature point information DB 326. Below are some examples of the information stored in each DB of the memory unit 320.

[0070] —Map Information DB322— The map information DB322 stores map information acquired by the map information acquisition unit 302 from the map information server 12. The stored map information is used by the feature point information generation unit 306 of the device 14 to generate information about the first feature point.

[0071] —Filming Information DB324— The imaging information DB324 stores imaging information generated by the imaging unit 304. The stored imaging information is used by the feature point information generation unit 306 to generate information about the second feature point, and is also acquired by the imaging information acquisition unit 106 of the device control server 10.

[0072] —Feature Point Information DB326— The feature point information DB326 stores information about feature points generated by the feature point information generation unit 306 on the terminal 14. The stored information about feature points is acquired by the feature point information acquisition unit 102 on the device control server 10.

[0073] The control unit 300 includes a movement control unit 301, a map information acquisition unit 302, an imaging unit 304, a feature point information generation unit 306, and a communication unit 308.

[0074] —Operation Control Unit 301— The motion control unit 301 performs a predetermined operation, which includes responding to changes specified by the execution unit 105. The predetermined operation includes, for example, changing the movement speed, changing the movement route, and notifying the terminal user, excluding acquiring or transmitting shooting information.

[0075] —Map Information Acquisition Unit 302— The map information acquisition unit 302 acquires map information from the map information DB 222 of the map information server 12. The acquired map information is stored in the map information DB 322.

[0076] —Photography Department 304— The imaging unit 304 generates first and second imaging information by operating the imaging device 362. The generated imaging information is stored in the imaging information DB 324.

[0077] —Feature point information generation unit 306— The feature point information generation unit 306 generates information about feature points based on the map information stored in the map information DB 322 and the first image information stored in the image information DB 324. The generated information about feature points is stored in the feature point information DB 326.

[0078] —Communications Department 308— The communication unit 308 communicates with the device control server 10 and the map information server 12 via the network interface unit 340. Specific examples of the information transmitted and received will be described later using Figure 4, etc.

[0079] The network interface unit 340 communicates with the device control server 10 and the map information server 12 via the communication network 16.

[0080] The input unit 360 includes an imaging device 362 and a GPS 364. The imaging device 362 is used by the imaging unit 304 to acquire imaging information. The GPS 364 is used to acquire the location information of the terminal itself. The location information is associated with the imaging information and stored in the imaging information DB 324. In addition to two-dimensional coordinates consisting of longitude and latitude, the location information may also include information such as the altitude at which the terminal 14 is located and the direction in which the terminal 14 is facing.

[0081] The output unit 380 includes a car navigation system 382. The car navigation system 382 displays various information to the user of the terminal 14.

[0082] <3. Operation Description> An example of the operation of the device control system 1 according to this embodiment will be explained using Figure 3-11. Figure 3 is a diagram illustrating an example of the operation of the device control server 10 according to the control mode when information regarding a change is detected. Figure 4-9 is a diagram illustrating an example of the operation when the device control server 10 activates the execution unit 105. Figure 10-11 is a diagram illustrating an example of the operation when the device control server 11 activates the image information acquisition unit 106.

[0083] —(1) An example of operation according to the control mode— Using Figure 3, an example of the operation of the device control server 10 according to the control mode when information about a change is detected will be explained. In this example, it is assumed that the change information acquisition unit 104 has already detected information about a change at the "start" stage. An example of the operation of the feature point information acquisition unit 102 and the change information acquisition unit 104, which are in the stage prior to detecting information about a change, will be described later using Figure 4-9.

[0084] First, the device control server 10 obtains the control mode setting by referring to the control mode information DB 121 (S12). If the control mode is set to "predetermined operation mode", proceed to S14. If the control mode is set to "shooting information acquisition mode", proceed to S16.

[0085] When the control mode is set to "predetermined operation mode", the device control server 10 activates the execution unit 105 (S14). That is, the device control server 10 instructs the terminal 14 to perform a predetermined operation, including responding to a change, and the terminal 14 activates the operation control unit 301 accordingly.

[0086] When the control mode is set to "shooting information acquisition mode", the device control server 10 activates the shooting information acquisition unit 106 (S16). In other words, the device control server 10 typically acquires second shooting information from the terminal 14.

[0087] After the device control server 10 acquires the second image capture information, the device control server 10 activates the update unit 108 (S18). Specifically, the device control server 10 typically causes the map information server 12 to update the map information.

[0088] —(2) An example of operation when the control mode is set to "predetermined operation mode"— Referring to Figure 4-9, an example of the operation of the device control system 1 when the control mode is a "predetermined operation mode" will be explained. Figure 4 is a sequence diagram of the device control system 1, including the device control server 10, the map information server 12, the first terminal 14a, and the second terminal 14b. Figure 5 is a diagram showing an example of map information acquired by the first terminal 14a. Figure 6 is an example of information about feature points generated based on the map information. Figure 7 is a diagram showing an example of photographic information acquired by the first terminal 14a. Figure 8 is an example of information about feature points generated based on the photographic information. Figure 9 is an example of a method for detecting changes in map information and photographic information by comparing information about feature points.

[0089] The first terminal 14a acquires map information using the map information acquisition unit 302 (S102). The first terminal 14a is a terminal that generates first image information and also generates information about feature points. On the other hand, the second terminal 14b is a terminal that performs predetermined operations in response to requests from the device control server 10.

[0090] Figure 5 shows an example of acquired map information. The map information DB322 of the first terminal 14a includes map information relating to rectangular prism-shaped buildings (hereinafter referred to as "map information 322a") and map information relating to truncated square pyramidal buildings (hereinafter referred to as "map information 322b").

[0091] Here, we assume that the rectangular prism-shaped building is located at position L1, and the truncated pyramidal building is located at position L2. That is, in map information DB322, position L1 is associated with map information 322a, and position L2 is associated with map information 322b.

[0092] In this example, it is assumed that the map information DB322 stores a 3D map. Specifically, the map information DB322 stores 3D models of various objects along with their location information, and map information 322a and map information 322b are 3D models of a rectangular prism-shaped building and a truncated pyramidal building, respectively. Information stored in this format can be used, for example, to create a metaverse space.

[0093] Furthermore, the acquisition of map information does not have to be frequent (e.g., every second or every minute), but may be done at relatively long intervals (e.g., every day or every week). For example, map information for all buildings in Tokyo may be acquired every week.

[0094] Map information typically does not need to be acquired and updated frequently. For example, since buildings are not usually rebuilt in short periods of time, such as one minute, it is preferable to acquire map information at somewhat longer intervals from the viewpoint of reducing communication and computational load.

[0095] The first terminal 14a, which has acquired map information, generates information about first feature points based on the map information using the feature point information generation unit 306 (S106). The generation of information about first feature points based on map information is performed, for example, by using image analysis.

[0096] Figure 6 shows an example of information regarding the generated first feature point. In this example, the items included in the feature point are the number of windows, the type of entrance, the height, the height-to-width ratio, and the shape of the front. The values ​​for each item in the first feature point of map information 322a are "6," "swing door," "12m," "2:1," and "rectangle," respectively. On the other hand, the values ​​for each item in the first feature point of map information 322b are "3," "automatic door," "9m," "3:2," and "trapezoid," respectively.

[0097] Typically, the amount of data related to the first feature point is less than or equal to the amount of data related to the map. For example, while the map information is 3D model data, the information related to the first feature point may be text data in JSON (JavaScript Object Notation) format. In this case, since text data generally takes up less data than 3D model data, the information related to the feature point can be handled with less communication and computation than the map information.

[0098] Just as the acquisition and updating of map information does not need to be frequent, the generation of information regarding the first feature point also does not need to be frequent. Typically, it is sufficient to generate information regarding the first feature point each time map information is acquired.

[0099] After information regarding the first feature point is obtained, the first terminal 14a generates first image information using the imaging unit 304 (S108). In this example, the first image information is assumed to be image data.

[0100] Figure 7 shows an example of the first image information that will be generated. The first terminal 14a generates first image information (hereinafter, "image information 324a") of a dome-shaped building 600 at location L3, which corresponds to the location associated with map information 322a (i.e., location L1). It also generates first image information (hereinafter, "image information 324b") of a truncated pyramidal building 602 at location L4, which corresponds to the location associated with map information 322b (i.e., location L2).

[0101] Here, we assume that the dome-shaped building 600 is located at position L1, and the truncated pyramidal building 602 is located at position L2. In other words, in the image information DB324, position L1 is associated with image information 324a, and position L2 is associated with image information 324b.

[0102] The features included in the first photographic information do not necessarily match the features associated with the map information. In this example, at location L1, there is a dome-shaped building according to photographic information 324a, while there is a rectangular-shaped building according to map information 322a.

[0103] If a feature changes (for example, is rebuilt) between the time the map information is generated and the time the first photographic information is generated, the map information will include information about the feature before the change, and the first photographic information will include information about the feature after the change.

[0104] Furthermore, as in this example, when the first captured information is image data, it is desirable to generate it at a reasonably high frequency (for example, every 0.1 seconds or every second). This is because the first terminal 14a is typically a mobile terminal, and if the generation frequency is low (for example, every minute or every hour), it will not be able to collect sufficient data on existing features.

[0105] Alternatively, the first image information may be generated not periodically, but triggered by the recognition of a feature. For example, the first image information concerning a building may be generated when the first terminal 14a approaches that building.

[0106] The first terminal 14a, having acquired the first image information, generates information about the second feature point based on the first image information using the feature point information generation unit 306 (S110). The generation of information about the second feature point based on the first image information is performed, for example, by using image analysis.

[0107] Figure 8 shows an example of information regarding the generated second feature points. In this example, the second feature points are assumed to have the same items as the first feature points (i.e., the number of windows, the type of entrance, the height, the height-to-width ratio, and the shape of the front). The values ​​for each item of the second feature point in the image information 324a are "6", "swing door", "9m", "3:2", and "dome-shaped", respectively. On the other hand, the values ​​for each item of the second feature point in the image information 324b are "3", "automatic door", "9m", "3:2", and "trapezoidal", respectively.

[0108] Typically, the amount of data related to the second feature point is less than or equal to the amount of data related to the first image. For example, while the first image is image data, the information related to the feature point is text data in JSON format.

[0109] Generally, since text data is smaller in size compared to image data, information about the second feature point can be handled with less computation and communication than the first image information.

[0110] The device control server 10 acquires information regarding the first and second feature points from the first terminal 14a using the feature point information acquisition unit 102 (S114).

[0111] As described above, typically, the amount of data related to the first and second feature points is less than that of the map information and the first image information, respectively. Therefore, by equipping the first terminal 14a with a feature point information generation unit 306, the amount of information transmitted from the first terminal 14a to the device control server 10 can be reduced. For example, transmitting information related to the first and second feature points, which are both text data, can reduce the amount of data transmitted compared to transmitting map information, which is 3D model data, and first image information, which is image data.

[0112] The device control server 10, having acquired information about the feature points, uses the change information acquisition unit 104 to acquire information about the changes, i.e., the difference between the information about the first feature point and the second feature point (S116), and determines whether or not there is a difference (S118).

[0113] Figure 9 illustrates an example of determining whether or not there is a difference. In this example, if the values ​​of the corresponding items for the first and second feature points match, the item is judged as "no difference." If they do not match, it is judged as "a difference exists." If there is one or more items judged as "a difference exists," it is determined that a difference exists.

[0114] Regarding the feature points of map information 322a and photographic information 324a at location L1, the number of windows and the type of entrance are judged as "no difference," but the height, the height-to-width ratio, and the shape of the front are judged as "difference." In this case, since there is one or more items judged as "difference," it is determined that the difference is greater than or equal to a predetermined threshold.

[0115] On the other hand, for the feature points of map information 322b and photographic information 324b at location L2, all items were determined to have "no difference". In this case, since there are no items determined to have a "difference", it is determined that the difference is below a predetermined threshold. Furthermore, two items being judged as "no difference" is not limited to cases where the items are completely identical; a certain degree of error may be permitted between the two items. The permissible amount of error may be determined based on the type of item and the performance requirements of the device control system 1, etc.

[0116] In this way, by having the device control server 10 equipped with a change information acquisition unit 104, the computational overhead between terminals can be reduced. If each terminal were equipped with a change information acquisition unit 104, multiple terminals would have to perform calculations to acquire change information for the same location, hindering the efficient use of computing resources. On the other hand, in the above example, because the device control server 10 performs the processing, calculations to acquire change information are performed only once for each location, allowing for the effective use of the computing resources of each terminal.

[0117] The device control server 10, having detected information regarding the change, determines the set control mode (S119). In this example, it is determined that the control mode is set to "predetermined operation mode".

[0118] When the control mode is set to "predetermined operation mode", the device control server 10 activates the execution unit 105 (see Figure 3). The execution unit 105 requests the second terminal 14b to perform a predetermined operation, including responding to a change (S120).

[0119] Upon receiving the request, the second terminal 14b performs a predetermined operation via the operation control unit 301 (S122). The predetermined operation is, for example, to notify the user of the second terminal 14b that there is a difference between the map information 322a and the photographic information 324a. Other possible actions include, for example, automatically reducing the speed of the second terminal 14b or changing the route provided by the car navigation system 382.

[0120] If there is a difference between the map information and the photographic information, it may be desirable for terminal 14 to behave differently than usual. For example, if terminal 14 is a car and it recognizes that a road is under construction (i.e., there is a difference between the map information and the photographic information), it is desirable for the car navigation system 382 to guide the user on a route that avoids the construction area.

[0121] Furthermore, by having the execution unit 105 cause a second terminal 14b, which is different from the first terminal 14a that acquired information about feature points, to perform a predetermined operation, more flexible device control can be achieved. For example, if the first terminal 14a is a car and recognizes that a road is under construction, it can cause the second terminal 14b, which is also a car, to select a different route via the device control server 10. In addition, for example, if a car accident occurs, the information about the accident can be shared with other terminals before it is registered in the car navigation system, and instructions can be given to reduce the speed of travel to prevent congestion.

[0122] When multiple terminals 14 are coordinated, they can be operated more efficiently than when each terminal 14 operates independently. For example, when each terminal 14 operates independently, an example is an autonomous vehicle that uses its onboard camera to recognize objects and drive. However, the events that can be recognized are limited to the range that the camera can capture, and if the camera malfunctions, the recognition performance deteriorates significantly. On the other hand, when multiple terminals 14 coordinate their operations via the device control server 10, each terminal can operate stably based on information that includes information outside of the field of view.

[0123] In summary, a program according to one aspect of the present invention provides a computer with a feature point information acquisition function 102 that acquires information about first and second feature points generated based on reference map information and first photographic information acquired by a first terminal at a reference position corresponding to the map information; a change information acquisition function 104 that acquires information about changes in the position detected based on the first and second feature points; and an execution function 105 that, when information about a change is detected, causes at least one of the first terminal and a second terminal different from the first terminal to perform a predetermined operation including responding to the change, the predetermined operation being the exception of acquiring or transmitting second photographic information at the reference position.

[0124] —(3) An example of operation when the control mode is "Shooting Information Acquisition Mode"— An example of operation when the control mode is "image information acquisition mode" will be explained using Figure 10-11. Figure 10 is a sequence diagram of the device control system 1 when the control mode is "image information acquisition mode". Figure 11 is an example of how map information is updated when a change is detected.

[0125] Steps S202 to S218 in Figure 10 are the same as steps S102 to S118 in Figure 4, so their explanation is omitted. In step S218, it is assumed that information regarding a change has been detected, similar to step S118. In this example, the second terminal 14b is the terminal that transmits the image information to the device control server 10.

[0126] The device control server 10, having detected information regarding the change, determines the set control mode (S219). In this example, it is determined that the control mode is set to "shooting information acquisition mode".

[0127] When the control mode is "shooting information acquisition mode", the device control server 10 activates the shooting information acquisition unit 106 (see Figure 3). The shooting information acquisition unit 106 requests the second terminal 14b to acquire the second shooting information (S220).

[0128] A request for acquisition means, for example, that the car navigation system 382 of the second terminal 14b notifies the user of the second terminal 14b that a change has been detected at location L2. The notification includes, for example, mapping the location where the change was detected onto a map.

[0129] Upon receiving the request, the second terminal 14b generates second imaging information at position L1 (corresponding to the position where the change was detected) using the imaging unit 304 (S222). Typically, the first imaging information and the second imaging information are information about the same object being photographed. Specifically, the second imaging information is the same as the first imaging information, i.e., the image data generated by photographing the dome-shaped building 600, i.e., the imaging information 324a.

[0130] The device control server 10 acquires the second image information from the second terminal 14b that captured the image (S224).

[0131] The device control server 10, which has captured the second image information, transmits the information to the map information server 12 (S228), and the update unit 108 causes the map information server 12 to update the map information contained in the map information DB 222 (S230).

[0132] Figure 11 shows an example of how the map information DB222 is updated. Before the update, the map information DB222 contains the map information acquired by the first terminal 14a in S202, namely map information 322a at location L1 and map information 322b at location L2 (see Figure 5).

[0133] Here, according to the results of S218, a change has been detected in the map information 322a for position L1 (see Figure 9), so the map information server 12 updates the map information based on the second image information (i.e., the image information 324a). At this time, the 3D model map information 324c is created based on the second image information, which is image data. Assuming that the updated map information 324c has been replaced, the updated map information DB 222 will contain both map information 324c and map information 322b.

[0134] In summary, the program according to this embodiment enables the computer to implement a feature point information acquisition function 102 that acquires information about first and second feature points generated based on reference map information and first photographic information acquired by the first terminal at a location corresponding to the map information; a change information acquisition function 104 that acquires information about changes at a location detected based on the first and second feature points; and a photographic information acquisition function 106 that acquires second photographic information at the reference location when information about changes is detected.

[0135] <4. Hardware Configuration> Referring to Figure 12, an example of a hardware configuration when the device control server 10 described above is implemented using a computer 70 will be explained. Note that the functions of each device can also be implemented by dividing them among multiple devices.

[0136] As shown in Figure 12, the computer 70 includes a processor 700, a storage device 702, an input interface 704, a data interface 706, a communication interface 708, and a display device 710.

[0137] The processor 700 controls various processes in the computer 70 by executing programs stored in the storage device 702. For example, each functional unit of the control unit 100 of the device control server 10 can be realized by the processor 700 executing programs temporarily stored in the storage device 702.

[0138] The storage device 702 is a storage medium such as RAM (Random Access Memory). RAM temporarily stores data necessary for program execution, such as the program code of the program executed by the processor 700.

[0139] The storage device 702 is also a non-volatile storage medium such as a hard disk drive (HDD) or flash memory. The storage device 702 stores various programs for realizing the above configurations, such as an operating system. In addition, the storage device 702 can also store tables for registering various information such as business information, and a database for managing those tables. Such programs and data are loaded into the storage device 702 as needed and referenced by the processor 700.

[0140] The Input I / F 704 is a device for receiving input from the user. Specific examples of the Input I / F 704 include keyboards, mice, touch panels, various sensors, and wearable devices. The Input I / F 704 may be connected to the computer 70 via an interface such as USB (Universal Serial Bus).

[0141] The data interface 706 is a device for inputting data from outside the computer 70. Specific examples of the data interface 706 include drive devices for reading data stored on various storage media. The data interface 706 may also be located outside the computer 70. In that case, the data interface 706 would be connected to the computer 70 via an interface such as USB.

[0142] The communication interface 708 is a device for communicating data with external devices of the computer 70 via a communication network 16, either wired or wirelessly. The communication interface 708 may also be located outside the computer 70. In that case, the communication interface 708 would be connected to the computer 70 via an interface such as USB.

[0143] The display device 710 is a device for displaying various types of information. Specific examples of the display device 710 include liquid crystal displays, organic EL (Electro-Luminescence) displays, and displays for wearable devices. The display device 710 may be located outside the computer 70. In that case, the display device 710 is connected to the computer 70 via, for example, a display cable. Furthermore, if a touch panel is used as the input I / F 704, the display device 710 can be configured as an integrated unit with the input I / F 704.

[0144] This embodiment is illustrative for explaining the present invention and is not intended to limit the invention to this embodiment alone. Furthermore, the present invention can be modified in various ways without departing from its essence. Moreover, those skilled in the art can adopt embodiments in which each of the elements described below is replaced with equivalent ones, and such embodiments are also included within the scope of the present invention.

[0145] Furthermore, the components of the identity verification device described in the above embodiment are such that a program stored in the storage device 702 is executed by the processor 700, thereby realizing a defined process in cooperation with other hardware. In other words, these components are envisioned as both software or firmware, and as corresponding hardware, and in both concepts, they may be described and interpreted as "function," "means," "part," "processing circuit," "unit," or "module," etc.

[0146] <5. Variation> Although the present invention has been described based on the embodiments described above, the following cases are also included in the present invention.

[0147] —(1) Variations regarding the functional configuration— In the above embodiment, the device control server 10, the map information server 12, and the terminal 14 were described as being single devices, but this is not limited to this. For example, the functions of the device control server 10 and the map information server 12 may be implemented by the same device, or the functions of the device control server 10 may be implemented by multiple devices. In addition, for example, the terminal 14 may have the functions of the device control server 10.

[0148] In the above embodiment, the first terminal 14a and the second terminal 14b were described as different terminals, but this is not limited to this. For example, the first terminal 14a and the second terminal 14b may be the same terminal, and both the first and second shooting information may be generated at that terminal.

[0149] In the above embodiment, it was explained that each device has a functional part as shown in Figure 2, but this is not limited to this. That is, a functional part provided by one device may also be provided by another device.

[0150] In the above embodiment, the map information acquisition unit 302 was described as being provided by the terminal 14, but this is not limited to that. For example, the device control server 10 may be equipped with the map information acquisition unit 302 and acquire map information from the map information server 12.

[0151] In other words, although the above embodiment described that the first terminal 14a acquires map information and generates information about the first feature point based on that map information, the first terminal 14a does not need to acquire map information or generate information about the first feature point. In this case, the device control server 10 generates the information about the first feature point.

[0152] This configuration can sometimes reduce the amount of data transmitted to send map information. For example, compared to sending map information from the map information server 12 to each of 100 terminals, sending it to a single device control server 10 can reduce the amount of data transmitted and received to 1 / 100.

[0153] In addition, although the above embodiment described the feature point information generation unit 306 as being provided by the terminal 14, it is not limited to this. For example, the device control server 10 may be equipped with the feature point information generation unit 306 and generate information about feature points based on map information and photographic information.

[0154] This configuration can sometimes reduce the amount of computation required on the terminal side. For example, compared to a case where each of 100 terminals generates information about the first feature point based on map information of a building at location L1, if a single device control server 10 that receives the map information generates the information about the first feature point, the amount of computation required to generate the information about the first feature point can be reduced to 1 / 100.

[0155] In addition, although the above embodiment described the change information acquisition unit 104 as being provided by the device control server 10, it is not limited to this. For example, a terminal 14 may be equipped with the change information acquisition unit 104 and generate change information based on information about feature points at a specific location. In this case, the terminal 14 may determine whether or not a change has occurred at that location and transmit change information to the device control server 10 according to the result of the determination.

[0156] This configuration can sometimes reduce the amount of communication from terminal 14 to device control server 10. Specifically, compared to sending information about feature points from terminal 14 to device control server 10 regardless of whether there is a change or not, sending information to device control server 10 only when a change is detected in terminal 14 can sometimes reduce the amount of communication to device control server 10.

[0157] In addition, although the above embodiment describes the update unit 108 as being provided by the device control server 10, it is not limited to this. For example, the map information server 12 may be equipped with the update unit 108 and update the map information DB 222 of the map information server 12.

[0158] In addition, although the above embodiment described the device control server 10 as activating either the execution unit 105 or the image acquisition unit 106 depending on the control mode (see Figure 3), it is not limited to this. Specifically, the device control server 10 may activate both the execution unit 105 and the image acquisition unit 106 when information regarding a change is detected. That is, the device control server 10 may cause the second terminal device 14b to perform a predetermined operation using the execution unit 105, and further acquire second image information using the image acquisition unit 106. Alternatively, the device control server 10 may consist of only one of the execution unit 105 or the image acquisition unit 106.

[0159] —(2) Modified example of the change information acquisition unit 104 — In the above embodiment, the presence or absence of a difference was determined based on whether the values ​​of each item of the first and second feature points matched, but this is not limited to this. For example, the information regarding the first and second feature points may be represented by a vector (e.g., a feature vector) in which each element is numerical, and the presence or absence of a change at the reference position may be determined based on the dot product of the respective vectors.

[0160] In the above embodiment, it was determined whether the values ​​matched for all items of the first and second feature points, but this is not limited to this. Specifically, the information regarding change may be generated by comparing a portion of the first feature point with at least a portion of the second feature point, or by comparing at least a portion of the first feature point with a portion of the second feature point. For example, the information regarding change may be generated by comparing only the "front shape" item among the items in Figure 9.

[0161] This configuration allows for the generation of information about changes with less computation. Specifically, when comparing only the shape of the front panel, the number of comparisons is reduced to 1 / 6 compared to comparing all six items in Figure 9: position, number of windows, type of entrance, height, height-to-width ratio, and front panel shape.

[0162] It is thought that humans acquire information about changes in geographical features in a similar way. For example, when humans recognize that a building has been rebuilt, they often make their judgment by focusing on a few items, such as the overall shape and color, rather than comprehensively considering multiple items such as the number of windows or the type of entrance.

[0163] The types and number of items used for comparison may vary depending on the type and condition of the feature. Specifically, the items used for comparison may be selected based on factors such as the type of feature, the design of the feature, and the movement speed of the terminal 14. For example, if the feature is a sign, the text on the sign may be recognized and used for comparison. In addition, for example, if the movement speed of the terminal 14 is slow, there is no need to perform calculations at high speed, so multiple items may be compared to ensure high-precision detection.

[0164] Furthermore, the change information acquisition unit 104 may acquire change information based on information about a first feature point stored in a long-term memory unit (e.g., HDD or SSD) and information about a second feature point stored in a short-term memory unit (e.g., main memory and cache memory) which has a shorter information retention period and a higher transfer speed than the long-term memory unit. The transfer speed includes at least one of the speed at which information is written to the memory unit and the speed at which information is read from the memory unit.

[0165] Information regarding the first feature point is based on map data and typically does not need to be updated frequently, so it can be stored in long-term memory. On the other hand, information regarding the second feature point is based on frequently acquired first image data and requires high-speed reading and writing. Therefore, it is desirable to store it in short-term memory, which has a shorter information retention period and higher transfer speed than long-term memory. Information regarding the first feature point stored in short-term memory may be deleted immediately after being used, for example, for calculating the difference (see Figure 9).

[0166] As described above, the change information acquisition unit 104 may detect information about changes by at least one of a simplified evaluation and a detailed evaluation. A simplified evaluation is a method that prioritizes the speed of calculation (or the amount of calculation) required to detect information about changes. For example, a simplified evaluation is a method of detecting information about changes based on a portion of the information about the first feature point stored in the long-term memory and a portion of the information about the second feature point stored in the short-term memory. On the other hand, a detailed evaluation is a method that prioritizes the accuracy of detection. For example, a detailed evaluation is a method of detecting information about changes based on all of the information about the first and second feature points, both of which are stored in the long-term memory.

[0167] Figure 13 is a diagram illustrating simplified and detailed evaluation. Figure 13(A) shows an example of simplified evaluation. Information regarding the first feature point is stored in the long-term memory unit 120a, which has a low transfer speed, and information regarding the second feature point is stored in the short-term memory unit 120b, which has a high transfer speed. According to the simplified evaluation, the change information acquisition unit 104 acquires information regarding change based on item A1, which is part of the information regarding the first feature point, and item A2, which is part of the information regarding the second feature point. Figure 13(B) shows an example of detailed evaluation. Information regarding both the first and second feature points is stored in the long-term memory unit 120a. According to the detailed evaluation, the change information acquisition unit 104 acquires information regarding change based on items A1-C1, which represent all of the information regarding the first feature point, and items A2-C2, which represent all of the information regarding the second feature point.

[0168] —(3) Modified example of the feature point information acquisition unit 102 — Furthermore, if the first terminal 14a is a mobile terminal, the ratio of the amount of data for the first image information to the amount of data for the second feature point information may vary depending on the speed at which the first terminal 14a moves. For example, with respect to the image information 324a, if the first terminal 14a is moving at 30 km / h, information for the feature point may be generated so that there are 5 items, as shown in Figure 8, and if it is moving at 60 km / h, information for the feature point may be generated so that there are 2 items.

[0169] For example, if the captured information is video data, it becomes necessary to process more captured information about features depending on the movement speed of the terminal 14. In such cases, the load on the terminal 14 can be reduced by changing the ratio of the amount of captured information data to the amount of feature point information data according to the movement speed.

[0170] In the above embodiment, the items included in the feature points were assumed to include the number of windows, the type of entrance, the height, the height-to-width ratio, and the shape of the front, but this is not limited to this. For example, the items of the first feature point may include the number of windows, the type of entrance, the height, the height-to-width ratio, and the shape of the front, while the items of the second feature point may include only the number of windows, the type of entrance, and the height. In other words, the items of the first feature point and the items of the second feature point do not have to be the same.

[0171] Furthermore, information regarding the first feature point may be stored in long-term memory, while information regarding the second feature point may be stored in short-term memory, which has a shorter information retention period and a higher transfer speed than long-term memory. Note that information regarding the first feature point may be stored in short-term memory at least temporarily, and information regarding the second feature point may be stored in long-term memory at least temporarily.

[0172] —(4) Modified version of the shooting information acquisition unit 106 — In the above embodiment, it was explained that when a change at the reference position is detected, the device control server 10 requests the second terminal 14b to transmit second image information at the reference position. In addition to this, information regarding the reference position may be transmitted to the map information server 12 so that the location where the change was detected is mapped on the map.

[0173] —(5) Other variations— In the above embodiment, terminal 14 was described as an automobile, but it is not limited to this. For example, it may be an autonomously operating drone, a smartphone, or a fixed-point camera. Alternatively, dedicated application software may be used to generate the image information.

[0174] Furthermore, each functional unit acquiring information may include having other devices acquire that information. For example, instead of the device control server 10 acquiring the information about the change, the change information acquisition unit 104 may have the map information server 12 acquire the information about the change. Alternatively, for example, instead of the device control server 10 acquiring the second image information, the image capture information acquisition unit 106 may have the map information server 12 acquire the second image capture information. [Explanation of symbols]

[0175] 1. Device control system 10 Device control servers 12 Map information server 14 devices 102 Feature Point Information Acquisition Unit 104 Change Information Acquisition Unit 105 Execution Unit 106 Shooting Information Acquisition Unit 108 Update Department

Claims

1. A program for controlling a first terminal equipped with a first mobile device, On the computer, A feature point information acquisition function that acquires information about a first feature point and information about a second feature point, The information regarding the first feature point is generated based on reference map information. The information relating to the second feature point is generated based on first photographic information acquired by the imaging device when a second terminal, which is equipped with a second mobile device and an imaging device different from the first mobile device, moves to a position corresponding to the map information by the second mobile device before the first terminal. Feature point information acquisition function, A change information acquisition function that acquires information regarding the change in the position detected based on the first feature point and the second feature point, When information regarding the aforementioned change is detected, a control function is provided to control the movement speed when the first terminal moves at the aforementioned position using the first moving device, A program that makes this a reality.

2. A program for controlling a first terminal equipped with a first imaging device and a first moving device, On the computer, A feature point information acquisition function that acquires information about a first feature point generated based on reference map information, and information about a second feature point generated based on first imaging information acquired by a second terminal equipped with a second imaging device different from the first imaging device at a location corresponding to the map information, A change information acquisition function that acquires information regarding the change in the position detected based on the first feature point and the second feature point, When information regarding the aforementioned change is detected, the system includes an imaging information acquisition function that acquires second imaging information at the aforementioned position via the first imaging device of the first terminal, A program that makes this a reality.

3. The program according to claim 1 or 2, wherein the information relating to the changes is information generated by comparing a portion of the first feature point with at least a portion of the second feature point, and comparing at least a portion of the first feature point with a portion of the second feature point.

4. The program according to claim 1 or 2, wherein the condition for detecting the change includes that the difference between the first feature point and the second feature point is greater than or equal to a predetermined threshold.

5. Information regarding the aforementioned feature point is stored in the long-term memory. The program according to claim 1 or 2, wherein the information relating to the second characteristic point is stored in a short-term memory unit which has a shorter information storage period and a higher transfer speed than the long-term memory unit.

6. The program according to claim 2, wherein the first shooting information and the second shooting information are information relating to the same object being photographed.

7. On the computer, The program according to claim 2, which implements an update function for updating the map information based on the second photographic information.

8. The program according to claim 1 or 2, wherein the amount of data relating to the first feature point and the second feature point is less than or equal to the amount of data relating to the map information and the first photographic information.

9. The program according to claim 1 or 2, wherein the information relating to the first feature point is not generated in the second terminal.

10. The program according to claim 1 or 2, wherein the information relating to the second feature point is generated in the second terminal.

11. The program according to claim 1 or 2, wherein the ratio of the amount of data of the first imaging information to the amount of data of the second feature point information varies according to the movement speed of the first terminal.

12. A storage unit for storing the program described in claim 1 or 2, A control unit that executes the aforementioned program, An information processing device equipped with the following features.

13. A storage unit for storing the program described in claim 1 or 2, An information processing system comprising a control unit that executes the aforementioned program.

14. An information processing method for causing a computer to execute the program described in claim 1 or 2.

Images