Information processing methods and computer programs

Abstract

The aim is to provide effective support for object positioning. [Solution] An information processing method is provided, comprising imaging object processing, virtual object processing, and comparison processing, wherein the imaging object processing acquires an imaging object, the virtual object processing acquires a virtual object, and the comparison processing acquires a comparison result between the imaging object and the virtual object.

Description

Technical Field

[0001] The present invention relates to an information processing method and a computer program executed on an information processing terminal.

Background Art

[0002] Patent Document 1 describes an information processing apparatus and the like related to image display of a building in construction work.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventionally, technologies for performing subject positioning using a terminal carried by a user have been developed. In such technologies, for example, a subject captured by a camera is displayed on a display screen, a measurement point is set on the subject on the display screen by a user operation, and the position of the measurement point in the real space is acquired. Here, effective support is required for the accuracy and burden reduction when the user sets a desired measurement point.

[0005] In addition, there is a need to grasp whether the situation in the real space such as a construction site matches or differs from the assumed state (design dimensions, etc.).

[0006] The object of the first invention according to the present application is effective support for subject positioning.

[0007] The object of the second invention according to the present application is to conveniently compare the situation in the real space with the assumed state.

Means for Solving the Problems

[0008] According to the first invention, an information processing method is provided that is performed by a portable information processing terminal, comprising display processing, positioning processing, and UI processing, wherein the display processing displays an image based on imaging data on a display screen and superimposes a UI image on the image based on imaging data, the image based on imaging data is a current image or a processed image obtained by processing the current image, the UI image is used to indicate a subject for which a positioning point is to be set, the positioning processing acquires subject position data which is the position data of the positioning point of the subject in response to an operation to set the positioning point of the subject, and the UI processing changes the UI image in response to the positioning status in the positioning processing or the content of the image based on imaging data in the display processing.

[0009] According to the first invention described above, subject positioning can be effectively supported.

[0010] According to the second invention, an information processing method is provided comprising imaging object processing, virtual object processing, and comparison processing, wherein the imaging object processing acquires an imaging object, the virtual object processing acquires a virtual object, and the comparison processing acquires a comparison result between the imaging object and the virtual object.

[0011] According to the second invention described above, the situation in the real space and the assumed state can be compared with ease. [Brief explanation of the drawing]

[0012] [Figure 1] This is an overall configuration diagram showing a user U carrying the information processing terminal 1 of the embodiment, and a positioning target Y in the field X (real space). [Figure 2] Figure 2A shows the rear side of the information processing terminal 1 (the terminal body 2 to which the location acquisition terminal 3 is attached), and Figure 2B shows its front side. [Figure 3] This diagram shows the hardware configuration of information processing terminal 1 in terms of functional blocks. [Figure 4]This is a functional block diagram showing how the computer program of the embodiment is executed on the information processing terminal 1. [Figure 5] This is a processing flow diagram illustrating the information processing method and computer program of the embodiment. [Figure 6] This is an example of a display screen 13 when user U attempts to set a positioning point P1 on the subject (upper top A) in the embodiment. [Figure 7] Figure 7A shows positioning points P1 to P8 and a three-dimensional shape Qr (an example of an imaged object) in the augmented reality space W corresponding to the site X (real space) in Figure 1. Figure 7B shows an example of a virtual object Qv. [Figure 8] Figure 8A shows an example of a web screen 200, a map screen 201, and a subject image 202 with location information uploaded to the cloud. Figure 8A is an enlarged view of the dashed frame B in Figure 8A. [Modes for carrying out the invention]

[0013] Embodiments of the present invention will be described below with reference to the drawings. The various features shown in the embodiments below can be combined with each other. Furthermore, each feature constitutes an independent invention.

[0014] 1. Description of the Configuration of the Embodiment 1-1. Overall Structure As shown in Figure 1, in this embodiment, as an example, user U measures the position of the target Y in a real-world location X using a portable information processing terminal 1. The information processing terminal 1 is a portable terminal. In this embodiment, the position acquisition terminal 3, described later, in the information processing terminal 1 is capable of communicating with at least one of the artificial satellite Z and the base station K. That is, the position acquisition terminal 3 may be configured to communicate with both the artificial satellite Z and the base station K, or it may be configured to communicate with only one of the artificial satellite Z and the base station K. In this embodiment, the target Y is a hexahedral concrete structure as an example, but this is just an example, and various objects with arbitrary sizes, shapes, and structures can be used as the target Y.

[0015] 1-2. Hardware Configuration 1-2-1. Appearance Configuration Figures 2A and 2B illustrate an example of the information processing terminal 1 (terminal body 2 with the position acquisition terminal 3 attached). The information processing terminal 1 includes the terminal body 2 and the position acquisition terminal 3. In the embodiment, as an example, the terminal body 2 is a smartphone. The position acquisition terminal 3 is attached to the terminal body 2. In the embodiment, as an example, the position acquisition terminal 3 can be attached to the back surface of the terminal body 2. And the position acquisition terminal 3 is preferably configured to be detachably connected to the terminal body 2, for example. The planar dimensions of the position acquisition terminal 3 are defined such that the portion other than the antenna 2b is substantially accommodated within the back surface of the terminal body 2. In addition, in the embodiment, the information processing terminal 1 is configured such that the terminal body 2 responsible for functions such as operations related to surveying and the position acquisition terminal 3 responsible for the function of acquiring position information are separate (independent) from each other. For example, by a simple connection operation of attaching the position acquisition terminal 3 to a generally used smartphone (terminal body 2), the terminal can be enhanced in functionality, and it has a configuration with high convenience, versatility, and usability, but it is not limited to this. The information processing terminal 1 may be an integrated terminal of the terminal body 2 and the position acquisition terminal 3.

[0016] In the position acquisition terminal 3 of the embodiment, various circuit configurations are housed in the plate-shaped housing 3a. In the embodiment, as shown in an example in Figure 2A, an antenna 3b is connected above the housing 3a, a battery 3c is provided within the housing 2a, and these are connected to the main circuit 3d that realizes functions such as the position acquisition function. Although an example of the form in which the position acquisition terminal 3 has the battery 3c is described, it is not limited to this, and the position acquisition terminal 3 may not have the battery 3c and may be configured to receive power supply from another terminal (for example, the terminal body 2), or may be configured to receive power supply from an external power source.

[0017] 1-2-2. Functional Blocks of Hardware Configuration As shown in FIG. 3, the terminal main body 2 includes a communication unit 10, a storage unit 11, a control unit 12, a display screen 13, a camera 14, and a LiDAR scanner 15, and these components are electrically connected via a communication bus 16 inside the terminal main body 2.

[0018] As an example, the communication unit 10 can adopt wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), and wired LAN network communication. Note that the communication unit 10 may be configured to be connected to the communication network 6 via wireless communication means such as wireless LAN network communication, mobile communication such as 3G / LTE / 5G, and Bluetooth (registered trademark) communication. Also, the communication unit 10 may be configured to use both the above-mentioned wired communication means and wireless communication means.

[0019] The control unit 12 is configured to execute processing and control related to the information processing of the information processing terminal 1. The control unit 12 can be constituted by, for example, a central processing unit (CPU). In the embodiment, the control unit 12 is an example of a processor capable of executing a program according to each step of the flowchart described later. The control unit 12 realizes various functions related to the information processing terminal 1 by, for example, reading out the program stored in the storage unit 11. Also, the information processing of the software in the information processing terminal 1 is realized by, for example, various programs stored in the storage unit 11 being processed by the control unit 12 as hardware.

[0020] In this embodiment, the display screen 13 is a touch panel. The touch panel has an integrated input unit and display unit. The display screen 13 is an input device that receives user U's input and also functions as a display device that displays various screens. The use of a touch panel is just one example and is not limited to it; the input unit that receives user U's input and the display unit that displays images, etc., may be configured separately. In this case, the input unit may be configured to receive input from, for example, an instruction unit (e.g., a button or lever) provided on the information processing terminal 1, or it may be configured to receive voice instructions (input) from a microphone. The display unit can be, for example, a liquid crystal display, an organic EL display, a plasma display, or various other display devices.

[0021] As shown in Figure 2A, the terminal body 2 of this embodiment has, as an example, three different cameras 14 and a LiDAR scanner 15, all arranged together in one place on the back of the terminal body 2. Of the three cameras 14, one may be a wide-angle camera, another an ultra-wide-angle camera, and another a telephoto camera. The user U may use these cameras to capture images of the positioning target Y. The configuration of the cameras 14 described here is merely an example and is not limited to this configuration.

[0022] As shown in Figure 3, the location acquisition terminal 3 can communicate with the storage unit 11 and other components within the terminal body 2 via the communication unit 10 and the communication bus 16. There are no limitations on the connection configuration between the location acquisition terminal 3 and the communication unit 10; any wireless connection can be used. Specifically, for example, Bluetooth® or Wi-Fi can be used. Furthermore, it is not limited to wireless connections; any wired connection (e.g., USB-TypeC) may also be used.

[0023] 1-3. Software Configuration (Functional Block Diagram) As shown in the functional blocks in Figure 4, the control unit 12 of the terminal body 2 includes a display unit 101, a positioning unit 102, a UI unit 103, an acquisition unit 104, an image capture object unit 105, a virtual object unit 106, and a comparison unit 107.

[0024] The display unit 101 performs various image processing related to the display screen 13. Figure 4 shows, as part of the images displayed by the display unit 101 in this embodiment, "image based on imaging data 101a," "UI image 101b," and "UI operation image 101c." Details of each image will be explained later.

[0025] The positioning unit 102 executes processing related to the positioning function of the information processing terminal 1. The positioning unit 102 can perform the process of setting a positioning point P and the process of acquiring subject position data Dy. The subject position data Dy is position data indicating the position of positioning point P. As specific examples of positioning point P, in this embodiment, positioning points P1 to P8 in Figure 7A and positioning point P9 in Figures 8A and 8B are exemplified. Specific examples of each process will be explained later.

[0026] The method by which the control unit 12 implements each function in Figure 4 is not particularly limited. In this embodiment, for example, the terminal body 2 is a smartphone. For example, an application program is downloaded from an external server to the terminal body 2 and installed on the terminal body 2, and the control unit 12 executes this application program to implement each function in Figure 4. However, this is not limited to this, and as another example, the program may be provided by being pre-stored on a non-temporary recording medium that can be read by a computer. Also, each function in Figure 4 is not limited to being implemented by software, but may also be implemented by hardware. Furthermore, in this embodiment, various information and the concepts encompassing it are handled, which are represented by high or low signal values ​​or qubits as a set of binary bits composed of 0s or 1s, and communication and calculations can be performed by the above-mentioned software or hardware configurations.

[0027] As shown in Figure 4, the location acquisition terminal 3 is communicated with each functional unit of the terminal body 2. The terminal location data Dt that the location acquisition terminal 3 can acquire may include, for example, latitude, longitude, and altitude, and in addition to these, it may also include, for example, geoid height. Furthermore, although the use of a coordinate system (geographic coordinate system) such as latitude and longitude is described here as an example, it is not limited to this, and a plane orthogonal coordinate system may also be used.

[0028] 1-4. Acquisition of various location data As shown in Figure 4, the information processing terminal 1 acquires terminal location data Dt, and specifically, in this embodiment, as an example, the location acquisition terminal 3 acquires the terminal location data Dt. The terminal location data Dt is the location data of the information processing terminal 1, and subject location data Dy is acquired based on this terminal location data Dt.

[0029] Terminal location data Dt can be obtained by communication with satellite Z, communication with base station K, or by performing SLAM (Simultaneous Localization and Mapping), and two or more of these three technologies may be used in any combination. In this case, RTK (Real Time Kinematic) may be used for communication between the location acquisition terminal 3 and satellite Z.

[0030] Specifically, as an example, the position acquisition terminal 3 may communicate with the artificial satellite Z (using, for example, RTK, GNSS, CLAS, or SLAS) to acquire terminal position data Dt. GNSS stands for Global Navigation Satellite System, CLAS for Centimeter-Level Augmentation Service, and SLAS for Submeter-Level Augmentation Service. As another example, a base station K capable of communicating with the position acquisition terminal 3 may be installed at the survey site, and the position acquisition terminal 3 may be configured to acquire terminal position data Dt from this base station K. In this case, the base station K is configured to acquire base station coordinates corresponding to the base station's location. The position acquisition terminal 3 can then acquire terminal position data Dt based on the base station coordinates and distance information.

[0031] The location acquisition terminal 3 can obtain base station coordinates from base station K. Distance information may be obtained by the location acquisition terminal 3 by calculating the distance information through communication between the location acquisition terminal 3 and base station K. Alternatively, base station K may communicate with the location acquisition terminal 3, calculate and obtain the distance information, and then transmit the distance information to the location acquisition terminal 3, thereby allowing the location acquisition terminal 3 to obtain the distance information.

[0032] Here, the distance information is based on the distance between base station K and location acquisition terminal 3. This distance information may, for example, represent the distance between base station K and location acquisition terminal 3 itself. Alternatively, it may represent, for example, the relative positional relationship between base station K and location acquisition terminal 3. This positional relationship can be understood, for example, as the relative position (X, Y, Z) of location acquisition terminal 3 with respect to base station K.

[0033] Furthermore, when base station K acquires base station coordinates, it can employ communication methods such as RTK, GNSS, CLAS, or SLAS.

[0034] To acquire terminal position data Dt, for example, coordinates obtained using inertial positioning with an inertial sensor may be used, or coordinates obtained using SLAM may be used. For inertial positioning, for example, an accelerometer or gyroscope built into the terminal body 2 may be used as the inertial sensor. For SLAM, SLAM using a camera 14 may be used, or SLAM using a LiDAR scanner 15 may be used. In SLAM, for example, coordinates can be obtained by combining the above-mentioned camera 14 or LiDAR scanner 15 with the above-mentioned inertial sensor.

[0035] In one embodiment, the information processing terminal 1 acquires relative position data Dr in addition to terminal position data Dt. Relative position data Dr is data based on the relative positional relationship between the information processing terminal 1 and the subject. In one embodiment, the relative position data Dr includes relative position and orientation. In one embodiment, the information processing terminal 1 acquires subject position data Dy based on terminal position data Dt and relative position data Dr.

[0036] In this embodiment, as an example, relative position data Dr is obtained by performing a LiDAR scan using the LiDAR scanner 15. The information processing terminal 1 (terminal body 2) calculates the coordinates of each point in the point cloud obtained by the LiDAR scan based on the terminal position data Dt, relative position, and orientation. The information processing terminal 1 (terminal body 2) also obtains subject position data Dy based on the coordinates of a specific point in the point cloud. Note that relative position data Dr may be obtained using any distance measuring sensor, etc., not just the LiDAR scanner 15.

[0037] The subject position data Dy obtained by the various methods described above may be stored in the storage unit 11 of the information processing terminal 1, displayed on the display screen 13, used in other processing, or transferred to an external location of the information processing terminal 1 via the communication unit 10.

[0038] 2. Description of the operation of the embodiment The operation of the embodiment will be explained using Figures 5 to 7B.

[0039] 2-1. Display Processing S1 In the processing flow of Figure 5, first, the display process S1 is executed, and the display screen 13 illustrated in Figure 6 is presented. Figure 6 is an example of the display screen 13 when user U attempts to set a positioning point in the embodiment. In the display process S1 of the embodiment, the display unit 101 displays an "image based on imaging data 101a" on the display screen 13 and superimposes a UI image 101b on the "image based on imaging data 101a". Furthermore, in the embodiment, as an example, a UI operation image 101c (UI operation buttons 101c1 to 101c8) is also superimposed on the display screen 13 (see Figure 6).

[0040] In Figure 6, UI operation button 101c1 is for setting position point P. UI operation button 101c2 is for opening the setting menu for various setting items. UI operation button 101c3 is for switching modes. UI operation button 101c4 is for canceling the operation of setting a position point. UI operation button 101c5 is for redoing the canceled position point setting operation. UI operation button 101c6 is for creating a new 3D object composed of position points. When this button is pressed, position points set from this point onward will be treated as vertices of the new 3D object. UI operation button 101c7 is for hiding various UI operation buttons. UI operation button 101c8 is for opening a menu for selecting the 3D model and position points to display on the screen. The screen in Figure 6 is an example and is not limited to this; for example, some of the above operation buttons may be omitted.

[0041] The "image 101a based on imaging data" may be a current situation image or a processed image. The current situation image is an image captured by the camera 14, and in this embodiment, it is an image of the site X. The processed image is a current situation image processed with any processing method. The image 101a based on imaging data is updated in real time when the user U points the camera 14 in any direction.

[0042] UI image 101b is used to indicate the subject on which to set the positioning point P. In one embodiment, UI image 101b includes an indicator figure 101b1 and an indicator point 101b2. The indicator figure 101b1 extends from a predetermined position toward the indicator point 101b2 within the display screen 13. In one embodiment, as shown in Figure 6, two straight lines (indicator figures 101b1) extend from the center of each vertical side of the display screen 13 toward the indicator point 101b2. Each straight line becomes thinner as it approaches the indicator point 101b2. In other words, as shown in Figure 6, the thickness of the tip (back) side of each straight line (indicator figure 101b1) corresponding to the indicator point 101b2 is thinner than the thickness of the front side. In other words, each straight line tapers towards the end, allowing the user to grasp the sense of depth and providing excellent convenience.

[0043] As a variation, the indicator figure 101b1 may move from a predetermined position toward the indicator point 101b2. For example, an image in which a straight line with a river-like flow is dynamically presented toward the indicator point 102b2 may be used. Alternatively, the indicator figure 101b1 may be replaced with another non-linear shape (e.g., a circle, sphere, triangle, pin, etc.), and this sphere image, etc., may move toward the indicator point 102b2. In this way, the indicator figure 101b1 is not limited to a straight line; any shape can be used.

[0044] The indicator point 101b2 points to the location where positioning point P is to be set. When user U presses the UI operation button 101c1 while indicator point 101b2 overlaps with the subject (specific part thereof), the position of that subject (specific part thereof) in the field X (real space) is acquired, as described in the next positioning process S2.

[0045] 2-2. Positioning Processing S2 In the positioning process S2 of this embodiment, the positioning unit 102 sets the positioning point P and acquires the subject position data Dy in response to user operation (pressing the UI operation button 101c1).

[0046] Please refer to Figure 6 for a concrete example. User U adjusts the direction of the camera 14 while viewing the display screen 13 in Figure 6, and aligns the indicator point 101b2 of the UI image 101b with the desired subject (upper top A in the example of Figure 6). In this state, User U performs the operation to set a positioning point P on the subject (upper top A) (by pressing the UI operation button 101c1). In the example of Figure 6, the indicator point 101b2 is superimposed on the upper top A, and when the UI operation button 101c1 is pressed in this state, a positioning point P1 is set on the subject (upper top A), and the subject position data Dy of positioning point P1 is acquired.

[0047] Please refer to Figure 7A. In the augmented reality space W of Figure 7A, positioning point P1 is set at a location corresponding to the upper top A of the real space (site X) in Figure 1. For the sake of explanation, positioning point P1 is shown in the augmented reality space W of Figure 7A and not in Figure 1, but positioning point P1 has real-world position information based on the subject position data Dy.

[0048] In one embodiment, as an example, when user U sets a positioning point P1 at the upper top A in Figure 6, the display unit 101 superimposes an image corresponding to the positioning point P1 (positioning point image) onto the display screen 13. As a result, the positioning point image is displayed in the augmented reality space W using AR (augmented reality display), and user U can see the positioning point P1 on the display screen 13. The positioning point image can be any image with any shape and color, for example, any point image or stake image.

[0049] In this embodiment, for example, when the operation to set a positioning point P in Figure 6 (by pressing the UI operation button 101c1) is performed, an image 101a based on the imaging data at that moment is also captured, and a "location-information-attached subject image" is obtained by superimposing the captured image and the positioning point image. This location-information-attached subject image can be saved, displayed, used in other processing, or transferred. Specific examples of location-information-attached subject images will be explained later using Figures 8A and 8B.

[0050] Figure 7A also shows examples of other positioning points P2 to P8. User U can capture images of each other peak of the positioning target Y with camera 14 and set each positioning point P2 to P8 on the display screen 13 in Figure 6, using each peak as the subject.

[0051] 2-3. UI Processing S3 In this embodiment, the positioning process S2 and the UI process S3 are executed in parallel. In the UI process S3, the UI image 101b is changed as follows, thereby providing improved convenience for the user U when performing positioning.

[0052] In the UI processing S3 of the embodiment, it is preferable that the UI unit 103 changes the UI image 101b (see Figure 6) according to the "positioning status in the positioning unit 102" or the "content of the image 101a based on the imaging data," as described below. In this UI process S3, at least one of the processes described in Section 2-3-1, Section 2-3-2, and Section 2-3-3 below will be executed. In other words, one of the processes in Section 2-3-1, Section 2-3-2, or Section 2-3-3 may be executed, two of these may be executed, or all of these may be executed.

[0053] 2-3-1. UI processing related to positioning status in positioning unit 102 "Positioning status" could specifically refer to, for example, "whether or not the subject is capable of distance measurement" or "the positioning accuracy of the subject." Whether or not distance measurement is possible may be determined based on, for example, the distance measured by the LiDAR scanner 15, or the presence or degree of diffuse reflection (e.g., on a water surface). "Positioning accuracy" may include, for example, positioning accuracy using LiDAR, or positioning accuracy associated with satellite reception sensitivity. "Positioning status" may include, for example, "reliability of positioning," and may also include, as an example, the reflectivity of the LiDAR scanner 15. "Positioning accuracy" indicates, for example, how accurate the position obtained by positioning is compared to the actual position (the expected degree of error), while "positioning reliability" indicates, for example, how stably the positioning result is maintained against factors such as reflection conditions on the measurement target, external noise, and weather. The method by which the control unit 12 calculates the positioning accuracy value (degree of error and degree of stability of positioning result) is not particularly limited, and known methods can be used.

[0054] For example, in the first case where distance measurement is possible, UI image 101b may be set as the first mode, and in the second case where distance measurement is not possible, UI image 101b may be set as the second mode. The modes described here are distinct modes that can be distinguished from each other, and one example is color. For example, if the mode is color, a configuration can be adopted in which the color of UI image 101b is green in the first case and gray in the second case. For example, in the first case where the positioning accuracy is greater than or equal to the first threshold accuracy, UI image 101b may be designated as the first embodiment; in the second case where the positioning accuracy is less than or equal to the second threshold accuracy (lower than the first threshold accuracy), UI image 101b may be designated as the second embodiment; and in the third case where the positioning accuracy is less than the first threshold accuracy and greater than the second threshold accuracy, it may be designated as the third embodiment. The embodiments described here are distinct embodiments that can be distinguished from each other, and one example is color. For example, if the embodiment is color, the color of UI image 101b may be green in the first case, gray in the second case, and yellow in the third case. In addition, the above has described embodiments divided into three cases, the first to the third, regarding positioning accuracy, but of course, it is not limited to these, and there may be two cases (i.e., one threshold) or four or more cases (i.e., three or more thresholds).

[0055] The above describes how the configuration can be changed in terms of distance measurement capability and positioning accuracy, but similarly, the configurations exemplified above may be changed depending on the level of positioning reliability, etc.

[0056] In the above explanation, the aspect was described as color, and therefore the color of UI image 101b is changed, but this is merely an example. In the embodiment, any kind of change in aspect that can be presented to the user U on the display screen 13 can be employed. In the embodiment, as an example, at least one of the following change aspects (f) may be changed. Change mode (f): Size, thickness, length, movement speed, quantity, shape, presence or absence of display, presence or absence of flashing, flashing time interval, rotation angle, operation, color and transparency of UI image 101b, and sound and vibration associated with UI image 101b.

[0057] 2-3-2. UI processing related to the content of image 101a based on imaging data The "content of image 101a based on imaging data" may specifically include, for example, the "condition of the subject." The "condition of the subject" may include the presence or degree of diffuse reflection by the LiDAR scanner 15. Alternatively, the "condition of the subject" may include, for example, if the subject (concrete, etc.) has cracks, the "location of the cracks, etc." may be automatically detected by AI (artificial intelligence), and the UI image 101b may be changed according to the detection result, and the coordinates of that location may be determined. Furthermore, regarding the "content of image 101a based on imaging data," any one or more of the aforementioned change patterns (f) can be adopted for the UI image 101b.

[0058] 2-3-3. Processing according to the distance to the subject (i) and processing to guide the subject (ii) Furthermore, the UI process S3 in the embodiment may include either process (i) or process (ii) described below. It is more preferable that both processes (i) and (ii) are performed.

[0059] In process (i), a process may be performed to change the UI image 101b according to the distance between the subject and the information processing terminal 1. The distance can be obtained, for example, from the relative position data Dr described above. In this process, for example, the closer the user U with the information processing terminal 1 gets to the subject (more precisely, the part that overlaps with the indicator point 101b2 on the display screen 13), the thicker the indicator shape 101b1 (UI image 101b) of the UI image 101b may be (may be made larger). Any one or more of the aforementioned change modes (f) can be adopted for the UI image 101b. Regarding process (i), a numerical value corresponding to the distance between the information processing terminal 1 and the subject may be displayed. This numerical value may be the distance itself (absolute value) or a relative value. Furthermore, the numerical value corresponding to the distance may change such that it decreases as the user U holding the information processing terminal 1 approaches the subject, and increases as the user moves away. The numerical value corresponding to the distance described here can be displayed, for example, above the indicator shape 101b1 or the indicator point 101b2 of the UI image 101b.

[0060] In process (ii), a process may be executed to change the UI image (which may be UI image 101b, or other UI images may be added) in order to guide the information processing terminal 1 to a predetermined subject location. The guidance direction may be determined, for example, using the orientation of the relative position data Dr described above. In this process, one or more of the aforementioned change modes (f) can be adopted for UI image 101b to indicate the positional relationship between the information processing terminal 1 and the predetermined subject location. For example, the blinking time interval of UI image 101b may be shortened as the information processing terminal 1 approaches the subject location, or any navigation UI image (e.g., arrow image, triangle, pin, etc.) indicating the guidance direction to the subject location may be added.

[0061] In one embodiment, for example, in UI processing S3, the UI unit 103 may change the UI image 101b in steps or continuously in accordance with the change in the relative positional relationship between the information processing terminal 1 and the subject. In this embodiment, for example, in combination with processing (i) or processing (ii) above, any one or more of the above change modes (f) can be changed in steps or continuously.

[0062] 2-4. Acquisition process S4 Through the processes S1 to S3 described above, user U can set positioning points P on any subject. In this embodiment, as an example, acquisition process S4 is executed using the set positioning points P (corresponding to positioning points P1 to P8 in Figure 7A). In acquisition process S4, the acquisition unit 104 of the information processing terminal 1 acquires (a) to (e) below using the subject position data Dy (specifically, through automatic calculation processing, etc.). In this embodiment, it is not necessary for all of (a) to (e) below to be acquired in acquisition process S4, but it is preferable that at least one is acquired. In this embodiment, it will be explained assuming that all are acquired.

[0063] (a) Coordinates of each positioning point P (In the example in Figure 7A, the coordinates of each positioning point P1 to P8 in real space (site X in Figure 1)) (b) The distance between each positioning point P from which coordinates have been obtained (for example, the distance L1 between positioning points P3 and P4 in the example in Figure 7A) (c) The area of ​​a planar figure whose vertices are each of the positioning points P whose coordinates have been obtained (for example, the area of ​​quadrilateral Mr consisting of positioning points P3, P4, P7 and P8 in the example of Figure 7A). (d) The volume or capacity of a three-dimensional figure whose vertices are each of the positioning points P from which coordinates have been obtained (in the example in Figure 7A, for the three-dimensional figure Qr whose vertices are positioning points P1 to P8, this is the volume if the three-dimensional figure Qr is a tangible object, or the capacity if the three-dimensional figure Qr is a hollow body). (e) The bearing, angle, or gradient between each positioning point P from which coordinates have been obtained. Furthermore, the 3D figure Qr corresponds to the object being imaged at the measurement site, and also corresponds to the imaged objects described later. Furthermore, regarding the point-to-point distance mentioned above, the distance components, such as horizontal distance and elevation difference, may be displayed separately, or conversely, only the point-to-point distance may be displayed without displaying these separately. Alternatively, at least one of the distance components, such as horizontal distance and elevation difference, may be displayed.

[0064] In one embodiment, augmented reality (AR) display is employed in connection with the acquisition process S4 described above. Specifically, for example, when the display unit 101 executes the display process S1, any image representing each of the above components (positioning points, lines, figures, directions, etc.) (positioning points, lines, figures, directions, etc.) may be superimposed on the display screen 13 (AR display) in response to a user request or automatically. For example, when any multiple positioning points P are set, a line image connecting them, a planar figure image connecting them, or a three-dimensional figure image with them as vertices may be superimposed on the display screen 13. Each of these images may also have the above numerical values ​​(a) to (e) displayed alongside it.

[0065] As an example, in Figure 7A, a three-dimensional figure Qr with vertices P1 to P8 is shown as a dashed line and superimposed on the positioning target Y. In Figure 7A, the three-dimensional figure Qr and the positioning target Y are slightly offset to make them distinguishable, but this is for convenience only. Ideally, the shape and size of the three-dimensional figure Qr should perfectly match those of the positioning target Y.

[0066] 2-5. Image Object Processing S5 Next, in the imaging object processing S5 of this embodiment, the imaging object unit 105 acquires an "imaging object". An imaging object is an object based on imaging data. An object is any digital element displayed in the digital space.

[0067] In this embodiment, as an example, a three-dimensional model representing the three-dimensional figure Qr shown in Figure 7A is adopted as the imaging object of the imaging object unit 105. The three-dimensional model of the three-dimensional figure Qr is just one example of an imaging object. This is because the three-dimensional figure Qr is composed of an image 101a based on imaging data with positioning points P1 to P8 set on it.

[0068] There are no limitations on the method of acquiring the imaging object; other acquisition methods are also acceptable. Any object (2D or 3D object) may be generated by any means, and this arbitrary object may be used as the imaging object. The imaging object may also be generated from point clouds, etc. For example, it may be generated by 3D scanning processing of the LiDAR scanner 15, SfM (Structure from Motion) processing, 3D reconstruction processing, photogrammetry processing, NeRF processing, or 3D Gaussian splatting processing. The object may be surface data or a 3D model, and may be a mesh or a solid.

[0069] The information processing terminal 1 may acquire an image object in the image object unit 105 using the 3D scanning process of the information processing terminal 1 (processing using the LiDAR scanner 15).

[0070] 2-6. Virtual Object Processing S6 In the virtual object processing S6 of this embodiment, the virtual object unit 106 acquires a virtual object Qv (see Figure 7B). In the information processing terminal 1 of this embodiment, the virtual object unit 106 generates a virtual object based on object data or based on a virtual shape set by the user in the virtual space.

[0071] "Object data" can be any digital data, such as any 2D or 3D model data, or design data (e.g., 2D or 3D CAD data). Terminal 2 may read object data from a designated cloud server (storage), for example. As an example, the virtual object Qv in Figure 7B is based on the design data of the positioning target Y (a concrete structure, for example).

[0072] A "virtual shape" is any shape set by user U. The display unit 101 may, for example, display a "virtual shape setting screen" on the display screen 13. On this setting screen, user U may input the virtual shape in the form of any line, plane, or solid, or superimpose the virtual shape onto the imaged object. In this embodiment, it may be possible to "divide an object by specifying any multiple points," and in this case, the lines and points used for division are also included in the virtual shape.

[0073] 2-7. Comparison process S7 In the comparison process S7 of this embodiment, the comparison unit 107 acquires the comparison result between the imaged object (for example, a three-dimensional figure Qr) and the virtual object Qv. The "comparison result" may be, for example, the superposition of multiple objects, or the difference between multiple objects. The display unit 101 may display, for example, a "comparison result presentation screen" on the display screen 13. Any display method can be used for the difference display; for example, the result of a Boolean operation on objects (e.g., subtraction) may be presented, or objects may be combined (union) and the non-overlapping difference parts may be visually presented (e.g., color coding or flashing display). Furthermore, the comparison result is not limited to a visual display in digital space, but may also be presented on the display screen 13 as text information, numerical values, etc. The acquired comparison result may be saved, displayed, used in other processing, or transferred.

[0074] In this embodiment, as an example, the comparison unit 107 aligns the position of the imaged object (for example, a three-dimensional figure Qr) with the virtual object Qv before acquiring the comparison result. By establishing a correspondence between the digital coordinate system and the coordinate system of the real space X (real coordinate system, geographic coordinate system), the virtual object Qv is precisely matched to the position of the imaged object.

[0075] After that, this processing flow will be completed.

[0076] 3. Description of the operation and effects of the embodiment As described above, according to the embodiment, when user U performs object positioning, UI processing S3 can effectively assist user U by changing the UI image 101b.

[0077] Furthermore, according to the embodiment, as explained in "1-4. Acquisition of various positional data" above, there is the advantage that various positional data necessary for positioning the subject can be appropriately acquired by making full use of various positioning technologies.

[0078] Furthermore, according to the embodiment, as explained in acquisition process S4, various information (a) to (e) can be obtained from the subject position data Dy, improving the convenience of the user U at the site X.

[0079] Furthermore, according to the embodiment, various improvements are made to the UI image 101b, as described in UI processing S3, etc. According to the embodiment, since the UI image 101b includes the instruction figure 101b1, good operability can be provided to the user U. Also, in the embodiment, by adopting the various change patterns listed in (f) above, or by providing stepwise or continuous changes, higher visibility can be provided to the changes in the UI image 101b. Also, in the embodiment, in UI processing S3, the UI image 101b can be changed according to various conditions that are helpful to the user U (whether or not distance measurement is possible, positioning accuracy, state of the subject, distance between the subject and the terminal, guidance to the subject's position).

[0080] Furthermore, according to the embodiment, objects can be compared with each other through the processes described in imaging object processing S5 to comparison processing S7. This makes it possible to compare the state of the site X (e.g., the three-dimensional figure Qr based on the positioning target Y in Figure 7A) with an arbitrary virtual object Qv (e.g., design data or an arbitrary virtual figure, etc.).

[0081] 4. Variations 4-1. Switching location data acquisition devices Terminal body 2 can also acquire terminal location data. The terminal location data of information processing terminal 1 is not limited to the terminal location data Dt of location acquisition terminal 3 (see Figure 2), but may also be the terminal location data of terminal body 2.

[0082] Depending on the accuracy of the terminal location data Dt of the location acquisition terminal 3 (in the example of this embodiment, the accuracy of RTK), it may be possible to switch from the terminal location data Dt acquired by the location acquisition terminal 3 to the terminal location data acquired by the terminal body 2, or conversely, it may be possible to switch from the terminal location data acquired by the terminal body 2 to the terminal location data Dt acquired by the location acquisition terminal 3. Specifically, if the predetermined conditions regarding the accuracy of the terminal location data of the location acquisition terminal 3 are met (in this example, if RTK accuracy can be ensured), the terminal location data Dt acquired by the location acquisition terminal 3 will be used as the terminal location data for the information processing terminal 1. If the predetermined conditions regarding the accuracy of the location data of the location acquisition terminal 3 are not met (in this example, if RTK accuracy cannot be ensured), the information processing terminal coordinates acquired by the terminal body 2 may be used as the location data. Depending on the accuracy of the terminal location data Dt from the location acquisition terminal 3, the terminal location data acquired by the terminal body 2 and the terminal location data Dt acquired by the location acquisition terminal 3 may be combined to form the configuration.

[0083] The above example describes switching between terminal location data acquisition devices (terminal body 2 and location acquisition terminal 3) or combining terminal location data depending on the accuracy, but it is not limited to this. For example, the terminal location data acquisition devices may be switched or terminal location data may be combined depending on various circumstances such as the processing speed of terminal body 2, the remaining battery level 2c, the balance between these, and user requests. In other words, it is not limited to the accuracy of the terminal location data, but rather it may be possible to switch between terminal location data acquisition devices or combine terminal location data depending on predetermined circumstances.

[0084] 4-2. Upload function to web cloud, etc. In addition to recording a photograph taken at site X (image 101a based on imaging data) and the subject's position (subject position data Dy) and displaying it on the display screen 13 using an application on the information processing terminal 1, a function may also be provided to synchronize this information with a server / web cloud and allow it to be viewed on a web screen 200. The web screen 200 can be viewed on any terminal (including personal computers, tablets, etc.) that can connect to the server / web cloud. In other words, the information processing method according to the embodiment may further include an acquired information display process. The acquired information display process is a process that acquires information from a storage unit such as a server / web cloud and displays the acquired information on the information processing terminal 1. The acquired information display process can be executed in parallel, independently of the control flow process described in Figure 5, for example. Here, the acquired information display process displays the following acquired information (A) to (C). (A) Image 101a based on imaging data (B) Subject position data Dy in image 101a based on imaging data (C) Positioning point image associated with the subject position data Dy of the subject in image 101a based on imaging data. The following provides a detailed explanation. The following example describes a configuration in which all of (A) to (C) are displayed, but it is not limited to this; at least one of (A) to (C) will be displayed.

[0085] Figure 8A is an example of a web screen 200. When the server is accessed, a map screen 201 is displayed. Clicking the pin icon on the map screen 201 displays a pop-up screen 201a. The pop-up screen 201a displays location information (e.g., shooting location coordinates XY, elevation, coordinate system) and a thumbnail image of the subject image 202 (corresponding to image 101a based on the imaging data). Clicking "View Details" at the bottom displays the subject image 202 and its detailed information (not shown). The subject image 202 is an image with location information (corresponding to subject location data Dy) attached. Figure 8B is an enlarged view of the dashed frame B, where the positioning point image (image of a point) of positioning point P9 is drawn on the subject image 202. This associates and records the location information of positioning point P9 with the subject image 202. The subject image 202 may also include information 202a indicating the orientation in which the subject image 202 was taken, along with the coordinate position (in the example of Figure 8A, it is "North").

[0086] The positioning point image of positioning point P9 may change in appearance (e.g., color or shape) depending on the radio wave conditions or positioning accuracy. For example, if the radio wave conditions or positioning accuracy are good, it may be green, and as they deteriorate, it may change in the order of yellow > gray. For example, green may indicate good radio wave conditions and positioning accuracy. In addition, a numerical value corresponding to the distance between the information processing terminal 1 and the subject may be displayed. This numerical value may be the distance itself (absolute value) or a relative value. The numerical value corresponding to the distance described here can be displayed, for example, at the bottom of the positioning point image of positioning point P9.

[0087] 4-3. Others The terminal body 2 may be a tablet device instead of a smartphone. Unlike laptop computers, smartphones or tablet devices are primarily provided as information processing terminals suitable for image capture and held in one hand. In typical use, the terminal is held in one hand, and the touch panel is operated with either the holding hand or the other hand. By attaching the small and lightweight position acquisition terminal 3 according to the embodiment to such a terminal body 2, positioning work can be easily performed with one hand. In particular, the position acquisition terminal 3 according to the embodiment is equipped with a battery 2c and an antenna 2b, and when attached to the terminal body 2, it is small enough to fit in one hand, thus providing high convenience for the user when moving during image acquisition. Furthermore, it can suppress a decrease in the mobility of the user's actions in civil engineering and construction work or infrastructure management sites (e.g., railways, roads, electricity, gas, water), or disaster sites. Furthermore, the embodiment can be applied to various fields in addition to civil engineering and construction sites, such as on-site surveys by civil engineering consultants, archaeological excavations, agricultural applications, and livestock research (e.g., recording the shape and location of cow dung). Moreover, while the user of the embodiment is not limited, it is suitable for, for example, civil engineering and construction workers, infrastructure workers, researchers, as well as land and building surveyors, media personnel, police, and the Self-Defense Forces.

[0088] In this embodiment, we have described the process as executing all of steps S1-S7 in the flow chart of Figure 5, but we are not limited to this. For example, it could be an information processing method (program) that executes steps S5-S7 of the flow. In other words, in the processing flow of Figure 5 above, only "imaging object processing S5, virtual object processing S6, and comparison processing S7" may be extracted and executed independently without being combined with other processes (S1-S4). This is because, as mentioned above, imaging objects can be acquired by various methods such as SfM processing. Furthermore, at least one of flows S1-S3 and flow S4 may be an additional information processing method (program) that is executed. Conversely, it may be an information processing method (program) that executes S1-S3 of the flow. Furthermore, at least one of the flows S4 and S5-S7 may be an additional information processing method (program) that is executed.

[0089] In display processing S1, when a subject corresponding to previously set subject position data Dy is included in the imaging range of the current display processing, a positioning point image associated with the previously set subject position data Dy may be superimposed and displayed on the image 101a based on the imaging data. The positioning point image is the same as that described in the embodiment, and is an image that is displayed in the augmented reality space W using AR (augmented reality display) and is visible on the display screen 13. Any image with any shape and color can be used, for example, any point image or stake image. In other words, in the display process S1 of the embodiment, a positioning point image corresponding to previously set subject position data Dy may be displayed. When a user has previously captured an image of a subject and set and saved a positioning point, a positioning point image will be set and saved at that positioning point. Then, when the user is currently positioning and points the camera again to include a subject, the previously set positioning point image will be displayed on the information processing terminal 1. This allows the user to easily understand the locations they have previously positioned.

[0090] In the positioning process, the constituent points that make up the depiction image are used as positioning points, and the position data for each subject is acquired. Furthermore, the depiction image (not shown) may be superimposed on the image 101a based on the imaging data and displayed. Here, the depiction image is an image that is depicted via the input device in the same image space as the UI image (i.e., the augmented reality space W). In other words, in the positioning process of the embodiment, when a user draws a shape or the like in the augmented reality space W using an input device such as an electronic pen, each point (the aforementioned constituent points) that makes up the drawn shape (drawn image) becomes a positioning point. Then, subject position data Dy is set and saved for each positioning point. As a result, even if the user changes the camera angle, the drawn image continues to be displayed in the augmented reality space W. It goes without saying that this depiction image may be displayed superimposed on the UI image 101b described in the embodiment. Furthermore, since positioning, display, and UI processing can be performed in parallel, although it is stated here that the depicted image is superimposed on the image 101a based on the captured data during the positioning process, the control flow can also be reorganized so that the superimposition occurs during the display process.

[0091] The functions described in section "4-2. Upload Function to Web Cloud, etc." may be implemented not only on the server / web cloud but also as functions of the information processing terminal 1 alone (the application of the information processing terminal 1). That is, in the application of the information processing terminal 1, clicking a pin icon in the map screen 201 displays a pop-up screen 201a, and the pop-up screen 201a may display location information, a thumbnail image of the subject image 202, etc. Also, the appearance of the positioning point image of the positioning point may change depending on the radio wave conditions or positioning accuracy. In addition, a numerical value corresponding to the distance between the information processing terminal 1 and the subject may be displayed. This numerical value may be the distance itself (absolute value) or a relative value. The numerical value corresponding to the distance described here can be displayed, for example, at the bottom of the positioning point image of positioning point P9.

[0092] 5. Addendum Various embodiments are illustrated below. The embodiments shown below can be combined with each other. [Note 1] An information processing method performed by a portable information processing terminal, It includes display processing, positioning processing, and UI processing. In the aforementioned display process, an image based on the captured data is displayed on the display screen, and a UI image is superimposed on the image based on the captured data. The image based on the aforementioned imaging data is either an existing image or a processed image obtained by processing the existing image. The aforementioned UI image is used to indicate the subject on which the positioning point is to be set. In the positioning process described above, in response to the operation to set the positioning point of the subject, subject position data, which is the position data of the positioning point of the subject, is acquired. The UI processing method involves changing the UI image according to the positioning status in the positioning process or the content of the image based on the imaging data in the display process. [Note 2] The information processing method according to claim 1, The information processing method involves the portable information processing terminal acquiring terminal location data, which is the location data of the portable information processing terminal, by communicating with an artificial satellite, communicating with a base station, or performing SLAM, and acquiring subject location data based on the terminal location data. [Note 3] The information processing method described in Appendix 2, The portable information processing terminal acquires relative position data in addition to the terminal position data, and acquires subject position data based on the terminal position data and the relative position data. An information processing method wherein the relative position data is data based on the relative positional relationship between the portable information processing terminal and the subject. [Note 4] The information processing method described in Appendix 3, The portable information processing terminal is an information processing method for acquiring relative position data, including relative position and orientation. [Note 5] The information processing method described in Appendix 4, The aforementioned portable information processing terminal is By performing a LiDAR scan, the relative position data is acquired. The coordinates of each point in the point cloud obtained by the LiDAR scan are calculated based on the terminal position data, the relative position, and the orientation. An information processing method for acquiring subject position data based on the coordinates of a specific point in the point cloud. [Note 6] An information processing method described in any one of the appendices 2 to 5, The aforementioned portable information processing terminal comprises a terminal body and a location acquisition terminal attached to the terminal body. The location acquisition terminal acquires the terminal location data, The terminal body is an information processing method for acquiring the subject location data. [Note 7] An information processing method described in any one of the appendices 1 to 6, Furthermore, the acquisition process is also included. The portable information processing terminal is an information processing method that, in the acquisition process, uses the subject location data acquired in the positioning process to acquire at least one of the following (a) to (e). (a) Coordinates of each of the aforementioned positioning points (b) distance between each of the positioning points whose coordinates have been obtained (c) The area of ​​a planar figure whose vertices are each of the aforementioned positioning points whose coordinates have been obtained. (d) The volume or capacity of a three-dimensional figure whose vertices are each of the aforementioned positioning points whose coordinates have been obtained. (e) The direction, angle, or gradient between each of the positioning points from which the coordinates have been obtained. [Note 8] An information processing method described in any one of the appendices 1 to 7, The aforementioned UI image includes an instruction shape, The aforementioned indicator figure extends from a predetermined position toward the indicator point within the display screen, or moves from the predetermined position toward the indicator point. An information processing method in which the indicated point points to the position where the positioning point is set. [Note 9] An information processing method described in any one of the appendices 1 to 8, The portable information processing terminal is an information processing method that changes at least one of the following (f) in the UI processing. (f) The size, thickness, length, movement speed, quantity, shape, presence or absence of display, presence or absence of flashing, flashing time interval, rotation angle, movement, color and transparency of the UI image, as well as the sound and vibration associated with the UI image. [Note 10] An information processing method described in any one of the appendices 1 to 9, The portable information processing terminal is an information processing method that, in the UI processing, changes the UI image in stages or continuously in accordance with changes in the relative positional relationship between the portable information processing terminal and the subject. [Note 11] An information processing method described in any one of the appendices 1 to 10, The portable information processing terminal provides an information processing method in which, in the UI processing, the UI image is changed according to whether the subject is measurable, the positioning accuracy of the subject, or the state of the subject. [Note 12] An information processing method described in any one of the appendices 1 to 11, The UI processing described above is an information processing method that performs either (i) or (ii) below. (i) Processing to change the UI image according to the distance between the subject and the portable information processing terminal. (ii) A process to change the UI image in order to guide the portable information processing terminal to a predetermined subject position. [Note 13] An information processing method described in any one of the appendices 1 to 12, The display processing method involves, when the subject corresponding to the pre-set subject position data is included in the imaging range of the current display processing, superimposing and displaying a positioning point image associated with the pre-set subject position data onto an image based on the imaging data. [Note 14] An information processing method described in any one of the appendices 1 to 13, In the positioning process described above, the constituent points constituting the depicted image are used as the respective positioning points, the subject position data is acquired for each of the constituent points, and the depicted image is superimposed on the image based on the imaging data and displayed. An information processing method wherein the aforementioned depiction image is an image depicted via an input device in the same image space as the aforementioned UI image. [Note 15] An information processing method described in any one of the appendices 1 to 14, It further includes image object processing, virtual object processing, and comparison processing. The aforementioned portable information processing terminal is In the aforementioned imaging object processing, an imaging object is obtained, In the virtual object processing described above, a virtual object is obtained, The comparison process described above is an information processing method that obtains the comparison result between the imaged object and the virtual object. [Note 16] The information processing method described in Appendix 15, The portable information processing terminal is an information processing method that, in the comparison process, acquires the comparison result after aligning the image object and the virtual object. [Note 17] The information processing method described in Appendix 15 or Appendix 16, The portable information processing terminal is used in the image object processing to acquire the image object using the 3D scanning process of the information processing terminal. [Note 18] An information processing method described in any one of the appendices 15 to 17, The portable information processing terminal is an information processing method that generates virtual objects based on object data or based on virtual shapes set by the user in a virtual space in the virtual object processing. [Note 19] An information processing method described in any one of the appendices 1 to 18, Further equipped with a function to display acquired information, The aforementioned information display process involves displaying at least one of the following (A) to (C) which are pieces of information that have been acquired. (A) Image based on the aforementioned imaging data (B) Subject position data of the subject in the image based on the imaging data (C) Positioning point image associated with the subject position data of the subject in the image based on the imaging data. [Note 20] An information processing method comprising image object processing, virtual object processing, and comparison processing, In the above imaging object processing, the imaging object is acquired, In the aforementioned virtual object processing, a virtual object is obtained, The comparison process described above is an information processing method that obtains the comparison result between the imaged object and the virtual object. [Note 21] A computer program that executes the information processing method described in any one of the appendices 1 through 20. [Explanation of Symbols]

[0093] 1: Information processing terminal 2: The main unit 2a: Enclosure 2b: Antenna 2c: Battery 3: Location acquisition device 3a: Enclosure 3b: Antenna 3c: Battery 3d: Main circuit 6: Communication Network 10: Communications Department 11: Storage section 12: Control Unit 13:Display screen 14: Camera 15: LiDAR scanner 16: Communications bus 101: Display section 101a: Image based on imaging data 101b :UI image 101b1: Indicator Figure 101b2 :Indication point 101c:UI operation image 101c1~101c8: UI operation buttons 102: Positioning Unit 102b2 :Indication point 103:UI section 104: Acquisition Department 105: Imaging object section 106: Virtual Object Section 107: Comparison Section 200: Web screen 201: Map screen 201a: Pop-up screen 202: Subject image (subject image with location information) 202a: Information indicating the orientation in which the image was taken. Dr: Relative position data Dt: Terminal location data Dy: Subject position data P, P1~P8: Positioning point Qr: Three-dimensional shape (an example of an object to be imaged) Qv: Virtual object U: User X: The scene (real space) Y: Positioning target Z:Artificial satellite A: Upper apex K:Base station W: Augmented Reality Space

Claims

1. An information processing method comprising image object processing, virtual object processing, and comparison processing, In the above imaging object processing, the imaging object is acquired, In the aforementioned virtual object processing, a virtual object is obtained, The comparison process described above is an information processing method that obtains the comparison result between the imaged object and the virtual object.

2. The information processing method according to claim 1, The aforementioned image object is acquired using a portable information processing terminal. An information processing method for acquiring position data of an image-captured object based on terminal position data of the aforementioned portable information processing terminal.

3. The information processing method according to claim 2, An information processing method for acquiring terminal location data by performing communication with an artificial satellite, communication with a base station, or SLAM using the aforementioned portable information processing terminal.

4. The information processing method according to claim 2, The aforementioned portable information processing terminal comprises a terminal body and a location acquisition terminal attached to the terminal body. An information processing method for acquiring position data of an imaged object using the positioning function of at least one of the position acquisition terminal and the terminal body.

5. The information processing method according to claim 2, An information processing method in which a positioning point can be set on the portable information processing terminal, and the imaged object is configured using the positioning point.

6. The information processing method according to claim 5, An information processing method that uses the aforementioned portable information processing terminal to acquire at least one of the following (a) to (e). (a) Coordinates of each of the aforementioned positioning points (b) The distance between each of the positioning points whose coordinates have been obtained. (c) The area of ​​a planar figure whose vertices are each of the positioning points from which the coordinates were obtained. (d) The volume or capacity of a three-dimensional figure whose vertices are each of the positioning points from which the coordinates have been obtained. (e) The direction, angle, or gradient between each of the positioning points from which the coordinates have been obtained.

7. The information processing method according to claim 6, An information processing method comprising superimposing and displaying at least one of the acquired (a) to (e) onto the imaged object on the display screen of the portable information processing terminal.

8. The information processing method according to claim 5, An information processing method in which, when the set position point is included in the imaging range of the portable information processing terminal, a position point image indicating the set position point is superimposed and displayed on the display screen of the portable information processing terminal.

9. The information processing method according to claim 1, An information processing method comprising displaying the comparison result between the imaged object and the virtual object obtained in the comparison process on the display screen of a portable information processing terminal.

10. The information processing method according to claim 1, An information processing method for obtaining the comparison result after aligning the image object and the virtual object in the comparison process.

11. The information processing method according to claim 1, An information processing method for acquiring an imaged object in the imaged object processing, using a three-dimensional scanning process with LiDAR or an SfM process with a camera.

12. The information processing method according to claim 11, The SfM processing is an information processing method in which the processing is performed in photogrammetry, NeRF, or 3D Gaussian splatting.

13. The information processing method according to claim 1, An information processing method for generating a virtual object based on object data or based on a virtual shape set by a user in a virtual space, in the virtual object processing described above.

14. The information processing method according to claim 13, The screen displays the settings screen for the virtual shape, An information processing method that accepts input of the virtual shape by the user on the aforementioned settings screen.

15. The information processing method according to claim 13, An information processing method for displaying the virtual figure superimposed on the imaged object on a display screen.

16. A computer program that performs the information processing method described in any one of claims 1 to 15.

Images