Information processing device, control method for information processing device, and program

Abstract

The objective is to provide a system for appropriately managing images of objects, including the object itself. [Solution] The information processing device includes an acquisition means for acquiring an object image including an object photographed by a photographing means and location information of the object; a storage control means for storing the object image and the location information of the object in association; and a display control means for controlling the display of a list of object images in group form when an operation to display a list of object images is performed, to display thumbnails of object images associated with the location information of the object indicating a location within a predetermined range.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, a control method for the information processing apparatus, and a program.

Background Art

[0002] Currently, there is a movement to promote the preparation of data related to road environments and road infrastructure facilities.

[0003] Patent Document 1 discloses a method of acquiring a photographed image and its photographing position, displaying a mark indicating the photographing position on a map, and displaying a reduced photographed image corresponding to the mark.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the method disclosed in Patent Document 1 shows the position of the imaging device and does not show the position of the imaging object. When attempting to appropriately manage the number of imaging objects using the captured image, it is necessary to confirm whether the same imaging object has been imaged.

[0006] Therefore, an object of the present invention is to provide a mechanism for appropriately managing an object image including an object.

Means for Solving the Problems

[0007] [[ID=四十九]] The information processing device includes an acquisition means for acquiring an object image including an object captured by a photographing means and location information of the object; a storage control means for storing the object image and the location information of the object in association; and a display control means for controlling the display of a list of object images in group form when an operation to display a list of object images is performed, to display thumbnails of object images associated with the location information of the object indicating a location within a predetermined range. [Effects of the Invention]

[0008] According to the present invention, object images including the object can be appropriately managed. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of the overall system configuration in an embodiment of the present invention. [Figure 2] This is a block diagram showing an example of the hardware configuration of various devices. [Figure 3] This is an example of a flowchart related to the processing of this system. [Figure 4] This diagram shows an example of a configuration when an imaging device is mounted on a vehicle. [Figure 5] This figure shows an example of a detection image (first image) from a convex mirror. [Figure 6] This figure shows an example of a detection image (second image) from a convex mirror. [Figure 7] This is an example of a diagram showing the diameter and center of a rectangle related to a convex mirror. [Figure 8] This figure shows the position of a vehicle and an example of the captured image when a convex mirror is photographed every few meters. [Figure 9] This figure shows an example of how to calculate a multiplier approximation formula. [Figure 10] This figure shows an example of a method for calculating angles. [Figure 11] This figure shows an example of a method for calculating the distance β between the imaging device and the convex mirror. [Figure 12A]This is a diagram showing an example of a screen to be modified on a map application. [Figure 12B] This is a diagram showing an example of a screen to be modified on a map application. [Figure 12C] This is a diagram showing an example of a screen to be modified on a map application. [Figure 12D] This is a diagram showing an example of a screen to be modified on a map application. [Figure 12E] This is a diagram showing an example of a screen to be modified on a map application. [Figure 13] This is a diagram showing an example in which two or more different curved mirrors are installed. [Figure 14] This is a diagram showing an example of a method for checking whether driving for detecting a curved mirror in a specific area is being correctly performed. [Figure 15A] This is a diagram showing an example of a method for collectively checking when the same curved mirror is detected multiple times or when an image that is not a curved mirror is recorded due to misrecognition. [Figure 15B] This is a diagram showing an example of a method for collectively checking when the same curved mirror is detected multiple times or when an image that is not a curved mirror is recorded due to misrecognition. [Figure 16] This is an example of a flowchart regarding display processing.

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0011] FIG. 1 is a diagram showing an example of the overall configuration of the system in an embodiment of the present invention.

[0012] In this embodiment, the present invention will be explained using a system for recording and managing the number and location of convex mirrors within a predetermined area by detecting them, which is one example of the application of the present invention. Specifically, an imaging device 100 is installed on a vehicle and driven to various locations to photograph convex mirrors within a predetermined area. The size of the convex mirror is determined using the diameter of the convex mirror on the image detected from the captured image (in this embodiment, a round convex mirror is assumed, so the diameter is used, but for a square convex mirror, the vertical size (the vertical size is preferable to the horizontal size considering that it is photographed from an oblique direction) and the X-coordinate position, along with the latitude and longitude of the vehicle obtained by GPS 101. Then, the latitude and longitude of the convex mirror are calculated based on the determined size. This makes it possible to record the number and location of convex mirrors and create a management ledger.

[0013] In this system, the in-vehicle system 10, the cloud system 20, and the office system 30 are connected and can communicate with each other via the network 110.

[0014] The in-vehicle system 10 includes an edge computer 102 that receives images captured by an imaging device 100 mounted on the vehicle and vehicle location information from a GPS 101. The edge computer 102 is, for example, a personal computer (hereinafter referred to as PC). The edge computer 102 performs processes such as detecting the convex mirror and calculating the convex mirror's location information using the vehicle's location information. Any system capable of measuring the vehicle's location information other than GPS 101 is acceptable. The in-vehicle system 10 also includes a client terminal 105. The client terminal 105 can be any terminal capable of displaying the detected convex mirror images and location information, such as a PC or a mobile terminal such as a smartphone. The client terminal 105 controls the application server 103 to display the detected convex mirror images and location information on its display.

[0015] The cloud system 20 includes an application server 103 and a database 104. The application server 103 stores detection images and location information of convex mirrors received from the edge computer 102 into the database 104. The application server 103 also transmits the detection results and location information to be displayed on the display of the client terminal 105. The database 104 stores the detection images and location information of convex mirrors received from the application server 103.

[0016] The office system 30 includes a client terminal 105. The client terminal 105 can be any device capable of displaying the detected images and location information of the convex mirror, such as a PC or a mobile device like a smartphone. The client terminal 105 controls the display to show the detected images and location information of the convex mirror received from the application server 103. In this embodiment, the client terminal 105 is configured to send and receive information via the cloud system 20, but it is also possible to configure the in-vehicle system 10 and the office system 30 to communicate via the network 110 without going through the cloud system 20.

[0017] Next, referring to Figure 2, an example configuration of an edge computer 102, an application server 103, or a client terminal 105, which is an example of a device to which the present invention can be applied, is shown.

[0018] In Figure 2, the CPU 201, memory 202, non-volatile memory 203, image processing unit 204, display 205, operation unit 206, recording medium interface 207, external interface 209, communication interface 210, and camera unit 212 are connected to the internal bus 250. Each unit connected to the internal bus 250 is configured to exchange data with each other via the internal bus 250.

[0019] Memory 202 consists of, for example, RAM (volatile memory using semiconductor elements). The CPU 201 controls various parts using memory 202 as work memory according to a program stored in, for example, non-volatile memory 203. Non-volatile memory 203 stores image data, audio data, other data, and various programs for the operation of the CPU 201. Non-volatile memory 203 consists of, for example, a hard disk (HD) or ROM.

[0020] The image processing unit 204 performs various image processing operations on image data stored in the non-volatile memory 203 and recording medium 208, video signals acquired via the external I / F 209, image data acquired via the communication I / F 210, and captured images, based on the control of the CPU 201. The image processing operations performed by the image processing unit 204 include A / D conversion, D / A conversion, image data encoding, compression, decoding, resizing, noise reduction, and color conversion. The image processing unit 204 may be composed of dedicated circuit blocks for performing specific image processing operations. Depending on the type of image processing, the CPU 201 may also perform image processing according to a program without using the image processing unit 204. The process of recognizing objects to be recognized from an image is performed by the CPU 201 in cooperation with the image processing unit 204.

[0021] The display 205 displays images and GUI (Graphical User Interface) screens based on the control of the CPU 201. The CPU 201 generates display control signals according to the program and controls various parts of the edge computer 102 or client terminal 105 to generate video signals for display on the display 205 and output them to the display 205. The display 205 displays images based on the output video signals. The configuration of the edge computer 102 or client terminal 105 itself is limited to an interface for outputting video signals for display on the display 205, and the display 205 may be configured as an external monitor (such as a television).

[0022] The operation unit 206 is an input device for receiving user input, including text input devices such as keyboards, pointing devices such as mice and touch panels, buttons, dials, joysticks, touch sensors, and touchpads. The touch panel is configured planarly by being superimposed on the display 205, and is an input device that outputs coordinate information corresponding to the position of contact.

[0023] The recording medium interface (I / F207) allows for the insertion of recording media 208 such as memory cards, CDs, and DVDs. Based on the control of the CPU 201, it reads data from the inserted recording media 208 and writes data to the recording media 208. The external interface (I / F209) is an interface for connecting to external devices via wired or wireless cables and for inputting and outputting video and audio signals. The communication interface (I / F210) is an interface for communicating with external devices and the internet 211 to send and receive various data such as files and commands.

[0024] The camera unit 212 is a camera unit composed of an image sensor (imaging sensor) such as a CCD or CMOS element that converts an optical image into an electrical signal.

[0025] Next, with reference to Figure 3, an example of the curve mirror detection process in an embodiment of the present invention is shown. When the in-vehicle system 10 is started, the process shown in Figure 3 begins. The CPU 201 executes each step of the process. The edge computer 102 continuously acquires information from the imaging device 100 and GPS 101 while the vehicle is in motion and executes the process shown in this flowchart.

[0026] In S301, the CPU 201 acquires an image from the imaging device 100.

[0027] In S302, the CPU201 obtains the vehicle's latitude, longitude, and azimuth from the GPS101. The azimuth indicates the angle of the direction the imaging device 100 installed on the vehicle is facing. For example, if it is pointing north, it will be obtained as 0°.

[0028] Figure 4 shows an example of the mounting positions for the imaging device 100 and the GPS 101 antenna on the vehicle 401. The imaging device 100 is positioned in the front center of the roof of the vehicle 401, with the center of the field of view 402 facing the direction of travel. Specifically, the horizontal field of view is oriented so that the center of the field of view is in the direction of travel of the vehicle, and the vertical field of view is positioned so that the center of the field of view is parallel to the road in the direction of travel. The GPS 101 antenna is installed in a location close to the imaging device 100.

[0029] In S316, CPU201 uploads the latitude and longitude information of vehicle 401, along with processing time (hours, minutes, seconds, milliseconds), to application server 103 at intervals of 2 seconds.

[0030] In S303, CPU201 performs object detection processing. Specifically, CPU201 uses AI to detect the curve mirror 403 in the image. In this embodiment, a curve mirror is used for explanation, but any other object that requires counting results to be managed, such as signs, traffic lights, streetlights and other lighting fixtures, guardrails, utility poles, pillars, street trees, benches and other road accessories, may be used.

[0031] In S304, CPU201 determines whether or not the curve mirror 403 has been detected. If it has been detected, the process proceeds to S305; otherwise, it proceeds to S314.

[0032] Figure 5 shows an example of an image including a convex mirror and the positional relationship between the vehicle and the convex mirror at the time of shooting. Image 501 is an image taken by the imaging device 100 at the position of vehicle 503. If a convex mirror is detected in image 501, a rectangle 502 is displayed at the detection location.

[0033] In S305, if multiple curved mirrors are detected, CPU201 acquires a representative curved mirror. In image 501, two curved mirrors, rectangle 502 and rectangle 504, are detected. The vertical lengths of rectangle 502 and rectangle 504 are compared, and the curved mirror corresponding to rectangle 502, which has the larger length, is acquired as the representative curved mirror.

[0034] In S306, CPU201 calculates the diameter and X-coordinate position of the representative curved mirror. As shown in Figure 7, CPU201 calculates the diameter 701 (px) and X-coordinate position 702 (center position of the curved mirror: px) of rectangle 502 in image 501.

[0035] In S307, CPU201 calculates the distance α of the detected curve mirror using the diameter 701 calculated in S307 and the multiplier approximation formula.

[0036] The method for deriving the multiplier approximation formula, which is a prerequisite for processing S307, will be explained in detail using Figures 8 and 9. Note that the multiplier approximation formula should be prepared in advance before the processing of this flowchart begins. Figure 8 shows an example of photographing a convex mirror from different distances in order to derive the formula. Although Figure 8 is a view from above, for explanatory purposes, only the convex mirror is shown from the front. The starting point is a distance α 15m away from the convex mirror, and photographs are taken at 1m intervals from there, ending at a point 3m away. Distance α is a value identified as the distance β between the imaging device 100 and the convex mirror, and indicates how far the convex mirror and the imaging point are. For example, it is the distance 805 between the position 801 of the convex mirror and the position 802 of the vehicle obtained from GPS 101. The distance α between the convex mirror and the imaging point, and the diameter of the convex mirror in the image taken at that point are measured to create the graph in Figure 9. Note that when photographing the convex mirror from different distances α, the same imaging device 100 as the one mounted on the vehicle is used. Furthermore, using images taken under the same conditions (e.g., magnification), the distance α between the convex mirror and the shooting point, and the diameter of the convex mirror in the image taken at that point are measured.

[0037] The distance α between the actual position of the convex mirror 801 and the vehicle position 802 obtained from GPS 101 can be determined. For example, the diameter 701 of the convex mirror in the image 804 taken at point 803, 6m away, is obtained. The obtained value 901 is plotted on a graph with the distance α between the convex mirror and the shooting point on the y-axis and the diameter of the convex mirror on the x-axis. The multiplier approximation formula obtained from this graph is shown below. Note that there are two types of convex mirrors, one with a diameter of 60cm and the other with a diameter of 80cm, and the multiplier approximation formula needs to be obtained for each.

[0038] Formula 1 The multiplier approximation formula for a 60cm curved mirror is y = 565.6 x ^ -1.069 The multiplier approximation formula for a curved mirror at 80cm is y = 812.34 x ^ -1.082 y: distance α(m) x: Diameter of the convex mirror (px)

[0039] In this step, since the size of the convex mirror is not specified as 60cm or 80cm, the distance α (distance from the vehicle to the convex mirror) when the convex mirror is 60cm and the distance α when it is 80cm are calculated using Equation 1. If a convex mirror is detected in the image to be processed in S308, and a convex mirror is also detected in the image acquired immediately before (i.e., they are detected consecutively), the actual size of the convex mirror is determined using the calculation results from this step. If they are not detected consecutively in S308, the calculation results are set in the output work area in S312.

[0040] In S308, the CPU 201 determines whether or not the convex mirror has been detected consecutively. If it has been detected consecutively, the process proceeds to S309; ​​otherwise, it proceeds to S312. Figure 6 shows the positional relationship between the vehicle and the convex mirror and the captured image after the vehicle 503 has moved to the position of vehicle 603. Image 601 is an image taken by the imaging device 100 at the position of vehicle 603. In other words, in this step, it is determined whether or not the convex mirror 403 has been detected in the images taken before and after the vehicle's movement.

[0041] In S309, the CPU 201 determines whether the detected curve mirror is the same as the curve mirror detected just before. If it is the same, the process proceeds to S310; otherwise, it proceeds to S313. Specifically, it is determined that the mirrors are the same if the difference between the X-coordinate position 702 of the previous curve mirror and the X-coordinate position 702 of the current curve mirror is less than 200px, and the diameter 701 of the current mirror is the same as or larger than the diameter 701 of the previous mirror. The value less than 200px takes into account road conditions such as whether it is a straight or curved road, vehicle speed, and the shooting interval. For example, Figure 13 shows an example where different curve mirrors are installed on the left and right sides of a vehicle. A curve mirror 1303 is installed in front of vehicle 1301, and a curve mirror 1304 appears when turning right. When the vehicle is traveling in a straight line, the curve mirror 1303 is detected around the center of image 1305, and as the vehicle turns right, the curve mirror 1303 is detected on the left side of the image. Then, in image 1306, taken while turning right (after turning right), the convex mirror 1304, which was installed on the right side of the vehicle, is detected. At this time, when comparing thresholds of 100px, 200px, and 300px, if the threshold is less than 100px, even a slight shift in pixel value will cause it to be detected as a different convex mirror. In particular, on straight roads, the vehicle speed is relatively high, so the pixel value will shift significantly, and even if it is the same, it will be detected as a different convex mirror. If the threshold is less than 300px, even if the convex mirror 1303 is shifted to the left side, as in image 1306, it will be detected as the same convex mirror. In other words, if images in which the target convex mirror is shifted to the edge of the image are included, they will be images that are not suitable for presentation in the screen examples in Figures 12A to 12E described later (images in which it is difficult for the user to see the convex mirror). Therefore, the vehicle speed changes depending on whether it is a straight road or a curve, and if the threshold is less than 200px, it will be possible to detect it as the same convex mirror regardless of the road conditions. 200px is just an example; an appropriate value will be set considering factors such as vehicle speed, shooting interval, and pixel count.

[0042] In S310, CPU201 determines the size of the convex mirror. There are mainly two types of convex mirrors: those with a diameter of 60 cm and those with a diameter of 80 cm. In this step, it is determined whether the diameter of the convex mirror is 60 cm or 80 cm (whether it is an object of the first size or an object of the second size).

[0043] When the distance between a convex mirror and a vehicle is unknown, it is difficult to determine whether the diameter of the convex mirror is 60 cm or 80 cm. When the size of the convex mirror cannot be determined, it is unclear which multiplier approximation formula to use when using Formula 1. If the wrong formula is used in the calculation, errors will occur in the calculation results, and the accurate location will not be managed when recording and managing the number and position of convex mirrors within a designated area. Therefore, this step explains a method for determining the actual size of a convex mirror.

[0044] Let's take an example where, after image 501 (hereinafter referred to as the "first detection image") shown in Figure 5 is taken, vehicle 503 moves to the position of vehicle 603, and image 601 (hereinafter referred to as the "second detection image") shown in Figure 6 is taken. Here, we assume that the vehicle is traveling at a speed of 25 km / h, that images are taken every 0.5 seconds, and that the latitude and longitude are obtained from GPS 101 at that time. Also, let's assume that the diameter of the convex mirror 701 in the first detection image was 50 px. The calculation results using Equation 1 for the cases where the size of the convex mirror is 60 cm and 80 cm are shown below.

[0045] "Equation 1: Calculation result when x = 50" The multiplier approximation for a 60cm curved mirror is 565.6 * 50^-1.069 = 8.64m The multiplier approximation for a curve mirror at 80cm is 812.34 * 50^-1.082 = 11.78m

[0046] Next, using the latitude and longitude of the vehicles obtained from GPS101 in S302, the distance traveled by vehicle 503 to the position of vehicle 603 is calculated. Under the above assumptions, the distance traveled by the vehicles is 3.5m.

[0047] To determine "x: diameter of the convex mirror," we calculate the value using the inverse multiplier approximation formula shown below, which is a modified version of Equation 1. When the travel distance is the same, the calculation result of the 60cm inverse multiplier approximation formula tends to be larger than that of the 80cm inverse multiplier approximation formula. We utilize this characteristic to determine the size of the convex mirror. That is, we determine which size the convex mirror is in the second detection image 601 by looking at which of the 60cm and 80cm calculation results the diameter 701 of the convex mirror is closer to.

[0048] Formula 2 The inverse multiplier approximation formula for a 60cm curved mirror is x = ( y / 565.6 )^ (1 / -1.069 ) The inverse multiplier approximation formula for a 80cm curved mirror is x = (y / 812.34) ^ (1 / -1.082) y: First calculation distance α - Movement distance x: Estimated diameter of the convex mirror

[0049] "Formula 2: Calculation Example" The inverse multiplier approximation formula for a 60cm curved mirror is 81.3 = ((8.64 -3.5) / 565.6 )^ (1 / -1.069) The inverse multiplier approximation formula for a 80cm curved mirror is 69.24 = ((11.78-3.5) / 812.34) ^ (1 / -1.082)

[0050] In the calculation example above, the result for 60cm is 81.3, and the result for 80cm is 69.24. For example, if the diameter 701 of the convex mirror in the second detection image 601 is 80px, the result is close to that of 60cm, so the convex mirror is determined to be 60cm.

[0051] This makes it possible to determine whether the size of the convex mirror is the first size (60 cm in this embodiment) or the second size (80 cm in this embodiment). Specifically, there is a conventional method for determining the size of an object using the distance (focal length) between the imaging device 100 and the object. However, in this embodiment, the vehicle is constantly moving, and the distance between the convex mirror and the imaging device 100 fluctuates. If the distance between the convex mirror and the imaging device 100 is known, the size of the convex mirror can be determined using prior art, but it is difficult to always determine the distance between the two when the distance fluctuates. Therefore, by processing in this step, it is possible to determine the actual size of the object even when the distance between the imaging device 100 and the object is unknown. In other words, it is possible to determine whether the convex mirror is 60 cm or 80 cm, and when managing it in a ledger, it becomes possible to manage which size the convex mirror installed at a certain location is.

[0052] Furthermore, if the size of the convex mirror could not be determined, the accurate positional information of the convex mirror, as described later, could not be calculated. When the size of the convex mirror cannot be determined, one possible method for creating a graph like Figure 9 by measuring the distance α between the convex mirror and the shooting point and the diameter of the convex mirror in the image taken at that point is to plot the values ​​for 60 cm and 80 cm on a single graph and derive a multiplier approximation formula from the average. However, in order to calculate more accurate positional information, it is desirable to determine the size of the convex mirror before calculating the positional information.

[0053] In S311, CPU201 calculates the position information of the convex mirror. This will be explained in detail using Figures 8 to 11 and several mathematical formulas.

[0054] To calculate the position information of the convex mirror, the distance α(A)1101 obtained from the diameter 701 and the angle (B)1102 obtained from the X coordinate position 702 are required, as shown in Figure 11.

[0055] First, the distance α(A)1101, which is determined from the diameter 701, is calculated using the aforementioned formula 1. In S310, the size of the convex mirror is determined. Depending on whether the determined diameter is 60 cm or 80 cm, the corresponding multiplier approximation formula of formula 1 is used to calculate the distance. That is, if the object has a diameter of 60 cm, the multiplier approximation formula for a 60 cm object is used to calculate the distance α.

[0056] Next, the calculation of angle (B) 1102 obtained from the X-coordinate position 702 will be explained using Figure 10. The contents of Figure 10 were prepared in advance before this flowchart began. Angle (B) is calculated using the width of the captured image 1001 (1280px in this embodiment), the ratio of the X-coordinate position 702 1002, and the triangle 1003. The ratio of the X-coordinate position 702 1002 is determined by the ratio of the X-coordinate position 702 to the image width 1001. The calculated angle 1004 is calculated using this ratio 1002 and a function (inverse tangent) relating to the inverse triangle connecting the X-coordinate position 702 and the imaging device 100. Angle 1005 is calculated using the calculated angle 1004. The specific formula for determining the angle is shown below.

[0057] "Equation 3" y = math.degress(math.atan(a ×(math.abs(b - x / b)))) y: angle (°) x:X coordinate (px) a: For an image with an angle of 48°, the width is calculated by setting the image height to 1. = math.tan(math.radians(48)) = 1.11 b: Half the width of the image (e.g., 1280 / 2 = 640) math.degress:Angle transformation function math.atan: Inverse tangent function in radians math.abs: absolute value function math.tan: Tangent function in radians math.radians:radian conversion function

[0058] Finally, the process of calculating the position information of the representative curve mirror will be explained using Figure 11. Note that Figure 11 is a view from above, but for the sake of explanation, only the curve mirror is shown as a front view. In this step, the position information of the representative curve mirror is calculated using pre-prepared equations 1 and 2. Specifically, equation 1 calculates the distance α(A) 1101 obtained from the diameter 701, and equation 3 calculates the angle (B) 1102 obtained from the X coordinate position 702. Once the distance α(A) and angle (B) are determined, the distance β(1103) between the imaging device 100 and the curve mirror can be calculated using the sine function. Therefore, the position information of the curve mirror can be calculated from the following equation. Note that distance β is a value that is distinguished from the aforementioned distance α and represents the distance between the curve mirror and the imaging device.

[0059] Formula 4 Y = A / math.sin(math.radians(90°-B)) Y: Distance β (m) from the imaging device A: Distance α (m) calculated from the diameter B: Angle calculated from the X coordinate (°) math.sin:Sine function in radians math.radians:radian conversion function

[0060] When imaging is taken with an on-board imaging device in a vehicle, the distance to the convex mirror is constantly fluctuating. Furthermore, GPS101 can only acquire the vehicle's position, making it difficult to measure the distance to the convex mirror. According to the processing in this step, the distance β between the vehicle and the convex mirror can be calculated based on the X-coordinate position (center position) and diameter obtained from the image including the convex mirror. This makes it possible to calculate the position information of the convex mirror, and to record and manage the number and position of convex mirrors within a predetermined area.

[0061] In this embodiment, an example in which the imaging device moves has been described, but images captured by two imaging devices may also be used. That is, an image from the first imaging device and an image from the second imaging device are acquired, and the distance between each imaging device is calculated. The actual size of the object may then be determined using the image from the first device, the image from the second device, and the distance between the imaging devices.

[0062] In S312, CPU201 performs the process of setting the output work area. Specifically, the following information is set.

[0063] curved mirror size Curve mirror latitude Longitude of a convex mirror Detected image Detected image name Previous diameter Previous X coordinate 60cm previous distance α 80cm previous distance α

[0064] The size of the convex mirror will be set to either 60cm or 80cm. The latitude and longitude of the convex mirror will be set based on the position information calculated by S310. The method for calculating the latitude and longitude of the convex mirror is shown below.

[0065] First, using the azimuth angle obtained from GPS101 in S302, the azimuth angle of the installed convex mirror is determined. Different calculations are performed depending on whether the convex mirror is to the right or left of the center of the image. Specifically, if the X-coordinate position 702 of the convex mirror is greater than the X-coordinate indicating the center of the image, the calculation is "Azimuth angle of the convex mirror = Azimuth angle obtained from GPS101 + Angle obtained using Formula 3". If the X-coordinate position 702 of the convex mirror is less than or equal to the X-coordinate indicating the center of the image, the calculation is "Azimuth angle of the convex mirror = Azimuth angle obtained from GPS101 - Angle obtained using Formula 3". For example, if the azimuth angle obtained from GPS101 is 5° (north) and the angle is 30°, the azimuth angle of the convex mirror is calculated to be 35°. This indicates that the convex mirror is installed at a position 35° to the right, with the front of the vehicle being 0°. Next, the latitude and longitude of the convex mirror are calculated using the vehicle's latitude and longitude obtained from GPS101, the azimuth angle of the convex mirror calculated above, and the distance from the imaging device calculated using formula 4. For example, it is calculated using the function "(latitude of convex mirror, longitude of convex mirror = grs80.fwd(latitude of vehicle, longitude of vehicle, azimuth angle of convex mirror, distance from imaging device))". grs80.fwd is a function that calculates the latitude and longitude of a location located far from the current latitude and longitude in the direction of the azimuth angle. In this way, the latitude and longitude of the convex mirror are determined, and the position of the convex mirror can be managed.

[0066] The detected image is set to a file saved in an image file format (JPEG, PNG, GIF, etc.). The detected image name is set to the name assigned to the detected image. The previous diameter and previous X coordinate are set to the diameter 701 and X coordinate position 702 of the convex mirror in the first detected image 501. The previous distance α at 60cm and the previous distance α at 80cm are set to the calculation results obtained using formula 1 for the first detected image 501, respectively.

[0067] Meanwhile, in S313, the CPU 201 uploads the information set in the output work area to the application server 103. In this step, if curve mirrors are detected consecutively but are not identical, the information related to the first detected curve mirror is uploaded. For example, if curve mirror A, curve mirror A, and curve mirror B are detected in that order, the actual size of curve mirror A is determined and set in the output work area. Subsequently, if curve mirror B is detected consecutively, the information related to curve mirror A set in the output work area is uploaded to the application server 103.

[0068] In S314, CPU201 uploads the information set in the output work area to application server103. This is the same process as in S313. In this step, for example, if curve mirror A is detected in that order, and nothing else is detected thereafter, the information related to curve mirror A set in the output work area is uploaded to application server103.

[0069] In S315, CPU201 clears the information set in the output work area.

[0070] Next, the control method for the application server 103 will be described. The application server 103 is an example of an information processing device. In S313 and S314 of Figure 3, the CPU 201 of the application server 103 functions as an acquisition unit and acquires (receives) information set in the output work area from the edge computer 102. The information set in the output work area includes the latitude of the convex mirror, the longitude of the convex mirror, and the detected image, as shown in S312.

[0071] The detected image is an image of an object, including the object, captured by the imaging device 100. The object is, for example, a convex mirror, but is not limited to that. The imaging device 100 is an example of an imaging unit. The convex mirror latitude and convex mirror longitude are examples of the object's position information. The following describes an example where the object is a convex mirror.

[0072] Here, using Figures 12A to 12E, we will explain an example of the screen displayed and modified on the client terminal 105 after uploading. The application server 103 stores the information set in the received output work area in the database 104. That is, the CPU 201 of the application server 103 functions as a memory control unit and stores the detected image and the position information of the curve mirror in the database 104, associating them.

[0073] Database 104 assigns a unique identification number to the information set in the output work area. It also obtains addresses by sending latitude and longitude to the reverse geocoding API and registers the ward, city, county, and town name from the address as the primary key. In the example, the CPU 201 of application server 103 registers the town names of Ward X in Tokyo in database 104. When the CPU 201 of application server 103 receives a screen display request from client terminal 105, it sends all the information set in the output work area stored in database 104 to client terminal 105. As a result, the list of town names in Ward X screen 1201 shown in Figure 12A is displayed on the display of client terminal 105. Clicking the town name link 1202 transitions to the map application screen 1203. On the map application screen 1203, all convex mirrors detected within the town name are displayed as pins 1231 at their latitude and longitude locations.

[0074] The CPU 201 of the application server 103 functions as a display control unit and, based on the operation of the client terminal 105, controls the display of the client terminal 105 to display a map application screen 1203 on the client terminal 105, in which the pins 1231 of the convex mirror are placed at positions corresponding to the location information of the convex mirror. The pins 1231 are an example of an indicator.

[0075] Clicking any pin 1231 on the map application screen 1203 displays the update screen 1204 shown in Figure 12B. On the update screen 1204, you can check the detection results by comparing the detected image with the Street View on the map application and then update the status. Clicking the detection result link 1205 displays image 1207, and clicking the StreetView button 1206 displays Street View 1208, allowing you to compare and confirm the results.

[0076] The CPU 201 of the application server 103 controls the Street View 1208, which corresponds to the location information of pin 1231 on the map application screen 1203, to display pin 1232 at the location corresponding to pin 1231. Street View 1208 is an example of a photographic image.

[0077] The CPU 201 of the application server 103 controls the display of the map application screen 1203, the Street View 1208 corresponding to the location information of pin 1231, and the detected image 1207 associated with the location information of pin 1231, all on the same screen.

[0078] When the CPU 201 of the application server 103 operates the link 1202 which specifies the location of town J on the town name list screen 1201, it controls the map on the map application screen 1203 to display the location information pins 1231 that are included in the specified location of town J, and to not display the location information pins that are not included in the specified location of town J. The above location can be specified by town or district, for example, as on the town name list screen 1201 or the town name / district list screen 1401 in Figure 14.

[0079] Check if the type of the nearest convex mirror matches type 1209. If the type is different, change it to the correct type. For status 1210, open the update screen of another nearby pin 1233 to check and make a final decision. If pin 1231 is selected, set status 1210 to "Selected" and click the update button 1211 to update. Clicking on a nearby pin 1233 will display the update screen 1212 shown in Figure 12C. Clicking on the detection result link 1213 will display image 1214. After checking, set status 1215 to "Delete" and click the update button 1216 to update. If you have driven on the same road multiple times and detected the same convex mirror, there will be multiple "Delete" statuses for a single "Selected" status.

[0080] Pins and detected images with a "deleted" status are not deleted from database 104. Also, pins with a "deleted" status are displayed on the map and Street View 1208 of the map application screen 1203. On the map and Street View 1208 of the map application screen 1203, pins with an "adopted" status and pins with a "deleted" status have different colors from each other. After the modifier of the office system 30 modifies and verifies the pins and detected images using the client terminal 105, the CPU 201 of the application server 103 may delete pins and detected images with a "deleted" status from database 104.

[0081] The latitude and longitude of the convex mirror are calculated based on the position of the in-vehicle GPS 101 and the size and position of the convex mirror in the image, but there may be errors. In Street View 1217 in Figure 12D, the error in the position of the pin that has been "accepted" is corrected. The position of the pin that has been "accepted" in Street View 1217 is corrected by dragging and dropping 1218 to the position of the convex mirror displayed in Street View 1217. Alternatively, the position of the pin that has been "accepted" displayed on the map in the map application screen 1203 can also be corrected by dragging and dropping 1219. If the pin is moved at the boundary of a town name and the town name changes, an alert screen 1220 (changed from town A to town B) will be displayed. If OK is selected, the area will be changed to town B, and the pin will disappear from the map application screen for town A. To check and correct the position of a pin whose position has been changed, open the map application screen for town B, and you can check the pin there and perform the operation.

[0082] When a drag-and-drop operation 1218 is performed to correct the position of a pin in Street View 1217, the CPU 201 of the application server 103 controls the Street View 1217 to display the pin with the corrected position.

[0083] Furthermore, when the application server 103's CPU 201 is displaying the Street View 1217 with the pins placed on it, and a drag-and-drop operation 1219 is performed to correct the position of the pins on the map application screen 1203, the CPU 201 controls the map application screen 1203 to display the pins with the corrected positions.

[0084] Furthermore, the CPU 201 of the application server 103 controls the map on the map application screen 1203 so as not to display the corrected pin if an operation is performed to correct the location of the pin that is included in the location specified in the town name list screen 1201 of Figure 12A to a location that is not included in the location specified in the town name list screen 1201.

[0085] When there are two convex mirrors on the left and right sides of a road, there may be cases where individual detection images for each mirror are not available. In such cases, the pin showing both convex mirrors can be duplicated, and two pins can be placed at the locations of the two convex mirrors. Clicking the duplicate button 1221 in Figure 12E creates a new pin 1222.

[0086] When the duplicate button 1221 for duplicating pin 1231 is operated, the CPU 201 of the application server 103 duplicates the pair of pin 1231 and the detection image associated with the location information of pin 1231, and controls the display of the new pin 1222 on the map of the map application screen 1203. Operating the duplicate button 1221 is an operation to duplicate pin 1231 which is placed on the map of the map application screen 1203.

[0087] The modifier / verifier within the office system 30 can use the client terminal 105 to create a duplicate pin 1222 of pin 1231, as described above. Pin 1231 has a detection image registered that includes two convex mirrors. The duplicated pin 1222 also has a detection image registered that includes the same two convex mirrors. The modifier / verifier can use the client terminal 105 to modify the detection image registered in pin 1231 to include only the left convex mirror and register it, and to modify the detection image registered in pin 1222 to include only the right convex mirror and register it. The application server 103 may also be configured to automatically modify the detection image registered in pin 1222 when the detection image registered in pin 1231 is modified.

[0088] Next, using Figure 16, the processing method for display control in Figures 12A to 12E will be explained. This processing method involves cooperation between the client terminal 105 and the application server 103. Specifically, after the processing in Figure 3, the modifier / verifier modifies the curve mirror detection result using the client terminal 105. Note that each step is controlled by the CPU 201 of the client terminal 105, but steps that can be processed by the application server 103 may be controlled by the CPU 201 of the application server 103.

[0089] In S1601, the CPU 201 accesses (logs in to) the application server 103 using the browser or other interface on the client terminal 105. The application server 103 retrieves a list of target town names registered in the database 104, sends it to the client terminal 105, and displays it on the client terminal 105's display 205. An example of this display is shown in Figure 12A, which shows the town name list screen.

[0090] In S1602, CPU201 identifies the location information (range) based on the town name selected by the modifier / verifier in 1201, retrieves information on the convex mirrors in the target town name based on the location information of the convex mirrors stored in database104, and displays the location of the convex mirrors as pins on the map application screen (for example, 1203 in Figure 12A).

[0091] In S1603, CPU201 displays the screens shown in Figures 12B, 12C, 12D, and 12E in response to the actions of the modifier and verifier, and updates the curve mirror detection results. The updated information is reflected in database 104. The processing in Figures 12B, 12C, and 12E in this step is as described above. The pin movement process in Figure 12D will be explained below.

[0092] In S1604, CPU201 moves the pin position, i.e., the position of the curve mirror detection result, in Figure 12D according to the actions of the modifier / verifier.

[0093] In S1605, CPU201 determines whether the town name will change based on the position changed by the movement of the curve mirror detection result (e.g., 1219). If the town name will change, the process moves to S1607; otherwise, the process moves to S1606.

[0094] In S1606, CPU201 reflects the corrected latitude and longitude data in database104 when movement occurs within a town.

[0095] In S1607, CPU201 displays alert screen 1220 if the destination is a different town name than the one specified.

[0096] In S1608, CPU201 detects whether the corrector / confirmer pressed the OK button or the cancel button on the alert screen 1220, and determines whether to update or not. If the OK button was pressed and it is determined that an update should be performed, the process moves to S1610. If the cancel button was pressed and it is determined that an update should not be performed, the process moves to S1609.

[0097] In S1609, CPU201 resets the pin position of the detected curve mirror to its coordinates before the movement.

[0098] In S1610, CPU201 confirms the update and deletes the detection result data from database104 that was available before the town name change.

[0099] In S1611, CPU201 registers the detection result data after the town name change in database104.

[0100] In S1612, CPU201 retrieves the detection results for the target town name from database104 and redisplays the map application screen. As a result, the detection results for curve mirrors that have moved outside the target town name range will no longer be displayed.

[0101] Using Figure 14, an example of a method for verifying whether driving to detect convex mirrors in a specific area is being performed correctly is explained. In the in-vehicle system 10, the driver's assistant can use the client terminal 105 to check the vehicle's location information and the detected convex mirrors. The application server 103 stores the received vehicle's latitude and longitude information in the database 104. In the database 104, the processing time (hours, minutes, seconds, milliseconds) of the received vehicle's latitude and longitude information is used as a unique identification number. The database 104 also obtains the address obtained by sending the latitude and longitude to the reverse geocoding API and registers the ward, city, county, town name, and block number from the address as the primary key. In this example, the CPU 201 of the application server 103 registers the town names and block numbers for each ward in Tokyo X in the database 104. When the CPU 201 of the application server 103 receives a screen display request from the client terminal 105, it sends all the information stored in the database 104 to the client terminal 105. As a result, the list of town names and districts for District X, screen 1401 (Figure 14), is displayed on the client terminal 105's screen. Clicking on a town name or district link 1402 transitions to the driving history screen 1403. The driving history screen 1403 displays the latitude and longitude of the vehicle at 2-second intervals within each town (all districts) as a polyline. The driving history screen 1403 also implements a function to display the location of convex mirrors using pins from the map application screen 1203 (Figure 12A) and a function to display detected images of convex mirrors. By using the driving history screen 1403, it is possible to instantly check the driving history 1405 of the time traveled while driving through any town, thus preventing missed routes. It also allows for checking whether any convex mirrors have been overlooked. The driver's assistant can view the driving history on a tablet or similar device and instruct the driver to correct the route, such as correcting locations missed in the driving plan or returning to the same route. This enables correct driving in specific areas.

[0102] This makes it possible to identify where and how many convex mirrors are installed within a designated area. For example, local governments that install and maintain convex mirrors can easily create a ledger for managing them.

[0103] Next, the control method for the application server 103 will be explained. In S316 of Figure 3, the CPU 201 of the application server 103 acquires (receives) location information detected by the GPS 101 installed in the vehicle from the edge computer 102. The GPS 101 is an example of a location detection unit. The application server 103 is an example of an information processing device. The edge computer 102 is an example of an edge terminal.

[0104] In steps S313 and S314 of Figure 3, the CPU 201 of the application server 103 receives from the edge computer 102 an image of a detected object, including a curve mirror, captured by an imaging device 100 installed in the vehicle, and the position information of the curve mirror. The imaging device 100 is an example of an imaging unit. The curve mirror is an example of an object. The detected image is an example of an object image.

[0105] The CPU 201 of the application server 103 controls the display (display device) 205 of the client terminal 105 installed in the vehicle to display a map of the driving history screen 1403, which has pins 1404 of the convex mirror placed at positions corresponding to the position information of the convex mirror, and which represents the vehicle's driving history 1405 acquired based on the position information detected by GPS 101. Pins 1404 are an example of an indicator.

[0106] Next, the data collection method will be explained. The vehicle is equipped with an imaging device 100 for photographing convex mirrors installed on the road and a GPS 101 for detecting its position. Along the route the vehicle is traveling, the CPU 201 of the edge computer 102 installed in the vehicle uses the imaging device 100 to photograph the convex mirrors.

[0107] In step S303 of Figure 3, the CPU 201 of the edge computer 102 detects the curve mirror in the image obtained by the imaging device 100 while moving along the path.

[0108] In step S311 of Figure 3, the CPU 201 of the edge computer 102 calculates the position information of the convex mirror using the GPS 101.

[0109] In steps S313 and S314 of Figure 3, the CPU 201 of the edge computer 102 transmits the detected image of the curved mirror and the location information of the curved mirror to the application server 103.

[0110] The client terminal 105 installed in the vehicle accesses the application server 103 to display the detected image on the display (display device) 205 of the client terminal 105 inside the vehicle.

[0111] If the edge computer 102 fails to detect the convex mirror, the vehicle is moved, and the CPU 201 of the edge computer 102 uses the imaging device 100 to take another picture of the convex mirror.

[0112] While traveling along the route, the CPU 201 of the client terminal 105 accesses the application server 103 to display on the display 205 a map of the travel history screen 1403, where the pins 1404 of the convex mirrors detected by the edge computer 102 are positioned at locations corresponding to the convex mirror's location information.

[0113] In step S302 of Figure 3, the CPU 201 of the edge computer 102 transmits the location information detected by the GPS 101 to the application server 103.

[0114] While traveling along the route, the CPU 201 of the client terminal 105 accesses the application server 103 to display a map of the driving history screen 1403 on the display 205, which represents the vehicle's driving history 1405 acquired based on the location information detected by the GPS 101.

[0115] The CPU 201 of the client terminal 105 displays the pins 1404 of the curve mirrors detected by the edge computer 102 on the map of the driving history screen 1403 at the positions corresponding to the location information of the curve mirrors.

[0116] As described above, the detected images are stored in database 104 along with the location information of the convex mirror. Therefore, if it is later discovered that there was a problem with detecting the convex mirror or that a detection image was not taken, a vehicle must be prepared again, the vehicle must be driven to that location, and the detection image must be taken again.

[0117] According to this embodiment, the vehicle is equipped with an imaging device 100 and a GPS 101 and moves. The assistant inside the vehicle can use a client terminal 105 to display and confirm the detected image inside the vehicle. Since the assistant can display and confirm the detected image using the client terminal 105, they can retake the image while the vehicle is moving.

[0118] Using Figures 15A and 15B, an example of a method for batch verification when the same convex mirror is detected multiple times or when an image that is not a convex mirror is recorded due to misrecognition will be explained. In the office system 30, the corrector / verifier can use the client terminal 105 to view a list of detected convex mirror images. The application server 103 stores the information set in the received output work area in the database 104. The database 104 assigns a unique identification number to the information set in the output work area. The database 104 also obtains the address obtained by sending latitude and longitude to the reverse geocoding API and registers the ward, city, county, and town name from the address as the primary key. In the example, the CPU 201 of the application server 103 registers the town names of Ward X in Tokyo in the database 104. When the CPU 201 of the application server 103 receives a screen display request from the client terminal 105, it sends all the information set in the output work area stored in the database 104 to the client terminal 105. As a result, the town name list screen 1501 for Ward X, shown in Figure 15A, is displayed on the client terminal 105's display. Clicking the town name / district link 1502 transitions to the detected image list screen 1503. The detected image list screen 1503 displays a list of images of curve mirrors detected for each district. Initially, the detection results for district 1 are displayed. If there are no districts in an area, all images are displayed in the list. When displaying the list, the latitude and longitude of each detection result held in the database 104 are used to display detection results within a radius of 10m as one group, as in 1504. The algorithm for grouping detection results within a radius of 10m is shown below.

[0119] 1. Add the first detection result to the first group. The average latitude and longitude of the group will be the latitude and longitude of the first detection result. 2. Repeat the following for the second and subsequent detection results. 3. Calculate the distance to other groups using the Haversine formula. 4. If the distance is within 10m, add the detection result to the group. Update the group's average latitude and longitude. 5. If the distance to all other groups is not within 10m, create a new group. Use the average latitude and longitude of that group as the latitude and longitude of the detected result.

[0120] Clicking the thumbnail of any detected image on the detected image list screen 1503 displays the update screen 1505 (Figure 15B). On the update screen 1505, the system compares the images displayed on the detected image list screen 1503 with the detection results and updates the status. If the same road has been driven on multiple times and the same curve mirror has been detected, one image's status is changed to "Accepted" and the others to "Deleted". Clicking the detection result link 1506 displays the detected image 1507. After reviewing, select a status and click the update button to update.

[0121] The detected image list screen 1503 has a corresponding checkbox 1508 for each displayed image. By selecting the checkboxes 1508 and pressing the delete button 1509, you can delete the detection result status of the selected images all at once. This is useful when there are multiple images of the same convex mirror in the same group and you want to select one image, or to delete images that have been misidentified as something other than a convex mirror. The thumbnail 1510 of the detected image whose status has been updated to delete will be displayed in a grayed-out state, making it easy to identify at a glance.

[0122] The number 1511 displayed at the top of the image is an ID corresponding to the detection result, and it matches the number displayed on the curve mirror pin in the map application screen 1203 in Figure 12A mentioned above. Editers and reviewers using the office system 30 can view the detected images of the curve mirror pins displayed on the map application screen 1203 at a glance by using the detection image list screen 1503 in a separate tab while the map application screen 1203 is open. The color of the number 1511 displayed at the top of the image corresponds to the status of the detection result, and it matches the color of the curve mirror pin displayed on the map application screen 1203.

[0123] This allows correctors and reviewers to instantly see images of the same convex mirror that were detected multiple times, as well as images that were misidentified as something other than a convex mirror. It eliminates the need to check each detection result individually from the map application screen 1203, and a function to delete unnecessary detection results in bulk is provided, thus reducing the workload for correctors and reviewers.

[0124] This section describes how to control the application server 103. The application server 103 is an example of an information processing device.

[0125] In steps S313 and S314 of Figure 3, the CPU 201 of the application server 103 receives from the edge computer 102 a detection image including the convex mirror captured by the imaging device 100, and the position information of the convex mirror derived using the vehicle's position information.

[0126] The CPU 201 of the application server 103 stores the detected image and the position information of the convex mirror in association with each other in the database 104.

[0127] When the application server 103's CPU 201 operates the link 1502 for displaying a list of detected images (object images) as shown in Figure 15A, it controls the client terminal 105's display 205 to group and display thumbnails of detected images associated with the location information of a convex mirror indicating a location within a predetermined range, as shown in the detected image list screen 1503 of Figure 15A. The predetermined range is, for example, within a radius of 10m. Group 1504 indicates one group within the predetermined range. If there is only one convex mirror in group 1504, it is necessary to set the status of "accepted" for one detected image in group 1504 and the status of "deleted" for the other detected images.

[0128] The CPU 201 of the application server 103 functions as a setting unit, and when the delete (reject) button 1509 is pressed on the thumbnail of a displayed detected image, it sets the status of the detected image to delete (reject). Note that the status of delete (reject) can also be set by updating the "delete" status on the update screen 1505 in Figure 15B.

[0129] When the CPU 201 of the application server 103 performs operations on the checkbox 1508 and the delete button 1509 for selecting thumbnails of multiple detected images to be deleted (rejected) within group 1504, the CPU 201 sets the status of the selected multiple detected images to "delete (rejected)".

[0130] As shown in Figure 15B, the CPU 201 of the application server 103 displays thumbnails 1521 of detected images that do not have a deleted (rejected) status set, and thumbnails 1510 of detected images that have a deleted (rejected) status set, differently. For example, thumbnails 1510 of detected images are displayed grayed out. However, as shown in Figure 15, if the detection result link 1506 is clicked, the detected image 1507, which is not grayed out, is displayed. Pins and detected images with a deleted (rejected) status are not deleted from the database 104.

[0131] While the example described shows pins and detected images being set to delete (reject) status together, this is not the only way. The above operation can also be used to set the delete (reject) status only for detected images. However, in that case, the pins corresponding to those detected images must also be set to delete (reject) status individually afterward.

[0132] As shown in Figure 15A, when the link 1502 for displaying a list of detected images at a specified location is accessed, the CPU 201 of the application server 103 controls the display of thumbnails of detected images associated with the location information of convex mirrors included in the specified location, as shown in the detected image list screen 1503. Link 1502 is, for example, a link for displaying a list of detected images at the location of "J town". The location is specified, for example, by town or district.

[0133] When the thumbnail of a detected image is clicked, the CPU 201 of the application server 103 controls the display 205 of the client terminal 105 to display the update screen 1505 shown in Figure 15B. Then, when the operation of the detection result link 1506 shown in Figure 15B is performed, the CPU 201 of the application server 103 controls the display of the detected image 1507 corresponding to the thumbnail of the detected image.

[0134] The CPU 201 of the application server 103 controls the display of thumbnails of detected images along with their corresponding image number, 1511. Number 1511 is an example of an ID.

[0135] The CPU 201 of the application server 103 controls the display of detection image number 1511 that does not have a deletion (rejection) status set and detection image number 1511 that has a deletion (rejection) status set, using different colors.

[0136] The CPU 201 of the application server 103 can control the display 205 of the client terminal 105 to display a map application screen 1203, as shown in Figure 12A, in which the pins 1231 of the convex mirror are placed at positions corresponding to the location information of the convex mirror.

[0137] The CPU 201 of the application server 103 controls the display of the pins 1231 of the convex mirror corresponding to detected images that do not have a deleted (rejected) status set, and the pins 1231 of the convex mirror corresponding to detected images that have a deleted (rejected) status set, to be in different colors.

[0138] For detected images that do not have a "delete (rejected)" status, the detection image number 1511 and the convex mirror pin 1231 are displayed in the same color (e.g., green). For detected images that have a "delete (rejected)" status, the detection image number 1511 and the convex mirror pin 1231 are displayed in the same color (e.g., black).

[0139] As described above, this embodiment provides a mechanism for determining the size of an object without using focal length. Specifically, the size of the object on the image can be obtained from two images containing the same object taken in succession, and the actual size of the object can be calculated using the distance the imaging device moved during the successive shooting. This makes it possible to calculate the actual size of the object even when either the imaging device or the object is moving and the exact distance between them cannot be measured.

[0140] The present invention can take the form of, for example, a system, apparatus, method, program, or recording medium. Specifically, it may be applied to a system consisting of multiple devices, or to an apparatus consisting of a single device.

[0141] Furthermore, the various controls described above, which are performed by CPU201, may be performed by a single piece of hardware, or multiple pieces of hardware (for example, multiple processors or circuits) may share the processing to control the entire device.

[0142] Furthermore, although the present invention has been described in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Moreover, each of the embodiments described above is merely one embodiment of the present invention, and it is possible to combine each embodiment as appropriate.

[0143] Furthermore, although the above-described embodiments used the application of the present invention to a PC as an example, the invention is not limited to this example and can be applied to any device capable of calculating the actual size of an object. In other words, the present invention can be applied to PDAs, mobile phone terminals (smartphones), tablet terminals, and the like.

[0144] (Other embodiments) The present invention can also be realized by performing the following process: supplying software (programs) that realize the functions of the embodiments described above to a system or device via a network or various storage media, and having the computer (or CPU, MPU, etc.) of that system or device read and execute the program code. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of Symbols]

[0145] 100 Imaging device 101 GPS 102 Edge Computers 103 Application Server 104 Databases 105 Client terminals 110 Network

Claims

1. An acquisition means for acquiring an object image including an object captured by a photographing means, and location information of the object, A storage control means for storing the object image and the object's position information in association with each other, When the operation to display a list of object images is performed, a display control means controls the display to group and display thumbnails of object images associated with the location information of the object indicating a position within a predetermined range. An information processing device characterized by comprising:

2. The information processing device according to claim 1, further comprising setting means for setting a rejected status for the object image for which a rejection operation was performed when a rejection operation was performed on the thumbnail of the displayed object image.

3. The information processing device according to claim 2, wherein when an operation is performed to select thumbnails of multiple object images to be rejected from the group, the setting means sets a rejection status for the selected multiple object images.

4. The information processing apparatus according to claim 2, characterized in that the display control means causes the display manner of the thumbnails of object images to be different from that of the thumbnails of object images to be displayed when the status of "rejected" is set.

5. The information processing apparatus according to claim 1, characterized in that when an operation is performed to display a list of object images of a specified location, the display control means controls the display to group and display thumbnails of object images associated with the location information of the object included in the specified location.

6. The information processing device according to claim 5, characterized in that the aforementioned location is designated by town or district.

7. The information processing apparatus according to claim 1, characterized in that the display control means controls the display of the object image corresponding to the thumbnail of the object image when an operation to display the object image corresponding to the thumbnail of the object image is performed.

8. The information processing apparatus according to claim 2, characterized in that the display control means controls the display of a thumbnail of the object image together with the ID of the object image.

9. The information processing apparatus according to claim 8, characterized in that the display control means controls the display of the IDs of object images for which the rejected status is not set and the IDs of object images for which the rejected status is set in different colors.

10. The information processing apparatus according to claim 2, characterized in that the display control means controls the display to show a map in which the indicators of the object are placed at positions corresponding to the location information of the object.

11. The information processing apparatus according to claim 10, characterized in that the display control means controls the display of an object index corresponding to an object image for which the "rejected" status is not set and an object index corresponding to an object image for which the "rejected" status is set, in mutually different colors.

12. The display control means is The system controls the display of a map in which the indicators of the object are placed at positions corresponding to the object's location information. The IDs and indicators of object images corresponding to object images that do not have the aforementioned rejection status set are displayed in the same color. The information processing apparatus according to claim 9, characterized in that the ID of the object image and the indicator of the object corresponding to the object image for which the status of rejection has been set are displayed in the same color to each other.

13. An acquisition step of acquiring an object image including the object captured by the shooting means and the position information of the object, A memory control step that stores the object image and the object's position information in association with each other, When the operation to display a list of object images is performed, a display control step is performed to control the display so that thumbnails of object images associated with the location information of the object indicating a position within a predetermined range are grouped and displayed. A control method for an information processing device, characterized by comprising the following:

14. A program for causing a computer to function as an information processing device according to any one of claims 1 to 12.

Images