Information processing system, information processing method, and program

Abstract

To provide a mechanism capable of specifying the size of an object without using a focal distance.SOLUTION: A first image including an object is acquired, and a second image including the object is acquired after the first image is acquired. A difference between the position of the imaging apparatus when the first image is acquired and the position of the imaging apparatus when the second image is acquired is calculated. Whether the object is an object of a first size or an object of a second size is identified using the first image, the second image, and the difference.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] In situations where either the subject or the camera is constantly moving, the distance between the camera and the object changes over time. For example, consider a vehicle equipped with a camera. In situations where either the subject or the camera is constantly moving, there is a known technique for determining the size of the subject using the focal length.

[0003] Patent Document 1 discloses that information regarding the movement of a subject is calculated from a captured image so that the state of the moving subject can be played back and confirmed, and that this information is used as control information for image playback. It also discloses that when determining the subject size on the image as information regarding the subject movement, the actual size and focal length of the subject are used for calculation. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-76738 DISCLOSURE OF THE INVENTION [Problem to be solved by the invention]

[0005] Patent Document 1 assumes that the focal length of the camera is known, so if the focal length of the camera, i.e., the distance from the camera to the subject, is unknown, the actual size of the subject cannot be determined.

[0006] Therefore, an object of the present invention is to provide a mechanism that can identify the size of an object without using the focal length. [Means for solving the problem]

[0007] In order to achieve the above object, the information processing system of the present invention comprises: a first acquisition means for acquiring a first image including the object; a second acquisition means for acquiring a second image including the object after acquiring the first image; a calculation means for calculating a difference between a position of the imaging device when the first image was acquired and a position of the imaging device when the second image was acquired; an identification means for identifying whether the object is an object of a first size or an object of a second size using the first image, the second image, and the difference; The present invention is characterized by comprising: [Effects of the Invention]

[0008] According to the present invention, the size of an object can be determined without using the focal length. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an example of an overall system configuration according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing an example of the hardware configuration of various devices. [Figure 3] 1 is an example of a flowchart relating to processing related to the present system. [Figure 4] FIG. 1 is a diagram illustrating an example of a configuration in which an imaging device is mounted on a vehicle. [Figure 5] FIG. 10 is a diagram showing an example of a detected image (first image) of a convex mirror. [Figure 6] FIG. 10 is a diagram showing an example of a detected image (second image) of a convex mirror. [Figure 7] 1 is an example of a diagram showing the diameter and center of a rectangle related to a convex mirror. [Figure 8] FIG. 1 is a diagram showing an example of vehicle positions and captured images when a curve mirror is photographed every few meters. [Figure 9] FIG. 10 is a diagram illustrating an example of calculation of a multiplier approximation formula. [Figure 10] FIG. 10 is a diagram illustrating an example of a method for calculating an angle. [Figure 11] FIG. 10 is a diagram illustrating an example of a method for calculating a distance β between an imaging device and a convex mirror. [Figure 12] FIG. 10 is a diagram illustrating an example of a screen displayed on a map application. [Figure 13] This is a diagram showing an example in which two or more other convex mirrors are installed. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0011] FIG. 1 is a diagram showing an example of the overall configuration of a system according to an embodiment of the present invention.

[0012] In this embodiment, the present invention will be described using a system for recording and managing the number and locations of convex mirrors within a specified area by detecting convex mirrors, which is one application of the present invention. Specifically, an imaging device 100 is installed in a vehicle and the vehicle travels to various locations to photograph convex mirrors within a specified area. The size of the convex mirror is identified using the diameter (in this embodiment, the diameter is used because a round convex mirror is assumed, but for a rectangular convex mirror, any information related to the size of the convex mirror will suffice, such as the vertical size (vertical size rather than horizontal size is preferable considering that the mirror will be photographed from an oblique angle)) of the convex mirror in the image detected from the photographed image, the X-coordinate position, and the latitude and longitude of the vehicle acquired by GPS 101. The latitude and longitude of the convex mirror are then calculated based on the identified size. This allows the number and locations of convex mirrors to be recorded and a management ledger to be created.

[0013] In this system, an in-vehicle system 10, a cloud system 20, and an office system 30 are connected via a network 110 so as to be able to communicate with each other.

[0014] The in-vehicle system 10 includes an edge computer 102 that receives images captured by an imaging device 100 mounted on the vehicle and vehicle position information from a GPS 101. The edge computer 102 is, for example, a personal computer (hereinafter referred to as PC). The edge computer 102 executes processes such as detecting convex mirrors and calculating position information. Note that any system other than the GPS 101 that can measure vehicle position information may be used.

[0015] The cloud system 20 includes an application server 103 and a database 104. The application server 103 stores the detected image of the convex mirror, position information, etc. received from the edge computer 102 in the database 104. The application server 103 also transmits the detection results and position information, etc. to be displayed on the display of the client terminal 105. The database 104 stores the detected image of the convex mirror, position information, etc. received from the application server 103.

[0016] The office system 30 includes a client terminal 105. The client terminal 105 may be any terminal capable of displaying the detected image of the convex mirror, position information, etc., such as a PC or a mobile terminal such as a smartphone. The client terminal 105 controls the display to display the detected image of the convex mirror, position information, etc., received from the application server 103. Note that, although this embodiment is configured to send and receive information via the cloud system 20, the in-vehicle system 10 and the office system 30 may also be configured to communicate via the network 110 without going through the cloud system 20.

[0017] Next, referring to FIG. 2, an example of the configuration of the edge computer 102, application server 103, or client terminal 105 is shown as an example of a device to which the present invention can be applied.

[0018] 2, a CPU 201, a memory 202, a nonvolatile memory 203, an image processing unit 204, a display 205, an operation unit 206, a recording medium I / F 207, an external I / F 209, and a communication I / F 210 are connected to an internal bus 250. The units connected to the internal bus 250 are capable of exchanging data with each other via the internal bus 250.

[0019] The memory 202 is formed, for example, from a RAM (such as a volatile memory using semiconductor elements). The CPU 201 controls each part of the client terminal 101 using the memory 202 as a work memory in accordance with a program stored, for example, in the nonvolatile memory 203. The nonvolatile memory 203 stores image data, audio data, other data, various programs for the operation of the CPU 201, and the like. The nonvolatile memory 203 is formed, for example, from a hard disk (HD) or a ROM.

[0020] Under the control of the CPU 201, the image processing unit 204 performs various image processing on image data stored in the nonvolatile memory 203 or the recording medium 208, video signals acquired via the external I / F 209, image data acquired via the communication I / F 210, captured images, etc. The image processing performed by the image processing unit 204 includes A / D conversion processing, D / A conversion processing, image data encoding processing, compression processing, decoding processing, enlargement / reduction processing (resizing), noise reduction processing, color conversion processing, etc. The image processing unit 204 may be configured with a circuit block dedicated to performing specific image processing. Furthermore, depending on the type of image processing, the CPU 201 may perform image processing according to a program without using the image processing unit 204. The CPU 201 works in cooperation with the image processing unit 204 to perform processing to recognize an object to be recognized from an image.

[0021] The display 205 displays images, GUI screens constituting a GUI (Graphical User Interface), and the like under the control of the CPU 201. The CPU 201 generates a display control signal according to a program, and controls each part of the edge computer 102 or the client terminal 105 to generate a video signal to be displayed on the display 205 and output it to the display 205. The display 205 displays a video based on the output video signal. Note that the edge computer 102 or the client terminal 105 itself is only provided with an interface for outputting a video signal to be displayed on the display 205, and the display 205 may be configured as an external monitor (such as a television).

[0022] The operation unit 206 is an input device for receiving user operations, including a character information input device such as a keyboard, a pointing device such as a mouse or touch panel, a button, a dial, a joystick, a touch sensor, a touch pad, etc. The touch panel is an input device that is configured as a plane overlaid on the display 205 and outputs coordinate information according to the position of contact.

[0023] A recording medium I / F 207 allows a recording medium 208 such as a memory card, CD, or DVD to be attached, and reads data from and writes data to the attached recording medium 208 under the control of the CPU 201. The external I / F 209 is an interface for connecting to an external device via a wired cable or wirelessly, and for inputting and outputting video signals and audio signals. The communication I / F 210 is an interface for communicating with external devices, the Internet 211, etc., and for sending and receiving various types of data such as files and commands.

[0024] The camera unit 212 is a camera unit including an image sensor (image sensor) that is configured with a CCD or CMOS element that converts an optical image into an electrical signal.

[0025] Next, an example of a convex mirror detection process according to an embodiment of the present invention will be shown with reference to Fig. 3. When the in-vehicle system 10 is started, the process of Fig. 3 starts. The process of each step is executed by the CPU 201. While the vehicle is traveling, the edge computer 103 continues to acquire information from the image capture device 100 and the GPS 102 and executes the process of this flowchart.

[0026] In S301, the CPU 201 acquires an image from the imaging device 100.

[0027] In S302, the CPU 201 acquires the latitude, longitude, and azimuth angle of the vehicle from the GPS 101. The azimuth angle indicates the angle of the direction in which the imaging device 100 installed in the vehicle is facing. For example, if the azimuth angle indicates north, 0° is acquired.

[0028] 4 shows an example of the positions at which the imaging device 100 and the GPS 101 antennas are mounted relative to the vehicle. The imaging device 100 is installed in the center of the front of the roof of the vehicle, with the center of the angle of view 402 facing the direction of travel. Specifically, the imaging device 100 is installed so that the center of the horizontal angle of view is oriented in the direction of travel of the vehicle, and the center of the vertical angle of view is parallel to the road in the direction of travel. The antenna of the GPS 101 is installed in a location close to the imaging device 100.

[0029] In S303, the CPU 201 executes an object detection process. Specifically, the process executes a process of detecting a convex mirror 403 in the image using AI. Note that, although the present embodiment will be described using a convex mirror, any object may be detected, such as a sign, a traffic light, a street lamp or other lighting fixture, a guardrail, a utility pole, a support, a roadside tree, a bench or other road accessories, or any other object for which the counting results need to be managed.

[0030] In S304, the CPU 201 determines whether or not the convex mirror 403 has been detected. If it has been detected, the process proceeds to S305, and if not, the process proceeds to S314.

[0031] An example of an image including a convex mirror and the positional relationship between the vehicle and the convex mirror at the time of capturing the image is shown in Fig. 5. Image 501 is an image captured by imaging device 100 at the position of vehicle 503. If a convex mirror is detected in image 501, rectangle 502 is displayed at the detected location.

[0032] In S305, if there are multiple detected convex mirrors, the CPU 201 acquires a representative convex mirror. Two convex mirrors, a rectangle 502 and a rectangle 504, are detected in the image 501. The vertical lengths of the rectangles 502 and 504 are compared, and the convex mirror associated with the rectangle 502, which is longer, is acquired as the representative convex mirror.

[0033] In S306, CPU 201 calculates the diameter and X coordinate position of the representative convex mirror. As shown in Fig. 7, diameter 701 (px) and X coordinate position 702 (center position of the convex mirror: px) of rectangle 502 in the image are calculated.

[0034] In S307, the CPU 201 calculates the distance α of the detected convex mirror using the diameter 701 calculated in S307 and the multiplier approximation formula.

[0035] The method for calculating the multiplier approximation formula, which is the premise of the processing in S307, will be described in detail using Figures 8 and 9. The multiplier approximation formula is assumed to be prepared before the processing of this flowchart begins. Figure 8 shows an example of photographing a convex mirror from different distances to calculate the formula. While Figure 8 is a view from above, for convenience of explanation, only the convex mirror is shown from the front. Starting at a distance α of 15 m from the convex mirror, photographs are taken at 1 m intervals from there, ending at a point 3 m away. The distance α is a value identified as the distance β between the imaging device and the convex mirror (described later) and indicates the distance between the convex mirror and the imaging point. For example, it is the distance 805 between the position 801 of the convex mirror and the vehicle position 802 obtained from the GPS 101. The distance α between the convex mirror and the imaging point and the diameter of the convex mirror in the image captured at that point are measured to create the graph shown in Figure 9. When photographing the convex mirror from different distances α, the same imaging device 100 as the imaging device 100 installed in the vehicle is used. In addition, using images taken under the same conditions (for example, magnification), the distance α between the convex mirror and the photographing point and the diameter of the convex mirror in the image taken at that point are measured.

[0036] The distance α between the actual position 801 of the convex mirror and the vehicle position 802 obtained from GPS 101 can be calculated. For example, the diameter 702 of the convex mirror in image 804 taken at point 803, 6 m away, is calculated. The calculated value 901 is plotted on a graph with the distance α between the convex mirror and the photographing point on the y-axis and the diameter of the convex mirror on the x-axis. The multiplier approximation formula calculated from this graph is shown below. Note that there are two types of convex mirrors, one with a diameter of 60 cm and one with a diameter of 80 cm, and it is necessary to calculate the multiplier approximation formula for each.

[0037] "Formula 1" Approximate multiplier for a 60cm convex mirror: y = 565.6 x ^ -1.069 Approximate multiplier for an 80cm convex mirror: y= 812.34 x ^ -1.082 y: distance α(m) x: diameter of the convex mirror (px)

[0038] In this step, since it is not specified whether the size of the convex mirror is 60 cm or 80 cm, the distance α (distance from the vehicle to the convex mirror) when the convex mirror is 60 cm and the distance α when it is 80 cm are calculated using Equation 1. If a convex mirror is detected in the image to be processed in S308 and also detected in the image acquired immediately before (if detected continuously), the calculation result in this step is used to determine the actual size of the convex mirror. If a convex mirror is not detected continuously in S308, the calculation result is set in the output work area in S312.

[0039] In S308, CPU 201 determines whether or not convex mirrors have been detected consecutively. If they have been detected consecutively, the process proceeds to S309; ​​if not, the process proceeds to S312. FIG. 6 is a diagram showing the positional relationship between the vehicle and the convex mirror and the captured image thereof after vehicle 503 has moved to the position of vehicle 603. Image 601 is an image captured by imaging device 100 at the position of vehicle 603. That is, in this step, it is determined whether or not convex mirror 403 has been detected in images captured before and after the vehicle moved.

[0040] In S309, the CPU 201 determines whether the detected convex mirror is the same as the convex mirror detected immediately before. If they are the same, the process proceeds to S310; if not, the process proceeds to S313. Specifically, if the difference between the X-coordinate position 702 of the previous convex mirror and the X-coordinate position 702 of the current convex mirror is less than 200 px and the current diameter 701 is the same as or larger than the previous diameter 701, the convex mirror is determined to be the same. Note that values ​​less than 200 px are determined based on factors such as road conditions (e.g., whether the road is straight or curved), vehicle speed, and capture intervals. For example, FIG. 13 shows an example in which separate convex mirrors are installed on the left and right sides of a vehicle. Convex mirror 1303 is installed in front of vehicle 1301, and convex mirror 1304 appears when turning right. Convex mirror 1303 is detected near the center of image 1305 when the vehicle is traveling straight, and as the vehicle makes a right turn, convex mirror 1303 is detected on the left side of the image. Then, in image 1306 taken during (after) a right turn, the convex mirror 1304 installed on the right side of the vehicle is detected. In this case, when comparing thresholds of 100px, 200px, and 300px, if the threshold is set to less than 100px, even a slight deviation in pixel value will result in the mirror being detected as a different convex mirror. In particular, on straight roads, vehicle speeds are relatively fast, resulting in a large deviation in pixel value, and even if the mirrors are the same, they will be detected as different convex mirrors. If the threshold is set to less than 300px, even if the convex mirror 1303 is shifted to the left, as in image 1306, it will be detected as the same convex mirror. In other words, including an image in which the target convex mirror is shifted to the edge of the image will result in an image that is inappropriate for presentation in the example screen shown in Figure 12 (described later) (an image in which the convex mirror is difficult for the user to see). Therefore, the vehicle speed changes depending on whether the road is straight or curved, and if the threshold is set to less than 200px, the mirrors can be detected as the same convex mirror regardless of the road conditions. 200px is just an example, and an appropriate value will be set taking into account factors such as vehicle speed, shooting interval, and number of pixels.

[0041] In S310, the CPU 201 determines the size of the convex mirror. There are two main types of convex mirrors, one with a diameter of 60 cm and the other with a diameter of 80 cm, and in this step, it is determined whether the diameter of the convex mirror is 60 cm or 80 cm (whether it is an object of the first size or an object of the second size).

[0042] When the distance between the convex mirror and the vehicle is unknown, it is difficult to determine whether the diameter of the convex mirror is 60 cm or 80 cm. If the size of the convex mirror cannot be specified, it is not clear which multiplier approximation formula to use when using Formula 1. If the wrong formula is used for calculation, an error will occur in the calculation result, and when recording and managing the number and location of convex mirrors within a specified area, the exact location will not be managed. Therefore, in this step, we will explain how to determine the actual size of the convex mirror.

[0043] An example will be described in which, after image 501 shown in FIG. 5 (hereinafter referred to as the "first detected image") is captured, vehicle 503 moves to the position of vehicle 603, and image 601 shown in FIG. 6 (hereinafter referred to as the "second detected image") is captured. Here, it is assumed that the vehicle is traveling at a speed of 25 km / h, images are captured every 0.5 seconds, and the latitude and longitude are acquired from GPS at each capture. Also, it is assumed that the diameter 701 of the convex mirror in the first detected image is 50 px. The calculation results using Equation 1 for the convex mirror sizes of 60 cm and 80 cm at that time are shown below.

[0044] "Formula 1: Calculation result when x = 50" The approximate multiplier for a 60cm convex mirror is 565.6 * 50 ^ -1.069 = 8. 64m Approximate multiplier for an 80cm convex mirror: 812.34 * 50 ^ -1.082 = 11.78m

[0045] Next, in S302, the travel distance when vehicle 503 moves to the position of vehicle 603 is calculated using the latitude and longitude of the vehicle acquired from GPS 101. Under the above assumptions, the travel distance of the vehicle is 3.5 m.

[0046] To find "x: diameter of the convex mirror," a value is calculated using the inverse multiplier approximation formula shown below, which is a modification of Equation 1. When using the inverse multiplier approximation formula for 60 cm and the inverse multiplier approximation formula for 80 cm, if the travel distance is the same, the calculation result of the inverse multiplier approximation formula for 60 cm tends to be larger than that of 80 cm. This characteristic is utilized to determine the size of the convex mirror. In other words, the size of the convex mirror is determined based on whether the diameter 701 of the convex mirror in the second detected image 601 is closer to the calculation result for 60 cm or 80 cm.

[0047] "Formula 2" Approximate inverse multiplier for a 60cm convex mirror: x = ( y / 565.6 )^ (1 / -1.069) Approximate inverse multiplier for an 80cm convex mirror: x = (y / 812.34) ^ (1 / -1.082) y: First calculation distance α - Movement distance x: Estimated diameter of the convex mirror

[0048] "Formula 2: Calculation example" Approximate inverse multiplier for a 60cm convex mirror: 81.3 = ((8.64 -3.5) / 565.6)^ (1 / -1.069) Approximate inverse multiplier for an 80cm convex mirror: 69.24 = ((11.78-3.5) / 812.34) ^ (1 / -1.082)

[0049] In the above calculation example, the result for 60 cm is 81.3, and the result for 80 cm is 69.24. For example, if the diameter 701 of the convex mirror in the second detected image 601 is 80px, it is determined that the convex mirror is 60 cm because it is close to the result of 60 cm.

[0050] This makes it possible to determine whether the size of the convex mirror is a first size (60 cm in this embodiment) or a second size (80 cm in this embodiment). Specifically, there are conventional methods for determining the size of an object using the distance (focal length) between the imaging device and the object. However, in this embodiment, the vehicle is constantly moving, and the distance between the convex mirror and the imaging device fluctuates. If the distance between the convex mirror and the imaging device is known, the size of the convex mirror can be determined using prior art, but it is difficult to constantly determine the distance between the two in situations where the distance fluctuates. Therefore, by processing this step, the actual size of the object can be determined even if the distance between the imaging device and the object is unknown. In other words, it is possible to determine whether the convex mirror is 60 cm or 80 cm, and when managing it in a ledger, it becomes possible to manage the size of the convex mirror installed at a certain location.

[0051] Furthermore, if the size of the convex mirror cannot be identified, it is not possible to calculate the accurate position information of the convex mirror, which will be described later.When the size of the convex mirror cannot be identified, one method is to measure the distance α between the convex mirror and the shooting location and the diameter of the convex mirror in the image taken at that location to create a graph like the one in Figure 9, by plotting the values ​​for 60 cm and 80 cm on the same graph and deriving the multiplier approximation formula from the average.However, in order to calculate more accurate position information, it is desirable to first identify the size of the convex mirror and then calculate the position information.

[0052] In S311, the CPU 201 calculates the position information of the convex mirror, which will be specifically described with reference to Figs. 8 to 11 and several formulas.

[0053] To calculate the position information of the convex mirror, distance α(A) 1101 calculated from diameter 701 and angle (B) 1102 calculated from X coordinate position 702 are required, as shown in FIG.

[0054] First, distance α(A) 1101, which is found from diameter 701, is calculated using the above-mentioned formula 1. The size of the convex mirror is identified in S310. Depending on whether the identified diameter is 60 cm or 80 cm, the calculation is performed using the corresponding multiplier approximation formula of formula 1. That is, if the object has a diameter of 60 cm, distance α is calculated using the multiplier approximation formula for a 60 cm object.

[0055] Next, calculation of angle (B) 1102 obtained from X coordinate position 702 will be described with reference to FIG. 10. The content of FIG. 10 is prepared in advance before the start of this flowchart. Angle (B) is calculated using width 1001 of the captured image (1280 px in this embodiment), ratio 1002 of X coordinate position 702, and triangle 1003. Ratio 1002 of X coordinate position 702 is obtained based on the ratio of X coordinate position 702 to the width of the image. Calculated angle 1004 is calculated using ratio 1002 and a function (inverse tangent) related to the inverted triangle connecting X coordinate position 702 and image capture device 100. Angle 1005 is calculated using calculated angle 1004. A specific formula for calculating the angle is shown below.

[0056] "Formula 3" y = math.degress(math.atan(a ×(math.abs(b - x / b)))) y: angle (°) x:X coordinate (px) a: An image at an angle of 48°, with the width when the height of the image is 1 = math.tan(math.radians(48)) = 1.11 b: Half the width of the image (e.g. 1280 / 2 = 640) math.degress: angle conversion functions math.atan: inverse tangent function in radians math.abs : absolute value function math.tan: Tangent function in radians math.radians: radian conversion functions

[0057] Finally, the process of calculating the position information of the representative convex mirror will be explained using Figure 11. Note that although Figure 11 is a view from above, for convenience of explanation, only the convex mirror is shown as viewed from the front. In this step, the position information of the representative convex mirror is calculated using formulas 1 and 2, which have been prepared in advance. That is, formula 1 calculates distance α(A) 1101 calculated from diameter 701, and formula 3 calculates angle (B) 1102 calculated from X coordinate position 702. Once distance α(A) and angle (B) have been calculated, distance β 1103 between the image capture device 100 and the convex mirror can be calculated using a sine function. Therefore, the position information of the convex mirror can be calculated using the following formula. Note that distance β is a value identified as the aforementioned distance α and indicates the distance between the convex mirror and the image capture device.

[0058] "Formula 4" Y = A / math.sin(math.radians(90°-B)) y: Distance β (m) from the imaging device A: Distance α (m) calculated from the diameter B: Angle calculated from the X coordinate (°) math.sin: sine function in radians math.radians: radian conversion functions

[0059] When capturing images using an imaging device mounted on a vehicle, the distance to the convex mirror is constantly changing. Furthermore, the GPS 101 can only obtain the vehicle's position, making it difficult to measure the distance to the convex mirror. The processing in this step allows the distance β between the vehicle and the convex mirror to be calculated based on the X coordinate position (center position) and diameter obtained from the image including the convex mirror. This makes it possible to calculate the position information of the convex mirror, making it possible to record and manage the number and positions of convex mirrors within a specified area.

[0060] In this embodiment, an example in which the imaging device moves has been described, but images captured by two imaging devices may also be used. That is, an image captured by a first imaging device and an image captured by a second imaging device are acquired, and the distance between the imaging devices is calculated. The actual size of the object may also be determined using the images captured by the first imaging device, the images captured by the second imaging device, and the distance between the imaging devices.

[0061] In S312, the CPU 201 performs processing to set an output work area. That is, the following information is set:

[0062] Convex mirror size Convex mirror latitude Convex mirror longitude Detected image Detected Image Name Previous diameter Previous X coordinate 60cm Previous distance α 80cm Previous distance α

[0063] The size of the convex mirror is set to either 60cm or 80cm. The latitude and longitude of the convex mirror are set based on the location information calculated by S310. The method for calculating the latitude and longitude of the convex mirror is shown below.

[0064] First, in S302, the azimuth angle at which the convex mirror is installed is calculated based on the azimuth angle acquired from GPS 101. Different calculations are performed depending on whether the convex mirror is displayed to the right or left of the center of the image. That is, if the X coordinate position 702 of the convex mirror is greater than the X coordinate indicating the center of the image, the calculation is performed as follows: "azimuth angle of convex mirror = azimuth angle acquired from GPS 101 + angle calculated using Equation 3." If the X coordinate position 702 of the convex mirror is equal to or less than the X coordinate indicating the center of the image, the calculation is performed as follows: "azimuth angle of convex mirror = azimuth angle acquired from GPS 101 - angle calculated using Equation 3." For example, if the azimuth angle acquired from GPS 101 is 5° (north) and the angle is 30°, the azimuth angle of the convex mirror is calculated to be 35°. This indicates that the convex mirror is installed 35° to the right of the vehicle, with 0° being the front of the vehicle. Next, the latitude and longitude of the convex mirror are calculated using the latitude and longitude of the vehicle obtained from GPS 101, the azimuth angle of the convex mirror calculated above, and the distance from the image capture device calculated using Equation 4. For example, this is calculated using the function "(latitude of convex mirror, longitude of convex mirror = grs80.fwd(latitude of vehicle, longitude of vehicle, azimuth angle of convex mirror, distance from image capture device))". grs80.fwd is a function that calculates the latitude and longitude of a location away from the current latitude and longitude in the azimuth angle. In this way, the latitude and longitude of the convex mirror are calculated, and the position of the convex mirror can be managed.

[0065] The detected image is set as a file saved in an image file format (JPEG, PNG, GIF, etc.). The detected image name is set as a name given to the detected image. The previous diameter and previous X coordinate are set as the diameter 701 and X coordinate position 702 of the convex mirror in the first detected image 501. The previous 60 cm distance α and the previous 80 cm distance α are set as the calculation results calculated using Equation 1 in the first detected image 501.

[0066] Meanwhile, in S313, the CPU 201 uploads the information set in the output work area to the application server 103. In this step, when convex mirrors are detected consecutively but are not the same, information related to the first detected convex mirror is uploaded. For example, when convex mirror A, convex mirror A, and convex mirror B are detected in this order, the actual size of convex mirror A is determined and set in the output work area. If convex mirror B is then detected consecutively, information related to convex mirror A set in the output work area is uploaded to the application server 103.

[0067] In S313, the CPU 201 uploads the information set in the output work area to the application server 103. This is the same process as S313. In this step, for example, if convex mirror A, convex mirror A are detected in that order and nothing is detected thereafter, the information related to convex mirror A set in the output work area is uploaded to the application server 103.

[0068] Here, an example of a screen displayed on the client terminal 105 after uploading will be described with reference to FIG. 12. The application server 103 stores the information set in the received output work area in the database 104. The database 104 assigns a unique identification number to the information set in the output work area. The database 104 also adds and registers the address obtained by sending the latitude and longitude to the reverse geocoding API. When the CPU 201 of the application server 103 receives a screen display request from the client terminal 105, it sends all of the information set in the output work area stored in the database 104 to the client terminal 105. As a result, a screen such as that shown in FIG. 12 is displayed on the display of the client terminal 105.

[0069] A pin 1201 is placed on the map app at the latitude and longitude position of the convex mirror. In FIG. 12, pin 1201 assigned with the identification number 1060 is placed, and when an operation on pin 1201 is accepted, detailed information 1202 related to it is displayed. The detailed information 1202 includes a street view button 1203 and a detected image link 1204. When a press of street view button 1203 is accepted, the position of pin 1201 is displayed in street view 1205. When an operation on detected image link 1204 is accepted, detected image 1206 set in the output work area related to the identification number 1060 is displayed.

[0070] This makes it possible to know where and how many convex mirrors are installed within a given area. For example, local governments install and maintain convex mirrors, and this will make it easier to create a ledger for managing this.

[0071] As described above, this embodiment provides a mechanism for identifying the size of an object without using focal length. That is, the size of the object in the image is obtained from two images of the same object captured consecutively, and the actual size of the object can be calculated using the distance traveled by the image capture device during the consecutive captures. This makes it possible to calculate the actual size of the object even when either the image capture device or the object is moving and the accurate distance between them cannot be measured.

[0072] The present invention can be embodied as, for example, a system, an apparatus, a method, a program, a recording medium, etc. Specifically, the present invention may be applied to a system consisting of multiple devices, or may be applied to an apparatus consisting of a single device.

[0073] The various controls described above as being performed by CPU 201 may be performed by a single piece of hardware, or the entire device may be controlled by multiple pieces of hardware (e.g., multiple processors or circuits) sharing the processing.

[0074] Furthermore, although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Furthermore, each of the above-described embodiments merely represents one embodiment of the present invention, and each embodiment can be combined as appropriate.

[0075] In the above-described embodiment, the present invention has been described as being applied to a PC, but the present invention is not limited to this example and can be applied to any device that can calculate the actual size of an object. In other words, the present invention can be applied to PDAs, mobile phone terminals (smartphones), tablet terminals, etc.

[0076] (Other embodiments) The present invention can also be realized by executing the following process. That is, software (programs) that realize the functions of the above-described embodiments are supplied to a system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads and executes the program code. In this case, the program and the storage media storing the program constitute the present invention. [Explanation of symbols]

[0077] 100 Imaging device 101 GPS 102 Edge Computer 103 Application Server 104 databases 105 client terminals 110 Network

Claims

1. a first acquisition means for acquiring a first image including the object; a second acquisition means for acquiring a second image including the object after acquiring the first image; a calculation means for calculating a difference between a position of the imaging device when the first image was acquired and a position of the imaging device when the second image was acquired; an identification means for identifying whether the object is an object of a first size or an object of a second size using the first image, the second image, and the difference; An information processing system comprising:

2. 2. The information processing system according to claim 1, wherein the position of the object is identified using the size of the identified object.

3. obtaining a diameter and a center of the object on the image from the first image; 2. The information processing system according to claim 1, wherein the position of the object is identified using the acquired diameter and center.

4. 2. The information processing system according to claim 1, further comprising display control means for controlling the display of the position of the object on a map.

5. 2. The information processing system according to claim 1, wherein the display control means controls the display so that an image relating to the object is displayed when an operation on the position of the displayed object is received.

6. 2. The information processing system according to claim 1, wherein the specifying means specifies whether the diameter of the object is 60 cm or 80 cm.

7. 2. The information processing system according to claim 1, wherein the imaging device is mounted on a vehicle.

8. 2. The information processing system according to claim 1, wherein the position of the imaging device is acquired from a GPS installed in a vehicle.

9. 9. The information processing system according to claim 1, wherein the object is a convex mirror.

10. a first acquisition step of acquiring a first image including the object; a second acquisition step of acquiring a second image including the object after acquiring the first image; a calculation step of calculating a difference between a position of the imaging device when the first image was acquired and a position of the imaging device when the second image was acquired; a step of identifying whether the object is an object of a first size or an object of a second size using the first image, the second image, and the difference; 1. A method for controlling an information processing system, comprising:

11. A program for causing at least one computer to function as each of the means of the information processing system according to any one of claims 1 to 8.

Images