Electronic apparatus, control method, and program

Abstract

To provide an electronic apparatus, a control method, and a program that can increase the accuracy of calculating the distance to an object to be imaged on the basis of an image of the object.SOLUTION: An electronic apparatus comprises: an imaging unit; and a control unit that, on the basis of an image of a movable body picked up by the imaging unit, generates information indicating the position of the movable body representing a position in a space in which the image is picked up. The control unit corrects a parameter that associates coordinates on the image picked up by the imaging unit with the position in the space in which the image is picked up, on the basis of a change over time of the position of the movable body obtained from the information indicating the position of the movable body.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present disclosure relates to an electronic device, a control method, and a program.

Background Art

[0002] Conventionally, for example, by using a road monitoring system or the like to obtain in advance the positional relationship between the road plane of the monitoring area and a camera, a method of measuring the position or speed of an object such as an automobile from the information acquired by the camera is known. As a calibration method for a camera in such a system, for example, Patent Document 1 proposes a method of calculating road plane parameters from the coordinates of a plurality of feature points detected from an image of an object imaged by a camera.

[0003] The surface of a flat land such as a road surface is hardly assumed to be strictly flat, and actually usually includes some unevenness. Thus, in an actual road surface that is not flat, it may not be possible to accurately measure the position of an object such as an automobile from the information acquired by a camera.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] For example, it is advantageous to improve the accuracy of calculating the distance to an object based on an image of the object captured by an imaging unit such as a digital camera.

[0006] An object of the present disclosure is to provide an electronic device, a control method, and a program that can improve the accuracy of calculating the distance to an object and the like based on an image of the object to be imaged.

Means for Solving the Problems

[0007] An electronic device according to an embodiment includes: an imaging unit; a control unit that generates information indicating the position of a moving object, which represents the position in the space where the image of the moving object is captured, based on the image of the moving object captured by the imaging unit; and includes the above. The control unit corrects a parameter for associating coordinates on the image captured by the imaging unit with the position in the space where the image is captured, based on the temporal change of the position of the moving object obtained from the information indicating the position of the moving object.

[0008] A control method according to an embodiment is a control method for an electronic device that generates information indicating the position of a moving object, which represents the position in the space where the image of the moving object is captured, based on the image of the moving object captured by the imaging unit, and includes: correcting a parameter for associating coordinates on the image captured by the imaging unit with the position in the space where the image is captured, based on the temporal change of the position of the moving object obtained from the information indicating the position of the moving object.

[0009] A program according to an embodiment causes an electronic device that generates information indicating the position of a moving object, which represents the position in the space where the image of the moving object is captured, based on the image of the moving object captured by the imaging unit, to execute: correcting a parameter for associating coordinates on the image captured by the imaging unit with the position in the space where the image is captured, based on the temporal change of the position of the moving object obtained from the information indicating the position of the moving object.

Advantages of the Invention

[0010] According to an embodiment, it is possible to provide an electronic device, a control method, and a program that can improve the accuracy of calculating the distance to an object to be imaged or the like based on the image of the object.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

MODE FOR CARRYING OUT THE INVENTION

[0012] In the present disclosure, the “electronic device” may be a device driven by electric power. Further, the “system” may be a device or devices including a device driven by electric power. Further, the “user” may be a person (typically a human) who uses the system and / or the electronic device according to an embodiment. By using the system and / or the electronic device according to an embodiment, the user can accurately know the actual position of an object imaged as an image and / or the actual distance to the object. Further, the “object” may be an object (any article) or a person (for example, a human or an animal) whose distance is calculated or estimated by the system and / or the electronic device according to an embodiment. Furthermore, the user may be a person whose distance is calculated or estimated by the system and / or the electronic device according to an embodiment.

[0013] Hereinafter, a system according to an embodiment will be described in detail with reference to the drawings.

[0014] FIG. 1 is a diagram showing a schematic configuration of a system according to an embodiment. As shown in FIG. 1, a system 100 according to an embodiment may include an electronic device 1 and an imaging unit 20. Further, the electronic device 1 according to an embodiment may include the imaging unit 20 or may be configured without including the imaging unit 20. The electronic device 1 and the imaging unit 20 may be connected by wire, wirelessly, or a combination of wire and wireless. Further, the system 100 according to an embodiment may include at least one of an output unit 30 and an input unit 40. At least one of the output unit 30 and the input unit 40 may be connected to the electronic device 1 by wire, wirelessly, or a combination of wire and wireless. Further, at least any one of the imaging unit 20, the output unit 30, and the input unit 40 may be included in the electronic device 1 or may be provided separately from the electronic device 1. The system 100 according to an embodiment may not include some of the functional units shown in FIG. 1 or may include functional units other than those shown in FIG. 1.

[0015] The electronic device 1 according to an embodiment generates information indicating the distance to an object based on an image of the object captured by the imaging unit 20. As shown in FIG. 1, the electronic device 1 according to an embodiment may include a control unit 10, a storage unit 12, and a communication unit 14. The electronic device 1 according to an embodiment may not include some of the functional units shown in FIG. 1 or may include functional units other than those shown in FIG. 1.

[0016] The control unit 10 controls and / or manages the entire electronic device 1, including each functional unit that constitutes the electronic device 1. The control unit 10 may be configured to include an arbitrary controller. More specifically, the control unit 10 may include at least one processor such as a CPU (Central Processing Unit) or a DSP (digital signal processor) in order to provide control and processing capabilities for executing various functions. The control unit 10 may be realized by a single processor, or may be realized by several processors, or may be realized by individual processors respectively. The processor may be realized as a single integrated circuit. The integrated circuit is also referred to as an IC (Integrated Circuit). The processor may be realized as a plurality of communicably connected integrated circuits and discrete circuits. The processor may be realized based on various other known technologies.

[0017] The control unit 10 may be configured, for example, as a CPU or an SDP and a program executed by the CPU or the SDP. Programs executed in the control unit 10, results of processes executed in the control unit 10, etc. may be stored, for example, in the storage unit 12. The control unit 10 may appropriately include a memory necessary for the operation of the control unit 10.

[0018] In one embodiment, the control unit 10 may acquire image data captured by the imaging unit 20. For this purpose, the control unit 10 may be connected to the imaging unit 20 by at least one of wired and wireless means. Also, the control unit 10 may perform image recognition or the like on the image data captured by the imaging unit 20. Thus, the control unit 10 may appropriately process the image data captured by the imaging unit 20. The operation of the control unit 10 of the electronic device 1 according to one embodiment will be described in further detail later.

[0019] The storage unit 12 may have a function as a memory for storing various kinds of information. The storage unit 12 may store, for example, a program executed in the control unit 10 and the result of the process executed in the control unit 10. The storage unit 12 may function as a working memory of the control unit 10. For this reason, the storage unit 12 may be connected to the control unit 10 by at least one of wired and wireless means. Further, the storage unit 12 may store the data of the image captured by the imaging unit 20. The storage unit 12 can be constituted by, for example, a semiconductor memory or the like, but is not limited thereto and can be any storage device. For example, the storage unit 12 may be a storage medium such as a memory card inserted into the electronic device 1 according to one embodiment. Further, the storage unit 12 may be an internal memory of a CPU or a DSP used as the control unit 10 described later, or may be connected to the control unit 10 as a separate unit.

[0020] The communication unit 14 has a function of an interface for communicating by wire or wirelessly. The communication method performed by the communication unit 14 in one embodiment may be a wireless communication standard. For example, the wireless communication standard may include communication standards for cellular phones such as 2G, 3G, 4G, and 5G. For example, the communication standards for cellular phones include LTE (Long Term Evolution), W-CDMA (Wideband Code Division Multiple Access), CDMA2000, PDC (Personal Digital Cellular), GSM (registered trademark) (Global System for Mobile communications), and PHS (Personal Handy-phone System). For example, the wireless communication standard includes WiMAX (Worldwide Interoperability for Microwave Access), IEEE802.11, WiFi, Bluetooth (registered trademark), IrDA (Infrared Data Association), and NFC (Near Field Communication). The communication unit 14 can support one or more of the above communication standards. The communication unit 14 may be configured to include, for example, an antenna for transmitting and receiving radio waves and an appropriate RF unit. Further, the communication unit 14 may be configured as an interface such as a connector for wired connection to the outside. Since the communication unit 14 can be configured by known techniques for performing wireless communication, a more detailed description of hardware and the like is omitted.

[0021] The various types of information received by the communication unit 14 may be supplied to, for example, the storage unit 12 and / or the control unit 10. For this reason, the communication unit 14 may be connected to at least one of the control unit 10 and the storage unit 12 by at least one of wired and wireless means. The various types of information received by the communication unit 14 may be stored, for example, in a memory built into the control unit 10. Further, the communication unit 14 may transmit, for example, the processing result by the control unit 10, the image data captured by the imaging unit 20, and / or the information stored in the storage unit 12 to the outside. The communication unit 14 may receive, for example, known values (such as the actual height Ho of the object Ob and / or the vertical height Ha at which the imaging unit 20 is installed) from an external communication device as described later.

[0022] The imaging unit 20 shown in FIG. 1 may be configured to include an image sensor that electronically captures an image, such as a digital camera. The imaging unit 20 may be, for example, a visible light camera or a FIR (Far Infrared Rays) camera. The imaging unit 20 may be configured to include an imaging element that performs photoelectric conversion, such as a CCD (Charge Coupled Device Image Sensor) or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The imaging unit 20 may capture an image of an arbitrary object Ob as described later. The imaging unit 20 may convert the captured image into a signal (data) and transmit it to the electronic device 1. For example, the imaging unit 20 may transmit a signal based on the captured image to the control unit 10, the storage unit 12, and / or the communication unit 14 of the electronic device 1. Therefore, the imaging unit 20 may be connected to the control unit 10, the storage unit 12, and / or the communication unit 14 by at least one of wired and wireless means. The imaging unit 20 is not limited to an imaging device such as a digital camera as long as it can capture the object Ob including its background, and may be any device.

[0023] The imaging unit 20 may be a still camera or a movie camera. In one embodiment, the imaging unit 20 may capture, for example, the object Ob as a still image at regular intervals (e.g., 15 frames per second). Also, in one embodiment, the imaging unit 20 may capture, for example, the object Ob as a continuous video. Thus, in one embodiment, the imaging unit 20 may capture an image of the object Ob.

[0024] The output unit 30 outputs the processing result by the electronic device 1 and the like. In one embodiment, the output unit 30 may display information regarding the processing result by the electronic device 1 as, for example, a display. Also, in one embodiment, the output unit 30 may display characters or symbols, various objects, and / or various images that constitute a screen for prompting the user to input predetermined information. Further, in one embodiment, the output unit 30 may display an image or video captured by the imaging unit 20. The data necessary for display in the output unit 30 is supplied from the control unit 10.

[0025] The output unit 30 may be any display device such as a liquid crystal display (LCD), an organic electro-luminescence panel (OLED), or an inorganic electro-luminescence panel (ILED). The output unit 30 may display various types of information such as characters, figures, symbols, or graphs. The output unit 30 may display objects that constitute various GUIs such as pointers and buttons, and icon images in order to prompt the user operating the electronic device 1 to perform an operation. Also, the output unit 30 may be configured to include a backlight or the like as appropriate.

[0026] Further, the output unit 30 is not necessarily limited to a device that gives a visual effect to the user. The output unit 30 may adopt any configuration as long as it can convey the necessary information to the user. For example, the output unit 30 may be replaced with a speaker that conveys various information by voice or the like. Further, such a speaker may be provided together with the output unit 30 such as a display.

[0027] In one embodiment, the output unit 30 may be configured as a touch screen display together with the input unit 40 described later. In this case, the touch screen display may include a display device such as a liquid crystal display or an organic EL display as the output unit 30. Further, in this case, the touch screen display may include a touch sensor or a touch panel that detects the presence or absence of contact by the user and the position of the contact as the input unit 40. In such a configuration, for example, a key such as a numeric keypad or an icon is displayed on the output unit 30 as an object, and an operation in which the operator touches the object can be detected by the input unit 40. The input unit 40 can adopt various types of touch panels such as a resistive film type, a capacitive type, or an optical type.

[0028] The input unit 40 can be any input device used by the user to perform operations, such as a key (physical key) like a keyboard and / or a pointing device like a mouse or a trackball. The input unit 40 may be configured to detect various input operations by the user. In one embodiment, since the input unit 40 can be various known input devices, a more detailed description is omitted. In one embodiment, the electronic device 1 may obtain, from the input unit 40, an input from the user that designates a predetermined object in the image captured by the imaging unit 20. Further, the input unit 40 may detect an operation by the user to input known values (for example, the actual height Ho of the object Ob and / or the height Ha in the vertical direction where the imaging unit 20 is installed) described later.

[0029] At least a part of each functional unit constituting the system 100 or the electronic device 1 according to one embodiment, as shown in FIG. 1, may be configured by a specific means in which software and hardware resources cooperate with each other.

[0030] Next, a basic usage mode of the system 100 according to one embodiment will be described.

[0031] FIG. 2 is a diagram schematically showing an example of a basic usage mode of the system 100 according to one embodiment. FIG. 2 shows an example of the arrangement of the imaging unit 20 among the system 100 according to one embodiment. In the system 100, the electronic device 1, the output unit 30, the input unit 40, etc. may be arranged at any position as long as they can receive the data of the image captured by the imaging unit 20. Therefore, in FIG. 2, the illustration of the functional units other than the imaging unit 20 such as the electronic device 1 is omitted.

[0032] As shown in FIG. 2, the imaging unit 20 may be installed at a position where the object Ob can be imaged. In FIG. 2, the object Ob is located at the point P on the plane Gr. The object Ob may be, for example, a human. Also, the object Ob may be other organisms other than humans, or may be, for example, a non-biological substance. The object Ob may be, for example, any moving body such as an automobile, or may be any stationary object such as a road sign.

[0033] In FIG. 2, the XY plane passing through the origin O, that is, the XY plane where the coordinates in the Z-axis direction are zero, is set as a predetermined plane Gr. The predetermined plane Gr may be, for example, a plane that is at least partially flat like the ground. The plane Gr shown in FIG. 2 is almost completely flat, and the point P where the object Ob (human) is located (standing) is on the plane Gr. That is, in FIG. 2, it is assumed that both the coordinate in the Z-axis direction of the plane Gr and the coordinate in the Z-axis direction of the point P are zero. Hereinafter, the coordinate in the Z-axis direction shown in FIG. 2 will also be simply referred to as "height". For example, in FIG. 2, the plane Gr and the point P are at the same height. Also, hereinafter, the positive direction of the Z-axis shown in FIG. 2 may be simply referred to as "up", and the negative direction of the Z-axis shown in FIG. 2 may be simply referred to as "down".

[0034] As shown in FIG. 2, the imaging unit 20 is disposed at the position of point A having a height of Ha with respect to the plane Gr. The imaging unit 20 may be attached to, for example, a stand or may be attached to a building such as a building or a roadside unit. Further, the imaging unit 20 may be disposed in a state where its orientation can be adjusted at point A.

[0035] As shown in FIG. 2, the imaging unit 20 is disposed at a position at a distance Ha vertically upward from the origin O. Also, the length from the origin O to the point P where the object Ob is located is denoted as So. Hereinafter, the distance from the imaging unit 20 to the object Ob in the plane Gr (that is, the length of the orthogonal projection of the distance from the imaging unit 20 to the object Ob onto the plane Gr) may be denoted as the distance So (length So).

[0036] The imaging unit 20 is assumed to have an optical axis in the direction of the dashed-dotted line Ac shown in FIG. 2. Also, the dashed lines Av1 and Av2 represent the angle of view of the imaging unit 20. For example, the dashed line Av1 may indicate the lower end in the vertical direction of the angle of view of the imaging unit 20, and the dashed line Av2 may indicate the upper end in the vertical direction of the angle of view of the imaging unit 20.

[0037] The imaging unit 20 arranged as shown in FIG. 2 can capture, for example, an image as shown in FIG. 3. FIG. 3 is a diagram showing an example of an image captured by the imaging unit 20 arranged as in FIG. 2. FIG. 3 may show, for example, a state where the result (image) captured by the imaging unit 20 is output (displayed) to the output unit 30 (for example, a display). As shown in FIG. 3, the object Ob is captured at a position substantially in the center of the image captured by the imaging unit 20. Also, the object Ob is located (standing) at the point P on the plane Gr.

[0038] Next, the operation of the system 100 according to an embodiment will be described.

[0039] First, an example of the operation of system 100 in an ideal environment will be described. FIG. 4 is a diagram for explaining an example of the operation of system 100 in an ideal environment. Here, the "ideal environment" may be an environment where, as shown in FIGS. 2 to 4, the coordinates in the Z-axis direction of the plane Gr and the coordinates in the Z-axis direction of the point P are both zero, that is, an environment where the plane Gr and the point P are at the same height.

[0040] In the situation (ideal environment) as shown in FIG. 4, the principle of the operation of measuring the distance So from the imaging unit 20 to the object Ob on the plane Gr using the system 100 will be described.

[0041] As shown in FIG. 4, the broken line L1 is a straight line representing the light incident on the imaging unit 20 from the point P (the grounding position B of the object Ob) where the object Ob is located in the image captured by the imaging unit 20 (position A). Hereinafter, such a straight line will also be referred to as the "first straight line". The boundary of an object such as the grounding position B of the object Ob can be determined or recognized by known image recognition processing or the like. As shown in FIG. 4, the first straight line L1 is a straight line passing through the position A and the point P (the grounding position B of the object Ob) where the object Ob is located. In the situation shown in FIG. 4, the first straight line L1 is a straight line indicating the light incident on the imaging unit 20 from the grounding position B of the object Ob.

[0042] Also, the broken line L2 may be a straight line representing the light incident on the imaging unit 20 from a predetermined position C of the object Ob (for example, the position C at the upper end of the object Ob) in the image captured by the imaging unit 20 (position A). Hereinafter, such a straight line will also be referred to as the "second straight line". The boundary of an object such as the position B at the upper end of the object Ob can be determined or recognized by known image recognition processing or the like. Also, the intersection point where the second straight line L2 intersects the plane Gr is denoted as point D. As shown in FIG. 4, the second straight line L2 is a straight line passing through the position A, a predetermined position C of the object Ob (for example, the position C at the upper end of the object Ob), and the point D. In the situation shown in FIG. 4, the second straight line L2 is a straight line indicating the light incident on the imaging unit 20 from the predetermined position C of the object Ob.

[0043] In a situation as shown in FIG. 4, the system 100 according to an embodiment can associate a pixel (picture element) constituting an image captured by the imaging unit 20 with the actual distance to a point on the plane Gr corresponding to the pixel. For example, as shown in FIG. 4, the electronic device 1 may obtain the distance α from the origin O to the point P (the grounding position B of the object Ob) where the object Ob is located by actual measurement or calculation or the like, and associate the point P with the distance α. For example, the electronic device 1 may store such an association between the point P and the distance α in the storage unit 12, or may acquire it from other electronic devices or the like via the communication unit 14.

[0044] By such an association, when a point P as shown in FIG. 3 is specified by the user from the input unit 40, for example, the electronic device 1 can output information indicating that the distance from the origin O to the point P is α. In this case, the electronic device 1 may display, for example, the image shown in FIG. 3 on the output unit 30. Then, the electronic device 1 may detect the position of the point specified by the user in the image shown in FIG. 3 by the input unit 40, for example. For example, assume that the user wants to know the actual distance to the point P in the image shown in FIG. 3. In this case, assume that the user makes an input indicating the point P by the input unit 40, for example. Then, the electronic device 1 may output information indicating the distance α to the point P to the output unit 30 or the like, for example. Specifically, when the distance α to the point P is associated with, for example, 8.6 m, the electronic device 1 may output information such as “Distance (So) from the origin O to the specified point (P) = 8.6 m” to the output unit 30 or the like.

[0045] Similarly, the electronic device 1 may associate not only the point P shown in FIG. 4 but also points at other locations with the pixels constituting the image captured by the imaging unit 20 and the distances to the points on the actual plane Gr corresponding to the pixels. For example, the electronic device 1 may store such an association between the points and the distances in the storage unit 12, or may acquire it from other electronic devices or the like via the communication unit 14. By performing such an association for each point on the plane Gr, when an arbitrary point such as that shown in FIG. 3 is specified by the user from, for example, the input unit 40, the electronic device 1 can output information indicating the distance from the origin O to the point.

[0046] On the other hand, in a case where the ideal environment as described above does not exist, it is also assumed that information indicating the distance from the origin O to the specified point is not correctly output due to the operation of the electronic device 1. Hereinafter, a case where information indicating the distance from the origin O to the specified point is not correctly output will be described.

[0047] FIGS. 5 and 6 are diagrams for explaining an example of the operation of the system 100 in a non-ideal environment. Here, the "non-ideal environment" may be an environment in which, unlike the situations (ideal environments) shown in FIGS. 2 to 4, the coordinate in the Z-axis direction of the plane Gr and the coordinate in the Z-axis direction of the point P are different, that is, an environment in which the plane Gr and the point P are at different heights.

[0048] As described above, land such as the flat surface Gr is generally difficult to assume to be strictly flat, and actually usually includes some unevenness or the like. FIG. 5 shows a state where the area around point P is slightly higher on the flat surface Gr. FIG. 6 shows a state where the area around point P is slightly lower on the flat surface Gr. In FIGS. 5 and 6, for the purpose of explanation, the state where the area around point P is high or low is shown somewhat exaggerated. Also, generally, the surface of flat land such as a road surface usually includes some unevenness or the like at every location, not limited to point P shown in FIG. 5 or FIG. 6. Also, there may be a certain degree of gradient on an actual road surface. In such a realistic environment, an aspect in which information indicating the distance from the origin O to a designated point is not correctly output will be described.

[0049] As shown in FIG. 5, when the area around point P is slightly higher on the flat surface Gr, point P and the grounding position B' of the object Ob do not have the same height. As shown in FIG. 5, let the height of the grounding position B' with respect to point P be h1. In such a case, even if in an ideal environment, a pixel (picture element) indicating a point on the flat surface Gr in the image and the distance to the corresponding point on the actual flat surface Gr are associated in advance, the information on the distance So to the designated point may not be correctly output.

[0050] In the ideal environment (FIGS. 2 to 4) as described above, when a pixel (picture element) indicating a point in the image captured by the imaging unit 20 and the distance to the corresponding point on the actual flat surface Gr are associated, the flat surface Gr and point P have the same height. Therefore, when using such associated data in a situation where the flat surface Gr and point P do not have the same height (FIG. 5 or FIG. 6), the information on the distance So to the designated point may not be correctly output.

[0051] For example, in an ideal environment (Figs. 2 to 4), pixels indicating points in the image captured by the imaging unit 20 are associated in advance with the distances to the points on the actual plane Gr corresponding to the pixels. After such association, assume a case where, in an actual non-ideal environment as shown in Fig. 5, the electronic device 1 calculates the distance So from the imaging unit 20 to the object Ob in the plane Gr.

[0052] In the ideal environment shown in Fig. 4, the point P where the object Ob is located and the grounding position B of the object Ob were at the same position. Therefore, in the situation shown in Fig. 4, the distance α between the origin O and the intersection point (point P) where the first straight line L1 intersects the plane Gr is the same as the distance So from the imaging unit 20 to the object Ob in the plane Gr. On the other hand, in the non-ideal environment shown in Fig. 5, the point P where the object Ob is located and the grounding position B’ of the object Ob are at different heights. Therefore, in the situation shown in Fig. 5, the distance α’ between the origin O and the intersection point (point F) where the first straight line L1’ intersects the plane Gr is not the same as the distance So from the imaging unit 20 to the object Ob in the plane Gr (the distance α’ is larger than the distance So).

[0053] Therefore, in Fig. 5, when calculating the distance So from the origin O to the point P where the object Ob is located, actually, the distance α’ between the origin O and the intersection point F where the first straight line L1’ intersects the plane Gr is calculated. That is, in this case, the value (α’) calculated as the distance So is larger than the actual distance So.

[0054] Also, in the situation shown in Fig. 4, the distance between the origin O and the intersection point (point D) where the second straight line L2 intersects the plane Gr is shown as the distance β. On the other hand, in the situation shown in Fig. 5, the distance between the origin O and the intersection point (point D’) where the second straight line L2’ intersects the plane Gr is shown as the distance β’. The distance β’ shown in Fig. 5 is larger than the distance β shown in Fig. 4.

[0055] As can be seen from the comparison between FIGS. 4 and 5, in the situation shown in FIG. 5, when calculating the length (height) Ho of the object Ob, actually the length (height) Ho’ of the object Ob shown in FIG. 5 is calculated. That is, in the situation shown in FIG. 5, when calculating the length (height) of the object Ob, it is calculated to be larger than the actual length (height) of the object Ob.

[0056] Also, as shown in FIG. 6, even when the area around the point P on the plane Gr is slightly lower, the point P and the grounding position B” of the object Ob are not at the same height. As shown in FIG. 6, let the depth of the grounding position B” with respect to the point P be h2. Even in such a case, in an ideal environment, even if the distance between the pixel (picture element) indicating the point on the plane Gr in the image and the actual point on the plane Gr corresponding to the pixel is associated in advance, the information on the distance So to the specified point may not be correctly output.

[0057] For example, in an ideal environment (FIGS. 2 to 4), assume that the distance between the pixel (picture element) indicating the point in the image captured by the imaging unit 20 and the actual point on the plane Gr corresponding to the pixel is associated in advance. After such an association, assume a case where, in the non-ideal actual environment shown in FIG. 6, the electronic device 1 calculates the distance So from the imaging unit 20 on the plane Gr to the object Ob.

[0058] In the non-ideal environment shown in FIG. 6, the point P where the object Ob is located and the grounding position B” of the object Ob are at different heights. Therefore, in the situation shown in FIG. 6, the distance α” between the origin O and the intersection point (point M) where the first straight line L1” intersects the plane Gr is not the same as the distance So from the imaging unit 20 on the plane Gr to the object Ob (the distance α” is smaller than the distance So).

[0059] Therefore, in FIG. 6, when calculating the distance So from the origin O to the point P where the object Ob is located, actually, the distance α” between the origin O and the intersection point M where the first straight line L1” intersects the plane Gr is calculated. That is, in this case, the value (α”) calculated as the distance So is smaller than the actual distance So.

[0060] In the situation shown in FIG. 5, the distance between the origin O and the intersection point (point D”) where the second straight line L2” intersects the plane Gr is shown as the distance β”. The distance β” shown in FIG. 5 is smaller than the distance β shown in FIG. 4.

[0061] As can be seen from the comparison between FIG. 4 and FIG. 6, when calculating the length (height) Ho of the object Ob in the situation shown in FIG. 6, actually, the length (height) Ho” of the object Ob shown in FIG. 6 is calculated. That is, in the situation shown in FIG. 6, when calculating the length (height) of the object Ob, it is calculated to be smaller than the actual length (height) of the object Ob.

[0062] As described above, when the point P where the object Ob is located and the plane Gr are not at the same height (FIG. 5 or FIG. 6), the calculated length (Ho’ or Ho”) of the object Ob does not become the same as the actual length Ho of the object Ob. Therefore, the electronic device 1 according to an embodiment may correct, for example, the distance α’ or α” by obtaining the actual height Ho of the object Ob as known information and comparing it with the calculated length (Ho’ or Ho”) of the object Ob. In this case, the electronic device 1 may correct the side lengths of the similar figures in FIG. 5 or FIG. 6 based on the ratio between the actual height Ho of the object Ob and the calculated length (Ho’ or Ho”) of the object Ob. By correcting the distance α’ or α”, the electronic device 1 can improve the accuracy of the distance So from the imaging unit 20 to the object Ob in the plane Gr.

[0063] Hereinafter, a method by which the control unit 10 corrects the distance So from the imaging unit 20 to the object Ob in the plane Gr based on the known height Ho of the object Ob will be further described.

[0064] In Fig. 5, △AOF (a right triangle with vertices at points A, O, and F) and △B’PF (a right triangle with vertices at points B’, P, and F) are similar. Therefore, the following relationship of formula (1) holds. OA:PB’ = OF:PF (1) In the above formula (1), the notation “OA” represents the length of the line segment connecting points O and A. Also, the notation “PB’” represents the length of the line segment connecting points P and B’, and the same applies to other notations.

[0065] Here, from Fig. 5, the length of PF can be expressed as OF - OP. From these, the following relationship of formula (2) holds. PB’ = OA(1 - OP / OF) (2)

[0066] From Fig. 5, the length of PB’ is equal to the length of OJ, and this length is denoted as h1. Also, as shown in Fig. 5, the length of OP is denoted as So. Furthermore, the length of OA is the height in the vertical direction where the imaging unit 20 is installed, and this length is denoted as Ha. From the above, the following relationship of formula (3) holds. h1 = Ha(1 - So / OF) (3)

[0067] From the above formula (3), for example, if Ha is known and the distance OF to the object Ob is estimated by the electronic device 1, the actual height h1 of the location where the object Ob is located can be estimated.

[0068] Also, in Fig. 5, △AOD’ (a right triangle with vertices at points A, O, and D’) and △C’PD’ (a right triangle with vertices at points C’, P, and D’) are similar. Therefore, the following relationship of formula (4) holds. OD’:PD’ = OA:PC’ (4)

[0069] Here, from Fig. 5, the length of PD’ can be expressed as OD’ - So. Also, the length of PC’ can be expressed as h1 + B’C’, and furthermore, the length of B’C’ is the length in the vertical direction of the object Ob, and this length is denoted as Ho. The length of OA is denoted as Ha as described above.

[0070] From the above, the relationship of the following formula (5) holds. OD’:(OD’-So)=Ha:(h1+Ho) (5)

[0071] When the above formula (5) is transformed, it becomes the following formula (6). (Ha·So)+(OD’·h1)=(OD’·Ha)-(OD’·Ho) (6)

[0072] When substituting the above-mentioned h1 (the above formula (3)) into the above formula (6), it becomes the following formula (7). So=-OD’·Ho / {Ha(1-OD’ / OF)} (7)

[0073] Also, in FIG. 5, △GFD’ (a right triangle with vertices at points G, F, and D’) and △AOD’ (a right triangle with vertices at points A, O, and D’) are similar. Therefore, the relationship of the following formula (8) holds. Ha:GF=OD’:FD’ (8)

[0074] Here, from FIG. 5, the length of FD’ can be expressed as OD’-OF. Also, in FIG. 5, the relationship of the following formula (9) holds. GF:Ho=OF:So (9)

[0075] Summarizing the above, the relationship of the following formula (10) holds. GF=OF·Ho / So (10)

[0076] From FIG. 5, the length of OF can be expressed as α’, and the length of OD’ can be expressed as β’. Then, the above formula (3) can be expressed as the following formula (11). h1=Ha(1-So / α’) (11)

[0077] Also, the above formula (7) can be expressed as the following formula (12). So=-β’·Ho / {Ha(1-β’ / α’)} (12)

[0078] Furthermore, from FIG. 5, the length of GF can be represented as Ho’. Then, the above formula (10) can be represented as the following formula (13). Ho’ = α’·Ho / So (13)

[0079] According to the above formula (12), the control unit 10 of the electronic device 1 can calculate the distance So from the distances α’ and β’ shown in FIG. 5 with the values of Ha and Ho being known. The known value Ha represents the height of the imaging unit 20 (the length of the line segment OA). Hereinafter, the information representing the value of Ha is also referred to as "first height information". The known value Ho represents the length (height) Ho of the object Ob (the length of the line segment C’B’). Hereinafter, the information representing the value of Ho is also referred to as "second height information". Also, the distance α’ is a distance calculated from the image captured by the imaging unit 20 (the length of the line segment OF). Hereinafter, the information representing the value of the distance α’ is also referred to as "first distance information". The distance β’ is a distance calculated from the image captured by the imaging unit 20 (the length of the line segment OD’). Hereinafter, the information representing the value of the distance β’ is also referred to as "second distance information". And the distance So is the distance from the imaging unit 20 to the object Ob in the plane Gr (the length of the line segment OP). Hereinafter, the information representing the value of the distance So is also referred to as "third distance information". That is, the control unit 10 acquires the first height information, the second height information, the first distance information, and the second distance information. Then, the control unit 10 generates the third distance information by using (based on) the first height information, the second height information, the first distance information, and the second distance information.

[0080] As described above, the method for accurately calculating the distance So by the electronic device 1 in the situation shown in FIG. 5 has been described. Also, in the situation shown in FIG. 6, by replacing the first distance information α’ with the first distance information α” and the second distance information β’ with the second distance information β”, the electronic device 1 can accurately calculate the distance So.

[0081] The electronic device 1 according to an embodiment can generate information indicating the distance to the object Ob based on the image of the object Ob captured by the imaging unit 20. As described above, according to the electronic device 1 according to an embodiment, the third distance information can be calculated or generated based on the first height information, the second height information, the first distance information, and the second distance information.

[0082] The first height information is information indicating the actual height Ha of the imaging unit 20 with respect to a predetermined plane (for example, the ground) Gr. The second height information is information indicating the actual height Ho from the grounding position B' of the object Ob to a predetermined position (for example, the upper end) C' of the object Ob. The first distance information is distance information obtained based on the image in which the object Ob is captured, and is information indicating the distance α corresponding to the grounding position B' of the object Ob. The second distance information is distance information obtained based on the image in which the object Ob is captured, and is information indicating the distance β corresponding to a predetermined position (for example, the position of the upper end of the object Ob) C' of the object Ob. Further, the third distance information is information indicating the distance So to the object Ob. In this way, according to the electronic device 1, the distance (corresponding to the length of the line segment OP) from the imaging unit 20 to the object Ob on the plane Gr can be accurately obtained.

[0083] Further, the first distance information may be information indicating the distance α between the intersection point F where the first straight line L1' passing through the position A of the imaging unit 20 and the grounding position B' of the object Ob intersects the predetermined plane Gr, and the intersection point O of the perpendicular line dropped from the position A of the imaging unit 20 to the predetermined plane Gr. Further, the second distance information may be information indicating the distance β between the intersection point D' where the second straight line L2' passing through the position A of the imaging unit 20 and the predetermined position C' of the object Ob intersects the predetermined plane Gr, and the intersection point O of the perpendicular line dropped from the position A of the imaging unit 20 to the predetermined plane Gr.

[0084] Further, when the difference between the vertical component of the first straight line L1' and the vertical component of the second straight line L2' at the position of the object Ob is equal within a predetermined error from the height indicated by the second height information, the electronic device 1 may generate the third distance information.

[0085] Further, the third distance information may be information indicating the distance So between the intersection point O of the perpendicular line dropped from the position A of the imaging unit 20 to a predetermined plane Gr and the intersection point P of the perpendicular line dropped from the grounding position B' of the object Ob to the predetermined plane Gr. Further, the electronic device 1 may generate information indicating the height h1 of the grounding position B' of the object Ob with respect to the predetermined plane Gr based on the third distance information.

[0086] In one embodiment, the electronic device 1 may correct, based on the third distance information, for example, parameters for converting coordinates on an image captured by the imaging unit 20 into coordinates on a predetermined plane Gr. Further, the electronic device 1 may correct, based on the third distance information, for example, parameters representing the distance between coordinates on an image captured by the imaging unit 20 and the intersection point O (origin O) of the perpendicular line dropped from the position A of the imaging unit 20 to a predetermined plane Gr. Furthermore, the electronic device 1 may acquire the second height information from another electronic device other than the electronic device 1.

[0087] In the above-described embodiment, the electronic device 1 has been described as calculating the distance So from the imaging unit 20 to the object Ob in the plane Gr (i.e., the length of the orthographic projection of the distance from the imaging unit 20 to the object Ob onto the plane Gr). However, in one embodiment, the electronic device 1 may obtain the distance from the position A of the imaging unit 20 to the position of the object Ob (for example, the distance between point A and point P shown in FIG. 4). In this case, the electronic device 1 may convert, for example, the calculated length of the distance So into the length of the line segment AP based on the ratio of the length of the distance So and the length of the line segment AP.

[0088] The electronic device 1 according to the above-described embodiment calculates (estimates) the distance to an object Ob such as a person whose height is known. However, in one embodiment, the object Ob does not necessarily have to be a person. In one embodiment, the electronic device 1 (control unit 10 thereof) may calculate the distance based on, for example, a fixed installation object (such as a sign) imaged by the imaging unit 20. Here, for a fixed installation object whose height is known, its known value may be used. Also, for a fixed installation object whose height is unknown, the height may be actually measured at that time. By such calculation, the electronic device 1 may perform calibration for calculating (estimating) various distances.

[0089] In the above-described embodiment, the electronic device 1 calculated the distance information to the object Ob based on the known height of the object Ob imaged by the imaging unit 20. Conversely, in one embodiment, the electronic device 1 may calculate the height of the object Ob imaged by the imaging unit 20 based on known accurate distance information. For example, assume that a predetermined distance from the imaging unit 20 or a predetermined distance from point O shown in FIG. 4 is known. In this case, for example, when a person who is the object Ob imaged by the imaging unit 20 moves and reaches the position of the predetermined distance, the electronic device 1 can calculate the height of the object Ob (for example, the height of the person) by inverse calculation based on the known distance information.

[0090] As described above, when the electronic device 1 calculates the height of the object Ob (for example, the height of a person) by inverse calculation, it may be used in combination with a sensor having high ranging accuracy such as LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging). In this case, in the electronic device 1, it may be used in combination with a sensor having high ranging accuracy only, for example, during calibration, or it may be fused so that only a part of the area overlaps.

[0091] (Setting of Correction Priority) Next, the setting of the correction priority by the electronic device 1 according to one embodiment will be further described.

[0092] As described above, according to the electronic device 1 according to one embodiment, it is possible to correct various parameters such as correcting the distance So (FIGS. 4 to 6) from the imaging unit 20 to the object Ob in the plane Gr. Here, the various parameters may be, for example, parameters for converting the coordinates on the image captured by the imaging unit 20 into coordinates on a predetermined plane Gr. Further, the various parameters may be, for example, parameters representing the distance from the intersection point O (origin O) of the perpendicular line dropped from the position A of the imaging unit 20 to a predetermined plane Gr to the coordinates on the image captured by the imaging unit 20. Thus, the various parameters corrected by the electronic device 1 according to one embodiment may be parameters for associating the coordinates on the image captured by the imaging unit 20 with the actual position in the space where the image was captured. Such parameters may hereinafter be simply referred to as "parameters". On the other hand, it is possible in principle to perform the above-described correction for all the pixels (picture elements) included in the image captured by the imaging unit 20, but the processing load may increase. Therefore, in one embodiment, instead of performing the correction of various parameters for all the pixels included in the image captured by the imaging unit 20, the priority for performing the correction may be set. Hereinafter, such an embodiment will be described.

[0093] As described above, when the point P where the object Ob is located and the plane Gr are not at the same height (FIG. 5 or FIG. 6), the calculated length (Ho' or Ho") of the object Ob does not become the same as the actual length Ho of the object Ob. Here, the greater the difference between the height of the point P where the object Ob is located and the height of the plane Gr, the greater the deviation of the calculated length (Ho' or Ho") of the object Ob from the actual length Ho of the object Ob. For this reason, the greater the difference between the height of the point P where the object Ob is located and the height of the plane Gr, the greater the deviation of the calculated distance α' or α" to the object Ob from the actual distance So to the object Ob.

[0094] Therefore, in one embodiment, the control unit 10 may set a higher priority for correcting various parameters at points where the difference between the height of the point P where the object Ob is located and the height of the plane Gr is larger in the image captured by the imaging unit 20. Also, the control unit 10 may set a higher priority for correcting various parameters for pixels including points where the difference in height as described above becomes larger in the image captured by the imaging unit 20. Further, the control unit 10 may set a higher priority for correcting various parameters for regions composed of a plurality of pixels including points where the difference in height as described above becomes larger or pixels including such points in the image captured by the imaging unit 20. Furthermore, the control unit 10 may set a lower priority for correcting various parameters at points where the difference between the height of the point P where the object Ob is located and the height of the plane Gr is smaller in the image captured by the imaging unit 20. Also, the control unit 10 may set a lower priority for correcting various parameters for pixels including points where the difference in height as described above becomes smaller in the image captured by the imaging unit 20. Additionally, the control unit 10 may set a lower priority for correcting various parameters for regions composed of a plurality of pixels including points where the difference in height as described above becomes smaller or pixels including such points in the image captured by the imaging unit 20.

[0095] Here, in the image captured by the imaging unit 20, a point where the difference between the height of the point P where the object Ob is located and the height of the plane Gr becomes relatively large tends to be a position where the height difference from the periphery of the point becomes large. Therefore, in one embodiment, the control unit 10 may track the position (including the height) of, for example, a moving object (e.g., an automobile) from the image captured by the imaging unit 20. By tracking the position (including the height) of the moving object from the image captured by the imaging unit 20, the control unit 10 can grasp the temporal change of the position (including the height) of the moving object. Then, based on the temporal change of the position (including the height) of the moving object obtained from the information indicating the position of the moving object, the control unit 10 may set the priority of the point where the above-described parameters should be corrected in the image captured by the imaging unit 20. Further, based on the temporal change of the position of the moving object as described above, the control unit 10 may set the priority of the pixel including the point where the parameter should be corrected in the image captured by the imaging unit 20. Further, based on the temporal change of the position of the moving object as described above, the control unit 10 may set the priority of the region composed of a plurality of pixels including the point where the parameter should be corrected or the pixel including the point in the image captured by the imaging unit 20. In these cases, the control unit 10 may set the priority for each of the various parameters to be corrected itself. Thus, based on the temporal change of the position of the moving object obtained from the information indicating the position of the moving object, the control unit 10 may set the priority for correcting the parameter that associates the coordinates on the image captured by the imaging unit 20 with the position in the space where the image was captured. Here, based on the temporal change of the position (including the height) of the moving object obtained from the information indicating the position of the moving object, the control unit 10 may also set the priority for correcting the information indicating the position of the moving object. In the following description, the moving object is assumed to be an automobile as an example. Further, the moving object may include, for example, a motorcycle and various other means of transportation that can travel on the road.

[0096] In order to perform the above-described processing, the control unit 10 may generate a graph showing the temporal change in the distance of a predetermined moving object based on the images captured by the imaging unit 20 at each moment. FIG. 7 is a graph showing an example of the temporal change in the distance of a predetermined moving object generated by the electronic device 1 according to an embodiment.

[0097] FIG. 7 is a diagram showing an example of a graph showing the temporal change in the distances of a plurality of moving objects (moving object Obα, moving object Obβ, moving object Obγ, and moving object Obδ) generated by the electronic device 1 according to an embodiment. The control unit 10 of the electronic device 1 can generate a graph of the temporal change in the distance of each moving object from the images captured by the imaging unit 20 at each moment. Each moving object is assumed to exist in area 1 from time t0 to t2 shown in FIG. 7 and in area 2 from time t1 to t3. That is, area 1 and area 2 shown in FIG. 7 partially overlap. Here, the distance of each moving object represents the actual position of the moving object in the space where the image was captured, generated using a parameter that associates the coordinates on the image captured by the imaging unit 20 with the actual position in the space where the image was captured.

[0098] For example, area 1 and area 2 may be areas captured by the imaging unit 20 of the electronic device 1 installed separately. That is, area 1 may be an area captured by the first imaging unit 20 of the electronic device 1, and area 2 may be an area captured by the second imaging unit 20 of the electronic device 1. The first imaging unit 10 and the second imaging unit 10 may each transmit various information to the same electronic device 1, or may each transmit various information to a plurality of different electronic devices 1 that control them. In the latter case, the plurality of different electronic devices 1 may share various information by communicating with each other and may each be capable of processing various information.

[0099] In FIG. 7, the dashed line St indicates a straight line. Therefore, the dashed line St indicates a uniform linear motion. If any of the plurality of moving bodies moves from area 1 to area 2 at a constant speed, a graph having a linear slope, like the dashed line St shown in FIG. 7, is generated.

[0100] On the other hand, for example, like the moving body Obα and the moving body Obδ in the time period from time t0 to t1 shown in FIG. 7, a graph having a portion with a relatively large slope will include a relatively large height difference. Also, for example, the moving body Obα in the time period from time t2 to t3 shown in FIG. 7 is also a graph having a portion with a relatively large slope, and thus includes a relatively large height difference. Thus, in the graph as shown in FIG. 7, a portion with a relatively large slope becomes a point including a relatively large height difference.

[0101] The plurality of moving bodies (moving body Obα, moving body Obβ, moving body Obγ, and moving body Obδ) shown in FIG. 7 may follow the same path or different paths. When these moving bodies follow the same path, the control unit 10 can grasp the tendency of the plurality of moving bodies following the same path by tracking the data of the plurality of moving bodies. Conversely, when tracking the data of the plurality of moving bodies, if there is a moving body whose data deviates more than a predetermined amount, the control unit 10 may ignore the behavior of the moving body as a moving body having a behavior different from the general behavior.

[0102] As described above, the electronic device 1 according to one embodiment may include, for example, an imaging unit 20 and a control unit 10. The control unit 10 may generate information indicating the position of the moving object, which represents the actual position in the space where the image of the moving object is captured, based on the image of the moving object captured by the imaging unit 20. Further, the control unit 10 corrects a parameter that associates the coordinates on the image captured by the imaging unit 20 with the actual position in the space where the image is captured, based on the temporal change of the position of the moving object obtained from the information indicating the position of the moving object. In one embodiment, the control unit 10 may set a priority for correcting the parameter based on the temporal change of the position of the moving object obtained from the information indicating the position of the moving object. In one embodiment, the control unit 10 may preferentially execute the correction of the parameter starting from those with a high priority set for correcting the parameter.

[0103] Further, when the temporal change of the position of any one of the plurality of moving objects obtained from the information indicating the position of the moving object exceeds a predetermined threshold, the control unit 10 may set the priority for correcting the parameter to be higher by a predetermined level. Here, the predetermined threshold may be appropriately set based on, for example, a demonstration experiment according to the environment where the electronic device 1 is installed. Also, the predetermined level of the priority for correcting the parameter may be appropriately set based on, for example, a demonstration experiment according to the environment where the electronic device 1 is installed. Furthermore, the control unit 10 may set the priority for correcting the parameter higher as the temporal change of the position of the moving object obtained from the information indicating the position of the moving object is larger.

[0104] Also, the control unit 10 may generate information indicating the position of the moving object in the height direction (Z-axis direction) as the information indicating the position of the moving object. In this case, the control unit 10 may set a priority for correcting the parameter based on the temporal change of the position of the moving object in the height direction obtained from the information indicating the position of the moving object in the height direction (Z-axis direction) of the moving object.

[0105] Further, the control unit 10 may generate information indicating a plurality of positions of the moving object 1 imaged by the imaging unit 20. In this case, the control unit 10 may set the priority for correcting the parameter based on the temporal change of the plurality of positions of the moving object 1 obtained from the information indicating the plurality of positions of the moving object 1.

[0106] Further, the control unit 10 may generate information indicating the positions of the plurality of moving objects based on the images of the plurality of moving objects imaged by the imaging unit 20. In this case, the control unit 10 may set the priority for correcting the parameter based on the temporal change of the positions of the plurality of moving objects obtained from the information indicating the positions of the plurality of moving objects.

[0107] Further, when the control unit 10 exceeds a predetermined threshold value set based on the temporal change of the positions of the plurality of moving objects, the control unit 10 may set the priority for correcting the parameter to be higher by a predetermined level. Here, the predetermined threshold value may be appropriately set based on, for example, a demonstration experiment according to the environment where the electronic device 1 is installed. Also, the predetermined level of the priority for correcting the parameter may be appropriately set based on, for example, a demonstration experiment according to the environment where the electronic device 1 is installed. Further, the control unit 10 may generate information indicating the positions of the moving objects at a plurality of points based on the images of the moving objects at the plurality of points imaged by the imaging unit 20. In this case, the control unit 10 may set the priority for correcting the parameter based on the temporal change of the positions of the moving objects at the plurality of points obtained from the information indicating the positions of the moving objects at the plurality of points.

[0108] Furthermore, the control unit 10 may increase the points for setting the priority for correcting the parameter based on the temporal change of the position of the moving object obtained from the information indicating the position of the moving object.

[0109] As described above, the electronic device 1 according to one embodiment can determine a point where the measurement error becomes large from the result of recognizing a location with a height difference. Therefore, according to the electronic device 1 according to one embodiment, the priority order for correcting various parameters can be set. According to the electronic device 1 according to one embodiment, since correction can be preferentially performed from the point where the error becomes large, the processing load can be reduced.

[0110] (Embodiment of Correction) Next, the embodiment of correction by the electronic device 1 according to one embodiment will be further described.

[0111] When setting the priority order for correcting various parameters as described above, when actually performing the correction, the control unit 10 may correct, for example, the distance So (Figs. 4 to 6) from the imaging unit 20 to the object Ob in the plane Gr as described above. Also, when actually performing the correction in the above-described case, the control unit 10 may correct, for example, the parameters for converting the coordinates of the pixels including the object Ob on the image captured by the imaging unit 20 into the coordinates on a predetermined plane Gr. Also, when actually performing the correction in the above-described case, the control unit 10 may correct, for example, the parameters representing the distance from the intersection point O (origin O) of the perpendicular line dropped from the position A of the imaging unit 20 to a predetermined plane Gr to the coordinates of the pixels including the object Ob on the image captured by the imaging unit 20. On the other hand, as another correction method, in one embodiment, the control unit 10 may correct, for example, the distance So from the imaging unit 20 to the object Ob in the plane Gr based on the information received from a moving body such as a connected car. Also, the control unit 10 may correct, for example, the parameters for converting the coordinates of the pixels including the object Ob on the image captured by the imaging unit 20 into the coordinates on a predetermined plane Gr based on the information received from a moving body such as a connected car. Also, the control unit 10 may correct, based on the above-described information, the parameters representing the distance from the intersection point O (origin O) of the perpendicular line dropped from the position A of the imaging unit 20 to a predetermined plane Gr to the coordinates of the pixels including the object Ob on the image captured by the imaging unit 20. Hereinafter, such an embodiment will be further described.

[0112] FIG. 8 is a diagram for explaining an embodiment of correction by the electronic device 1 according to one embodiment.

[0113] In FIG. 8, Ob1 represents a moving body such as a connected car traveling on a lane of a road. As shown in FIG. 8, it is assumed that the moving body Ob1 is traveling at a speed V on the lane of the road from area 1 toward area 2. As shown in FIG. 8, area 1 and area 2 may at least partially overlap.

[0114] The moving body Ob1, which is a connected car, can transmit various types of information to the electronic device 1. Therefore, the electronic device 1 can receive various types of information transmitted from the moving body Ob1, which is a connected car. The various types of information received by the electronic device 1 from the moving body Ob1 may include, for example, the position information of the moving body Ob1. Also, the moving body Ob1, which is a connected car, can acquire information such as an accurate time from, for example, an NTP (Network Time Protocol) server.

[0115] As shown in FIG. 8, the position of the moving body Ob1 is set as an arbitrary point (first position). Also, as shown in FIG. 8, it is assumed that there is a calibration point (second position) in the traveling direction of the moving body Ob1. The position (coordinates) of the calibration point (second position) or the distance from the imaging device 20 is assumed to be known. Here, the calibration point (second position) by the electronic device 1 may be, for example, where the electronic device 1 is installed, or where the imaging unit 20 is installed. In the example shown in FIG. 8, the moving body Ob1 at the first position is in a state of approaching the calibration point at the second position. On the other hand, the moving body Ob1 at the first position may be in a state of moving away from the calibration point at the second position.

[0116] Let the distance between the calibration point (second position) shown in FIG. 8 and an arbitrary point (first position) which is the position of the moving body Ob1 be the distance L. Here, the distance L is the distance estimated from the image captured by the imaging unit 20 in the electronic device 1. Also, let the predetermined time including the time point when this distance L is detected be the predetermined time t. Further, among the information transmitted from the moving body Ob1 which is a connected car and received by the electronic device 1, let the speed of the moving body Ob1 be V. Then, let the product of the speed V of the moving body Ob1 and the above-described predetermined time t be the distance L'. In this case, the control unit 10 may correct various parameters based on the difference between the above-described distance L and the distance L'.

[0117] For example, the control unit 10 may correct the distance So from the imaging unit 20 to the object Ob in the plane Gr based on the difference between the above-described distance L and the distance L'. For example, when the first position and the second position exist on one straight line extended from the intersection O (origin O), the control unit 10 corrects the distance So from the imaging unit 20 to the moving body Ob1 when the moving body Ob1 exists at the first position based on the difference. For example, when the first position and the second position do not exist on one straight line extended from the intersection O (origin O), the control unit 10 corrects the distance So from the imaging unit 20 to the moving body Ob1 when the moving body Ob1 exists at the first position using the following information. That is, in this case, the control unit 10 may use the distance from the intersection O (origin O) to the second position, the directions from the intersection O (origin O) to the first position and the second position based on the image captured by the imaging unit 20, the direction from the second point to the first point based on the direction in which the moving body Ob1 travels acquired from the moving body Ob1, and the distance (distance L') obtained by adding or subtracting the above-described difference to or from the distance L.

[0118] Further, for example, the control unit 10 may correct a parameter for converting the coordinates of pixels including the object Ob on the image captured by the imaging unit 20 into coordinates on a predetermined plane Gr based on the difference between the above-described distance L and distance L'. Further, for example, the control unit 10 may correct a parameter representing the distance from the intersection point O (origin O) of the perpendicular line dropped from the position A of the imaging unit 20 to the predetermined plane Gr to the coordinates of pixels including the object Ob on the image captured by the imaging unit 20 based on the difference as described above. For example, when the first position and the second position exist on one straight line extended from the intersection point O (origin O), the control unit 10 corrects a parameter representing the distance from the intersection point O (origin O) to the coordinates of pixels including the moving object Ob1 when the moving object Ob1 exists at the first position based on the difference. For example, when the first position and the second position do not exist on one straight line extended from the intersection point O (origin O), the control unit 10 corrects a parameter representing the distance from the intersection point O (origin O) to the coordinates of pixels including the moving object Ob1 when the moving object Ob1 exists at the first position using the following information. That is, in this case, the control unit 10 may use the distance from the intersection point O (origin O) to the second position, the directions from the intersection point O (origin O) to the first position and the second position based on the image captured by the imaging unit 20, the direction from the second point to the first point based on the direction in which the moving object Ob1 travels acquired from the moving object Ob1, and the distance (distance L') obtained by adding or subtracting the above-described difference to or from the distance L.

[0119] The control unit 10 may correct the parameters for converting the coordinates at an arbitrary point (first position) on the image captured by the imaging unit 20 into the coordinates on a predetermined plane Gr based on the difference between the above-described distance L and distance L'. The control unit 10 may correct the parameter representing the distance from the imaging unit 20 (intersection point O) to an arbitrary point (first position) on the image captured by the imaging unit 20 based on the difference between the above-described distance L and distance L'. For example, when the first position and the second position exist on one straight line extending from the intersection point O (origin O), the control unit 10 corrects the parameter representing the distance to the first position based on the difference. For example, when the first position and the second position do not exist on one straight line extending from the intersection point O (origin O), the control unit 10 corrects the parameter representing the distance to the first position using the following information. That is, in this case, the control unit 10 may use the distance from the intersection point O (origin O) to the second position, the directions from the intersection point O (origin O) to the first position and the second position based on the image captured by the imaging unit 20, the direction from the second point to the first point based on the direction in which the moving body Ob1 travels acquired from the moving body Ob1, and the distance (distance L') obtained by adding or subtracting the above-described difference to or from the distance L. Also in this embodiment, the various parameters corrected by the electronic device 1 may be parameters for associating the coordinates on the image captured by the imaging unit 20 with the actual positions in the space where the image is captured.

[0120] In one embodiment, the control unit 10 may execute the process by adopting, for example, the median value of the information received from a plurality of moving bodies that are connected cars. In this way, the electronic device 1 can reduce the error of the correction value of various parameters such as the distance So from the imaging unit 20 to the object Ob in the plane Gr.

[0121] In addition, in one embodiment, the control unit 10 may preferentially correct from points with large measurement errors by recording a plurality of positions received from a plurality of moving bodies that are connected cars. In this way, the electronic device 1 can average the data by using the data of the plurality of moving bodies to create correction data. Therefore, the electronic device 1 can generate correction data that is less likely to be affected by sudden acceleration and / or sudden deceleration by each moving body.

[0122] FIG. 9 is a graph showing the time change of each detected (measured) speed V by the electronic device 1 based on an image in which a plurality of moving bodies are imaged by the imaging unit 20. In the graph of FIG. 9, the horizontal axis represents time, and the vertical axis represents the speed V of the moving body. FIG. 9 shows, as an example, the time change of the speeds of three moving bodies (moving body Ob1, moving body Ob2, and moving body Ob3) traveling at a predetermined point.

[0123] If the moving body is traveling in a lane on a normal road, each moving body will perform uniform motion or uniformly accelerated motion. For example, in an ideal (flat) situation as shown in FIG. 4, each moving body is assumed to perform uniform motion or uniformly accelerated motion. However, as shown in FIG. 9, at a specific location, it is observed that each moving body does not perform simple uniform motion or uniformly accelerated motion. For example, in a non-ideal (not flat, for example, having unevenness) situation as shown in FIG. 5 or FIG. 6, it is assumed that each moving body does not perform simple uniform motion or uniformly accelerated motion. Therefore, when a characteristic pattern is detected (for example, at time t1' in FIG. 9), such as an error being included in the speed detection, the control unit 10 may determine that the situation is not ideal, such as the position where the moving body is traveling having unevenness. Then, the control unit 10 may correct various parameters at such a position, or may increase the priority of correcting various parameters.

[0124] In the above-described process, one lane on which the moving object travels may be simply regarded as a straight line for processing. By doing so, when the electronic device 1 measures the speed of the moving object during travel, the movement of the moving object can be regarded as a linear motion, so that the processing load can be reduced. In this case, by regarding one lane as a straight line, correction can be performed in units of one lane. By such processing, for example, when a passenger car and / or a large vehicle is used as the moving object, correction of various parameters can be performed.

[0125] In one embodiment, the electronic device 1 may include a plurality of imaging units 20 (which may include, for example, a first imaging unit 21 and a second imaging unit 22). In this case, the control unit 10 may fuse the information received from the plurality of imaging units 20. When fusing the information in this way, in the control unit 10, an arbitrary point and a calibration point may be set so as to span from area 1 to area 2. By doing so, for example, when using a plurality of electronic devices 1 and / or a plurality of imaging units 20, the same correction value can be set, and the deviation in the coordinate system for each area can be eliminated.

[0126] As described above, the electronic device 1 according to one embodiment may include, for example, an imaging unit 20 and a control unit 10. The control unit 10 may generate information indicating the position of a predetermined point in the image, which represents the position in the space where the image is captured, based on the image captured by the imaging unit 20. Here, the information indicating the position of the predetermined point represents the actual position in the space where the image is captured. The control unit 10 may use the position of the predetermined point in the image captured by the imaging unit 20 as the first position (an arbitrary point). Also, the control unit 10 may use the position of a predetermined known point as the second position (calibration point). The control unit 10 may use the distance between the first position and the second position as the first distance L. Further, the control unit 10 may use the distance obtained from the speed information and time information of a moving body moving from the first position to the second position or from the second position to the first position as the second distance L'. Then, the control unit 10 may correct the parameter that associates the coordinates on the image captured by the imaging unit 20 with the actual position in the space where the image is captured based on the first distance L and the second distance L'. In this case, the control unit 10 may correct the parameter based on the difference between the first distance L and the second distance L'.

[0127] Also, the control unit 10 may set a higher priority for correcting the parameter as the difference between the first distance L and the second distance L' is larger.

[0128] In one embodiment, the electronic device 1 may include a plurality of imaging units 20. The imaging unit 20 may include, for example, at least a first imaging unit 21 and a second imaging unit 22. In this case, the first imaging unit 21 and the second imaging unit 22 may be arranged to capture images each including the first position and the second position. In this case, the control unit 10 may correct the parameter based on each of the images captured by the first imaging unit 21 and the second imaging unit 22.

[0129] Also, in one embodiment, the control unit 10 may correct the parameter based on the second distance obtained from the speed information and time information obtained from a moving body such as a connected car.

[0130] Further, the control unit 10 may correct the parameter based on a second distance obtained from average speed information and average time information of a plurality of moving bodies that move from the first position to the second position or from the second position to the first position.

[0131] As described above, when the electronic device 1 performs calibration, the distance to a predetermined object can be obtained with relatively high accuracy at the points actually measured. On the other hand, at the points estimated without actual measurement by the electronic device 1, the measured distance may deviate from the actual distance due to factors such as the unevenness of the road surface.

[0132] According to the electronic device 1 according to one embodiment, the measured value of the position of the connected car can be corrected from the speed information of the connected car during traveling and the information based on the image captured by the imaging unit 20. Further, according to the electronic device 1 according to one embodiment, by correcting the measured value of the position of the connected car, the accuracy of measuring the distance to a predetermined object can be automatically improved during operation.

[0133] Although the embodiments according to the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications or corrections based on the present disclosure. Therefore, it should be noted that these modifications or corrections are included in the scope of the present disclosure. For example, the functions included in each component or each step, etc. can be rearranged so as not to be logically contradictory, and a plurality of components or steps, etc. can be combined into one or divided. Although the embodiments according to the present disclosure have been described centering on the device, the embodiments according to the present disclosure can also be realized as a method including the steps executed by each component of the device. The embodiments according to the present disclosure can also be realized as a method, program, or storage medium recording the program executed by a processor included in the electronic device, or as a recording medium. It should be understood that these are also included in the scope of the present disclosure.

[0134] The above-described embodiments are not limited to being implemented as system 100. For example, the above-described embodiments may be implemented as the electronic device 1 included in system 100. Further, the above-described embodiments may be implemented as a monitoring method by a device such as the electronic device 1. Furthermore, the above-described embodiments may be implemented as a program executed by a device such as the electronic device 1 or an information processing apparatus (e.g., a computer). Also, in the technology of the present disclosure, not all of the components of the electronic device 1 shown in FIG. 1 need to exist in one housing or server, etc. For example, each part such as the control unit and / or the storage unit of the components of the electronic device 1 may be connected by a network consisting of wired, wireless, or a combination thereof. In this case, each part of the components of the electronic device 1 may be arbitrarily arranged in different housings, servers, devices, rooms, buildings, regions, or countries, etc.

Explanation of Reference Numerals

[0135] 1 Electronic device 10 Control unit 12 Storage unit 14 Communication unit 20 Imaging unit 30 Output unit 40 Input unit 100 System

Claims

1. An imaging unit, a control unit that generates information indicating the position of a moving object that represents the position of the moving object in the space where the image was captured based on an image of the moving object captured by the imaging unit, and comprising, the control unit corrects a parameter that associates coordinates on an image captured by the imaging unit with a position in the space where the image was captured based on a temporal change in the position of the moving object obtained from the information indicating the position of the moving object, an electronic device.

2. The control unit sets a priority for correcting the parameter based on a temporal change in the position of the moving object, the electronic device according to claim 1.

3. The control unit sets the priority for correcting the parameter to a predetermined level higher when a temporal change in the position of the moving object obtained from the information indicating the position of the moving object exceeds a predetermined threshold, the electronic device according to claim 2.

4. The control unit sets the priority for correcting the parameter higher as the temporal change in the position of the moving object obtained from the information indicating the position of the moving object is greater, the electronic device according to claim 2.

5. The information indicating the position of the moving object is information indicating the position of the moving object in the height direction, the control unit, sets a priority for correcting the parameter based on a temporal change in the position of the moving object in the height direction obtained from the information indicating the position of the moving object in the height direction, the electronic device according to claim 2.

6. The control unit, generates information indicating a plurality of positions of one moving object based on a plurality of images of the one moving object captured by the imaging unit, sets a priority for correcting the parameter based on a temporal change in the plurality of positions of the one moving object obtained from the information indicating the plurality of positions of the one moving object, the electronic device according to claim 2.

7. The control unit, generates information indicating the positions of a plurality of moving objects based on images of the plurality of moving objects captured by the imaging unit, sets a priority for correcting the parameter based on a temporal change in the positions of the plurality of moving objects obtained from the information indicating the positions of the plurality of moving objects, the electronic device according to claim 2.

8. The control unit sets the priority for correcting the parameter to a predetermined level higher when a predetermined threshold set based on a temporal change in the positions of the plurality of moving objects is exceeded, the electronic device according to claim 7.

9. The control unit, Based on images of the moving object at a plurality of points captured by the imaging unit, information indicating the positions of the moving objects at the plurality of points is generated. The electronic device according to claim 2, wherein a priority for correcting the parameter is set based on a temporal change in the positions of the moving objects at the plurality of points obtained from the information indicating the positions of the moving objects at the plurality of points.

10. The electronic device according to claim 2, wherein the control unit increases a point at which a priority for correcting the parameter is set based on a temporal change in the position of the moving object obtained from the information indicating the position of the moving object.

11. A control method for an electronic device that generates information indicating the position of a moving object, representing the position in the space where the image was captured, based on an image of the moving object captured by an imaging unit, comprising: a step of correcting a parameter for associating coordinates on an image captured by the imaging unit with a position in the space where the image was captured, based on a temporal change in the position of the moving object obtained from the information indicating the position of the moving object.

12. In an electronic device that generates information indicating the position of a moving object, representing the position in the space where the image was captured, based on an image of the moving object captured by an imaging unit, A program for causing a step of correcting a parameter for associating coordinates on an image captured by the imaging unit with a position in the space where the image was captured to be executed, based on a temporal change in the position of the moving object obtained from the information indicating the position of the moving object.

Images