Observation device and method for calibrating observation device

The observation device self-calibrates by capturing images of markers and detecting shifts in recognition points and edges, addressing accuracy issues caused by vibrations and ensuring precise distance measurements.

JP7579137B2Active Publication Date: 2024-11-07KYOCERA CORP
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2020215264
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-12-24
Publication Date
2024-11-07
Estimated Expiration
2040-12-24

AI Technical Summary

Technical Problem

Existing observation devices installed near roads face challenges in maintaining detection accuracy due to vibrations, necessitating calibration without the use of measurement vehicles and detecting appropriate calibration timing.

Method used

An observation device that captures images of markers, identifies recognition points and edges, and determines shifts in position or orientation based on predetermined conditions, allowing for self-calibration without external vehicles.

Benefits of technology

Enables accurate detection of calibration needs, ensuring precise distance measurements by updating coordinate-distance correspondence data, thereby maintaining device functionality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007579137000001
    Figure 0007579137000001
  • Figure 0007579137000002
    Figure 0007579137000002
  • Figure 0007579137000003
    Figure 0007579137000003
Patent Text Reader

Abstract

To provide an observation system and a calibration method of the observation system capable of detecting without using a measurement vehicle the fact that a deviation necessitating calibration has taken place.SOLUTION: An observation system (5) includes a detector (50) that images an object of imaging including a marker so as to acquire a shot image, a memory unit (52) that stores positions of plural recognition points, each of which is an intersection at which the contour of the marker included in the shot image intersects an edge of the shot image, in the shot image, and a control unit (53) that, when the number of recognition points whose positions have changed by a predetermined distance or more from the stored positions meets a predetermined requirement, determines that the position or orientation of the detector has deviated.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to an observation device and a method for calibrating an observation device. [Background technology]

[0002] In recent years, development of a safe driving support system has been progressing, in which an observation device is installed on the roadside to detect vehicles and pedestrians, and notify approaching vehicles of danger according to the distance to the vehicle and pedestrian. The observation device is generally installed in a position close to the road and may be affected by vibrations from passing vehicles. If the installation angle of the observation device fluctuates due to vibration, the distance to the vehicle detected by the observation device will no longer be detected accurately. Therefore, the observation device needs to maintain the accuracy of detection by calibration. For example, Patent Document 1 discloses a technology for calibrating a roadside unit, which is a type of observation device, using a measurement vehicle equipped with a marker. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2012-10036 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is a demand for technology that can calibrate observation devices without using measurement vehicles, etc. Furthermore, such technology would be even more useful if it could detect the appropriate timing for calibration.

[0005] An object of the present disclosure, made in consideration of the above circumstances, is to provide an observation device and a calibration method for an observation device that can detect the occurrence of a deviation that requires calibration without using a measurement vehicle or the like. [Means for solving the problem]

[0006] An observation device according to an embodiment of the present disclosure includes: A detection device that captures an image of a target including a marker; a storage unit that stores positions of a plurality of recognition points in the photographed image, the recognition points being intersections between an outline of the marker included in the photographed image and an end portion of the photographed image; and a control unit that determines that a shift has occurred in the position or orientation of the detection device when the number of recognition points among the plurality of recognition points that have changed from their stored positions by a predetermined distance or more satisfies a predetermined condition.

[0007] A method for calibrating an observation device according to an embodiment of the present disclosure includes: A method for calibrating an observation device including a detection device, comprising: Obtaining a photographed image of a photographing target including a marker; storing positions of a plurality of recognition points in the photographed image, the recognition points being points at which an outline of the marker included in the photographed image intersects with an end portion of the photographed image; determining that a shift has occurred in the position or orientation of the detection device when the number of recognition points among the plurality of recognition points that have changed from their stored positions by a predetermined distance or more satisfies a predetermined condition. Effect of the Invention

[0008] According to the present disclosure, it is possible to provide an observation device and a calibration method for an observation device that can detect the occurrence of a deviation that requires calibration without using a measurement vehicle or the like. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a communication system including an observation device according to an embodiment. [Diagram 2] FIG. 2 is an example of a functional block diagram of an observation device according to an embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of the configuration of an observation device according to an embodiment. [Figure 4] FIG. 4 is a diagram showing an example of installation of an observation device according to an embodiment. [Diagram 5] FIG. 5 is a diagram illustrating the deviation and calibration of the detection device. [Figure 6] FIG. 6 is a diagram illustrating the misalignment and calibration of the detection device. [Figure 7] FIG. 7 is a diagram illustrating recognition points and edges. [Figure 8] FIG. 8 is a diagram illustrating the amount of change in the position or orientation of the detection device. [Figure 9] FIG. 9 is a flowchart showing an example of a calibration method for an observation device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] (Communications Systems) 1 shows a configuration example of a communication system 1 including an observation device 5 according to an embodiment. The communication system 1 is, for example, a safe driving support communication system of an intelligent transport system (ITS). The safe driving support communication system is also called a safe driving support system or a safe driving support wireless system.

[0011] The observation device 5 is a device that observes vehicles, objects, and people on the road, and is, for example, a roadside device or a surveillance camera device. In this embodiment, the observation device 5 is a roadside device that is placed near an intersection where multiple roads 7 (carriageways) intersect, and observes the road surface. Here, the observation device 5 may be placed on the roadside other than at the intersection. Details of the observation device 5 will be described later.

[0012] In the communication system 1, an observation device 5 and a vehicle 6, such as an automobile traveling on a road 7, are capable of wireless communication with each other. Furthermore, a plurality of vehicles 6 may be capable of wireless communication with each other. The communication system 1 may further be capable of wireless communication including with an electronic device carried by a pedestrian 9. The electronic device is, for example, a mobile terminal device such as a smartphone.

[0013] The observation device 5 can notify the vehicle 6 of safe driving support information for supporting the driver of the vehicle 6 in safe driving. The safe driving support information may include, for example, information on the illumination of the traffic lights 4, information on road regulations, road linear information indicating the shape of the intersection where the observation device 5 is located (the shape of the road 7), and the like. The observation device 5 also detects the vehicles 6 and pedestrians 9 on the road being observed. The observation device 5 can detect, for example, the pedestrians 9 crossing the crosswalk 3. The observation device 5 can detect the vehicles 6 approaching the intersection. The observation device 5 can detect the distance to the detected objects such as the vehicles 6 and pedestrians. The observation device 5 can include information on the detected vehicles 6 and pedestrians 9 in the safe driving support information and notify the vehicle 6. The observation device 5 can also include information notified from the vehicle 6 in the safe driving support information and notify other vehicles 6. As described above, the observation device 5 may also be able to notify such information to an electronic device carried by the pedestrian 9.

[0014] The vehicle 6 may transmit vehicle information to the observation device 5 or the like periodically, for example. The vehicle information may include, for example, information on the position, speed, and blinker of the vehicle 6. The vehicle 6 may receive various information notified from the observation device 5 or the like by the electronic device mounted thereon. The electronic device mounted on the vehicle 6 may be, for example, a car navigation system. The electronic device mounted on the vehicle 6 may support the driver in safe driving by notifying the driver of a warning or the like based on the information notified from the observation device 5. The notification to the driver may be, for example, when turning at an intersection, that another vehicle 6 is approaching from the front, that a pedestrian 9 is present on the crosswalk 3 beyond the turn, and the like.

[0015] In this way, the communication system 1 supports safe driving of the driver of the vehicle 6. Here, the vehicle 6 is not limited to an automobile. For example, the vehicle 6 may include a motorcycle, a bus, a streetcar, and a bicycle.

[0016] (Functional block diagram of the observation device) 2 is a functional block diagram of the observation device 5 according to this embodiment. The observation device 5 includes a detection device 50, a communication unit 51, a storage unit 52, and a control unit 53. There may be a plurality of detection devices 50. Each of the plurality of detection devices 50 may be capable of operating independently.

[0017] The detection device 50 is a device that detects vehicles 6, pedestrians 9, and the like. The detection device 50 includes a sensor. The detection device 50 may be, for example, a visible light camera including an image sensor. The detection device 50 may also be an infrared camera including an infrared sensor. In this embodiment, the detection device 50 includes at least a visible light camera, and captures an image of a subject including a marker M, which will be described later.

[0018] Here, the sensor included in the detection device 50 is not limited to an image sensor and an infrared sensor. The sensor included in the detection device 50 may include, for example, a 3D laser scanner. The detection device 50 outputs the detection signal detected by the sensor, the above-mentioned captured image, and the like to the control unit 53.

[0019] The communication unit 51 is controlled by the control unit 53 to perform wireless communication with the vehicle 6. The communication unit 51 may be configured with a communication circuit and an antenna. The antenna may be, for example, an omnidirectional antenna. The communication unit 51 may perform wireless communication using, for example, the 700 MHz band allocated to ITS. The communication unit 51 may also perform wireless communication using, for example, a wireless LAN (Local Area Network).

[0020] The communication unit 51 performs various processes such as amplification on the signal received by the antenna, and outputs the processed received signal to the control unit 53. The control unit 53 performs various processes on the inputted received signal to obtain information contained in the received signal. The control unit 53 also outputs a transmission signal including information to the communication unit 51. The communication unit 51 performs various processes such as amplification on the inputted transmission signal, and wirelessly transmits the processed transmission signal from the antenna.

[0021] The storage unit 52 may function as a memory that stores various types of information. The storage unit 52 may store, for example, a program executed in the control unit 53 and the results of processing executed in the control unit 53. The storage unit 52 may also function as a work memory for the control unit 53. The storage unit 52 may be configured, for example, by semiconductor memories such as ROM and RAM, but is not limited thereto and may be any storage device. The storage unit 52 may include, for example, a small hard disk drive and an SSD (Solid State Drive).

[0022] In this embodiment, the storage unit 52 stores data related to the calibration executed by the control unit 53. As data related to the calibration, the storage unit 52 may store recognition point data 521 including the positions of the recognition points described below. The storage unit 52 may also store ridge line data 522 described below. The storage unit 52 may also store coordinate-distance correspondence data indicating the correspondence between the position of the subject to be photographed in the photographed image and the distance to the subject to be photographed. The storage unit 52 may also store a program for causing the control unit 53 to execute calculation of the amount of change in the detection device 50 and calibration described below.

[0023] The control unit 53 generally manages the operation of the observation device 5 by controlling other components of the observation device 5. The control unit 53 may include one or more processors. The processor may load a program from an accessible memory to realize various functions of the observation device 5. The processor may include at least one of a general-purpose processor that loads a specific program and executes a specific function, and a dedicated processor specialized for a specific process. The dedicated processor may include an application specific integrated circuit (ASIC). The processor may include a programmable logic device (PLD). The PLD may include a field-programmable gate array (FPGA). The control unit 53 may include at least one of a system-on-a-chip (SoC) and a system in a package (SiP) in which one or more processors work together.

[0024] The control unit 53 can generate safe driving support information based on the acquired detection signal from the detection device 50. For example, the control unit 53 may measure the distance to the pedestrian 9 included in the captured image acquired from the detection device 50, and warn the vehicle 6 when it is determined that the pedestrian 9 is approaching. In addition, in this embodiment, when the number of recognition points described later satisfies a predetermined condition, the control unit 53 calculates the amount of change in the position or orientation of the detection device 50 and performs calibration based on the calculated amount of change. Details of these processes will be described later.

[0025] (Configuration of observation device) Fig. 3 is a diagram showing an example of the configuration of the observation device 5 according to this embodiment. As shown in Fig. 3, the observation device 5 may be provided as an on-road observation unit including multiple detection devices 50, which are connected to a connection section 57 to determine the relative positional relationship between them.

[0026] The connection part 57 is a member for connecting the detection device 50. In this embodiment, the connection part 57 is a frame-shaped member made of metal. However, the material and shape of the connection part 57 are not limited. Here, as shown in FIG. 3, an orthogonal coordinate system can be defined for the observation device 5, which is a road observation unit. For example, the u-axis direction, the v-axis direction, and the w-axis direction each correspond to the direction in which the sides that intersect at right angles at the vertex of the connection part 57 extend. As shown in FIG. 3, in the observation device 5 according to this embodiment, multiple detection devices 50 are arranged side by side in the w-axis direction.

[0027] FIG 4 shows an example of the installation of observation device 5. Observation device 5 is installed on a pillar 59 near an intersection where road 7 to be observed intersects. A connection part 57 is fixed to pillar 59, and the orientation of detection device 50 may be adjusted with respect to connection part 57. Here, when observation device 5 is installed to observe the road, the w-axis direction is the vertical direction. Here, in FIG 4, the z-axis direction is the vertical direction, and the x-axis and y-axis indicate horizontal directions perpendicular to the z-axis.

[0028] Here, the detection device 50 may be attached to the connection part 57 by, for example, a screw 56 (see FIG. 3), fitting, or the like, and the orientation may be adjusted. The detection device 50 may have an adjustment mechanism for changing the installation angle with respect to the connection part, and may be capable of rotating in, for example, the pan, tilt, and yaw directions. The adjustment mechanism may be a structure such as a camera platform, but is not limited thereto. The detection device 50 may have a structure capable of adjusting the movable part in three axial directions with respect to the base part, for example, by having axes in the u-axis, v-axis, and w-axis directions shown in FIG. 3. When the connection part 57 is fixed to the pillar 59, the detection device 50 is adjusted so as to observe the road at an appropriate depression angle, and is fixed by a screw or the like so as not to change the orientation.

[0029] (calibration) As described above, the observation device 5 is installed, for example, near an intersection where the road 7 to be observed intersects, and so may be affected by vibrations from passing vehicles, etc. For example, continued exposure to vibrations from passing vehicles, etc. may cause the screws 56 for fixing the detection device 50 to the connection part 57, the fitting parts, etc. to loosen or rotate, causing the detection direction of the detection device 50 to shift. Alternatively, the detection direction of the detection device 50 may shift due to an impact being applied to the pillar 59 on which the observation device 5 is attached. In such cases, calibration is required so that the observation device 5 can continue to perform accurate distance measurements.

[0030] 5 and 6 are diagrams for explaining the deviation and calibration of the detection device 50. FIG. 5 illustrates an example of the detection device 50 at the initial position photographing a marker M 50 m ahead, and the photographed image. The number of markers M is not limited to one, and may be multiple. Here, the marker M is an object located at a specific distance from the observation device 5 among the photographed objects of the detection device 50. Therefore, by using the marker M, it is possible to associate the coordinates in the photographed image with the distance from the observation device 5 to the photographed object. As the marker M, for example, an object that is difficult to move, such as a building, an installation indicating a bus stop (hereinafter, a bus stop), a signboard, or a white line drawn on the road 7, is selected. In the photographed image shown in FIG. 5, the marker M is located at the coordinate (50, 40), and this coordinate is associated with a position 50 m ahead from the observation device 5. Here, the coordinates of the photographed image are set with the horizontal direction as the X axis and the vertical direction as the Y axis, and the upper left vertex is the coordinate (0, 0). Information on the coordinates in the captured image and the distance from the observation device 5 may be stored as coordinate-distance correspondence data in the storage unit 52. The coordinate-distance correspondence data may be data in which the distance from the observation device 5 is associated with the coordinates of all pixels in the captured image, or may be data in which the coordinates of some pixels are associated with each other.

[0031] FIG. 6 illustrates an example of the detection device 50 photographing a marker M 50 m ahead after being displaced in the tilt direction from the state of FIG. 5, and the photographed image. As shown in FIG. 6, the marker M located at the coordinate (50, 40) in FIG. 5 has moved to the coordinate (45, 45). When the detection device 50 changes to the state of FIG. 6, the observation device 5 cannot perform accurate distance measurement with the initial correspondence in which the coordinate (50, 40) is the position 50 m ahead. Therefore, calibration is required to newly associate the coordinate (45, 45) with the position 50 m ahead. Calibration is a correction of the correspondence relationship between the position (coordinate) in the photographed image and the distance between the photographed object and the observation device at that position. In this embodiment, calibration is performed by updating the coordinate-distance correspondence data stored in the storage unit 52. The coordinate-distance correspondence data is data that defines the correspondence between the position (coordinate) in the photographed image and the distance to the photographed object at that coordinate. When performing distance measurement, the control unit 53 of the observation device 5 calculates the distance to the subject to be photographed based on the coordinate-distance correspondence data read out from the storage unit 52.

[0032] Here, depending on the installation location of the observation device 5, a movable object such as a signboard attached on the road may be selected as the marker M. Or, an object may be placed on the white line selected as the marker M, making it impossible to correctly recognize it, or it may disappear or change in shape due to aging. In such a case, if the detection device 50 erroneously determines that the marker M has moved or that it has been recognized as having a different shape and performs calibration, the observation device 5 cannot perform accurate distance measurement. Therefore, it is necessary to appropriately determine that the detection device 50 has shifted. In addition, the detection device 50 may have a shift other than the tilt direction (for example, a shift in the pan direction, a shift in one direction, etc.). Therefore, it is necessary to appropriately determine the direction and amount of the shift. The observation device 5 according to this embodiment detects that a shift requiring calibration has occurred and calculates the amount of change in the position or orientation of the detection device 50 by the method described below.

[0033] (Recognition points) The control unit 53 of the observation device 5 sets the marker M according to the installation location of the observation device 5. Then, the control unit 53 sets the intersection point where the outline of the marker M included in the photographed image acquired from the detection device 50 intersects with the end of the photographed image as the recognition point. The control unit 53 determines one or more markers M so that there are multiple recognition points. The control unit 53 also sets a ridge line. The ridge line is a part of the outline of the marker M in the photographed image and is a line extending from the recognition point. The ridge line is selected to be a straight line or a line that can be approximated to a straight line. Here, the marker M forming the recognition point and the ridge line may be an object used to generate the coordinate-distance correspondence data, or may be another object. Various objects included in the photographed image acquired from the detection device 50 can be the marker M forming the recognition point and the ridge line.

[0034] FIG. 7 is a diagram for explaining the recognition points and ridge lines. FIG. 7 includes an example of a captured image acquired from the detection device 50, an example of data 521 of the recognition points, and an example of data 522 of the ridge lines. In the captured image shown in FIG. 7, the marker M is a building in the upper left, a white line of the road 7 in the center, and a bus stop in the lower right. Here, the ends of the captured image constitute four sides (upper side, right side, lower side, and left side). The recognition point P1 is an intersection point where the outline of the bus stop intersects with the lower side. The recognition point P2 is an intersection point where the outline of the building intersects with the left side. Moreover, the recognition point P3 is an intersection point where the outline of the white line of the road 7 intersects with the upper side. In the captured image shown in FIG. 7, the other circled parts can be recognition points. The control unit 53 may set all such intersection points as recognition points, or may select some of them as recognition points. The control unit 53 sets a plurality of recognition points. As an example, the control unit 53 may set 10 to 20 recognition points. Here, the recognition points must be present on three or more of the four sides that constitute the end of the captured image so that the amount of change in the position or orientation of the detection device 50 can be calculated. The recognition points may be distributed so that there are an equal number of recognition points on each side of the captured image. The positions of the recognition points on each side may be set to be separated by a predetermined distance or more in the captured image. This is because adjacent recognition points are often formed by the same object, or by different objects that are adjacent to each other. It is desirable that the objects in the captured image that form the recognition points and ridgelines are different objects for each recognition point and ridgeline. This is because if many recognition points are set by one object, the positional deviation of the object may be erroneously determined as a deviation of the detection device. Here, the recognition points may be set automatically by the control unit by image recognition of the object in the captured image, or may be set by the operator selecting one from the captured image.

[0035] In the captured image shown in FIG. 7, ridge line L1 is part of the contour of the bus stop and extends from recognition point P1. Moreover, ridge line L2 is part of the contour of the white line of road 7 and extends from recognition point P3. Other parts of the contour of marker M may be set as ridge lines. The control unit 53 may set all such parts as ridge lines, or may select parts as ridge lines. The control unit 53 may set one or more ridge lines. As an example, the control unit 53 may set 10 to 20 ridge lines.

[0036] The control unit 53 stores information about the set recognition points in the storage unit 52 as recognition point data 521. The recognition point data 521 includes the positions (coordinates) of multiple recognition points in the captured image. In the example of FIG. 7, the recognition point data 521 associates the recognition point number, the position on the captured image, and the angle change of the detection device 50 with respect to a one-pixel shift of the recognition point. Here, the X-direction value of the angle change indicates the amount of change in the pan direction. Also, the Y-direction value of the angle change indicates the amount of change in the tilt direction.

[0037] Furthermore, control unit 53 stores information about the set ridge line in storage unit 52 as ridge line data 522. Ridge line data 522 includes the positions (coordinates) of the start point and end point of the ridge line in the captured image. In the example of Fig. 7, ridge line data 522 associates the number in the ridge line data, the positions of the start point and end point, and the angle that the ridge line makes with the end of the captured image (i.e., the outer periphery of the image) in the captured image.

[0038] When the control unit 53 acquires a new captured image from the detection device 50, the control unit 53 extracts the recognition points and edge lines from the new captured image and determines their positions (coordinates). The control unit 53 determines whether the detection device 50 has shifted by comparing the recognition point data 521 and edge line data 522 stored in the memory unit 52 with information such as positions. If it is determined that the detection device 50 has shifted, the control unit 53 calculates the amount of change in the detection device 50. Furthermore, the control unit 53 performs calibration of the correspondence between the position of the subject in the captured image and the distance to the subject based on the calculated amount of change.

[0039] Fig. 8 is a diagram illustrating the amount of change in the position or orientation of the detection device 50. In the example of Fig. 8, a captured image It0 is a previously captured image including recognition points and ridge lines corresponding to recognition point data 521 and ridge line data 522 stored in the storage unit 52. A captured image It1 is a new captured image.

[0040] The control unit 53 sets a plurality of recognition points (broken circles in FIG. 8) and ridgelines in the captured image It0. The control unit 53 specifies the positions of the plurality of recognition points and stores data 521 of the recognition points in the memory unit 52. The control unit 53 also specifies the inclination (angle) of the ridgelines and stores data 522 of the ridgelines in the memory unit 52. When a new captured image It1 is acquired, the control unit 53 extracts recognition points (solid circles at the ends of the captured image It1 in FIG. 8) and ridgelines in the captured image It1. The objects constituting the recognition points in the captured image It1 are the same as the objects constituting the recognition points in the captured image It0. The positions of the recognition points and the inclination of the ridgelines in the captured images It0 and It1 are compared to determine whether the detection device 50 has shifted.

[0041] The control unit 53 determines that the detection device 50 has shifted when the number of recognition points that have changed by a predetermined distance or more from the stored position among the multiple recognition points set for the captured image It0 satisfies a predetermined condition. Here, the predetermined distance is set so that the detection device 50 is not erroneously determined to have shifted due to small everyday shaking of the detection device 50 caused by vibrations transmitted by a large vehicle traveling nearby. The predetermined distance may be set to about 3 pixels, for example. In the example of FIG. 8, an arrow is attached to the recognition point that has changed by a predetermined distance or more. For example, the recognition point P1 and the recognition point P2 are recognition points that have changed by a predetermined distance or more. Also, for example, the recognition point P3 is a recognition point that has not changed by a predetermined distance or more. Here, in this embodiment, the predetermined condition is that the number of recognition points that have changed exceeds half of the multiple recognition points. This is because if the number of recognition points that have changed is the majority of the total, it is considered that the detection device 50 has shifted, not that some of the markers M have moved. The observation device 5 according to this embodiment can determine with sufficient accuracy that the detection device 50 has shifted and requires calibration. Here, the threshold value of the ratio of the number of changed recognition points to the total number used as the predetermined condition is not limited to 50%. For example, this threshold value may be set to a value less than 50% of the total (e.g., 30%) so that calibration is performed more frequently. From a similar viewpoint, the predetermined condition may be that the number of changed recognition points exceeds a predetermined value (e.g., 5). Here, the amount of change in the recognition points due to the shift of the detection device 50 is usually larger for recognition points with markers M that are farther away. Therefore, instead of setting the predetermined distance to the same value for all recognition points, the predetermined distance may be set to be larger for recognition points with markers M that are farther away.

[0042] When the control unit 53 determines that the detection device 50 has shifted, it calculates the amount of change in the position or orientation of the detection device 50 based on a comparison result of the position of the recognition point and the inclination of the ridgeline between the captured images It0 and It1. If the recognition point has changed in the X direction in the captured image, it can be determined that the detection device has rotated in the pan direction. If the recognition point has changed in the Y direction in the captured image, it can be determined that the detection device has rotated in the tilt direction. If the angle of the ridgeline has changed in the captured image, it can be determined that the detection device has rotated in the roll direction. The control unit 53 may calculate the amount of change in the detection device 50 by a known method used in coordinate rotation processing.

[0043] The control unit 53 updates the correspondence between the position of the subject in the captured image and the distance to the subject based on the calculated amount of change. Specifically, the control unit 53 updates the coordinate-distance correspondence data stored in the storage unit 52 as calibration. This allows the observation device 5 to continue to perform accurate distance measurement. Even if it is determined that a deviation requiring calibration has occurred in the detection device 50, calibration may not be performed in a predetermined case, and a notification may be sent to the administrator who manages the observation device 5. The predetermined case is, for example, when the calculated amount of change is very large, when there is a contradiction in the direction of change in the position of the recognition point, or when there is a contradiction in the direction of change in the ridge line. An example of an inconsistency in the changes in the recognition points and ridge lines is when there are a certain number or more of recognition points and ridge lines that are changing in opposite directions. Or, an example of an inconsistency in the changes in the recognition points and ridge lines is when many recognition points have disappeared. In such a case, since there is a possibility that the observation device 5 has broken down or a serious fault has occurred, the administrator needs to check the condition.

[0044] Furthermore, the control unit 53 updates the recognition point data 521 and the edge line data 522 stored in the storage unit 52 using the positions of the recognition points and the inclinations of the edge lines extracted from the captured image It1.

[0045] (Method of calibrating the observation device) 9 is a flowchart showing an example of a calibration method for the observation device 5 according to one embodiment. The observation device 5 executes a series of processes for calibration in accordance with the flowchart shown in FIG.

[0046] The observation device 5 acquires a captured image (step S1). The captured image in step S1 corresponds to, for example, the above-mentioned captured image It0. The observation device 5 sets a plurality of recognition points and edge lines in the captured image It0.

[0047] The observation device 5 stores the position of the recognition point (step S2). The observation device 5 stores the set position of the recognition point in the storage unit 52, for example, by including it in the above-mentioned recognition point data 521. The processes of steps S1 and S2 are also performed, for example, when the observation device 5 is first installed.

[0048] The observation device 5 acquires a captured image (step S3). The captured image in step S3 corresponds to, for example, the above-mentioned captured image It1. The observation device 5 identifies the positions of a plurality of recognition points and the inclinations of ridgelines in the captured image It1.

[0049] The observation device 5 determines whether the number of recognition points that have changed by a predetermined distance or more from the positions stored in step S2, among the multiple recognition points identified in the captured image It0, satisfies the above-mentioned predetermined condition (step S4).

[0050] When the number of changed recognition points does not satisfy the predetermined condition (No in step S4), the observation device 5 determines that no deviation of the detection device 50 has occurred, and returns to the processing in step S3.

[0051] When the number of recognition points where a change has occurred satisfies a predetermined condition (Yes in step S4), the observation device 5 determines that a deviation of the detection device 50 has occurred, and proceeds to the processing in step S5.

[0052] The observation device 5 determines that a shift has occurred in the position or orientation of the detection device 50 (step S5). In addition, the observation device 5 calculates the amount of change in the position or orientation of the detection device 50 based on a comparison result of the positions of the recognition points and the inclinations of the ridgelines between the captured images It0 and It1.

[0053] The observation device 5 performs calibration based on the calculated amount of change (step S6). Then, the observation device 5 returns to the process of step S2. In the process of step S2 after performing the calibration, the observation device 5 updates the data 521 of the recognition point using the position of the recognition point extracted in the captured image It1, and stores it in the storage unit 52.

[0054] Here, the observation device 5 may execute the processes from step S3 onward in the flowchart of FIG. 9 periodically (for example, once a day).

[0055] As described above, the observation device 5 and the calibration method for the observation device 5 according to this embodiment can detect the occurrence of a deviation that requires calibration by using the above-mentioned configuration and steps, without using a measurement vehicle or the like.

[0056] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these modifications and corrections are included in the scope of the present disclosure.

[0057] For example, in the above embodiment, the control unit 53 included in the observation device 5 calculates the amount of change in the position or orientation of the detection device 50, and performs calibration based on the calculated amount of change. Here, as described above, the observation device 5 includes a communication unit 51. The observation device 5 may communicate with a server in the communication system 1 via the communication unit 51. The observation device 5 may transmit necessary data to the server, and the server may perform at least one of calculation of the amount of change in the detection device 50 and calibration. Here, the server in the communication system 1 is a computer that controls the entire communication system 1.

[0058] For example, in the above embodiment, the observation device 5 sets a plurality of recognition points and edge lines. Here, the observation device 5 may define only a plurality of recognition points without setting edge lines.

[0059] For example, in the above embodiment, the observation device 5 treats each of the multiple recognition points equally, but weighting may be performed according to the position and type of the recognition point. [Explanation of symbols]

[0060] 1. Communication Systems 3. Crosswalks 4. Traffic lights 5 Observation equipment 6 Vehicles 7 road 9. Pedestrians 50 Detection Device 51 Communications Department 52 Storage section 53 Control section 56 Screw 57 Connection 59 Pillars 521 Recognition Point Data 522 Ridge Data

Claims

1. A detection device that captures an image of the marker; a storage unit that stores positions of a plurality of recognition points in the photographed image, the recognition points being defined by contours of the markers included in the photographed image; a control unit that determines that a position or orientation of the detection device has shifted when a recognition point that has changed by a predetermined distance or more from a stored position among the plurality of recognition points satisfies a predetermined condition, The predetermined distance is a larger value as the distance of the marker forming the recognition point from the detection device increases. Observation equipment.

2. An observation device as described in Claim 1, wherein the recognition point is an intersection where the contour of the marker and the end of the captured image intersect.

3. The control unit determines that a deviation has occurred in the position or orientation of the detection device when the number of the recognition points that have changed by a predetermined distance or more from a stored position among the plurality of recognition points satisfies a predetermined condition.

3. The observation device according to claim 1 or 2.

4. 4. The observation device according to claim 1, wherein when a deviation in a position or orientation of the detection device is detected, the control unit performs a calibration regarding a correspondence between a position in the captured image and a distance to a subject present at that position.

5. The observation device according to claim 1 , wherein the predetermined condition is that the number of the recognition points at which the change has occurred exceeds half of the plurality of recognition points.

6. The observation device according to claim 1 , wherein the control unit issues an alert when a predetermined number or more of the recognition points have changed in a direction opposite to a direction in which the positions of some of the recognition points have changed.

7. A method for calibrating an observation device including a detection device, comprising: a control unit that stores in a storage unit positions of a plurality of recognition points in the acquired photographed image, the recognition points being parts of the contours of the markers included in the photographed image; the control unit determines that a position or orientation of the detection device has shifted when a recognition point that has changed by a predetermined distance or more from a stored position among the plurality of recognition points satisfies a predetermined condition, A method for calibrating an observation device, wherein the specified distance is a larger value as the marker forming the recognition point is farther away from the detection device.

8. The control unit performs calibration of a correspondence between a position of the marker in the captured image and a distance to the marker based on the calculated amount of change in position or orientation of the detection device. The method for calibrating an observation device according to claim 7.

Citation Information

Patent Citations

  • Video camera monitoring system

    JP1993037937A

  • Method and device for adjusting viewing angle deviation of monitoring camera and medium for controlling the device

    JP2002232875A

  • Apparatus and method for detecting image shift

    JP2007208513A

  • Camera calibration system, and measuring vehicle and roadside device for the same

    JP2012010036A

  • Detection of camera misalignment

    JP2014003599A