Method for a configuration of a camera, camera arrangement, computer program and storage medium

EP4466849A1Pending Publication Date: 2024-11-27ROBERT BOSCH GMBH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
EP2022700335
Authority / Receiving Office
EP · EP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2022-01-18
Publication Date
2024-11-27

AI Technical Summary

Technical Problem

Existing video surveillance systems require manual measurement of camera parameters such as position, orientation, and opening angle, which is complex and time-consuming, especially when multiple cameras are involved in monitoring large areas.

Method used

A method using a mobile device, such as a smartphone, to determine the camera's geolocation by manually placing it in the surrounding area and providing its geolocation to the camera, eliminating the need for user estimation or expensive sensors, and allowing for automatic configuration and calibration of camera positions and orientations.

Benefits of technology

This method simplifies the camera configuration process, reduces manual effort, and enables accurate determination of camera positions and orientations for effective image analysis and object tracking in surveillance systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 1.1
    Figure 1.1
  • Figure 1.2
    Figure 1.2
Patent Text Reader

Abstract

Method for a configuration of a camera (2), wherein the camera (2) is arranged and / or mounted stationary, wherein the camera (2) has a recording area and is configured to take image data for the recording area, wherein the camera (2) comprises and / or stores a camera geolocation, wherein a mobile device (3) is placed in a surrounding area (4) of the camera (2), wherein the mobile device (3) determines a mobile device geolocation and provides the mobile device geolocation to the camera (2), characterized in that the camera (2) stores the mobile device geolocation as the camera geolocation and / or the mobile device geolocation is assigned to camera (2) as the camera geolocation.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Description

[0002] Title

[0003] Method for a configuration of a camera, camera arrangement, computer program and storage medium

[0004] Background of the invention

[0005] The invention relates to a method for a configuration of a camera, which is arranged and / or mounted stationary, wherein the camera takes images of a recording area.

[0006] Cameras are often used in video surveillance systems for surveillance of streets, intersections, and squares but also of public buildings, museums, and prisons. Such video surveillance systems often comprise many cameras, which are located at various positions, along with a central surveillance office where the image streams taken by the surveillance cameras are united. The image data streams collected in this way are monitored by surveillance workers on-line or are evaluated in automated fashion. In the latter case, image processing algorithms are typically used for evaluating the video sequences recorded. Often, moving objects are then separated from the essentially static background and are tracked over time, and alarms are tripped if relevant movements occur. These image processing algorithms employ so-called object segmentation, in which the differences between the current camera image and a so-called scene reference image, which models the static scene background, are evaluated.

[0007] As a result, in such image processing algorithms, information is obtained about the position, size of the segmented image regions / objects and about the speed of the segmented image regions / objects in pixels, or pixels traveled per second. Video monitoring systems that conversely express the results of the image regions / objects in the scene in metric sizes, such as lengths in meters and speed in meters per second, or the absolute and / or relative positioning in a camera network, require that such camera parameters as the position, orientation and opening angle of the camera or cameras be known. These parameters are usually measured manually, which is complicated, in the installation process.

[0008] A surveillance system is descripted in DE 10 2010 038341 Al. The video surveillance system comprises an input device comprising an analysis device for evaluating images of a surveillance scene from a surveillance region, said images having been recorded by a surveillance camera, wherein a user is represented and / or can be represented on the images, wherein the analysis device and / or the input device are / is designed for generating user object data, which are generated on the basis of the images with the user and / or the input device.

[0009] Disclosure of the invention

[0010] The invention presents a method for a configuration of a camera according to claim 1. The invention also discloses a camera arrangement, a computer program and a storage medium. Preferred and / or advantageous embodiments are descripted in the sub claims, the description and the attached figures.

[0011] The invention concerns a method to configure a camera, especially a surveillance camera and / or a camera of a surveillance system. Especially, configuration of the camera includes and / or is adapted as a calibration of the camera, e.g. a position and / or orientation calibration. The camera is arranged, positioned and / or mounted stationary. The camera has a recording area, also called as and / or defined by a field of view. The camera is suitable and / or designed for monitoring a surveillance region such as, for example, individual rooms of a building, places, roads, etc. The video surveillance system comprises preferably a plurality of cameras, for example more than 10, 20 or 50 cameras.

[0012] The camera is for example arranged, positioned and / or mounted at a surveillance position. The surveillance position is in particular stationary relative to the surveillance area. The camera is preferably embodied as a CCD or CMOS camera, but in principle can be based on any arbitrary camera technology. In particular, both the use of fixedly installed PZT cameras (Pan / Tilt / Zoom) as well as the use of cameras with a fixed viewing field are within the scope of the invention. The optical equipment of the surveillance camera is also in general freely selectable; that is, the surveillance camera may be implemented with a wide angle lens, for instance, or with a 360[deg.] fisheye lens attachment.

[0013] The stationary surveillance position preferably includes the absolute and / or relative position and / or orientation and / or opening angle of the camera. In particular, the method is embodied for locating the camera and / or determining the surveillance position and / or orientation of the camera.

[0014] The camera has a recording area, e.g. field of view. Especially, the recording area defines and / or is given by a view direction, also called recording direction. The camera is configured to capture and / or take image data for its recording area. The image data comprise preferably pictures, images, a video, sensor data and / or audio data for the recording area.

[0015] The camera comprises, incudes and / or stores a camera geolocation. The camera geolocation is adapted as and / or comprises for example the surveillance position, camera position and / or orientation of the camera. Preferably, the camera stores the camera geolocation as computer and / or machine readable data. The camera geolocation is used and / or needed for example for image analysis, object detection and / or object tracking based on the image data.

[0016] A mobile device is placed, arranged and / or positioned in a surrounding area of the camera. The mobile device is for example adapted as a mobile phone, smartphone, mobile computer or tablet. The surrounding area is preferably adapted as surrounding volume, e.g. a sphere around a centre of the camera. Especially, the surrounding area is an area or volume around the camera, wherein the surrounding area is for example defined that all points inside the surrounding area have distanced to the camera or centre of the camera less then a surrounding area threshold. The surrounding area threshold is for example larger then 5 cm and / or smaller than 100 cm. The mobile device is preferably manually and / or handhold placed, arranged and / or positioned in the surrounding area. Especially, the mobile device is placed, arranged and / or positioned outside the surrounding area after applying the method for the configuration of the camera. The mobile device determines a mobile device geolocation. The mobile device especially determines the mobile device geolocation during its placement in the surrounding area. The mobile device geolocation comprises for example a position of the mobile device and / or an orientation of the mobile device, especially as three-dimensional position and / or in six degrees of freedom. The mobile device provides the mobile device geolocation to the camera. For example the mobile device sends the mobile device geolocation to the camera, preferably using a wireless connection between camera and mobile device. Especially, the mobile device provides the mobile device geolocation to the camera during its placement in the surrounding area of the camera.

[0017] The camera stores and / or uses the provided mobile device geolocation as its camera geolocation. Additionally and / or alternatively, the mobile device geolocation is assigned to the camera as its camera geolocation.

[0018] The invention is based on the idea to use a mobile device, e.g. a smart phone, to configure and / or calibrate the camera, especially to determine the cameras geolocation. To use the camera in a surveillance system and / or for image analyses the knowledge of the position and / or orientation of the camera needed. Instead of using estimating the geolocation by a user or the integration of expensive sensors for location determination in the camera, the method uses a separate device to determine a geolocation of the device in the surrounding area and uses this geolocation as the geolocation of the camera.

[0019] Optionally, the mobile device determines the device geolocation using GPS, satellite locating radio locating or Wi-Fi locating. For example, the mobile device comprises a sensor for determining the mobile device geolocation. The sensor is for example adapted as a GPS, radio or Wi-Fi sensor. Preferably, the sensor is adapted and / or configured to determine a orientation of the mobile device, e.g. a sensor adapted as a gyroscope.

[0020] The mobile device geolocation is preferably determined as a position and / or orientation in or according to a reference coordinate system. For example, the mobile device geolocation is determined as coordinates and / or angles using the reference coordinate system. The reference coordinate system is preferably a real-world coordinate system, a coordinate system defined by the surveillance area or a geographic coordinate system.

[0021] Especially, the mobile device geolocation and / or the camera geolocation is determined and / or comprising a latitude and / or longitude, where in the latitude and longitude is preferably based on the geographic coordinate system of the earth. Furthermore, the device and / or camera geolocation may comprise an altitude and / or height.

[0022] In a preferred embodiment the camera geolocation comprises a system time. The system time is for example the time at the position of the camera. The mobile device may determine a location time. The mobile device provides the location time to the camera, especially providing it as a part of the mobile device geolocation. The camera stores and / or uses the provided location time as its system time. This embodiment is based on the idea to provide a method for time synchronisation during configuring and / or calibrating the camera.

[0023] Optionally, a distance between the camera and the mobile device is determined. The distance between the camera and the mobile device is preferably determined by the mobile device, alternatively by the camera. The distance between camera and mobile device is for example the distance between the mobile device and a reference point of the camera, e.g. the centre of the camera. The distance may be determined as a length or as an abstract distance, e.g. a signal strength of an exchange signal between camera and mobile device. Based on the determined distance it is signalised, indicated and / or reported if the mobile device is placed, arranged and / or positioned inside and / or outside the surrounding area of the camera. For example the camera or the mobile device signalises the placement of the mobile device inside the surrounding area using a light and / or first colour and the placement of the mobile device outside the surrounding area using no light or second colour. This is based on the idea to show a user if the placement of the mobile device may be used to configure and / or calibrate the cameras geolocation.

[0024] Additionally and / or alternatively, it is determined based on the distance between the camera and the mobile device, if the mobile device is placed, arranged and / or positioned inside or outside the surrounding area of the camera. Especially, the mobile device is blocked to determine its mobile device geolocation and / or blocked to provide its mobile device geolocation whenever it is outside the surrounding area. Alternatively, the mobile device is triggered to determine and / or provide the mobile device geolocation when it is inside the surrounding area. Furthermore, the camera may be blocked to store, use or assign the mobile device geolocation as its camera geolocation when the mobile device is outside the surrounding area. Alternatively, the camera checks if the mobile device is inside the surrounding area before storing, using or assigning the mobile device geolocation as its camera geolocation.

[0025] Optionally, an orientation, especially an azimuth as orientation and / or as part of the orientation is determined based on a connection line between the geolocation and / or position of the mobile device and the geolocation and / or position of the camera. The connection line is preferably the closest connection between the camera and the mobile device, e.g. the position of the camera and the position of the mobile device, wherein the position is for example the latitude and the longitude. The position is for example the GPS-Position and / or comprised by the geolocation. The orientation is especially determined based on the connection line and a recording direction of the camera. Preferably, for determining the orientation and / or the connection the mobile device, e.g. person with the mobile device, is positioned in the recording area of the camera, wherein optionally an image and / or video is taken, the mobile devices determines its position, e.g. GPS-coordinates, wherein the connection is for example the closest connection between the position of the camera and the position of the mobile device, wherein the connection is for example used to determine and / or used as the orientation of the camera and / or the azimuth of the camera.

[0026] Preferably, the distance between the camera and the mobile device is determined based on a signal, especially wireless, optical or radio signal. The signal is for example exchanged between the camera and the mobile device. The signal is preferably used to exchange data between the camera and the mobile device, especially to provide the mobile device geolocation.

[0027] The mobile device has, comprises and / or defines a device recording direction.

[0028] For example the mobile device comprises at least one camera, for example our front and / or back camera. The recording direction is for example in the direction of the camera view of the mobile device. Additionally and / or alternatively, the recording direction of the mobile device is static and / or indicated using a sign, marking and / or optical indicator on the mobile device, e.g. on the housing of the mobile device. The camera comprises, defines and / or has a camera recording direction, for example the cameras optical axis and / or an axis pointing in the direction of the recording area. For determination of the orientation of the camera, an orientation of the mobile device and / or the mobile device geolocation or for providing the mobile device geolocation to the camera, the mobile device is and / or has to be arranged and / or aligned relative to the camera recording direction with parallel device recording direction. With other words, the mobile device geolocation and / or the orientation of the mobile device is determined when the device recording direction is parallel to the camera recording direction. The orientation is especially an orientation relative to the earth reference system. The determined orientation is provided to the camera, wherein the camera stores, uses and / or assigns this orientation as the cameras orientation.

[0029] According to a preferred embodiment of the invention, a user and / or an object is placed, positioned and / or arranged in the recording area. The user and / or the object is in other words used as a reference object. Especially, the user and / or the object is placed and / or positioned as a static object in the recording area the camera takes an image, picture and / or image data. The object and / or the user is shown in the image, picture and / or image data taken by the camera in a first view. The mobile device is placed, positioned and / or arranged in the surrounding area of the camera, especially with the device recording direction parallel to the camera recording direction, wherein the mobile device takes further image, further picture and / or further image data of the object and / or user. Especially, a relative orientation between the mobile device and the camera is determined, for example based on the signal, a signal strength and / or the images. For example, the relative orientation is determined by an comparison of the picture, image and / or image data with the further picture, further image and / or further image data, e.g. comparing the first view with the view to the object and / or user according to the further image or picture. The orientation of the camera is preferably determined and based on the orientation and / or geolocation of the mobile device, especially determined by the mobile device, and the image, picture and / or image data of the camera and the further, the further image and / or further image data. Especially, the determined orientation of the camera is used in the geolocation of the camera and / or for the calibration and / or a configuration of the camera.

[0030] A further object of the invention is a camera arrangement. The camera arrangement comprises at least one of the camera and one of the mobile devices. The camera is arranged and / or amounted stationary and comprises and / or stores a camera geolocation. The mobile device is adapted and / or configured to determine its mobile device geolocation, especially to determine it when the mobile device is arranged and / or positioned inside a surrounding area of the camera. For example, the mobile device is adapted and / or configured to detect the arrangement and / or placement inside the surrounding area and to trigger the determination of the mobile device geolocation when detected to be inside the surrounding area. The mobile device is adapted and / or configured to provide the determined mobile device geolocation to the camera. The camera is adapted and / or configured to store, use and / or assign the provided mobile device geolocation as the camera geolocation. Especially the mobile device and the camera are adapted and / or configured to implement and / or use the method for configuration of the camera.

[0031] A further object of the invention is a computer program, wherein the computer program is adapted and / or configured for execution on a computer, the camera arrangement, the camera and / or the mobile device. The computer program is adapted and / or configured to implement, use and / or execute the method for configuration of the camera.

[0032] A further object of the invention is a storage medium, especially a machine- readable storage medium. The computer program is stored on the storage medium.

[0033] Further embodiments and advantages are disclosed in the figure and its description. Wherein

[0034] Figure 1 shows an example of a camera arrangement.

[0035] Figure 1 shows an example of a camera arrangement 1 comprising a camera 2 and a mobile device 3. The camera 2 is mounted stationary, for example mounted on a wall. The camera 2 is adapted as PTZ camera (Pan, Tilt, Zoom) and has a camera geolocation. The camera geolocation comprises in this embodiment the position of the camera 2 as a position in three-dimensional space and an orientation of the camera 2, for example three Euler angles. For example the position in the geolocation is given by the coordinates of grid point P. The grid point P is for example given by the latitude and longitude of the camera 2. The position of the geolocation may also comprise the height h, which is in principle is easily measurable during and / or after mounting of the camera 2.

[0036] To determine the longitude and latitude of the camera in the camera geolocation a device geolocation is measured and / or determined. Therefore, the mobile device 3 is arranged in a surrounding area 4 of the camera 2, wherein the surrounding area 4 is a sphere around the camera 2 with the radius less than 30 cm. For example the camera 2 and the mobile device 3 are coupled for data exchange using a wireless connection 5, e.g. Bluetooth. Based on the wireless connection 5, for example its signal strength, the distance between the mobile device 3 and the camera 2 is determined. The mobile device 3 is adapted and / or configured to indicate a user if the mobile device 3 is placed inside the camera surrounding area 4, especially using a display 6 or signal lamp for indication. The mobile device 3 determines its position, especially geolocation, and / or orientation using GPS and / or inertia sensors.

[0037] The orientation is especially determined when the mobile devices 3 device recording direction 7 is aligned parallel to the camera’s 2 camera recording direction 8. Additionally and / or alternatively, the orientation is determined based on a picture taken by the camera 2 and a further picture 9 taken by the mobile device 3. Both pictures show the same object, mostly in different views. Both pictures are taken when the mobile device is placed inside the surrounding area. Hence, based on the two pictures, especially by comparing the views, and based on the knowledge of the orientation of the mobile device 3 the orientation of the camera 2 is determined and provided to the camera 2 and / or used in the camera geolocation.

Claims

Claims1. Method for a configuration of a camera (2), wherein the camera (2) is arranged and / or mounted stationary, wherein the camera (2) has a recording area and is configured to take image data for the recording area, wherein the camera (2) comprises and / or stores a camera geolocation, wherein a mobile device (3) is placed in a surrounding area (4) of the camera (2), wherein the mobile device (3) determines a mobile device geolocation and provides the mobile device geolocation to the camera (2), wherein the camera (2) stores the mobile device geolocation as the camera geolocation and / or the mobile device geolocation is assigned to camera (2) as the camera geolocation.

2. Method according to claim 1, wherein the mobile device (3) determines the mobile device geolocation based on GPS, radiolocation and / or WiFi- localization.

3. Method according to claim 1 or 2, wherein the mobile device (3) determines the mobile device geolocation in 6 degrees of freedom and / or as location in a real world coordinate system, especially in a geographic coordinate system of the earth.

4. Method according to one of the previous claims, wherein the mobile device geolocation comprises and / or is descripted using a latitude and a longitude.Method according to one of the previous claims, wherein the camera geolocation comprises a system time, wherein the mobile device geolocation comprises a location time, wherein the camera (2) stores the location time as the system time and / or the location time is assigned to camera (2) as the system time. Method according to one of the previous claims, wherein the camera (2) and / or the mobile device (3) determines a distance between the camera (2) and the mobile device (3), wherein the placement of the mobile device (3) inside and / or outside the surrounding area (4) of the camera is signalled based on the determined distance. Method according to one of the previous claims, wherein the camera (2) and / or the mobile device (3) determines the distance between the camera (2) and the mobile device (3), wherein the placement of the mobile device (3) inside and / or outside the surrounding of the camera (2) is determined based on the determined distance, wherein the mobile device (3) is blocked to determine and / or to provide the mobile device (3) geolocation to the camera (2). Method according to one of the previous claims, wherein the distance between the camera (2) and the mobile device (3) is determined based on a wireless signal. Method according to one of the previous claims, wherein the camera (2) comprises and / or defines a camera recording direction (8), wherein the mobile device (3) is arranged in the recording area of the camera (2), wherein an orientation, especially an azimuth of the camera (2), is determined based on the connection line between the geolocation of the mobile device (2), the position and / or geolocation of the camera (2) and the camera recording direction (8). Method according to one of the previous claims, wherein the mobile device (3) comprises and / or defines device recording direction (7), wherein the camera (2) comprises and / or defines a camera recording direction (8),wherein the mobile device (3) is arranged with the device recording direction (7) parallel to the camera recording direction (8), wherein an orientation of the mobile device (3) is determined when the recording directions are parallel, wherein the orientation of the mobile device (3) is provided to the camera (2) and / or the orientation of the mobile device (3) is set as an orientation of the camera (2). Method according to one of the previous claims, wherein a user and / or object is positioned in the recording area and the camera (2) takes an image of the user and / or object in the recording area, wherein the mobile device (3) is arranged in the surrounding of the camera (2) and the mobile device (3) takes a further image of the user and / or object in the recording area, wherein the orientation of the mobile device (3) and / or an orientation of the camera (2) is determined based on the image and the further image. Camera arrangement with at least one camera (2) and an mobile device (3), wherein the camera (2) is arranged and / or amounted stationary and comprises and / or stores a camera geolocation, wherein the mobile device (3) is adapted and / or configured to determine its mobile device (3) geolocation when the mobile device (3) is arranged and / or positioned inside a surrounding area (4) of the camera (2), wherein the mobile device (3) is adapted and / or configured to provide the determined mobile device (3) geolocation to the camera (2), wherein the camera (2) is adapted and / or configured to store, use and / or assign the provided device geolocation as the camera geolocation. Computer program, wherein the computer program is adapted and / or configured for execution on a computer and / or on the camera arrangement according to claim 12, wherein the computer program is adapted and / or configured to implement, use and / or execute the method according to one of the claims 1 to 11. Storage medium, wherein the computer program is stored on the storage medium.