A 360° panoramic camera positioning system and positioning method thereof
Through the 360° panoramic camera positioning system and image processing technology, the panoramic mirror and positioning markers are used to solve the blind spots and cost problems of video surveillance equipment, and panoramic camera and precise positioning are achieved, which is suitable for large-scale area monitoring.
Patent Information
- Application Number
- CN202010442044.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-05-22
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2040-05-22
AI Technical Summary
Existing video surveillance equipment is prone to blind spots when monitoring large areas, which is costly and lacks positioning functions, making it difficult to accurately locate objects in the image.
The 360° panoramic imaging positioning system is adopted to realize the 360° imaging function through the panoramic reflection mirror, and multiple positioning marks are used to calculate the position of the object to be measured to realize the panoramic imaging and positioning functions.
It realizes panoramic photography without dead angles, reduces costs, and can efficiently and accurately locate objects in the scene. It is suitable for monitoring needs of relatively open scenes, with simple structure, convenient installation and good waterproof performance.
Smart Images

Figure CN111447421B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a 360-degree panoramic camera positioning system and a positioning method thereof, and belongs to the technical field of panoramic camera. Background Art
[0002] With the development of science and technology, video surveillance equipment is everywhere in the world. Video surveillance equipment can help people to view specific people and things in specific locations in real time and accurately, and ensure the safety and stability of society. The development of video surveillance equipment is developing from a single angle to full angles. For large open areas, video equipment with wider camera angles can not only save monitoring costs, but also avoid blind spots in monitoring.
[0003] Wide-angle cameras currently on the market are generally single cameras with rotatable angles, camera devices composed of multiple cameras, or a combination of the first two. Although cameras with rotatable angles can monitor a wider range, it is inevitable that other angles cannot be monitored during the rotation of the camera. The camera device composed of multiple cameras invisibly increases the monitoring cost, and if one of the cameras is damaged, subsequent repairs and replacements are also troublesome.
[0004] In addition, people's current demand for panoramic cameras is not just to record images, but also to locate or track objects in the images, because the image-based positioning function can further improve the efficiency of video surveillance and create more possibilities for video surveillance. Summary of the invention
[0005] In order to solve the problems of blind spots in monitoring, high cost and lack of positioning function in the existing camera devices, the present invention proposes a 360° panoramic camera positioning system and a positioning method thereof. The system of the present invention reflects objects within a 360° range around the device into the shooting range of the camera through a panoramic reflective mirror, thereby realizing a panoramic camera function, and at the same time uses multiple positioning markers to realize the positioning function of surrounding objects. The method of the present invention analyzes the positioning markers and the object to be measured in the panoramic image through image processing technology, and calculates the position of the object to be measured.
[0006] In order to solve the above technical problems, the present invention adopts the following technical means:
[0007] A 360° panoramic camera positioning system comprises a panoramic camera device and a plurality of positioning markers, wherein the panoramic camera device comprises a view cup, a panoramic reflective mirror, a top cover, a view cup mounting seat, a camera, a camera bracket and a base, wherein the top cover and the view cup mounting seat are respectively provided with view cup mounting grooves, the top of the view cup is fixed in the view cup mounting groove of the top cover, the bottom of the view cup is fixed in the view cup mounting groove of the view cup mounting seat, the base is connected to the bottom of the view cup mounting seat by bolts, the camera bracket is fixedly installed inside the base; the camera is fixed to the camera bracket by screws, and the camera passes through the view cup mounting seat and enters the view cup, the panoramic reflective mirror is fixedly installed inside the view cup and is located directly above the camera; and the positioning markers are evenly distributed around the panoramic camera device.
[0008] Furthermore, the width of the sight cup installation groove is consistent with the thickness of the sight cup, and the inner side of the sight cup installation groove is higher than the outer side.
[0009] Furthermore, O-rings are provided between the sight cup and the top cover and the sight cup mounting seat; and a sealing gasket is provided between the sight cup mounting seat and the base.
[0010] Furthermore, the panoramic reflective mirror surface is a conical mirror surface, a regular pyramid mirror surface or a hemispherical mirror surface, and the perpendicular bisector of the panoramic reflective mirror surface coincides with the perpendicular bisector of the camera.
[0011] Furthermore, the camera bracket is provided with a plurality of waist-shaped holes.
[0012] Furthermore, the positioning marker is a luminous body or a highly reflective object.
[0013] A positioning method based on a panoramic camera positioning system specifically comprises the following steps:
[0014] S1. Set up a panoramic camera positioning system according to monitoring requirements and enter basic information of the panoramic camera positioning system;
[0015] S2, using a panoramic camera device to collect original panoramic images in real time, and preprocessing the collected original panoramic images;
[0016] S3, using the closing operation in grayscale morphology to perform denoising and smoothing on the preprocessed image to obtain an image after the closing operation;
[0017] S4, defining a structure element sequence {B}, and performing image thinning processing on the image after the closing operation using the structure element sequence {B} to obtain a thinned image;
[0018] S5. Selecting a ROI area and a search quantity from the refined image according to the object to be tested, searching for the edges of the object to be tested and the positioning marker in the ROI area by edge point detection technology, and obtaining a search line corresponding to the object to be tested and the positioning marker;
[0019] S6. Establish a rectangular coordinate system in the ROI area, calculate the angle value of each search line in the rectangular coordinate system, and calculate the position of the object to be measured according to the angle value of the object to be measured, the angle value of the positioning marker and the basic information.
[0020] Furthermore, the basic information in step S1 includes the number of the positioning marker, the coordinates of the positioning marker, the coordinates of the panoramic camera device, and the straight-line distance between the positioning marker and the panoramic camera device.
[0021] Furthermore, the specific operations of the preprocessing in step S2 are:
[0022] The original panoramic image is gray-processed to obtain a gray-scale image, and the image is binarized according to the gray-scale histogram of the gray-scale image to obtain a processed binary image.
[0023] Furthermore, the specific operations of step S5 are as follows:
[0024] S51, selecting an object to be detected in the refined image, and selecting a ROI area and a search quantity according to the object to be detected, wherein the ROI area includes all objects to be detected and at least one positioning marker, and the search quantity refers to the total number of objects to be detected and positioning markers in the ROI area;
[0025] S52, selecting an object to be detected or a positioning marker as a search target, and using a search tool to perform edge point detection on the search target along multiple search paths in the ROI area;
[0026] S53, performing curve fitting on the multiple edge points detected in S52 to obtain the edge line of the search target, that is, the search line;
[0027] S54, repeat steps S52 and S53 until the search lines corresponding to all the objects to be detected and the positioning markers in the ROI area are obtained.
[0028] The following advantages can be obtained by using the above technical means:
[0029] The present invention proposes a 360° panoramic camera positioning system and a positioning method thereof. The panoramic reflective mirror in the system of the present invention can map out objects within a 360° range around the panoramic camera device. The camera collects the image reflected by the panoramic reflective mirror in real time to realize the 360° panoramic camera function. Since positioning markers are distributed around the device, the camera can collect the positioning markers while collecting the surrounding images. The position of the object in the image can be judged according to the positioning markers to realize the panoramic positioning function. The system of the present invention can change the distance between the camera and the panoramic reflective mirror by adjusting the position of the camera on the camera bracket, and then adjust the coverage of the panoramic camera to meet the camera requirements of different scenes. The system of the present invention has a simple structure, is easy to install, and has good waterproof performance. In addition, the system only needs one camera, which is lower in cost than other panoramic camera devices and can meet the panoramic camera positioning requirements of relatively open scenes such as parks, fields, and large factories. The method of the present invention processes the images collected by the panoramic camera positioning system through a morphological method, and uses tools to search for the edge of the object to be measured, calculates the angle value of the object to be measured, and finally calculates the position of the object to be measured by combining the angle value of the object to be measured, the data of the positioning marker, etc., to achieve efficient and accurate positioning function. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 The figure is a schematic structural diagram of a 360° panoramic camera positioning system of the present invention.
[0031] Figure 2 For the present invention Figure 1 Schematic diagram of the structure from the A-A' perspective.
[0032] Figure 3 It is a structural schematic diagram of the viewing cup mounting seat in the device of the present invention.
[0033] Figure 4 It is a structural schematic diagram of the camera bracket in the invention device.
[0034] Figure 5 The present invention is a flowchart of the steps of a positioning method based on a panoramic camera positioning system.
[0035] Figure 6 Schematic diagram of an image after a closing operation in an embodiment of the method of the present invention.
[0036] Figure 7 Schematic diagram of the structural element sequence {B} in the embodiment of the method of the present invention.
[0037] Figure 8 It is a schematic diagram of a refined image in an embodiment of the method of the present invention.
[0038] Fig. 9 Schematic diagram of searching for a straight line in an embodiment of the method of the present invention.
[0039] Fig.10 Schematic diagram of angle value calculation in an embodiment of the method of the present invention.
[0040] In the figure, 1 is a sight cup, 2 is a panoramic reflective mirror, 3 is a top cover, 4 is a waterproof brim, 5 is a sight cup mounting base, 6 is a camera, 7 is a camera bracket, 8 is a base, 9 is a sealing gasket, 10 is an O-ring, 11 is a sight cup mounting groove, 12 is a screw hole, 13 is a waist-shaped hole, 14 is a flange, 15 is a positioning marker, 16 is an object to be measured, 17 is a panoramic camera device, a is the angle value of the positioning marker, and b and c are the angle values of the object to be measured. DETAILED DESCRIPTION
[0041] The technical solution of the present invention is further described below in conjunction with the accompanying drawings:
[0042] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0043] A 360° panoramic camera positioning system includes a panoramic camera device and multiple positioning markers, wherein the panoramic camera device is used to realize a 360° no-dead-angle camera function, and the positioning markers are used to assist in determining the position of a specific object in the captured image.
[0044] like Figure 1 , 2 As shown in Figures 3 and 4, the panoramic camera device mainly includes a view cup 1, a panoramic reflective mirror 2, a top cover 3, a waterproof brim 4, a view cup mounting seat 5, a camera 6, a camera bracket 7 and a base 8, wherein the top cover and the view cup mounting seat are respectively provided with view cup mounting grooves 11, the groove width of the view cup mounting groove is consistent with the thickness of the view cup, and the inner side of the view cup mounting groove is higher than the outer side, a plurality of screw holes 12 are provided on the top cover, the view cup mounting seat and the outer side of the view cup mounting groove, an opening for the camera to pass through is also provided at the center position of the view cup mounting seat, screw holes are provided on the short side of the camera bracket, and a plurality of waist-shaped holes 13 for adjusting the mounting position are provided on the long side of the camera bracket.
[0045] The top of the sight cup is inserted into the sight cup installation groove of the top cover, and the bottom of the sight cup is inserted into the sight cup installation groove of the sight cup mounting seat. The sight cup is fixed to the top cover and the sight cup mounting seat by using a set screw through the screw hole on the outer side of the sight cup installation groove. The base and the bottom of the sight cup mounting seat are fixedly connected by bolts. For sealing, a sealing gasket 9 is installed between the base and the sight cup mounting seat. The short side of the camera bracket is fixedly installed on the top of the base by screws, the long side of the camera bracket is perpendicular to the ground, the camera is fixed on the long side of the camera bracket by screws, and the camera passes through the opening in the center of the sight cup mounting seat to enter the inside of the sight cup. The contact part between the camera and the sight cup mounting seat is connected by a threaded form, and the panoramic reflective mirror surface is also fixedly installed in the sight cup by a threaded form, and is located directly above the camera. The midpoint perpendicular line of the panoramic reflective mirror surface coincides with the midpoint perpendicular line of the camera. The panoramic camera device of the present invention can adjust the distance between the camera and the panoramic reflective mirror surface by adjusting the position where the camera is installed on the camera bracket.
[0046] Since the system of the present invention is mainly suitable for large outdoor spaces, in order to improve the waterproofness of the system, a waterproof brim is installed on the top cover in a tight fit manner, and the edge of the waterproof brim has a downward fringe to prevent rainwater and other liquids from penetrating into the interior of the device. In addition, O-rings 10 are provided between the top cover and the sight cup and between the sight cup mounting seat and the sight cup, thereby achieving a double waterproof splash or penetration effect.
[0047] In order to improve the clarity of the panoramic camera, the viewing cup is preferably made of tempered borosilicate glass with high light transmittance and high rigidity. In order to facilitate installation and balance of the adjustment device, a flange 14 is provided at the bottom of the base of the device.
[0048] The panoramic reflective mirror in the device is mainly used to reflect the surrounding scenes. The panoramic reflective mirror can be a conical mirror, a regular pyramid mirror or a hemispherical mirror. In the embodiment of the present invention, that is, in the accompanying drawings, the panoramic reflective mirror adopts a conical mirror with a taper of 45°. The taper, number of edges or radius of the panoramic reflective mirror are different, and the range of scenes that can be reflected is different. In addition, the distance between the camera and the panoramic reflective mirror is different, and the range of panoramic photography is also different. In actual use, the panoramic reflective mirror and the distance can be selected according to specific needs.
[0049] The positioning markers in the system of the present invention are generally luminous bodies or highly reflective objects (objects with high reflectivity). After the camera range is determined, the panoramic camera device can be installed in the central area of the camera range, or can be installed on movable patrol cars, robots and other objects, and the positioning markers are evenly distributed within the camera range, that is, distributed around the panoramic camera device.
[0050] The working principle of a 360° panoramic camera positioning system of the present invention is as follows:
[0051] Panoramic cameras and positioning markers are installed as needed according to the camera range. The panoramic reflective mirror in the panoramic camera can map objects within a 360° range around the device. The camera located directly below the panoramic reflective mirror collects the image projected on the mirror in real time to obtain the panoramic image required for monitoring. Since the positioning marker is also within the camera range, the approximate position of an object in the panoramic image can be obtained through the position of the positioning marker in the panoramic image. For example, if the camera takes a panoramic image in which a person is near a positioning marker in the southeast direction, it can be determined that the person was located in the southeast corner of the camera range when the camera was shooting.
[0052] A positioning method based on a panoramic camera positioning system, such as Figure 5 As shown, the specific steps include:
[0053] S1. Set up a panoramic camera positioning system according to monitoring requirements and enter the basic information of the panoramic camera positioning system; install panoramic camera devices and positioning markers according to the location and monitoring range to be monitored. Generally, the panoramic camera device is fixedly installed at the center of the monitoring range, and the positioning markers are evenly distributed within the monitoring range. After completing the installation of the panoramic camera positioning system, enter the basic information of the panoramic camera positioning system into the system. The basic information mainly includes the number of the positioning marker, the coordinates of the positioning marker, the coordinates of the panoramic camera device, and the straight-line distance between the positioning marker and the panoramic camera device. In order to facilitate the distinction between positioning markers, identification marks can be installed on different positioning markers, and then the identification marks are matched one by one with the number of the positioning marker. The coordinates of the positioning marker and the coordinates of the panoramic camera device are generally geodetic coordinates.
[0054] S2. Using a panoramic camera device to collect original panoramic images in real time, and preprocessing the collected original panoramic images, wherein the specific operations of the preprocessing are:
[0055] Perform grayscale processing on the original panoramic image, take the R value in the RGB channel of the original panoramic image as the grayscale value, and obtain a grayscale image;
[0056] The image is binarized according to the grayscale histogram of the grayscale image. When the grayscale value of a pixel is greater than the grayscale threshold, the grayscale value of the pixel is modified to 255. When the grayscale value of a pixel is less than the grayscale threshold, the grayscale value of the pixel is modified to 0 to obtain a processed binary image. In the embodiment of the present invention, the grayscale threshold is 150.
[0057] S3. Use the closing operation in grayscale morphology to denoise and smooth the preprocessed image to obtain an image after the closing operation. The preprocessed image generally has more noise, and it is necessary to use the closing operation in morphology to further process the image. First, select the structural element B according to the shape of the image to be processed, and then use the structural element B to perform a closing operation on the preprocessed image set A. The formula is as follows:
[0058]
[0059] Among them, A·B represents the closing operation of B on A. It means A is expanded by B, and Θ means erosion.
[0060] The closing operation can eliminate isolated black spots in bright areas and smooth the boundaries. Figure 6 Schematic diagram of an image after a closing operation in an embodiment of the present invention.
[0061] S4. Define the structure element sequence {B}, and use the structure element sequence {B} to perform image thinning on the image after the closing operation to obtain the thinned image; image thinning refers to the process of changing the width of each particle in the image to one pixel. Image thinning will not change the original shape and direction information of the object, and has the characteristic of translation invariance. Depending on the image to be thinned, the selected structure element sequence is also different. Suppose the structure element sequence is {B} = {B 1 ,B 2 ,…,B i-1 ,B i ,…,B n}, where B i represents the i-th structural element, B i It is B i-1 In the rotated form, i = 1, 2, ..., n, where n is the total number of structural elements, image refinement can be defined based on hit or miss transformations, as follows:
[0062]
[0063]
[0064] Among them, A' represents the image to be refined, and A' is represented by the structural element B 1 After one refinement, the result is 2 Refine it again, and continue like this until A' is replaced by B n The whole process is repeated until the result does not change, and the image is refined. Figure 7 is a schematic diagram of a structural element sequence {B} in an embodiment of the present invention, Figure 8 It is a schematic diagram of the refined image.
[0065] S5. Select the ROI area and the search quantity from the refined image according to the object to be tested, search the edges of the object to be tested and the positioning marker in the ROI area by edge point detection technology, and obtain the search straight line corresponding to the object to be tested and the positioning marker; the specific operation is as follows:
[0066] S51. Select an object to be tested in the refined image, and select a ROI area and a search quantity according to the object to be tested. The selected ROI area needs to include all objects to be tested and at least one positioning marker. The search quantity refers to the total number of objects to be tested and positioning markers in the ROI area.
[0067] S52, select an object to be detected or a positioning marker as a search target, and use the search tool to detect edge points of the search target along multiple search paths in the ROI area. Generally, the ROI area selects a rectangular area, and the search tool uses a rectangular rake, which can search for edge points based on multiple lines inside the rectangular ROI and parallel to the rectangular ROI, and can search for rising edges, falling edges, the first and last or all edge points on each search path, as well as the best edge point with the largest edge strength.
[0068] S53, performing curve fitting on the multiple edge points detected in S52 to obtain the edge line of the search target, that is, the search line, such as Fig. 9 shown.
[0069] S54, repeat steps S52 and S53 until the search lines corresponding to all the objects to be detected and the positioning markers in the ROI area are obtained.
[0070] S6. Establish a rectangular coordinate system in the ROI area, calculate the angle value of each search line in the rectangular coordinate system, and calculate the position of the object to be measured according to the angle value of the object to be measured, the angle value of the positioning marker and the basic information. Fig.10 It is a schematic diagram of the angle value in an embodiment of the present invention. In the figure, a rectangular coordinate system is established with the panoramic camera device 17 as the coordinate origin. The angle value of the positioning marker 15 is a, and the angle values of the two objects to be measured are b and c respectively. Since the coordinates of the positioning marker and the distance from the positioning marker to the panoramic camera device are known, the coordinates of the object to be measured can be calculated by the angle value of the object to be measured.
[0071] The panoramic camera device in the system of the present invention performs camera operation based on the principle of light reflection. There is no blind spot in camera shooting. Only one camera is needed to simultaneously collect 360° images around the device. The cost is lower and the camera angle is wider. The device has a simple structure, is easy to install, and has good waterproof performance and stability. It is particularly suitable for monitoring needs in open and empty scenes. The method of the present invention performs image processing and analysis based on the collected panoramic images and positioning markers, and can accurately obtain the specific position of any object within the camera range, thereby providing an efficient and accurate positioning function based on panoramic camera shooting.
[0072] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in the field without departing from the purpose of the present invention.
Claims
1. A positioning method based on a panoramic camera positioning system, characterized in that: The following steps are involved: S1. Set up a panoramic camera positioning system according to monitoring requirements and enter basic information of the panoramic camera positioning system; S2, using a panoramic camera device to collect original panoramic images in real time, and preprocessing the collected original panoramic images; S3, using the closing operation in grayscale morphology to perform denoising and smoothing on the preprocessed image to obtain an image after the closing operation; S4, defining a structure element sequence {B}, and performing image thinning processing on the image after the closing operation using the structure element sequence {B} to obtain a thinned image; S5. Selecting a ROI area and a search quantity from the refined image according to the object to be tested, searching for the edges of the object to be tested and the positioning marker in the ROI area by edge point detection technology, and obtaining a search line corresponding to the object to be tested and the positioning marker; S6. Establish a rectangular coordinate system in the ROI area, calculate the angle value of each search line in the rectangular coordinate system, and calculate the position of the object to be measured according to the angle value of the object to be measured, the angle value of the positioning marker and the basic information; The panoramic camera positioning system comprises a panoramic camera device and a plurality of positioning markers, wherein the panoramic camera device comprises a sight cup, a panoramic reflective mirror, a top cover, a sight cup mounting seat, a camera, a camera bracket and a base, wherein the top cover and the sight cup mounting seat are respectively provided with sight cup mounting grooves, the top of the sight cup is fixed in the sight cup mounting groove of the top cover, the bottom of the sight cup is fixed in the sight cup mounting groove of the sight cup mounting seat, the base is connected to the bottom of the sight cup mounting seat by bolts, and the camera bracket is fixedly installed inside the base; the camera is fixed to the camera bracket by screws, and the camera passes through the sight cup mounting seat and enters the sight cup, and the panoramic reflective mirror is fixedly installed inside the sight cup and is located directly above the camera; The positioning markers are evenly distributed around the panoramic camera device.
2. A positioning method based on a panoramic camera positioning system according to claim 1, characterized in that: The width of the sight cup installation groove is consistent with the thickness of the sight cup, and the inner side edge of the sight cup installation groove is higher than the outer side edge.
3. The positioning method based on a panoramic camera positioning system according to claim 1, characterized in that: O-type sealing rings are arranged between the sight cup and the top cover and the sight cup mounting seat; and a sealing pad is arranged between the sight cup mounting seat and the base.
4. The positioning method based on a panoramic camera positioning system according to claim 1, characterized in that: The panoramic reflective mirror surface is a conical mirror surface, a regular pyramid mirror surface or a hemispherical mirror surface, and the perpendicular bisector of the panoramic reflective mirror surface coincides with the perpendicular bisector of the camera.
5. The positioning method based on a panoramic camera positioning system according to claim 1, characterized in that: The camera bracket is provided with a plurality of waist-shaped holes.
6. The positioning method based on a panoramic camera positioning system according to claim 1, characterized in that: The positioning marker is a luminous body or a highly reflective object.
7. The positioning method based on a panoramic camera positioning system according to claim 1, characterized in that: The basic information in step S1 includes the number of the positioning marker, the coordinates of the positioning marker, the coordinates of the panoramic camera device, and the straight-line distance between the positioning marker and the panoramic camera device.
8. The positioning method based on a panoramic camera positioning system according to claim 1, characterized in that: The specific operations of the pre-processing in step S2 are: The original panoramic image is gray-processed to obtain a gray-scale image, and the image is binarized according to the gray-scale histogram of the gray-scale image to obtain a processed binary image.
9. The positioning method based on a panoramic camera positioning system according to claim 1, characterized in that: The specific operation of step S5 is as follows: S51, selecting an object to be detected in the refined image, and selecting a ROI area and a search quantity according to the object to be detected, wherein the ROI area includes all objects to be detected and at least one positioning marker, and the search quantity refers to the total number of objects to be detected and positioning markers in the ROI area; S52, selecting an object to be detected or a positioning marker as a search target, and using a search tool to perform edge point detection on the search target along multiple search paths in the ROI area; S53, performing curve fitting on the multiple edge points detected in S52 to obtain the edge line of the search target, that is, the search line; S54, repeat steps S52 and S53 until the search lines corresponding to all the objects to be detected and the positioning markers in the ROI area are obtained.
