A video preset point calibration method

Abstract

The present invention relates to the field of video surveillance technology, and in particular to a video preset point calibration method, which presets images and collects images in a matrix manner on a two-dimensional coordinate system constructed with a camera preset point of 0 degrees as the origin, and stores reference image related information obtained in the image in a database, wherein the reference image related information includes a reference image, the camera PTZ coordinates of the center point of the reference image, and the working status of the camera. The present invention studies the video surveillance preset point calibration algorithm and the video surveillance omnidirectional reference image fitting technology, and develops a preset point calibration system, provides preset point management services, and realizes automatic deviation correction calibration of camera preset points. Video preset point calibration can improve the availability of video preset points, significantly reduce maintenance costs, realize centralized management and service of preset points, and improve the reliability of video intelligent inspection. The system is suitable for various substation intelligent inspection projects.

Description

Technical Field

[0001] The present invention relates to the technical field of video surveillance, and in particular to a video preset point calibration method. Background Art

[0002] Smart inspections using cameras in substations rely on a number of preset points set for the cameras. These points are linked to the inspection content by calling different preset points. Currently, preset point management is performed through regular on-site manual maintenance, requiring collaboration between production personnel and security professionals. A 110kV substation video smart inspection involves 42 cameras and 900 preset points, and a full station calibration takes five working days. The number of preset points for 330kV and 750kV reaches 2,700 to 3,500, requiring even longer maintenance time. Preset points are prone to drift, and the calibration and maintenance workload is high, severely limiting the reliability of smart video inspections. Summary of the Invention

[0003] In order to solve the problem that preset points of substation cameras are easily offset and have poor timeliness, the present invention provides a video preset point calibration method and device.

[0004] A video preset point calibration method, characterized by comprising:

[0005] Preset image acquisition: collect images in a matrix manner on a two-dimensional coordinate system constructed with the camera preset point 0 degrees as the origin, and store the reference image related information obtained from the image in a database, where the reference image related information includes the reference image, the camera PTZ coordinates of the center point of the reference image, and the working status of the camera;

[0006] Calibrate OSD. After completing the preset image acquisition, calibrate the OSD of the acquired reference image to obtain a calibration image.

[0007] Calibration trigger, manual or automatic control of camera connection, at least one camera forms a queue for one calibration, each calibration generates a unique UUID as the calibration task number;

[0008] Calibrate the camera preset points, obtain the current camera offset angle and zoom ratio, rotate and adjust the focal length according to the camera PTZ coordinates, use the computer server to remotely call the service interface provided by the recorder through the network, operate the camera, and capture images of the camera's correct reference point preset points. Use the calibration algorithm to compare the preview image captured by the camera with the calibration image to obtain the offset value between the preview image and the calibration image, and calibrate the camera based on the offset value;

[0009] Returns the calibration result. Gets the calibration task number. Based on the task number, gets the calibration result of a single camera that is being calibrated or gets the calibration results of all cameras that have completed calibration.

[0010] Furthermore, the preset image acquisition is performed in a row-by-row scanning manner in an image acquisition matrix within a range of 3 degrees above and below and 12 degrees left and right of the camera preset point, obtaining multiple reference images named according to serial numbers, as well as 1 preset point image before calibration and 1 preset point image after calibration.

[0011] Furthermore, the route of collecting the reference image is S-shaped in the two-dimensional coordinate system. When the motion coordinates of the two-dimensional coordinate system reach the camera motion boundary, the collected reference image is truncated with the preset point as the center.

[0012] Furthermore, the OSD calibration includes the following steps:

[0013] Segment the potential white and black characters in each collected benchmark image, and discard the potential connected domains that do not meet the width and height requirements;

[0014] The segmentation results of each benchmark image are superimposed and normalized, and the frequency of potential regions is counted;

[0015] After counting the frequencies, perform bilateral filtering and sharpening on the benchmark image, and then segment the high-frequency areas of the benchmark image;

[0016] After segmenting the high-frequency regions, the reference image is subjected to morphological dilation and erosion processing, and the non-character regions in the reference image are filtered to obtain the OSD region mask;

[0017] The OSD area mask is projected horizontally and vertically to obtain the OSD area coordinates. Nine reference images in the center of all reference images are selected according to the camera PTZ coordinates. The mapping relationship between the pixel offset and the angle offset in the x and y directions is calibrated in the OSD area coordinates.

[0018] Furthermore, the camera preset point calibration includes the following steps:

[0019] The calibration image is matched with the best calibration image in the image sequence of the correct reference preset point, and the offset between the preview image and the calibration image is calculated;

[0020] Calculate the offset between the calibration image and the central reference image among the nine reference images located in the center;

[0021] Calculate the offset between the central reference image and the pre-calibration preset point image among the 9 reference images located in the middle;

[0022] The sum of the above three offsets is the offset between the preview image and the correct reference preset point image.

[0023] Furthermore, the median pixel offset between the image pairs in the x and y directions of the 9 reference images located in the center is calculated, and the calibration relationship is calculated from the median pixel offset and the PTZ coordinates.

[0024] Furthermore, the pixel offset between the preview image currently captured by the camera and the calibration image is continuously calculated, and the angle offset is calculated using the calibration relationship.

[0025] Furthermore, calculating the pixel offset between two images includes the following steps:

[0026] Utilize surf feature point extraction to delete feature points in the text area, calculate feature point descriptors, extract matching feature point pairs based on the nearest and second nearest distance thresholds, filter feature point pairs using the basic matrix constraints, and then use the matching slope to filter feature point pairs, and count the pixel offsets between feature point pairs.

[0027] When the number of matches found is greater than the threshold, the calibration is successful and the angle offset of the match at this time is returned.

[0028] Beneficial effects of the invention:

[0029] Through research on video surveillance preset point calibration algorithms and video surveillance omnidirectional reference image fitting technology, the present invention has developed a preset point calibration system, which provides preset point management services and realizes automatic correction and calibration of camera preset points. Video preset point calibration can improve the availability of video preset points, significantly reduce maintenance costs, realize centralized management and services of preset points, and improve the reliability of video intelligent inspection. It is suitable for various types of substation intelligent inspection projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 This is a calibration algorithm flow chart of a video preset point calibration method proposed by the present invention;

[0032] Figure 2 A schematic diagram of a preset image acquisition method for video preset point calibration proposed by the present invention;

[0033] Figure 3 This is a flowchart of pixel offset calculation of a video preset point calibration method proposed by the present invention;

[0034] Figure 4This is a schematic diagram of the system workflow of a video preset point calibration method proposed by the present invention. DETAILED DESCRIPTION

[0035] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with examples and drawings. The exemplary embodiments of the present invention and their descriptions are only used to explain the present invention and are not intended to limit the present invention.

[0036] Example 1

[0037] refer to Figure 1 、 Figure 2 、 Figure 3 and Figure 4 The present invention proposes a video preset point calibration method including preset image acquisition, wherein the preset image acquisition needs to acquire a total of 177 images, including 175 reference images named according to serial numbers, a preset point image first.png specified before calibration, and a preset point image now.png that is called again after calibration;

[0038] The preset acquisition rule is to capture 3 degrees above and below the preset point, and 12 degrees left and right, including the current 0 degree of the preset point, to generate a capture matrix with a total of 7 rows and 25 columns, for a total of 175 images. When acquisition begins, the camera moves to the upper left corner of the acquisition matrix, and then moves from left to right, performing row-by-row scanning acquisition. The acquisition route is S-shaped, and the coordinate control error is controlled within 0.1 degrees. The image is in PNG lossless format. If the motion coordinate reaches the camera motion boundary, the image is truncated with the preset point as the center, and the image number remains unchanged.

[0039] Also includes calibration OSD:

[0040] (1) Read 1-175 reference images in sequence. Since the images captured by the camera have text descriptions and the text colors are inconsistent, if the background color of the image is black and the text is white, when the text spans the two colors, half of the text will be white and half will be black. It is necessary to segment the potential white and black characters for each reference image and discard the potential connected domains that are too large or do not meet the width and height requirements;

[0041] (2) The segmentation results of all benchmark images are superimposed and normalized to count the frequency of potential regions;

[0042] (3) After counting the frequencies, bilateral filtering is performed to sharpen the images and then separate the regions with higher frequencies;

[0043] (4) Perform morphological dilation and corrosion processing based on the results to filter out larger and smaller non-character areas and obtain the OSD area mask;

[0044] (5) After obtaining the regional mask, perform horizontal and vertical projection to obtain the OSD regional coordinates. After preset mapping or OSD calibration, select the 9 images located in the center, including the 62nd, 63rd, 64th, 87th, 88th, 89th, 112th, 113th, and 114th images, based on all the reference images and PTZ coordinates of the correct reference preset points collected. Calibrate the mapping relationship between the pixel offset and the angle offset in the x and y directions, and record the calibration information. Since these 9 reference images can be approximately considered to be in the same plane, the calibration result will be more accurate.

[0045] It also includes a calibration trigger, which is a camera calibration trigger. The camera calibration trigger can be set to a timed automatic trigger or manually triggered. The condition for the camera to be successfully triggered is that the camera connection is normal.

[0046] It also includes camera calibration, using the PTZ coordinates recorded by rotating the camera to the preset point to capture the image of the preset point. In the correct reference preset point image sequence, the best matching image is searched and defined as ref_match. The offset is calculated, and the offset between the best matching image ref_match and the preset point center image 88.png is calculated. The offset between the preset point center image 88.png and the correct reference preset point image ref_first is calculated. The sum of these three offsets is the offset between the preview image and the correct reference preset point image.

[0047] The purpose of performing three offset calculations is to:

[0048] (1) If the preset point offset is too large, the preset point image lu_first that is reached by the control cannot be directly compared with the correct reference preset point image ref_first. Similarly, it cannot be directly compared with ref_88. Therefore, a search and match is required to find the best matching image around the preset point.

[0049] (2) If there is an error or delay in the camera motion control, there will be an error between ref_88 and ref_first. ref_match is a map taken with ref_88 as the center, so the deviation can only be calculated with ref_88, and cannot be directly calculated with ref_first.

[0050] The final deviation calculation formula (ref_first-lu_first) is:

[0051] (ref_first-ref_88)+(ref_88-ref_match)+(ref_match-lu_first).

[0052] Select the 9 images in the middle, images 62, 63, 64, 87, 88, 89, 112, 113, and 114, and calculate the median pixel offsets offset_x and offset_y between the image pairs in the x and y directions:

[0053] (1) Take 62-63, 63-64, 87-88, 88-89, 112-113, 113-114 to calculate the x direction.

[0054] (2) Take 62-87, 87-112, 63-88, 88-113, 64-89, 89-114 to calculate the y direction.

[0055] Then, the calibration relationship is obtained based on the PTZ angle difference and pixel offset difference:

[0056] bx = tan(angle_diff_x / 180*3.1415926 / 2) / offset_x

[0057] by=tan(angle_diff_y / 180*3.1415926 / 2) / offset_y.

[0058] Among them, angle_diff_x and angle_diff_y are obtained by subtracting the corresponding shooting angles in PTZ.

[0059] The pixel offset between the collected preview image and the reference image is continuously calculated, and the angular offset ref_match - lu_first is calculated using the calibration relationship, and ref_88 - ref_match and ref_first - ref_88 are superimposed.

[0060] Example 2

[0061] refer to Figure 3 , based on Example 1, this embodiment proposes a method for calculating the pixel offset of two images.

[0062] (1) Use surf feature point extraction to delete the feature points in the text area, calculate the feature point descriptor, extract the most matching feature point pairs according to the nearest distance threshold, use the basic matrix constraint to filter the feature point pairs, use the matching slope to filter the feature point pairs, and count the pixel offsets between the feature point pairs.

[0063] (2) When the number of matches found is greater than the threshold of 500, the calibration is considered successful, and the angle offset of the match at this time is returned, and the process ends early. Otherwise, the best match is found and the angle offset of the best match is returned.

[0064] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A video preset point calibration method, characterized in that: include: Preset image acquisition: collect images in a matrix manner on a two-dimensional coordinate system constructed with the camera preset point 0 degrees as the origin, and store the reference image related information obtained from the image in a database, where the reference image related information includes the reference image, the camera PTZ coordinates of the center point of the reference image, and the working status of the camera; Calibrate OSD. After completing the preset image acquisition, calibrate the OSD of the acquired reference image to obtain a calibration image. Calibration trigger, manual or automatic control of camera connection, at least one camera forms a queue for one calibration, each calibration generates a unique UUID as the calibration task number; Calibrate the camera preset points, obtain the current camera offset angle and zoom ratio, rotate and adjust the focal length according to the camera PTZ coordinates, use the computer server to remotely call the service interface provided by the recorder through the network, operate the camera, and capture images of the camera's correct reference point preset points. Use the calibration algorithm to compare the preview image captured by the camera with the calibration image to obtain the offset value between the preview image and the calibration image, and calibrate the camera based on the offset value; Return the calibration result, obtain the calibration task number, and obtain the calibration result of a single camera in the calibration process or the calibration results of all cameras that have completed calibration based on the task number; The method of calibrating the camera preset points comprises the following steps: The calibration image is matched with the best calibration image in the image sequence of the correct reference preset point, and the offset between the preview image and the calibration image is calculated; Calculate the offset between the calibration image and the central reference image among the nine reference images located in the center; Calculate the offset between the central reference image and the pre-calibration preset point image among the 9 reference images located in the middle; The sum of the above three offsets is the offset between the preview image and the correct reference preset point image; Calculate the median pixel offset between the image pairs in the x and y directions of the 9 reference images located in the center, and calculate the calibration relationship from the median pixel offset and the PTZ coordinates.

2. A video preset point calibration method according to claim 1, characterized in that: The preset image acquisition is performed in a row-by-row scanning manner in an image acquisition matrix within a range of 3 degrees above and below and 12 degrees left and right of the camera preset point, obtaining multiple reference images named according to serial numbers, as well as 1 preset point image before calibration and 1 preset point image after calibration.

3. A video preset point calibration method according to claim 2, characterized in that: The route of collecting the reference image is S-shaped in the two-dimensional coordinate system. When the motion coordinates of the two-dimensional coordinate system reach the camera motion boundary, the collected reference image is truncated with the preset point as the center.

4. A video preset point calibration method according to claim 1, characterized in that: The OSD calibration comprises the following steps: Segment the potential white and black characters in each collected benchmark image, and discard the potential connected domains that do not meet the width and height requirements; The segmentation results of each benchmark image are superimposed and normalized, and the frequency of potential regions is counted; After counting the frequencies, perform bilateral filtering and sharpening on the benchmark image, and then segment the high-frequency areas of the benchmark image; After segmenting the high-frequency regions, the reference image is subjected to morphological dilation and erosion processing, and the non-character regions in the reference image are filtered to obtain the OSD region mask; The OSD area mask is projected horizontally and vertically to obtain the OSD area coordinates. Nine reference images in the center of all reference images are selected according to the camera PTZ coordinates. The mapping relationship between the pixel offset and the angle offset in the x and y directions is calibrated in the OSD area coordinates.

5. A video preset point calibration method according to claim 1, characterized in that: The pixel offset between the preview image currently captured by the camera and the calibration image is continuously calculated, and the angle offset is calculated using the calibration relationship.

6. A video preset point calibration method according to claim 5, characterized in that: Calculating the pixel offset between two images involves the following steps: Utilize surf feature point extraction to delete feature points in the text area, calculate feature point descriptors, extract matching feature point pairs based on the nearest and second nearest distance thresholds, filter feature point pairs using the basic matrix constraints, and then use the matching slope to filter feature point pairs, and count the pixel offsets between feature point pairs. When the number of matches found is greater than the threshold, the calibration is successful and the angle offset of the match at this time is returned.

Images