Image equipment displacement abnormity early warning system and method based on feature point detection

Abstract

The invention provides an image equipment displacement abnormity early warning system and method based on feature point detection, and belongs to the technical field of abnormity early warning, and the method specifically comprises the steps: carrying out the feature point detection and descriptor calculation of a reference image and a current collection image through employing an ORB algorithm, matching the feature descriptors of the reference image and the current image to obtain image comparison image feature points, determining feature point similarity coefficients of different comparison image feature points by using a distance function, and matching the feature points of the different comparison image feature points by using the feature point similarity coefficients; according to the method, the homography matrix is constructed based on the matched feature points, geometric transformation information is extracted based on the homography matrix to obtain the rotation angle, the scaling ratio and the translation amount, whether the image equipment deviates or not is determined in combination with the preset threshold value, an early warning signal is sent out, and the reliability of monitoring processing is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of abnormality warning, and in particular relates to an image device displacement abnormality warning system and method based on feature point detection. Background Art

[0002] With the popularization of intelligent devices and Internet of Things technologies, image acquisition devices (such as cameras) are widely used in monitoring, navigation, intelligent manufacturing and other fields. However, during operation, the acquisition equipment may be displaced due to external interference (such as vibration, collision or manual adjustment), which may affect the accuracy of image acquisition and even cause system abnormalities.

[0003] Therefore, in order to dynamically monitor the installation status of the image acquisition device and issue an early warning in time when the deviation is abnormal, in the invention patent application CN202120809098.0 "Video surveillance device dedicated to tourist attractions", the locking mechanism is clamped on the tree trunk or the crossbar of the lamp post through lateral movement during use, and the tension spring controls the locking mechanism to close to prevent it from falling off, and then it can be powered on for use. The locking mechanism prevents the monitoring device from sliding and causing the monitoring position to shift. However, through analysis, the following technical problems exist: The prior art solutions do not disclose the use of the analysis results of the monitoring images to perform abnormal detection of abnormal displacement of the installation position of the image acquisition device, and thus cannot accurately realize the recognition and processing of abnormal changes in the installation position of the image device.

[0004] Specifically, the present invention proposes an image device displacement abnormality warning system and method based on feature point detection, which determines whether the device is offset by analyzing the geometric transformation relationship (rotation angle, scaling ratio and translation vector) between the image acquisition device in the initial installation state and the current operating state. Summary of the invention

[0005] To achieve the purpose of the present invention, the present invention adopts the following technical solutions: In a first aspect, the present application provides an image device displacement abnormality warning system based on feature point detection, specifically comprising: Feature point screening module, matrix calculation and information extraction module, displacement recognition and early warning module; The feature point screening module is responsible for detecting feature points and calculating descriptors of the reference image and the current captured image using the ORB algorithm, and obtaining image feature points for image comparison by matching the feature descriptors of the reference image and the current image; The matrix calculation and information extraction module is responsible for determining the feature point similarity coefficients of different comparison image feature points using the distance function, and matching feature points of different comparison image feature points using the feature point similarity coefficients, and constructing a homography matrix based on the matching feature points; The displacement identification and warning module is responsible for extracting geometric transformation information based on the homography matrix to obtain the rotation angle, scaling ratio and translation amount, and determines whether the image device is offset and issues a warning signal in combination with a preset threshold.

[0006] A further technical solution is that the feature point similarity coefficient is determined according to a Hamming distance function.

[0007] A further technical solution is that when any one of the rotation angle, the scaling ratio and the translation amount is greater than a preset threshold, it is determined that the image device is offset and a warning signal is issued.

[0008] In a second aspect, the present application provides an image device displacement abnormality warning method based on feature point detection, which is applied to the above-mentioned image device displacement abnormality warning system based on feature point detection, and specifically includes: S1 determines the change of light data in different time periods on different dates based on the analysis results of the monitoring images of the imaging device, and determines the feature comparison time period in the time period based on the change; S2: obtaining changes in different ORB image feature points and initial image feature points in the monitoring image during the feature comparison period, and determining the credibility coefficients of the comparison image feature points and different comparison image feature points during the feature comparison period by using the changes in the image feature points; S3: obtaining the change of the feature points of the compared image and the feature points of the initial image during the feature comparison period, and combining the credibility coefficients of different compared image feature points to determine that the image device has a suspected change, and then proceeding to the next step; S4 determines whether to issue a warning signal based on changes in the feature points of the compared images in different time periods.

[0009] The beneficial effects of the present invention are: Whether a suspected change has occurred in the image device is determined based on the changes in the feature points of the compared image during the feature comparison period and the initial image feature points, as well as the credibility coefficients of different compared image feature points. This enables accurate assessment of the probability of displacement change of the image feature points based on the changes in the feature points of the compared image during the feature comparison period and the credibility coefficients, and also lays the foundation for differentiated recognition and processing of the displacement of image feature points.

[0010] Whether to issue a warning signal is determined based on the changes in the feature points of the compared images in different time periods, avoiding the technical problem of insufficient recognition accuracy caused by simply considering the changes in the feature points of the compared images in a certain time period. It realizes the recognition and processing of warning signals from the change data of the feature points of the compared images in multiple time periods, thereby improving the reliability and accuracy of the output processing of warning signals of abnormal changes in image equipment.

[0011] A further technical solution is that the change of the light data is determined according to the brightness of the monitoring image of the image device in the time period.

[0012] A further technical solution is that the light data is determined according to a preset light brightness corresponding to the brightness of the monitoring image.

[0013] A further technical solution is that the method for determining the feature comparison period in the period is: Determine the light brightness of the time periods on different dates by using the light data of the time periods on different dates, and determine the reference light brightness by taking the average light brightness of the time periods on different dates; Determining the light change date among the dates according to the deviation between the light brightness on different dates and the reference light brightness; Whether the time period is a feature comparison time period is determined by the proportion of the number of light change dates.

[0014] A further technical solution is that when the proportion of the number of light change dates is less than the preset proportion, the time period is determined to be a feature comparison time period.

[0015] A further technical solution is that the light change date is a date whose deviation from the reference light brightness is not within a preset deviation range.

[0016] A further technical solution is to determine whether to issue an early warning signal, specifically including: Using the matching results of the feature points of the comparison images in different time periods, using the distance function to determine the feature point similarity coefficients of different feature points of the comparison images, and using the feature point similarity coefficients to match the feature points of different feature points of the comparison images; A homography matrix is ​​constructed based on the matching feature points, geometric transformation information is extracted based on the homography matrix to obtain a rotation angle, a scaling ratio, and a translation amount, and a preset threshold is combined to determine whether the image device is offset.

[0017] A further technical solution is that when any one of the rotation angle, zoom ratio and translation amount in multiple time periods does not meet the requirements of a preset threshold, it is determined that the image device is offset and an early warning signal is output.

[0018] Other features and advantages will be described in the following description. The objectives and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description and the drawings.

[0019] In order to make the above-mentioned objectives, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other features and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the accompanying drawings.

[0021] Figure 1 It is a framework diagram of an abnormal early warning system for image device displacement based on feature point detection; Figure 2 It is a flow chart of an abnormal early warning method for image device displacement based on feature point detection; Figure 3 is a flow chart of a method for determining a feature comparison period in a period; Figure 4 is a flow chart of a method for determining feature points of a comparison image in a feature comparison period; Figure 5 It is a flow chart for determining a suspected change of an imaging device. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0023] With the popularization of intelligent devices and Internet of Things technology, image acquisition equipment (such as cameras) are widely used in monitoring, navigation, intelligent manufacturing and other fields. However, during operation, the acquisition equipment may be displaced from the installation position due to external interference (such as vibration, collision or manual adjustment), which may affect the accuracy of image acquisition and even cause system abnormalities. Therefore, there is an urgent need for an efficient and reliable method to dynamically monitor the installation status of the image acquisition equipment and issue a warning in time when the displacement is abnormal. In the existing technology, there is still a lack of a simple, easy-to-use, efficient and robust solution.

[0024] The present invention proposes a method for monitoring the installation displacement of image equipment and abnormal warning based on ORB feature detection. By analyzing the geometric transformation relationship (rotation angle, scaling ratio and translation vector) between the image acquisition device in the initial installation state and the image in the current operating state, it is determined whether the device is offset. When the transformation information exceeds the set threshold, abnormal warning information is generated.

[0025] The implementation steps of this method include image preprocessing, feature extraction and matching, geometric transformation calculation, offset threshold judgment and warning output. By superimposing the offset information text on the matching image, the device status is intuitively displayed, and automatic alarm output is supported to meet the needs of intelligent operation and maintenance of the device.

[0026] 1. Initial image acquisition and preprocessing After the device is installed, use the camera to capture its initial state image and store it as a reference image.

[0027] The subsequently acquired images are grayed out to improve the computational efficiency and robustness of ORB feature extraction and matching.

[0028] Grayscale processing: Convert a color image to a grayscale image to simplify subsequent calculations by reducing color information. The calculation formula is as follows: in: Igray: Gray value after conversion.

[0029] R,G,B: The red, green, and blue channel values ​​of each pixel in the original image.

[0030] Weights 0.299, 0.587, 0.114: weighted values ​​that simulate the sensitivity of the human eye to different colors.

[0031] Feature point detection and descriptor calculation The ORB algorithm is used to detect feature points and calculate descriptors for the reference image and the current captured image. The ORB algorithm combines the advantages of FAST corner point detection and BRIEF descriptor calculation, has high real-time performance and stability, and is suitable for dynamic monitoring scenarios.

[0032] The ORB feature detection process consists of two parts: FAST Corner Detection: Quickly identify corner points in an image.

[0033] FAST (Features from Accelerated Segment Test) is a fast corner detection algorithm. The detection process is as follows: Set the pixel point p and its neighborhood with radius r.

[0034] There are 16 pixels in the neighborhood: if at least N pixels have brightness significantly greater than (or less than) that of the center point p, then p is considered a corner point.

[0035] The judgment formula for N pixels: satisfying the condition I(q)>I(p)+δ or I(q) <I(p)−δ in: I(p): Gray value of the center point.

[0036] I(q): Gray value of the neighborhood pixel.

[0037] δ: The set threshold.

[0038] BRIEF descriptor calculation: Generate a short but unique binary descriptor to identify corner points. BRIEF (BinaryRobust Independent Elementary Features) is a binary descriptor that uses simple pixel intensity comparison for feature description. The generation process is as follows: Define a fixed-size image patch around the feature point (usually an S×S area) Pick a random pair (pi,qi): Generate a binary string with the following formula: in: pi,qi: randomly selected pixel pairs.

[0039] I(pi),I(qi): Grayscale value of pixel.

[0040] Binary descriptor: A string of 0s and 1s that serves as the feature vector of a feature point.

[0041] ORB adds rotation invariance when detecting feature points, so that it can maintain high detection accuracy even when the angle is offset.

[0042] Feature point matching and optimization BFMatcher (Brute-Force Matcher) is used to match the feature descriptors of the reference image and the current image. The matching results are sorted by distance and the best 50 matching points are selected to reduce noise interference and improve calculation stability.

[0043] BFMatcher is a brute force matching algorithm used to match feature point descriptors. The matching process is as follows: Compute the distance between two descriptors.

[0044] Find the matching pair with the smallest distance as the matching result of the feature points, where the distance is calculated using the Hamming distance between descriptors, and the calculation formula is as follows: in: d1,d2: two binary descriptors.

[0045] n: length of the descriptor (usually 256 bits).

[0046] ⊕: Bitwise XOR operation, calculate whether each bit is different.

[0047] The matching distance is calculated using the Hamming distance to measure the similarity of descriptors.

[0048] Hamming distance is used to compare the differences between two binary strings. The formula is as follows: in: A, B: two binary strings.

[0049] n: The number of bits in the binary string.

[0050] ⊕: Bitwise XOR operation.

[0051] Homography matrix calculation Based on the matched feature points, the homography matrix between images is calculated using the RANSAC (Random Sample Consensus) algorithm. During the calculation process, RANSAC randomly selects a set of matching points, solves the homography matrix, and improves the robustness of the calculation by excluding outliers.

[0052] The homography matrix H represents the perspective transformation relationship between two images and is defined as: in: H: Matrix decomposition can be used to extract rotation, scaling and translation information; x′, y′: transformed image coordinates (coordinates of the point on the target image); x, y: image coordinates before transformation (coordinates of the point on the reference image); w′: Normalization factor used to convert homogeneous coordinates to Cartesian coordinates.

[0053] Geometric transformation information extraction Extract the first two rows and two columns of the homography matrix to form the rotation matrix R:

[0054] in: R: Rotation matrix, describing the rotation state of the image.

[0055] hij: Rotation-related components of the homography matrix.

[0056] Rotation angle: Calculate the rotation angle θ through the rotation matrix: in: θ: rotation angle, in radians (can be converted to degrees later).

[0057] atan2(h21,h11): The inverse tangent function used to calculate the angle, taking the quadrant into account.

[0058] Scaling: Obtain Sx and Sy by taking the row vector norm of the rotation matrix: in: sx: scaling factor in the x direction sy: scaling factor in the y direction.

[0059] Translation vector: Extract the translation vector T from the homography matrix: in: T: Translation vector, indicating the translation state of the image.

[0060] h 13 ,h 23 : Parameters describing the translation in the homography matrix.

[0061] 6. Offset threshold determination To set a predefined threshold range: Rotation angle threshold: within 5 degrees is the normal range.

[0062] Scaling ratio range: For example, 0.95 to 1.05 means no significant scaling.

[0063] Translation threshold: If the translation amount is within 10 pixels, it is normal.

[0064] Based on the extracted information, the current calculated value is compared with the threshold to determine whether it exceeds the set range to detect whether the device is offset.

[0065] Alarm and visual output If the device status is abnormal, an early warning message is generated and detailed offset parameters such as rotation angle, scaling ratio and translation vector are output. Transformation information text is superimposed on the matching image to make the offset situation intuitive. The operation and maintenance personnel are notified in real time through printing alarm prompts, sound reminders or message push notifications on the console.

[0066] Embodiment 1 To solve the above problem, according to one aspect of the present invention, Figure 1 As shown, a system for early warning of abnormal displacement of an image device based on feature point detection is provided, which specifically includes: Feature point screening module, matrix calculation and information extraction module, displacement recognition and early warning module; The feature point screening module is responsible for detecting feature points and calculating descriptors of the reference image and the current captured image using the ORB algorithm, and obtaining image feature points for image comparison by matching the feature descriptors of the reference image and the current image; The matrix calculation and information extraction module is responsible for determining the feature point similarity coefficients of different comparison image feature points using the distance function, and matching feature points of different comparison image feature points using the feature point similarity coefficients, and constructing a homography matrix based on the matching feature points; The displacement identification and warning module is responsible for extracting geometric transformation information based on the homography matrix to obtain the rotation angle, scaling ratio and translation amount, and determines whether the image device is offset and issues a warning signal in combination with a preset threshold.

[0067] Furthermore, the feature point similarity coefficient is determined according to a Hamming distance function.

[0068] Specifically, when any one of the rotation angle, the scaling ratio, and the translation amount is greater than a preset threshold, it is determined that the image device is offset and a warning signal is issued.

[0069] Embodiment 2 In the second aspect, as Figure 2 As shown, the present application provides an image device displacement abnormality warning method based on feature point detection, which is applied to the above-mentioned image device displacement abnormality warning system based on feature point detection, and specifically includes: S1 determines the change of light data in different time periods on different dates based on the analysis results of the monitoring images of the imaging device, and determines the feature comparison time period in the time period based on the change; Furthermore, the change of the light data is determined according to the brightness of the monitoring image of the image device in the time period.

[0070] Specifically, the light data is determined according to a preset light brightness corresponding to the brightness of the monitoring image.

[0071] Specifically, Figure 3 As shown, the method for determining the feature comparison period in the period is: Determine the light brightness of the time periods on different dates by using the light data of the time periods on different dates, and determine the reference light brightness by using the average light brightness of the time periods on different dates; Determining the light change date among the dates according to the deviation between the light brightness on different dates and the reference light brightness; Whether the time period is a feature comparison time period is determined by the number ratio of the light change dates.

[0072] Furthermore, when the proportion of the number of light change dates is less than the preset proportion, the time period is determined to be a feature comparison time period.

[0073] It can be understood that the light change date is a date whose deviation from the reference light brightness is not within a preset deviation range.

[0074] Optionally, the method for determining the feature comparison period in the period is: Determine the light brightness of the time periods on different dates by using the light data of the time periods on different dates, and determine the reference light brightness by using the average light brightness of the time periods on different dates; According to the deviations of the light brightness on different dates from the reference light brightness, averaging the deviations on different dates; Whether the period is a feature comparison period is determined by the average value of the deviations on different dates.

[0075] Specifically, when the average value of the deviations on different dates does not meet the requirement, the period is determined as the feature comparison period.

[0076] In another embodiment, the method for determining the feature comparison period in the period is: S11: determining the light brightness of the time periods on different dates by using the light data of the time periods on different dates, and determining the reference light brightness by using the average value of the light brightness of the time periods on different dates; S12 determines the light change date in the date according to the deviation between the light brightness in different dates and the reference light brightness; S13 determines the feature point comparison coefficient of the time period according to the number ratio of the light change dates and the reference light brightness, and uses the feature point comparison coefficient to determine whether the time period is a feature comparison time period.

[0077] Specifically, the method for determining the feature point comparison coefficient of the time period is: Based on the reference light brightness, determining a preset comparison coefficient corresponding to the reference light brightness; The characteristic point ratio coefficient of the time period is determined according to the product of the proportion of the number of light change dates and the preset comparison coefficient.

[0078] Furthermore, the value range of the feature point comparison coefficient is between 0 and 1, and when the feature point comparison coefficient of the time period is less than a preset comparison coefficient threshold, the time period is determined to be a feature comparison time period.

[0079] Optionally, the above step S11 includes the following contents: S111 determines the light brightness of the time periods on different dates by using the light data of the time periods on different dates, and determines the reference light brightness by taking the average value of the light brightness of the time periods on different dates. When the reference light brightness is less than a preset brightness threshold, it is determined that the time period does not belong to the feature comparison time period. When the reference light brightness is not less than the preset brightness threshold, the process proceeds to step S112. S112: when determining the date when the light brightness is less than the preset brightness threshold based on the light brightness of the time periods on different dates, the process proceeds to step S112; when there is no date when the light brightness is less than the preset brightness threshold, the process proceeds to step S12; S113: When the number of dates with light brightness lower than the preset brightness threshold is greater than the preset number of dates, it is determined that the period does not belong to the feature comparison period; when the number of dates with light brightness lower than the preset brightness threshold is not greater than the preset number of dates, proceed to step S12.

[0080] Optionally, the above step S12 includes the following contents: S121: if it is determined that there is no light change date in the date according to the deviation between the light brightness in different dates and the reference light brightness, then the process proceeds to S122; if there is a light change date in the date, then the process proceeds to S123; S122: when the average value of the deviations between the light brightness on different dates and the reference light brightness is less than a preset deviation threshold, it is determined that the period belongs to the feature comparison period; when the average value of the deviations between the light brightness on different dates and the reference light brightness is not less than the preset deviation threshold, it is determined that the period does not belong to the feature comparison period; S123: obtaining the number of light change dates. When the number of light change dates is greater than the preset number of change dates, determining that the period does not belong to the feature comparison period. When the number of light change dates is not greater than the preset number of change dates, proceeding to step S124. S124 determines the comprehensive deviation coefficient based on the proportion of the number of light change dates and the deviation of different light change dates. When the comprehensive deviation coefficient does not meet the requirements, it is determined that the time period does not belong to the feature comparison time period. When the comprehensive deviation coefficient meets the requirements, proceed to step S13.

[0081] S2: obtaining changes in different ORB image feature points and initial image feature points in the monitoring image during the feature comparison period, and determining the credibility coefficients of the comparison image feature points and different comparison image feature points during the feature comparison period by using the changes in the image feature points; Furthermore, the change of the ORB image feature points and the initial image feature points is determined according to the deviation between the ORB image feature points and the initial image feature points in the monitoring images corresponding to different dates.

[0082] Specifically, the initial image feature points are image feature points of a reference image corresponding to the image device when it is installed.

[0083] Specifically, Figure 4 As shown, the method for determining the feature points of the comparison image in the feature comparison period is: Determine the similarity coefficients between the ORB feature points and the initial image feature points on different dates based on the changes between the ORB image feature points and the initial image feature points; Based on the similarity coefficients with the feature points of the initial image, determining an average value of the similarity coefficients on different dates; The credibility coefficient of the ORB image feature point is determined according to the average value of the similarity coefficients on different dates, and the credibility coefficient is used to determine whether the ORB feature point is a comparison image feature point.

[0084] Further, using the credibility coefficient to determine whether the ORB feature point is a comparison image feature point specifically includes: The ORB feature points whose credibility coefficients meet the requirements are used as the feature points of the comparison image.

[0085] In another embodiment, the method for determining the feature points of the comparison image in the feature comparison period is: Determine the similarity coefficients between the ORB feature points and the initial image feature points on different dates based on the changes between the ORB image feature points and the initial image feature points; Determining similar dates among the dates based on similarity coefficients with the feature points of the initial image; The credibility coefficient of the ORB image feature point is determined according to the quantity ratio of the similar dates, and the credibility coefficient is used to determine whether the ORB feature point is a comparison image feature point.

[0086] Furthermore, the similar date is a date whose similarity coefficient with the initial image feature point is greater than a preset similarity coefficient.

[0087] Optionally, the method for determining the feature points of the comparison image in the feature comparison period is: S11 determines the similarity coefficients between the ORB feature points and the initial image feature points on different dates based on the changes between the ORB image feature points and the initial image feature points; S12, based on the similarity coefficient with the feature point of the initial image, determining similar dates among the dates, and determining weight coefficients of different similar dates using the similarity coefficients of different similar dates; S13 determines the credibility coefficient of the ORB image feature point according to the quantity ratio of the similar dates and the average value of the weight coefficients of different similar dates, and determines whether the ORB feature point is a comparison image feature point by using the credibility coefficient.

[0088] Specifically, the method for determining the credibility coefficient of the ORB image feature point is: The average weight coefficient is determined by averaging the weight coefficients of different similar dates; The credibility coefficient of the ORB image feature point is determined based on the product of the quantity ratio of the similar dates and the average weight coefficient.

[0089] Optionally, the above step S11 includes the following contents: S111 determines the similarity coefficients between the ORB image feature points and the initial image feature points on different dates based on the changes between the ORB image feature points and the initial image feature points. When there is a date where the similarity coefficient does not meet the requirement, it is determined that the ORB image feature points do not belong to the comparison image feature points. When there is no date where the similarity coefficient does not meet the requirement, the process proceeds to step S112. S112: determining the average value of the similarity coefficients of different dates based on the similarity coefficients of the ORB feature points on different dates and the initial image feature points; when the average value of the similarity coefficients of different dates is less than a preset similarity coefficient threshold, determining that the ORB image feature point does not belong to the comparison image feature point; and when the average value of the similarity coefficients of different dates is not less than the preset similarity coefficient threshold, proceeding to step S113; S113 uses the date whose similarity coefficient is within the preset similarity coefficient range as the screening deviation date. When the number ratio of the screening deviation dates does not meet the requirement, it is determined that the ORB image feature point does not belong to the comparison image feature point. When the average value of the similarity coefficients of different dates is not less than the preset similarity coefficient threshold and the number ratio of the screening deviation dates meets the requirement, it proceeds to step S12.

[0090] Optionally, the above step S12 includes the following contents: Based on the similarity coefficient with the feature point of the initial image, similar dates among the dates are determined, and weight coefficients of different similar dates are determined by using the similarity coefficients of different similar dates. S121: when it is determined based on the similarity coefficient with the initial image feature point that there is no similar date in the date, it is determined that the ORB image feature point does not belong to the comparison image feature point; when there is a similar date in the date, the process proceeds to step S122; S122: obtaining the number ratio of similar dates in the date; when the number ratio of the similar dates is less than the preset number ratio, determining that the ORB image feature point does not belong to the comparison image feature point; when the number ratio of the similar dates is not less than the preset number ratio, proceeding to step S123; S123 uses the similarity coefficients of different similar dates to determine the weight coefficients of different similar dates. When the sum of the weight coefficients of different similar dates is less than the preset weight coefficient threshold, it is determined that the ORB image feature point does not belong to the comparison image feature point. When the sum of the weight coefficients of different similar dates is not less than the preset weight coefficient threshold, it proceeds to step S13.

[0091] S3: obtaining the change of the feature points of the compared image and the feature points of the initial image during the feature comparison period, and combining the credibility coefficients of different compared image feature points to determine that the image device has a suspected change, and then proceeding to the next step; Specifically, Figure 5 As shown, determining that a suspected change has occurred in the image device specifically includes: Determine the deviations of different compared image feature points from the initial image feature points based on the changes of the compared image feature points and the initial image feature points, and determine the deviation coefficients of the compared image feature points based on the deviations; Determine the sum of deviation coefficients according to the sum of the products of deviation coefficients and credibility coefficients of different comparison image feature points; Based on the deviation coefficient, it is determined whether a suspected change occurs in the image device.

[0092] Furthermore, the deviation coefficient of the feature point of the comparison image is determined according to the product of the deviation amount and a preset scale factor.

[0093] It should be noted that when the sum of the deviation coefficients is greater than a preset deviation coefficient threshold, it is determined that a suspected change has occurred in the imaging device.

[0094] Optionally, determining that a suspected change occurs to the image device specifically includes: Determine the deviations of different compared image feature points from the initial image feature points based on the changes of the compared image feature points and the initial image feature points, and when the deviations of different compared image feature points from the initial image feature points are all within a preset deviation range, determine that no suspected changes have occurred in the image device; When there is a comparison image feature point whose deviation from the initial image feature point is not within the preset deviation range: The compared image feature points whose deviations from the initial image feature points are not within the preset deviation range are used as deviation image feature points, and when the number of the deviation image feature points is less than the preset number threshold: Obtaining an average value of deviations between different compared image feature points and the initial image feature points, and when the average value of deviations between different compared image feature points and the initial image feature points meets the requirement, determining that a suspected change occurs to the image device; When the average value of the deviations between different comparison image feature points and the initial image feature points does not meet the requirement or the number of the deviation image feature points is not less than the preset number threshold: When the number of the deviation image feature points is greater than a preset feature point number threshold, it is determined that the image device has a suspected change; When the number of feature points of the deviation image is not greater than the preset feature point number threshold: Based on the credibility coefficients of different deviation image feature points, a credibility coefficient sum is determined, and when the credibility coefficient sum is greater than a preset credibility coefficient threshold, it is determined that a suspected change has occurred in the image device; When the sum of the credibility coefficients is not greater than the preset credibility coefficient threshold: Determine the deviation coefficient of the comparison image feature point by using the deviation amount, determine different correction deviation coefficients of the comparison image feature points according to the product of the deviation coefficients of different comparison image feature points and the credibility coefficient, and when there is no comparison image feature point whose correction deviation coefficient does not meet the requirement, determine that the image device has not undergone suspected changes; When there are feature points in the comparison image whose correction deviation coefficient does not meet the requirements: When the number of the compared image feature points whose correction deviation coefficients do not meet the requirements is greater than a preset deviation feature point number threshold, it is determined that the image device has a suspected change; When the number of feature points of the compared images for which the correction deviation coefficient does not meet the requirement is not greater than the preset deviation feature point number threshold: The change probability is determined based on the correction deviation coefficients of different comparison image feature points, and the change probability is used to determine whether the image device has a suspected change.

[0095] S4 determines whether to issue a warning signal based on changes in the feature points of the compared images in different time periods.

[0096] Specifically, determine whether to issue an early warning signal, including: Using the matching results of the feature points of the comparison images in different time periods, using the distance function to determine the feature point similarity coefficients of different feature points of the comparison images, and using the feature point similarity coefficients to match the feature points of different feature points of the comparison images; A homography matrix is ​​constructed based on the matching feature points, geometric transformation information is extracted based on the homography matrix to obtain a rotation angle, a scaling ratio, and a translation amount, and a preset threshold is combined to determine whether the image device is offset.

[0097] Furthermore, when any one of the rotation angle, the scaling ratio, and the translation amount in multiple time periods does not meet the requirement of a preset threshold, it is determined that the image device is offset, and an early warning signal is output.

[0098] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0099] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0100] The above description is only one or more embodiments of this specification and is not intended to limit this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of this specification.

Claims

1. An image device displacement abnormality warning system based on feature point detection, characterized in that: Specifically include: Feature point screening module, matrix calculation and information extraction module, displacement recognition and early warning module; The feature point screening module is responsible for detecting feature points and calculating descriptors of the reference image and the current captured image using the ORB algorithm, and obtaining image feature points for image comparison by matching the feature descriptors of the reference image and the current image; The matrix calculation and information extraction module is responsible for determining the feature point similarity coefficients of different comparison image feature points using the distance function, and matching feature points of different comparison image feature points using the feature point similarity coefficients, and constructing a homography matrix based on the matching feature points; The displacement identification and warning module is responsible for extracting geometric transformation information based on the homography matrix to obtain the rotation angle, scaling ratio and translation amount, and determines whether the image device is offset and issues a warning signal in combination with a preset threshold.

2. The image device displacement abnormality warning system based on feature point detection as described in claim 1, wherein the feature point similarity coefficient is determined according to a Hamming distance function.

3. The image device displacement abnormality warning system based on feature point detection as described in claim 1, wherein when any one of the rotation angle, scaling ratio and translation amount is greater than a preset threshold, it is determined that the image device is offset and a warning signal is issued.

4. A method for warning abnormal displacement of an image device based on feature point detection, applied to a warning system for warning abnormal displacement of an image device based on feature point detection as claimed in any one of claims 1 to 3, characterized in that: Specifically include: Determine the change of light data in different time periods on different dates based on the analysis results of the monitoring images of the imaging device, and use the change to determine the feature comparison period in the time period; Obtaining changes in different ORB image feature points and initial image feature points in the monitoring image during the feature comparison period, and determining the credibility coefficients of the comparison image feature points and different comparison image feature points during the feature comparison period using the changes in the image feature points; Obtaining the change of the feature points of the compared image and the feature points of the initial image during the feature comparison period, and combining the credibility coefficients of different compared image feature points to determine that the image device has a suspected change, and then proceeding to the next step; Determine whether to issue a warning signal based on the changes in the feature points of the compared images in different time periods.

5. The method for warning abnormal displacement of an image device based on feature point detection as described in claim 4 is characterized in that the change of the light data is determined based on the brightness of the monitoring image of the image device in the time period.

6. The method for warning abnormal displacement of an image device based on feature point detection as described in claim 4, wherein the light data is determined based on a preset light brightness corresponding to the brightness of the monitored image.

7. The method for early warning of abnormal displacement of an image device based on feature point detection according to claim 4, wherein the method for determining the feature comparison period in the period is: Determine the light brightness of the time periods on different dates by using the light data of the time periods on different dates, and determine the reference light brightness by using the average light brightness of the time periods on different dates; Determining the light change date among the dates according to the deviation between the light brightness on different dates and the reference light brightness; Whether the time period is a feature comparison time period is determined by the number ratio of the light change dates.

8. The image device displacement abnormality warning method based on feature point detection as described in claim 7, wherein the characteristic point is that when the proportion of the number of light change dates is less than the preset proportion, the time period is determined to be a feature comparison period.

9. The method for early warning of abnormal displacement of an image device based on feature point detection as claimed in claim 4, wherein the method comprises: determining whether to issue an early warning signal; and Using the matching results of the feature points of the comparison images in different time periods, using the distance function to determine the feature point similarity coefficients of different feature points of the comparison images, and using the feature point similarity coefficients to match the feature points of different feature points of the comparison images; A homography matrix is ​​constructed based on the matching feature points, geometric transformation information is extracted based on the homography matrix to obtain a rotation angle, a scaling ratio, and a translation amount, and a preset threshold is combined to determine whether the image device is offset.

10. The method for warning abnormal displacement of an image device based on feature point detection as described in claim 9 has the characteristic point that when any one of the rotation angle, zoom ratio and translation amount in multiple time periods does not meet the requirements of a preset threshold, it is determined that the image device is offset and an early warning signal is output.

Images