Target object displacement monitoring method and device, computer device and storage medium
By acquiring the displacement of the target object through image processing technology and using Gaussian filtering and SIFT feature point matching, the problems of low accuracy and large environmental interference in infrastructure monitoring are solved, and high-precision, interference-resistant displacement monitoring is achieved.
Patent Information
- Application Number
- CN202210004925.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-04
- Publication Date
- 2025-12-23
- Estimated Expiration
- 2042-01-04
AI Technical Summary
Existing technologies for infrastructure monitoring suffer from low accuracy, susceptibility to environmental interference, and poor applicability. In particular, it is difficult to achieve high accuracy and strong anti-interference capabilities for displacement monitoring of facilities such as bridges, ancient buildings, slopes, track deformation, tunnels, foundation pits, houses, dams, ponds, and reservoirs.
By acquiring images of the target object, preprocessing template and matching images, extracting and matching feature points, calculating pixel displacement distance, and combining the ratio between pixel size and physical size, the actual displacement distance of the target object is determined. Gaussian filtering and SIFT feature point matching technology are used to reduce environmental deployment requirements and external interference.
It achieves high-precision displacement monitoring, has strong anti-interference capabilities, low maintenance costs, no strict requirements for environmental deployment, high monitoring accuracy, and is less affected by external factors.
Smart Images
Figure CN114387306B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to displacement monitoring methods, in particular to a target object displacement monitoring method and device, computer equipment and a storage medium. BACKGROUND
[0002] Worldwide, due to the influence of climate and geological changes, some major infrastructure has been severely damaged, such as bridges, ancient buildings, slopes, track deformation, tunnels, foundation pit monitoring, houses, dams, ponds, reservoirs, etc. If these changes in infrastructure cannot be discovered and understood in time, it will cause the destruction of infrastructure and cause major consequences such as loss of life and property.
[0003] The main monitoring methods for infrastructure include the following three kinds, the first is to use a total station for monitoring, the total station is a contact monitoring method, which needs to install and arrange sensors on the monitored object in a good view, undisturbed, stable and reliable position, this method will cause damage to the monitored object itself, has poor applicability, small application range, harsh monitoring conditions, and large interference, in addition, the monitoring precision is generally millimeter level, and the precision is poor; The second is to use GNSS (Global Navigation Satellite System) for monitoring, GNSS automatic monitoring collects GNSS signals of each monitoring point and reference point, analyzes the real-time three-dimensional coordinates of each monitoring point, and compares them with the initial coordinates to obtain the change of the monitoring point. This method has low monitoring precision and is greatly affected by signal factors; The third is to use artificial targets for monitoring based on vision, which is a non-contact monitoring method. By designing artificial targets, the target information is converted into image signals by a camera, and the displacement between the template and the target is calculated by using image processing algorithms to obtain the change of the monitored target. This method has high flexibility and automation, and is often used in dangerous places where manual operation is not suitable. The main disadvantage of this method is that the artificial target needs to be installed in advance, and the target is greatly affected by environmental factors.
[0004] Therefore, it is necessary to design a new method to achieve high monitoring precision, no strict requirement for environmental deployment, and small external interference factors. SUMMARY
[0005] The present application aims to overcome the defects of the prior art and provide a target object displacement monitoring method, device, computer equipment and storage medium.
[0006] To achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows: a target object displacement monitoring method, comprising:
[0007] Obtaining an image of the monitored target object to obtain a template image;
[0008] acquire a matching image according to the template image;
[0009] preprocess the template image and the matching image to obtain a processed template image and a processed matching image;
[0010] calculate a proportional relationship between a pixel size and a physical size according to the processed template image;
[0011] extract feature points of the processed template image and feature points of the processed matching image, and match the feature points of the processed template image and the feature points of the processed matching image to obtain a matching result;
[0012] calculate a pixel displacement distance of the target object according to the matching result;
[0013] determine an actual displacement distance of the target object according to the pixel displacement distance of the target object and the proportional relationship between the pixel size and the physical size.
[0014] Further technical solutions of the present application are as follows:
[0015] acquire an image corresponding to a region at the same position as the template image to obtain the matching image.
[0016] Further technical solutions of the present application are as follows:
[0017] perform Gaussian filtering processing on the template image and the matching image to obtain a denoised template image and a denoised matching image;
[0018] perform gray scale processing on the denoised template image and the denoised matching image to obtain the processed template image and the processed matching image.
[0019] Further technical solutions of the present application are as follows:
[0020] acquire an actual size of an inner rectangular region of a measurement target by placing the measurement target in a template region corresponding to the processed template image;
[0021] cut an image containing the entire inner rectangular region in a circumscribed rectangular region of the measurement target to obtain a region image;
[0022] calculate an average gray scale value of the region image;
[0023] calculate a binary image of the region image;
[0024] calculate the contour of the region image, determine the minimum circumscribed rectangle of the contour, and determine the pixel length and width of the minimum circumscribed rectangle;
[0025] calculate the actual size corresponding to one pixel in the x direction and the y direction of the region image, to obtain the proportional relationship between the pixel size and the physical size.
[0026] Further technical solutions thereof are as follows: the feature points of the processed template image and the feature points of the processed matching image are extracted, and the feature points of the processed template image and the feature points of the processed matching image are matched to obtain a matching result, comprising:
[0027] sift feature points are extracted from the processed template image and the processed matching image to obtain template image feature points and matching image feature points;
[0028] feature vectors of the template image feature points and the matching image feature points are calculated and normalized to obtain processed template image feature points and processed matching image feature points;
[0029] the processed template image feature points and the processed matching image feature points are matched, and abnormal feature points are eliminated to obtain a matching result.
[0030] Further technical solutions thereof are as follows: the pixel displacement distance of the target object is calculated according to the matching result, comprising:
[0031] the difference between the coordinates of two feature points in the matching result is calculated to obtain an x coordinate offset and a y coordinate offset;
[0032] the x coordinate offset and the y coordinate offset are sorted according to their absolute values from small to large, and coordinate offsets meeting the requirements are screened to obtain target coordinate offsets;
[0033] the x direction pixel offset and the y direction pixel offset are calculated according to the target coordinate offsets to obtain the pixel displacement distance of the target object.
[0034] Further technical solutions thereof are as follows: the actual displacement distance of the target object is determined according to the pixel displacement distance of the target object and the proportional relationship between the pixel size and the physical size, comprising:
[0035] the product of the pixel displacement distance of the target object and the proportional relationship between the pixel size and the physical size is calculated to obtain the actual displacement distance of the target object.
[0036] The application also provides a target object displacement monitoring device, comprising:
[0037] A template image acquisition unit is configured to acquire an image of a monitored target object to obtain a template image.
[0038] A matching image generation unit is configured to acquire a matching image according to the template image.
[0039] A preprocessing unit is configured to pre-process the template image and the matching image to obtain a processed template image and a processed matching image.
[0040] A scale relationship calculation unit is configured to calculate a scale relationship between a pixel size and a physical size according to the processed template image.
[0041] A matching unit is configured to extract feature points of the processed template image and feature points of the processed matching image, and match the feature points of the processed template image and the feature points of the processed matching image to obtain a matching result.
[0042] A pixel displacement calculation unit is configured to calculate a pixel displacement distance of the target object according to the matching result.
[0043] An actual displacement calculation unit is configured to determine an actual displacement distance of the target object according to the pixel displacement distance of the target object and the scale relationship between the pixel size and the physical size.
[0044] The present application also provides a computer device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the above method when executing the computer program.
[0045] The present application also provides a storage medium, which stores a computer program, and the computer program implements the above method when executed by a processor.
[0046] Compared with the prior art, the present application has the following beneficial effects: the present application sets a template image, generates a matching image, pre-processes the template image and the matching image, reduces the requirement for environmental deployment in the monitoring process, and is not easily affected by external interference factors; the scale relationship between the pixel size and the physical size is determined for the processed template image, and the target object pixel displacement distance is determined by extracting and matching the target sift features of the two images, and then the actual displacement distance of the target object is determined; the entire method has strong anti-interference, high precision, small influence of external environment, and low maintenance cost, so as to realize high monitoring precision, no strict requirement for environmental deployment, and small influence of external interference factors.
[0047] The present application will be further described below in combination with the drawings and specific embodiments. BRIEF DESCRIPTION OF DRAWINGS
[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.
[0049] Figure 1 The application scenario diagram of the target object displacement monitoring method provided by the embodiments of the present application is shown.
[0050] Figure 2 The flowchart of the target object displacement monitoring method provided by the embodiments of the present application is shown.
[0051] Figure 3 The sub-flowchart of the target object displacement monitoring method provided by the embodiments of the present application is shown.
[0052] Figure 4 The sub-flowchart of the target object displacement monitoring method provided by the embodiments of the present application is shown.
[0053] Figure 5 The sub-flowchart of the target object displacement monitoring method provided by the embodiments of the present application is shown.
[0054] Figure 6 The sub-flowchart of the target object displacement monitoring method provided by the embodiments of the present application is shown.
[0055] Figure 7 The schematic block diagram of the target object displacement monitoring device provided by the embodiments of the present application is shown.
[0056] Figure 8 The schematic block diagram of the preprocessing unit of the target object displacement monitoring device provided by the embodiments of the present application is shown.
[0057] Figure 9 The schematic block diagram of the proportional relationship calculation unit of the target object displacement monitoring device provided by the embodiments of the present application is shown.
[0058] Figure 10 The schematic block diagram of the matching unit of the target object displacement monitoring device provided by the embodiments of the present application is shown.
[0059] Figure 11 The schematic block diagram of the pixel displacement calculation unit of the target object displacement monitoring device provided by the embodiments of the present application is shown.
[0060] Figure 12 The schematic block diagram of the computer device provided by the embodiments of the present application is shown. DETAILED DESCRIPTION
[0061] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative efforts belong to the scope of the present application.
[0062] It should be understood that the terms "comprising" and "including" as used in the specification and the appended claims indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.
[0063] It should also be understood that the terms used in the present application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise.
[0064] It should be further understood that the term "and / or" used in the present application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations thereof, and includes these combinations.
[0065] Please refer to Figure 1 and Figure 2 , Figure 1 The application scenario diagram of the target object displacement monitoring method provided by the embodiments of the present application. Figure 2 The schematic flowchart of the target object displacement monitoring method provided by the embodiments of the present application. The target object displacement monitoring method is applied to a server. The server interacts with a camera, captures an image of the monitored target object through the camera as a template image, extracts a matching image from the template image, calculates the proportional relationship of pixel size and physical size and matches feature points from the two images, thereby determining the actual displacement distance of the target object.
[0066] Figure 2 The flowchart of the target object displacement monitoring method provided by the embodiments of the present application. As shown in Figure 2 , the method comprises the following steps S110 to S170.
[0067] S110, acquiring an image of the monitored target object to obtain a template image.
[0068] In the embodiment, the template image is obtained by a camera to monitor the image of the target object, and a rectangular region corresponding to a stable, flat and environment-unchanged region in the image region is selected.
[0069] The template image is used to record the initial state of the target object, and the position in the image coordinate system is the initial position of the target in the image coordinate system. The displacement change of the template image region is used to replace the displacement change of the entire target object. A stable, flat and environment-unchanged region is selected as the template image on the image by using a program and a mouse, and the position information of the template image in the image coordinate system is calculated and recorded as (x0, y0, width, height).
[0070] S120, obtaining a matching image according to the template image.
[0071] In the embodiment, the matching image refers to an image corresponding to the region at the same position as the template image.
[0072] Specifically, an image corresponding to the region at the same position as the template image is obtained to obtain the matching image.
[0073] The matching image is used to record the subsequent state of the target object, and the region (x0, y0, width, height) in the image coordinate system is selected as the matching image in the subsequent image frame. The displacement difference between the positions of the template image and the matching image is calculated, and the displacement difference of the target object can be known.
[0074] S130, pre-processing the template image and the matching image to obtain a processed template image and a processed matching image.
[0075] In the embodiment, the processed template image refers to the result obtained after the template image is de-noised and gray scaled; and the processed matching image refers to the result obtained after the matching image is de-noised and gray scaled.
[0076] In an embodiment, referring to Figure 3 The above step S130 can include steps S131-S132.
[0077] S131, performing Gaussian filtering processing on the template image and the matching image to obtain a de-noised template image and a de-noised matching image.
[0078] In the embodiment, the filtering processing on the template image and the matching image can make the template image and the matching image smoother.
[0079] Specifically, the filtering process is implemented by an image Gaussian filtering function, and the image Gaussian filtering function is: wherein σ = 1, x, y represent the coordinates of the image pixel points respectively.
[0080] S132, performing gray scale processing on the denoised template image and the denoised matching image to obtain a processed template image and a processed matching image.
[0081] In the embodiment, performing gray scale processing on the denoised template image and the denoised matching image can reduce the influence of illumination change.
[0082] The gray scale of the image is I(x, y), and the gray scale of the processed image is I'(x, y), then: I'(x, y) = [I(x, y) * 0.1] * 10; wherein [] represents an integer.
[0083] S140, calculating the proportional relationship between the pixel size and the physical size according to the processed template image.
[0084] In the embodiment, the proportional relationship between the pixel size and the physical size refers to the actual size corresponding to one pixel.
[0085] The proportional relationship between the pixel and the actual size is obtained by measuring the actual size of the measurement target on the region where the measurement target is located and calculating the corresponding image pixel size.
[0086] In an embodiment, referring to Figure 4 The above step S40 can include steps S141-S146.
[0087] S141, placing a measurement target in the template region corresponding to the processed template image to obtain the actual size of the inner rectangular region of the measurement target.
[0088] In the embodiment, the measurement target is placed in the template region corresponding to the template image, i.e. the region corresponding to the template image, to calculate the relationship between the pixel size and the physical size. The measurement target is composed of two concentric squares, the inner rectangle is black filled, and the white filled region is between the outer rectangle and the inner rectangle.
[0089] S142, intercepting an image containing the entire inner rectangular region in the circumscribed rectangular region of the measurement target to obtain a region image.
[0090] In the embodiment, the region image refers to the image containing the entire inner rectangular region intercepted in the circumscribed rectangular region of the measurement target.
[0091] The actual size of the inner rectangular region is denoted as L n . The region image containing the entire inner rectangle in the circumscribed rectangular region is denoted as I c .
[0092] S143, calculate the average gray value of the region image.
[0093] In this embodiment, the average gray value a c of the region image I c is calculated by using the prior art.
[0094] S144, calculate the binary image of the region image.
[0095] In this embodiment, the binary image I c of the region image I b is calculated by using the prior art.
[0096] S145, calculate the contour of the region image, determine the minimum circumscribed rectangle of the contour, and determine the pixel length and width of the minimum circumscribed rectangle.
[0097] In this embodiment, the contour of the region image I c is calculated by using the prior art, and the minimum circumscribed rectangle of the contour is obtained, and the pixel length and width of the minimum circumscribed rectangle are denoted as w and l respectively.
[0098] S146, calculate the actual size corresponding to one pixel in the x direction and the y direction of the region image, to obtain the proportional relationship between the pixel size and the physical size.
[0099] The actual size L c corresponding to one pixel in the x direction and the y direction of the region image I x and L y , L x and L y denote the proportional relationship between the pixel size and the physical size in the x direction and the y direction, and the calculation formula is as follows:
[0100] S150, extract the feature points of the processed template image and the feature points of the processed matching image, and match the feature points of the processed template image and the feature points of the processed matching image to obtain a matching result.
[0101] In this embodiment, the matching result refers to the point combination matched by the feature points of the processed template image and the feature points of the processed matching image.
[0102] In an embodiment, referring to Figure 5 , the above-mentioned step S150 can include steps S151-S153.
[0103] S151, extract the sift feature points of the processed template image and the processed matching image to obtain the template image feature points and the matching image feature points.
[0104] In the embodiment, the template image feature points refer to sift feature points on the template image; the matching image feature points refer to sift feature points on the matching image. The sift feature points refer to stable and unchangeable corners.
[0105] Specifically, the sift feature point extraction process is as follows: a Gaussian scale space of an image is constructed by using L(x, y, σ) = G(x, y, σ) * I(x, y), wherein the Gaussian kernel function is I(x, y) is an input image, and L(x, y, σ) is a scale space of the input image constructed. The difference between the images of adjacent layers of the Gaussian scale space is calculated, and a DOG (Difference of Guassian) operator is used to calculate the key points to determine the feature points.
[0106] S152, calculate the feature vectors of the template image feature points and the matching image feature points, and normalize to obtain processed template image feature points and processed matching image feature points.
[0107] In the embodiment, the processed template image feature points refer to the results obtained after the feature vectors of the template image feature points are normalized; and the processed matching image feature points refer to the results obtained after the feature vectors of the matching image feature points are normalized.
[0108] Specifically, the direction of the key point, i.e., the feature point, is determined according to the gradient direction and the modulus of the center adjacent window pixels of the key point, and a 128-dimensional feature vector is generated. The 128-dimensional feature vector of the key point is normalized, for example, as follows: wherein X is an input vector, (x'0, x'1,..., x'n-1) is the normalized vector. 127
[0109] S153, perform feature matching on the processed template image feature points and the processed matching image feature points, and eliminate abnormal feature points to obtain a matching result.
[0110] In the embodiment, the similarity of the feature points in the template image and the matching image is determined by the feature vectors of the key points, i.e., the feature points, and the Euclidean distance. The feature points with high similarity can be included in the matching result, and the abnormal feature points are eliminated, and the stable matched feature points are retained.
[0111] Extracting natural target sift features for related matching calculation has the advantages of strong anti-interference, high precision, small influence of external environment, low maintenance cost, etc.
[0112] S160, calculate the pixel displacement distance of the target object according to the matching result.
[0113] In the embodiment, the pixel displacement distance of the target object refers to the displacement of the target object on the pixel.
[0114] In an embodiment, referring to Figure 6 , the step S160 can include steps S161-S163.
[0115] S161, calculate the difference of the two feature point coordinates in the matching result to obtain the x coordinate offset and the y coordinate offset.
[0116] In the embodiment, the x coordinate offset and the y coordinate offset respectively refer to the offset of the target object in the x direction and the y direction of the pixel.
[0117] Specifically, subtract the feature point coordinates of the matched template image from the feature point coordinates of the matched image to calculate the x coordinate offset and the y coordinate offset
[0118] S162, sort the x coordinate offset and the y coordinate offset according to their absolute values from small to large, and screen the required coordinate offset to obtain the target coordinate offset.
[0119] In the embodiment, the target coordinate offset refers to the coordinate offset of the target object.
[0120] Specifically, sort and according to their absolute values from small to large, select the top 30%-50% of feature points as reliable feature points, and the difference of these feature points can obtain the target coordinate offset.
[0121] S163, calculate the pixel offset in the x direction and the pixel offset in the y direction according to the target coordinate offset to obtain the pixel displacement distance of the target object.
[0122] Sum and average the reliable points to calculate the pixel offset in the x direction a x and the pixel offset in the y direction a y , so that a x and a y constitute the pixel displacement distance of the target object.
[0123] S170, determine the actual displacement distance of the target object according to the pixel displacement distance of the target object and the proportional relationship between the pixel size and the physical size.
[0124] In the embodiment, the actual displacement distance of the target object refers to the physical displacement of the target object.
[0125] Specifically, the product of the pixel displacement distance of the target object and the proportional relationship between the pixel size and the physical size is calculated to obtain the actual displacement distance of the target object. The pixel size of the monitored target object displacement is calculated by matching the feature points, and the actual displacement size of the target object is calculated by using the proportional relationship between the pixel and the actual size.
[0126] The physical size of the displacement of the target object relative to the template is calculated by calculating the pixel displacement between the matching image and the template image, and then calculating the actual size L x and L y of one pixel corresponding to the calculated pixel. x and d y , then d x =a x *L x ; d y =a y *L y .
[0127] The target object displacement monitoring method described above sets a template image, generates a matching image, pre-processes the template image and the matching image, reduces the requirements for environmental deployment in the monitoring process, and is not easily affected by external interference factors. The proportional relationship between the pixel size and the physical size of the processed template image is determined, and the target object pixel displacement distance is determined by extracting and matching the target sift features of the two images, and then the actual displacement distance of the target object is determined. The entire method has strong anti-interference, high precision, small influence from the external environment, and low maintenance cost, so as to realize high monitoring precision, no strict requirements for environmental deployment, and small influence from external interference factors.
[0128] Figure 7 is a schematic block diagram of a target object displacement monitoring device 300 provided by an embodiment of the present application. As Figure 7 indicated, corresponding to the above target object displacement monitoring method, the present application further provides a target object displacement monitoring device 300. The target object displacement monitoring device 300 includes units for executing the above target object displacement monitoring method, and the device can be configured in a server. Specifically, please refer to Figure 7 , the target object displacement monitoring device 300 includes a template image acquisition unit 301, a matching image generation unit 302, a pre-processing unit 303, a proportional relationship calculation unit 304, a matching unit 305, a pixel displacement calculation unit 306, and an actual displacement calculation unit 307.
[0129] The template image acquisition unit 301 is configured to acquire an image of a monitored target object to obtain a template image; the matching image generation unit 302 is configured to acquire a matching image according to the template image; the preprocessing unit 303 is configured to pre-process the template image and the matching image to obtain a processed template image and a processed matching image; the scale relationship calculation unit 304 is configured to calculate a scale relationship between a pixel size and a physical size according to the processed template image; the matching unit 305 is configured to extract feature points of the processed template image and feature points of the processed matching image, and match the feature points of the processed template image and the feature points of the processed matching image to obtain a matching result; the pixel displacement calculation unit 306 is configured to calculate a pixel displacement distance of the target object according to the matching result; and the actual displacement calculation unit 307 is configured to determine an actual displacement distance of the target object according to the pixel displacement distance of the target object and the scale relationship between the pixel size and the physical size.
[0130] In an embodiment, the matching image generation unit 302 is configured to acquire an image corresponding to a region at a same position as the template image to obtain the matching image.
[0131] In an embodiment, as shown in Figure 8 The preprocessing unit 303 includes a de-noising sub-unit 3031 and a scaling sub-unit 3032.
[0132] The de-noising sub-unit 3031 is configured to perform Gaussian filtering processing on the template image and the matching image to obtain a de-noised template image and a de-noised matching image; and the scaling sub-unit 3032 is configured to perform gray scale scaling processing on the de-noised template image and the de-noised matching image to obtain the processed template image and the processed matching image.
[0133] In an embodiment, as shown in Figure 9 The scale relationship calculation unit 304 includes a size acquisition sub-unit 3041, a region image acquisition sub-unit 3042, a gray value calculation sub-unit 3043, a binary image calculation sub-unit 3044, a circumscribed rectangle determination sub-unit 3045, and a relationship determination sub-unit 3046.
[0134] The size obtaining sub-unit 3041 is configured to obtain the actual size of the inner rectangular region of the measurement target by placing the measurement target in the template region corresponding to the processed template image; the region image obtaining sub-unit 3042 is configured to intercept an image containing the entire inner rectangular region in the circumscribed rectangular region of the measurement target to obtain a region image; the gray value calculating sub-unit 3043 is configured to calculate the average gray value of the region image; the binary image calculating sub-unit 3044 is configured to calculate a binary image of the region image; the circumscribed rectangle determining sub-unit 3045 is configured to calculate the contour of the region image, determine the minimum circumscribed rectangle of the contour, and determine the pixel length and width of the minimum circumscribed rectangle; and the relationship determining sub-unit 3046 is configured to calculate the actual size corresponding to one pixel of the region image in the x direction and the y direction to obtain the proportional relationship between the pixel size and the physical size.
[0135] In an embodiment, as shown in FIG. 3, the matching unit 305 includes an extracting sub-unit 3051, a normalization sub-unit 3052, and a feature matching sub-unit 3053. Figure 10
[0136] The extracting sub-unit 3051 is configured to extract SIFT feature points from the processed template image and the processed matching image to obtain template image feature points and matching image feature points; the normalization sub-unit 3052 is configured to calculate the feature vectors of the template image feature points and the matching image feature points, and perform normalization to obtain processed template image feature points and processed matching image feature points; and the feature matching sub-unit 3053 is configured to perform feature matching on the processed template image feature points and the processed matching image feature points, and eliminate abnormal feature points to obtain a matching result.
[0137] In an embodiment, as shown in FIG. 3, the matching unit 305 includes an extracting sub-unit 3051, a normalization sub-unit 3052, and a feature matching sub-unit 3053. Figure 11
[0138] The difference calculating sub-unit 3061 is configured to calculate the difference between the coordinates of two feature points in the matching result to obtain the x coordinate offset and the y coordinate offset; the sorting sub-unit 3062 is configured to sort the x coordinate offset and the y coordinate offset in ascending order according to their absolute values, and screen the coordinate offsets meeting the requirements to obtain a target coordinate offset; and the offset determining sub-unit 3063 is configured to calculate the pixel offset in the x direction and the pixel offset in the y direction according to the target coordinate offset to obtain the pixel displacement distance of the target object.
[0139] In an embodiment, the actual displacement calculation unit 307 is configured to calculate a product of the pixel displacement distance of the target object and a proportional relationship between the pixel size and the physical size, to obtain the actual displacement distance of the target object.
[0140] It should be noted that the specific implementation process of the target object displacement monitoring device 300 and each unit can be clearly understood by those skilled in the art, and can refer to the corresponding description in the foregoing method embodiments. For the convenience and brevity of description, it will not be repeated here.
[0141] The target object displacement monitoring device 300 described above can be implemented in the form of a computer program, which can run on a computer device as shown in the drawings. Figure 12
[0142] Please refer to Figure 12 , Figure 12 is a schematic block diagram of a computer device provided by an embodiment of the present application. The computer device 500 can be a server, wherein the server can be a stand-alone server or a server cluster composed of multiple servers.
[0143] Refer to Figure 12 , the computer device 500 includes a processor 502, a memory, and a network interface 505 connected through a system bus 501, wherein the memory can include a non-volatile storage medium 503 and an internal memory 504.
[0144] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, which when executed, can cause the processor 502 to perform a target object displacement monitoring method.
[0145] The processor 502 is configured to provide computing and control capabilities to support the operation of the entire computer device 500.
[0146] The internal memory 504 provides an environment for the running of the computer program 5032 in the non-volatile storage medium 503, which when executed by the processor 502, can cause the processor 502 to perform a target object displacement monitoring method.
[0147] The network interface 505 is configured to perform network communication with other devices. Those skilled in the art can understand that Figure 12 the structure shown in the drawings is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device 500 to which the scheme of the present application is applied. The specific computer device 500 can include more or fewer components than those shown in the drawings, or combine certain components, or have a different arrangement of components.
[0148] The processor 502 is configured to run the computer program 5032 stored in the memory to implement the following steps:
[0149] An image of the monitored target object is acquired to obtain a template image; a matching image is acquired according to the template image; the template image and the matching image are preprocessed to obtain a processed template image and a processed matching image; a proportional relationship between a pixel size and a physical size is calculated according to the processed template image; feature points of the processed template image and feature points of the processed matching image are extracted, and the feature points of the processed template image and the feature points of the processed matching image are matched to obtain a matching result; a pixel displacement distance of the target object is calculated according to the matching result; and an actual displacement distance of the target object is determined according to the pixel displacement distance of the target object and the proportional relationship between the pixel size and the physical size.
[0150] In an embodiment, when implementing the step of acquiring a matching image according to the template image, the processor 502 specifically implements the following steps:
[0151] An image corresponding to an area at the same position as the template image is acquired to obtain a matching image.
[0152] In an embodiment, when implementing the step of preprocessing the template image and the matching image to obtain a processed template image and a processed matching image, the processor 502 specifically implements the following steps:
[0153] The template image and the matching image are subjected to Gaussian filtering processing to obtain a denoised template image and a denoised matching image; and the denoised template image and the denoised matching image are subjected to gray scale scaling processing to obtain a processed template image and a processed matching image.
[0154] In an embodiment, when implementing the step of calculating a proportional relationship between a pixel size and a physical size according to the processed template image, the processor 502 specifically implements the following steps:
[0155] A measurement target is placed in a template area corresponding to the processed template image, an actual size of an inner rectangular area of the measurement target is acquired; an image containing the entire inner rectangular area is intercepted in a circumscribed rectangular area of the measurement target to obtain an area image; an average gray scale value of the area image is calculated; a binary image of the area image is calculated; an outline of the area image is calculated, a minimum circumscribed rectangle of the outline is determined, and a pixel length and width of the minimum circumscribed rectangle are determined; and an actual size corresponding to one pixel in an x direction and a y direction of the area image is calculated to obtain a proportional relationship between a pixel size and a physical size.
[0156] In an embodiment, the processor 502 implements the following steps when implementing the feature points of the processed template image and the processed matching image, and matching the feature points of the processed template image and the processed matching image to obtain the matching result step:
[0157] extracting the sift feature points of the processed template image and the processed matching image to obtain the template image feature points and the matching image feature points; calculating the feature vectors of the template image feature points and the matching image feature points, and normalizing to obtain the processed template image feature points and the processed matching image feature points; performing feature matching on the processed template image feature points and the processed matching image feature points, and eliminating abnormal feature points to obtain the matching result.
[0158] In an embodiment, the processor 502 implements the following steps when implementing the step of calculating the pixel displacement distance of the target object according to the matching result:
[0159] calculating the difference between the coordinates of two feature points in the matching result to obtain the x-coordinate offset and the y-coordinate offset; sorting the x-coordinate offset and the y-coordinate offset according to their absolute values from small to large, and screening the coordinate offsets that meet the requirements to obtain the target coordinate offset; and calculating the x-direction pixel offset and the y-direction pixel offset according to the target coordinate offset to obtain the pixel displacement distance of the target object.
[0160] In an embodiment, the processor 502 implements the following steps when implementing the step of determining the actual displacement distance of the target object according to the pixel displacement distance of the target object and the proportional relationship between the pixel size and the physical size:
[0161] calculating the product of the pixel displacement distance of the target object and the proportional relationship between the pixel size and the physical size to obtain the actual displacement distance of the target object.
[0162] It should be understood that, in the embodiments of the present application, the processor 502 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.
[0163] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the above-mentioned embodiments.
[0164] Therefore, the present application further provides a storage medium. The storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program is executed by a processor to make the processor perform the following steps:
[0165] An image of the monitored target object is acquired to obtain a template image; a matching image is acquired according to the template image; the template image and the matching image are preprocessed to obtain a processed template image and a processed matching image; a proportional relationship between pixel size and physical size is calculated according to the processed template image; feature points of the processed template image and feature points of the processed matching image are extracted, and the feature points of the processed template image and the feature points of the processed matching image are matched to obtain a matching result; a pixel displacement distance of the target object is calculated according to the matching result; and an actual displacement distance of the target object is determined according to the pixel displacement distance of the target object and the proportional relationship between pixel size and physical size.
[0166] In an embodiment, when the processor executes the computer program to implement the step of acquiring a matching image according to the template image, the following steps are implemented:
[0167] An image corresponding to an area at the same position as the template image is acquired to obtain a matching image.
[0168] In an embodiment, the processor, when executing the computer program to implement the step of pre-processing the template image and the matching image to obtain a processed template image and a processed matching image, implements the following steps:
[0169] Gaussian filtering the template image and the matching image to obtain a denoised template image and a denoised matching image; and performing gray scale scaling on the denoised template image and the denoised matching image to obtain the processed template image and the processed matching image.
[0170] In an embodiment, the processor, when executing the computer program to implement the step of calculating the proportional relationship between the pixel size and the physical size according to the processed template image, implements the following steps:
[0171] placing a measurement target in a template region corresponding to the processed template image to obtain an actual size of an inner rectangular region of the measurement target; intercepting an image containing the entire inner rectangular region in a circumscribed rectangular region of the measurement target to obtain a region image; calculating an average gray scale value of the region image; calculating a binary image of the region image; calculating a contour of the region image, determining a minimum circumscribed rectangle of the contour, and determining a pixel length and width of the minimum circumscribed rectangle; and calculating an actual size corresponding to one pixel in the x direction and the y direction of the region image to obtain the proportional relationship between the pixel size and the physical size.
[0172] In an embodiment, the processor, when executing the computer program to implement the step of extracting feature points of the processed template image and feature points of the processed matching image, and matching the feature points of the processed template image and the feature points of the processed matching image to obtain a matching result, implements the following steps:
[0173] extracting sift feature points from the processed template image and the processed matching image to obtain template image feature points and matching image feature points; calculating feature vectors of the template image feature points and the matching image feature points, and normalizing to obtain processed template image feature points and processed matching image feature points; and performing feature matching on the processed template image feature points and the processed matching image feature points, and eliminating abnormal feature points to obtain the matching result.
[0174] In an embodiment, the processor, when executing the computer program to implement the step of calculating the pixel displacement distance of the target object according to the matching result, implements the following steps:
[0175] Calculate the difference between the two feature point coordinates in the matching result to obtain an x coordinate offset and a y coordinate offset; sort the x coordinate offset and the y coordinate offset according to their absolute values from small to large, and screen the coordinate offsets meeting the requirements to obtain a target coordinate offset; and calculate the pixel offset in the x direction and the pixel offset in the y direction according to the target coordinate offset to obtain the pixel displacement distance of the target object.
[0176] In an embodiment, when the processor executes the computer program to implement the step of determining the actual displacement distance of the target object according to the pixel displacement distance of the target object and the ratio relationship between the pixel size and the physical size, the processor specifically implements the following steps:
[0177] Calculate the product of the pixel displacement distance of the target object and the ratio relationship between the pixel size and the physical size to obtain the actual displacement distance of the target object.
[0178] The storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, and various computer readable storage media that can store program codes.
[0179] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.
[0180] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not implemented.
[0181] The steps in the method embodiments of the present application can be adjusted, combined and reduced in sequence according to actual needs. The units in the device embodiments of the present application can be combined, divided and reduced according to actual needs. In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.
[0182] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part of the prior art that contributes, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions to make a computer device (which can be a personal computer, terminal, or network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application.
[0183] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.
Claims
1. A method of monitoring displacement of an object, characterized by, The method comprises the following steps: acquiring an image of a monitored target object to obtain a template image; acquiring a matching image according to the template image; preprocessing the template image and the matching image to obtain a processed template image and a processed matching image; calculating a proportional relationship between a pixel size and a physical size according to the processed template image; extracting feature points of the processed template image and feature points of the processed matching image, and matching the feature points of the processed template image and the feature points of the processed matching image to obtain a matching result; calculating a pixel displacement distance of the target object according to the matching result; determining an actual displacement distance of the target object according to the pixel displacement distance of the target object and the proportional relationship between the pixel size and the physical size; the step of calculating the proportional relationship between the pixel size and the physical size according to the processed template image comprises the following steps: placing a measurement target in a template region corresponding to the processed template image to acquire an actual size of an inner rectangular region of the measurement target; cutting an image containing the entire inner rectangular region in a circumscribed rectangular region of the measurement target to obtain a region image; calculating an average gray value of the region image; calculating a binary image of the region image; calculating a contour of the region image, determining a minimum circumscribed rectangle of the contour, and determining a pixel length and width of the minimum circumscribed rectangle; calculating actual sizes corresponding to one pixel in the x direction and the y direction of the region image to obtain the proportional relationship between the pixel size and the physical size; the step of extracting the feature points of the processed template image and the feature points of the processed matching image, and matching the feature points of the processed template image and the feature points of the processed matching image to obtain the matching result comprises the following steps: extracting sift feature points from the processed template image and the processed matching image to obtain template image feature points and matching image feature points; calculating feature vectors of the template image feature points and the matching image feature points, and normalizing to obtain processed template image feature points and processed matching image feature points; performing feature matching on the processed template image feature points and the processed matching image feature points, and eliminating abnormal feature points to obtain the matching result; the step of calculating the pixel displacement distance of the target object according to the matching result comprises the following steps: calculating a difference value of two feature point coordinates in the matching result to obtain an x coordinate offset and a y coordinate offset; sorting the x coordinate offset and the y coordinate offset in ascending order according to their absolute values, and screening coordinate offsets meeting the requirements to obtain target coordinate offsets; calculating an x direction pixel offset and a y direction pixel offset according to the target coordinate offsets to obtain the pixel displacement distance of the target object; the step of determining the actual displacement distance of the target object according to the pixel displacement distance of the target object and the proportional relationship between the pixel size and the physical size comprises the following steps: calculating a product of the pixel displacement distance of the target object and the proportional relationship between the pixel size and the physical size to obtain the actual displacement distance of the target object.
2. The target displacement monitoring method according to claim 1, wherein the step of acquiring a matching image according to the template image comprises the following steps: An image corresponding to the region at the same position as the template image is acquired to obtain a matching image.
3. The target displacement monitoring method according to claim 1, wherein The template image and the matching image are preprocessed to obtain a processed template image and a processed matching image, including: The template image and the matching image are subjected to Gaussian filtering processing to obtain a denoised template image and a denoised matching image; The denoised template image and the denoised matching image are subjected to gray scale scaling processing to obtain the processed template image and the processed matching image.
4. Object displacement monitoring device, characterized in that The device uses the target object displacement monitoring method according to any one of claims 1 to 3, including: A template image acquisition unit is configured to acquire an image of a monitored target object to obtain a template image; A matching image generation unit is configured to acquire a matching image based on the template image; A preprocessing unit is configured to preprocess the template image and the matching image to obtain a processed template image and a processed matching image; A proportional relationship calculation unit is configured to calculate a proportional relationship between a pixel size and a physical size based on the processed template image; A matching unit is configured to extract feature points of the processed template image and feature points of the processed matching image, and match the feature points of the processed template image and the feature points of the processed matching image to obtain a matching result; A pixel displacement calculation unit is configured to calculate a pixel displacement distance of the target object based on the matching result; An actual displacement calculation unit is configured to determine an actual displacement distance of the target object based on the pixel displacement distance of the target object and the proportional relationship between the pixel size and the physical size; The matching unit includes: An extraction subunit is configured to extract SIFT feature points from the processed template image and the processed matching image to obtain template image feature points and matching image feature points; a normalization subunit is configured to calculate feature vectors of the template image feature points and the matching image feature points, and normalize the feature vectors to obtain processed template image feature points and processed matching image feature points; and a feature matching subunit is configured to perform feature matching on the processed template image feature points and the processed matching image feature points, and eliminate abnormal feature points to obtain the matching result. The pixel displacement calculation unit includes: A difference calculation subunit is configured to calculate a difference between coordinates of two feature points in the matching result to obtain an x-coordinate offset and a y-coordinate offset; an ordering subunit is configured to order the x-coordinate offset and the y-coordinate offset from small to large according to their absolute values, and screen coordinate offsets meeting a requirement to obtain target coordinate offsets; and an offset determination subunit is configured to calculate an x-direction pixel offset and a y-direction pixel offset based on the target coordinate offsets to obtain the pixel displacement distance of the target object.
5. A computer device, comprising: The computer device includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the method according to any one of claims 1 to 3.
6. A storage medium, characterized by The storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1 to 3.
Citation Information
Patent Citations
Building structure displacement measurement method based on dynamic visual sensor DVS
CN113155032A