Bridge structure safety early warning method and system, storage medium and electronic equipment

Abstract

The invention provides a bridge structure safety early warning method and system, a storage medium and electronic equipment, and the method comprises the steps: obtaining an initial image, and carrying out the transformation through Gaussian convolution and Gaussian difference, and obtaining a Gaussian difference space; fitting the Gaussian difference space curve to obtain extreme points, namely feature points; rotating the feature points to obtain a gradient direction, and obtaining region representation according to the gradient direction; comparing region representations of different feature points, and removing pixel points which do not meet requirements; performing point cloud generation on the reserved pixel points to obtain an initial point cloud set, and further obtaining a complete point cloud set; optimizing the complete point cloud set to obtain an optimized point cloud set, and performing triangular meshing to obtain a Gaussian curvature field; the number of adjacent point clouds, the area density and the space density are calculated to find out disease point cloud blocks, and an alarm is given, the bridge contour and the diseases are reserved, meanwhile, the calculated amount is reduced, meanwhile, the curvature research method is adopted, the diseases hidden in the point clouds are found out, and the monitoring precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge monitoring, and in particular to a bridge structure safety early warning method, system, storage medium and electronic equipment. Background Art

[0002] At present, potential damage to bridges can easily lead to serious damage during bridge operation. Accurately obtaining the potential damage area is a necessary prerequisite for accurate detection of bridge damage, which can then make safety warnings in advance.

[0003] In the existing technology, laser radar is generally used to scan various parts of the bridge from multiple angles to obtain the point cloud of the bridge. However, since the radar is not penetrating, different point cloud sets need to be obtained from various angles and then aligned. At present, due to the large number of point cloud sets during alignment, the number of point clouds in the point cloud set is huge, which occupies a large amount of computing resources. If some point clouds are deleted, it will lead to inaccurate alignment, and the subsequent identification of bridge defects will also lead to inaccurate identification, which in turn affects the safety monitoring results of the bridge. Summary of the invention In view of the deficiencies in the prior art, the purpose of the present invention is to provide a bridge structure safety early warning method, aiming to solve the technical problems mentioned in the background technology.

[0004] In order to achieve the above object, the present invention is implemented by the following technical solutions: A bridge structure safety early warning method comprises the following steps: Acquire initial images of the bridge at different angles through a camera, and transform the initial images through Gaussian convolution and Gaussian difference to obtain a Gaussian difference space of the initial images; Performing curve fitting on the Gaussian difference space of the initial image to obtain extreme points, and using the extreme points as feature points of the initial image; Rotating the feature points of the initial image toward adjacent pixels to obtain a gradient direction of the feature points, and obtaining a region representation of the feature points according to the gradient direction; Based on a matching algorithm, the regional representations of the feature points of different initial images are compared, and all the pixels in the regional representations that fail the comparison are eliminated; Generating a point cloud for all pixel points under the reserved area representation to obtain an initial point cloud set, and obtaining a complete point cloud set based on a multi-view stereo vision algorithm; Performing filtering and registration processing on the complete point cloud set to obtain an optimized point cloud set; Triangulating the optimized point cloud set to obtain a Gaussian curvature field of the optimized point cloud set; The number of adjacent point clouds, area density, and spatial density of a single research point in the Gaussian curvature field are calculated to find point cloud blocks representing bridge defects and generate an alarm at the monitoring center.

[0005] According to one aspect of the above technical solution, the initial images of the bridge at different angles are obtained by a camera, and the initial images are transformed by Gaussian convolution and Gaussian difference to obtain the Gaussian difference space of the initial images, specifically including: Acquire initial images of the bridge at different angles through a camera; Based on the intrinsic and extrinsic parameters of the camera, the initial image is reflected in the scale space by Gaussian convolution method; ; Where G represents the Gaussian convolution function, C is the scale space, φ represents the parameter variable of the scale space, (x, y) is the coordinate value of the pixel point in the initial image, * is the convolution factor in the Gaussian convolution function, and U is the initial image; Transforming the scale space of the initial image by using a Gaussian difference method to obtain a Gaussian difference space of the initial image; ; Where k represents the scale factor of the adjacent scale space, and D represents the Gaussian difference function.

[0006] According to one aspect of the above technical solution, performing curve fitting on the Gaussian difference space of the initial image to obtain extreme points, and using the extreme points as feature points of the initial image, specifically includes: Performing Taylor expansion on the Gaussian difference space; ; Among them, X represents the original function value corresponding to the extreme point in the initial image, T is the transpose, and ∂ is the sign for partial derivative; Derivative the Gaussian difference function after Taylor expansion to obtain the extreme point X'; ; The extreme points are used as feature points of the initial image.

[0007] According to one aspect of the above technical solution, rotating the feature points of the initial image toward adjacent pixels to obtain the gradient direction of the feature points, and obtaining the region representation of the feature points according to the gradient direction specifically includes: Rotating the feature points of the initial image toward adjacent pixels using an image rotation formula to obtain multiple gradient directions of the feature points; ; Among them, θ is the rotation angle of the gradient direction, V is the scale space where the extreme point is located, and f is the original function of the initial image; Add the amplitudes of the feature points located in the same gradient direction, select the gradient direction corresponding to the feature point with the highest amplitude as the reference direction, and calculate the position and spatial scale of the feature point located in the reference direction; Taking the position of the feature point as the reference point and the spatial scale of the feature point as the radius, all pixel points within the query range are vectorized with reference to the reference direction, and the vectorized pixel points are used as the region representation of the feature point.

[0008] According to one aspect of the above technical solution, the matching algorithm is used to compare the regional representations of the feature points of different initial images, and all the pixels in the regional representations that fail the comparison are eliminated, specifically including: Based on the SIFT algorithm, comparing the region representations where the same feature points in different initial images are located; A similarity threshold and a distance threshold are set, and the pixel points that meet the similarity threshold and the distance threshold in the region representations of different initial images are matched by the SIFT algorithm, and the pixel points that do not meet the similarity threshold and the distance threshold are removed.

[0009] According to one aspect of the above technical solution, triangulating the optimized point cloud set to obtain the Gaussian curvature field of the optimized point cloud set specifically includes: The outermost point cloud in the optimized point cloud set is used as a point cloud surface, and a simulation shell is established based on the point cloud surface; Using the feature point as an original point on the simulated shell, using the original point as a corner point of a triangle, and using adjacent pixel points of the original point as other corner points of the triangle to generate an initial triangular mesh; Extending the initial triangular mesh outward, gradually generating multiple triangular meshes with the remaining pixel points as corner points, and taking the surface of each triangular mesh as a triangular facet; Based on the Gaussian curvature calculation formula, the Gaussian curvature of the triangular facet extending outward from the original point is calculated, and the Gaussian curvature of each triangular facet is combined to form a Gaussian curvature field; ; ; Among them, P represents Gaussian curvature, i represents the serial number of the original point, j represents the serial number of the pixel point adjacent to the original point, ξ represents the angle of the triangle patch, δ represents the original point, and S represents the area of ​​the triangle patch.

[0010] According to one aspect of the above technical solution, the calculation of the number of adjacent point clouds, area density, and spatial density of a single research point in the Gaussian curvature field to find point cloud blocks representing bridge defects and to issue an alarm at a monitoring center specifically includes: A Gaussian curvature classification table is preset to statistically classify the curvature of each triangular face in the Gaussian curvature field; In the Gaussian curvature field, with the original point as the center, a search radius is preset, the number of triangles in each Gaussian curvature classification within the search range is calculated, and the number of pixels covered is calculated accordingly as the number of adjacent point clouds; In the Gaussian curvature field, a research surface is preset, the number of triangular facets in each Gaussian curvature classification in the research surface is calculated, and the number of pixel points covered is calculated accordingly as the area density; In the Gaussian curvature field, a research volume is preset, the number of triangular facets in each Gaussian curvature classification in the research volume is calculated, and the number of pixel points covered is calculated accordingly as the spatial density; According to different parts of the bridge, it is studied whether one or more indicators of the number of adjacent point clouds, area density and space density of the part meet the preset requirements. If they do not meet the preset requirements, an alarm is sent to the monitoring center.

[0011] The present invention also provides a bridge structure safety early warning system, comprising: Transformation module: used to obtain initial images of the bridge at different angles through a camera, and transform the initial images through Gaussian convolution and Gaussian difference to obtain Gaussian difference space of the initial images; The transformation module is specifically used to: obtain initial images of the bridge at different angles through a camera; Based on the intrinsic and extrinsic parameters of the camera, the initial image is reflected in the scale space by Gaussian convolution method; ; Where G represents the Gaussian convolution function, C is the scale space, φ represents the parameter variable of the scale space, (x, y) is the coordinate value of the pixel point in the initial image, * is the convolution factor in the Gaussian convolution function, and U is the initial image; Transforming the scale space of the initial image by using a Gaussian difference method to obtain a Gaussian difference space of the initial image; ; Where k represents the scale factor of the adjacent scale space, and D represents the Gaussian difference function; An obtaining module; used for performing curve fitting on the Gaussian difference space of the initial image to obtain extreme points, and using the extreme points as feature points of the initial image; The obtaining module is specifically used for: performing Taylor expansion on the Gaussian difference space; ; Among them, X represents the original function value corresponding to the extreme point in the initial image, T is the transpose, and ∂ is the sign for partial derivative; Derivative the Gaussian difference function after Taylor expansion to obtain the extreme point X'; ; Taking the extreme points as feature points of the initial image; A representation module: used for rotating the feature points of the initial image toward adjacent pixel points to obtain the gradient direction of the feature points, and obtaining the regional representation of the feature points according to the gradient direction; The representation module is specifically used to: rotate the feature points of the initial image toward adjacent pixel points by using an image rotation formula to obtain multiple gradient directions of the feature points; ; Among them, θ is the rotation angle of the gradient direction, V is the scale space where the extreme point is located, and f is the original function of the initial image; Add the amplitudes of the feature points located in the same gradient direction, select the gradient direction corresponding to the feature point with the highest amplitude as the reference direction, and calculate the position and spatial scale of the feature point located in the reference direction; Taking the position of the feature point as a reference point and the spatial scale of the feature point as a radius, querying all pixel points within a range, vectorizing the pixel points with reference to the reference direction, and using the vectorized pixel points as the region representation of the feature point; Elimination module: used for comparing the regional representations of the feature points of different initial images based on a matching algorithm, and eliminating all the pixels in the regional representations that fail the comparison; The elimination module is specifically used to: compare the region representations where the same feature points are located in different initial images based on the SIFT algorithm; Setting a similarity threshold and a distance threshold, matching the pixel points that meet the similarity threshold and the distance threshold in the region representations of different initial images by the SIFT algorithm, and removing the pixel points that do not meet the similarity threshold and the distance threshold; A generation module: used to generate a point cloud from all the pixels represented by the retained area to obtain an initial point cloud set, and obtain a complete point cloud set based on a multi-view stereo vision algorithm; Optimization module: used for performing filtering and registration processing on the complete point cloud set to obtain an optimized point cloud set; A calculation module is used to triangulate the optimized point cloud set to obtain a Gaussian curvature field of the optimized point cloud set; The calculation module is specifically used to: use the outermost point cloud in the optimized point cloud set as a point cloud surface, and establish a simulated shell based on the point cloud surface; Using the feature point as an original point on the simulated shell, using the original point as a corner point of a triangle, and using adjacent pixel points of the original point as other corner points of the triangle to generate an initial triangular mesh; Extending the initial triangular mesh outward, gradually generating multiple triangular meshes with the remaining pixel points as corner points, and taking the surface of each triangular mesh as a triangular facet; Based on the Gaussian curvature calculation formula, the Gaussian curvature of the triangular facet extending outward from the original point is calculated, and the Gaussian curvature of each triangular facet is combined to form a Gaussian curvature field; ; ; Where P represents Gaussian curvature, i represents the number of the original point, j represents the number of the pixel point adjacent to the original point, ξ represents the angle of the triangle, δ represents the original point, and S represents the area of ​​the triangle. Alarm module: used to calculate the number of adjacent point clouds, area density, and spatial density of a single research point in the Gaussian curvature field to find out the point cloud blocks representing bridge defects and issue an alarm at the monitoring center; The alarm module is specifically used to: preset a Gaussian curvature classification table, and statistically classify the curvature of each triangular facet in the Gaussian curvature field; In the Gaussian curvature field, with the original point as the center, a search radius is preset, the number of triangles in each Gaussian curvature classification within the search range is calculated, and the number of pixels covered is calculated accordingly as the number of adjacent point clouds; In the Gaussian curvature field, a research surface is preset, the number of triangular facets in each Gaussian curvature classification in the research surface is calculated, and the number of pixel points covered is calculated accordingly as the area density; In the Gaussian curvature field, a research volume is preset, the number of triangular facets in each Gaussian curvature classification in the research volume is calculated, and the number of pixel points covered is calculated accordingly as the spatial density; According to different parts of the bridge, it is studied whether one or more indicators of the number of adjacent point clouds, area density and space density of the part meet the preset requirements. If they do not meet the preset requirements, an alarm is sent to the monitoring center.

[0012] The present invention also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the bridge structure safety early warning method as described above is implemented.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the bridge structure safety early warning method as described above when executing the computer program.

[0014] Compared with the prior art, the present invention has the following beneficial effects: By transforming the initial image through Gaussian convolution and Gaussian difference methods, we can get Gaussian difference space to reflect the rate of change of the initial image at different scales, and then we can find the extreme points of the initial image through the rate of change at different scales. The extreme points can reflect the specific outline of the bridge. Therefore, using the extreme points as feature points can greatly reduce the initial calculation of a large number of point clouds while retaining the basic characteristics of the bridge, and the feature points also retain the characteristics of bridge diseases; then the gradient direction is obtained by rotating the initial image to find the regional representation of the feature points, and then the pixels in the regional representation that are not passed are eliminated. This step can find the reference direction of the feature points, and at the same time, the regional representation that is not in the reference direction of the feature points is abandoned through the machine algorithm, thereby optimizing the quality of the pixels and improving the calculation speed; the remaining pixels are used as the initial point cloud to construct the initial The initial point cloud reflects the specific outline of the bridge, and then the complete point cloud is obtained by iterative optimization step by step according to the stereo vision algorithm, which is similar to the idea of ​​taking a surface from a line and taking a body from a surface. Since the complete point cloud contains noise, filtering is required. In order to splice the complete point cloud of multiple surfaces, coarse and fine registration are required to obtain the optimized point cloud. The optimized point cloud is triangulated, and the Gaussian curvature of the triangulated mesh relative to the base point is calculated. Each triangular mesh is spliced ​​together to obtain the Gaussian curvature field, and the Gaussian curvature of each part of the bridge is intuitively reflected through the Gaussian curvature field. Finally, the number of adjacent point clouds, area density, and spatial density of a single research point are calculated in the Gaussian curvature field. According to the properties of each part of the bridge, it is decided which parameter index is used to analyze the bridge's defects. If defects are found, the monitoring center is alarmed to achieve the purpose of rapid maintenance. The present invention greatly reduces the amount of calculation of the initial point cloud while retaining the specific outline and defects of the bridge. At the same time, the curvature research method is used to find out the defects hidden in the point cloud, thereby improving the monitoring accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of the bridge structure safety early warning method in the first embodiment of the present invention; Figure 2It is a structural block diagram of a bridge structure safety early warning system in a second embodiment of the present invention; Figure 3 is a structural block diagram of an electronic device in a third embodiment of the present invention; The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0016] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0017] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0019] See also Figure 1 , which is a bridge structure safety early warning method in the first embodiment of the present invention, comprising the following steps: S10, acquiring initial images of the bridge at different angles through a camera, and transforming the initial images through Gaussian convolution and Gaussian difference to obtain a Gaussian difference space of the initial images; S20, performing curve fitting on the Gaussian difference space of the initial image to obtain extreme points, and using the extreme points as feature points of the initial image; S30, rotating the feature point of the initial image toward the adjacent pixel point to obtain the gradient direction of the feature point, and obtaining the region representation of the feature point according to the gradient direction; S40, comparing the regional representations of the feature points of different initial images based on a matching algorithm, and removing all the pixel points in the regional representations that fail the comparison; S50, generating a point cloud for all pixel points represented by the retained region to obtain an initial point cloud set, and obtaining a complete point cloud set based on a multi-view stereo vision algorithm; S60, performing filtering and registration processing on the complete point cloud set to obtain an optimized point cloud set; S70, triangulating the optimized point cloud set to obtain a Gaussian curvature field of the optimized point cloud set; S80, calculating the number of adjacent point clouds, area density, and spatial density of a single research point in the Gaussian curvature field to find out the point cloud blocks representing bridge defects, and issuing an alarm at the monitoring center.

[0020] It can be understood that the present invention transforms the initial image by the method of Gaussian convolution and Gaussian difference to obtain a Gaussian difference space to reflect the rate of change of the initial image at different scales, and then the extreme points of the initial image can be obtained through the rate of change at different scales. The extreme points can reflect the specific outline of the bridge. Therefore, using the extreme points as feature points can greatly reduce the initial calculation of a large number of point clouds while retaining the basic characteristics of the bridge, and the feature points also retain the characteristics of the bridge disease; then the gradient direction is obtained by rotating the initial image to obtain the regional representation of the feature points, and then the pixels in the regional representation that are not passed are eliminated. This step can find the reference direction of the feature points, and at the same time, the regional representation that is not in the reference direction of the feature points is abandoned through a machine algorithm, thereby optimizing the quality of the pixel points and improving the calculation speed; the remaining pixel points are used as the initial point cloud , to construct an initial point cloud set, which reflects the specific outline of the bridge. Then, according to the stereo vision algorithm, it is optimized step by step through iteration to obtain a complete point cloud set, which is similar to the idea of ​​taking a surface from a line and taking a body from a surface. Since the complete point cloud set contains noise, filtering processing is required. In order to splice the complete point cloud sets of multiple surfaces, coarse and fine registration are required to obtain the optimized point cloud set. The optimized point cloud set is triangulated, and the Gaussian curvature of the triangulated mesh relative to the base point is calculated. Each triangular mesh is spliced ​​together to obtain a Gaussian curvature field, and the Gaussian curvature of each part of the bridge is intuitively reflected through the Gaussian curvature field. Finally, the number of adjacent point clouds, area density, and spatial density of a single research point are calculated in the Gaussian curvature field. According to the properties of each part of the bridge, it is decided which parameter index is used to analyze the bridge disease. If a disease is found, an alarm is issued to the monitoring center to achieve the purpose of rapid maintenance. The present invention greatly reduces the amount of calculation of the initial point cloud while retaining the specific outline and defects of the bridge. At the same time, the curvature research method is used to find out the defects hidden in the point cloud, thereby improving the monitoring accuracy.

[0021] Furthermore, the step S10 specifically includes: Acquire initial images of the bridge at different angles through a camera; Based on the intrinsic and extrinsic parameters of the camera, the initial image is reflected in the scale space by Gaussian convolution method; ; Where G represents the Gaussian convolution function, C is the scale space, φ represents the parameter variable of the scale space, (x, y) is the coordinate value of the pixel point in the initial image, * is the convolution factor in the Gaussian convolution function, and U is the initial image; Transforming the scale space of the initial image by using a Gaussian difference method to obtain a Gaussian difference space of the initial image; ; Where k represents the scale factor of the adjacent scale space, and D represents the Gaussian difference function.

[0022] It can be understood that the present application does not directly use lidar scanning to obtain a point cloud, but constructs a three-dimensional model through multiple two-dimensional initial images. Therefore, it is necessary to find feature points and improve adjacent points based on the feature points to construct the model. Specifically, the initial image is mapped to the scale space through a Gaussian convolution operation to make the initial image three-dimensional and obtain the performance of the initial image in different scale spaces. Then, the rate of change of the initial image in the adjacent scale space is obtained through a Gaussian difference operation. The extreme value point of the rate of change is used as the feature point of the initial image, indicating that this is a structural change point or a damage point of the bridge, and both points have mutations.

[0023] Furthermore, the step S20 specifically includes: Performing Taylor expansion on the Gaussian difference space; ; Among them, X represents the original function value corresponding to the extreme point in the initial image, T is the transpose, and ∂ is the sign for partial derivative; Derivative the Gaussian difference function after Taylor expansion to obtain the extreme point X'; ; The extreme points are used as feature points of the initial image.

[0024] It is understandable that the purpose of Taylor expansion of the Gaussian difference function is to fit curves in multiple scale spaces to find the limit, and then capture the extreme points more accurately. Then, the partial derivative of the Gaussian difference function after Taylor expansion is taken, and the extreme points can be found in the continuous scale space after curve fitting, and the mutation occurs between two adjacent scale spaces, and the extreme points are used as feature points.

[0025] Furthermore, the step S30 specifically includes: Rotating the feature points of the initial image toward adjacent pixels using an image rotation formula to obtain multiple gradient directions of the feature points; ; Among them, θ is the rotation angle of the gradient direction, V is the scale space where the extreme point is located, and f is the original function of the initial image; Add the amplitudes of the feature points located in the same gradient direction, select the gradient direction corresponding to the feature point with the highest amplitude as the reference direction, and calculate the position and spatial scale of the feature point located in the reference direction; Taking the position of the feature point as the reference point and the spatial scale of the feature point as the radius, all pixel points within the query range are vectorized with reference to the reference direction, and the vectorized pixel points are used as the region representation of the feature point.

[0026] It can be understood that the extreme points obtained by the Gaussian difference function after Taylor expansion are stable under scaling at any scale. Then, the immobility of the extreme points during rotation is verified. By rotating the extreme points toward adjacent pixels, multiple gradient directions can be obtained, some of which are located on the same straight line. The gradient directions of the feature points will be consistent when they rotate toward the pixels located on the same straight line. By adding the amplitudes of the feature points located in the same gradient direction and selecting the gradient direction corresponding to the feature point with the highest amplitude as the reference direction, this step is to extract the direction with the most pixels as the reference direction to ensure the accuracy and completeness of the data for subsequent point cloud establishment, and use the spatial scale corresponding to the reference direction as the search radius to search for all pixels within the range, and vectorize the pixels as a regional representation. The idea of ​​regional representation is to describe the distribution characteristics of the feature points and pixels at that location through vectorized pixels, in preparation for the subsequent establishment of point clouds.

[0027] Furthermore, the step S40 specifically includes: Based on the SIFT algorithm, comparing the region representations where the same feature points in different initial images are located; A similarity threshold and a distance threshold are set, and the pixel points that meet the similarity threshold and the distance threshold in the region representations of different initial images are matched by the SIFT algorithm, and the pixel points that do not meet the similarity threshold and the distance threshold are removed.

[0028] It can be understood that the SIFT algorithm is an algorithm that transforms and observes images in different environments and at different distances. It can automatically and accurately identify the differences between the regional representations of each part of the initial image and the regional representations of each part of other initial images. By setting the similarity threshold and distance threshold, the SIFT algorithm will automatically retain the pixels that meet both requirements and remove those that do not. It has a fast processing speed. The removed pixels are duplicate pixels that were not considered in the previous algorithm, or pixels that are not feature points and are affected by the direction of the illumination vector.

[0029] Furthermore, point clouds are generated for all pixel points under the retained area representation to obtain an initial point cloud set, and a complete point cloud set is obtained based on a multi-view stereo vision algorithm. This step can be performed by an existing machine algorithm, which can be summarized as follows: first, based on the retained pixel points, the machine algorithm will find the same feature points in different images, and then compare the area representations where the feature points are located, find the paired connection points connecting the two images, and piece together the basic outline of the bridge step by step to form an initial point cloud set. Then, a machine algorithm is used to construct a surface based on the pixel points of the basic outline, and the extended surface is iteratively calculated to generate a complete point cloud set.

[0030] Furthermore, the complete point cloud set is filtered and registered to obtain an optimized point cloud set. This step is to remove noise in the complete point cloud set because filtering is performed, and in order to splice the complete point cloud sets of multiple faces, coarse registration and fine registration are performed.

[0031] Furthermore, the step S70 specifically includes: The outermost point cloud in the optimized point cloud set is used as a point cloud surface, and a simulation shell is established based on the point cloud surface; Using the feature point as an original point on the simulated shell, using the original point as a corner point of a triangle, and using adjacent pixel points of the original point as other corner points of the triangle to generate an initial triangular mesh; Extending the initial triangular mesh outward, gradually generating multiple triangular meshes with the remaining pixel points as corner points, and taking the surface of each triangular mesh as a triangular facet; Based on the Gaussian curvature calculation formula, the Gaussian curvature of the triangular facet extending outward from the original point is calculated, and the Gaussian curvature of each triangular facet is combined to form a Gaussian curvature field; ; ; Among them, P represents Gaussian curvature, i represents the serial number of the original point, j represents the serial number of the pixel point adjacent to the original point, ξ represents the angle of the triangle patch, δ represents the original point, and S represents the area of ​​the triangle patch.

[0032] It can be understood that after obtaining the optimized point cloud, the surface of the optimized point cloud is used as the simulation shell, and then a triangular mesh is randomly established based on a feature point (original point). The other two points of the triangular mesh are the adjacent pixel points of the original point. The triangular mesh is then extended outward to gradually generate all the triangular meshes. The Gaussian curvature of the triangular facets on the surface of the triangular mesh is then calculated using the Gaussian curvature calculation formula. After calculating the Gaussian curvature of all triangular facets, a Gaussian curvature field can be obtained. The Gaussian curvature field reflects the Gaussian curvature in different parts of the bridge.

[0033] Furthermore, the step S80 specifically includes: A Gaussian curvature classification table is preset to statistically classify the curvature of each triangular face in the Gaussian curvature field; In the Gaussian curvature field, with the original point as the center, a search radius is preset, the number of triangles in each Gaussian curvature classification within the search range is calculated, and the number of pixels covered is calculated accordingly as the number of adjacent point clouds; In the Gaussian curvature field, a research surface is preset, the number of triangular facets in each Gaussian curvature classification in the research surface is calculated, and the number of pixel points covered is calculated accordingly as the area density; In the Gaussian curvature field, a research volume is preset, the number of triangular facets in each Gaussian curvature classification in the research volume is calculated, and the number of pixel points covered is calculated accordingly as the spatial density; According to different parts of the bridge, it is studied whether one or more indicators of the number of adjacent point clouds, area density and space density of the part meet the preset requirements. If they do not meet the preset requirements, an alarm is sent to the monitoring center.

[0034] It can be understood that the Gaussian curvature of each triangular face on the bridge can be recorded in a table, and the curvature of the bridge can be reflected by using different shades of colors on the bridge. Then, the three indicators of the number of adjacent point clouds, area density and spatial density at various parts of the bridge are gradually counted. For example, for the bridge deck, the number of adjacent point clouds whose Gaussian curvature exceeds the threshold is 12, the area density is 10, and the spatial density is 30. Since the bridge deck is relatively flat, the area density is used as the main indicator to determine whether an alarm is needed. This can be adjusted based on actual conditions. For bridge piers, the number of adjacent point clouds and spatial density are used as the main indicators to determine whether an alarm is needed.

[0035] In summary, the bridge structure safety early warning method in the above-mentioned embodiment of the present invention greatly reduces the calculation amount of the initial point cloud while retaining the specific outline and defects of the bridge. At the same time, it adopts the curvature research method to find out the defects hidden in the point cloud, thereby improving the monitoring accuracy. Please refer to Figure 2, which is a bridge structure safety early warning system in a second embodiment of the present invention, comprising: Transformation module 11: used to obtain initial images of the bridge at different angles through a camera, and transform the initial images through Gaussian convolution and Gaussian difference to obtain Gaussian difference space of the initial images; The transformation module 11 is specifically used to: obtain initial images of the bridge at different angles through a camera; Based on the intrinsic and extrinsic parameters of the camera, the initial image is reflected in the scale space by Gaussian convolution method; ; Where G represents the Gaussian convolution function, C is the scale space, φ represents the parameter variable of the scale space, (x, y) is the coordinate value of the pixel point in the initial image, * is the convolution factor in the Gaussian convolution function, and U is the initial image; Transforming the scale space of the initial image by using a Gaussian difference method to obtain a Gaussian difference space of the initial image; ; Where k represents the scale factor of the adjacent scale space, and D represents the Gaussian difference function; The obtaining module 12 is used for performing curve fitting on the Gaussian difference space of the initial image to obtain extreme points, and taking the extreme points as feature points of the initial image; The obtaining module 12 is specifically used for: performing Taylor expansion on the Gaussian difference space; ; Among them, X represents the original function value corresponding to the extreme point in the initial image, T is the transpose, and ∂ is the sign for partial derivative; Derivative the Gaussian difference function after Taylor expansion to obtain the extreme point X'; ; Taking the extreme points as feature points of the initial image; Representation module 13: used for rotating the feature points of the initial image toward adjacent pixel points to obtain the gradient direction of the feature points, and obtaining the regional representation of the feature points according to the gradient direction; The representation module 13 is specifically used to: rotate the feature points of the initial image toward adjacent pixel points by using an image rotation formula to obtain multiple gradient directions of the feature points; ; Among them, θ is the rotation angle of the gradient direction, V is the scale space where the extreme point is located, and f is the original function of the initial image; Add the amplitudes of the feature points located in the same gradient direction, select the gradient direction corresponding to the feature point with the highest amplitude as the reference direction, and calculate the position and spatial scale of the feature point located in the reference direction; Taking the position of the feature point as a reference point and the spatial scale of the feature point as a radius, querying all pixel points within a range, vectorizing the pixel points with reference to the reference direction, and using the vectorized pixel points as the region representation of the feature point; Elimination module 14: used for comparing the regional representations of the feature points of different initial images based on a matching algorithm, and eliminating all the pixel points in the regional representations that fail the comparison; The elimination module 14 is specifically used to: compare the region representations where the same feature points are located in different initial images based on the SIFT algorithm; Setting a similarity threshold and a distance threshold, matching the pixel points that meet the similarity threshold and the distance threshold in the region representations of different initial images by the SIFT algorithm, and removing the pixel points that do not meet the similarity threshold and the distance threshold; Generating module 15: used to generate point clouds for all pixel points under the reserved area representation to obtain an initial point cloud set, and obtain a complete point cloud set based on a multi-view stereo vision algorithm; Optimization module 16: used for performing filtering and registration processing on the complete point cloud set to obtain an optimized point cloud set; Calculation module 17: used for triangulating the optimized point cloud set to obtain a Gaussian curvature field of the optimized point cloud set; The calculation module 17 is specifically used to: use the outermost point cloud in the optimized point cloud set as a point cloud surface, and establish a simulated shell based on the point cloud surface; Using the feature point as an original point on the simulated shell, using the original point as a corner point of a triangle, and using adjacent pixel points of the original point as other corner points of the triangle to generate an initial triangular mesh; Extending the initial triangular mesh outward, gradually generating multiple triangular meshes with the remaining pixel points as corner points, and taking the surface of each triangular mesh as a triangular facet; Based on the Gaussian curvature calculation formula, the Gaussian curvature of the triangular facet extending outward from the original point is calculated, and the Gaussian curvature of each triangular facet is combined to form a Gaussian curvature field; ; ; Where P represents Gaussian curvature, i represents the number of the original point, j represents the number of the pixel point adjacent to the original point, ξ represents the angle of the triangle, δ represents the original point, and S represents the area of ​​the triangle. Alarm module 18: used to calculate the number of adjacent point clouds, area density, and spatial density of a single research point in the Gaussian curvature field to find out the point cloud blocks representing bridge defects and to issue an alarm at the monitoring center; The alarm module 18 is specifically used to: preset a Gaussian curvature classification table, and statistically classify the curvature of each triangular facet in the Gaussian curvature field; In the Gaussian curvature field, with the original point as the center, a search radius is preset, the number of triangles in each Gaussian curvature classification within the search range is calculated, and the number of pixels covered is calculated accordingly as the number of adjacent point clouds; In the Gaussian curvature field, a research surface is preset, the number of triangular facets in each Gaussian curvature classification in the research surface is calculated, and the number of pixel points covered is calculated accordingly as the area density; In the Gaussian curvature field, a research volume is preset, the number of triangular facets in each Gaussian curvature classification in the research volume is calculated, and the number of pixel points covered is calculated accordingly as the spatial density; According to different parts of the bridge, it is studied whether one or more indicators of the number of adjacent point clouds, area density and space density of the part meet the preset requirements. If they do not meet the preset requirements, an alarm is sent to the monitoring center.

[0036] The present invention also provides an electronic device, see Figure 3 , shown is an electronic device in the third embodiment of the present invention, including a memory 10, a processor 20, and a computer program 30 stored in the memory 10 and executable on the processor 20, and when the processor 20 executes the computer program 30, the above-mentioned bridge structure safety early warning method is implemented.

[0037] Among them, the memory 10 includes at least one type of storage medium, and the storage medium includes a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 10 can be an internal storage unit of an electronic device, such as a hard disk of the electronic device. In other embodiments, the memory 10 can also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Further, the memory 10 can also include both an internal storage unit of the electronic device and an external storage device. The memory 10 can not only be used to store application software and various types of data installed in the electronic device, but also can be used to temporarily store data that has been output or is to be output.

[0038] Among them, in some embodiments, the processor 20 can be an electronic control unit (Electronic Control Unit, abbreviated as ECU, also known as a vehicle computer), a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, a microprocessor or other data processing chip, used to run the program code stored in the memory 10 or process data, such as executing access restriction programs, etc.

[0039] It should be pointed out that Figure 3 The structure shown does not constitute a limitation on the electronic device. In other embodiments, the electronic device may include fewer or more components than those shown in the figure, or combine certain components, or arrange the components differently.

[0040] The embodiment of the present invention further provides a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the bridge structure safety early warning method as described above is implemented.

[0041] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0042] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0043] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0044] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0045] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A bridge structure safety early warning method, characterized in that: The steps include: Acquire initial images of the bridge at different angles through a camera, and transform the initial images through Gaussian convolution and Gaussian difference to obtain a Gaussian difference space of the initial images; Performing curve fitting on the Gaussian difference space of the initial image to obtain extreme points, and using the extreme points as feature points of the initial image; Rotating the feature points of the initial image toward adjacent pixels to obtain a gradient direction of the feature points, and obtaining a region representation of the feature points according to the gradient direction; Based on a matching algorithm, the regional representations of the feature points of different initial images are compared, and all the pixels in the regional representations that fail the comparison are eliminated; Generating a point cloud for all pixel points under the reserved area representation to obtain an initial point cloud set, and obtaining a complete point cloud set based on a multi-view stereo vision algorithm; Performing filtering and registration processing on the complete point cloud set to obtain an optimized point cloud set; Triangulating the optimized point cloud set to obtain a Gaussian curvature field of the optimized point cloud set; The number of adjacent point clouds, area density, and spatial density of a single research point in the Gaussian curvature field are calculated to find point cloud blocks representing bridge defects and generate an alarm at the monitoring center.

2. The bridge structure safety early warning method according to claim 1 is characterized in that: The method of acquiring initial images of the bridge at different angles through a camera and transforming the initial images through Gaussian convolution and Gaussian difference to obtain a Gaussian difference space of the initial images specifically includes: Acquire initial images of the bridge at different angles through a camera; Based on the intrinsic and extrinsic parameters of the camera, the initial image is reflected in the scale space by Gaussian convolution method; ； Where G represents the Gaussian convolution function, C is the scale space, φ represents the parameter variable of the scale space, (x, y) is the coordinate value of the pixel point in the initial image, * is the convolution factor in the Gaussian convolution function, and U is the initial image; Transforming the scale space of the initial image by using a Gaussian difference method to obtain a Gaussian difference space of the initial image; ； Where k represents the scale factor of the adjacent scale space, and D represents the Gaussian difference function.

3. The bridge structure safety early warning method according to claim 2 is characterized in that: The performing curve fitting on the Gaussian difference space of the initial image to obtain extreme points, and using the extreme points as feature points of the initial image, specifically includes: Performing Taylor expansion on the Gaussian difference space; ； Among them, X represents the original function value corresponding to the extreme point in the initial image, T is the transpose, and ∂ is the sign for partial derivative; Derivative the Gaussian difference function after Taylor expansion to obtain the extreme point X'; ； The extreme points are used as feature points of the initial image.

4. The bridge structure safety early warning method according to claim 3 is characterized in that: The step of rotating the feature points of the initial image toward adjacent pixels to obtain a gradient direction of the feature points, and obtaining a region representation of the feature points according to the gradient direction, specifically includes: Rotating the feature points of the initial image toward adjacent pixels using an image rotation formula to obtain multiple gradient directions of the feature points; ； Among them, θ is the rotation angle of the gradient direction, V is the scale space where the extreme point is located, and f is the original function of the initial image; Add the amplitudes of the feature points located in the same gradient direction, select the gradient direction corresponding to the feature point with the highest amplitude as the reference direction, and calculate the position and spatial scale of the feature point located in the reference direction; Taking the position of the feature point as the reference point and the spatial scale of the feature point as the radius, all pixel points within the query range are vectorized with reference to the reference direction, and the vectorized pixel points are used as the region representation of the feature point.

5. The bridge structure safety early warning method according to claim 4 is characterized in that: The comparing the regional representations of the feature points of the different initial images based on the matching algorithm and removing all the pixel points in the regional representations that fail the comparison specifically includes: Based on the SIFT algorithm, comparing the region representations where the same feature points in different initial images are located; A similarity threshold and a distance threshold are set, and the pixel points that meet the similarity threshold and the distance threshold in the region representations of different initial images are matched by the SIFT algorithm, and the pixel points that do not meet the similarity threshold and the distance threshold are removed.

6. The bridge structure safety early warning method according to claim 5, characterized in that: The step of triangulating the optimized point cloud set to obtain a Gaussian curvature field of the optimized point cloud set specifically includes: The outermost point cloud in the optimized point cloud set is used as a point cloud surface, and a simulation shell is established based on the point cloud surface; Using the feature point as an original point on the simulated shell, using the original point as a corner point of a triangle, and using adjacent pixel points of the original point as other corner points of the triangle to generate an initial triangular mesh; Extending the initial triangular mesh outward, gradually generating multiple triangular meshes with the remaining pixel points as corner points, and taking the surface of each triangular mesh as a triangular facet; Based on the Gaussian curvature calculation formula, the Gaussian curvature of the triangular facet extending outward from the original point is calculated, and the Gaussian curvature of each triangular facet is combined to form a Gaussian curvature field; ； ； Among them, P represents Gaussian curvature, i represents the serial number of the original point, j represents the serial number of the pixel point adjacent to the original point, ξ represents the angle of the triangle patch, δ represents the original point, and S represents the area of ​​the triangle patch.

7. The bridge structure safety early warning method according to claim 6, characterized in that: The calculation of the number of adjacent point clouds, area density, and spatial density of a single research point in the Gaussian curvature field to find point cloud blocks representing bridge defects and to issue an alarm at the monitoring center specifically includes: A Gaussian curvature classification table is preset to statistically classify the curvature of each triangular face in the Gaussian curvature field; In the Gaussian curvature field, with the original point as the center, a search radius is preset, the number of triangles in each Gaussian curvature classification within the search range is calculated, and the number of pixels covered is calculated accordingly as the number of adjacent point clouds; In the Gaussian curvature field, a research surface is preset, the number of triangular facets in each Gaussian curvature classification in the research surface is calculated, and the number of pixel points covered is calculated accordingly as the area density; In the Gaussian curvature field, a research volume is preset, the number of triangular facets in each Gaussian curvature classification in the research volume is calculated, and the number of pixel points covered is calculated accordingly as the spatial density; According to different parts of the bridge, it is studied whether one or more indicators of the number of adjacent point clouds, area density and space density of the part meet the preset requirements. If they do not meet the preset requirements, an alarm is sent to the monitoring center.

8. A bridge structure safety early warning system, characterized in that: include: Transformation module: used to obtain initial images of the bridge at different angles through a camera, and transform the initial images through Gaussian convolution and Gaussian difference to obtain Gaussian difference space of the initial images; An obtaining module; used for performing curve fitting on the Gaussian difference space of the initial image to obtain extreme points, and using the extreme points as feature points of the initial image; A representation module: used for rotating the feature points of the initial image toward adjacent pixel points to obtain the gradient direction of the feature points, and obtaining the regional representation of the feature points according to the gradient direction; Elimination module: used for comparing the regional representations of the feature points of different initial images based on a matching algorithm, and eliminating all the pixels in the regional representations that fail the comparison; A generation module: used to generate a point cloud from all the pixels represented by the retained area to obtain an initial point cloud set, and obtain a complete point cloud set based on a multi-view stereo vision algorithm; Optimization module: used for performing filtering and registration processing on the complete point cloud set to obtain an optimized point cloud set; A calculation module is used to triangulate the optimized point cloud set to obtain a Gaussian curvature field of the optimized point cloud set; Alarm module: used to calculate the number of adjacent point clouds, area density, and spatial density of a single research point in the Gaussian curvature field to find out the point cloud blocks representing bridge defects and issue an alarm in the monitoring center.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the bridge structure safety early warning method as described in any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the bridge structure safety early warning method according to any one of claims 1 to 7 is implemented.

Images