Image data acquisition method, equipment and system for bridge engineering

Abstract

The present invention relates to the field of image processing, and in particular to an image data acquisition method, device and system applied to bridge engineering, wherein the method comprises: obtaining at least one group of images of a target object, each group of images comprising a plurality of images obtained by photographing the target object in multiple directions at the same horizontal height and at a stable shooting distance; for each image in each group of images, obtaining a plurality of target extreme points of the image, and determining a plurality of stable extreme points of the image according to the texture features of each target extreme point; for each group of images, using a SIFT algorithm to perform image stitching on a plurality of images in the group of images based on the stable extreme points of each image in the group of images, and obtaining a panoramic image of the target object at the horizontal height corresponding to the group of images. In this way, the image quality of the panoramic image of the target object can be improved, and a more accurate data basis can be provided for subsequent quality inspection of the target object through the panoramic image.

Description

Technical Field

[0001] The present invention relates to the field of image data processing, and in particular to an image data acquisition method, device and system applied to bridge engineering. Background Art

[0002] In construction projects, buildings need to be inspected regularly to ensure that they can be used safely and normally (for example, bridge pillars need to be inspected regularly in bridge projects). When inspecting the quality of a building, it is usually observed whether there are irregular textures on the surface of the building to determine whether there are cracks or other defects on the surface of the building. Before inspecting the quality of a building, you can use auxiliary means (such as drones) to obtain panoramic images of the building to achieve remote inspection, thereby improving the efficiency of quality inspection.

[0003] Buildings are large three-dimensional structures. When obtaining panoramic images of buildings, existing methods collect images around the building and perform image processing to obtain panoramic images of the building. When taking images of buildings, due to some influences of the shooting environment (for example, some parts may be in a backlight environment, etc.), it is easy to increase the difficulty of processing the captured images, which in turn leads to low quality of the stitched panoramic images, which is not conducive to the subsequent inspection of the quality of the building through the panoramic images of the building. Summary of the invention

[0004] In order to solve the above technical problems, the object of the present invention is to provide an image data acquisition method applied to bridge engineering, comprising:

[0005] Acquire at least one set of images of the target object, wherein each set of images includes a plurality of images obtained by photographing the target object along a plurality of directions at the same horizontal height and at a stable shooting distance;

[0006] For each image in each group of images, a scale-invariant feature transform (SIFT) algorithm is used to obtain multiple target extreme value points of the image, and multiple stable extreme value points of the image are determined from the multiple target extreme value points according to the texture features of each target extreme value point;

[0007] For each group of images, a SIFT algorithm is used to stitch multiple images in the group of images based on the stable extreme points of each image in the group of images to obtain a panoramic image of the target object at a horizontal height corresponding to the group of images.

[0008] Optionally, determining a plurality of stable extreme value points of the image from the plurality of target extreme value points according to the texture feature of each target extreme value point comprises:

[0009] For each target extreme point of the image, a final stability degree of the target extreme point is calculated according to the texture features of the target extreme point and the texture features of other target extreme points in the texture growth region where the target extreme point is located;

[0010] Determine the target extreme point whose final stability degree is greater than the first preset value as the stable extreme point of the image.

[0011] Optionally, the adopting SIFT algorithm to obtain multiple target extreme points of the image includes:

[0012] The image is analyzed using the SIFT algorithm to obtain a plurality of first extreme value points of the image;

[0013] For each first extreme point of the image, the initial stability of the first extreme point is calculated according to the distribution of the first extreme point within the neighborhood of the first extreme point and the contrast between the first extreme point and adjacent pixel points;

[0014] A first extreme point whose initial stability degree is greater than a second preset value is determined as a target extreme point of the image.

[0015] Optionally, for each first extreme point of the image, calculating the initial stability of the first extreme point according to the distribution of the first extreme point in a neighborhood of the first extreme point and the contrast between the first extreme point and adjacent pixels, includes:

[0016] Calculate the distribution stability characteristics of each first extreme value point according to the number of first extreme value points in the neighborhood of each first extreme value point in the difference image and the distance between each first extreme value point in the difference image and other first extreme value points in its neighborhood;

[0017] According to the number of adjacent pixels of each first extreme point in the difference image in the Gaussian difference DOG pyramid, the gray value of each first extreme point and the gray value of its adjacent pixels in the DOG pyramid, the contrast feature of each first extreme point and its adjacent pixels is calculated;

[0018] For each first extreme point of the image, the initial stability of the first extreme point is calculated according to the distribution stability characteristics of the first extreme point and the contrast characteristics between the first extreme point and adjacent pixel points.

[0019] Optionally, for each target extreme value point of the image, calculating the final stability of the target extreme value point according to the texture features of the target extreme value point and the texture features of other target extreme value points in the texture growth region where the target extreme value point is located includes:

[0020] The texture feature of each target extreme point is calculated according to the information entropy of each target extreme point in the differential image in different directions within its texture growth region and the fluctuation degree of different gradient values ​​of each target extreme point in the differential image within its texture growth region;

[0021] For each target extreme point of the image, the final stability of the target extreme point is calculated according to the texture feature similarity between the target extreme point and other target extreme points in its texture growth region in the differential image.

[0022] Optionally, the adopting of SIFT algorithm to analyze the image to obtain a plurality of first extreme points of the image includes:

[0023] The SIFT algorithm is used to analyze the image and obtain multiple extreme points of the image;

[0024] A main body region of the image is identified, and an extreme point located in the main body region is determined as a first extreme point.

[0025] Optionally, the multiple images obtained by photographing the target object along multiple directions include: multiple images obtained by rotationally photographing the target object while rotating along the same rotation direction.

[0026] Optionally, acquiring at least one set of images of the target object includes:

[0027] Acquire at least two sets of images of the target object, wherein the at least two sets of images are images obtained by photographing the target object at different levels;

[0028] After the method uses the SIFT algorithm to stitch multiple images in each group of images based on the stable extreme points of each image in the group of images to obtain a panoramic image of the target object at a horizontal height corresponding to the group of images, the method further includes:

[0029] The panoramic images at the horizontal heights respectively corresponding to the at least two groups of images are spliced ​​along the height direction to obtain an overall panoramic image of the target object.

[0030] The present invention also provides an image data acquisition device applied to bridge engineering, comprising a memory, at least one processor and at least one program stored in the memory and executable by the at least one processor. When the at least one program is executed by the at least one processor, the steps in any one of the above-mentioned image data acquisition methods applied to bridge engineering are implemented.

[0031] The present invention also provides an image data acquisition system applied to bridge engineering, the system comprising:

[0032] An acquisition module, used to acquire at least one set of images of a target object, wherein each set of images includes a plurality of images obtained by photographing the target object along multiple directions at the same horizontal height and at a stable shooting distance;

[0033] An analysis module, for obtaining multiple target extreme value points of each image in each group of images by using a scale-invariant feature transform (SIFT) algorithm, and determining multiple stable extreme value points of the image from the multiple target extreme value points according to a texture feature of each target extreme value point;

[0034] The stitching module is used to stitch multiple images in each group of images using the SIFT algorithm based on the stable extreme points of each image in the group of images to obtain a panoramic image of the target object at a horizontal height corresponding to the group of images.

[0035] The present invention has the following beneficial effects: when the panoramic image of the target object is obtained by using the SIFT algorithm, the texture features of the extreme points are analyzed to screen out stable extreme points, and the image is stitched based on the screened stable extreme points to obtain a panoramic image. The adverse effects of the shooting environment on the panoramic image can be effectively reduced, thereby improving the image quality of the panoramic image of the target object, and providing a more accurate data basis for subsequent quality inspection of the target object through the panoramic image. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A first schematic diagram of an image data acquisition method applied to bridge engineering provided by an embodiment of the present invention;

[0038] Figure 2 A second schematic diagram of the image data acquisition method applied to bridge engineering provided by an embodiment of the present invention;

[0039] Figure 3 A schematic diagram of an image data acquisition device applied to bridge engineering provided by an embodiment of the present invention;

[0040] Figure 4 A schematic diagram of an image data acquisition system applied to bridge engineering provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the image data acquisition method for bridge engineering proposed by the present invention, its specific implementation method, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0042] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0043] The present invention provides an image data acquisition method applied to bridge engineering. The scheme of the present invention is described in detail below in conjunction with the accompanying drawings.

[0044] See also Figure 1 , which shows a first schematic diagram of an image data acquisition method applied to bridge engineering provided by an embodiment of the present invention, the image data acquisition method applied to bridge engineering (hereinafter referred to as "the method") comprises the following steps:

[0045] Step 101: Acquire at least one set of images of a target object, wherein each set of images includes a plurality of images obtained by photographing the target object along multiple directions at the same horizontal height and at a stable shooting distance.

[0046] In this step, the method obtains at least one set of images of the target object, wherein each set of images includes multiple images obtained by photographing the target object along multiple directions at the same horizontal height and at a stable shooting distance. In bridge engineering, the target object may be a bridge pillar. It is understandable that the image data acquisition method applied to bridge engineering of the present invention may also be applied to image data acquisition of other projects, such as other construction projects, or image data acquisition of other objects, and the present invention does not limit this.

[0047] At least one group of images of the target object may be images taken by a drone equipped with a high-definition camera, or may be images taken manually by using a camera. The following describes the shooting process in detail by taking a drone equipped with a high-definition camera as an example. When shooting a group of images of the surface of the target object, the drone should try to maintain the same horizontal height when moving, and keep the relative distance between the drone and the target object stable, and shoot the target object along multiple different directions to obtain a group of images of the target object. In the case where the overall height of the target object is not high, the drone can be used to shoot the image of the target object at a horizontal height to obtain a group of images; in the case where the overall height of the target object is high, the drone can be used to shoot the image of the target object at multiple different horizontal heights to obtain multiple groups of images, each group of images includes multiple images shot at the same horizontal height. The shooting of the target object in multiple different directions can be to select a few fixed directions around the target object to shoot the target object, or the drone can be rotated along the same rotation direction to shoot. For example, in the case where the target object is a bridge pillar, the drone can rotate counterclockwise around the bridge pillar to shoot, and 4 or more images can be shot at the same horizontal height. In some specific embodiments, a drone equipped with a high-definition CCD (Charge-coupled Device) camera may be used for shooting.

[0048] The method may acquire at least one set of images of the target object by directly receiving at least one set of images of the target object sent from other devices, or the method may control a drone to take pictures and then receive at least one set of images of the target object transmitted from the drone. This embodiment of the present invention does not specifically limit this.

[0049] Step 102: for each image in each group of images, a SIFT (Scale-invariant feature transform) algorithm is used to obtain a plurality of target extreme value points of the image, and a plurality of stable extreme value points of the image are determined from the plurality of target extreme value points according to the texture features of each target extreme value point.

[0050] In order to better understand the present invention, before describing this step, the SIFT algorithm is briefly described here. The SIFT algorithm is a machine vision algorithm, which mainly detects the scale space extreme value by establishing a Gaussian pyramid, establishing a DOG (Difference of Gaussian) pyramid, and DOG local extreme value detection, thereby obtaining the extreme point of the image.

[0051] Detecting the extreme value of scale space is to search for the image position on all scales, and identify the points of interest that are invariant to scale and rotation through Gaussian differential functions. The scale space of an image is defined as the convolution of a Gaussian function of varying scales and the original image. The scale space can be represented by constructing a Gaussian pyramid when it is implemented. The construction of the Gaussian pyramid includes downsampling the image (sampling at every other point) and Gaussian blurring the image at different scales. The pyramid model of an image refers to a pyramid-shaped model that is formed from large to small and from bottom to top by continuously downsampling the original image to obtain a series of images of different sizes. The original image is the first layer of the pyramid, and the new image obtained by each downsampling is the upper layer of the pyramid (one image per layer). The number of layers of the pyramid is determined by the size of the bottom image and the size of the top image. In order to make the scale reflect its continuity, the Gaussian pyramid adds Gaussian filtering on the basis of simple downsampling. An image in each layer of the pyramid is Gaussian blurred using multiple different parameters to obtain multiple pyramids corresponding to multiple different parameters. In other words, the Gaussian pyramid is not just one pyramid, but includes many groups of pyramids, each of which includes several layers of images processed by Gaussian blur.

[0052] The DOG pyramid is constructed on the basis of the Gaussian pyramid. The i-th group of the h-th layer image of the DOG pyramid is obtained by subtracting the i-th group of the h-th layer image from the i-th group of the Gaussian pyramid. For example, for one group of pyramids in the Gaussian pyramid, the first layer of the corresponding DOG pyramid is obtained by subtracting the first layer from the second layer of the pyramid in this group. By analogy, this group of pyramids generates a differential image layer by layer, and all the differential images obtained constitute the DOG pyramid. In terms of the number of layers in each group, the DOG pyramid has one less layer than the Gaussian pyramid. In the constructed DOG pyramid, the SIFT algorithm detects extreme points by comparing each pixel with its neighboring pixels in the image domain and scale domain. These extreme points are feature points that are significant at different scales.

[0053] In this step, for each image in each group of images, the method uses the SIFT algorithm to obtain multiple target extreme value points of the image, and determines multiple stable extreme value points of the image from the multiple target extreme value points according to the texture features of each target extreme value point. The multiple target extreme value points may be all extreme value points of the image directly obtained by using the SIFT algorithm, or may be extreme value points obtained after preliminary screening of all extreme value points of the image, wherein the preliminary screening may include screening extreme value points located in the main area of ​​the image as target extreme value points, and / or screening extreme value points whose initial stability meets specific conditions among all extreme value points as target extreme value points.

[0054] For each image, the method can calculate the final stability of each target extreme point based on the texture features of all target extreme points in the image, and then determine the target extreme point whose final stability is greater than the first preset value as the stable extreme point of the image. Specifically, for each target extreme point in the image, the method can calculate the final stability of the target extreme point based on the texture features of the target extreme point and the texture features of other target extreme points in the texture growth area where the target extreme point is located.

[0055] Step 103 : for each group of images, a SIFT algorithm is used to stitch multiple images in the group of images based on the stable extreme points of each image in the group of images to obtain a panoramic image of the target object at a horizontal height corresponding to the group of images.

[0056] After obtaining the stable extreme points of each image, in this step, for each group of images, the method uses the SIFT algorithm to stitch multiple images in the group of images based on the stable extreme points of each image in the group of images to obtain a panoramic image of the target object at the horizontal height corresponding to the group of images. Specifically, the method can use the SIFT algorithm to assign directions to the stable extreme points, and obtain feature descriptors of the stable extreme points for matching and stitching to obtain the corresponding panoramic image. Image matching and stitching based on extreme points using the SIFT algorithm belongs to the scope of the prior art and will not be described in detail here.

[0057] When multiple groups of images of the target object are acquired, the method may stitch each group of images separately to obtain a panoramic image corresponding to each group of images, and then stitch the panoramic images corresponding to the multiple groups of images in the height direction to obtain an overall panoramic image of the target object.

[0058] In an embodiment of the present invention, the method obtains at least one group of images of the target object, wherein each group of images includes multiple images obtained by photographing the target object in multiple directions at the same horizontal height and at a stable shooting distance; for each image in each group of images, a scale-invariant feature transformation SIFT algorithm is used to obtain multiple target extreme points of the image, and multiple stable extreme points of the image are determined from the multiple target extreme points according to the texture features of each target extreme point; for each group of images, a SIFT algorithm is used to perform image stitching on multiple images in the group of images based on the stable extreme points of each image in the group of images to obtain a panoramic image of the target object at the horizontal height corresponding to the group of images. In this way, when the panoramic image of the target object is obtained by using the SIFT algorithm, the texture features of the extreme points are analyzed to screen out the stable extreme points, and the images are stitched based on the screened stable extreme points to obtain the panoramic image. The adverse effects of the shooting environment on the panoramic image imaging can be effectively reduced, thereby improving the image quality of the panoramic image of the target object, and providing a more accurate data basis for subsequent quality inspection of the target object through the panoramic image.

[0059] Optionally, the method of determining multiple stable extreme points of the image from the multiple target extreme points based on the texture features of each target extreme point includes: for each target extreme point of the image, calculating the final stability of the target extreme point based on the texture features of the target extreme point and the texture features of other target extreme points in the texture growth area where the target extreme point is located; and determining the target extreme point whose final stability is greater than a first preset value as the stable extreme point of the image.

[0060] There are usually some textures on the surface of the target object. When the extreme point is at the junction of the hierarchical texture of the target object, the more significant its features are, the more likely it is to be a stable extreme point. Therefore, the stable extreme point can be screened by analyzing the texture features of the target extreme point. For example, when the target object is a bridge pillar, the bridge pillar usually shows a certain sense of hierarchy during construction. For example, the splicing marks of the prefabricated parts of the bridge pillar will produce obvious hierarchical textures, and generally appear rectangular in the bridge pillar image, which is relatively regular. When the extreme point is at the junction of the hierarchical texture of the bridge pillar, the more significant its features are, the more likely it is to be a stable extreme point. And the junction of the hierarchical texture of the bridge pillar is generally an area with complex textures, so the area with complex textures can effectively provide stable extreme points.

[0061] In this embodiment, for each target extreme point in the image, the method calculates the final stability of the target extreme point based on the texture features of the target extreme point and the texture features of other target extreme points in the texture growth area where the target extreme point is located, and then determines the target extreme point whose final stability is greater than the first preset value as the stable extreme point of the image. In this way, the accuracy of screening stable extreme points can be effectively improved by analyzing the texture features of the target extreme points.

[0062] Please read further Figure 2 , Figure 2 is a second schematic diagram of the image data acquisition method applied to bridge engineering provided by an embodiment of the present invention, such as Figure 2 As shown, the method comprises the following steps:

[0063] Step 201: Acquire at least one set of images of a target object, wherein each set of images includes a plurality of images obtained by photographing the target object along multiple directions at the same horizontal height and at a stable shooting distance.

[0064] The step 201 and Figure 1 Step 101 in the illustrated embodiment is the same and will not be described again here.

[0065] Step 202: For each image in each group of images, use the SIFT algorithm to analyze the image to obtain a plurality of first extreme points of the image.

[0066] In this step, for each image in each group of images, the method uses the SIFT algorithm to analyze the image to obtain multiple first extreme points of the image. The multiple first extreme points can be all extreme points of the image directly obtained by the SIFT algorithm, or extreme points located in the main area of ​​the image. The method of obtaining extreme points of an image by using the SIFT algorithm belongs to the scope of the prior art and has been Figure 1 The step 102 in the illustrated embodiment has been briefly described and will not be described in detail here.

[0067] In the case where the multiple first extreme points are extreme points located in the main area of ​​the image, the method can identify the image, identify the main area of ​​the image, and determine the extreme points located in the main area as the first extreme points. Specifically, the image can be processed by a semantic segmentation network to identify the main area and background area of ​​the image, and then the extreme points belonging to the background area are removed, and the multiple extreme points located in the main area are determined as the multiple first extreme points of the image. For example, the semantic segmentation network can be a pyramid scene parsing network - PSPNet (PyramidScene Parsing Network, a deep convolutional neural network model for image semantic segmentation).

[0068] Step 203: For each first extreme point of the image, the initial stability of the first extreme point is calculated according to the distribution of the first extreme point in the neighborhood of the first extreme point and the contrast between the first extreme point and adjacent pixel points.

[0069] The target object may be affected by light to a greater or lesser extent when photographing, which may easily lead to too many extreme points in the image of the target object, increasing the redundancy of the extreme points. The surface structure of the target object is generally similar, and there will be relatively regular splicing traces. Therefore, the distribution of extreme points at different positions should be relatively uniform. If some extreme points are unevenly distributed, there may be extreme points affected by light among these extreme points. In addition, for stable extreme points, the contrast difference between them and other pixel points in the neighborhood will also be relatively large. The greater the contrast difference, the more significant the extreme point is in the differential image, and it is more likely to be a stable feature point, and it is less likely to be lost due to noise interference. Therefore, the extreme points can be preliminarily screened according to the distribution of extreme points and the contrast between extreme points and adjacent pixels.

[0070] In this step, for each first extreme point of the image, the method calculates the initial stability of the first extreme point according to the distribution of the first extreme point in the neighborhood of the first extreme point and the contrast between the first extreme point and the adjacent pixels. The neighborhood of the extreme point is the area within a preset range centered on the extreme point in the differential image. For example, for a length of , the width is The neighborhood range of a certain extreme point in the difference image of can be the extreme point centered in the difference image. The adjacent pixels of the extreme point can be the adjacent pixels around the extreme point in the DOG pyramid.

[0071] Step 204: Determine a first extreme point whose initial stability is greater than a second preset value as a target extreme point of the image.

[0072] In this step, the method selects a first extreme point whose initial stability is greater than a second preset value as a target extreme point of the image. For example, the second preset value may be 0.8.

[0073] Step 205: For each target extreme value point of the image, the final stability of the target extreme value point is calculated according to the texture features of the target extreme value point and the texture features of other target extreme value points in the texture growth region where the target extreme value point is located.

[0074] Since extreme points are mainly corner points, edge points, bright spots in dark areas or dark spots in volume areas on the surface of the target object, the above steps 203 and 204 preliminarily screen the extreme points according to the distribution of the extreme points and the contrast between the extreme points and the adjacent pixels, but do not fully consider the overall structural information of the target object. For further optimization, further analysis can be considered in combination with the texture information and shape characteristics of the surface of the target object.

[0075] There are usually some textures on the surface of the target object. When the extreme point is at the junction of the hierarchical texture of the target object, the more significant its features are, the more likely it is to be a stable extreme point. Therefore, the stable extreme point can be screened by analyzing the texture features of the target extreme point. For example, taking the target object as a bridge pillar, the bridge pillar usually shows a certain sense of hierarchy during construction, such as the splicing marks of the bridge pillar prefabricated parts will produce obvious hierarchical textures, and generally appear rectangular in the bridge pillar image, which is relatively regular. When the extreme point is at the junction of the hierarchical texture of the bridge pillar, the more significant its features are, the more likely it is to be a stable extreme point. And the junction of the hierarchical texture of the bridge pillar is generally a complex texture area, so the complex texture area can effectively provide a stable extreme point. Therefore, in this step, for each target extreme point in the image, the method calculates the final stability of the target extreme point based on the texture features of the target extreme point and the texture features of other target extreme points in the texture growth area where it is located.

[0076] Step 206: Determine the target extreme point whose final stability degree is greater than the first preset value as the stable extreme point of the image.

[0077] In this step, the method selects the target extreme point whose final stability is greater than the first preset value as the stable extreme point of the image. For example, the first preset value may be 0.8. It should be noted that the first preset value and the second preset value may be the same preset value or different preset values, which is not specifically limited in the embodiment of the present invention.

[0078] Step 207: for each group of images, use the SIFT algorithm to stitch multiple images in the group of images based on the stable extreme points of each image in the group of images to obtain a panoramic image of the target object at a horizontal height corresponding to the group of images.

[0079] The step 207 and Figure 1 Step 103 in the illustrated embodiment is the same and will not be described again here.

[0080] In this embodiment, when the SIFT algorithm is used to obtain a panoramic image of the target object, the extreme points are preliminarily screened according to their distribution and the contrast between the extreme points and the adjacent pixels, and then the texture features of the extreme points are analyzed to further screen out stable extreme points from the preliminary screening structure, and image stitching is performed based on the screened stable extreme points to obtain a panoramic image. In this way, multiple factors such as the distribution of extreme points, the contrast between extreme points and adjacent pixels, and the texture features of the target object itself are fully considered, the accuracy and rationality of screening stable extreme points are improved, the adverse effects of the shooting environment on the panoramic image can be effectively reduced, and the image quality of the panoramic image of the target object is improved, thereby providing a more accurate data basis for subsequent quality inspection of the target object through the panoramic image.

[0081] Optionally, for each first extreme point of the image, the initial stability of the first extreme point is calculated according to the distribution of the first extreme point in the neighborhood of the first extreme point and the contrast between the first extreme point and adjacent pixels, including: calculating the distribution stability characteristics of each first extreme point according to the number of first extreme points in the neighborhood of each first extreme point in the differential image and the distance between each first extreme point in the differential image and other first extreme points in its neighborhood; calculating the contrast characteristics of each first extreme point and adjacent pixels according to the number of adjacent pixels of each first extreme point in the differential image in the Gaussian difference DOG pyramid, the grayscale value of each first extreme point and the grayscale value of its adjacent pixels in the DOG pyramid; for each first extreme point of the image, the initial stability of the first extreme point is calculated according to the distribution stability characteristics of the first extreme point and the contrast characteristics of the first extreme point and adjacent pixels.

[0082] In this embodiment, the method can calculate the distribution stability characteristics of each first extreme point according to the number of first extreme points in the neighborhood of each first extreme point in the differential image and the distance between each first extreme point in the differential image and other first extreme points in its neighborhood. The first extreme point can be calculated by the following formula to calculate its distribution stability characteristics :

[0083]

[0084] in, is the first The distribution stability characteristics of the first extreme points; is the first The number of first extreme points in the neighborhood of the first extreme point; is the mean value of the number of first extreme value points in the neighborhood of all first extreme value points in the differential image; is the first The average distance between the first extreme point and other first extreme points in the neighborhood; is the mean of the distances between all first extreme points in the differential image and other first extreme points in the neighborhood; is a linear normalization function used for normalization processing.

[0085] The more evenly the first extreme point is distributed in the differential image, the higher the distribution stability characteristic of the first extreme point is. In the above formula, Indicates the difference image The difference between the distribution density of the first extreme point in the neighborhood of the first extreme point and the average distribution density of the first extreme points in the neighborhood of other first extreme points in the differential image is smaller, indicating that the first extreme point is The less the first extreme point is affected by the environment, the Distribution stability characteristics of the first extreme point The higher; The difference image is The difference between the distribution distance of the first extreme point in the neighborhood of the first extreme point and the average distribution distance of the first extreme points in the neighborhood of other first extreme points in the differential image. The smaller the difference in distribution distance, the greater the difference in the distribution distance. The less the first extreme point is affected by the environment, the Distribution stability characteristics of the first extreme point The higher.

[0086] The method can calculate the contrast characteristics of each first extreme point and its adjacent pixels according to the number of adjacent pixels of each first extreme point in the difference image in the Gaussian difference DOG pyramid, the gray value of each first extreme point and the gray value of its adjacent pixels in the DOG pyramid. The first extreme point can be used to calculate the contrast characteristics between it and the adjacent pixels by the following formula :

[0087]

[0088] in, is the first The contrast characteristics of the first extreme point and the adjacent pixel points; is the first The number of all adjacent pixels of the first extreme point in the DOG pyramid; is the first The gray value of the first extreme point; is the first The first extreme point in the DOG pyramid is adjacent to the The gray value of each pixel; is a linear normalization function used for normalization processing.

[0089] Indicates the difference image The contrast difference between the first extreme point and its adjacent pixels in the DOG pyramid is analyzed by analyzing the average contrast difference of all adjacent pixels in the DOG pyramid to obtain the first extreme point in the differential image. The contrast feature of the first extreme point and the adjacent pixel points, the larger the contrast feature, the The more significant the first extreme point is in the image, the more likely it is to be a stable feature point and the less likely it is to be lost due to noise interference.

[0090] For each first extreme point of the image, the method calculates the initial stability of the first extreme point according to the distribution stability characteristics of the first extreme point and the contrast characteristics between the first extreme point and the adjacent pixel points. Distribution stability characteristics of the first extreme point and The contrast characteristics of the first extreme point and the adjacent pixels Then, the method can calculate the first The initial stability of the first extreme point :

[0091]

[0092] in, is the first The initial stability of the first extreme point; is the first The distribution stability characteristics of the first extreme points; is the first The contrast characteristics of the first extreme point and the adjacent pixel points; is a linear normalization function used for normalization processing.

[0093] Optionally, for each target extreme point of the image, the final stability of the target extreme point is calculated based on the texture features of the target extreme point and the texture features of other target extreme points in its texture growth region, including: calculating the texture features of each target extreme point based on the information entropy of each target extreme point in the differential image in different directions within its texture growth region and the degree of fluctuation of different gradient values ​​of each target extreme point in the differential image within its texture growth region; for each target extreme point of the image, calculating the final stability of the target extreme point based on the similarity of texture features between the target extreme point in the differential image and other target extreme points in its texture growth region.

[0094] For any target extreme point, in the differential image where the target extreme point is located, the target extreme point is used as the starting point for regional growth, and the growth criterion is that the gradient difference between the starting point and the neighborhood point is less than a preset threshold (for example, 20), and the texture growth region of the target extreme point is obtained. The angle between the line connecting each pixel point and the initial point in the texture growth region and the horizontal line is recorded. In some embodiments of the present invention, the angle can be defined as the angle formed by the direction of 0 degrees to the right horizontally and the counterclockwise direction as the positive direction. Of course, this is not limited to this. In other embodiments, the angle can also be defined as the angle in other directions, as long as all angles are determined by the same definition standard. For example, the angle can be defined as the angle formed by the direction of 0 degrees to the left horizontally and the counterclockwise direction as the positive direction.

[0095] In this embodiment, the method can calculate the texture features of each target extreme point in the differential image based on the information entropy of each target extreme point in different directions in its texture growth region and the degree of fluctuation of different gradient values ​​of each target extreme point in the differential image in its texture growth region. The target extreme point can be calculated by the following formula to calculate its texture feature :

[0096]

[0097] in, is the first Texture features of target extreme points; is the first The number of different angles of pixels within the texture growth area of ​​the target extreme point; is the first The texture growth area of ​​the target extreme point The frequency of occurrence of angle values; is a logarithmic function with base 2; is the first The number of different gradient values ​​of pixels in the texture growth area of ​​the target extreme point; is the first The texture growth area of ​​the target extreme point The frequency of occurrence of gradient values; is the first The texture growth area of ​​the target extreme point Gradient value; is the first The mean of the gradient values ​​within the texture growth region at the target extreme point.

[0098] In the above formula, Indicates the difference image The information entropy of the target extreme point in different directions within its texture growth area is smaller, indicating that The distribution direction of the pixels in the texture growth area of ​​the target extreme point is relatively regular. The more regular the texture, the greater the texture feature. Indicates the difference image The fluctuation degree of different gradient values ​​of a target extreme point in its texture growth area. The smaller the fluctuation degree, the better the texture information performance of the texture growth area and the greater the texture feature.

[0099] In this embodiment, for each target extreme point, the method can calculate the final stability of the target extreme point based on the texture feature similarity between the target extreme point in the differential image and other target extreme points in its texture growth region. The final stability of the target extreme point can be calculated by the following formula :

[0100]

[0101] in, is the first The final stability of the target extreme point; is the first The number of other target extreme points within the texture growth area of ​​the target extreme point; is the first Texture features of target extreme points; is the first The texture growth area of ​​the target extreme point Texture features of target extreme points; is an exponential function with a natural constant as its base.

[0102] In the above formula, Indicates the difference image The texture feature similarity between a target extreme point and other target extreme points in the texture growth area is calculated. The greater the similarity, the more stable the target extreme point.

[0103] Optionally, the adopting SIFT algorithm to analyze the image to obtain multiple first extreme points of the image includes: adopting SIFT algorithm to analyze the image to obtain multiple extreme points of the image; identifying the main area of ​​the image, and determining the extreme points located in the main area as the first extreme points.

[0104] After using the SIFT algorithm to detect the extreme points of the image, since the detected extreme points are distributed throughout the entire image, the subsequent calculation amount will be large and the mismatch rate will be high. In fact, for image stitching, the main area in the image is more important. Therefore, in this embodiment, the method identifies the image, identifies the main area of ​​the image, and determines the extreme point located in the main area as the first extreme point. Specifically, the image can be processed by a semantic segmentation network to identify the main area and background area of ​​the image, and then the extreme points belonging to the background area are removed, and the multiple extreme points located in the main area are determined as multiple first extreme points of the image. For example, the semantic segmentation network can be a pyramid scene parsing network-PSPNet (Pyramid Scene Parsing Network, a deep convolutional neural network model for image semantic segmentation).

[0105] In this way, by performing semantic segmentation on the image and obtaining the extreme points in the main area of ​​the image as the first extreme points of the image, the interference of the extreme points in the background area of ​​the image on subsequent processing can be reduced, providing a more accurate data basis for the subsequent screening of stable extreme points.

[0106] Optionally, the multiple images obtained by photographing the target object along multiple directions include: multiple images obtained by rotationally photographing the target object while rotating along the same rotation direction.

[0107] Optionally, acquiring at least one group of images of the target object includes: acquiring at least two groups of images of the target object, wherein the at least two groups of images are images obtained by photographing the target object at different horizontal heights; for each group of images, using a SIFT algorithm to stitch multiple images in the group of images based on stable extreme points of each image in the group of images to obtain a panoramic image of the target object at a horizontal height corresponding to the group of images, the method further includes: stitching the panoramic images at the horizontal heights corresponding to the at least two groups of images along a height direction to obtain an overall panoramic image of the target object.

[0108] The embodiment of the present invention also provides an image data acquisition device for bridge engineering, please refer to Figure 3 The image data acquisition device 300 applied to bridge engineering includes a memory 301, at least one processor 302, and at least one program stored in the memory 301 and executable by the at least one processor 302. When the at least one program is executed by the at least one processor 302, the following steps are implemented:

[0109] Acquire at least one set of images of the target object, wherein each set of images includes a plurality of images obtained by photographing the target object along a plurality of directions at the same horizontal height and at a stable shooting distance;

[0110] For each image in each group of images, a scale-invariant feature transform (SIFT) algorithm is used to obtain multiple target extreme value points of the image, and multiple stable extreme value points of the image are determined from the multiple target extreme value points according to the texture features of each target extreme value point;

[0111] For each group of images, a SIFT algorithm is used to stitch multiple images in the group of images based on the stable extreme points of each image in the group of images to obtain a panoramic image of the target object at a horizontal height corresponding to the group of images.

[0112] Optionally, determining a plurality of stable extreme value points of the image from the plurality of target extreme value points according to the texture feature of each target extreme value point comprises:

[0113] For each target extreme point of the image, a final stability degree of the target extreme point is calculated according to the texture features of the target extreme point and the texture features of other target extreme points in the texture growth region where the target extreme point is located;

[0114] Determine the target extreme point whose final stability degree is greater than the first preset value as the stable extreme point of the image.

[0115] Optionally, the adopting SIFT algorithm to obtain multiple target extreme points of the image includes:

[0116] The image is analyzed using the SIFT algorithm to obtain a plurality of first extreme value points of the image;

[0117] For each first extreme point of the image, the initial stability of the first extreme point is calculated according to the distribution of the first extreme point within the neighborhood of the first extreme point and the contrast between the first extreme point and adjacent pixel points;

[0118] A first extreme point whose initial stability degree is greater than a second preset value is determined as a target extreme point of the image.

[0119] Optionally, for each first extreme point of the image, calculating the initial stability of the first extreme point according to the distribution of the first extreme point in a neighborhood of the first extreme point and the contrast between the first extreme point and adjacent pixels, includes:

[0120] Calculate the distribution stability characteristics of each first extreme value point according to the number of first extreme value points in the neighborhood of each first extreme value point in the difference image and the distance between each first extreme value point in the difference image and other first extreme value points in its neighborhood;

[0121] According to the number of adjacent pixels of each first extreme point in the difference image in the Gaussian difference DOG pyramid, the gray value of each first extreme point and the gray value of its adjacent pixels in the DOG pyramid, the contrast feature of each first extreme point and its adjacent pixels is calculated;

[0122] For each first extreme point of the image, the initial stability of the first extreme point is calculated according to the distribution stability characteristics of the first extreme point and the contrast characteristics between the first extreme point and adjacent pixel points.

[0123] Optionally, for each target extreme value point of the image, calculating the final stability of the target extreme value point according to the texture features of the target extreme value point and the texture features of other target extreme value points in the texture growth region where the target extreme value point is located includes:

[0124] The texture feature of each target extreme point is calculated according to the information entropy of each target extreme point in the differential image in different directions within its texture growth region and the fluctuation degree of different gradient values ​​of each target extreme point in the differential image within its texture growth region;

[0125] For each target extreme point of the image, the final stability of the target extreme point is calculated according to the texture feature similarity between the target extreme point and other target extreme points in its texture growth region in the differential image.

[0126] Optionally, the adopting of SIFT algorithm to analyze the image to obtain a plurality of first extreme points of the image includes:

[0127] The SIFT algorithm is used to analyze the image and obtain multiple extreme points of the image;

[0128] A main body region of the image is identified, and an extreme point located in the main body region is determined as a first extreme point.

[0129] Optionally, the multiple images obtained by photographing the target object along multiple directions include: multiple images obtained by rotationally photographing the target object while rotating along the same rotation direction.

[0130] Optionally, acquiring at least one set of images of the target object includes:

[0131] Acquire at least two sets of images of the target object, wherein the at least two sets of images are images obtained by photographing the target object at different levels;

[0132] After the method uses the SIFT algorithm to stitch multiple images in each group of images based on the stable extreme points of each image in the group of images to obtain a panoramic image of the target object at a horizontal height corresponding to the group of images, the method further includes:

[0133] The panoramic images at the horizontal heights respectively corresponding to the at least two groups of images are spliced ​​along the height direction to obtain an overall panoramic image of the target object.

[0134] In this embodiment, when the image data acquisition device used in bridge engineering uses the SIFT algorithm to obtain a panoramic image of a target object, the texture features of the extreme points are analyzed to screen out stable extreme points, and image stitching is performed based on the screened stable extreme points to obtain a panoramic image. This can effectively reduce the adverse effects of the shooting environment on panoramic image imaging, thereby improving the image quality of the panoramic image of the target object, and providing a more accurate data basis for subsequent quality inspection of the target object through the panoramic image.

[0135] The embodiment of the present invention also provides an image data acquisition system for bridge engineering, please refer to Figure 4 , the image data acquisition system 400 applied to bridge engineering includes:

[0136] An acquisition module 401 is used to acquire at least one set of images of a target object, wherein each set of images includes a plurality of images obtained by photographing the target object along multiple directions at the same horizontal height and at a stable shooting distance;

[0137] An analysis module 402 is used to obtain multiple target extreme value points of each image in each group of images by using a scale-invariant feature transform (SIFT) algorithm, and determine multiple stable extreme value points of the image from the multiple target extreme value points according to the texture feature of each target extreme value point;

[0138] The stitching module 403 is used to stitch multiple images in each group of images using the SIFT algorithm based on the stable extreme points of each image in the group of images to obtain a panoramic image of the target object at a horizontal height corresponding to the group of images.

[0139] Optionally, the analysis module 402 determines a plurality of stable extreme value points of the image from the plurality of target extreme value points according to the texture feature of each target extreme value point, including:

[0140] For each target extreme point of the image, a final stability degree of the target extreme point is calculated according to the texture features of the target extreme point and the texture features of other target extreme points in the texture growth region where the target extreme point is located;

[0141] Determine the target extreme point whose final stability degree is greater than the first preset value as the stable extreme point of the image.

[0142] Optionally, the analysis module 402 uses a SIFT algorithm to obtain multiple target extreme points of the image, including:

[0143] The image is analyzed using the SIFT algorithm to obtain a plurality of first extreme value points of the image;

[0144] For each first extreme point of the image, the initial stability of the first extreme point is calculated according to the distribution of the first extreme point within the neighborhood of the first extreme point and the contrast between the first extreme point and adjacent pixel points;

[0145] A first extreme point whose initial stability degree is greater than a second preset value is determined as a target extreme point of the image.

[0146] Optionally, the analysis module 402 calculates, for each first extreme point of the image, the initial stability of the first extreme point according to the distribution of the first extreme point in a neighborhood of the first extreme point and the contrast between the first extreme point and adjacent pixels, including:

[0147] Calculate the distribution stability characteristics of each first extreme value point according to the number of first extreme value points in the neighborhood of each first extreme value point in the difference image and the distance between each first extreme value point in the difference image and other first extreme value points in its neighborhood;

[0148] According to the number of adjacent pixels of each first extreme point in the difference image in the Gaussian difference DOG pyramid, the gray value of each first extreme point and the gray value of its adjacent pixels in the DOG pyramid, the contrast feature of each first extreme point and its adjacent pixels is calculated;

[0149] For each first extreme point of the image, the initial stability of the first extreme point is calculated according to the distribution stability characteristics of the first extreme point and the contrast characteristics between the first extreme point and adjacent pixel points.

[0150] Optionally, the analysis module 402 calculates the final stability of each target extreme point of the image according to the texture features of the target extreme point and the texture features of other target extreme points in the texture growth region where the target extreme point is located, including:

[0151] The texture feature of each target extreme point is calculated according to the information entropy of each target extreme point in the differential image in different directions within its texture growth region and the fluctuation degree of different gradient values ​​of each target extreme point in the differential image within its texture growth region;

[0152] For each target extreme point of the image, the final stability of the target extreme point is calculated according to the texture feature similarity between the target extreme point and other target extreme points in its texture growth region in the differential image.

[0153] Optionally, the analysis module 402 uses a SIFT algorithm to analyze the image to obtain a plurality of first extreme points of the image, including:

[0154] The SIFT algorithm is used to analyze the image and obtain multiple extreme points of the image;

[0155] A main body region of the image is identified, and an extreme point located in the main body region is determined as a first extreme point.

[0156] Optionally, the multiple images obtained by photographing the target object along multiple directions include: multiple images obtained by rotationally photographing the target object while rotating along the same rotation direction.

[0157] Optionally, the acquisition module 401 acquires at least one set of images of the target object, including:

[0158] Acquire at least two sets of images of the target object, wherein the at least two sets of images are images obtained by photographing the target object at different levels;

[0159] For each group of images, after stitching multiple images in the group of images using the SIFT algorithm based on the stable extreme points of each image in the group of images to obtain a panoramic image of the target object at a horizontal height corresponding to the group of images, the stitching module 403 is further used to:

[0160] The panoramic images at the horizontal heights respectively corresponding to the at least two groups of images are spliced ​​along the height direction to obtain an overall panoramic image of the target object.

[0161] In this embodiment, when the image data acquisition system applied to bridge engineering uses the SIFT algorithm to obtain the panoramic image of the target object, the texture features of the extreme points are analyzed to screen out stable extreme points, and the image is stitched based on the screened stable extreme points to obtain the panoramic image. This can effectively reduce the adverse effects of the shooting environment on the panoramic image, thereby improving the image quality of the panoramic image of the target object, and providing a more accurate data basis for subsequent quality inspection of the target object through the panoramic image.

[0162] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0163] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. An image data acquisition method applied to bridge engineering, characterized in that: The method comprises: Acquire at least one set of images of the target object, wherein each set of images includes a plurality of images obtained by photographing the target object along a plurality of directions at the same horizontal height and at a stable shooting distance; For each image in each group of images, a scale-invariant feature transform (SIFT) algorithm is used to obtain multiple target extreme value points of the image, and multiple stable extreme value points of the image are determined from the multiple target extreme value points according to the texture features of each target extreme value point; For each group of images, a SIFT algorithm is used to stitch multiple images in the group of images based on the stable extreme points of each image in the group of images to obtain a panoramic image of the target object at a horizontal height corresponding to the group of images; The step of determining a plurality of stable extreme value points of the image from the plurality of target extreme value points according to the texture feature of each target extreme value point comprises: For each target extreme point of the image, a final stability degree of the target extreme point is calculated according to the texture features of the target extreme point and the texture features of other target extreme points in the texture growth region where the target extreme point is located; Determine a target extreme point whose final stability degree is greater than a first preset value as a stable extreme point of the image; The method of using the SIFT algorithm to obtain multiple target extreme points of the image includes: The image is analyzed using the SIFT algorithm to obtain a plurality of first extreme value points of the image; For each first extreme point of the image, the initial stability of the first extreme point is calculated according to the distribution of the first extreme point within the neighborhood of the first extreme point and the contrast between the first extreme point and adjacent pixel points; Determine a first extreme point whose initial stability is greater than a second preset value as a target extreme point of the image; The step of calculating the initial stability of each first extreme point of the image according to the distribution of the first extreme point in the neighborhood of the first extreme point and the contrast between the first extreme point and adjacent pixels comprises: Calculate the distribution stability characteristics of each first extreme value point according to the number of first extreme value points in the neighborhood of each first extreme value point in the difference image and the distance between each first extreme value point in the difference image and other first extreme value points in its neighborhood; According to the number of adjacent pixels of each first extreme point in the difference image in the Gaussian difference DOG pyramid, the gray value of each first extreme point and the gray value of its adjacent pixels in the DOG pyramid, the contrast feature of each first extreme point and its adjacent pixels is calculated; For each first extreme point of the image, calculating the initial stability of the first extreme point according to a distribution stability feature of the first extreme point and a contrast feature between the first extreme point and adjacent pixel points; For each target extreme value point of the image, calculating the final stability of the target extreme value point according to the texture features of the target extreme value point and the texture features of other target extreme value points in the texture growth region where the target extreme value point is located, comprises: The texture feature of each target extreme point is calculated according to the information entropy of each target extreme point in the differential image in different directions within its texture growth region and the fluctuation degree of different gradient values ​​of each target extreme point in the differential image within its texture growth region; For each target extreme point of the image, the final stability of the target extreme point is calculated according to the texture feature similarity between the target extreme point and other target extreme points in its texture growth region in the differential image.

2. The image data acquisition method applied to bridge engineering according to claim 1 is characterized in that: The SIFT algorithm is used to analyze the image to obtain a plurality of first extreme points of the image, including: The SIFT algorithm is used to analyze the image and obtain multiple extreme points of the image; A main body region of the image is identified, and an extreme point located in the main body region is determined as a first extreme point.

3. The image data acquisition method for bridge engineering according to any one of claims 1 to 2, characterized in that: The multiple images obtained by photographing the target object along multiple directions include: multiple images obtained by rotationally photographing the target object while rotating along the same rotation direction.

4. The image data acquisition method for bridge engineering according to any one of claims 1 to 2, characterized in that: The acquiring of at least one set of images of the target object comprises: Acquire at least two sets of images of the target object, wherein the at least two sets of images are images obtained by photographing the target object at different levels; After the method uses the SIFT algorithm to stitch multiple images in each group of images based on the stable extreme points of each image in the group of images to obtain a panoramic image of the target object at a horizontal height corresponding to the group of images, the method further includes: The panoramic images at the horizontal heights respectively corresponding to the at least two groups of images are spliced ​​along the height direction to obtain an overall panoramic image of the target object.

5. An image data acquisition device for bridge engineering, comprising a memory, at least one processor, and at least one program stored in the memory and executable on the at least one processor, characterized in that: When the at least one program is executed by the at least one processor, the steps of the image data acquisition method applied to bridge engineering described in any one of claims 1 to 4 are implemented.

6. An image data acquisition system for bridge engineering, the system being used to implement the steps of the image data acquisition method for bridge engineering as claimed in any one of claims 1 to 4, characterized in that: The system comprises: An acquisition module, used to acquire at least one set of images of a target object, wherein each set of images includes a plurality of images obtained by photographing the target object along multiple directions at the same horizontal height and at a stable shooting distance; An analysis module, for obtaining multiple target extreme value points of each image in each group of images by using a scale-invariant feature transform (SIFT) algorithm, and determining multiple stable extreme value points of the image from the multiple target extreme value points according to a texture feature of each target extreme value point; The stitching module is used to stitch multiple images in each group of images using the SIFT algorithm based on the stable extreme points of each image in the group of images to obtain a panoramic image of the target object at a horizontal height corresponding to the group of images.

Images