Fan blade image splicing method, device and system

Abstract

The invention discloses a fan blade image splicing method, device and system, and belongs to the field of image splicing. The inspection images are shot at different positions at the same height and at the same distance from the fan, so that the width of the same part of the target blade in different inspection images is unchanged, and a characteristic curve related to the width change of the target blade is obtained based on the width; and then determining the transverse offset and the longitudinal offset of the adjacent images according to the characteristic curve, and finally moving the inspection image according to the transverse offset and the longitudinal offset to obtain a spliced image. According to the scheme, the image splicing can be realized without extracting the features of the target blade, the operation is simple, and the splicing success rate and precision are high.

Description

Technical Field

[0001] The present application relates to the field of image stitching, and in particular, to a method, device and system for stitching images of wind turbine blades. Background Art

[0002] During the operation of a wind turbine, the status monitoring of the wind turbine blades is the key to ensuring the safe and stable operation of the wind turbine. However, due to the large size of the wind turbine blades, it is difficult for the airborne image acquisition equipment to obtain a complete blade image at one time during the inspection of the blades by drones. Therefore, it is necessary to use image stitching technology to stitch multiple local images into a complete blade. Existing image stitching methods often have problems such as low matching accuracy and slow stitching speed, which are difficult to meet the needs of practical applications.

[0003] During the operation of the wind turbine, the status monitoring of the wind turbine blades is the key to ensuring the safe and stable operation of the wind turbine. However, due to the large size of the wind turbine blades, it is difficult for the airborne image acquisition equipment to obtain a complete blade image at one time during the drone blade inspection process. Therefore, it is necessary to use image stitching technology to stitch multiple local images into a complete blade. Existing image stitching methods are usually implemented using image registration methods, which require feature extraction and feature point matching of the entire image to calculate the transformation matrix between adjacent images, making the stitching speed slow. In addition, since the wind turbine blades themselves have fewer texture features, and the rich texture features in the background will seriously interfere with the stitching of the blades, resulting in low blade stitching success rate and matching accuracy. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the present application provides a wind blade image stitching method, device and system to solve the problem that the existing image stitching method has low blade stitching success rate and matching accuracy due to the fact that the wind blades themselves have fewer texture features and the rich texture features in the background will seriously interfere with the stitching of the blades.

[0005] The technical solution adopted by this application to solve its technical problem is:

[0006] In a first aspect, a method for stitching images of wind turbine blades is provided, comprising:

[0007] Obtaining inspection images of the target blades, and obtaining a mask image of the target blades according to the inspection images, wherein the inspection images are images taken at different positions at the same height and at the same distance from the wind turbine, and there are overlapping parts of the target blades in adjacent inspection images;

[0008] Acquire the position and posture of the target blade based on the mask image, wherein the position and posture includes the horizontal and vertical coordinates of the starting point and the angle;

[0009] Acquire a characteristic curve of a target blade based on the mask image and the position and posture, wherein the characteristic curve is used to characterize a width change of the target blade at different positions along the x-direction;

[0010] Determine the lateral offset and longitudinal offset of adjacent inspection images based on the characteristic curve, wherein the lateral offset is used to characterize the displacement in the x direction when adjacent inspection images are spliced, and the longitudinal offset is used to characterize the displacement in the y direction when adjacent inspection images are spliced;

[0011] The inspection images are sequentially spliced ​​based on the posture, the lateral offset, and the longitudinal offset to obtain a complete image.

[0012] Further, the determining of the lateral offset and the longitudinal offset of adjacent inspection images based on the characteristic curve includes:

[0013] Controlling the x-axis and y-axis of the first characteristic curve of the previous image and the second characteristic curve of the next image to coincide with each other;

[0014] Controlling the first characteristic curve to be fixed while the second characteristic curve slides along the x direction;

[0015] Calculating target parameters of the first characteristic curve and the second characteristic curve during the sliding process of the second characteristic curve;

[0016] When the target parameter is the minimum, the sliding distance of the second characteristic curve is used as the lateral offset;

[0017] Acquire a first ordinate when the abscissa in the previous image is the lateral displacement, and a second ordinate of the starting point of the target blade in the x-direction in the next image;

[0018] The values ​​of the first longitudinal coordinate and the second longitudinal coordinate are used as the longitudinal offset.

[0019] Furthermore, the calculating of target parameters of the first characteristic curve and the second characteristic curve during the sliding process of the second characteristic curve includes:

[0020] acquiring an overlapping area between the first characteristic curve and the second characteristic curve during a sliding process of the second characteristic curve;

[0021] Calculating the Euclidean distance between the first characteristic curve and the second characteristic curve in the overlapping area;

[0022] The Euclidean distance is used as a target parameter.

[0023] Furthermore, the calculating of target parameters of the first characteristic curve and the second characteristic curve during the sliding process of the second characteristic curve includes:

[0024] acquiring an overlapping area between the first characteristic curve and the second characteristic curve during a sliding process of the second characteristic curve;

[0025] Calculate an area ratio of a total area between the first characteristic curve and the second characteristic curve and the x-axis in the overlapping region to a first area between the first characteristic curve and the x-axis in the overlapping region;

[0026] The area ratio is used as the target parameter.

[0027] Further, obtaining a mask image of the target blade according to the inspection image includes:

[0028] Using a deep learning instance segmentation method to perform foreground segmentation on the inspection image to obtain a mask image of the inspection image;

[0029] Calculate the connected domain of the mask image of each inspection image;

[0030] The connected domain is screened according to the aspect ratio of the target leaf to remove false detection and noise in the non-leaf area, so as to obtain a mask image of each leaf;

[0031] If there is only one blade in the inspection image mask, the mask image of the blade is the mask image of the target blade. If there are multiple blades in the inspection image mask, the mask image of one blade is determined as the mask image of the target blade according to the span direction of the blades in other inspection image masks of the same target blade.

[0032] Further, the acquiring the position and posture of the target leaf based on the mask image includes:

[0033] The mask matrix of the target leaf is summed along the x direction to obtain a one-dimensional vector, and the non-zero area in the one-dimensional vector is the interval where the target leaf is located;

[0034] Based on the one-dimensional vector in the x direction and the y value distribution of the interval where the target leaf is located, the x1, y1, x2, and y2 values ​​of the target leaf are obtained, wherein x1 is the horizontal coordinate of the starting point of the target leaf in the x direction in the image; y1 is the vertical coordinate of the starting point; x2 is the horizontal coordinate of the end point of the target leaf in the x direction in the image, and y2 is the vertical coordinate of the end point;

[0035] Based on the x1, y1, x2, y2 values, calculate the spanwise direction vector of the target blade in the image;

[0036] Calculate the α value of the target blade according to the direction vector, where the α value is the angle between the direction vector and the x-axis;

[0037] [x1, y1, x2, y2, α] is taken as the pose of the target leaf.

[0038] Further, the acquiring a characteristic curve of a target leaf based on the mask image and the posture includes:

[0039] Obtaining a mask matrix of the target leaf based on the mask image;

[0040] The mask matrix of the target leaf in each image is summed along the x direction to obtain a feature vector of the width change of the target leaf along the x direction, and a feature curve of the target leaf is obtained.

[0041] Further, the step of sequentially stitching the inspection images based on the posture, the lateral offset, and the longitudinal offset to obtain a complete image includes:

[0042] The first inspection image of each target blade is used as a fixed image, and a coordinate system is established based on the position and posture of the first inspection image;

[0043] Calculating the displacement of each inspection image in the coordinate system relative to the fixed image in the x direction and in the y direction based on the lateral offset and the longitudinal offset respectively;

[0044] A canvas is created, and the inspection image is moved in the canvas based on the displacement to obtain a complete image.

[0045] In a second aspect, a wind turbine blade image stitching device is provided, comprising:

[0046] A blade image acquisition module is used to acquire an inspection image of a target blade and obtain a mask image of the target blade according to the inspection image. The inspection image is an image taken at different positions at the same height and the same distance from the wind turbine. There are overlapping parts of the target blades in adjacent inspection images.

[0047] A blade posture acquisition module, used to acquire the posture of the target blade based on the mask image, wherein the posture includes the horizontal and vertical coordinates of the starting point and the angle;

[0048] A characteristic curve acquisition module, which acquires a characteristic curve of a target blade based on the mask image and the position and posture, wherein the characteristic curve is used to characterize a width change of the target blade at different positions along the x direction;

[0049] An offset determination module, used to determine the lateral offset and longitudinal offset of adjacent inspection images based on the characteristic curve, wherein the lateral offset is used to characterize the displacement in the x direction when adjacent inspection images are spliced, and the longitudinal offset is used to characterize the displacement in the y direction when adjacent inspection images are spliced;

[0050] The inspection image stitching module is used to stitch the inspection images in sequence based on the posture, the lateral offset and the longitudinal offset to obtain a complete image.

[0051] In a third aspect, a wind turbine blade image stitching system is provided, comprising:

[0052] at least one processor and at least one memory;

[0053] The memory stores executable instructions of the processor;

[0054] The processor is configured to execute the above-mentioned wind turbine blade image stitching method.

[0055] Beneficial effects:

[0056] The technical solution of the present application provides a method, device and system for stitching images of wind turbine blades. First, an inspection image of a target blade is obtained, and a mask image of the target blade is obtained according to the inspection image; then, the position and posture of the target blade is obtained based on the mask image, and then, a characteristic curve of the target blade is obtained based on the mask image and the position and posture, and the lateral offset and longitudinal offset of adjacent images are determined based on the characteristic curve, and the inspection images are stitched in sequence based on the lateral offset and longitudinal offset to obtain a complete image. Since the inspection images are images taken at different positions at the same height and the same distance from the wind turbine, the width of the same part of a target blade is constant in different inspection images, and a characteristic curve related to the change in the width of the target blade is obtained based on the width, and then the lateral offset and longitudinal offset of the adjacent images are determined based on the characteristic curve, and finally, the inspection image is moved according to the lateral offset and longitudinal offset to obtain a stitched image. The present application solution can achieve image stitching without extracting the characteristics of the target blade, and the operation is simple, and the stitching success rate and accuracy are high. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 This is a flow chart of a wind turbine blade image stitching method provided in an embodiment of the present application;

[0059] Figure 2 is a schematic diagram of an inspection image mask provided in an embodiment of the present application;

[0060] Figure 3 is a schematic diagram of a blade mask provided in an embodiment of the present application;

[0061] Figure 4 It is a mask image and a one-dimensional vector schematic diagram of a leaf provided in an embodiment of the present application;

[0062] Figure 5 This is a schematic diagram of obtaining a target leaf mask provided in an embodiment of the present application;

[0063] Figure 6 is a schematic diagram of a target blade characteristic curve provided in an embodiment of the present application;

[0064] Figure 7 is a schematic diagram of determining the lateral offset of adjacent inspection images based on the characteristic curve provided in an embodiment of the present application;

[0065] Figure 8 is a schematic diagram of obtaining a longitudinal offset provided in an embodiment of the present application;

[0066] Fig. 9 It is a structural schematic diagram of a wind turbine blade image stitching device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application is described in detail below in conjunction with the accompanying drawings and embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other implementation methods obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0068] The most common method of image stitching in the prior art is to use feature point matching. However, if feature point matching is to be accurate, it is necessary to ensure that there are more texture features in the image. However, the fan blade itself has relatively few feature points, and most of the inspection images taken are backgrounds. Therefore, when the feature point matching method is used, the background will also affect the success rate of matching. Therefore, stitching based on feature point matching has low accuracy and slow speed.

[0069] To solve the above technical problems, refer to Figure 1 , the embodiment of the present application provides a wind turbine blade image stitching method, comprising:

[0070] S11: acquiring an inspection image of a target blade, and obtaining a mask image of the target blade according to the inspection image, wherein the inspection image is an image taken at different positions at the same height and the same distance from the wind turbine, and there are overlapping parts of the target blades in adjacent inspection images;

[0071] It should be noted that the present application uses a drone to take images of wind turbine blades. During specific shooting, in order to facilitate subsequent processing, the drone selects several hovering points for fixed-point shooting. The number of hovering points is determined according to the size of the wind turbine or wind turbine blades and the size of the image taken by the drone. Because the present application solution relies on the width of the wind turbine blades in the picture for splicing, in order to ensure that the width of the same part on different images is the same, the height of the hovering point and the distance to the plane where the wind turbine is located are the same. Multiple groups of photos are taken at each hovering point. Since the wind turbine blades are always rotating, multiple groups of photos of three wind turbine blades can be obtained at one hovering point according to the blade rotation speed and shooting time. For example, for a dynamic inspection of a wind turbine, it is known that the inspection image is divided into six groups by dividing the blade image, namely, the pressure surface of blade A, the suction surface of blade A, the pressure surface of blade B, the suction surface of blade B, the pressure surface of blade C, and the suction surface of blade C. Splicing each group of images can obtain a complete image of the pressure surface and suction surface of the three blades.

[0072] The adjacent images referred to in this application are images of the same blade taken at adjacent hovering points.

[0073] Wherein, obtaining a mask image of the target blade according to the inspection image includes:

[0074] The inspection image is segmented by deep learning instance segmentation method to obtain the following Figure 2 The mask image of the inspection image shown;

[0075] Calculate the connected domain of the mask image of each inspection image;

[0076] The connected domain is screened according to the aspect ratio of the target leaf to remove false detection and noise in the non-leaf region, and the following is obtained: Figure 3 The mask image of each leaf is shown;

[0077] like Figure 5 As shown, if there is only one blade in the inspection image mask, the mask image of the blade is the mask image of the target blade. If there are multiple blades in the inspection image mask, the mask image of one blade is determined as the mask image of the target blade based on the span direction of the blades in other inspection image masks of the same target blade.

[0078] Among them, the mask image of a blade is determined as the target blade according to the blade span direction in other inspection image masks of the same target blade. The mask image of the blade can be obtained according to the α value in its posture. If there is only one blade in the image, then the blade is the target blade in the image. For a group of inspection images, all images containing only one fan blade are screened out, and the α values ​​of all the screened blades are averaged as the α standard value of the group of inspection images. If there are multiple blades in the image, the difference between the α value of each blade and the standard value is calculated respectively, and the blade with the smallest difference in the image is selected as the target blade.

[0079] S12: acquiring a position and posture of a target blade based on the mask image, wherein the position and posture includes a horizontal and vertical coordinate of a starting point and an angle;

[0080] Specifically, Figure 4 As shown, the mask matrix of the target blade is summed along the x direction to obtain a one-dimensional vector, and the non-zero area in the one-dimensional vector is the interval where the target blade is located; based on the one-dimensional vector in the x direction and the y value distribution of the interval where the target blade is located, the x1, y1, x2, y2 values ​​of the target blade are obtained, wherein x1 is the horizontal coordinate of the starting point of the target blade in the x direction in the image; y1 is the vertical coordinate of the starting point; x2 is the horizontal coordinate of the end point of the target blade in the x direction in the image, and y2 is the vertical coordinate of the end point; based on the x1, y1, x2, y2 values, the span direction vector a of the target blade in the image is calculated as (x2-x1, y2-y1); the α value of the target blade is calculated according to the direction vector, α=<a,b> ,b=(1,0),<a,b> To find the angle between vector a and vector b, where vector b is a unit vector on the x-axis; the α value is the angle between the direction vector and the x-axis; and [x1, y1, x2, y2, α] is taken as the position and posture of the target blade.

[0081] It should be noted that if an image includes multiple leaves, it is necessary to calculate the pose of each leaf first, and then determine the target leaf. Of course, it is also possible to calculate the pose of each leaf when obtaining the mask image of each leaf, and then determine the target leaf based on it.

[0082] S13: acquiring a characteristic curve of the target blade based on the mask image and the posture, wherein the characteristic curve is used to characterize a width change of the target blade at different positions along the x direction;

[0083] Specifically, the mask matrix of the target leaf is obtained based on the mask image; the mask matrix of the target leaf in each image is summed along the x direction to obtain the characteristic vector of the width change of the target leaf along the x direction, and the characteristic curve of the target leaf is obtained. Figure 6As shown, the upper part is the mask image of the target leaf, and the lower part is the corresponding feature curve.

[0084] S14: determining a lateral offset and a longitudinal offset of adjacent inspection images based on the characteristic curve, wherein the lateral offset is used to characterize a displacement in the x direction when adjacent inspection images are spliced, and the longitudinal offset is used to characterize a displacement in the y direction when adjacent inspection images are spliced;

[0085] Specifically, Figure 7 As shown, the x-axis and y-axis of the first characteristic curve of the previous image and the second characteristic curve of the next image are controlled to coincide with each other; wherein the previous image and the next image are adjacent images. The first characteristic curve is controlled to be fixed while the second characteristic curve slides along the x direction; during the sliding process of the second characteristic curve, the target parameters of the first characteristic curve and the second characteristic curve are calculated; when the target parameter is the smallest, the sliding distance of the second characteristic curve is used as the lateral offset offset-x.

[0086] In one embodiment, the overlapping area of ​​the first characteristic curve and the second characteristic curve is obtained during the sliding process of the second characteristic curve, wherein the overlapping area does not refer to the overlap of the curves, but refers to the overlapping area of ​​the coordinate system where the curves are located; the Euclidean distance between the first characteristic curve and the second characteristic curve in the overlapping area is calculated; and the Euclidean distance is used as the target parameter. When the second characteristic curve slides, when the same part of the first characteristic curve and the second characteristic curve moves together, the first characteristic curve and the second characteristic curve overlap, and the Euclidean distance is minimum. At other times, the two characteristic curves may not overlap at all, or only partially overlap, and the Euclidean distance is larger. Therefore, its lateral offset can be obtained according to the Euclidean distance.

[0087] In another embodiment, the overlapping area of ​​the first characteristic curve and the second characteristic curve during the sliding process of the second characteristic curve is obtained; the area ratio of the total area between the first characteristic curve and the second characteristic curve and the x-axis in the overlapping area to the first area between the first characteristic curve and the x-axis in the overlapping area is calculated; and the area ratio is used as the target parameter. During the sliding process of the second characteristic curve, when the first characteristic curve and the second characteristic curve do not overlap in the overlapping area, the total area between the first characteristic curve and the second characteristic curve and the x-axis in the overlapping area is greater than the area between any characteristic curve and the x-axis, and the area ratio is greater than 1. When only two curves overlap, the total area = the first area.

[0088] The calculation of the longitudinal offset is as follows: obtain the first longitudinal coordinate when the horizontal coordinate in the previous image is the horizontal displacement, and the second longitudinal coordinate of the starting point of the target leaf in the x direction in the next image; and use the values ​​of the first longitudinal coordinate and the second longitudinal coordinate as the longitudinal offset. Exemplarily, according to offset-x and the position and posture of the target leaf in two adjacent images, the leaf positions on the fixed image and the sliding image are [x 11 ,y 11 ,x 12 ,y 12 ,α1],[x 21 ,y 21 ,x 22 ,y 22 ,α2], first, find the y coordinate of the leaf at the offset-x position on the fixed image, such as Figure 8 As shown, denoted as y', the displacement in the y direction can be obtained as offset-y = y'-y 21 .

[0089] S15: Based on the posture, the lateral offset, and the longitudinal offset, the inspection images are stitched in sequence to obtain a complete image.

[0090] The first inspection image of each target blade is used as a fixed image, and a coordinate system is established with the position and posture of the first inspection image; the displacement of each inspection image in the coordinate system relative to the fixed image in the x direction and in the y direction is calculated based on the lateral offset and the longitudinal offset respectively; a canvas is created, and the inspection image is moved in the canvas based on the displacement to obtain a complete image.

[0091] When creating a canvas, the width and height of the canvas may be directly determined according to the width and height of each inspection image and the number of inspection images of a leaf that need to be spliced.

[0092] However, the canvas created in this way is relatively large, so in order to obtain a suitable canvas, the present application also calculates the width and height of the canvas based on the horizontal offset and the vertical offset. For example, for a set of inspection images, the first image is used as a fixed image to establish a coordinate system, and the displacement relationship between adjacent images, offset-x, offset-y, is first calculated respectively, and the width W and height H of the spliced ​​image are calculated based on the displacement relationship between adjacent images. The canvas is created and each image is mapped to the canvas in turn for image fusion. The fused image is the leaf splicing image.

[0093] The wind turbine blade image stitching method provided in the embodiment of the present application first obtains the inspection image of the target blade, and obtains the mask image of the target blade according to the inspection image; then obtains the position and posture of the target blade based on the mask image, and then obtains the characteristic curve of the target blade based on the mask image and the position and posture, determines the lateral offset and longitudinal offset of adjacent images based on the characteristic curve, and stitches the inspection images in sequence based on the lateral offset and longitudinal offset to obtain a complete image. Since the inspection images are images taken at different positions at the same height and the same distance from the wind turbine, the width of the same part of a target blade is constant in different inspection images, and a characteristic curve related to the change in the width of the target blade is obtained based on the width, and then the lateral offset and longitudinal offset of the adjacent images are determined based on the characteristic curve, and finally the inspection image is moved according to the lateral offset and longitudinal offset to obtain a stitched image. The present application scheme can achieve image stitching without extracting the characteristics of the target blade, and the operation is simple, and the stitching success rate and accuracy are high.

[0094] Based on the same inventive concept, Fig. 9 As shown, the present application provides a wind turbine blade image stitching device 90, comprising:

[0095] The blade image acquisition module 91 is used to acquire the inspection image of the target blade and obtain the mask image of the target blade according to the inspection image. The inspection image is an image taken at different positions at the same height and the same distance from the wind turbine. There are overlapping parts of the target blades in adjacent inspection images.

[0096] The step of obtaining a mask image of the target blade according to the inspection image comprises:

[0097] The deep learning instance segmentation method is used to perform foreground segmentation on the inspection image to obtain a mask image of the inspection image; a connected domain is calculated for the mask image of each inspection image; the connected domain is screened according to the aspect ratio of the target blade to remove false detection and noise in the non-leaf area, so as to obtain a mask image of each blade; if there is only one blade in the inspection image mask, the mask image of the blade is the mask image of the target blade; if there are multiple blades in the inspection image mask, the mask image of a blade is determined as the mask image of the target blade according to the span direction of the blades in other inspection image masks of the same target blade.

[0098] The blade posture acquisition module 92 is used to acquire the posture of the target blade based on the mask image, wherein the posture includes the horizontal and vertical coordinates of the starting point and the angle; the acquiring the posture of the target blade based on the mask image includes:

[0099] The mask matrix of the target blade is summed along the x direction to obtain a one-dimensional vector, and the non-zero area in the one-dimensional vector is the interval where the target blade is located; based on the one-dimensional vector in the x direction and the y value distribution of the interval where the target blade is located, the x1, y1, x2, y2 values ​​of the target blade are obtained, wherein x1 is the horizontal coordinate of the starting point of the target blade in the x direction in the image; y1 is the vertical coordinate of the starting point; x2 is the horizontal coordinate of the end point of the target blade in the x direction in the image, and y2 is the vertical coordinate of the end point; based on the x1, y1, x2, y2 values, the span direction vector of the target blade in the image is calculated; according to the direction vector, the α value of the target blade is calculated, and the α value is the angle between the direction vector and the x-axis; [x1, y1, x2, y2, α] is taken as the pose of the target blade.

[0100] A characteristic curve acquisition module 93 is used to acquire a characteristic curve of a target blade based on the mask image and the position and posture, wherein the characteristic curve is used to characterize a width change of the target blade at different positions along the x direction;

[0101] The acquiring a characteristic curve of a target leaf based on the mask image and the posture includes:

[0102] A mask matrix of the target leaf is obtained based on the mask image; the mask matrix of the target leaf in each image is summed along the x direction to obtain a characteristic vector of the width change of the target leaf along the x direction, and a characteristic curve of the target leaf is obtained.

[0103] An offset determination module 94, used to determine the lateral offset and longitudinal offset of adjacent inspection images based on the characteristic curve, wherein the lateral offset is used to characterize the displacement in the x direction when adjacent inspection images are spliced, and the longitudinal offset is used to characterize the displacement in the y direction when adjacent inspection images are spliced;

[0104] Determining the lateral offset and the longitudinal offset of adjacent inspection images based on the characteristic curve includes:

[0105] The first characteristic curve of the previous image and the second characteristic curve of the next image are controlled to overlap in their x-axis and y-axis respectively; the first characteristic curve is controlled to be fixed while the second characteristic curve slides along the x-direction; the target parameters of the first characteristic curve and the second characteristic curve are calculated during the sliding process of the second characteristic curve; when the target parameter is the minimum, the sliding distance of the second characteristic curve is used as the lateral offset; the first ordinate when the abscissa in the previous image is the lateral displacement, and the second ordinate of the starting point of the target blade in the x-direction in the next image are obtained; and the values ​​of the first ordinate and the second ordinate are used as the longitudinal offset.

[0106] In one embodiment, the calculating of target parameters of the first characteristic curve and the second characteristic curve during the sliding process of the second characteristic curve includes:

[0107] Acquire an overlapping area between the first characteristic curve and the second characteristic curve during the sliding of the second characteristic curve; calculate the Euclidean distance between the first characteristic curve and the second characteristic curve in the overlapping area; and use the Euclidean distance as a target parameter.

[0108] In another embodiment, calculating target parameters of the first characteristic curve and the second characteristic curve during the sliding process of the second characteristic curve includes:

[0109] Acquire an overlapping area of ​​the first characteristic curve and the second characteristic curve during the sliding process of the second characteristic curve; calculate an area ratio of a total area between the first characteristic curve and the second characteristic curve and the x-axis in the overlapping area to a first area between the first characteristic curve and the x-axis in the overlapping area; and use the area ratio as the target parameter.

[0110] The inspection image stitching module 95 is used to stitch the inspection images in sequence based on the posture, the lateral offset and the longitudinal offset to obtain a complete image.

[0111] Specifically, the first inspection image of each target blade is used as a fixed image, and a coordinate system is established with the position and posture of the first inspection image; the displacement of each inspection image in the coordinate system relative to the fixed image in the x direction and the y direction is calculated based on the lateral offset and the longitudinal offset respectively; a canvas is created, and the inspection image is moved in the canvas based on the displacement to obtain a complete image.

[0112] The wind turbine blade image stitching device provided in the embodiment of the present application first obtains the inspection image of the target blade, and obtains the mask image of the target blade according to the inspection image; then obtains the position and posture of the target blade based on the mask image, and then obtains the characteristic curve of the target blade based on the mask image and the position and posture, determines the lateral offset and longitudinal offset of adjacent images based on the characteristic curve, and stitches the inspection images in sequence based on the lateral offset and longitudinal offset to obtain a complete image. Since the inspection images are images taken at different positions at the same height and the same distance from the wind turbine, the width of the same part of a target blade is constant in different inspection images, and a characteristic curve related to the change in the width of the target blade is obtained based on the width, and then the lateral offset and longitudinal offset of the adjacent images are determined based on the characteristic curve, and finally the inspection image is moved according to the lateral offset and longitudinal offset to obtain a stitched image. The present application solution can achieve image stitching without extracting the characteristics of the target blade, and the operation is simple, and the stitching success rate and accuracy are high.

[0113] Based on the same inventive concept, the present application provides a wind turbine blade image stitching system, comprising:

[0114] at least one processor and at least one memory;

[0115] The memory stores executable instructions of the processor;

[0116] The processor is configured to execute the above-mentioned wind turbine blade image stitching method.

[0117] The wind turbine blade image stitching system provided by the embodiment of the present application stores the executable instructions of the processor in the memory. When the executable instructions are executed, the processor can first obtain the inspection image of the target blade, and obtain the mask image of the target blade according to the inspection image; then obtain the position and posture of the target blade based on the mask image, and then obtain the characteristic curve of the target blade based on the mask image and the position and posture, determine the lateral offset and longitudinal offset of the adjacent images based on the characteristic curve, and stitch the inspection images in sequence based on the lateral offset and longitudinal offset to obtain a complete image. Since the inspection images are images taken at different positions at the same height and the same distance from the wind turbine, the width of the same part of a target blade is constant in different inspection images, and a characteristic curve related to the change in the width of the target blade is obtained based on the width, and then the lateral offset and longitudinal offset of the adjacent images are determined based on the characteristic curve, and finally the inspection image is moved according to the lateral offset and longitudinal offset to obtain a stitched image. The present application scheme can achieve image stitching without extracting the characteristics of the target blade, and the operation is simple, and the stitching success rate and accuracy are high.

[0118] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0119] It should be noted that, in the description of this application, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" refers to at least two.

[0120] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0121] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0122] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0123] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0124] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0125] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0126] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A wind turbine blade image stitching method, It is characterized in that include: Obtaining inspection images of the target blades, and obtaining a mask image of the target blades according to the inspection images, wherein the inspection images are images taken at different positions at the same height and at the same distance from the wind turbine, and there are overlapping parts of the target blades in adjacent inspection images; Acquire the position and posture of the target blade based on the mask image, wherein the position and posture includes the horizontal and vertical coordinates of the starting point and the angle; Acquire a characteristic curve of a target blade based on the mask image and the position and posture, wherein the characteristic curve is used to characterize a width change of the target blade at different positions along the x-direction; Determine the lateral offset and longitudinal offset of adjacent inspection images based on the characteristic curve, wherein the lateral offset is used to characterize the displacement in the x direction when adjacent inspection images are spliced, and the longitudinal offset is used to characterize the displacement in the y direction when adjacent inspection images are spliced; The inspection images are sequentially spliced ​​based on the posture, the lateral offset, and the longitudinal offset to obtain a complete image.

2. The method according to claim 1, Features: The determining of the lateral offset and the longitudinal offset of adjacent inspection images based on the characteristic curve comprises: Controlling the x-axis and y-axis of the first characteristic curve of the previous image and the second characteristic curve of the next image to coincide with each other; Controlling the first characteristic curve to be fixed while the second characteristic curve slides along the x direction; Calculating target parameters of the first characteristic curve and the second characteristic curve during the sliding process of the second characteristic curve; When the target parameter is the minimum, taking the sliding distance of the second characteristic curve as the lateral offset; Acquire a first ordinate when the abscissa in the previous image is the lateral displacement, and a second ordinate of a starting point of the target blade in the x-direction in the next image; The values ​​of the first longitudinal coordinate and the second longitudinal coordinate are used as the longitudinal offset.

3. The method according to claim 2, Features: The calculating of target parameters of the first characteristic curve and the second characteristic curve during the sliding process of the second characteristic curve includes: acquiring an overlapping area between the first characteristic curve and the second characteristic curve during a sliding process of the second characteristic curve; Calculating the Euclidean distance between the first characteristic curve and the second characteristic curve in the overlapping area; The Euclidean distance is used as a target parameter.

4. The method according to claim 2, Features: The calculating of target parameters of the first characteristic curve and the second characteristic curve during the sliding process of the second characteristic curve includes: acquiring an overlapping area between the first characteristic curve and the second characteristic curve during a sliding process of the second characteristic curve; Calculate an area ratio of a total area between the first characteristic curve and the second characteristic curve and the x-axis in the overlapping region to a first area between the first characteristic curve and the x-axis in the overlapping region; The area ratio is used as the target parameter.

5. The method according to claim 1, Features: The step of obtaining a mask image of the target blade according to the inspection image comprises: Using a deep learning instance segmentation method to perform foreground segmentation on the inspection image to obtain a mask image of the inspection image; Calculate the connected domain of the mask image of each inspection image; The connected domain is screened according to the aspect ratio of the target leaf to remove false detection and noise in the non-leaf area, so as to obtain a mask image of each leaf; If there is only one blade in the inspection image mask, the mask image of the blade is the mask image of the target blade. If there are multiple blades in the inspection image mask, the mask image of one blade is determined as the mask image of the target blade according to the span direction of the blades in other inspection image masks of the same target blade.

6. The method according to claim 1, Features: The acquiring the position and posture of the target leaf based on the mask image includes: The mask matrix of the target leaf is summed along the x direction to obtain a one-dimensional vector, and the non-zero area in the one-dimensional vector is the interval where the target leaf is located; Based on the one-dimensional vector in the x direction and the y value distribution of the interval where the target leaf is located, the x1, y1, x2, and y2 values ​​of the target leaf are obtained, wherein x1 is the horizontal coordinate of the starting point of the target leaf in the x direction in the image; y1 is the vertical coordinate of the starting point; x2 is the horizontal coordinate of the end point of the target leaf in the x direction in the image, and y2 is the vertical coordinate of the end point; Based on the x1, y1, x2, y2 values, calculate the spanwise direction vector of the target blade in the image; Calculate the α value of the target blade according to the direction vector, where the α value is the angle between the direction vector and the x-axis; [x1, y1, x2, y2, α] is taken as the pose of the target leaf.

7. The method according to claim 1, Features: The acquiring a characteristic curve of a target leaf based on the mask image and the posture includes: Obtaining a mask matrix of the target leaf based on the mask image; The mask matrix of the target leaf in each image is summed along the x direction to obtain a feature vector of the width change of the target leaf along the x direction, and a feature curve of the target leaf is obtained.

8. The method according to claim 1, Features: The step of sequentially stitching the inspection images based on the posture, the lateral offset, and the longitudinal offset to obtain a complete image includes: The first inspection image of each target blade is used as a fixed image, and a coordinate system is established based on the position and posture of the first inspection image; Calculating the displacement of each inspection image in the coordinate system relative to the fixed image in the x direction and in the y direction based on the lateral offset and the longitudinal offset respectively; A canvas is created, and the inspection image is moved in the canvas based on the displacement to obtain a complete image.

9. A wind turbine blade image stitching device, It is characterized in that include: A blade image acquisition module is used to acquire an inspection image of a target blade and obtain a mask image of the target blade according to the inspection image. The inspection image is an image taken at different positions at the same height and the same distance from the wind turbine. There are overlapping parts of the target blades in adjacent inspection images. A blade posture acquisition module, used to acquire the posture of the target blade based on the mask image, wherein the posture includes the horizontal and vertical coordinates of the starting point and the angle; A characteristic curve acquisition module, which acquires a characteristic curve of a target blade based on the mask image and the position and posture, wherein the characteristic curve is used to characterize a width change of the target blade at different positions along the x direction; An offset determination module, used to determine the lateral offset and longitudinal offset of adjacent inspection images based on the characteristic curve, wherein the lateral offset is used to characterize the displacement in the x direction when adjacent inspection images are spliced, and the longitudinal offset is used to characterize the displacement in the y direction when adjacent inspection images are spliced; The inspection image stitching module is used to stitch the inspection images in sequence based on the posture, the lateral offset and the longitudinal offset to obtain a complete image.

10. A wind turbine blade image stitching system, It is characterized in that include: at least one processor and at least one memory; The memory stores executable instructions of the processor; The processor is configured to execute the method according to any one of claims 1 to 8.

Images