Defect location calculation methods, systems, equipment, and storage media

Abstract

This disclosure relates to a method, system, device, and storage medium for calculating defect locations, including an image acquisition unit for acquiring a plurality of images covering at least one complete surface of an object under test; an image stitching unit for stitching the plurality of acquired images of the complete surface to obtain a stitched image of the complete surface of the object under test; a foreground extraction unit for performing foreground segmentation processing on the obtained stitched image to obtain a foreground mask image of the complete surface of the object under test; a position determination unit for acquiring the position of a defect in the foreground mask image relative to at least one endpoint of the object under test; and a position conversion unit for determining the position of the defect on the actual object under test based on the ratio of the actual object under test to the foreground mask image.

Description

Technical Field

[0001] This disclosure relates to the field of location determination technology, specifically to a method, system, device, and storage medium for calculating defect locations. Background Technology

[0002] Wind power generation refers to converting the kinetic energy of wind into electrical energy. The principle of wind power generation is to use wind power to drive the rotation of windmill blades, and then use a speed increaser to increase the rotation speed, thereby driving a generator to generate electricity. Due to the environmental friendliness of wind power generation, it has received great attention as a clean energy source.

[0003] A wind turbine generator set, or simply a wind turbine, generally consists of three parts: a wind turbine, a generator, and a tower. The wind turbine is the important component that converts the kinetic energy of the wind into mechanical energy. It consists of several blades. When the wind blows on the blades, the aerodynamic force generated on the blades drives the wind turbine to rotate. The generator takes the constant speed obtained from the wind turbine and transmits it to the power generation mechanism to operate evenly, thus converting mechanical energy into electrical energy. The tower is the frame that supports the wind turbine and the generator.

[0004] The blades are an important part of the wind turbine generator. Whether wind energy can be captured to the maximum extent often depends on whether the blades have excellent aerodynamic shape, smoothness of the blade surface, and other properties.

[0005] The complexity of the blade structure and the environment in which it is located makes the operation and maintenance of wind turbine blades very difficult, and it is also difficult to determine the location of blade defects.

[0006] The existing methods for determining the location of blade defects mainly include the following: setting scales on the blade for estimation, that is, before the actual installation or operation of the blade, markings are painted or pasted on the blade in advance, and when a defect appears on the blade, it can be read directly using these markings; using known location markers (lightning arresters) on the blade for estimation, that is, checking the distance of the defect location from the marker, and making a rough estimate based on the location data of the marker; using actual measurement estimation based on human stride length, which requires climbing onto the blade to make actual walking estimation, as well as ground building triangulation and reverse estimation. Summary of the Invention

[0007] This disclosure addresses the problem of the coarse measurement of blade defect locations in the prior art by providing a method, system, device, and storage medium for calculating defect locations.

[0008] To solve the above problems, the present disclosure adopts the following technical solution.

[0009] One aspect of this disclosure provides a method for calculating the location of a defect, which includes the following steps:

[0010] Acquire several images covering at least one complete surface of the object under test;

[0011] Several images of a complete surface are stitched together to obtain a stitched image of the complete surface of the object under test.

[0012] The obtained stitched image is subjected to foreground segmentation processing to obtain a foreground mask image of a complete surface of the object under test;

[0013] The position of the defect region in the foreground mask image relative to at least one endpoint of the object under test is obtained;

[0014] The location of the defect region on the actual object being tested is determined based on the ratio between the actual object being tested and the foreground mask image.

[0015] According to an embodiment of this disclosure, obtaining the position of the defect region in the foreground mask image relative to at least one endpoint of the object under test further includes the following steps:

[0016] Image recognition is performed on the foreground mask image using an endpoint recognition model to identify the endpoints of the object being measured and determine the position coordinates of the endpoints in the foreground mask image.

[0017] The defect identification model is used to identify defects in a single image among several images that make up the complete surface, obtain the bounding box where the defect area is located, and determine the position coordinates of several corner points of the bounding box in the single image.

[0018] The position coordinates of several corner points of the marker box in a single image are transformed by the homography matrix to determine the position coordinates of several corner points of the marker box in the foreground mask image in the single image;

[0019] Based on the position coordinates of the endpoint in the foreground mask image and the position coordinates of several corner points of the marker box in the foreground mask image, determine the distances of several corner points of the marker box relative to the endpoint in the foreground mask image;

[0020] A corner point of a marker frame is determined as a representative point of the defect area, and the distance of the defect area relative to the endpoint in the foreground mask image is determined.

[0021] According to an embodiment of this disclosure, the step of performing image recognition on the foreground mask image using an endpoint recognition model to identify the endpoints of the object under test and determine the position coordinates of the endpoints in the foreground mask image specifically includes:

[0022] Acquire several images of the object under test with endpoints, and mark the endpoints;

[0023] Several images with endpoint markers were used as training samples;

[0024] An endpoint identification model is established, and the model is trained using the training samples as sample data.

[0025] The foreground mask image is input into the endpoint recognition model to identify endpoints and determine the position coordinates of the identified endpoints in the foreground mask image.

[0026] According to an embodiment of this disclosure, the step of performing defect identification on a single image among several images constituting the complete surface using a defect identification model, obtaining a bounding box containing the defect region, and determining the position coordinates of several corner points of the bounding box in the single image includes:

[0027] Acquire several defect images of the object under test, and mark the defects in the defect images;

[0028] Several defect images bearing the aforementioned defect markers are used as training samples;

[0029] A defect identification model is established, and the model is trained using the training samples as sample data.

[0030] A single image from among several images that make up the complete surface is input into the defect recognition model to identify the defect, obtain the bounding box where the defect area is located, and determine the position coordinates of several corner points of the bounding box in the single image.

[0031] According to an embodiment of this disclosure, the step of transforming the position coordinates of several corner points of the marker box in a single image using a homography matrix to determine the position coordinates of several corner points of the marker box in the foreground mask image further includes:

[0032] Determine the first set of position coordinates of the four corner points of the single image where the defective region is located;

[0033] Determine the second set of position coordinates of the four corner points of the single image in the coordinate system of the foreground mask image or the stitched image;

[0034] The homography matrix between the single image and the foreground mask image or the stitched image is determined based on the first set of position coordinates and the second set of position coordinates of the four corner points;

[0035] Based on the homography matrix, the position coordinates of several corner points of the marker box in a single image are transformed to the position coordinates of the foreground mask image.

[0036] According to one embodiment of this disclosure, determining the location of the defect region on the actual object being tested based on the ratio of the actual object to the foreground mask image further includes the following steps:

[0037] Obtain the distance between the endpoints of the actual measured object and the distance between the endpoints of the foreground mask image;

[0038] The ratio is determined based on the distance between the endpoints of the actual measured object and the distance between the endpoints of the foreground mask image.

[0039] The position of the defect on the real object under test is determined by multiplying the ratio of the distance between the endpoints of the real object under test to the distance between the endpoints of the foreground mask image and the distance of the defect region in the foreground mask image relative to one endpoint of the object under test.

[0040] According to one embodiment of this disclosure, determining the location of the defect region on the actual object being tested based on the ratio of the actual object to the foreground mask image further includes the following steps:

[0041] Evenly distributed markers are pre-set on the actual object being measured, and the distances between each actual marker and the actual endpoints are recorded.

[0042] Obtain the distance between each marker and the endpoint in the foreground mask image or stitched image;

[0043] The correction parameters are determined based on the distance between the real identifier and the real endpoint and the ratio of the distance between the identifier and the endpoint in the corresponding foreground mask image or stitched image.

[0044] Adjust the position of defects and / or markings on the actual object being tested according to the correction parameters.

[0045] Another aspect of this disclosure provides a defect location calculation system for implementing the defect location calculation method as described above. The identification system includes:

[0046] An image acquisition unit is used to acquire several images covering at least one complete surface of the object being measured;

[0047] An image stitching unit is used to stitch together several images of a complete surface to obtain a stitched image of the complete surface of the object under test.

[0048] The foreground extraction unit is used to perform foreground segmentation processing on the obtained stitched image to obtain a foreground mask image of a complete surface of the object under test.

[0049] A position determination unit is used to obtain the position of the defect region in the foreground mask image relative to at least one endpoint of the object under test;

[0050] The position conversion unit is used to determine the position of the defect region on the real object being tested based on the ratio of the real object being tested to the foreground mask image.

[0051] Another aspect of this disclosure provides a defect location calculation and identification device. The identification device includes:

[0052] The memory is used to store the processing program;

[0053] A processor, which executes the processing program to implement the defect location calculation method in one embodiment of the present disclosure.

[0054] The final aspect of this disclosure is to provide a readable storage medium. The readable storage medium stores a processing program that, when executed by a processor, implements the defect location calculation method according to an embodiment of this disclosure.

[0055] The advantages of this disclosure are:

[0056] Compared with existing technologies, the advantages of the defect location calculation method, system, equipment, and storage medium provided in this disclosure are as follows: This disclosure uses a drone to collect several images of the wind turbine blades and stitches them together to extract the images. The defect location is determined from the acquired images, and a more accurate actual defect location is determined based on the ratio of the actual blade to the blade image. This facilitates maintenance personnel in locating the defect for maintenance and simplifies the defect location search process. Furthermore, this disclosure pre-determines evenly distributed markings on the blades. Based on the distances between the actual blade tip and root and the evenly distributed markings, a recursive proportional adjustment method is used to optimize the actual distances between the defect and the blade tip and root, obtaining a more precise defect location on the actual blade. Attached Figure Description

[0057] Figure 1 This is a flowchart of a defect location calculation method in one embodiment of this disclosure;

[0058] Figure 2 This is a schematic diagram of the defect location calculation system in one embodiment of the present disclosure;

[0059] Figure 3 This is a schematic diagram of the defect location calculation device in one embodiment of the present disclosure;

[0060] Figure 4 This is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of the present disclosure. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure. The following embodiments use a wind turbine blade as an example. The endpoints mentioned below represent the blade tip and blade root; the several corner points of the marked frame represent the corner points of the defect frame of the image recognition defect, and the defect frame includes regular and irregular shapes; the four corner points of a single image represent the four vertices of a captured single image.

[0062] Existing methods for determining the location of blade defects mainly include the following: estimating using scales on the blade (by painting or affixing markings to the blade before actual installation or operation, allowing direct reading of the scale when a defect appears); estimating using known markers (lightning arresters) on the blade (by checking the distance of the defect from the marker and making a rough estimate based on the marker's location data); and estimating using human stride length (requiring climbing onto the blade for actual walking estimation, as well as ground triangulation and reverse estimation). These existing methods are all relatively coarse and cannot achieve precise defect location determination. Because the precise location of the defect cannot be determined, the error is relatively large, making it difficult to implement artificial intelligence-based maintenance operations.

[0063] To address the technical problems in the prior art, this disclosure provides a method, system, device, and storage medium for calculating defect locations, which can determine more accurate true defect locations and facilitate maintenance operations using artificial intelligence solutions.

[0064] Embodiment 1 of this disclosure provides a specific implementation of a defect location calculation method, see [link to previous document]. Figure 1 The defect location calculation method specifically includes the following: S110: Acquire several images covering at least one complete surface of the object under test. The object under test includes one, two, three, or even countless surfaces, and the information of a single surface cannot be obtained from a single image. For example, wind turbine blades require the inspection of four surfaces, and the blades are huge, so it is impossible to obtain a complete surface with just one image; several images completely covering the blade are needed to obtain one complete surface.

[0065] S120: Several images of a complete surface are stitched together to obtain a stitched image of the complete surface of the object under test. Several images containing information about a complete surface are stitched together to obtain a complete surface of the object under test.

[0066] S130: Perform foreground segmentation processing on the obtained stitched image to obtain a foreground mask image of a complete surface of the object under test. The foreground mask image includes the object under test itself, i.e., it contains target-related information. Correspondingly, there is a background mask image including all parts other than the object under test. The foreground mask image and background mask image can be segmented using methods such as GrabCut or watershed segmentation to extract the foreground mask image. For example, when acquiring several images of wind turbine blades, the background environment of the wind turbine is also captured. In this embodiment, the GrabCut algorithm is used to segment the foreground and background images of the blades. GrabCut is an image segmentation algorithm based on graph cut. The user defines one or more rectangular objects for a stitched image of a complete surface of the blade. The area outside the rectangle is automatically identified as the background with a 100% probability. For the user-defined rectangular area, data in the background can be used to distinguish whether it is a foreground or background area. The user-defined background and foreground are saved in a mask image. The mask image includes a definite background (0), a definite foreground (1), a possible background (2), and a possible foreground (3). A Gaussian Mixture Model (GMM) is used to build models for the background and foreground respectively, and undefined pixels are marked as possible foreground or background. The probabilities of possible background and possible foreground are estimated. Each pixel in the blade image is considered to be connected to surrounding pixels through virtual edges, and each edge has a probability of belonging to the foreground or background. The probability can be based on the color similarity between each edge and the surrounding pixels. Each pixel is connected to either a foreground or background pixel. After the pixels are connected (possibly to the background or foreground), if the edges between the pixels belong to different endpoints (i.e., one pixel belongs to the foreground and the other to the background), the edges between these two pixels are cut, thus segmenting the image into different parts.

[0067] S140: Obtain the position of the defect region in the foreground mask image relative to at least one endpoint of the object under test.

[0068] S150: Determine the position of the defect area on the real object being tested based on the ratio of the real object being tested to the foreground mask image.

[0069] Step S140 further includes the following steps:

[0070] S141: Perform image recognition on the foreground mask image using an endpoint recognition model to identify the endpoints of the object being measured and determine the position coordinates of the endpoints in the foreground mask image.

[0071] The process involves acquiring several images of the object under test with endpoints and marking these endpoints; using these marked images as training samples; establishing an endpoint recognition model and training the model using the training samples as sample data; inputting the foreground mask image into the endpoint recognition model to identify the endpoints and determine the position coordinates of the identified endpoints in the foreground mask image. The endpoint recognition model can employ deep learning models such as Artificial Neural Networks (ANN), Convolutional Neural Networks (CNN), or Feature Pyramid Networks (FPN). For example, the endpoints are the leaf tip and leaf root, and the leaf tip and leaf root are identified using a DenseNet model of a Convolutional Neural Network (CNN). Specifically, this includes: dividing a randomly collected dataset of leaf samples with leaf roots and tips into a training dataset; constructing a dense network model consisting of an input layer, four dense block layers, three transition layers that downsample the convolutional and pooling layers respectively, and a classification layer; setting a growth rate of 32, the input layer has 64 7×7 convolutions, the dense block layer has 4 layers, three transition layers are placed between the dense block layers, the classification layer has 7×7 average pooling, 1×1 convolutions, and an output layer. The input layer includes a 7×7 convolution, a convolutional layer with a stride of 2, and a 3×3 pooling layer with a max pooling layer with a stride of 2, used to process the input image; the dense block layer contains multiple dense blocks, with input and output connected along the channel dimension between each dense block, and the channel number growth rate is K. If the input feature map has K0 channels, then the number of channels in the Lth layer is K. L =K0 + (L-1)K. A training image of size 224×224 is input to the input layer. The input image is processed by 7×7 convolution operations with a stride of 2 and 3×3 max pooling operations with a stride of 2, outputting 64 feature maps of size 56×56, i.e., 64 output channels. These 64 56×56 feature maps are then sequentially fed into the first dense block layer, the second dense block layer, the third dense block layer, and the fourth dense block layer, outputting feature maps. These feature maps are then fed into the classification layer. After average pooling and convolutional filtering, the classification result is output. The dense network model is evaluated and its parameters are tuned using a validation set to optimize the endpoint recognition model. The adjusted endpoint recognition model is used to perform image recognition on the foreground mask image to identify the leaf tip and / or leaf root. The foreground mask image is used as a coordinate system to determine the position coordinates of the leaf tip and / or leaf root within the foreground mask image.

[0072] S142: Using a defect recognition model, defects are identified in a single image among several images that make up the complete surface, the marked box where the defect area is located is obtained, and the position coordinates of several corner points of the marked box in the single image are determined.

[0073] The process involves acquiring several defect images of the object under test and marking these images for defects; using these defect images with the defect marks as training samples; establishing a defect recognition model and training the model using the training samples as sample data; inputting a single image from among the images constituting the complete surface into the defect recognition model to identify defects, obtaining the bounding box containing the defect region, and determining the position coordinates of several corner points of the bounding box in the single image. For example, the defect recognition model can employ deep learning models such as Artificial Neural Network (ANN), Convolutional Neural Network (CNN), and Feature Pyramid Network (FPN).

[0074] S143: The position coordinates of several corner points of the marker box in a single image are transformed by the homography matrix to determine the position coordinates of several corner points of the marker box in the foreground mask image in the single image.

[0075] Specifically, the steps include: determining the first set of position coordinates of the four corner points of the single image containing the defect area; determining the second set of position coordinates of the four corner points of the single image in the coordinate system of the foreground mask image or the stitched image; determining the homography matrix of the single image and the foreground mask image or the stitched image based on the first and second set of position coordinates of the four corner points; and converting the position coordinates of the corner points of the marking box in the single image to the position coordinates of the foreground mask image based on the homography matrix. S144: Determining the distance of the corner points of the marking box in the foreground mask image relative to the endpoints based on the position coordinates of the endpoints in the foreground mask image and the position coordinates of the corner points of the marking box in the foreground mask image. The position coordinates of the endpoints in the foreground mask or the stitched image, and the position coordinates of the corner points of the defect marking box in the foreground mask or the stitched image have already been determined. The distance of the corner points of the defect marking box in the foreground mask or the stitched image relative to each endpoint is then determined, where the endpoints are the leaf tip and the leaf root.

[0076] S145: Determine a corner point of a marker frame as a representative point of the defect region, and determine the distance of the defect region in the foreground mask image relative to the endpoint. By determining a corner point of a marker frame as a representative point of the defect region, the distance of the representative point of the defect region in the foreground mask image relative to the endpoint is determined, where the representative point of the defect region represents the defect region, thereby determining the distance of the defect region relative to the endpoint. For example, the corner point of a marker frame as a representative point of the defect region can be the corner point of a marker frame closest to the endpoint in the foreground mask image, the center focus of a corner point of a marker frame, the largest point among the corner points of the marker frame, etc. Determining a corner point of a marker frame as a representative point of the defect region is arbitrary or engineering-defined, and can be any corner point, center point, or intersection point of a marker frame.

[0077] In one embodiment of step S150, the position of the defect region on the actual measured object is determined by multiplying the ratio of the distance between the endpoints of the actual measured object to the distance between the endpoints of the foreground mask image by the distance of the defect region in the foreground mask image relative to one endpoint of the measured object. For example, the distances between the endpoints of the actual measured object and the distances between the endpoints of the foreground mask image are obtained. For instance, if the endpoints are the blade tip and blade root, the actual distance D between the blade tip and blade root of a wind turbine blade and the distance d between the blade tip and blade root in the foreground mask image or stitched image are obtained. The ratio is determined based on the obtained distances between the endpoints of the actual measured object and the distances between the endpoints of the foreground mask image, for example, by determining the ratio based on the actual distance D between the blade tip and blade root and the distance d between the blade tip and blade root in the foreground mask image or stitched image. Determine the ratio of real space to image space; based on the known distance A of the defect relative to the leaf tip in the foreground mask image, obtain the distance A of the defect relative to the leaf tip in the real leaf. Similarly, the distance of a defect relative to the leaf root can be obtained, and the true distance of other defects relative to the leaf tip and / or leaf root can also be obtained.

[0078] Another implementation involves pre-setting uniformly distributed markers on the actual object being tested and recording the distances between each actual marker and its actual endpoint; obtaining the distances between each marker and its endpoint in the foreground mask image or stitched image; determining correction parameters based on the ratio of the distances between actual markers and their actual endpoints to the corresponding distances between markers and their endpoints in the foreground mask image or stitched image; and adjusting the position of defects and / or markers on the actual object being tested based on the correction parameters.

[0079] For example, three evenly distributed markers are pre-set on a real leaf. This solution uses the leaf root as a reference to calculate the distance from the markers and defects to the leaf root. Those skilled in the art should understand that this solution is equally applicable when the leaf tip is used as a reference. The distances from the leaf root to the leaf tip are denoted as lps-01, lps-02, and lps-03, respectively. The position data of each real marker is recorded, generally referring to the position data of the lightning arrester LPS, indicating the distance between the marker and the leaf root. That is, the distances from points lps-01, lps-02, and lps-03 to the real leaf root are lps-01-D, lps-02-D, and lps-03-D, respectively. The marker positions lps-01, lps-02, and lps-03 in the foreground mask image or stitched image are determined according to the aforementioned method. The positions of s-02 and lps-03 from the leaf root are lps-01-d, lps-02-d, and lps-03-d, respectively. The position of the defect from the leaf root in the foreground mask image or stitched image is determined. For example, there are two defects, with positions from the leaf root of defects-01-d and defects-02-d, respectively. Among them, defects-01-d is less than lps-03-d and greater than lps-02-d, and defects-02-d is less than lps-01-d.

[0080] The adjustment ratio is determined based on the ratio of the pre-acquired position data of each real identifier to the identifier position data in the foreground mask image or stitched image obtained by the aforementioned method. Starting from the identifier lps-03, which is farthest from the leaf root, the adjustment ratio R03 of lps-03 is determined as R03 = lps-03-D / lps-03-d. For marker positions and defect positions with a distance less than lps-03-d, the scale is adjusted. For example, the actual positions corresponding to the distances of marker positions lps-01 and lps-02 from the leaf root in the foreground mask image or stitched image are adjusted to lps-01-d' and lps-02-d', where lps-01-d' = (lps-01-d) × R03 and lps-02-d' = (lps-02-d) × R03. The actual positions corresponding to the distances of defects from the leaf root in the foreground mask image or stitched image, defects-01-d and defects-02-d, are adjusted to defects-01-d' = (defects-01-d) × R03 and defects-02-d' = (defects-02-d) × R03, thus completing the adjustment and optimization based on lps-03.

[0081] Determine the adjustment ratio R02 = lps-02-D / lps-02-d for lps-02. For marker and defect locations smaller than lps-02-d, adjust the ratio accordingly. For example, in the foreground mask image or stitched image, adjust the actual position corresponding to the distance of marker lps-01 from the leaf root to lps-01-d', where lps-01-d' = (lps-01-d) × R02; in the foreground mask image or stitched image, adjust the actual position corresponding to the distance of defect defects-01-d from the leaf root to defects-01-d' = (defects-01-d) × R02, thus completing the adjustment and optimization based on lps-02.

[0082] Determine the adjustment ratio R01 of lps-01 = lps-01-D / lps-01-d. For marker positions and defect positions smaller than lps-01-d, make proportional adjustments. For example, in the foreground mask image or stitched image, the actual position corresponding to the defect distance from the leaf root, defect-01-d, is adjusted to defect-01-d' = (defects-01-d) × R01. This completes the adjustment and optimization based on lps-01, resulting in more accurate defect position data.

[0083] Embodiment 2 of this disclosure provides a specific implementation of a defect location calculation system 600, see [link to implementation details]. Figure 2 It includes:

[0084] Image acquisition unit 210: used to acquire several images covering at least one complete surface of the object under test;

[0085] Image stitching unit 220: used to stitch together several images of a complete surface to obtain a stitched image of the complete surface of the object under test;

[0086] Foreground extraction unit 230: used to perform foreground segmentation processing on the obtained stitched image to obtain a foreground mask image of a complete surface of the object under test;

[0087] Position determination unit 240: used to obtain the position of the defect in the foreground mask image relative to at least one end of the object under test;

[0088] Position conversion unit 250: used to determine the position of the defect on the real object being measured based on the ratio of the real object being measured to the foreground mask image.

[0089] It is understood that various aspects of the technical solutions disclosed herein can be implemented as systems, methods, or program products. Therefore, various aspects of the technical solutions disclosed herein can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which can be collectively referred to herein as "circuit", "unit", or "platform".

[0090] Those skilled in the art should understand that the various units or steps of the present disclosure described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computing device-executable program code, thereby storing them in a storage medium for execution by the computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit units, or multiple units or steps can be fabricated as a single integrated circuit unit.

[0091] Figure 3 A schematic diagram of a defect location calculation device is shown according to some embodiments of this disclosure. Referring below... Figure 3 The defect location calculation device 600 implemented according to the embodiment described herein will be described in detail below. It will be understood that... Figure 3 The defect location calculation device 600 shown is merely an example and should not impose any limitation on the functionality and scope of use of any embodiment of the technical solution disclosed herein.

[0092] like Figure 3 As shown, the defect location computing device 600 is presented in the form of a general-purpose computing device. The components of the defect location computing device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including storage unit 620 and processing unit 610), a display unit 640, etc.

[0093] The storage unit stores program code, which can be executed by the processing unit 610, enabling the processing unit 610 to implement the various functional modules in the audio and video signal synchronization processing system described above in this embodiment.

[0094] Storage unit 620 may include a readable medium in the form of a volatile storage unit, such as random access unit (RAM) 6201 and / or cache storage unit 6202, and may further include read-only storage unit (ROM) 6203.

[0095] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each of these examples or some combination thereof may include an implementation of a network environment.

[0096] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, an image acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0097] The defect location computing device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with the defect location computing device 600, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, the electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of the electronic device 600 via bus 630. It should be understood that, although... Figure 2 As not shown in the diagram, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0098] In some embodiments of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, can implement the various functional units in the defect location calculation system disclosed above.

[0099] Although other specific implementations are not listed exhaustively in this embodiment, in some possible implementations, the various aspects described in this disclosure can also be implemented as a program product, which includes program code. When the program product is run on a terminal device, the program code is used to cause the terminal device to perform the steps described in the image stitching method area of ​​this disclosure according to the various embodiments of this disclosure.

[0100] Figure 4 A schematic diagram of the structure of a computer-readable storage medium is shown according to some embodiments of this disclosure. For example... Figure 4As shown, a program product 800 for implementing the above-described method according to an embodiment of the present disclosure is described. This product may be a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a terminal device, such as a personal computer. Of course, the program product generated according to this embodiment is not limited to this. In the present disclosure, the readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0101] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0102] Computer-readable storage media may include data signals propagated in baseband or as a carrier wave region, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0103] Program code for performing the operations of the disclosed technical solutions can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, locally on the user's device, as a standalone software package, locally on the user's computing device, locally on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0104] In summary, the technical solution proposed in this disclosure enables the accurate determination of the true defect location on the tested object, facilitating maintenance personnel in pinpointing the defect location and simplifying the defect location search process. By pre-distributing uniformly arranged markers on the tested object, and optimizing the actual distance between the defect and the endpoint using a proportional recursive adjustment method based on the distance between the actual endpoint of the tested object and the uniformly arranged markers, a more precise defect location in the actual blade can be obtained.

[0105] The above description is only a description of the preferred embodiment of the technical solution disclosed herein, and is not intended to limit the scope of the technical solution disclosed herein. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. A defect position calculation method characterized by comprising: The method comprises the following steps: acquiring a plurality of images covering at least one complete surface of a measured object; stitching the plurality of images of the acquired complete surface to obtain a stitched image of the complete surface of the measured object; performing foreground segmentation on the obtained stitched image to obtain a foreground mask image of the complete surface of the measured object; acquiring the position of a defect region in the foreground mask image relative to at least one end point of the measured object, comprising the following steps: performing image recognition on the foreground mask image through an end point recognition model to recognize the end point of the measured object and determine the position coordinates of the end point in the foreground mask image; performing defect recognition on a single image of the plurality of images constituting the complete surface through a defect recognition model to obtain a marking box in which the defect region is located and determine the position coordinates of a plurality of corner points of the marking box in the single image; converting the position coordinates of the plurality of corner points of the marking box in the single image through a homography matrix to determine the position coordinates of the plurality of corner points of the marking box in the foreground mask image in the single image; determining the distance of the plurality of corner points of the marking box relative to the end point in the foreground mask image according to the position coordinates of the end point in the foreground mask image and the position coordinates of the plurality of corner points of the marking box in the foreground mask image; determining a corner point of a marking box as a representative point of a defect region and determining the distance of the defect region relative to the end point in the foreground mask image; determining the position of the defect region in the real measured object according to the ratio of the real measured object to the foreground mask image.

2. The defect position calculation method according to claim 1, wherein The method of performing image recognition on the foreground mask image through an end point recognition model to recognize the end point of the measured object and determine the position coordinates of the end point in the foreground mask image specifically comprises: acquiring a plurality of images of the measured object with an end point and marking the end point; taking the plurality of images with the end point mark as training samples; establishing an end point recognition model and training the model with the training samples as sample data; inputting the foreground mask image into the end point recognition model to recognize the end point and determine the position coordinates of the recognized end point in the foreground mask image.

3. The defect position calculation method according to claim 1, wherein The method of performing defect recognition on a single image of the plurality of images constituting the complete surface through a defect recognition model to obtain a marking box in which the defect region is located and determine the position coordinates of a plurality of corner points of the marking box in the single image comprises: acquiring a plurality of defect images of the measured object and marking the defect images; taking the plurality of defect images with the defect marks as training samples; establishing a defect recognition model and training the model with the training samples as sample data; inputting a single image of the plurality of images constituting the complete surface into the defect recognition model to recognize defects, obtain a marking box in which the defect region is located, and determine the position coordinates of a plurality of corner points of the marking box in the single image.

4. The defect position calculation method according to claim 1, wherein The method of converting the position coordinates of the plurality of corner points of the marking box in the single image through a homography matrix to determine the position coordinates of the plurality of corner points of the marking box in the foreground mask image in the single image further comprises: determining a first set of position coordinates of four corner points of the single image where the defect region is located; determining a second set of position coordinates of the four corner points of the single image in a coordinate system of the foreground mask image or the stitched image; determining a homography matrix of the single image and the foreground mask image or the stitched image according to the first set of position coordinates and the second set of position coordinates of the four corner points; converting position coordinates of the corner points of the mark frame in the single image to position coordinates of the foreground mask image according to the homography matrix.

5. The defect position calculation method according to claim 1, wherein The step of determining the position of the defect region in the real object according to the proportion of the real object and the foreground mask image further comprises the following steps: obtaining the distance between the end points of the real object and the distance between the end points of the foreground mask image; determining the ratio according to the obtained distance between the end points of the real object and the distance between the end points of the foreground mask image; determining the position of the defect in the real object according to the product of the ratio of the distance between the end points of the real object and the distance between the end points of the foreground mask image and the distance of the defect region in the foreground mask image relative to one end point of the object.

6. The defect position calculation method according to claim 5, wherein The step of determining the position of the defect region in the real object according to the proportion of the real object and the foreground mask image further comprises the following steps: previously setting uniform distribution of marks on the real object and recording the distance of each real mark to the real end point respectively; obtaining the distance of each mark to the end point in the foreground mask image or the stitched image; determining the correction parameter according to the distance of the real mark to the real end point and the ratio of the distance of the mark to the end point in the foreground mask image or the stitched image corresponding to the real mark to the real end point; adjusting the position of the defect and / or the mark in the real object according to the correction parameter.

7. A defect position calculation system characterized by comprising: It comprises: an image acquisition unit for obtaining a plurality of images covering at least one complete surface of the object to be measured; an image stitching unit for stitching the plurality of images obtained to obtain a stitched image of the complete surface of the object to be measured; a foreground extraction unit for performing foreground segmentation on the obtained stitched image to obtain a foreground mask image of the complete surface of the object to be measured; a position determination unit for obtaining the position of the defect region in the foreground mask image relative to at least one end point of the object to be measured, comprising image recognition of the foreground mask image by an end point recognition model to recognize the end point of the object to be measured and determine the position coordinates of the end point in the foreground mask image; The single image in the several images constituting the complete surface is subjected to defect recognition by a defect recognition model, a mark box in which a defect region is located is obtained, and position coordinates of several corner points of the mark box in the single image are determined; the position coordinates of the several corner points of the mark box in the single image are converted by a homography matrix to determine position coordinates of the several corner points of the mark box in the foreground mask image in the single image; and distances of the several corner points of the mark box relative to the end point in the foreground mask image are determined according to the position coordinates of the end point in the foreground mask image and the position coordinates of the several corner points of the mark box in the foreground mask image. A corner point of a mark box is determined as a representative point of a defect region, and a distance of the defect region relative to the end point in the foreground mask image is determined. A position conversion unit is configured to determine a position of the defect region in a real object according to a ratio of the real object to the foreground mask image.

8. A computer device, comprising: The method comprises: a memory configured to store a processing program; a processor configured to implement the defect position calculation method according to any one of claims 1 to 6 when executing the processing program.

9. A readable storage medium, characterized by, The readable storage medium stores a processing program, and the processing program is configured to implement the defect position calculation method according to any one of claims 1 to 6 when executed by a processor.

Images