Multi-stage image enhancement and adaptive MPC fused unmanned ship control method

Abstract

The invention provides a multi-stage image enhancement and adaptive MPC fused unmanned ship control method, and relates to the technical field of unmanned ship intelligent control. The method comprises the following steps: acquiring a tunnel inner wall image acquired by an unmanned ship, analyzing a leading texture direction formed by long-term water flow scouring, and determining a texture direction reference for constraining image enhancement; performing multi-stage image enhancement based on the texture direction reference to generate an enhanced image for suppressing reflection interference; a texture abnormal area of the enhanced image is predicted according to the texture direction reference, and an inspection target area is determined; in combination with the current navigation state of the unmanned ship and the position of the inspection target area, a navigation trajectory is predicted by adopting a self-adaptive model prediction control method, and a course angle reaching the inspection target area is determined and output accordingly, so that accurate inspection and stable navigation control of the unmanned ship in the complex tunnel environment are realized.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology for unmanned vessels, and in particular to an unmanned vessel control method that integrates multi-stage image enhancement and adaptive MPC. Background Technology

[0002] With the rapid expansion of water conservancy engineering facilities and the continuous improvement of the sophistication of operation and maintenance management, the inspection tasks for enclosed hydraulic structures such as tunnels and culverts are becoming increasingly frequent. However, traditional manual inspection methods are not only inefficient and costly, but also limited by the complex environment and poor lighting conditions inside tunnels, making it difficult to accurately detect and locate anomalies on the facility surface. Although unmanned surface vessel (USV) technology has been gradually applied to the field of water conservancy facility inspection in recent years, existing USV inspection methods often lack effective mechanisms for handling complex light and shadow interference and texture blurring in the special environment of water conservancy facilities. This results in insufficient environmental adaptability and poor positioning accuracy during the inspection process, making it difficult to meet the high standards required for precise inspection of modern water conservancy facilities.

[0003] Furthermore, most existing unmanned surface vessel (USV) inspection methods implement image processing and motion control independently, lacking an effective integration of the synergistic relationship between image enhancement and path planning control. This results in situations where visual perception and navigation control cannot effectively coordinate during actual inspections, thus affecting the accuracy and stability of inspection operations. Therefore, there is an urgent need for an USV inspection method that can effectively integrate image processing and motion control processes to overcome the complexity of the water conservancy project inspection environment and improve the overall quality and efficiency of inspection operations. Summary of the Invention

[0004] This invention aims to solve at least one of the technical problems existing in the prior art; to this end, this invention proposes a multi-stage image enhancement and adaptive method. MPC An integrated unmanned vessel control method.

[0005] To achieve the above objectives, the present invention provides the following technical solution: Multi-stage image enhancement and adaptation MPC The integrated unmanned vessel control method includes: Images of the tunnel interior wall collected by an unmanned surface vessel are acquired, and a texture direction reference for constraining the image enhancement process is determined based on the dominant texture direction formed by long-term water erosion in the tunnel interior wall images. ; Based on the texture direction reference Multi-stage image enhancement was performed on the tunnel interior wall image to obtain an enhanced image that suppresses reflection interference and has continuous texture features; Based on the texture direction reference Predict the texture distribution in the enhanced image to obtain texture anomalous regions. ; and based on the texture anomaly region Determine the target area for inspection; Based on the current navigation status of the unmanned surface vessel (USV) and its location relative to the target area being inspected, the USV's navigation trajectory is predicted. ; and based on the navigation trajectory Based on the relative positional relationship with the navigation state, determine and output the heading angle of the unmanned vessel to reach the inspection target area. .

[0006] Furthermore, the texture orientation reference used to constrain the image enhancement process The methods for determining this include: Based on the grayscale characteristics of the tunnel inner wall image, the texture region is divided to obtain multiple initial texture regions; The pixel grayscale gradient directions of each initial texture region are counted separately to obtain the gradient direction distribution vector of each initial texture region. ; Gradient direction distribution vector between initial texture regions The included angle is used as a similarity condition, which will satisfy the condition that the included angle is less than a preset angle threshold. The adjacent initial texture regions are merged to obtain a merged texture region with the same texture direction; The area of ​​each merged texture region is calculated, and the average gray-level gradient direction of the merged texture region with the largest area is determined as the texture direction reference. .

[0007] Furthermore, the process of dividing the multiple initial texture regions is as follows: A spatial coordinate system is established based on the images of the tunnel's inner wall to determine the spatial position of each pixel. Calculate the grayscale gradient magnitude of each pixel. and grayscale gradient direction The grayscale gradient magnitude Greater than the preset gradient threshold The pixels are marked as texture seed pixels; Starting from each texture seed pixel, based on spatial location and grayscale gradient direction Constraints are imposed, and the region growing method is used to expand outwards, generating multiple initial texture regions with clear boundaries.

[0008] Furthermore, the enhancement process of the multi-stage image enhancement includes: Based on the texture direction reference As a directional constraint, a gradient direction-correlated spatial filtering process is performed on the tunnel inner wall image to obtain a gradient direction-filtered image that characterizes the main texture direction response; Based on the gray-level gradient direction of each pixel in the gradient direction filtered image With respect to the texture direction reference The angle difference between Calculate the filter gain coefficient for each pixel. ; Based on the filter gain coefficient The gradient direction filtered image is weighted and enhanced to generate a textured image with direction-selective enhancement; Calculate the local pixel grayscale variance based on the textured image. A reflective area is obtained; and the texture direction is referenced. Under the constraints, directional texture reconstruction and illumination compensation are performed on the reflective areas to output an enhanced image that suppresses reflective interference and has continuous texture features.

[0009] Furthermore, the directional texture reconstruction and illumination compensation employ a dual-domain decomposition and repair method based on joint modeling of the structural domain and the texture domain. The repair method includes: The reflective area and its corresponding neighborhood image are decomposed to obtain the structural layer component representing the illumination and geometric changes of the tunnel inner wall and the texture layer component representing the surface detail features. For the structural layer components, based on the geometric continuity of the tunnel wall within the neighborhood of the reflective region, a least-squares fitting method is used to model the neighborhood illumination distribution using a polynomial surface. This polynomial surface is then used to predict the structural layer pixel values ​​within the reflective region. ; For the texture layer components, based on the texture direction reference An anisotropic structural tensor field reflecting the dominant direction of texture is constructed, and a texture diffusion model is established under the constraint of the structural tensor field. Solving the texture diffusion model allows neighborhood texture features to propagate along the principal feature direction of the structure tensor field into the reflective region, thereby obtaining texture layer restoration values. ; Repair the texture layer value With the pixel values ​​of the structural layer The process involves fusion to complete the directional texture reconstruction and lighting compensation of the reflective areas.

[0010] Furthermore, the texture abnormality region The formation process includes: Reference along the texture direction The enhanced image is divided into multiple projection units arranged along the texture direction reference. And establish a spatial mapping index between each projection unit and the pixel coordinates in the enhanced image. ; Calculate the variation in texture continuity length between adjacent projection units. To obtain the distribution of texture continuity variation ; Based on the distribution of texture continuity changes Filter out the texture continuity length variation value Meets the preset texture continuity length variation threshold projection unit ; Using the spatial mapping index Extract the projection unit The corresponding pixel coordinate sets are merged to form texture anomaly regions. .

[0011] Furthermore, the obtained texture continuity variation distribution The acquisition process includes: Statistics satisfy the preset grayscale similarity conditions The length of the continuous pixel segment is used to obtain the projection unit. Corresponding texture duration ; Texture continuity length of adjacent projection units Perform differential calculations to obtain the variation value of texture continuity length. ; Changes in the length of each texture Arranged sequentially along the projection unit numbers, forming a texture continuity and variation distribution. .

[0012] Furthermore, the method for determining the inspection target area includes: Statistical analysis of the texture anomaly regions The set of pixel coordinates is the extreme values ​​along the horizontal and vertical axes in the image coordinate system where the enhanced image is located, and the minimum bounding rectangle region is constructed. ; Acquire preset imaging parameters and real-time attitude information of the unmanned vessel, and establish a spatial mapping relationship between the image coordinate system and the navigation coordinate system in which the unmanned vessel is currently located; Using the aforementioned spatial mapping relationship, the minimum bounding rectangle region is calculated. The physical location coordinates of the geometric center in the navigation coordinate system and the minimum bounding rectangle region and its physical location coordinates The area is identified as the inspection target area.

[0013] Furthermore, the current navigation status of the unmanned vessel includes: current navigation speed, heading angle, and position; The predicted navigation trajectory of the unmanned vessel employs a model predictive control method, including: Based on the current speed, heading angle, and position of the unmanned vessel in the navigation coordinate system, determine the state evolution characteristics of the unmanned vessel in the prediction time domain; The physical location coordinates of the inspection target area By introducing the aforementioned state evolution characteristics, a trajectory optimization objective function that satisfies preset constraints is constructed. ; The trajectory optimization objective function is solved in the prediction time domain to obtain the optimal control input sequence of the unmanned surface vessel in the navigation coordinate system. and the corresponding navigation trajectory .

[0014] Furthermore, the heading angle of the unmanned vessel upon reaching the inspection target area is determined and output. ,include: Based on the rolling optimization principle of model predictive control, from the optimal control input sequence Extract the optimal control input vector at the current moment; The optimal control input vector is analyzed to obtain the optimal heading angular velocity, and the target heading angle for the unmanned surface vessel to reach the inspection target area is calculated and output in combination with the current heading angle. .

[0015] Compared with the prior art, the beneficial effects of the present invention are: This invention constructs a multi-stage image enhancement method based on the dominant texture direction of the tunnel inner wall, which eliminates the problems of image reflection interference and texture discontinuity during the inspection of water conservancy facilities, improves the effectiveness of abnormal area identification and the reliability of image processing results, and provides technical support for the accurate determination of subsequent inspection targets.

[0016] This invention achieves accurate conversion of abnormal regions from two-dimensional images to physical space through spatial mapping between texture abnormal regions and navigation coordinate systems, solving the problem of large target area positioning errors in existing unmanned vessel inspection methods and improving the accuracy of determining the physical location of inspection target areas.

[0017] This invention integrates multi-stage image enhancement and adaptive model predictive control methods to achieve collaborative processing between visual perception results and unmanned vessel motion control, thereby improving the stability and accuracy of unmanned vessel trajectory planning and control in complex water conservancy environments, and thus enhancing the automation level and practical application effect of water conservancy facility inspection operations. Attached Figure Description

[0018] Figure 1 This is a flowchart of the unmanned vessel control method based on the multi-stage image enhancement and adaptive MPC fusion method in Example 1.

[0019] Figure 2As in Example 1 Sobel A schematic diagram of operator image gray-level gradient calculation.

[0020] Figure 3 This is a schematic diagram illustrating the principle of texture projection anomaly region detection in Example 1.

[0021] Figure 4 This is an architecture diagram of the unmanned vessel control system based on the fusion of multi-stage image enhancement and adaptive MPC in Example 2. Detailed Implementation

[0022] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Example 1 Please see Figure 1 This invention provides a method for controlling unmanned vessels that integrates multi-stage image enhancement and adaptive MPC, including: Images of the tunnel interior wall collected by an unmanned surface vessel are acquired, and based on the dominant texture direction formed by long-term water erosion in the interior wall images, a texture direction reference is determined to constrain the image enhancement process. ; Based on the texture direction reference Multi-stage image enhancement was performed on the tunnel interior wall image to obtain an enhanced image that suppresses reflection interference and has continuous texture features; Based on the texture direction reference Predict the texture distribution in the enhanced image to obtain texture anomalous regions. ; and based on the texture anomaly region Determine the target area for inspection; Based on the current navigation status of the unmanned surface vessel (USV) and its location relative to the target area being inspected, the USV's navigation trajectory is predicted. ; and based on the navigation trajectory Based on the relative positional relationship with the navigation state, determine and output the heading angle of the unmanned vessel to reach the inspection target area. .

[0024] It should be noted that the tunnel interior images in this embodiment were obtained from an industrial-grade high-resolution image mounted on an unmanned vessel. CCD The camera captures data in real time; CCDThe camera is mounted on a fixed gimbal at the front of the unmanned vessel and collects real-time image sequences of the tunnel wall in the direction the unmanned vessel is moving. The resolution of each frame is set to 1920×1080 pixels, and the sampling frame rate is set to 30 frames per second to ensure the spatial continuity of the image data and the clarity of the texture features.

[0025] Specifically, the method for determining the texture orientation reference used to constrain the image enhancement process includes: Based on the grayscale characteristics of the tunnel inner wall image, the texture region is divided to obtain multiple initial texture regions; It should be noted that in this embodiment, the image of the tunnel inner wall is first converted into a grayscale image using a grayscale processing method to reduce redundant color information and lower computational complexity; the calculation formula used for grayscale processing is as follows: ; In the formula, This indicates the image after grayscale processing at the [number]th [time]. line, number The grayscale value of the column pixels, , , These represent the red, green, and blue components of the corresponding pixels in the original color image, respectively. This indicates the coordinate position of a pixel.

[0026] In implementation, the process of dividing the initial texture region is as follows: A spatial coordinate system is established based on the images of the tunnel's inner wall to determine the spatial position of each pixel. In specific implementation, a two-dimensional image space coordinate system is established with the upper left corner of the image as the origin, where the horizontal axis ( X The vertical axis ( ) increases along the column direction of the image, and the vertical axis ( ) increases along the column direction of the image. Y The axis increases along the row direction of the image; any pixel in the image The corresponding spatial coordinates are defined as follows: ; In the formula, Indicates the first in the image line, number The pixel at the column position, Represents pixels The position index value along the horizontal axis in the image space coordinate system. Represents pixels The position index value along the vertical axis in the image space coordinate system.

[0027] Calculate the grayscale gradient magnitude of each pixel. and grayscale gradient direction The grayscale gradient magnitude Greater than the preset gradient threshold The pixels are marked as texture seed pixels; It should be noted that the grayscale gradient magnitude of each pixel in this embodiment... and grayscale gradient direction pass Sobel The gradient operator is calculated using the following formula: , ; In the formula, Indicates the first in the image line, number The grayscale gradient magnitude of the pixel at the column position is used to describe the texture intensity at that position; Indicates the first in the image line, number The grayscale gradient direction of the column position pixel, with a value range of -90° to 90°; , These represent the first and second parts of the image, respectively. line, number The grayscale gradient components of the column position pixel in the horizontal and vertical directions.

[0028] Wherein, the gray-level gradient component value and Obtained through the following convolution calculation: , ; In the formula, Represents the grayscale value of neighboring pixels. and They represent Sobel Gradient calculation kernels for the operator in the horizontal and vertical directions. and These represent the relative offset positions of the convolution kernel function in the vertical and horizontal directions, respectively.

[0029] It should be understood that the gradient calculation kernel in the horizontal axis direction gradient calculation kernel along the vertical axis It can be represented as: , ; like Figure 2 As shown, the gradient calculation kernel in the horizontal axis direction This is used to calculate the gray-level gradient components of an image along the horizontal axis, thereby responding to vertical edge features; the gradient calculation kernel along the vertical axis... It is used to calculate the gray-level gradient components of an image along the vertical axis, thereby responding to lateral edge features.

[0030] It should be noted that the relative offset position in the vertical direction... Relative offset position in the horizontal direction The value range is −1, 0, +1, and it is used to perform gradient calculations within a local 3×3 pixel neighborhood to capture the spatial variation characteristics of gray levels in local areas of the image.

[0031] Subsequently, this embodiment sets a preset gradient threshold. The gradient threshold is calculated using the following formula to identify valid texture seed pixels: ; In the formula, This represents the gradient threshold scaling factor, used to limit pixels with significant texture features; its value ranges from 0.05 to 0.1. This represents the maximum value of the grayscale gradient of all pixels in the entire image.

[0032] Image elements that satisfy the grayscale gradient magnitude condition The pixels marked are used as texture seed pixels for the generation of subsequent initial texture regions.

[0033] Starting from each texture seed pixel, based on spatial location and grayscale gradient direction Constraints are imposed, and the region growing method is used to expand outwards, generating multiple initial texture regions with clear boundaries.

[0034] In a specific implementation, the process of the region growing method is as follows: First, starting from each marked texture seed pixel, examine all unmarked pixels within an 8-neighborhood of the seed pixel to determine whether they simultaneously satisfy the spatial position constraint and the gray-level gradient direction constraint. The spatial location constraint means that candidate neighboring pixels must be located within the 8-neighborhood of the current seed pixel and must not be marked as belonging to other texture regions, in order to ensure the spatial connectivity of texture regions; specifically, for the current seed pixel Examine its surrounding coordinates as The adjacent pixels, where, And not all of them are 0.

[0035] The gray-level gradient direction constraint refers to the difference in the angle between the gray-level gradient directions of the candidate neighboring pixels and the current seed pixel. The following conditions must be met: ; In the formula, Indicates the direction of the gray-level gradient of the candidate neighboring pixels. This indicates the grayscale gradient direction of the current texture seed pixel. This represents the allowable extension deviation angle threshold for the gradient direction, which is set to 15° in this embodiment.

[0036] When the above two constraints are met, the candidate neighboring pixels are marked and added to the current initial texture region, and the pixel is used as a new seed pixel to continue the region expansion; the above region expansion process is repeated until there are no new pixels around the current initial texture region that meet the expansion conditions; by processing all texture seed pixels through the above region growth method, multiple spatially continuous and clearly defined initial texture regions are finally obtained as the basis for further texture region merging processing.

[0037] Calculate the pixel grayscale gradient direction for each initial texture region. Obtain the gradient direction distribution vector for each initial texture region. ; In this specific implementation, the gradient directions of all pixels within the initial texture region are statistically analyzed to obtain the gradient direction distribution characteristics corresponding to each initial texture region. Specifically, a statistical method of averaging direction vectors is used to reduce angular periodic errors. The specific process is as follows: First, define the gradient direction vector for each pixel. Represented as: ; Then, the gradient direction vectors of all pixels within the region are averaged to obtain the gradient direction distribution feature vector of the initial texture region. .

[0038] Gradient direction distribution vector between initial texture regions The included angle is used as a similarity condition, which will satisfy the condition that the included angle is less than a preset angle threshold. The adjacent initial texture regions are merged to obtain a merged texture region with the same texture direction; It is understandable that this embodiment uses gradient direction distribution feature vectors. The included angle between two adjacent initial texture regions serves as the criterion for region merging. A and B Angle between The calculation method is as follows: ; In the formula, Represents the initial texture region A and B The angle between the characteristic vectors of the gradient direction distribution between them. and These represent the initial texture regions. A andB The gradient direction distribution characteristic vector, This represents the magnitude of a vector.

[0039] When the calculated included angle Meet the conditions At that time, the initial texture area A and B If the directions are similar, merge the elements; a preset angle threshold is used. The value ranges from 10° to 15°.

[0040] Calculate the area of ​​each merged texture region, select the merged texture region with the largest area, and determine the average gray-level gradient direction of this merged texture region as the texture direction reference. .

[0041] It should be noted that the area of ​​the merged texture region is obtained by counting the total number of pixels within the region, and the merged texture region with the largest number of pixels is selected as the dominant texture region; the texture direction reference corresponding to the dominant texture region is... The calculation method, using vector averaging, is as follows: , ; In the formula, This represents the average vector value of the pixel gradient direction within the dominant texture region. This indicates the number of pixels in the dominant texture area. Indicates the first in the dominant texture region The grayscale gradient direction of each pixel. This represents the final determined texture orientation reference.

[0042] In practice, the enhancement process of the multi-stage image enhancement includes: Based on the texture direction reference As a directional constraint, a gradient direction-correlated spatial filtering process is performed on the tunnel inner wall image to obtain a gradient direction-filtered image that characterizes the main texture direction response; It should be noted that this embodiment is based on a texture direction reference. A direction-selective Gaussian filter is used to spatially filter the original tunnel wall image to highlight the reference texture direction. Consistent dominant texture features, and effective suppression of background noise and interference information from other directions.

[0043] Based on the gray-level gradient direction of each pixel in the gradient direction filtered image With respect to the texture direction reference The angle difference between Calculate the filter gain coefficient for each pixel. The specific calculation formula is as follows: ; In the formula, This represents the directional difference attenuation parameter.

[0044] It should be noted that the directional difference attenuation parameter Used to adjust the degree of gain attenuation as the angle difference increases, with a value ranging from 10° to 20°.

[0045] Based on the filter gain coefficient The gradient direction filtered image is weighted and enhanced to generate a textured image with direction-selective enhancement; In specific implementation, the filter gain coefficient Gradient direction filtering is applied pixel-by-pixel to the grayscale values ​​of the image. This enhances directional selectivity and generates images with clear textures; the calculation formula is as follows: ; In the formula, This represents the image after weighted enhancement by the filter gain coefficients. line, number The grayscale value of the column pixels, Indicates the first gradient direction filtered image. line, number The grayscale value of the column pixels.

[0046] Based on the local pixel grayscale variance of the textured image Determine the reflective area and base it on the texture direction. Under the constraints, directional texture reconstruction and illumination compensation are performed on the reflective area to output an enhanced image that suppresses reflective interference and has continuous texture features.

[0047] It should be noted that this embodiment calculates the pixel grayscale variance of local regions in a textured image. The reflective area is defined as the location where the grayscale variance value is greater than or equal to a preset variance threshold. The calculation formula is as follows: ; In the formula, Represented in pixels The grayscale variance within a local window centered on the value. Represents pixels A 5x5 pixel neighborhood window centered on the center, This represents the total number of pixels within the neighborhood window. This represents the average grayscale value of the pixels within the window.

[0048] The directional texture reconstruction and illumination compensation employ a dual-domain decomposition and repair method based on joint modeling of the structural domain and the texture domain. The repair method includes: The reflective area and its corresponding neighborhood image are decomposed to obtain the structural layer component representing the illumination and geometric changes of the tunnel inner wall and the texture layer component representing the surface detail features. It should be noted that this embodiment uses the dual-domain image decomposition method to decompose the reflective area to be repaired and its neighboring images into a structural layer and a texture layer, respectively. The structural layer is used to describe low-frequency information such as illumination and geometry, while the texture layer is used to characterize high-frequency information such as texture details. Specifically, this embodiment uses guided filtering to smooth the image to obtain the structural layer component, and subtracts the structural layer component from the original image to obtain the texture layer component.

[0049] For the structural layer components, based on the geometric continuity of the tunnel wall within the neighborhood of the reflective region, a least-squares fitting method is used to model the neighborhood illumination distribution using a polynomial surface. This polynomial surface is then used to predict the structural layer pixel values ​​within the reflective region. ; In this specific implementation, the structural layer pixels within the neighborhood of the reflective area are selected as sample points to perform polynomial surface modeling on the illumination distribution. The polynomial surface model can be expressed as: ; In the formula, Indicates spatial location The predicted pixel values ​​of the structural layer are obtained at that point. , , , , as well as This represents the polynomial coefficients determined by fitting the pixel data of the structural layer within the neighborhood of the reflective region, used to characterize the trend of illumination and geometric changes within the neighborhood.

[0050] By substituting the structural layer pixel values ​​and their corresponding spatial coordinates within the neighborhood of the reflective area into the above model, and using the least squares method to solve the model parameters, the predicted structural layer values ​​corresponding to each pixel position within the reflective area can be obtained, thereby completing the reconstruction of the structural layer components of the reflective area.

[0051] For the texture layer components, based on the texture direction reference An anisotropic structural tensor field reflecting the dominant direction of texture is constructed, and a texture diffusion model is established under the constraint of the structural tensor field. In practical implementation, this embodiment is based on texture direction reference. An anisotropic structure tensor field is constructed in the texture layer components to describe the propagation characteristics of the texture in different directions; the structure tensor field at each pixel location is represented as: ; In the formula, Indicates pixel position The structure tensor matrix at the location is used to describe the diffusion weights of the texture in the dominant direction and its orthogonal directions.

[0052] Under the constraints of the structure tensor field, a texture diffusion model is established, and its diffusion process is described by the following partial differential equation: ; In the formula, This represents the grayscale value distribution of the texture layer components. The evolution parameters representing the diffusion process, and These represent the gradient operator and the divergence operator, respectively.

[0053] Solving the texture diffusion model allows neighborhood texture features to propagate along the principal feature direction of the structure tensor field into the reflective region, thereby obtaining texture layer restoration values. ; Repair the texture layer value With the pixel values ​​of the structural layer The process involves fusion to complete the directional texture reconstruction and lighting compensation of the reflective areas.

[0054] In the implementation process, this embodiment uses the finite difference method to numerically iterate and solve the above texture diffusion model, so that the texture layer information at the boundary of the reflective area gradually propagates into the interior of the area, obtaining the texture layer restoration value corresponding to each pixel in the reflective area. ; and with the pixel values ​​of the structural layer Pixel-by-pixel stacking is performed to obtain the final pixel grayscale value within the reflective area. Specifically, it is expressed as: .

[0055] Through the above-mentioned structural layer reconstruction and texture layer diffusion fusion processing, directional texture reconstruction and illumination compensation of reflective areas are achieved, thereby obtaining an enhanced image that suppresses reflective interference and has continuous texture features, which can be used for subsequent texture anomaly detection and unmanned vessel inspection control.

[0056] During implementation, the process of forming the textured abnormal region includes: Reference along the texture direction The enhanced image is decomposed into directions and divided into multiple sequentially arranged projection units. And establish each projection unit Spatial mapping index of pixel coordinates in enhanced image ; Specifically, in this embodiment, as Figure 3 As shown, the reference along the texture direction The enhanced image is oriented and unfolded according to a reference perpendicular to the texture direction. The direction is divided into a series of references along the texture direction. Extended rectangular strip regions, each rectangular strip region is defined as a projection unit. The projection unit Arranged closely along the direction perpendicular to the texture reference, forming a texture projection sequence. .

[0057] It should be noted that each projection unit width Set to a width of 10 to 20 pixels, with the width direction perpendicular to the texture direction reference; each projection unit length Set as reference along the texture direction The length of the extension is equal to the full span of the enhanced image on the texture orientation reference to ensure that full-length texture continuity can be statistically accounted for.

[0058] Subsequently, each projection unit was established. The set of spatial coordinates of corresponding pixels in the original enhanced image The mapping relationship between them generates a spatial mapping index. The spatial mapping index Specifically, it is expressed as follows: .

[0059] Calculate the variation in texture continuity length between adjacent projection units. To obtain the distribution of texture continuity variation ; The method for calculating the variation in texture continuity length between adjacent projection units includes: Statistics satisfy the preset grayscale similarity conditions The length of the continuous pixel segment is used to obtain the projection unit. Corresponding texture duration ; It should be understood that grayscale similarity conditions This refers to any two adjacent pixels. and If its corresponding enhanced image grayscale value A color is considered similar in grayscale if the following conditions are met: ; In the formula, This represents the grayscale similarity threshold, with a value ranging from 5 to 10 grayscale levels.

[0060] Based on the above conditions, in each projection unit Internal reference along the texture direction Statistical analysis of each consecutive condition satisfying grayscale similarity The pixel segment length is determined, and the maximum length value is selected as the corresponding projection unit. Texture duration .

[0061] Texture continuity length of adjacent projection units Perform differential calculations to obtain the variation value of texture continuity length. ; It is understood that the difference calculation can be expressed as: ; In the formula, This represents the total number of parametric projection units, characterizing the projection units divided after the enhanced image is unfolded in the direction of projection. Total number This represents the variation in texture length, reflecting the degree of difference in texture continuity between adjacent projection units.

[0062] Changes in the length of each texture Along the projection unit number Arranged sequentially, forming a continuous and varied texture distribution. Specifically, it is expressed as: ; It is understandable that the texture continues to vary in distribution. Used to characterize enhanced images in texture orientation reference The textural continuity variation features.

[0063] Based on the distribution of texture continuity changes Filter out the texture continuity length variation value Meets the preset texture continuity length variation threshold projection unit ; In specific implementation, this embodiment first defines a threshold value that satisfies the preset texture continuity length variation. Projection unit set Specifically, it is expressed as: ; It should be noted that the preset texture continuity length variation threshold The value is set based on the actual inspection environment and the sensitivity to changes in the tunnel wall texture, and the range is the length of the projection unit. 10% to 20% of the data is used to identify locations in enhanced images where texture changes are significant, thus enabling effective screening of texture anomaly locations.

[0064] Using the spatial mapping index Extract the projection unit The corresponding pixel coordinate sets are merged to form texture anomaly regions. .

[0065] It is understood that this embodiment uses spatial mapping indexing. Map each of the projection units that meet the above conditions and extract each projection unit. The complete set of pixel coordinates within the enhanced image is obtained by merging the extracted pixel coordinate sets. The corresponding set of pixel coordinates.

[0066] Specifically, the method for determining the inspection target area includes: Statistical analysis of the texture anomaly regions The set of pixel coordinates is the extreme values ​​along the horizontal and vertical axes in the image coordinate system where the enhanced image is located, and the minimum bounding rectangle region is constructed. ; It should be noted that, although the texture abnormality area It contains all abnormal pixels, but their shapes are usually irregular and discrete; to facilitate navigation planning, it iterates through all pixel coordinates in the set of pixel coordinates corresponding to the texture abnormal regions. The minimum and maximum values ​​of the region along the horizontal and vertical axes in the image coordinate system are calculated to obtain the minimum bounding rectangle region. boundary coordinates This rectangular region not only filters out discrete noise points at the edges, but also provides a stable geometric descriptor for subsequent visual tracking.

[0067] Acquire preset imaging parameters and real-time attitude information of the unmanned vessel, and establish a spatial mapping relationship between the image coordinate system and the navigation coordinate system in which the unmanned vessel is currently located; It should be noted that the unmanned surface vessel (USV) operates in a tunnel environment without satellite signals. To achieve precise control, this embodiment predefines a navigation coordinate system. Specifically, the navigation coordinate system is a local inertial reference system, with the time when the USV enters the tunnel or the mission start point as its origin. , X The axis points in the direction of the tunnel's extension. Y The axis is perpendicular to the tunnel wall and points towards the water surface. ZThe axis obeys the right-hand rule; the unmanned surface vessel (USV) uses its onboard inertial measurement unit (IMU) IMU ) and Doppler velocimeters ( DVL It performs dead reckoning and maintains its own position in the coordinate system in real time.

[0068] To convert two-dimensional pixel information in an image into three-dimensional physical position in a navigation coordinate system, this embodiment establishes a pinhole camera imaging model based on homogeneous coordinates, specifically represented as follows: ; In the formula, This represents the homogeneous pixel coordinate vector of the target point in the image coordinate system, where , These are the pixel's horizontal and vertical coordinates, and their values ​​are... To augment homogeneity components; This represents the three-dimensional physical position coordinate vector of the target in the navigation coordinate system. This represents the real-time position coordinate vector of the unmanned vessel in the navigation coordinate system; This represents the intrinsic parameter matrix of the camera; The rotation matrix represents the transformation from the navigation coordinate system to the ship's coordinate system, and is constructed from the real-time attitude information (roll angle, pitch angle, and heading angle) of the unmanned vessel. This represents the depth distance of the target point relative to the optical center of the camera.

[0069] It should be understood that the preset imaging parameters mainly include the camera's focal length and the principal point coordinates of the optical axis on the image plane. These parameters together constitute the matrix in the above formula. In specific implementation, the preset imaging parameters can be obtained and pre-stored in the control system of the unmanned vessel in one of the following two ways: one is to directly read the standard optical parameters provided in the technical specifications provided by the camera at the time of manufacture as preset values; the other is to perform optical calibration of the camera in a laboratory environment using a standard checkerboard calibration board before the unmanned vessel performs the mission, and calculate the accurate intrinsic parameter data as preset values. In this embodiment, in order to ensure the measurement accuracy in the dim lighting environment of the tunnel, the second method of obtaining accurate parameters is preferred.

[0070] It should be noted that the depth distance It uses a laser rangefinder mounted on the front of the unmanned vessel to measure the real-time depth distance between the center point of the target area and the optical center of the camera.

[0071] It is understood that the pinhole camera imaging model based on homogeneous coordinates describes the spatial mapping relationship between the image coordinate system and the navigation coordinate system.

[0072] Using the spatial mapping relationship, the physical position coordinates of the geometric center of the minimum bounding rectangle region in the navigation coordinate system are calculated, and the minimum bounding rectangle region and its physical position coordinates are determined as the inspection target region.

[0073] In specific implementation, this embodiment first calculates the minimum bounding rectangle region. Geometric center pixel coordinates Specifically, it is expressed as: .

[0074] Subsequently, using the spatial mapping relationship established above, the geometric center pixel coordinates are... Mapping to the navigation coordinate system of the unmanned vessel, the physical coordinates of the geometric center in the navigation coordinate system are calculated. , serving as the location center of the inspection target area; The solution obtained This refers to the center location of the inspection target, which has a physical distance metric (unit: meters); ultimately, the system will define the minimum bounding rectangle area. (For visual display and tracking selection) and physical location coordinates (The distance error used for MPC controller calculation) is used to jointly determine the inspection target area.

[0075] After establishing the inspection target area and its physical location coordinates in the navigation coordinate system, this embodiment enters the automatic control phase; in order to achieve accurate close-range observation of the unmanned vessel in an environment without satellite signals, this embodiment adopts model predictive control (MMC). MPC Combined with a pure tracking control strategy.

[0076] Specifically, the current navigation status of the unmanned vessel includes: current navigation speed, heading angle, and position; The predicted navigation trajectory of the unmanned vessel employs a model predictive control method, including: The current speed, heading angle, and position of the unmanned vessel in the navigation coordinate system are obtained, and the state evolution characteristics of the unmanned vessel in the prediction time domain are determined. In practice, firstly, the current state information of the unmanned vessel in the navigation coordinate system is obtained, namely, its speed, heading angle, and position; then, the definition is... State vector at time step Specifically, it is expressed as: ; in, , Indicates the first The horizontal and vertical coordinate positions in the navigation coordinate system at any time. Indicates the first longitudinal speed at all times Indicates the first Heading angle at any time.

[0077] At the same time, define the control input vector. Specifically, it is expressed as: ; in, Indicates the first longitudinal acceleration at all times This indicates the angular velocity of the heading.

[0078] This embodiment, based on the discrete-time sampling principle, determines the state evolution characteristics of the unmanned vessel in the prediction time domain, specifically as follows: ; In the formula, Indicates the sampling time interval. Indicates the number of prediction steps, and , This indicates the length of the prediction time domain.

[0079] The physical location coordinates of the inspection target area are incorporated into the state evolution features to construct a trajectory optimization objective function that satisfies preset constraints. ; It is understandable that, using the aforementioned physical location coordinates Construct the following trajectory optimization objective function: ; In the formula, Represents the predicted trajectory point With physical location coordinates The weighted sum of squares of the Euclidean distance deviations between them was used to constrain the unmanned vessel to precisely navigate to the physical target point calculated by vision. This indicates a penalty for the magnitude of the control input, used to prevent excessive control input from saturating the actuator; The penalty represents the rate of change of the control quantity (i.e., the rate of change of acceleration and the rate of change of angular velocity), used to ensure the stability of the navigation attitude and avoid violent swaying of the hull. , and These represent the state error weight matrix, the control weight matrix, and the control increment weight matrix, respectively.

[0080] It should be understood that the preset constraints include accuracy constraints, control input amplitude, and control quantity constraints, and the specific values ​​are set according to the actual application requirements.

[0081] It should be noted that the weight matrix , and The specific values ​​were determined through experimental debugging based on different tunnel flow environments and unmanned vessel dynamic characteristics. This embodiment does not limit the specific weight values, but those skilled in the art should understand that increasing... It will improve the accuracy of arrival and increase It will improve sailing stability, and suitable parameters can be obtained through routine adjustments.

[0082] The trajectory optimization objective function is solved in the prediction time domain to obtain the optimal control input sequence and corresponding navigation trajectory of the unmanned vessel in the navigation coordinate system.

[0083] In this specific implementation, the trajectory optimization problem is modeled as a nonlinear optimization problem with state constraints and control input constraints, and a numerical optimization algorithm is used to solve the trajectory optimization objective function. Specifically, a sequential quadratic programming algorithm is selected as the numerical optimization algorithm. The optimization process is as follows: First, based on the current state vector of the unmanned vessel... and control input Using the state evolution characteristics of each prediction step within the prediction time domain as constraints, a nonlinear constraint set is formed; then, the objective function is used... To optimize the objective, a numerical iterative search process is executed. When a preset termination condition is met, the optimal control input sequence and optimal state vector corresponding to each prediction step in the prediction time domain are obtained. ; ; In the formula, This represents the optimal control input sequence. Represents the optimal state vector. Indicates the first The optimal control input vector corresponding to each prediction step contains the optimal longitudinal acceleration. With optimal heading angular velocity .

[0084] Furthermore, based on the aforementioned optimal state vector, a complete navigation trajectory sequence in the navigation coordinate system is obtained: ; The above navigation trajectory sequence The spatial position and attitude change trends of the unmanned vessel in the predicted time domain were determined, providing a basis for the next step of navigation trajectory tracking and heading control.

[0085] During implementation, the heading angle of the unmanned vessel upon reaching the inspection target area is determined and output. ,include: Based on the rolling optimization principle of model predictive control, from the optimal control input sequence Extract the optimal control input vector at the current moment; Understandably, based on model predictive control (… MPC The rolling optimization mechanism in this embodiment selects the first control component in the optimal control input sequence as the actual execution instruction, specifically as follows: ; In the formula, Indicates the current The actual control input vector applied to the unmanned vessel at all times. Represents the optimal control input sequence The first element in.

[0086] The optimal control input vector is analyzed to obtain the optimal heading angular velocity, and the target heading angle for the unmanned surface vessel to reach the inspection target area is calculated and output in combination with the current heading angle. .

[0087] In specific implementation, the extracted optimal control input vector is first processed... The analysis separates the longitudinal acceleration command and the yaw rate command, which are specifically expressed as follows: ; In the formula, This represents the optimal longitudinal acceleration at the current moment. This represents the optimal heading angular velocity at the current moment.

[0088] Subsequently, this embodiment, based on the Euler integral method, utilizes the optimal heading angular velocity. The current heading angle of the unmanned vessel Perform a simulation to calculate the target heading angle for the next control cycle. Specifically, it is expressed as: ; In the formula, This indicates the actual heading angle of the unmanned vessel in the navigation coordinate system at the current moment. This indicates the sampling time interval of the controller.

[0089] Finally, the system will determine the target heading angle. Output to the underlying heading hold controller (e.g.) PID The servo controller drives the unmanned vessel's actuators to adjust its course, enabling the unmanned vessel to navigate precisely and smoothly to the inspection target area, achieving efficient automatic inspection and real-time control.

[0090] Example 2 like Figure 2As shown in the example, the parts not detailed in this embodiment are as shown in Example 1. This embodiment discloses a multi-stage image enhancement and adaptive method. MPC The integrated unmanned vessel control system includes: Texture reference determination module: Used to acquire images of the tunnel interior wall collected by the unmanned surface vessel, and determine the texture direction reference used to constrain the image enhancement process based on the dominant texture direction formed by long-term water flow erosion in the interior wall image. ; Image multi-level enhancement module: used to enhance the image based on the texture direction reference. Multi-stage image enhancement was performed on the tunnel interior wall image to obtain an enhanced image that suppresses reflection interference and has continuous texture features; Anomaly region identification module: used to identify regions based on the texture direction reference. Predict the texture distribution in the enhanced image to obtain texture anomalous regions. ; and based on the texture anomaly region Determine the target area for inspection; MPC The trajectory control module is used to predict the navigation trajectory of the unmanned surface vessel (USV) based on its current navigation status and the location relationship with the target area being inspected. ; and based on the navigation trajectory Based on the relative positional relationship with the navigation state, determine and output the heading angle of the unmanned vessel to reach the inspection target area. .

[0091] Some of the data in the above formula are calculated by removing dimensions and taking their numerical values. The formula is the closest to the real situation obtained by software simulation of a large amount of collected data. The preset parameters and preset thresholds in the formula are set by those skilled in the art according to the actual situation or obtained through simulation of a large amount of data.

[0092] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. Multi-stage image enhancement and adaptation MPC The integrated unmanned vessel control method is characterized by, include: Images of the tunnel interior wall collected by an unmanned surface vessel are acquired, and a texture direction reference for constraining the image enhancement process is determined based on the dominant texture direction formed by long-term water erosion in the tunnel interior wall images. ; Based on the texture direction reference Multi-stage image enhancement was performed on the tunnel interior wall image to obtain an enhanced image that suppresses reflection interference and has continuous texture features; Based on the texture direction reference Predict the texture distribution in the enhanced image to obtain texture anomalous regions. ; And based on the texture anomaly region Determine the target area for inspection; Based on the current navigation status of the unmanned surface vessel (USV) and its location relative to the target area being inspected, the USV's navigation trajectory is predicted. ; and based on the navigation trajectory Based on the relative positional relationship with the navigation state, determine and output the heading angle of the unmanned vessel to reach the inspection target area. .

2. The multi-stage image enhancement and adaptation method according to claim 1 MPC The integrated unmanned vessel control method is characterized by, The texture orientation reference used to constrain the image enhancement process The methods for determining this include: Based on the grayscale characteristics of the tunnel inner wall image, the texture region is divided to obtain multiple initial texture regions; The pixel grayscale gradient directions of each initial texture region are counted separately to obtain the gradient direction distribution vector of each initial texture region. ; Gradient direction distribution vector between initial texture regions The included angle is used as a similarity condition, and angles smaller than a preset angle threshold are considered. The adjacent initial texture regions are merged to obtain a merged texture region with the same texture direction; The area of ​​each merged texture region is calculated, and the average gray-level gradient direction of the merged texture region with the largest area is determined as the texture direction reference. .

3. The multi-stage image enhancement and adaptation method according to claim 2 MPC The integrated unmanned vessel control method is characterized by, The process of dividing the multiple initial texture regions is as follows: A spatial coordinate system is established based on the images of the tunnel's inner wall to determine the spatial position of each pixel. Calculate the grayscale gradient magnitude of each pixel. and grayscale gradient direction The grayscale gradient magnitude Greater than the preset gradient threshold The pixels are marked as texture seed pixels; Starting from each texture seed pixel, based on spatial location and grayscale gradient direction Constraints are imposed, and the region growing method is used to expand outwards, generating multiple initial texture regions with clear boundaries.

4. The multi-stage image enhancement and adaptation method according to claim 1 MPC The integrated unmanned vessel control method is characterized by, The multi-stage image enhancement process includes: Based on the texture direction reference As a directional constraint, a gradient direction-correlated spatial filtering process is performed on the tunnel inner wall image to obtain a gradient direction-filtered image that characterizes the main texture direction response; Based on the gray-level gradient direction of each pixel in the gradient direction filtered image With respect to the texture direction reference The angle difference between Calculate the filter gain coefficient for each pixel. ; Based on the filter gain coefficient The gradient direction filtered image is weighted and enhanced to generate a textured image with direction-selective enhancement; Calculate the local pixel grayscale variance based on the textured image. A reflective area is obtained; and the texture direction is referenced. Under the constraints, directional texture reconstruction and illumination compensation are performed on the reflective areas to output an enhanced image that suppresses reflective interference and has continuous texture features.

5. The multi-stage image enhancement and adaptation method according to claim 4 MPC The integrated unmanned vessel control method is characterized by, The directional texture reconstruction and illumination compensation employs a dual-domain decomposition and repair method based on joint modeling of the structural domain and the texture domain. The repair method includes: The reflective area and its corresponding neighborhood image are decomposed to obtain the structural layer component representing the illumination and geometric changes of the tunnel inner wall and the texture layer component representing the surface detail features. For the structural layer components, based on the geometric continuity of the tunnel wall within the neighborhood of the reflective region, a least-squares fitting method is used to model the neighborhood illumination distribution using a polynomial surface. This polynomial surface is then used to predict the structural layer pixel values ​​within the reflective region. ; For the texture layer components, based on the texture direction reference An anisotropic structural tensor field reflecting the dominant direction of texture is constructed, and a texture diffusion model is established under the constraint of the structural tensor field. Solving the texture diffusion model allows neighborhood texture features to propagate along the principal feature direction of the structure tensor field into the reflective region, thereby obtaining texture layer restoration values. ; Repair the texture layer value With the pixel values ​​of the structural layer The process involves fusion to complete the directional texture reconstruction and lighting compensation of the reflective areas.

6. The multi-stage image enhancement and adaptation method according to claim 1 MPC The integrated unmanned vessel control method is characterized by, The texture abnormal area The formation process includes: Reference along the texture direction The enhanced image is divided into multiple projection units arranged along the texture direction reference. And establish a spatial mapping index between each projection unit and the pixel coordinates in the enhanced image. ; Calculate the variation in texture continuity length between adjacent projection units. To obtain the distribution of texture continuity variation ; Based on the distribution of texture continuity changes Filter out the texture continuity length variation value Meets the preset texture continuity length variation threshold projection unit ; Using the spatial mapping index Extract the projection unit The corresponding pixel coordinate sets are merged to form texture anomaly regions. .

7. The multi-stage image enhancement and adaptation method according to claim 6 MPC The integrated unmanned vessel control method is characterized by, The obtained texture continuity variation distribution The acquisition process includes: Statistics satisfy the preset grayscale similarity conditions The length of the continuous pixel segment is used to obtain the projection unit. Corresponding texture duration ; Texture continuity length of adjacent projection units Perform differential calculations to obtain the variation value of texture continuity length. ; Changes in the length of each texture Arranged sequentially along the projection unit numbers, forming a texture continuity and variation distribution. .

8. The multi-stage image enhancement and adaptation method according to claim 1 MPC The integrated unmanned vessel control method is characterized by, The method for determining the inspection target area includes: Statistical analysis of the texture anomaly regions The set of pixel coordinates is the extreme values ​​along the horizontal and vertical axes in the image coordinate system where the enhanced image is located, and the minimum bounding rectangle region is constructed. ; Acquire preset imaging parameters and real-time attitude information of the unmanned vessel, and establish a spatial mapping relationship between the image coordinate system and the navigation coordinate system in which the unmanned vessel is currently located; Using the aforementioned spatial mapping relationship, the minimum bounding rectangle region is calculated. The physical location coordinates of the geometric center in the navigation coordinate system and the minimum bounding rectangle region and its physical location coordinates The area is identified as the inspection target area.

9. The multi-stage image enhancement and adaptation method according to claim 8 MPC The integrated unmanned vessel control method is characterized by, The current navigation status of the unmanned vessel includes: current navigation speed, heading angle, and position; The predicted navigation trajectory of the unmanned vessel employs a model predictive control method, including: Based on the current speed, heading angle, and position of the unmanned vessel in the navigation coordinate system, determine the state evolution characteristics of the unmanned vessel in the prediction time domain; The physical location coordinates of the inspection target area By introducing the aforementioned state evolution characteristics, a trajectory optimization objective function that satisfies preset constraints is constructed. ; The trajectory optimization objective function is solved in the prediction time domain to obtain the optimal control input sequence of the unmanned surface vessel in the navigation coordinate system. and the corresponding navigation trajectory .

10. The multi-stage image enhancement and adaptation method according to claim 9 MPC The integrated unmanned vessel control method is characterized by, The heading angle of the unmanned vessel upon reaching the inspection target area is determined and output. ,include: Based on the rolling optimization principle of model predictive control, from the optimal control input sequence Extract the optimal control input vector at the current moment; The optimal control input vector is analyzed to obtain the optimal heading angular velocity, and the target heading angle for the unmanned surface vessel to reach the inspection target area is calculated and output in combination with the current heading angle. .

Images