River real scene modeling method, device and equipment based on real-time image sequence, storage medium and program product

Abstract

The application relates to a river real scene modeling method and device based on a real-time image sequence, a computer device, a readable storage medium and a program product, and relates to the technical field of artificial intelligence. The application can improve the modeling efficiency of river real scene modeling. The method comprises the following steps: performing directional deblurring processing on a river image sequence by using bank line linear parameters in structure priori, and performing partition illumination correction on the river image sequence by using water body distribution threshold values in environment priori, to obtain a target river image sequence after preprocessing; obtaining optimization pose information based on key facility geometric features and river section contour templates in structure priori, and obtaining a target reconstruction area based on illumination interference parameters and dynamic interference features in environment priori; obtaining an initial river real scene model according to the target river image sequence, the optimization pose information and the target reconstruction area, updating the initial river real scene model by using structure priori and time sequence priori, and obtaining a river real scene three-dimensional semantic model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, computer device, computer-readable storage medium, and computer program product for real-time image sequence-based river scene modeling. Background Technology

[0002] River channel real-world modeling refers to the operation of modeling the actual situation of a river channel using various technical means. It can not only provide effective support for scientific research and engineering design, but also help relevant departments to carry out scientific management and decision-making.

[0003] Traditional techniques primarily involve collecting point cloud data of targets such as riverbanks, bridge piers, and sluice gates using fixed base stations or mobile platforms (such as unmanned surface vessels equipped with lidar). This data is then combined with positioning information from GPS (Global Positioning System) to stitch the point cloud together, thereby generating a dense 3D geometric model. However, this approach generates a massive amount of data (point cloud data for a single kilometer of river can reach tens of gigabytes under high-precision scanning), requiring significant time to process and resulting in low modeling efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for real-time image sequence-based river scene modeling to address the aforementioned technical problems.

[0005] Firstly, this application provides a method for real-time image sequence-based river channel scene modeling, including:

[0006] A sequence of river images from a motion perspective is acquired, and the sequence of river images is input into a pre-constructed river prior knowledge base; the river prior knowledge base includes structural priors, environmental priors, semantic-geometric association priors, and temporal priors.

[0007] The river image sequence is subjected to directional deblurring using the shoreline linear parameters in the structural prior, and the river image sequence is subjected to partitioned illumination correction using the water distribution threshold in the environmental prior, to obtain the preprocessed target river image sequence.

[0008] Based on the geometric features of key facilities and the river cross-section contour template in the prior structure, the motion pose of the prior constrained motion is optimized in real time to obtain optimized pose information. Based on the illumination interference parameters and dynamic interference features in the prior environment, prior-guided regional dense reconstruction is performed to obtain the target reconstruction area.

[0009] Based on the target river image sequence, the optimized pose information, and the target reconstruction region, an initial river scene model is obtained by fusion modeling through the semantic-geometric association prior. The initial river scene model is then updated using the structural prior and the temporal prior to obtain a three-dimensional semantic model of the river scene.

[0010] In one embodiment, the real-time optimization of motion pose based on the geometric features of key facilities and the river cross-section contour template in the prior structure includes:

[0011] Feature extraction and matching are performed on the geometric features of the key facilities to obtain matching results of similar features. Scale correction of geographical prior constraints is performed on the river channel cross-section contour template. If a target of known size is detected, the actual distance is inferred from the target's actual size, image pixel size, and camera intrinsic parameters to correct the depth value. If no target of known size is detected, the scale factor is calculated using the prior river segment width and the pixel distance between the left and right banks in the image to obtain the corrected pose. Based on the matching results of similar features and the actual distance-corrected depth value or the corrected pose, the motion pose of the prior constraints is optimized in real time.

[0012] In one embodiment, the prior-guided regional dense reconstruction based on the illumination interference parameters and dynamic interference characteristics in the prior environment includes:

[0013] Based on the illumination interference parameters and the dynamic interference characteristics, the real-world river scene is divided into regions, resulting in a static core area, a dynamic interference area, and a special water body area. High-precision reconstruction is performed on the static core area, lightweight reconstruction is performed on the dynamic interference area, and prior fitting reconstruction is performed on the special water body area.

[0014] In one embodiment, the step of performing directional deblurring on the river image sequence using the shoreline linearity parameter in the structural prior, and performing zonal illumination correction on the river image sequence using the water distribution threshold in the environmental prior, includes:

[0015] Based on the linear parameters of the shoreline, effective shorelines in the prior knowledge base of the river channel are screened by Hough transform line detection. The main blur direction is determined according to the pixel gradient direction of the effective shoreline. The blur kernel is generated adaptively by combining the inter-frame shoreline optical flow to perform directional restoration of the blurred images in the river channel image sequence. Based on the water body distribution threshold, the water body area and non-water body area are divided by the prior water body coordinate range. The Retinex algorithm is used to suppress reflection in the water body area. Histogram equalization is performed on the non-water body area where the grayscale value of the dark edge of the backlit shoreline is lower than the prior backlit shoreline dark edge grayscale threshold.

[0016] In one embodiment, the step of obtaining an initial river scene model by fusing and modeling based on the target river image sequence, the optimized pose information, and the target reconstructed region through the semantic-geometric association prior includes:

[0017] The lightweight semantic segmentation model is trained using the optimized pose information and the target reconstruction region, and the semantic-geometric association prior is incorporated during training to generate multiple pixel-level semantic labels; the geometric features of the 3D point cloud are extracted from the target river image sequence, and the initial river scene model is obtained by integrating the pixel-level semantic labels and the geometric features.

[0018] In one embodiment, the method further includes:

[0019] Acquire river GIS data, historical modeling data, and manually labeled samples. Based on the river GIS data, historical modeling data, and manually labeled samples, construct a river prior knowledge base that includes structural priors, environmental priors, semantic-geometric association priors, and temporal priors. Dynamically update the river prior knowledge base based on feedback data.

[0020] Secondly, this application also provides a device for real-time image sequence-based river channel scene modeling, comprising:

[0021] The image acquisition module is used to acquire a sequence of river images from a motion perspective and input the river image sequence into a pre-constructed river prior knowledge base; the river prior knowledge base includes structural priors, environmental priors, semantic-geometric association priors, and temporal priors.

[0022] The image processing module is used to perform directional deblurring on the river image sequence using the shoreline linear parameters in the structural prior, and to perform zonal illumination correction on the river image sequence using the water distribution threshold in the environmental prior, so as to obtain the preprocessed target river image sequence.

[0023] The optimization and reconstruction module is used to perform real-time optimization of motion pose based on the geometric features of key facilities and the river cross-section contour template in the prior structure, to obtain optimized pose information, and to perform prior-guided regional dense reconstruction based on the illumination interference parameters and dynamic interference features in the prior environment, to obtain the target reconstruction area.

[0024] The fusion modeling module is used to perform fusion modeling based on the target river image sequence, the optimized pose information, and the target reconstruction region through the semantic-geometric association prior to obtain an initial river scene model. The initial river scene model is then updated using the structural prior and the temporal prior to obtain a three-dimensional semantic model of the river scene.

[0025] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0026] A sequence of river images from a motion perspective is acquired and input into a pre-constructed river prior knowledge base. This prior knowledge base includes structural priors, environmental priors, semantic-geometric association priors, and temporal priors. The river image sequence is then subjected to directional deblurring using the shoreline linear parameters from the structural priors, and partitioned illumination correction is performed using the water distribution thresholds from the environmental priors, resulting in a pre-processed target river image sequence. Based on the geometric features of key facilities and the river cross-section contour template from the structural priors, real-time motion pose optimization with prior constraints is performed to obtain optimized pose information. Furthermore, prior-guided regional dense reconstruction is performed based on illumination interference parameters and dynamic interference features from the environmental priors to obtain the target reconstruction region. Finally, based on the target river image sequence, the optimized pose information, and the target reconstruction region, a fusion model is performed using the semantic-geometric association priors to obtain an initial river scene model. This initial river scene model is then updated using the structural and temporal priors to obtain a three-dimensional semantic model of the river scene.

[0027] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0028] A sequence of river images from a motion perspective is acquired and input into a pre-constructed river prior knowledge base. This prior knowledge base includes structural priors, environmental priors, semantic-geometric association priors, and temporal priors. The river image sequence is then subjected to directional deblurring using the shoreline linear parameters from the structural priors, and partitioned illumination correction is performed using the water distribution thresholds from the environmental priors, resulting in a pre-processed target river image sequence. Based on the geometric features of key facilities and the river cross-section contour template from the structural priors, real-time motion pose optimization with prior constraints is performed to obtain optimized pose information. Furthermore, prior-guided regional dense reconstruction is performed based on illumination interference parameters and dynamic interference features from the environmental priors to obtain the target reconstruction region. Finally, based on the target river image sequence, the optimized pose information, and the target reconstruction region, a fusion model is performed using the semantic-geometric association priors to obtain an initial river scene model. This initial river scene model is then updated using the structural and temporal priors to obtain a three-dimensional semantic model of the river scene.

[0029] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0030] A sequence of river images from a motion perspective is acquired and input into a pre-constructed river prior knowledge base. This prior knowledge base includes structural priors, environmental priors, semantic-geometric association priors, and temporal priors. The river image sequence is then subjected to directional deblurring using the shoreline linear parameters from the structural priors, and partitioned illumination correction is performed using the water distribution thresholds from the environmental priors, resulting in a pre-processed target river image sequence. Based on the geometric features of key facilities and the river cross-section contour template from the structural priors, real-time motion pose optimization with prior constraints is performed to obtain optimized pose information. Furthermore, prior-guided regional dense reconstruction is performed based on illumination interference parameters and dynamic interference features from the environmental priors to obtain the target reconstruction region. Finally, based on the target river image sequence, the optimized pose information, and the target reconstruction region, a fusion model is performed using the semantic-geometric association priors to obtain an initial river scene model. This initial river scene model is then updated using the structural and temporal priors to obtain a three-dimensional semantic model of the river scene.

[0031] The aforementioned method, apparatus, computer equipment, computer-readable storage medium, and computer program product for river scene modeling based on real-time image sequences first acquire a sequence of river images from a moving perspective and input it into a pre-constructed river prior knowledge base. Within this prior knowledge base, structural priors are used to optimize real-time pose estimation and dense reconstruction efficiency from a moving perspective; environmental priors are used to improve modeling robustness under complex lighting and dynamic disturbances; semantic-geometric association priors are used to achieve deep fusion of semantics and geometry; and temporal priors are used to achieve incremental 3D semantic updates from a moving perspective. This application addresses the high computational resource consumption of existing technologies such as traditional 3D river reconstruction techniques (e.g., LiDAR point cloud stitching and global SfM dense reconstruction) and real-time modeling techniques from a moving perspective (e.g., lightweight SLAM). By combining 3D geometric constraints of the river, enhancing reconstruction accuracy and computational efficiency through semantic categories, and solving scale drift and slow real-time updates through motion adaptation optimization using structural and temporal priors, the modeling efficiency of river scene modeling is effectively improved. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is an application environment diagram of a river scene modeling method based on real-time image sequences in one embodiment;

[0034] Figure 2 This is a flowchart illustrating a method for real-time image sequence-based river scene modeling in one embodiment.

[0035] Figure 3 This is a flowchart illustrating the real-time motion pose optimization steps in one embodiment;

[0036] Figure 4 This is a flowchart illustrating a method for real-time image sequence-based river scene modeling in a specific embodiment.

[0037] Figure 5 This is a flowchart illustrating a method for real-time image sequence-based river scene modeling in an application embodiment.

[0038] Figure 6 This is a structural block diagram of a river scene modeling device based on real-time image sequences in one embodiment;

[0039] Figure 7This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0041] Traditional technologies for 3D semantic modeling of river scenes have not fully utilized "river scene-specific prior features," resulting in significant shortcomings in four core requirements: "adaptability to motion perspective, robustness to complex environments, semantic-geometric fusion accuracy, and real-time performance." These shortcomings are detailed below:

[0042] (1) Failure to utilize prior knowledge of river structure leads to an imbalance between modeling accuracy and efficiency.

[0043] The river scenario has strong structural priors such as "linear shoreline distribution, regular geometric structure of bridge piers / gates, and fixed river width," but current technologies have not incorporated these into the modeling process.

[0044] Traditional SfM (Structure from Motion) and SLAM (Simultaneous Localization and Mapping) methods employ "global feature matching + global pose optimization," failing to utilize "linear shoreline features" to simplify pose estimation. This results in redundant feature matching between frames and decreased real-time performance in meandering river sections (such as serpentine channels).

[0045] When using lidar and oblique photogrammetry for reconstruction, prior knowledge such as "bridge pier column structure and gate plate structure" was not utilized. All areas were modeled with the same precision, resulting in a large amount of data and low efficiency in subsequent semantic overlay.

[0046] (2) Without utilizing prior knowledge of the river environment, it has poor robustness in complex scenarios.

[0047] Environmental priors such as the "lighting dynamics (backlighting, reflection) and dynamic disturbances (waves, floating objects)" of the river channel have not been effectively utilized by existing technologies, resulting in modeling and semantic segmentation being significantly affected by the environment.

[0048] The semantic segmentation model did not utilize prior lighting knowledge such as "the shoreline appears dark when the river is backlit and the reflective areas on the water surface are locally overexposed". In backlit scenes, the misjudgment rate of the "bank-sky" boundary was significantly increased, and the error rate of "floating garbage being misjudged as water" due to water surface reflection exceeded 25%.

[0049] The real-time reconstruction method does not take advantage of the prior knowledge that "the dynamic disturbances in the river channel are mostly small-scale floating objects". It models all dynamic targets (such as floating objects and moving vessels), which leads to the interference of reconstruction accuracy of static areas (such as riverbanks and bridge piers).

[0050] (3) The semantic-geometric association prior of the river channel was not utilized, resulting in a separation between semantics and geometry.

[0051] The environmental priors of the river channel, such as "lighting dynamics (backlighting, reflection) and dynamic disturbances (waves, floating objects)," have not been effectively utilized by existing technologies, resulting in modeling and semantic segmentation being significantly affected by the environment.

[0052] Semantic segmentation is based solely on two-dimensional image features and does not incorporate three-dimensional geometric constraints (such as judging "hard slope - soft slope" by slope), leading to the error of "misjudging steep slope vegetation as soft slope".

[0053] 3D reconstruction only outputs geometric information and does not utilize semantic categories to optimize reconstruction strategies (e.g., water areas do not require detailed reconstruction, while bridge piers require high-precision reconstruction), resulting in wasted modeling resources and poor spatial consistency between semantic labels and 3D geometry.

[0054] (4) Insufficient adaptability to motion perspective, making it difficult to balance real-time performance and accuracy.

[0055] Existing technologies are mostly designed based on general scenarios and do not utilize the prior knowledge that "the trajectory of the river movement platform extends along the shoreline," resulting in bottlenecks in real-time modeling from a motion perspective.

[0056] Lightweight SLAM does not utilize the prior knowledge of "the river platform moving along the bank in a straight line / gentle curve". Pose prediction relies on inter-frame feature matching, and the pose delay cannot meet the real-time modeling requirements when moving rapidly.

[0057] Real-time reconstruction and semantic segmentation are separate processes. The prior knowledge of "continuous segmentation of river modeling area according to shoreline" is not utilized. Global synchronous processing is adopted, which means that the global model needs to be recalculated when local areas (such as newly added shoreline segments) are updated, resulting in poor real-time performance.

[0058] To address this, this application takes "prior features of the river environment" as its core driver, constructing a technical framework of "prior knowledge base - full-process prior application - real-time 3D semantic output." Through "prior constraint accuracy and prior reduction overhead," it achieves coordinated optimization of "real-time performance, accuracy, and robustness" in 3D semantic modeling of river scenes from a motion perspective. The overall technical solution follows a pipeline logic of "input preprocessing → pose optimization → dense reconstruction → semantic fusion → model update," with each module deeply embedded with river-specific priors. It should be noted that the application scenarios of this application include, but are not limited to, maritime regulatory departments, river management departments' inspection and monitoring of rivers and estuaries, and environmental identification and positioning for emergency rescue in rivers and estuaries.

[0059] The river channel real-scene modeling method based on real-time image sequences provided in this application can be applied to, for example... Figure 1 The application environment shown illustrates this. In this environment, the terminal can communicate with the server via a network. The data storage system can store the data that the server needs to process. The data storage system can be integrated onto the server or located on the cloud or other network servers. In situations such as... Figure 1 In the application environment shown, the terminal can be, but is not limited to, various personal computers, laptops, smartphones, and tablets. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0060] In one embodiment, such as Figure 2 As shown, a method for river channel real-scene modeling based on real-time image sequences is provided. This method can be applied to... Figure 1 In a terminal, the method may include the following steps:

[0061] Step S201: Obtain the river channel image sequence from the motion perspective, and input the river channel image sequence into the pre-constructed river channel prior knowledge base; the river channel prior knowledge base includes structural priors, environmental priors, semantic-geometric association priors, and temporal priors.

[0062] Among them, the structural prior is used to store the linear parameters of the shoreline, the geometric features of key facilities, and the profile template of the river channel cross section; the environmental prior is used to store the water distribution threshold, the illumination interference parameters, and the dynamic interference features; the semantic-geometric association prior is used to store the "semantic category-geometric feature" mapping rules (such as "hard bank slope" corresponding to a slope > 70°, "water body" corresponding to a surface normal vector variance < 0.1, and "bridge pier" corresponding to a columnar structure radius range); and the temporal prior is used to store the temporal stability parameters of the river scene (such as the minimum number of frames in which the semantic category of a certain area remains unchanged, and the temporal offset threshold of the shoreline position).

[0063] Specifically, in response to the received river real-scene modeling command, the terminal acquires a sequence of river images from a motion perspective, and then inputs the river image sequence into a pre-built river prior knowledge base for subsequent processing.

[0064] Step S202: The river image sequence is deblurred using the shoreline linear parameters in the structural prior, and the river image sequence is partitioned for illumination correction using the water distribution threshold in the environmental prior, to obtain the preprocessed target river image sequence.

[0065] Among them, the shoreline linearity parameter may include the slope range of a certain river section and the smoothness threshold, etc., and the water body distribution threshold may include the y coordinate range of the water body region in the image, etc.

[0066] Specifically, the terminal utilizes structural and environmental priors to perform targeted optimization on three major problems in river channel images from a motion perspective: "jitter blur, uneven illumination, and reflection interference," and outputs a pre-processed high-quality image sequence for subsequent modeling.

[0067] Step S203: Based on the geometric features of key facilities and the profile template of the river channel in the prior structural information, the motion pose of the prior constrained motion is optimized in real time to obtain the optimized pose information. Based on the illumination interference parameters and dynamic interference features in the prior environmental information, the region-specific dense reconstruction is performed under prior guidance to obtain the target reconstruction region.

[0068] Among them, the geometric features of key facilities may include the diameter / height range of bridge piers and the length-to-width ratio of gates, etc.; the profile template of river channel cross section may include trapezoidal / rectangular cross section parameters, etc.; the illumination interference parameters may include the gray scale threshold of the dark edge of the shoreline under backlight and the brightness threshold of the reflective area of ​​the water surface, etc.; the dynamic interference features may include the floating object area threshold and the range of the Aspect Ratio of the vessel outline, etc.

[0069] Specifically, the terminal utilizes structural priors to optimize camera pose estimation from a motion perspective, ensuring real-time performance while avoiding scale drift, thus providing a precise pose foundation for 3D reconstruction. It also uses environmental priors to divide the river scene into "static core area, dynamic interference area, and special water body area," and adopts a differentiated reconstruction strategy to balance accuracy and efficiency.

[0070] Step S204: Based on the target river image sequence, optimized pose information, and target reconstruction region, a fusion model is performed using semantic-geometric association priors to obtain an initial river scene model. The initial river scene model is then updated using structural and temporal priors to obtain a three-dimensional semantic model of the river scene.

[0071] Specifically, the terminal utilizes the semantic-geometric association prior of the river channel to achieve bidirectional optimization of "geometric constraint on semantics and semantic guidance of geometry", generating an integrated modeling result of "three-dimensional geometry + semantic attributes" to obtain an initial river channel real scene model. Then, the initial river channel real scene model is updated using structural priors and temporal priors to obtain a three-dimensional semantic model of the river channel real scene.

[0072] In this embodiment, a sequence of river images from a moving perspective is first acquired and input into a pre-constructed river prior knowledge base. Within this knowledge base, structural priors are used to optimize real-time pose estimation and dense reconstruction efficiency from a moving perspective; environmental priors are used to improve modeling robustness under complex lighting and dynamic disturbances; semantic-geometric association priors are used to achieve deep fusion of semantics and geometry; and temporal priors are used to achieve incremental 3D semantic updates from a moving perspective. This application addresses the high computational resource consumption of existing technologies such as traditional 3D river reconstruction techniques (e.g., LiDAR point cloud stitching and global SfM dense reconstruction) and real-time modeling techniques from a moving perspective (e.g., lightweight SLAM). By combining 3D geometric constraints of the river, semantic categories are used to enhance reconstruction accuracy and computational efficiency. Furthermore, motion adaptation optimization using structural and temporal priors solves the problems of scale drift and slow real-time updates, thereby effectively improving the real-time performance and modeling efficiency of river scene modeling and providing core technical support for smart river construction.

[0073] In one embodiment, such as Figure 3 As shown, in step S203 above, the real-time optimization of motion pose based on the geometric features of key facilities and the river cross-section contour template in the prior structural data may include the following steps:

[0074] Step S301: Perform feature extraction and matching on the geometric features of key facilities to obtain matching results for similar features.

[0075] Step S302: Scale correction of the river channel cross-section contour template based on geographical prior constraints is performed. If a target of known size is detected, the depth value is corrected by back-calculating the actual distance using the actual target size, image pixel size, and camera intrinsic parameters. If the target of known size is not detected, the scale factor is calculated using the prior river section width and the pixel distance between the left and right banks in the image to obtain the corrected pose.

[0076] Step S303: Based on the matching results of similar features and the actual distance, the depth value is corrected or the pose is corrected, and the motion pose of the prior constraints is optimized in real time.

[0077] Specifically, this embodiment utilizes river channel geographical priors and structural priors to optimize camera pose estimation from a motion perspective, ensuring real-time performance while avoiding scale drift, thus providing a precise pose foundation for 3D reconstruction.

[0078] ① Prior simplified feature extraction and matching:

[0079] Prior application: Utilizing the prior that "the structure of key facilities (piers / gates) is stable", only feature points of "shoreline + key facilities" in the image are extracted (discarding the massive features of general scenarios). The feature extraction algorithm adopts the lightweight MobileViT, which only retains key points that match the geometric features of facilities in the prior library (such as the circular features of the pier area and the rectangular features of the gate area).

[0080] Technical implementation: Invalid features (such as suspected facility feature points with an area of ​​less than 0.01㎡) are filtered out by the "facility size range" in the prior library. When matching features, only features of the same type are calculated (such as shoreline features → shoreline features, pier features → pier features).

[0081] ② Scale correction of geographical prior constraints:

[0082] Prior application: Utilizing geographical priors such as "known river width / facility height" (e.g., the width of a certain river segment stored in a priori database). Pier height ), to apply scale constraints to the pose estimated by SLAM.

[0083] Technical Implementation: When a target of known size (such as a bridge pier) is detected in an image, the pixel height of the target in the image is calculated. Combined with camera internal parameters (Focal length), inversely estimating the actual distance Correct the depth value output by SLAM;

[0084] When no clear target is detected, the width of the river segment in the prior reservoir is used. The pixel distance between the left and right shorelines in the image Calculate the scaling factor To correct for scale drift in inter-frame pose, the formula is:

[0085]

[0086] In the above formula, This is the original pose of SLAM. For the corrected pose.

[0087] ③ Sliding window lightweight BA (Bundle Adjustment) optimization:

[0088] Prior application: Utilizing the prior trajectory of "the river channel movement platform moving gently along the shoreline", the sliding window size is dynamically adjusted to 5-10 frames (general SLAM is 15-20 frames), and only the pose parameters related to the shoreline / facilities within the window are optimized;

[0089] Technical Implementation: The pose of the next frame is predicted by combining Kalman filtering with the "temporal change threshold of shoreline slope" in the prior library, thereby reducing the number of iterations for BA optimization.

[0090] In one embodiment, step S203 above, which involves prior-guided regional dense reconstruction based on prior lighting interference parameters and dynamic interference characteristics in the environment, may include the following steps:

[0091] Based on the illumination interference parameters and dynamic interference characteristics, the real river scene is divided into regions, resulting in a static core area, a dynamic interference area, and a special water body area. High-precision reconstruction is performed for the static core area, lightweight reconstruction is performed for the dynamic interference area, and prior fitting reconstruction is performed for the special water body area.

[0092] Specifically, in this embodiment, by utilizing river dynamic priors and structural priors, the river scene is divided into "static core area, dynamic interference area, and special water body area," and a differentiated reconstruction strategy is adopted to balance accuracy and efficiency.

[0093] For the static core area, which includes structurally stable areas such as bank slopes, piers and gates, the location is determined by the "facility location range" and "static area grayscale variance threshold (<50)" in the prior library.

[0094] For dynamic interference zones, including floating objects and moving vessels, the location is determined by the "dynamic interference area threshold (<0.05㎡)" and "aspect ratio range (0.5-2.0)" in the prior library.

[0095] For special water bodies: the depth estimation error in this area is large due to reflection caused by the "water body distribution range" in the prior reservoir, and special processing is required.

[0096] Differentiated reconstruction strategies are adopted for different partitions:

[0097] ① Static core area (high-precision reconstruction):

[0098] Using the prior of "facility regular geometric structure", an improved Lite-Mono depth estimation model (incorporating "facility geometric features" from the prior library, such as circular constraints of bridge piers) is used to generate a depth map. Then, dense point clouds are generated through inter-frame depth fusion to preserve structural details.

[0099] For the bridge pier area, the depth map is forced to meet the "circular outline" constraint (non-circular depth points are filtered out by Hough circle detection) to ensure geometric accuracy.

[0100] ②Dynamic interference zone (lightweight reconstruction):

[0101] Leveraging the prior knowledge that "dynamic targets are not the core of modeling," target contours are extracted solely through instance segmentation masks to generate simplified point clouds (retaining <100 contour points per target), thus avoiding interference with static regions.

[0102] Minor disturbances of less than 0.01㎡ are filtered out by the "floating object area threshold" in the prior library and are not reconstructed.

[0103] ③ Special water body areas (prior fitting reconstruction):

[0104] Using the prior that "the water body is distributed in a plane" ("the variance of the water body surface normal vector is <0.1" in the prior library), the water body plane is fitted by the three-dimensional coordinates of the left and right shorelines to replace pixel-by-pixel depth estimation.

[0105] The plane equation is fitted using the least squares method:

[0106]

[0107] Input the fitting results of the "water body plane slope threshold (<5°)" constraint from the prior library to ensure plane consistency.

[0108] In one embodiment, step S202 above, which involves using the shoreline linear parameters from the structural prior to perform directional deblurring on the river image sequence and using the water distribution threshold from the environmental prior to perform zonal illumination correction on the river image sequence, may include the following steps:

[0109] Based on the linear parameters of the shoreline, effective shorelines in the prior knowledge base of the river channel are screened by Hough transform line detection. The main direction of blurring is determined according to the pixel gradient direction of the effective shoreline. Combined with the inter-frame shoreline optical flow, a blur kernel is adaptively generated to restore the blurred image in the river channel image sequence. Based on the water body distribution threshold, the water body area and non-water body area are divided by the prior water body coordinate range. The Retinex algorithm is used to suppress reflection in the water body area, and histogram equalization is performed on the area in the non-water body area that is lower than the gray scale threshold of the dark edge of the backlit shoreline.

[0110] Specifically, in this embodiment, by utilizing prior knowledge of river channel structure and environment, targeted optimization is performed on three major problems of river channel images from a motion perspective: "jitter blur, uneven illumination, and reflection interference," outputting high-quality images for subsequent modeling.

[0111] ① Application of directional defuzzification prior based on shoreline structure priors:

[0112] Using the structural prior of "the shoreline is linearly distributed", the shoreline region in the image is located (by detecting straight lines through Hough transform, matching the shoreline slope range in the prior library, and filtering valid shorelines).

[0113] Technical implementation: Extract the pixel gradient direction of the shoreline region (the direction of the shoreline extension is the main blur direction), and adaptively calculate the directional blur kernel (generate kernel function only along the main blur direction to avoid the indiscriminate calculation of general deblurring);

[0114] By combining inter-frame optical flow (calculating only the optical flow in the shoreline region to reduce computational cost), directional restoration of blurred images is performed, and the calculation formula is as follows:

[0115]

[0116] In the above formula, The original blurred image, For inter-frame optical flow in the shoreline region. , which is the optical flow weight (value 0.3-0.5, determined by the shoreline blur threshold in the prior library).

[0117] ② Zonal illumination correction based on prior knowledge of the aquatic environment:

[0118] Prior application: Utilizing the prior knowledge that "water bodies are mostly concentrated in the lower half of the image," the system divides the image into "water body regions" and "non-water body regions" (based on the y-coordinate range of water bodies in the prior database). position);

[0119] Water body area technology implementation: To address reflective interference, the Retinex algorithm is used to suppress reflective light in a targeted manner, with the "water surface reflective brightness threshold" input from the prior library. The pixels with excessive brightness are subjected to logarithmic domain compression, calculated as follows:

[0120]

[0121] In the above formula, This is the reflection suppression coefficient, with a value ranging from 0.6 to 0.8;

[0122] Technical implementation for non-water areas (shoreline / facilities): To address backlighting interference, the "grayscale threshold of dark edge of backlit shoreline" is input from the prior library, and histogram equalization is performed on shoreline areas with grayscale values ​​below the threshold to enhance details.

[0123] In one embodiment, step S204 above, based on the target river channel image sequence, optimized pose information, and target reconstruction region, performs fusion modeling through semantic-geometric association priors to obtain an initial river channel reality model, which may include the following steps:

[0124] A lightweight semantic segmentation model is trained using optimized pose information and target reconstruction region, and semantic-geometric association priors are incorporated during training to generate multiple pixel-level semantic labels. Geometric features of 3D point clouds are extracted from the target river image sequence, and an integrated modeling is performed based on pixel-level semantic labels and geometric features to obtain an initial river scene model.

[0125] This embodiment is the core innovation point. By utilizing the semantic-geometric association prior of the river channel, it achieves bidirectional optimization of "geometric constraint semantics and semantic-guided geometry", generating an integrated modeling result of "3D geometry + semantic attributes". Specifically, it includes the following steps:

[0126] ① Prior-driven river-specific semantic segmentation:

[0127] Prior application: Perform prior verification on the segmentation results, such as marking areas with "slope < 30° and marked as hard bank slope" as suspected errors, pending geometric constraint correction.

[0128] Technical Implementation: Construct a lightweight semantic segmentation model (based on an improvement of SegFormer-Lite), incorporate "semantic-geometric association rules" from the prior library during training (such as using "slope > 70°" as an additional supervision signal for "hard bank"), and output pixel-level semantic labels (which can contain 8 core semantic categories: water body / hard bank / soft bank / bridge pier / gate / floating garbage / vegetation / boat).

[0129] ② Two-way constraint optimization process:

[0130] Step 1: Geometric prior correction of semantics (solving "semantic segmentation errors").

[0131] Technical implementation: Extract the geometric features (slope, surface normal vector, structural morphology) of the 3D point cloud, match them with the "semantic-geometric association rules" in the prior library, and correct erroneous semantic labels.

[0132] Example: If a region is semantically labeled as "vegetation", but its slope is calculated to be greater than 70° and its structure is linear (matching the "hard bank slope" rule in the prior library), then the semantics of the region should be corrected to "hard bank slope".

[0133] Step 2: Semantic prior optimization of geometry (to solve the problem of "coarse geometric reconstruction").

[0134] Technical implementation: Based on the semantic tags, the "semantic-specific reconstruction rules" in the prior library are called to optimize the geometric models of different semantic regions.

[0135] Example: For the semantic region of "bridge pier", the "columnar structure smoothing rule" in the prior library is called to perform Gaussian smoothing (σ=0.1) on its point cloud, preserving columnar details; for the semantic region of "water body", the "planar consistency rule" is called to remove outliers that deviate from the fitted plane by more than 0.1m.

[0136] Then, this embodiment further utilizes the river channel temporal prior and structural prior to achieve "simultaneous data acquisition, modeling, and updating" from a motion perspective, avoiding global recalculation and ensuring real-time performance and model consistency.

[0137] ① Prior-guided rasterized partitioning:

[0138] Prior application: Utilizing the structural prior of "linear shoreline distribution", the river channel is divided into 10m×10m rectangular grids along the shoreline direction (the long side of the grid is parallel to the shoreline), and each grid is used as an independent update unit.

[0139] Technical implementation: The grid orientation is determined by the "shoreline slope" in the prior library to ensure that the grid matches the river morphology and reduce redundant calculations across grids.

[0140] ②Incremental update and timing verification:

[0141] Prior application: Call the "semantic temporal stability parameter" in the prior library (e.g., if the semantics of a certain region remain unchanged for 5 consecutive frames, it is considered stable) to verify the semantic label updated in a single frame. If the difference between the semantics of the previous 5 frames is greater than 30%, the current update is rejected and the average of the historical semantics is used.

[0142] Technical implementation: When the motion platform moves, only the currently covered grid is updated (no need for global reconstruction modeling). After the update, the semantic consistency within the grid is verified through temporal prior.

[0143] In one embodiment, the method of this application further includes the following steps:

[0144] Acquire river channel GIS (Geographic Information System) data, historical modeling data, and manually labeled samples. Based on the river channel GIS data, historical modeling data, and manually labeled samples, construct a river channel prior knowledge base that includes structural priors, environmental priors, semantic-geometric association priors, and temporal priors. Dynamically update the river channel prior knowledge base based on feedback data.

[0145] The feedback data from the river prior knowledge base can be historical output data obtained after processing by the river prior knowledge base.

[0146] Specifically, in response to the received knowledge base construction instruction, the terminal acquires river GIS data, historical modeling data, and manually labeled samples. Based on the river GIS data, historical modeling data, and manually labeled samples, it constructs a river prior knowledge base that includes structural priors, environmental priors, semantic-geometric association priors, and temporal priors. It acquires feedback data from the river prior knowledge base at a preset frequency and dynamically updates the river prior knowledge base based on the feedback data.

[0147] In one embodiment, such as Figure 4 As shown, a method for river scene modeling based on real-time image sequences is provided in a specific embodiment, which includes the following steps:

[0148] Step S401: Obtain the river channel image sequence from the motion perspective, and input the river channel image sequence into the pre-constructed river channel prior knowledge base; the river channel prior knowledge base includes structural priors, environmental priors, semantic-geometric association priors, and temporal priors.

[0149] Step S402: Based on the linear parameters of the shoreline, effective shorelines in the prior knowledge base of the river channel are screened by Hough transform line detection. The main blur direction is determined according to the pixel gradient direction of the effective shoreline. The blur kernel is generated adaptively by combining the inter-frame shoreline optical flow to perform directional restoration of the blurred images in the river channel image sequence. Based on the water body distribution threshold, the water body area and non-water body area are divided by the prior water body coordinate range. The Retinex algorithm is used to suppress reflection in the water body area. Histogram equalization is performed on the non-water body area where the grayscale value of the dark edge of the backlit shoreline is lower than the prior backlit shoreline. The preprocessed target river channel image sequence is obtained.

[0150] Step S403: Feature extraction and matching are performed on the geometric features of key facilities to obtain matching results of similar features; scale correction of geographical prior constraints is performed on the river channel cross-section contour template. If a target of known size is detected, the actual distance is back-calculated and the depth value is corrected by using the actual size of the target, the pixel size of the image, and the camera intrinsic parameters; if the target of known size is not detected, the scale factor is calculated using the prior river section width and the pixel distance between the left and right banks in the image to obtain the corrected pose; the motion pose of the prior constraints is optimized in real time based on the matching results of similar features and the actual distance correction depth value or the corrected pose to obtain the optimized pose information.

[0151] Step S404: Divide the river scene into regions based on illumination interference parameters and dynamic interference characteristics to obtain a static core area, a dynamic interference area, and a special water body area; perform high-precision reconstruction on the static core area, lightweight reconstruction on the dynamic interference area, and prior fitting reconstruction on the special water body area to obtain the target reconstruction area.

[0152] Step S405: The lightweight semantic segmentation model is trained using optimized pose information and target reconstruction region, and semantic-geometric association priors are incorporated during training to generate multiple pixel-level semantic labels; geometric features of 3D point cloud are extracted based on target river image sequence, and integrated modeling is performed based on pixel-level semantic labels and geometric features to obtain an initial river real scene model; the initial river real scene model is updated using structural priors and temporal priors to obtain a 3D semantic model of river real scene.

[0153] The beneficial effects of the above embodiments are as follows:

[0154] (1) Utilize prior knowledge of river channel structure to optimize real-time pose estimation and dense reconstruction efficiency from the perspective of motion:

[0155] By leveraging prior knowledge such as "linear shoreline features, fixed river width, and regular facility structure," the feature matching and optimization process for pose estimation is simplified, reducing unnecessary computations. At the same time, it achieves differentiated reconstruction that balances real-time performance and modeling accuracy by ensuring "high accuracy in key areas (bridge piers / gates) and high efficiency in non-key areas (open water).

[0156] (2) Utilize prior knowledge of the river environment to improve the robustness of modeling under complex lighting and dynamic disturbances:

[0157] By leveraging prior knowledge such as "backlit shoreline features, water surface reflection distribution, and dynamic interference scale," image preprocessing (e.g., directional deblurring and regional illumination correction) and dynamic target filtering are optimized to address feature loss issues under backlit, reflective, and rainy / foggy conditions. This improves semantic segmentation accuracy to over 90% and reduces the error in modeling static regions due to dynamic interference to within 5%.

[0158] (3) Utilizing the semantic-geometric association prior of river channels to achieve deep integration of semantics and geometry:

[0159] Establish association rules between semantic categories and geometric features (such as slope and structural morphology) to achieve bidirectional fusion of geometric constraint semantic segmentation and semantic-guided geometric optimization, solve the problem of misalignment between semantic and geometric boundaries (misalignment ≤ 3 pixels), and customize reconstruction strategies for different semantic categories to improve the utilization of modeling resources.

[0160] (4) Utilizing prior knowledge of river channel movement trajectories, incremental 3D semantic updates are achieved from a motion perspective:

[0161] Based on the prior of "segmented movement of the motion platform along the shoreline", the river modeling area is rasterized according to the shoreline to achieve "incremental update of local areas and dynamic splicing of global models", avoiding global recalculation and meeting the real-time requirements of "collecting, modeling and updating at the same time" from the motion perspective.

[0162] To more clearly illustrate the river channel real-scene modeling method based on real-time image sequences provided in this application, the following specific description uses an application embodiment to illustrate this method. In one embodiment, such as Figure 5 As shown, this application also provides a method for river channel real-scene modeling based on real-time image sequences, specifically including the following steps:

[0163] Step 1: Prior-driven image preprocessing;

[0164] Step 2: Real-time optimization of motion poses under prior constraints;

[0165] Step 3: Prior-guided dense reconstruction of sub-regions;

[0166] Step 4: Prior fusion of geometric-semantic bidirectional optimization;

[0167] Step 5: Incremental 3D semantic update with prior support.

[0168] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0169] Based on the same inventive concept, this application also provides a real-time image sequence-based river real-scene modeling device for implementing the above-mentioned method for real-time image sequence-based river real-scene modeling. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the real-time image sequence-based river real-scene modeling device provided below can be found in the limitations of the real-time image sequence-based river real-scene modeling method described above, and will not be repeated here.

[0170] In one exemplary embodiment, such as Figure 6 As shown, a device for real-time image sequence-based river scene modeling is provided, which may include:

[0171] The image acquisition module 601 is used to acquire a sequence of river images from a motion perspective and input the river image sequence into a pre-built river prior knowledge base; the river prior knowledge base includes structural priors, environmental priors, semantic-geometric association priors, and temporal priors;

[0172] The image processing module 602 is used to perform directional deblurring on the river image sequence using the shoreline linear parameters in the structural prior, and to perform partitioned illumination correction on the river image sequence using the water distribution threshold in the environmental prior, so as to obtain the preprocessed target river image sequence.

[0173] The optimization and reconstruction module 603 is used to perform real-time optimization of motion pose based on the geometric features of key facilities and the profile template of the river channel in the prior structural information, to obtain optimized pose information, and to perform prior-guided regional dense reconstruction based on the illumination interference parameters and dynamic interference features in the prior environmental information, to obtain the target reconstruction area.

[0174] The fusion modeling module 604 is used to perform fusion modeling based on the target river image sequence, optimized pose information and target reconstruction region through semantic-geometric association priors to obtain an initial river scene model. The initial river scene model is then updated using structural priors and temporal priors to obtain a three-dimensional semantic model of the river scene.

[0175] In one embodiment, the optimization and reconstruction module 603 is further configured to perform feature extraction and matching on the geometric features of key facilities to obtain matching results of similar features; perform scale correction of geographical prior constraints on the river channel profile template; if a target of known size is detected, the actual distance is inferred and the depth value is corrected by using the actual size of the target, the pixel size of the image, and the camera intrinsic parameters; if the target of known size is not detected, the scale factor is calculated using the prior river section width and the pixel distance between the left and right banks in the image to obtain the corrected pose; and the motion pose of the prior constraints is optimized in real time based on the matching results of similar features and the actual distance correction depth value or the corrected pose.

[0176] In one embodiment, the optimization and reconstruction module 603 is further used to divide the river scene into regions based on the illumination interference parameters and dynamic interference characteristics to obtain a static core area, a dynamic interference area, and a special water body area; to perform high-precision reconstruction for the static core area, lightweight reconstruction for the dynamic interference area, and prior fitting reconstruction for the special water body area.

[0177] In one embodiment, the image processing module 602 is further configured to filter effective shorelines in the prior knowledge base of the river channel based on the shoreline linear parameters through Hough transform line detection, determine the main blur direction according to the pixel gradient direction of the effective shoreline, and combine the inter-frame shoreline optical flow to adaptively generate a blur kernel to perform directional restoration of the blurred image in the river channel image sequence; and divide the water body region and non-water body region based on the water body distribution threshold through the prior water body coordinate range, use the Retinex algorithm to suppress reflection in the water body region, and perform histogram equalization on the non-water body region where the grayscale value of the dark edge of the backlit shoreline is lower than the prior backlit shoreline.

[0178] In one embodiment, the fusion modeling module 604 is further used to train a lightweight semantic segmentation model using optimized pose information and target reconstruction region, and incorporate semantic-geometric association priors during training to generate multiple pixel-level semantic labels; extract the geometric features of the three-dimensional point cloud based on the target river image sequence, and perform integrated modeling based on pixel-level semantic labels and geometric features to obtain an initial river scene model.

[0179] In one embodiment, the device may further include: a knowledge base construction module, used to acquire river GIS data, historical modeling data, and manually labeled samples; to construct a river prior knowledge base containing structural priors, environmental priors, semantic-geometric association priors, and temporal priors based on the river GIS data, historical modeling data, and manually labeled samples; and to dynamically update the river prior knowledge base according to feedback data from the river prior knowledge base.

[0180] The modules in the aforementioned real-time image sequence-based river scene modeling device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0181] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for real-time river scene modeling based on real-time image sequences. The display unit of the computer device is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0182] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0183] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0184] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0185] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0186] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0187] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0188] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0189] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for real-time river scene modeling based on real-time image sequences, characterized in that, The method includes: A sequence of river images from a motion perspective is acquired, and the sequence of river images is input into a pre-constructed river prior knowledge base; the river prior knowledge base includes structural priors, environmental priors, semantic-geometric association priors, and temporal priors. The river image sequence is subjected to directional deblurring using the shoreline linear parameters in the structural prior, and the river image sequence is subjected to partitioned illumination correction using the water distribution threshold in the environmental prior, to obtain the preprocessed target river image sequence. Based on the geometric features of key facilities and the river cross-section contour template in the prior structure, the motion pose of the prior constrained motion is optimized in real time to obtain optimized pose information. Based on the illumination interference parameters and dynamic interference features in the prior environment, prior-guided regional dense reconstruction is performed to obtain the target reconstruction area. Based on the target river image sequence, the optimized pose information, and the target reconstruction region, an initial river scene model is obtained by fusion modeling through the semantic-geometric association prior. The initial river scene model is then updated using the structural prior and the temporal prior to obtain a three-dimensional semantic model of the river scene.

2. The method according to claim 1, characterized in that, The real-time optimization of motion pose based on the geometric features of key facilities and the river cross-section contour template in the prior structure includes: Feature extraction and matching are performed on the geometric features of the key facilities to obtain matching results of similar features; Scale correction of the river channel profile template is performed based on geographical prior constraints. If a target of known size is detected, the depth value is corrected by back-calculating the actual distance using the actual target size, image pixel size, and camera intrinsic parameters. If the target of known size is not detected, the scale factor is calculated using the prior river segment width and the pixel distance between the left and right banks in the image to obtain the corrected pose. Based on the matching results of the same type of features and the actual distance correction depth value or the corrected pose, the motion pose of the prior constraint is optimized in real time.

3. The method according to claim 2, characterized in that, The prior-guided regional dense reconstruction based on the illumination interference parameters and dynamic interference characteristics in the environment prior includes: Based on the light interference parameters and the dynamic interference characteristics, the river landscape is divided into regions to obtain a static core area, a dynamic interference area, and a special water body area. High-precision reconstruction is performed for the static core area, lightweight reconstruction is performed for the dynamic interference area, and prior fitting reconstruction is performed for the special water body area.

4. The method according to claim 1, characterized in that, The process of performing directional deblurring on the river image sequence using the shoreline linear parameters in the structural prior, and performing zonal illumination correction on the river image sequence using the water distribution threshold in the environmental prior, includes: Based on the linear parameters of the shoreline, the effective shoreline in the prior knowledge base of the river channel is screened by Hough transform line detection. The main blur direction is determined according to the pixel gradient direction of the effective shoreline. The blur kernel is adaptively generated by combining the inter-frame shoreline optical flow to perform directional restoration of the blurred image in the river channel image sequence. Based on the water body distribution threshold, the water body area and non-water body area are divided by the prior water body coordinate range. The Retinex algorithm is used to suppress reflection in the water body area, and histogram equalization is performed on the non-water body area where the grayscale value of the dark edge of the backlit shoreline is lower than the prior backlit shoreline dark edge.

5. The method according to claim 1, characterized in that, The step of fusing and modeling based on the target river channel image sequence, the optimized pose information, and the target reconstructed region through the semantic-geometric association prior to obtain an initial river channel reality model includes: The lightweight semantic segmentation model is trained using the optimized pose information and the target reconstruction region, and the semantic-geometric association prior is incorporated into the training to generate various pixel-level semantic labels. Geometric features of the three-dimensional point cloud are extracted from the target river image sequence, and integrated modeling is performed based on the pixel-level semantic labels and the geometric features to obtain the initial river scene model.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Acquire river GIS data, historical modeling data, and manually labeled samples. Based on the river GIS data, the historical modeling data, and the manually labeled samples, construct a river prior knowledge base that includes the structural prior, the environmental prior, the semantic-geometric association prior, and the temporal prior. The river prior knowledge base is dynamically updated based on feedback data.

7. A device for real-time image sequence-based river scene modeling, characterized in that, The device includes: The image acquisition module is used to acquire a sequence of river images from a motion perspective and input the river image sequence into a pre-constructed river prior knowledge base; the river prior knowledge base includes structural priors, environmental priors, semantic-geometric association priors, and temporal priors. The image processing module is used to perform directional deblurring on the river image sequence using the shoreline linear parameters in the structural prior, and to perform zonal illumination correction on the river image sequence using the water distribution threshold in the environmental prior, so as to obtain the preprocessed target river image sequence. The optimization and reconstruction module is used to perform real-time optimization of motion pose based on the geometric features of key facilities and the river cross-section contour template in the prior structure, to obtain optimized pose information, and to perform prior-guided regional dense reconstruction based on the illumination interference parameters and dynamic interference features in the prior environment, to obtain the target reconstruction area. The fusion modeling module is used to perform fusion modeling based on the target river image sequence, the optimized pose information, and the target reconstruction region through the semantic-geometric association prior to obtain an initial river scene model. The initial river scene model is then updated using the structural prior and the temporal prior to obtain a three-dimensional semantic model of the river scene.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images