Method and system for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology

Through optical SAR fusion and binocular depth estimation technology, the problem of distance measurement accuracy and speed in dangerous tree monitoring along the railway was solved, high-quality video streaming and accurate dangerous tree recognition were achieved, and the effect of railway safety monitoring was improved.

CN118778064BActive Publication Date: 2024-11-19ZHONGKE LANZHUO (BEIJING) INFORMATION TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411232710.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-04
Publication Date
2024-11-19
Estimated Expiration
2044-09-04

AI Technical Summary

Technical Problem

In the monitoring of dangerous trees along the railway, the accuracy of longitudinal parallax distance measurement is low, the distance measurement of lidar is limited and the scanning speed is slow, and the monitoring accuracy of monocular depth estimation is highly dependent on the quality of video streams and distance measurement accuracy, which cannot be effectively guaranteed. Railway driving safety and normal operation of facilities and equipment.

Method used

Using optical SAR fusion and binocular depth estimation technology, the calibration images are obtained through binocular imaging device, the monocular internal reference data and binocular position data are calculated, image correction and feature matching are performed, and combined with synthetic aperture radar images to generate high-quality to determine the dangerous tree area along the railway.

Benefits of technology

It improves the video stream quality and ranging accuracy, can more accurately identify and locate dangerous tree areas along the railway, and enhances the reliability and efficiency of railway safety monitoring.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118778064B_ABST
    Figure CN118778064B_ABST
Patent Text Reader

Abstract

The present invention provides a method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology, comprising: responding to a dangerous tree monitoring instruction, acquiring a calibration image collected by a binocular camera device, processing the calibration image, and acquiring monocular internal reference data and binocular position data; receiving a visible light image collected by the binocular camera device on the environment along the railway, correcting the visible light image based on the monocular internal reference data and the binocular position data, and obtaining a binocular corrected image; receiving a SAR image collected by a synthetic aperture radar, fusing the binocular corrected image with the SAR image, and generating a fusion result image, wherein the fusion result image includes a left-camera fusion image and a right-camera fusion image; acquiring feature pixel points of the left-camera fusion image and the right-camera fusion image, acquiring feature matching pairs based on the feature pixel points, and generating a binocular disparity map based on each feature matching pair; converting the binocular disparity map into a depth map according to the monocular internal reference data and the binocular position data, and determining a dangerous tree area along the railway based on the depth map.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a method and system for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology. Background Art

[0002] During their growth, trees planted or growing naturally along the railway are affected by many factors such as bad weather, which can easily cause problems such as blocking signals, affecting observation, damaging facilities and equipment, invading limits, tilting, and falling, forming safety hazards, endangering railway driving safety and the normal operation of facilities and equipment. These trees are called railway dangerous trees. Therefore, there is an urgent need to effectively monitor dangerous trees along the railway. In order to ensure railway driving safety and the normal operation of facilities and equipment, it is necessary to monitor dangerous trees along the railway.

[0003] At present, dangerous trees along the railway are usually monitored by longitudinal parallax, lidar matrix or monocular depth estimation. However, there are problems such as low ranging accuracy of longitudinal parallax, limited ranging and slow scanning speed of lidar, and high dependence of monocular depth estimation monitoring accuracy on the quality of the acquired video stream and ranging accuracy. Summary of the invention

[0004] The embodiment of the present invention provides a method and system for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology, which can improve video stream quality and ranging accuracy.

[0005] To solve the above technical problems, a technical solution adopted in an embodiment of the present invention is: to provide a method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology, including: responding to a dangerous tree monitoring instruction, obtaining a calibration image collected by a binocular camera device, processing the calibration image, obtaining monocular intrinsic parameter data and binocular position data, receiving a visible light image collected by the binocular camera device of the railway environment, correcting the visible light image based on the monocular intrinsic parameter data and the binocular position data to obtain a binocular corrected image, receiving a SAR image collected by a synthetic aperture radar, fusing the binocular corrected image with the SAR image, and generating a fusion result image, the fusion result image including a left-camera fusion image and a right-camera fusion image, obtaining feature pixel points of the left-camera fusion image and the right-camera fusion image, obtaining feature matching pairs based on the feature pixel points, generating a binocular disparity map based on each feature matching pair, converting the binocular disparity map into a depth map according to the monocular intrinsic parameter data and the binocular position data, and determining a dangerous tree area along the railway based on the depth map.

[0006] Optionally, in response to a dangerous tree monitoring instruction, obtaining a calibration image captured by a binocular camera device, processing the calibration image, and obtaining monocular internal reference data and binocular position data, including:

[0007] In response to a dangerous tree monitoring instruction, a calibration image collected by a binocular camera device for a target calibration object is obtained, a corner point detection algorithm is called to process the calibration image, and corner point position data is obtained, the corner point position data includes the actual pixel position of each corner point in the calibration image, a world coordinate system is established with an arbitrary corner point as the origin, a calibration size of the target calibration object is obtained, the corner point world coordinates corresponding to each corner point are located in the world coordinate system based on the calibration size, the corner point world coordinates are input into a camera imaging model, a predicted pixel position is output based on the camera imaging model, the camera imaging model includes camera intrinsic parameter data, the predicted pixel position and the actual pixel position of each corner point are respectively input into a preset optimization algorithm, a reprojection error is obtained based on the preset optimization algorithm, the camera intrinsic parameter data when the reprojection error is minimized is determined as the monocular intrinsic parameter data, and the binocular position data is obtained based on the monocular intrinsic parameter data and the corner point position data.

[0008] Optionally, the calibration image includes a left-camera calibration image and a right-camera calibration image, and the binocular position data is obtained based on the monocular intrinsic parameter data and the corner point position data, including: performing corner point matching according to the corner point position data of the left-camera calibration image and the corner point position data of the right-camera calibration image to obtain multiple corner point matching pairs, calling a preset matrix acquisition rule based on the multiple corner point matching pairs, constructing a constraint matrix according to the preset matrix acquisition rule, converting the constraint matrix into a position matrix based on the monocular intrinsic parameter data, decomposing the position matrix, and obtaining the binocular position data.

[0009] Optionally, the monocular intrinsic reference data includes left-camera intrinsic reference data and right-camera intrinsic reference data, and the visible light image captured by the binocular camera device of the environment along the railway is received; the visible light image is corrected based on the monocular intrinsic reference data and the binocular position data to obtain a binocular corrected image, including: responding to a calibration completion instruction, receiving the visible light image captured by the binocular camera device of the environment along the railway, the visible light image includes a left-camera image and a right-camera image, performing distortion correction on the left-camera image based on the left-camera intrinsic reference data to obtain a left-camera distortion-corrected image, performing distortion correction on the right-camera image based on the right-camera intrinsic reference data to obtain a right-camera distortion-corrected image, and performing row alignment on the left-camera distortion-corrected image and the right-camera distortion-corrected image based on the binocular position data to obtain a binocular corrected image.

[0010] Optionally, a SAR image acquired by a synthetic aperture radar is received, and the binocular correction image is fused with the SAR image to generate a fusion result graph, including: inputting the binocular correction image and the SAR image into a feature extractor, outputting a visible light modal feature graph and a SAR modal feature graph based on the feature extractor, calling a preset fusion rule to process the visible light modal feature graph and the SAR modal feature graph to obtain a preliminary fusion feature graph, inputting the preliminary fusion feature graph into a decoder, generating a preliminary fusion graph based on the decoder, inputting the preliminary fusion graph and the preset fusion graph into a discriminator, performing adversarial training based on the discriminator, and obtaining a fusion result graph.

[0011] Optionally, the feature pixel points include left-camera feature pixel points and right-camera feature pixel points, and the feature pixel points of the left-camera fusion image and the right-camera fusion image are obtained, feature matching pairs are obtained based on the feature pixel points, and a binocular disparity map is generated based on each feature matching pair, including: performing feature extraction on the left-camera fusion image, obtaining multiple left-camera visual key points, and determining the multiple left-camera visual key points as the left-camera feature pixel points; performing feature extraction on the right-camera fusion image, obtaining multiple right-camera visual key points, and determining the multiple right-camera visual key points as the right-camera feature pixel points; calling a preset matching rule to perform feature matching on the left-camera feature pixel points and the right-camera feature pixel points to obtain feature matching pairs; calling a preset difference acquisition rule, calculating the horizontal distance difference of the current feature matching pair according to the preset difference acquisition rule, and determining the horizontal distance difference as the disparity value corresponding to the current feature matching pair; repeating the above step of obtaining the disparity value until the disparity values ​​corresponding to all feature matching groups are obtained, and combining the disparity values ​​corresponding to all feature matching groups to generate a binocular disparity map.

[0012] Optionally, the binocular disparity map is converted into a depth map according to the monocular intrinsic parameter data and the binocular position data, including: obtaining a baseline distance of the binocular camera device based on the binocular position data, the baseline distance being the distance between the optical centers of a left camera and a right camera, determining the focal length of the binocular camera device based on the monocular intrinsic parameter data, converting a current disparity value into a current depth value according to the baseline distance and the focal length, repeating the above step of converting the current disparity value into a current depth value until depth values ​​corresponding to all disparity values ​​are obtained, and generating a depth map based on multiple depth values.

[0013] Optionally, converting the current disparity value into a current depth value according to the baseline distance and the focal length includes:

[0014] The current depth value is calculated using the following formula:

[0015] in, is the focal length, is the baseline distance, is the parallax value.

[0016] Optionally, determining the dangerous tree area along the railway based on the depth map includes: acquiring multiple depth values ​​of the depth map in real time, judging whether each depth value is within a preset depth range, retaining the depth value when the depth value is within the preset depth range to obtain a depth value set, analyzing each depth value in the depth value set to obtain a depth attribute, marking multiple independent clustering areas based on the depth attribute, obtaining clustering area features of each independent clustering area, calling a feature mapping library to obtain preset area features corresponding to the dangerous trees along the railway, comparing the clustering area features with the preset area features, obtaining feature similarity between each independent clustering area and the preset area, and when the feature similarity is greater than or equal to a preset similarity threshold, determining the corresponding independent clustering area as a dangerous tree area along the railway.

[0017] Optionally, when the feature similarity is greater than or equal to a preset similarity threshold, the corresponding independent clustering area is determined as a dangerous tree area along the railway, including: obtaining the first positioning information corresponding to the dangerous tree area along the railway, generating a corresponding processing work order and sending it to the maintenance end, receiving the dangerous tree processing information corresponding to the dangerous tree area along the railway fed back by the maintenance end, dividing each railway line into multiple processing segments according to a preset segmentation distance, obtaining the center point of each processing segment, constructing a corresponding collection reference range based on the center point according to a preset radius, obtaining the dangerous tree processing information of all railways within the collection reference range, and performing training processing based on the dangerous tree processing information to update the preset similarity threshold.

[0018] Optionally, the method divides each railway line into multiple processing sections according to a preset segmentation distance, obtains the center point of each processing section, and constructs a corresponding acquisition reference range based on the center point according to a preset radius, including: taking each processing section along the first railway line as a first processing section, and taking each center point as a first center point, constructing a corresponding first acquisition reference range based on the first center point according to a preset radius, obtaining other first processing sections within the first acquisition reference range corresponding to the first processing section as first reference sections, and obtaining other processing sections along the second railway line corresponding to the first processing section as second reference sections, obtaining the number of first reference sections and second reference sections to obtain a first reference number, if the first reference number is greater than or equal to a first preset number, determining the current first acquisition reference range as the first acquisition reference range for subsequent training, if the first reference number is less than the first preset number, then constructing a corresponding second acquisition reference range after enlarging the preset radius, until the first reference number is greater than or equal to the first preset number and the first acquisition reference range for training is obtained.

[0019] Optionally, the acquiring and collecting dangerous tree processing information along all railways within the reference range, and performing training processing on the dangerous tree processing information to update the preset similarity threshold, includes:

[0020] The dangerous tree processing information includes at least one of the existence of dangerous trees, the number of dangerous trees and the non-existence of dangerous trees. The dangerous tree processing information of all first processing sections within the acquisition reference range is obtained. If the dangerous tree processing information is the existence of dangerous trees and the number of dangerous trees, a first adjustment coefficient is calculated based on the number of dangerous trees. If the dangerous tree processing information is the non-existence of dangerous trees, a preset second adjustment coefficient is obtained. The dangerous tree processing information of all second processing sections within the acquisition reference range is obtained. A coefficient offset weight is obtained based on the distance between the center point of each second processing section and the first center point. The dangerous tree processing information of all second processing sections within the acquisition reference range is obtained. If the dangerous tree processing information is the existence of dangerous trees and the number of dangerous trees, a third adjustment coefficient is calculated based on the number of dangerous trees and the coefficient offset weight. If the dangerous tree processing information is the non-existence of dangerous trees, a preset fourth adjustment coefficient is obtained. The preset similarity threshold is updated based on the first adjustment coefficient, the second adjustment coefficient, the third adjustment coefficient and the fourth adjustment coefficient.

[0021] Optionally, the updating of the preset similarity threshold based on the first adjustment coefficient, the second adjustment coefficient, the third adjustment coefficient and the fourth adjustment coefficient includes: adding the first adjustment coefficient, the second adjustment coefficient, the third adjustment coefficient and the fourth adjustment coefficient to obtain an overall adjustment coefficient, performing weighted processing on the overall adjustment coefficient and then updating the preset similarity threshold, and calculating the updated preset similarity threshold by the following formula:

[0022] in, is the preset similarity threshold after updating. is the first adjustment coefficient, is the number of dangerous trees in the first treatment section, is the first reverse adjustment coefficient, For the The coefficient offset weights of the second processing stage, For the The horizontal coordinate value of the second processing section, is the horizontal coordinate value of the first center point, For the The ordinate value of the second processing section, is the ordinate value of the first center point, is the second adjustment coefficient, is the third adjustment factor, For the The coefficient offset weights of the second processing stage, For the The number of dangerous trees in the second treatment section, is the upper limit value of the second processing section, is the second reverse adjustment coefficient, is the fourth adjustment coefficient, is a constant value, It is the preset similarity threshold before updating processing.

[0023] In order to solve the above technical problems, another technical solution adopted by the embodiment of the present invention is: to provide a dangerous tree monitoring system along the railway based on optical SAR fusion and binocular depth estimation technology, including: a data acquisition module, used to respond to the dangerous tree monitoring instruction, obtain the calibration image collected by the binocular camera device, process the calibration image, obtain the monocular internal reference data and the binocular position data, and an image correction module, used to receive the visible light image collected by the binocular camera device of the railway environment, correct the visible light image based on the monocular internal reference data and the binocular position data, and obtain the binocular correction image , an image fusion module, used to receive the SAR image collected by the synthetic aperture radar, fuse the binocular correction image with the SAR image, and generate a fusion result image, wherein the fusion result image includes a left-camera fusion image and a right-camera fusion image, a feature matching module, used to obtain feature pixel points of the left-camera fusion image and the right-camera fusion image, obtain feature matching pairs based on the feature pixel points, and generate a binocular disparity map based on each feature matching pair, and an area determination module, used to convert the binocular disparity map into a depth map according to the monocular internal reference data and the binocular position data, and determine the dangerous tree area along the railway based on the depth map.

[0024] Different from the prior art, the present invention provides a method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology, comprising: responding to a dangerous tree monitoring instruction, obtaining a calibration image collected by a binocular camera device, processing the calibration image, and obtaining monocular internal reference data and binocular position data; receiving a visible light image collected by the binocular camera device of the railway environment, and correcting the visible light image based on the monocular internal reference data and the binocular position data to obtain a binocular corrected image; receiving a SAR image collected by a synthetic aperture radar, fusing the binocular corrected image with the SAR image, and generating a fusion result image, wherein the fusion result image includes a left-camera fusion image and a right-camera fusion image; obtaining feature pixel points of the left-camera fusion image and the right-camera fusion image, obtaining feature matching pairs based on the feature pixel points, and generating a binocular disparity map based on each feature matching pair; converting the binocular disparity map into a depth map according to the monocular internal reference data and the binocular position data, and determining a dangerous tree area along the railway based on the depth map. The present invention fuses the visible light image acquired by the binocular camera device with the SAR image acquired by the synthetic aperture radar through the optical SAR fusion technology, obtains high-quality image information that is not affected by the environment, and improves the quality of the video stream. In addition, the image fused by the optical SAR is used as the image to be detected, and the ranging accuracy is improved through parallax calculation and depth estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] One or more embodiments are exemplarily described by corresponding drawings, which do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and the figures in the drawings do not constitute proportional limitations unless otherwise stated.

[0026] Figure 1 It is a flow chart of a method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology provided in an embodiment of the present application;

[0027] Figure 2 This is a schematic diagram of binocular ranging provided in an embodiment of the present application;

[0028] Figure 3 It is a schematic diagram of dividing the processing sections along the first railway provided in an embodiment of the present application;

[0029] Figure 4 It is a structural schematic diagram of a dangerous tree monitoring system along a railway based on optical SAR fusion and binocular depth estimation technology provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0031] It should be noted that, if there is no conflict, the various features in the embodiments of the present invention can be combined with each other, and all are within the protection scope of the present invention. In addition, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a different order from the module division in the device schematic diagram or the order in the flow chart.

[0032] The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in sequences other than those illustrated or described herein.

[0033] It should be understood that in the present invention, "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0034] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0035] See also Figure 1 , Figure 1 This is a flow chart of a method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology provided by an embodiment of the present application, including steps S1 to S5, which are as follows:

[0036] S1. Respond to a dangerous tree monitoring instruction, obtain a calibration image captured by a binocular camera device, process the calibration image, and obtain monocular internal reference data and binocular position data.

[0037] Among them, the dangerous tree monitoring instruction represents an instruction used to trigger the system to start monitoring dangerous trees along the railway; the binocular camera device represents a device composed of two cameras, specifically a left camera and a right camera, which is used to simulate the human eye and can simultaneously capture two images of the same scene; the calibration image represents an image used to calibrate and determine the parameters of the binocular camera device; the monocular internal reference data represents the internal parameters of a single camera in the binocular camera device, including but not limited to focal length, optical center, distortion parameters, etc. The left and right cameras of the binocular camera device can be selected to be of the same model and specifications; the binocular position data represents the relative position between the two cameras, specifically the rotation matrix R and translation vector t of the right camera relative to the left camera.

[0038] It should be noted that obtaining monocular intrinsic reference data can provide basic correction data for the subsequent correction of distortion in visible light images. The corrected images will provide more realistic visual information. The binocular position data can ensure that the images taken by the subsequent left camera and right camera can be aligned in space, thereby providing more accurate positioning of dangerous trees along the railway.

[0039] In some embodiments, step S1 (responding to the dangerous tree monitoring instruction, obtaining the calibration image captured by the binocular camera device, processing the calibration image, and obtaining the monocular internal reference data and binocular position data) specifically includes steps S11-S16:

[0040] S11. Respond to the dangerous tree monitoring instruction and obtain a calibration image captured by the binocular camera device of the target calibration object.

[0041] The target calibration object represents an object of known shape and size used to calibrate the camera, and specifically may be a chessboard; the calibration image represents an image of the target calibration object captured by the left camera and the right camera of the binocular camera device.

[0042] Specifically, the dangerous tree monitoring system along the railway is communicated with the railway management platform. The system can receive dangerous tree monitoring instructions sent by train staff through the railway management platform, and notify the binocular camera device to start working based on the dangerous tree monitoring instructions. The left and right cameras synchronously shoot multiple groups of calibration images according to the notification, such as checkerboard images, as the data basis for subsequent calibration. The left and right cameras can get a group of checkerboard images by shooting synchronously once.

[0043] S12, calling a corner point detection algorithm to process the calibration image to obtain corner point position data, where the corner point position data includes the actual pixel position of each corner point in the calibration image.

[0044] Among them, the corner point monitoring algorithm represents an algorithm for detecting the position of corner points in the calibration image, which can be specifically the Harris corner point detection algorithm. The application of the Harris corner point detection algorithm is well known to those skilled in the art and will not be described in detail here; the corner point position data represents the pixel position of the corner point detected in the calibration image; the corner point represents a significant point on the target calibration object in the calibration image. When the target calibration object is a chessboard, the corner point is usually the intersection of the chessboard; the actual pixel position represents the two-dimensional coordinate of the corner point in the calibration image.

[0045] Specifically, the Harris corner detection algorithm can be used to find the positions of the checkerboard intersections in the calibration image. Each checkerboard intersection is detected in the calibration image, and its pixel position is recorded, such as (150, 200), (250, 300), etc.

[0046] S13, establishing a world coordinate system with an arbitrary corner point as the origin, obtaining a calibration size of the target calibration object, and locating the corner point world coordinates corresponding to each corner point in the world coordinate system based on the calibration size.

[0047] Among them, the origin represents a reference point selected during the calibration process; the world coordinate system represents a three-dimensional coordinate system used to describe the spatial position of the target calibration object; the calibration size represents the actual physical size of the target calibration object, which is a known quantity and can be used to calculate the position of the corner point in the world coordinate system; the corner point world coordinates represent the three-dimensional coordinates of the corner point in the world coordinate system, indicating the actual spatial position of the corner point.

[0048] Specifically, a corner point is selected as the origin of the world coordinate system, and the three-dimensional position of each corner point in the world coordinate system is calculated based on the known size of the chessboard (such as the side length of each small square).

[0049] It should be noted that the Z axis of the world coordinate system is perpendicular to the chessboard surface (since the chessboard is two-dimensional, the Z coordinate is 0 for all corner points).

[0050] For example: If the size of each grid on the chessboard is 10mm, with the upper left corner as the origin, the world coordinates of the first corner point are (0, 0, 0), the world coordinates of the second corner point are (10, 0, 0), and so on.

[0051] S14, inputting the world coordinates of the corner point into a camera imaging model, and outputting a predicted pixel position based on the camera imaging model, wherein the camera imaging model includes camera intrinsic parameter data.

[0052] Among them, the camera imaging model represents a mathematical model that describes the camera imaging process, including the relationship of how to map three-dimensional world coordinates to a two-dimensional image plane; the predicted pixel position represents the theoretical position of the corner point in the image calculated based on the camera imaging model and the world coordinates of the corner point; the camera intrinsic parameter data represents the parameters of the internal characteristics of the camera, including focal length, optical center, distortion parameters, etc.

[0053] Specifically, a preset camera imaging model is used to map the three-dimensional world coordinates of the corner points onto a two-dimensional image plane to obtain the predicted pixel positions.

[0054] For example, based on the camera's current focal length, distortion parameters and other camera intrinsic parameter data, the corner point world coordinates (10, 10, 0) are mapped to the image plane, and the output predicted pixel position may be (150, 200).

[0055] It should be noted that the camera imaging model usually includes the camera's intrinsic parameter data and extrinsic parameters (the rotation and translation relationship between the camera coordinate system and the world coordinate system), which can convert three-dimensional points in the real world into two-dimensional points in the image through perspective projection. The application of the camera imaging model is well known to those skilled in the art.

[0056] S15, inputting the predicted pixel position and the actual pixel position of each corner point into a preset optimization algorithm respectively, obtaining a reprojection error based on the preset optimization algorithm, and determining the camera intrinsic parameter data when the reprojection error is minimum as the monocular intrinsic parameter data.

[0057] Among them, the preset optimization algorithm represents an algorithm for adjusting and optimizing camera parameters to minimize the error between the predicted pixel position and the actual pixel position. Specifically, it can be a least squares method, a gradient descent method, etc., by continuously adjusting the camera intrinsic parameter data and extrinsic parameters of the camera to minimize the reprojection error and achieve the optimal camera parameter configuration; the reprojection error represents the difference between the preset pixel position and the actual pixel position.

[0058] It should be noted that the above calibration process is to calibrate a single camera in the binocular camera device, that is, to calibrate the left camera and the right camera in the binocular camera device respectively.

[0059] S16. Acquire binocular position data based on the monocular intrinsic reference data and the corner point position data.

[0060] It should be noted that determining the relative position and posture between the two cameras in a binocular camera device can provide accurate basic data for subsequent stereo vision and depth estimation.

[0061] In some embodiments, the calibration image includes a left calibration image and a right calibration image, and step S16 (obtaining binocular position data based on the monocular intrinsic reference data and the corner point position data) specifically includes steps S161-S163:

[0062] S161 . Perform corner point matching according to the corner point position data of the left calibration image and the corner point position data of the right calibration image to obtain a plurality of corner point matching pairs.

[0063] Among them, the left calibration image represents the image of the target calibration object taken by the left camera of the binocular camera device; the right calibration image represents the image of the target calibration object taken by the right camera of the binocular camera device; and the corner point matching pair represents a group of corner points corresponding to each other in the left calibration image and the right calibration image.

[0064] Specifically, corresponding corner point pairs are found in the left camera calibration image and the right camera calibration image, and by matching the corner points, the correspondence between the same object points captured by the left and right cameras in the two images can be determined.

[0065] For example, in the image taken by the left camera, the corner point A of the chessboard corresponds to the corner point B in the image taken by the right camera, so A and B are a corner point matching pair.

[0066] S162: Calling a preset matrix acquisition rule based on multiple corner point matching pairs, and constructing a constraint matrix according to the preset matrix acquisition rule.

[0067] Among them, the constraint matrix, that is, the basic matrix, can describe the geometric relationship between the corner point matching pairs, and the preset matrix acquisition rules represent the pre-defined rules and methods for generating the basic matrix based on the corner point matching pairs. Specifically, the eight-point algorithm can be used for calculation, which is well known to those skilled in the art.

[0068] S163: Convert the constraint matrix into a position matrix based on the monocular internal reference data, decompose the position matrix, and obtain binocular position data.

[0069] The position matrix, namely the essential matrix, is a matrix that describes the relative positions and postures of the two cameras of the binocular camera device.

[0070] Specifically, the basic matrix is ​​normalized using the monocular intrinsic parameter data of the camera to convert the basic matrix into an essential matrix, and then the essential matrix is ​​decomposed into a rotation matrix and a displacement vector. The conversion and decomposition methods are well known to those skilled in the art and will not be described in detail here.

[0071] S2. Receive the visible light image captured by the binocular camera device of the environment along the railway, and correct the visible light image based on the monocular intrinsic reference data and the binocular position data to obtain a binocular corrected image.

[0072] Among them, the railway environment refers to the natural and artificial environment of the railway track and its two sides, including trees, buildings, terrain, etc.; the visible light image refers to the image taken by the binocular camera device within the visible light range, which contains detailed information of the railway environment; correction means processing the image obtained by the camera to eliminate shooting errors caused by the optical lens characteristics of the camera, assembly errors, etc., and the image group obtained after processing is the binocular correction image.

[0073] It should be noted that obtaining binocular correction images can correct the distortion and alignment errors in the images, and provide high-quality image data for subsequent stereo vision processing and depth estimation, so as to ensure accurate monitoring of the environment along the railway, that is, to improve the ranging accuracy.

[0074] In some embodiments, the monocular intrinsic reference data includes left-camera intrinsic reference data and right-camera intrinsic reference data, and step S2 (receiving the visible light image captured by the binocular camera device of the environment along the railway, and correcting the visible light image based on the monocular intrinsic reference data and the binocular position data to obtain a binocular corrected image) specifically includes steps S21-S24:

[0075] S21. Responding to a calibration completion instruction, receiving visible light images captured by the binocular camera device of the environment along the railway, wherein the visible light images include a left-camera image and a right-camera image.

[0076] Among them, the left-camera internal parameter data represents the internal parameters of the left camera in the binocular camera device, including focal length, optical center, distortion parameters, etc.; the right-camera internal parameter data represents the internal parameters of the right camera in the binocular camera device, including focal length, optical center, distortion parameters, etc.; the calibration completion instruction represents an instruction indicating that the calibration process of the binocular camera has been completed; the left-camera image represents the visible light image of the environment along the railway taken by the left camera in the binocular camera device; the right-camera image represents the visible light image of the environment along the railway taken by the right camera in the binocular camera device.

[0077] It should be noted that the left image and the right image are visible light images of the same railway environment captured simultaneously by the left camera and the right camera of the binocular camera device, that is, the binocular camera device simultaneously captures the scenery on both sides of the railway, such as tracks, trees, buildings, etc., to generate two images, one from the left camera and the other from the right camera.

[0078] S22. Perform distortion correction on the left-camera image based on the left-camera internal reference data to obtain a left-camera distortion-corrected image.

[0079] The distortion correction refers to the process of correcting the geometric distortion in the image; the left-photographed distortion-corrected image refers to the left-photographed image after the geometric distortion is eliminated.

[0080] Example: The straight lines in the left image appear bent. After applying distortion correction, these straight lines will return to their original straight shape.

[0081] S23. Perform distortion correction on the right camera image based on the right camera internal reference data to obtain a right camera distortion corrected image.

[0082] The right-photographed distortion-corrected image refers to the right-photographed image after geometric distortion is eliminated.

[0083] S24. Performing line alignment on the left-photographed distortion-corrected image and the right-photographed distortion-corrected image based on the binocular position data to obtain a binocular correction image.

[0084] The row alignment means aligning pixels of the left image and the right image on the same row.

[0085] Specifically, the left-camera distortion-corrected image and the right-camera distortion-corrected image are geometrically adjusted according to the relative positions and postures of the left and right cameras, so that the pixels of each row are aligned.

[0086] It should be noted that aligning the images from the left and right cameras can ensure that the pixels of the two images are aligned on the horizontal line, thereby performing effective matching.

[0087] S3, receiving a SAR image collected by a synthetic aperture radar, fusing the binocular correction image with the SAR image, and generating a fusion result image, wherein the fusion result image includes a left-shot fusion image and a right-shot fusion image.

[0088] Among them, the synthetic aperture radar represents an active remote sensing technology, namely SAR, which can acquire ground images by emitting electromagnetic waves and receiving their echoes; the SAR image represents a radar image acquired by the synthetic aperture radar, which may specifically include detailed information of ground objects; fusion represents the joint processing of the binocular correction image and the SAR image to generate an image containing information of both, and the generated image is the fusion result image; the left-camera fusion image represents an image generated by fusing the correction image corresponding to the left camera with the SAR image; the right-camera fusion image represents an image generated by fusing the correction image corresponding to the right camera with the SAR image.

[0089] It should be noted that SAR can penetrate obstacles such as clouds and vegetation to provide high-resolution ground images. The binocular camera device simulates the stereoscopic vision of the human eye. By fusing the images of the left and right cameras respectively, it can obtain richer depth information and stereoscopic visual effects, which is helpful for the subsequent accurate identification and location of dangerous tree areas along the railway.

[0090] In some embodiments, step S3 (receiving a SAR image acquired by a synthetic aperture radar, fusing the binocular correction image with the SAR image, and generating a fusion result image) specifically includes steps S31-S34:

[0091] S31, inputting the binocular correction image and the SAR image into a feature extractor, and outputting a visible light modal feature map and a SAR modal feature map based on the feature extractor.

[0092] Among them, the feature extractor, i.e., encoder, represents an algorithm or model for extracting features from an image; the visible light modal feature map can represent features extracted from a binocular corrected image, and includes important information in the visible light image, such as the appearance and outline of trees; the SAR modal feature map can represent features extracted from a SAR image, and includes important information in the SAR image, such as the density of trees and the underlying ground structure.

[0093] Specifically, the binocular correction image is input into the feature extractor, the features in the image are extracted, and the visible light modal feature map is output; the SAR image is input into the feature extractor, the features in the image are extracted, and the SAR modal feature map is output.

[0094] It should be noted that the feature extractor consists of four convolutional layers and four pooling layers, which can extract low-level and high-level features of the image. The application of feature extractors is well known to those skilled in the art. The extracted visible light modal feature map and SAR modal feature map can effectively represent the important information in the original image and provide basic data for subsequent image fusion and analysis.

[0095] S32: calling a preset fusion rule to process the visible light modal feature map and the SAR modal feature map to obtain a preliminary fusion feature map.

[0096] Among them, the preset fusion rule represents the preset rule and method for fusing the visible light modal feature map and the SAR modal feature map, which can specifically be a weighted average, feature selection and other methods that can combine the features of different modalities; the preliminary fusion feature map represents a map containing comprehensive feature information from visible light images and SAR images.

[0097] For example, weighted averaging can synthesize the important features in the visible light modal feature map and the SAR modal feature map according to the weights, retaining the useful information of both.

[0098] It should be noted that by fusing feature maps of different modalities, the advantages of visible light images and SAR images can be comprehensively utilized to generate images containing more information.

[0099] S33, inputting the preliminary fusion feature map into a decoder, and generating a preliminary fusion map based on the decoder.

[0100] Among them, the decoder is a deep learning network structure that can convert the preliminary fusion feature map into an image; the preliminary fusion map represents an image generated based on the fusion of the binocular correction image and the SAR image. After being processed by the decoder, the preliminary fusion map is closer to the actual situation in terms of resolution and details.

[0101] It should be noted that the decoder consists of four deconvolution layers and four upsampling layers, which can reconstruct the fused feature information into a complete visual image, retaining more details and texture information. The application of the decoder is well known to those skilled in the art.

[0102] S34, inputting the preliminary fusion graph and the preset fusion graph into the discriminator, performing adversarial training based on the discriminator, and obtaining a fusion result graph.

[0103] Among them, the preset fusion image represents a reference image for training the discriminator, which is usually a high-quality fusion result image; the discriminator represents a deep learning model for evaluating the quality of the fusion result image; adversarial training represents a method for training a discriminant model so that the discriminant model can accurately distinguish the image quality; the fusion result image represents a map that integrates the best features of the binocular correction image and the SAR image, and the visual effects and details are more in line with the actual situation, that is, the final image after adversarial training optimization.

[0104] It should be noted that the discriminator consists of five convolutional layers and one fully connected layer, which can determine whether the input image is a real fused image or a generated fused image, and its output is a binary classification probability value, which indicates the probability that the input image is a real fused image. The application of the discriminator is well known to those skilled in the art.

[0105] S4. Acquire feature pixel points of the left-camera fusion image and the right-camera fusion image, acquire feature matching pairs based on the feature pixel points, and generate a binocular disparity map based on each feature matching pair.

[0106] Among them, the feature pixel points represent pixel points with significant features in the image, usually corner points, edge points, etc.; the feature matching pairs represent feature pixel pairs that correspond to each other in the left-camera fusion image and the right-camera fusion image; the binocular disparity map is an image showing the difference in viewing angles between the left camera and the right camera.

[0107] It should be noted that the binocular disparity map is an image that shows the difference in viewing angles between the left camera and the right camera. Through the disparity map, the distance from each pixel to the camera can be calculated subsequently to achieve depth perception of the scene.

[0108] In some embodiments, the feature pixel points include left-camera feature pixel points and right-camera feature pixel points, and step S4 (obtaining feature pixel points of the left-camera fusion image and the right-camera fusion image, obtaining feature matching pairs based on the feature pixel points, and generating a binocular disparity map based on each feature matching pair) specifically includes steps S41-S45:

[0109] S41, performing feature extraction on the left-camera fusion image to obtain a plurality of left-camera visual key points, and determining the plurality of left-camera visual key points as left-camera feature pixel points.

[0110] Among them, the left-camera feature pixel points represent representative pixel points extracted from the left-camera fusion image, and these points have significant features; the right-camera feature pixel points represent representative pixel points extracted from the right-camera fusion image, and these points have significant features; feature extraction represents the process of extracting representative and significant pixel points from the left-camera fusion image, and specifically, algorithms such as SIFT and SURF can be used; the left-camera visual key points represent important visual points in the left-camera fusion image.

[0111] S42: extract features from the right-camera fusion image to obtain a plurality of right-camera visual key points, and determine the plurality of right-camera visual key points as right-camera feature pixel points.

[0112] The feature extraction refers to the process of extracting representative and significant pixels from the right-camera fusion image, and specifically, algorithms such as SIFT and SURF may be used; the right-camera visual key points refer to important visual points in the right-camera fusion image.

[0113] It should be noted that obtaining the key feature points in the left-camera fusion image and the right-camera fusion image can lay the foundation for subsequent feature matching and disparity calculation.

[0114] S43, calling a preset matching rule to perform feature matching on the left-photographed feature pixel points and the right-photographed feature pixel points to obtain a feature matching pair.

[0115] Among them, the preset matching rules represent pre-set rules and methods for matching left-camera feature pixel points and right-camera feature pixel points, which can determine which feature points are a corresponding set of matching pairs, and can specifically be Euclidean distance, Manhattan distance, etc., which are well known to those skilled in the art and will not be repeated here; feature matching means finding points with similar or identical features in different images; feature matching pairs refer to corresponding feature point pairs found in two images, and these two feature points are considered to be, to some extent, the representation of the same object point under different perspectives.

[0116] It should be noted that by matching feature points, the positional relationship of the same object in the two camera views can be determined to ensure the accuracy of parallax calculation.

[0117] S44, calling a preset difference acquisition rule, calculating a horizontal distance difference of a current feature matching pair according to the preset difference acquisition rule, and determining the horizontal distance difference as a disparity value corresponding to the current feature matching pair.

[0118] Among them, the preset difference acquisition rule represents a pre-set rule and method for calculating the horizontal distance difference between feature matching pairs; the current feature matching pair represents a matching pair of a left-camera feature pixel point and a right-camera feature pixel point processed at a specific moment; the horizontal distance difference represents the horizontal displacement between the left-camera feature pixel point and the right-camera feature pixel point in the current feature matching pair, that is, it reflects the horizontal displacement of the same object point in the two camera views; the disparity value represents the position difference of the same object point in the left-camera image and the right-camera image. The larger the parallax, the closer the object is to the camera, and vice versa.

[0119] S45 , repeat the above step of obtaining the disparity value until the disparity values ​​corresponding to all feature matching groups are obtained, and combine the disparity values ​​corresponding to all feature matching groups to generate a binocular disparity map.

[0120] The binocular disparity map may represent the disparity of each pixel in the scene.

[0121] Specifically, the disparity values ​​of all feature matching pairs are integrated together to generate a binocular disparity map of the entire scene.

[0122] S5. Convert the binocular disparity map into a depth map according to the monocular internal reference data and the binocular position data, and determine the dangerous tree area along the railway based on the depth map.

[0123] The depth map is an image representing the distance from each point in the scene to the camera; the dangerous tree area along the railway represents an area where dangerous trees exist.

[0124] It should be noted that the identification of dangerous tree areas along the railway can be used for maintenance and safety warning to prevent fallen trees or branches from affecting the normal operation of trains on the railway.

[0125] In some embodiments, step S5 (converting the binocular disparity map into a depth map according to the monocular intrinsic reference data and the binocular position data, and determining the dangerous tree area along the railway based on the depth map) specifically includes steps S51-S53:

[0126] S51. Acquire a baseline distance of the binocular camera device based on the binocular position data, where the baseline distance is a distance between optical centers of a left camera and a right camera.

[0127] Among them, the baseline distance represents the actual distance between the two cameras in the binocular camera device, which can affect the accuracy and range of depth measurement; the left camera represents the left camera in the binocular camera device; the right camera represents the right camera in the binocular camera device; the optical center represents the geometric center of the camera lens, the center point through which the light passes.

[0128] S52: determining a focal length of the binocular camera device based on the monocular intrinsic reference data, and converting a current disparity value into a current depth value according to the baseline distance and the focal length.

[0129] Among them, the focal length represents the distance from the optical center of the lens to the imaging plane; the current disparity value represents the disparity value of a specific pixel point in the binocular disparity map, indicating the horizontal displacement difference of the pixel point in all images; the current depth value represents the actual distance of the pixel point from the camera.

[0130] See also Figure 2 , Figure 2 is a schematic diagram of binocular ranging provided in an embodiment of the present application. P is a point on the target to be measured, and are the optical centers of the two cameras respectively. The imaging points of point P on the two camera sensors are P and P' respectively (the imaging plane of the camera is placed in front of the lens after rotation). f is the focal length of the camera, B is the baseline distance, and Z is the depth value to be determined. The focal length f and the camera center distance B can be obtained through calibration.

[0131] Converting the current disparity value into the current depth value according to the baseline distance and the focal length includes:

[0132] The current depth value is calculated using the following formula:

[0133] in, is the focal length, is the baseline distance, is the disparity value, the disparity value is d, that is, Z=fB / d.

[0134] Assuming that the coordinates of the target point in the left image are (x, y), the parallax formed on the left image and the right image is d, and the coordinates of the target point in the world coordinate system with the optical center of the left camera as the origin are (X, Y, Z), then there exists the transformation matrix Q shown below, so that Q*[xyd 1]*=[XYZW]*;

[0135]

[0136] To accurately determine the distance Z of a point in three-dimensional space, we need to obtain the focal length f, parallax d, and baseline distance B. To obtain the X and Y coordinates, we need to know the offset of the coordinate system of the left and right images and the origin in the world coordinate system. and . Where f, d, and The initial value can be obtained through stereo calibration, and the two cameras can be placed completely parallel to each other through stereo calibration optimization. , Similar to f, stereo matching can obtain the last variable, namely disparity d, to complete the three-dimensional coordinates corresponding to a point.

[0137] It is understandable that the camera calibration and binocular correction steps in binocular stereo vision ranging improve f, d, B, and accuracy, making the distance measurement more accurate.

[0138] S53 , repeating the above step of converting the current disparity value into the current depth value until the depth values ​​corresponding to all the disparity values ​​are obtained, and generating a depth map based on the multiple depth values.

[0139] The depth map contains an image of the distance from each point in the scene to the camera, which provides three-dimensional structural information of objects in the scene.

[0140] It should be noted that subsequent analysis of the depth map can identify areas with trees that are close to the railway and may be dangerous. This information is very important for railway safety monitoring and can detect and deal with potential dangers in advance.

[0141] In some embodiments, step S5 (determining the dangerous tree area along the railway based on the depth map) specifically includes steps S54-S57:

[0142] S54, acquiring multiple depth values ​​of the depth map in real time, determining whether each depth value is within a preset depth range, and retaining the depth value when the depth value is within the preset depth range to obtain a depth value set.

[0143] The preset depth range refers to a preset numerical range for screening depth values, which may be set as a common distance range along a railway; and the depth value set refers to a set of all depth values ​​within the preset depth range.

[0144] It should be noted that the depth value set is a set of all depth values ​​within a preset depth range. By filtering and retaining the depth value set, it is possible to focus on a target area within a specific depth range, which facilitates further analysis and processing.

[0145] S55 , analyzing each depth value in the depth value set to obtain a depth attribute, marking a plurality of independent cluster regions based on the depth attribute, and obtaining a cluster region feature of each independent cluster region.

[0146] Among them, the depth attribute represents the statistical characteristics of each depth value, such as depth range, depth distribution, etc.; the independent cluster area is a group of pixels with similar characteristics identified in the depth map, such as a group of pixels with similar depth values ​​in the depth map; the cluster area feature represents the size, shape, depth distribution, etc. of the area.

[0147] Specifically, each independent cluster region may be found in the depth map through a clustering algorithm, and the clustering algorithm may specifically be a K-means algorithm, which is well known to those skilled in the art.

[0148] S56, calling a feature mapping library to obtain preset regional features corresponding to dangerous trees along the railway, comparing the clustering regional features with the preset regional features, and obtaining feature similarity between each independent clustering region and the preset region.

[0149] The feature mapping library indicates that preset regional features of dangerous trees along the railway are stored; the feature similarity indicates the degree of similarity between the clustered regional features and the preset regional features.

[0150] S57: When the feature similarity is greater than or equal to a preset similarity threshold, the corresponding independent clustering area is determined as a dangerous tree area along the railway.

[0151] The preset similarity threshold represents a threshold used to determine whether the feature similarity is high enough.

[0152] It should be noted that the preset similarity threshold is a numerical limit. When the feature similarity reaches or exceeds the threshold, the monitored independent clustering area is a dangerous tree area.

[0153] In some embodiments, step S57 (when the feature similarity is greater than or equal to a preset similarity threshold, determining the corresponding independent clustering area as a dangerous tree area along the railway) specifically includes steps S571-S573:

[0154] S571. Obtain first positioning information corresponding to the dangerous tree area along the railway, generate a corresponding processing work order, and send it to the maintenance end, and receive the dangerous tree processing information corresponding to the dangerous tree area along the railway fed back by the maintenance end.

[0155] Among them, the dangerous tree area along the railway in the present invention can be referred to as the dangerous tree area, and the first positioning information represents the geographical location information of the dangerous tree area, such as GPS coordinates, etc.; the processing work order represents the generated work task, including but not limited to specific instructions for processing the discovered dangerous tree areas, the specific location and potential risks of each dangerous tree area, etc.; the maintenance end represents the terminal equipment used by the department or staff responsible for the equipment and safety maintenance along the railway, which can be specifically a mobile phone, computer, etc.; the dangerous tree processing information represents the specific situation of the dangerous trees, including but not limited to whether there are dangerous tree areas, the number of dangerous tree areas and the specific location corresponding to each dangerous tree area, etc.

[0156] Specifically, when the system detects the existence of dangerous tree areas along the railway, it obtains the geographic positioning information of the dangerous tree areas, uses the geographic positioning information as the first positioning information, generates a processing work order based on the first positioning information and sends it to the maintenance end. The processing work order records in detail the specific location and potential risks of the dangerous tree areas. The maintenance end can obtain the specific situation of the dangerous trees based on the received processing work order, that is, whether there are dangerous tree areas, the number of dangerous tree areas and the specific location of each dangerous tree area, etc., and generate and display dangerous tree processing information based on the above data. The staff can inspect and process each dangerous tree area separately according to the number of dangerous tree areas and the specific location of each dangerous tree area in the dangerous tree processing information to ensure the safety of train driving.

[0157] S572. Divide each railway line into multiple processing sections according to a preset segmentation distance, obtain the center point of each processing section, and construct a corresponding collection reference range based on the center point according to a preset radius.

[0158] Among them, the preset segmentation distance represents a fixed length pre-set along the railway according to the needs of railway safety monitoring, which can be used for segmentation processing and data analysis; the railway line represents an area within a certain range on both sides of the railway line, including railway tracks and surrounding areas that may affect railway operation; the processing segment represents each section along the railway divided according to the preset segmentation distance; the center point represents the section center of each processing segment, which can be used as a reference point for calculation and reference; the preset radius represents the length extending outward from the center point of the processing segment as the center, which can define the area for data collection; the collection reference range represents the area formed by extending outward with the center point as the reference and the preset radius as the extension length.

[0159] Specifically, the railway line is divided into multiple relatively independent sections, each of which is a processing section. The length of each section is a preset fixed distance (preset segment distance). The center point of each section is obtained, and the center point of each section is used as the reference and the preset radius is used as the extension length to extend outward to construct the corresponding collection reference range.

[0160] It should be noted that dividing the railway line according to preset segment distances can facilitate the monitoring and processing of each section of the railway line.

[0161] In some embodiments, step S572 (dividing each railway line into multiple processing segments according to a preset segmentation distance, obtaining the center point of each processing segment, and constructing a corresponding acquisition reference range based on the center point according to a preset radius) specifically includes steps S5721-S5724:

[0162] S5721. Take each processing section along the first railway as a first processing section, and take each center point as a first center point, and construct a corresponding first acquisition reference range according to a preset radius based on the first center point.

[0163] Among them, the first railway line represents the railway line segment that is currently focused on; the first railway line is divided into multiple smaller sections, each section is a first processing section; the first center point represents the section center point of each first processing section, which is usually used to determine the spatial position of the processing section and serve as a reference for subsequent operations; the first acquisition reference range represents the area range based on the first center point and delineated according to a preset radius, covering the area around the center point.

[0164] Specifically, the first railway line is divided into a plurality of first processing sections, and the center point of each first processing section is determined. Then, based on these center points, circular area ranges can be created according to preset radii to cover the area around the center point, and each circular area range is used as the first acquisition reference range.

[0165] See also Figure 3 , Figure 3 This is a schematic diagram of the division of processing sections along the first railway provided in an embodiment of the present application. The length of the first railway along the section is 1000 meters, which is divided into 4 processing sections, each of which is a first processing section, and each first processing section is 250 meters. The center point of each first processing section is the center of its section, such as the center point of the first processing section A is located at 125 meters. If the preset radius is 50 meters, the first acquisition reference range corresponding to the first processing section A is a circular area with a radius of 50 meters and a section center at 125 meters as the center.

[0166] S5722. Acquire other first processing segments within the first acquisition reference range corresponding to the first processing segment as first reference segments, and acquire other processing segments along the second railway corresponding to the first processing segment as second reference segments.

[0167] Among them, the first reference segment represents other processing segments except the current first processing segment within the first acquisition reference range corresponding to the current first processing segment. These processing segments are also located within the first railway line and are considered to be processing segments with similar characteristics to the current processing segment; the other second railway lines represent other railway lines except the first railway line, which are usually used for comparison or reference; the second reference segment represents the processing segment in the second railway line that is located within the current first acquisition reference range.

[0168] Specifically, within the first acquisition reference range, other processing segments along the same railway line as the current processing segment are found as the first reference segment. Meanwhile, processing segments along other railway lines that are within the same reference range are found as the second reference segment.

[0169] Example: Please continue to see Figure 3 , if the center point of the first processing segment A is located at 125 meters, and the first acquisition reference range with a preset radius of 50 meters contains a part of the first processing segment B, then the first processing segment B is the first reference segment. At the same time, along another railway line (the second railway line), there is a processing segment F that is also located within the first acquisition reference range of the first processing segment A, then the processing segment F is the second reference segment.

[0170] S5723. Obtain the number of first reference segments and second reference segments to obtain a first reference number. If the first reference number is greater than or equal to a first preset number, determine that the current first acquisition reference range is the first acquisition reference range for subsequent training.

[0171] Among them, the first reference number represents the sum of the numbers of all reference segments (first reference segment and second reference segment) within the current first acquisition reference range; the first preset number represents the minimum reference segment number threshold set by the system, which can be used to determine whether the number of reference segments within the current first acquisition reference range is sufficient.

[0172] Specifically, the first acquisition reference range corresponding to the current first processing segment is the current first acquisition reference range. The total number of all reference segments (including the first reference segment and the second reference segment) within the current first acquisition reference range is calculated. If the number reaches or exceeds the minimum number threshold (first preset number) set by the system, the acquisition reference range can be used for subsequent model training.

[0173] For example: if the first preset number set by the system is 3, and there are 2 first reference segments and 1 second reference segment in the first acquisition reference range corresponding to the current first processing segment, for a total of 3 reference segments, then the conditions are met and can be used for subsequent training.

[0174] S5724. If the first reference number is less than the first preset number, the preset radius is enlarged to construct a corresponding second acquisition reference range until the first reference number is greater than or equal to the first preset number and the first acquisition reference range participating in the training is obtained.

[0175] The second acquisition reference range represents a new reference range after the amplification process, and the range is expanded to include more processing segments until the number of reference segments within the current first acquisition reference range reaches a first preset number.

[0176] Specifically, if the number of reference segments within the current first acquisition reference range is insufficient, the preset radius will be increased and the first acquisition reference range will be expanded to increase the number of reference segments. The expanded first acquisition reference range is the second acquisition reference range. When the number of reference segments within the second acquisition reference range reaches the first preset number, the second acquisition reference range will be used as the first acquisition reference range participating in the training. The first acquisition reference range is the determined acquisition reference range.

[0177] For example, if there are only 2 reference segments in the current first acquisition reference range, which is not enough to meet the requirement of the first preset number 3, then the radius can be expanded from 50 meters to 75 meters, and the number of reference segments included can be recalculated. Assuming that the new acquisition reference range contains 4 reference segments, the preset requirement is met, and the range can be determined for training.

[0178] In this embodiment, when the system detects a dangerous tree area, it can gradually expand the collection range to obtain sufficient reference data to ensure the accuracy and robustness of the model.

[0179] S573, obtaining the dangerous tree processing information along all railways within the collection reference range, and performing training processing based on the dangerous tree processing information to update the preset similarity threshold.

[0180] The training process refers to optimizing and updating the thresholds and features of the model based on historical data and dangerous tree treatment information collected within a reference range.

[0181] It should be noted that updating the model or threshold based on new data or processing results can improve the accuracy of the monitoring system.

[0182] In some embodiments, step S573 (obtaining the dangerous tree processing information along all railways within the acquisition reference range, and performing training processing based on the dangerous tree processing information to update the preset similarity threshold) specifically includes steps S5731-S5734:

[0183] The dangerous tree processing information includes at least one of the existence of dangerous trees, the number of dangerous trees, and the absence of dangerous trees. The existence of dangerous trees means that a dangerous tree area is detected in a specific area; the number of dangerous trees means the number of dangerous tree areas detected within a certain acquisition reference range; and the absence of dangerous trees means that no dangerous tree area is detected in the specific area.

[0184] S5731. Obtain the dangerous tree processing information of all the first processing sections within the collection reference range. If the dangerous tree processing information is the existence of dangerous trees and the number of dangerous trees, calculate the first adjustment coefficient based on the number of dangerous trees. If the dangerous tree processing information is the absence of dangerous trees, obtain the preset second adjustment coefficient.

[0185] Among them, the first adjustment coefficient represents a coefficient calculated based on the number of dangerous trees when there are dangerous tree areas within a certain collection reference range and the number of dangerous tree areas is known, and is used to optimize the preset similarity threshold; the second adjustment coefficient represents a preset fixed coefficient used when there are no dangerous tree areas within a certain collection reference range.

[0186] Specifically, within a certain collection reference range, it is detected whether there is a dangerous tree area, and if so, the specific number of the dangerous tree area is determined. If there is a dangerous tree area and a specific number of dangerous tree areas in the dangerous tree processing information, a first adjustment coefficient is calculated according to the specific number, and if there is no dangerous tree area within the collection reference range, a preset second adjustment coefficient is directly applied.

[0187] For example, if three dangerous tree areas are detected within a certain collection reference range, then the dangerous tree processing information includes "dangerous trees exist" and "number of dangerous tree areas = 3", etc. Based on the three detected dangerous tree areas, the system can calculate an adjustment coefficient, which can be 0.8, indicating that the impact of dangerous trees in this range is greater and requires higher attention; if there is no dangerous tree area within the collection reference range, the system can directly use the default adjustment coefficient, such as 0.5.

[0188] S5732. Acquire the dangerous tree processing information of all second processing sections within the acquisition reference range, and obtain the coefficient offset weight based on the distance between the center point of each second processing section and the first center point.

[0189] Among them, the second processing section represents the processing sections corresponding to other railway lines except the first railway line; the coefficient offset weight represents the distance between the center point of each second processing section within the acquisition reference range and the reference center point (the current first center point), which is used to adjust the weight during calculation to reflect the influence of distance on the adjustment coefficient.

[0190] For example, if the center point of a second processing section is far from the first center point, the weight may be lower, such as 0.7, which means that the dangerous tree processing information corresponding to the second processing section has less impact on the overall situation.

[0191] S5733. Obtain the dangerous tree processing information of all the second processing sections within the collection reference range. If the dangerous tree processing information is the existence of dangerous trees and the number of dangerous trees, then obtain the third adjustment coefficient based on the number of dangerous trees and the coefficient offset weight calculation. If the dangerous tree processing information is that there are no dangerous trees, then obtain the preset fourth adjustment coefficient.

[0192] Among them, the third adjustment coefficient represents the existence of dangerous tree areas and the number of dangerous tree areas in the second processing section along other railways within the collection reference range, and is a coefficient calculated based on the number of dangerous tree areas and the coefficient offset weight; the fourth adjustment coefficient represents the absence of dangerous tree areas in the second processing section along other railways within the collection reference range, and a preset fixed coefficient is used.

[0193] It should be noted that if the number of dangerous trees in the second processing section within the collection reference range is large and is close to the current first processing section, the third adjustment coefficient will be higher, indicating that the potential impact of this area on the current first processing section is greater; for the case where there are no dangerous trees in the second processing section within the collection reference range, the system defaults to having a small impact on the target processing section, so this coefficient is usually a fixed value, which can be 0.3.

[0194] S5734. Update the preset similarity threshold based on the first adjustment coefficient, the second adjustment coefficient, the third adjustment coefficient, and the fourth adjustment coefficient.

[0195] Specifically, after comprehensively calculating the first adjustment coefficient, the second adjustment coefficient, the third adjustment coefficient, and the fourth adjustment coefficient, a new threshold value may be updated to determine the similarity between the dangerous tree area and the standard area.

[0196] The first adjustment coefficient, the second adjustment coefficient, the third adjustment coefficient and the fourth adjustment coefficient are added together to obtain an overall adjustment coefficient. After weighting the overall adjustment coefficient, the preset similarity threshold is updated. The preset similarity threshold after the update is calculated by the following formula:

[0197] in, is the preset similarity threshold after updating. is the first adjustment coefficient, is the number of dangerous trees in the first treatment section, is the first reverse adjustment coefficient, For the The coefficient offset weights of the second processing stage, For the The horizontal coordinate value of the second processing section, is the horizontal coordinate value of the first center point, For the The ordinate value of the second processing section, is the ordinate value of the first center point, is the second adjustment coefficient, is the third adjustment factor, For the The coefficient offset weights of the second processing stage, For the The number of dangerous trees in the second treatment section, is the upper limit value of the second processing section, is the second reverse adjustment coefficient, is the fourth adjustment coefficient, is a constant value, It is the preset similarity threshold before updating processing.

[0198] Specifically, the adjustment coefficients are further modified by distance or difference calculation to ensure that they accurately reflect the current state of dangerous trees. By accumulating the differences in these adjustment coefficients, the system's preset similarity threshold is updated. .

[0199] It should be noted that through the calculation and cumulative correction of a series of adjustment coefficients, dynamic optimization of the preset similarity threshold is achieved, ensuring that when dealing with the problem of dangerous trees along the railway, specific training can be carried out according to the plant properties in different geographical locations, and precise adjustments can be made based on real-time data to improve the reliability and safety of the system.

[0200] In the present embodiment, a method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology is provided, comprising: responding to a dangerous tree monitoring instruction, obtaining a calibration image collected by a binocular camera device, processing the calibration image, and obtaining monocular intrinsic parameter data and binocular position data; receiving a visible light image collected by the binocular camera device of the railway environment, and correcting the visible light image based on the monocular intrinsic parameter data and the binocular position data to obtain a binocular corrected image; receiving a SAR image collected by a synthetic aperture radar, fusing the binocular corrected image with the SAR image, and generating a fusion result image, wherein the fusion result image includes a left-camera fusion image and a right-camera fusion image; obtaining feature pixel points of the left-camera fusion image and the right-camera fusion image, obtaining feature matching pairs based on the feature pixel points, and generating a binocular disparity map based on each feature matching pair; converting the binocular disparity map into a depth map according to the monocular intrinsic parameter data and the binocular position data, and determining a dangerous tree area along the railway based on the depth map. The present invention fuses the visible light image acquired by the binocular camera device with the SAR image acquired by the synthetic aperture radar through the optical SAR fusion technology, obtains high-quality image information that is not affected by the environment, and improves the quality of the video stream. In addition, the image fused by the optical SAR is used as the image to be detected, and the ranging accuracy is improved through parallax calculation and depth estimation.

[0201] like Figure 4 FIG. 1 is a schematic diagram of a structure of a railway dangerous tree monitoring system based on optical SAR fusion and binocular depth estimation technology provided by an embodiment of the present invention. The railway dangerous tree monitoring system based on optical SAR fusion and binocular depth estimation technology includes:

[0202] The data acquisition module is used to respond to the dangerous tree monitoring instruction, obtain the calibration image collected by the binocular camera device, process the calibration image, and obtain the monocular internal reference data and binocular position data.

[0203] The image correction module is used to receive the visible light image collected by the binocular camera device of the environment along the railway, and correct the visible light image based on the monocular internal reference data and the binocular position data to obtain a binocular corrected image.

[0204] The image fusion module is used to receive the SAR image collected by the synthetic aperture radar, fuse the binocular correction image with the SAR image, and generate a fusion result image, wherein the fusion result image includes a left-photographed fusion image and a right-photographed fusion image.

[0205] The feature matching module is used to obtain feature pixel points of the left-camera fusion image and the right-camera fusion image, obtain feature matching pairs based on the feature pixel points, and generate a binocular disparity map based on each feature matching pair.

[0206] The area determination module is used to convert the binocular disparity map into a depth map according to the monocular internal reference data and the binocular position data, and determine the dangerous tree area along the railway based on the depth map.

[0207] It should be noted that the above-mentioned dangerous tree monitoring system along railways based on optical SAR fusion and binocular depth estimation technology can execute the dangerous tree monitoring method along railways based on optical SAR fusion and binocular depth estimation technology provided in the embodiment of the present invention. For technical details and beneficial effects that are not fully described in the embodiment of the dangerous tree monitoring system along railways based on optical SAR fusion and binocular depth estimation technology, please refer to the dangerous tree monitoring method along railways based on optical SAR fusion and binocular depth estimation technology provided in the embodiment of the present invention.

[0208] Through the description of the above embodiments, it can be clearly understood by those skilled in the art that each embodiment can be implemented by means of software plus a general hardware platform, or by hardware. It can be understood by those skilled in the art that all or part of the processes in the above embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0209] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may also be combined, the steps may be implemented in any order, and there are many other changes in different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology, characterized in that: include: In response to the dangerous tree monitoring instruction, a calibration image collected by a binocular camera device is obtained, and the calibration image is processed to obtain monocular internal reference data and binocular position data; Receiving a visible light image captured by the binocular camera device of the environment along the railway, and correcting the visible light image based on the monocular intrinsic reference data and the binocular position data to obtain a binocular corrected image; Receiving a SAR image collected by a synthetic aperture radar, fusing the binocular correction image with the SAR image, and generating a fusion result image, wherein the fusion result image includes a left-photographed fusion image and a right-photographed fusion image; Acquire characteristic pixel points of the left-photographed fusion image and the right-photographed fusion image, acquire characteristic matching pairs based on the characteristic pixel points, and generate a binocular disparity map based on each characteristic matching pair; Converting the binocular disparity map into a depth map according to the monocular internal reference data and the binocular position data, and determining a dangerous tree area along the railway based on the depth map; In response to the dangerous tree monitoring instruction, a calibration image collected by a binocular camera device is obtained, and the calibration image is processed to obtain monocular internal reference data and binocular position data, including: Responding to the dangerous tree monitoring command, obtaining the calibration image of the target calibration object captured by the binocular camera device; Calling a corner point detection algorithm to process the calibration image to obtain corner point position data, wherein the corner point position data includes an actual pixel position of each corner point in the calibration image; Establishing a world coordinate system with an arbitrary corner point as the origin, obtaining the calibration size of the target calibration object, and locating the corner point world coordinates corresponding to each corner point in the world coordinate system based on the calibration size; Inputting the corner point world coordinates into a camera imaging model, and outputting a predicted pixel position based on the camera imaging model, wherein the camera imaging model includes camera intrinsic parameter data; The predicted pixel position and the actual pixel position of each corner point are respectively input into a preset optimization algorithm, a reprojection error is obtained based on the preset optimization algorithm, and the camera intrinsic parameter data with the minimum reprojection error is determined as the monocular intrinsic parameter data; The binocular position data is acquired based on the monocular intrinsic parameter data and the corner point position data.

2. The method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology according to claim 1 is characterized in that: The calibration image includes a left calibration image and a right calibration image; Acquiring binocular position data based on the monocular intrinsic reference data and the corner point position data includes: Performing corner point matching according to the corner point position data of the left calibration image and the corner point position data of the right calibration image to obtain a plurality of corner point matching pairs; Calling a preset matrix acquisition rule based on a plurality of corner point matching pairs, and constructing a constraint matrix according to the preset matrix acquisition rule; The constraint matrix is ​​converted into a position matrix based on the monocular internal parameter data, and the position matrix is ​​decomposed to obtain binocular position data.

3. The method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology according to claim 1 is characterized in that: The monocular internal reference data includes left-camera internal reference data and right-camera internal reference data; Receiving a visible light image collected by the binocular camera device of the environment along the railway, and correcting the visible light image based on the monocular intrinsic reference data and the binocular position data to obtain a binocular corrected image, including: In response to the calibration completion instruction, receiving a visible light image of the environment along the railway collected by the binocular camera device, wherein the visible light image includes a left-camera image and a right-camera image; Performing distortion correction on the left-camera image based on the left-camera internal reference data to obtain a left-camera distortion-corrected image; Performing distortion correction on the right camera image based on the right camera internal reference data to obtain a right camera distortion corrected image; The left-photographed distortion-corrected image and the right-photographed distortion-corrected image are aligned based on the binocular position data to obtain a binocular correction image.

4. The method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology according to claim 3 is characterized in that: Receiving a SAR image collected by a synthetic aperture radar, fusing the binocular correction image with the SAR image, and generating a fusion result graph, including: Inputting the binocular correction image and the SAR image into a feature extractor, and outputting a visible light modal feature map and a SAR modal feature map based on the feature extractor; Calling a preset fusion rule to process the visible light modal feature map and the SAR modal feature map to obtain a preliminary fusion feature map; Inputting the preliminary fusion feature map into a decoder, and generating a preliminary fusion map based on the decoder; The preliminary fusion graph and the preset fusion graph are input into the discriminator, adversarial training is performed based on the discriminator, and a fusion result graph is obtained.

5. The method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology according to claim 1 is characterized in that: The characteristic pixel points include left-camera characteristic pixel points and right-camera characteristic pixel points; Acquiring characteristic pixel points of the left-camera fusion image and the right-camera fusion image, acquiring characteristic matching pairs based on the characteristic pixel points, and generating a binocular disparity map based on each characteristic matching pair, including: Performing feature extraction on the left-camera fusion image to obtain a plurality of left-camera visual key points, and determining the plurality of left-camera visual key points as the left-camera feature pixel points; Performing feature extraction on the right camera fusion image to obtain a plurality of right camera visual key points, and determining the plurality of right camera visual key points as the right camera feature pixel points; Calling a preset matching rule to perform feature matching on the left-camera feature pixel points and the right-camera feature pixel points to obtain a feature matching pair; Calling a preset difference acquisition rule, calculating a horizontal distance difference of a current feature matching pair according to the preset difference acquisition rule, and determining the horizontal distance difference as a disparity value corresponding to the current feature matching pair; The above step of obtaining the disparity value is repeated until the disparity values ​​corresponding to all feature matching groups are obtained, and the disparity values ​​corresponding to all feature matching groups are combined to generate a binocular disparity map.

6. The method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology according to claim 1 is characterized in that: Converting the binocular disparity map into a depth map according to the monocular intrinsic reference data and the binocular position data includes: Acquire a baseline distance of the binocular camera device based on the binocular position data, wherein the baseline distance is a distance between optical centers of a left camera and a right camera; Determine the focal length of the binocular camera device based on the monocular internal reference data, and convert the current disparity value into a current depth value according to the baseline distance and the focal length; Repeat the above step of converting the current disparity value into the current depth value until the depth values ​​corresponding to all disparity values ​​are obtained, and generate a depth map based on the multiple depth values.

7. The method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology according to claim 6 is characterized in that: Converting the current disparity value into the current depth value according to the baseline distance and the focal length includes: The current depth value is calculated using the following formula: in, is the focal length, is the baseline distance, is the parallax value.

8. The method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology according to claim 6 is characterized in that: Determine the dangerous tree area along the railway based on the depth map, including: Acquire multiple depth values ​​of the depth map in real time, determine whether each depth value is within a preset depth range, and retain the depth value when the depth value is within the preset depth range to obtain a depth value set; Analyze each depth value in the depth value set to obtain a depth attribute, mark a plurality of independent cluster regions based on the depth attribute, and obtain a cluster region feature of each independent cluster region; Calling a feature mapping library to obtain preset regional features corresponding to dangerous trees along the railway, comparing the clustering regional features with the preset regional features, and obtaining feature similarity between each independent clustering region and the preset region; When the feature similarity is greater than or equal to a preset similarity threshold, the corresponding independent clustering area is determined as a dangerous tree area along the railway.

9. The method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology according to claim 8 is characterized in that: When the feature similarity is greater than or equal to a preset similarity threshold, the corresponding independent clustering area is determined as a dangerous tree area along the railway, including: Obtaining first positioning information corresponding to the dangerous tree area along the railway, generating a corresponding processing work order and sending it to the maintenance end, and receiving the dangerous tree processing information corresponding to the dangerous tree area along the railway fed back by the maintenance end; Divide each railway line into multiple processing sections according to a preset segmentation distance, obtain the center point of each processing section, and construct a corresponding collection reference range based on the center point according to a preset radius; The dangerous tree processing information along all railways within the collection reference range is obtained, and a training process is performed based on the dangerous tree processing information to update a preset similarity threshold.

10. The method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology according to claim 9, characterized in that: The method of dividing each railway line into a plurality of processing sections according to a preset segmentation distance, obtaining a center point of each processing section, and constructing a corresponding acquisition reference range according to a preset radius based on the center point includes: Taking each processing section along the first railway as a first processing section, and taking each center point as a first center point, and constructing a corresponding first acquisition reference range according to a preset radius based on the first center point; Acquire other first processed segments within the first acquisition reference range corresponding to the first processed segment as first reference segments, and acquire other processed segments along the second railway corresponding to the first processed segment as second reference segments; Acquire the number of the first reference segment and the second reference segment to obtain a first reference number, and if the first reference number is greater than or equal to a first preset number, determine the current first acquisition reference range as the first acquisition reference range for subsequent training; If the first reference number is less than the first preset number, the preset radius is enlarged to construct a corresponding second acquisition reference range until the first reference number is greater than or equal to the first preset number and the first acquisition reference range participating in the training is obtained.

11. The method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology according to claim 10, characterized in that: The acquiring and collecting dangerous tree processing information along all railways within the reference range, and performing training processing based on the dangerous tree processing information to update the preset similarity threshold, include: The dangerous tree processing information includes at least one of the existence of dangerous trees, the number of dangerous trees, and the absence of dangerous trees; Acquire the dangerous tree processing information of all the first processing sections within the collection reference range, if the dangerous tree processing information is the existence of dangerous trees and the number of dangerous trees, calculate the first adjustment coefficient based on the number of dangerous trees, if the dangerous tree processing information is the absence of dangerous trees, calculate the preset second adjustment coefficient; Acquire the dangerous tree processing information of all second processing sections within the acquisition reference range, and obtain the coefficient offset weight based on the distance between the center point of each second processing section and the first center point; Acquire the dangerous tree processing information of all the second processing sections within the collection reference range. If the dangerous tree processing information is the existence of dangerous trees and the number of dangerous trees, calculate the third adjustment coefficient based on the number of dangerous trees and the coefficient offset weight. If the dangerous tree processing information is the absence of dangerous trees, obtain the preset fourth adjustment coefficient. The preset similarity threshold is updated based on the first adjustment coefficient, the second adjustment coefficient, the third adjustment coefficient and the fourth adjustment coefficient.

12. The method for monitoring dangerous trees along railways based on optical SAR fusion and binocular depth estimation technology according to claim 11, characterized in that: The updating process of the preset similarity threshold based on the first adjustment coefficient, the second adjustment coefficient, the third adjustment coefficient and the fourth adjustment coefficient includes: The first adjustment coefficient, the second adjustment coefficient, the third adjustment coefficient and the fourth adjustment coefficient are added together to obtain an overall adjustment coefficient. After weighting the overall adjustment coefficient, the preset similarity threshold is updated. The preset similarity threshold after the update is calculated by the following formula: in, is the preset similarity threshold after updating. is the first adjustment coefficient, is the number of dangerous trees in the first treatment section, is the first reverse adjustment coefficient, For the The coefficient offset weights of the second processing stage, For the The horizontal coordinate value of the second processing section, is the horizontal coordinate value of the first center point, For the The ordinate value of the second processing section, is the ordinate value of the first center point, is the second adjustment coefficient, is the third adjustment factor, For the The coefficient offset weights of the second processing stage, For the The number of dangerous trees in the second treatment section, is the upper limit value of the second processing section, is the second reverse adjustment coefficient, is the fourth adjustment coefficient, is a constant value, It is the preset similarity threshold before updating processing.

13. A railway dangerous tree monitoring system based on optical SAR fusion and binocular depth estimation technology, characterized in that: include: A data acquisition module is used to respond to the dangerous tree monitoring instruction, acquire the calibration image collected by the binocular camera device, process the calibration image, and acquire the monocular internal reference data and binocular position data; An image correction module is used to receive the visible light image collected by the binocular camera device of the environment along the railway, and correct the visible light image based on the monocular internal reference data and the binocular position data to obtain a binocular correction image; An image fusion module is used to receive a SAR image collected by a synthetic aperture radar, fuse the binocular correction image with the SAR image, and generate a fusion result image, wherein the fusion result image includes a left-photographed fusion image and a right-photographed fusion image; A feature matching module, used to obtain feature pixel points of the left-camera fusion image and the right-camera fusion image, obtain feature matching pairs based on the feature pixel points, and generate a binocular disparity map based on each feature matching pair; An area determination module, configured to convert the binocular disparity map into a depth map according to the monocular internal reference data and the binocular position data, and determine a dangerous tree area along the railway based on the depth map; In response to the dangerous tree monitoring instruction, a calibration image collected by a binocular camera device is obtained, and the calibration image is processed to obtain monocular internal reference data and binocular position data, including: Responding to the dangerous tree monitoring command, obtaining the calibration image of the target calibration object captured by the binocular camera device; Calling a corner point detection algorithm to process the calibration image to obtain corner point position data, wherein the corner point position data includes an actual pixel position of each corner point in the calibration image; Establishing a world coordinate system with an arbitrary corner point as the origin, obtaining the calibration size of the target calibration object, and locating the corner point world coordinates corresponding to each corner point in the world coordinate system based on the calibration size; Inputting the corner point world coordinates into a camera imaging model, and outputting a predicted pixel position based on the camera imaging model, wherein the camera imaging model includes camera intrinsic parameter data; The predicted pixel position and the actual pixel position of each corner point are respectively input into a preset optimization algorithm, a reprojection error is obtained based on the preset optimization algorithm, and the camera intrinsic parameter data with the minimum reprojection error is determined as the monocular intrinsic parameter data; The binocular position data is acquired based on the monocular intrinsic parameter data and the corner point position data.