An effective feature point output method and device, electronic equipment and computer program product

Abstract

The application belongs to the technical field of machine vision, and particularly relates to an effective feature point output method and device, electronic equipment and computer program product. The method comprises the following steps: firstly, acquiring a current frame image collected by a binocular camera and corresponding ambient light information, performing light correction on the current frame image according to the ambient light information to obtain a corrected image; secondly, performing feature point extraction on the corrected image, and in the case that the number of extracted feature points is lower than a first quantity threshold, fusing historical static feature points cached with feature points extracted in the current frame to form a supplementary feature point set; finally, screening the supplementary feature point set based on binocular geometric constraints, eliminating mis-matching feature points, and outputting an effective feature point set, so as to solve the technical problems of SLAM system tracking interruption and positioning drift in a weak texture-light mutation concurrent scene, and significantly improve the spatial consistency and matching reliability of the output feature set.

Description

Technical Field

[0001] This application belongs to the field of machine vision technology, and in particular relates to a method, apparatus, electronic device and computer program product for outputting effective feature points. Background Technology

[0002] With the rapid development of intelligent manufacturing and smart homes, binocular SLAM technology, due to its moderate cost and high positioning accuracy, has become a core technology for autonomous navigation and environmental perception in devices such as home service robots, mobile robots, and autonomous vehicles. Binocular SLAM systems simultaneously acquire images from both eyes, extract and match feature points, and calculate feature point depth using binocular geometric constraints, ultimately achieving device localization and map construction. In practical applications, mainstream SLAM systems such as ORB-SLAM2 / 3 employ feature point matching and optimization algorithms, with front-end feature extraction, back-end optimization, and loop closure detection modules working collaboratively to complete environmental perception, localization, and mapping tasks. In existing technologies, for situations with poor image quality, single processing methods such as increasing the number of corner detection iterations, expanding the feature extraction range, or performing histogram equalization are typically used to improve feature extraction results.

[0003] However, in complex scenarios with both weak textures and sudden changes in illumination, existing technologies cannot simultaneously guarantee the quality and quantity of feature points, leading to a decrease in the reliability of feature matching and affecting the stable operation of the SLAM system. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and computer program product for outputting effective feature points, aiming to solve the technical problems of SLAM positioning drift and tracking interruption in the combined scenes of weak texture and abrupt changes in illumination mentioned above.

[0005] Firstly, this application provides a method for outputting effective feature points, including: Step S1: Obtain the current frame image captured by the binocular camera and the corresponding ambient lighting information, and perform lighting correction on the current frame image according to the ambient lighting information to obtain a corrected image; Step S2: Extract feature points from the corrected image. If the number of extracted feature points is lower than a first threshold, merge the cached historical static feature points with the feature points extracted in the current frame to form a supplementary feature point set. Step S3: Based on the binocular geometric constraints, the supplementary feature point set is filtered to remove mismatched feature points and output the effective feature point set.

[0006] The technical effect of this application is as follows: This application constructs a three-level closed-loop enhancement path of illumination adaptive correction, condition-triggered feature reuse, and binocular geometry unified verification. The three are sequentially dependent and have closed-loop feedback, which jointly solves the technical problems of tracking interruption and positioning drift in SLAM system under concurrent weak texture-illumination change scenarios, and significantly improves the spatial consistency and matching reliability of the output feature set.

[0007] Preferably, step S1 further includes: Step S11: Calculate the grayscale mean difference rate between the current frame image and the previous frame image. If the grayscale mean difference rate exceeds the first change rate threshold, it is determined to be a sudden change in illumination. Step S12: When a sudden change in illumination is determined, nonlinear grayscale transformation and grayscale distribution equalization processing are sequentially performed on the current frame image; The technical advantage of this embodiment is that by detecting first and then correcting, correction is triggered only when there is a sudden change in illumination, thus reducing invalid calculations.

[0008] Preferably, in step S12, the target transformation parameters of the nonlinear grayscale transformation are dynamically adjusted according to the ambient lighting information, including: When the ambient light intensity is higher than the second intensity threshold, the transformation parameter for suppressing the highlight region is used as the target transformation parameter for the nonlinear grayscale transformation. When the ambient light intensity is below the third intensity threshold, the transformation parameters that enhance dark details are used as the target transformation parameters for the nonlinear grayscale transformation. The second intensity threshold is used to distinguish strong light scenes, the third intensity threshold is used to distinguish weak light scenes, and the third intensity threshold is less than the second intensity threshold.

[0009] The technical effect of this embodiment is that it dynamically adjusts the correction parameters according to the ambient light intensity to achieve adaptive correction that suppresses strong light and enhances weak light.

[0010] Preferably, step S2, "extracting feature points from the corrected image, and when the number of extracted feature points is lower than a first threshold, fusing cached historical static feature points with the feature points extracted in the current frame to form a supplementary feature point set," specifically includes: Step S21: Extract feature points from the corrected image, count the number of extracted feature points, and compare it with the first number threshold. Step S22: If the number of extracted feature points is lower than the first number threshold, select feature points with inter-frame displacement less than the first displacement threshold from the cached historical keyframes and use them as static feature points. Step S23: Project the selected static feature points onto the coordinate system of the current frame image and merge them with the feature points extracted from the current frame to form a supplementary feature point set; The first quantity threshold is used to determine whether the current scene is a weak texture scene, and the first displacement threshold is used to determine whether the feature point is a static feature point.

[0011] The technical advantages of this embodiment are: it adopts a conditional triggering mechanism to initiate reuse only when there are insufficient feature points; and it filters static feature points through displacement thresholds to ensure the stability of reused feature points.

[0012] Preferably, step S3 further includes: Step S31: Based on the internal parameters of the stereo camera, for each feature point in the supplementary feature point set, the depth value of the feature point is calculated by triangulation using its pixel coordinates in the left and right images of the current frame of the stereo camera, to obtain the first depth value. Step S32: Remove feature points whose first depth value exceeds the range of the preset minimum depth threshold and the maximum depth threshold from the supplementary feature point set to obtain the first filter point set; Step S33: For each feature point in the first set of filtering points, calculate the depth value of the feature point in the left eye image as the left eye depth value and the depth value of the feature point in the right eye image as the right eye depth value. Calculate the difference rate between the left eye depth value and the right eye depth value. Remove feature points whose difference rate is greater than a preset first difference threshold from the first set of filtering points to obtain the second set of filtering points. Step S34: Use the random sampling consensus algorithm to optimize the reprojection error of the feature points in the second set of selected points, remove feature points whose reprojection error is greater than the preset second displacement threshold, and output the effective feature point set; The minimum depth threshold and the maximum depth threshold are used to limit the effective depth range, the first difference threshold is used to determine whether the left eye depth value and the right eye depth value are consistent, and the second displacement threshold is used to determine whether the reprojection error is acceptable.

[0013] The technical effect of this embodiment is that it significantly improves the accuracy of feature point matching through a triple progressive verification of depth range, left and right eye consistency, and reprojection error.

[0014] Preferably, after step S3 filters the supplementary feature point set based on binocular geometric constraints, removes mismatched feature points, and outputs the effective feature point set, the method further includes: Step S4: Store the feature points in the effective feature point set that satisfy the static condition into the key frame cache library to obtain the stored static feature points; wherein, the static condition includes that the inter-frame displacement of the feature point in multiple consecutive frames is less than a third displacement threshold.

[0015] Step S5: For the corrected image corresponding to the subsequent frame image, repeat the step of extracting feature points from the corrected image. If the number of extracted feature points is lower than the first threshold, during the process of fusing the cached historical static feature points with the feature points extracted in the current frame, read the stored static feature points from the key frame cache library as the cached historical static feature points, and use them to fuse with the feature points extracted from the corrected image corresponding to the subsequent frame image.

[0016] The technical effect of this embodiment is that by feeding back the verified high-quality static feature points to the cache, a closed-loop mechanism for feature reuse is established, which enables the static feature point library in the cache to be continuously optimized; by clarifying the reading steps when reusing subsequent frames, the integrity and feasibility of the reuse mechanism are ensured.

[0017] Secondly, this application also provides an output device for effective feature points, comprising: An adaptive illumination correction module is used to acquire the current frame image captured by the binocular camera and the corresponding ambient illumination information, and to perform illumination correction on the current frame image according to the ambient illumination information to obtain a corrected image. The condition-triggered feature extraction and cross-frame reuse module is used to extract feature points from the corrected image. When the number of extracted feature points is lower than a first threshold, a cross-frame static feature point reuse mechanism is triggered to fuse cached historical static feature points with the feature points extracted in the current frame to form a supplementary feature point set. The binocular geometric constraint verification module is used to filter the supplementary feature point set based on binocular geometric constraints, remove mismatched feature points, and output the valid feature point set.

[0018] Thirdly, this application also provides an electronic device, including a memory, a controller, and a computer program stored in the memory and executable on the controller, wherein the controller executes the computer program to implement the method described in the first aspect above.

[0019] Fourthly, this application also provides a computer program product, which stores a computer program that, when executed by a controller, implements the method described in the first aspect above.

[0020] It is understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the main process of an effective feature point output method provided in this application; Figure 2 This is a schematic flowchart of an embodiment of the effective feature point output method provided in this application; Figure 3 This is a schematic flowchart of another embodiment of the effective feature point output method provided in this application; Figure 4 This is a schematic flowchart of another embodiment of an effective feature point output method provided in this application; Figure 5 This is a schematic flowchart of another embodiment of an effective feature point output method provided in this application; Figure 6 This is a schematic diagram of the structure of an effective feature point output device provided in an embodiment of this application.

[0023] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application; Detailed Implementation In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0024] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0025] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0026] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0027] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0028] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0029] Understandably, the applicant of this invention has discovered that in complex scenarios with both weak texture and abrupt changes in illumination, binocular SLAM systems often experience tracking interruptions or positioning drift due to low image signal-to-noise ratio, insufficient number of feature points, and poor matching robustness. In existing technologies, using only histogram equalization correction easily leads to overexposure, while simply expanding the feature detection range cannot solve the gradient annihilation problem under illumination distortion. Simultaneously, static feature points lack a cross-frame consistency management mechanism, resulting in insufficient continuity of feature supplementation in weak texture regions. Furthermore, mismatch elimination does not incorporate binocular geometric priors for joint depth-scale constraints, limiting the reliability of the selection results.

[0030] Based on the above issues, see Figure 1 As shown, this application provides a method for outputting effective feature points, which includes the following steps S1 to S3: Step S1: Acquire the current frame image and corresponding ambient lighting information captured by the binocular camera, and perform lighting correction on the current frame image based on the ambient lighting information to obtain the corrected image; It should be noted that the current frame image includes the left eye image and the right eye image. As an example, the two are synchronized via the PTP protocol, and the time synchronization error is less than 10 ms. Ambient lighting information can be acquired by a built-in light sensor, with a range of 0–1000 lux and an accuracy of ±10 lux. Illumination correction is used to improve the stability of image grayscale distribution and the discriminability of local gradients, ensuring that weakly textured areas (such as white walls or monochrome floors) retain extractable structural responses even under abrupt changes in illumination. For example, this application may dynamically select Gamma correction parameters based on ambient lighting intensity and perform histogram equalization on the Gamma-corrected image. In a specific implementation, this embodiment, as one approach, may determine whether to initiate the fusion correction process based on a combination of ambient light intensity and the inter-frame grayscale mean difference rate. Furthermore, as another specific implementation, this embodiment may also involve a collaborative processing of nonlinear grayscale mapping and global contrast adjustment of the image based on ambient light information. Based on any of the above specific implementation methods, this application can obtain a stable, high signal-to-noise ratio corrected image, providing reliable input for subsequent feature extraction.

[0031] For example, this application can be a binocular camera that simultaneously acquires the left and right eye images of the current frame at a resolution of 1920×1080 and a frame rate of 30 fps; the light sensor reads the current ambient light intensity as 320 lux in real time; the system determines that this intensity is within the normal lighting range (100–500 lux), triggers a light Gamma correction (Gamma=1.0) and superimposes histogram equalization processing; after processing, the overall contrast of the image is improved by 22%, the gray standard deviation of the central region is improved by 35%, and the edge response value is improved by ≥40%, which meets the basic requirements of the ORB feature detector for gradient intensity.

[0032] Step S2: Extract feature points from the corrected image. If the number of extracted feature points is lower than the first threshold, trigger the cross-frame static feature point reuse mechanism to fuse the cached historical static feature points with the feature points extracted in the current frame to form a supplementary feature point set. It should be noted that the first quantity threshold is used to determine whether the current scene is a weak texture scene. For example, its value can be set to 50. Historical static feature points can be selected from the previous 5-10 keyframes, which meet the requirements that the inter-frame displacement is less than the first displacement threshold and the response value is not lower than the preset response threshold. The first displacement threshold is used to determine whether the feature point is a static feature point. For example, its value can be set to 2 pixels. The response threshold is set to 100. The cross-frame static feature point reuse mechanism is only activated when the number of feature points is insufficient to avoid redundant calculations and noise introduction.

[0033] In a specific implementation, one approach of this embodiment is to perform a standard ORB feature detection algorithm on the corrected image, count the number of effective feature points, and compare it with a first quantity threshold. Another approach is to determine whether the weak texture condition is met based on a combination of feature point response values ​​and spatial distribution density. Furthermore, another approach is to determine whether a multiplexing mechanism is triggered based on the low-frequency energy proportion of the image gradient magnitude histogram. Based on any of the above implementation methods, this application can obtain a supplementary feature point set containing feature points of the current frame and static feature points after projection, ensuring that the total number of effective feature points in a single frame is not less than the first quantity threshold.

[0034] As an example, this step performs ORB feature extraction on the corrected image processed by S1, detecting a total of 37 valid feature points, which is less than the first quantity threshold of 50. The system reads 64 static feature points from the cached keyframe library that meet the requirements of inter-frame displacement ≤ 2 pixels and response value ≥ 100 from the previous 8 frames. Using the intrinsic parameter matrix of the stereo camera and the pose estimation results, the 64 static feature points are projected onto the coordinate system of the current frame image and merged with the 37 feature points of the current frame to form a supplementary feature point set containing 101 feature points. All feature points retain their original descriptors for subsequent matching.

[0035] Step S3: Based on the binocular geometric constraints, the supplementary feature point set is filtered to remove mismatched feature points and output the effective feature point set; It is understood that the binocular geometric constraints in the embodiments of this application may include, but are not limited to, intrinsic parameters such as the baseline length, focal length, and principal point coordinates of the binocular camera; mismatched feature points may refer to feature points that do not meet the three-dimensional spatial consistency or exceed the reprojection error limit; the screening process does not distinguish between the extracted points and reused points in the current frame, and uniformly performs depth calculation, depth range verification, left and right eye depth consistency verification, and reprojection error optimization. In a specific implementation, one approach of this embodiment is to calculate the corresponding depth value of each feature point in the left and right eye images based on triangulation, and then perform preliminary screening based on preset minimum and maximum depth thresholds. Another approach is to calculate the difference rate between the left and right eye depth values ​​separately, and remove feature points with a difference rate greater than a preset first difference threshold. Furthermore, this embodiment can also employ a random sampling consensus algorithm to model reprojection errors and remove outliers from the remaining feature points. Based on any of the above implementation methods, this embodiment can obtain an effective feature point set that satisfies binocular geometric consistency, ensuring the reliability and robustness of the SLAM backend input data.

[0036] As an example, steps S1 to S3 can be encapsulated as a front-end enhancement module. This front-end enhancement module is independent of the back-end processing flow of the SLAM system. It takes the effective feature point set output from step S3 as input and passes it to the feature matching module of the SLAM system. The back-end optimizer and loop closure detection algorithm of the SLAM system receive the output of the feature matching module and perform pose optimization and loop closure correction. The core code of the back-end optimizer and loop closure detection algorithm remains unchanged. This effectively solves the problem that existing enhancement schemes require modification of the core algorithm of the SLAM system, resulting in poor compatibility and high integration costs. This example adopts a front-end modular enhancement approach, encapsulating the enhancement steps as independent modules without intruding on the core code of the SLAM system, achieving high compatibility and low-cost robustness improvement.

[0037] For example, in this embodiment, all 101 feature points in the supplementary feature point set are triangulated based on the intrinsic parameters of the binocular camera (baseline length 12 cm, focal length 800 pixels) to obtain the depth value of each point; feature points with depth values ​​less than 0.5 m or greater than 5.0 m are removed, leaving 92; the depth difference rate between the left and right eyes is further calculated, and feature points with a difference rate > 10% are removed, leaving 86; finally, the RANSAC algorithm is used to fit the reprojection error of the 86 feature points, and feature points with a reprojection error > 2 pixels are removed, finally outputting 79 valid feature points; the feature point matching accuracy in this set is ≥ 90%, and it can be directly input into the front-end module of the ORB-SLAM2 system for subsequent feature matching and pose estimation.

[0038] The technical effect of this application embodiment is as follows: by acquiring the current frame image and corresponding ambient lighting information captured by the binocular camera, the current frame image is illuminated according to the ambient lighting information to obtain a calibrated image; on this basis, feature points are extracted from the calibrated image, and when the number of extracted feature points is lower than a first threshold, a cross-frame static feature point reuse mechanism is triggered to fuse the cached historical static feature points with the feature points extracted from the current frame to form a supplementary feature point set; then, based on binocular geometric constraints, the supplementary feature point set is filtered to remove mismatched feature points and output a valid feature point set. This scheme constructs a three-level closed-loop enhancement path: illumination adaptive correction, conditional triggering feature reuse, and unified binocular geometric verification. Step S1 improves image quality, ensuring that weak texture regions retain extractable gradients even under sudden illumination changes. Step S2 dynamically introduces high-confidence static feature points based on a first quantity threshold, ensuring the continuity of feature supply. Finally, step S3 applies strict binocular geometric constraints to all feature points (including reused points), jointly eliminating mismatches from both depth scale and reprojection dimensions, significantly improving the spatial consistency and matching reliability of the output feature set. These three steps, in a sequential and closed-loop feedback manner, jointly address the technical problems of tracking interruption and localization drift in SLAM systems under concurrent weak texture and illumination change scenarios.

[0039] refer to Figure 2 In one embodiment, step S1 may further include S11 to S12: Step S11: Calculate the grayscale mean difference rate between the current frame image and the previous frame image. If the grayscale mean difference rate exceeds the first change rate threshold, it is determined to be a sudden change in illumination. In this embodiment, the grayscale mean difference rate can be the ratio of the absolute value of the difference between the grayscale mean of the current frame image and the grayscale mean of the previous frame image to the grayscale mean of the previous frame image. The grayscale mean is calculated using a weighted average method, for example, the pixel weight in the central region is 60%, and the pixel weight in the edge region is 40%, to suppress the interference of image edge shadows or local occlusion on the judgment of illumination changes. For another example, a first change rate threshold of 30%, ranging from 25% to 35%, is used to determine whether a significant illumination change occurs between consecutive frames. This threshold can be used to accurately identify the time nodes where the illumination state changes abruptly in the image sequence, providing a trigger basis for subsequently initiating the composite correction process only when necessary, avoiding the introduction of noise or artifacts due to redundant correction in stable illumination scenarios.

[0040] In a specific implementation, one approach of this embodiment is to determine whether there is a sudden change in illumination between the current frame and the previous frame by comparing weighted gray-level averages. Another approach is to determine whether there is a step change in illumination state by analyzing the gray-level statistical change trend dominated by the central region. Furthermore, another approach is to determine the occurrence of a sudden change in illumination by combining the slope of the gray-level average change across multiple frames within a time sliding window with the difference rate of a single frame. This embodiment can obtain reliable detection results for sudden illumination events based on any of the above methods, thereby ensuring the accuracy of the conditional triggering of the S12 correction operation.

[0041] As an example, in this embodiment, after the binocular camera acquires the left-eye image of the current frame, the image and illumination data acquisition module synchronously reads the left-eye image of the previous frame; the average grayscale value of the central region (a rectangular area occupying 60% of the total image area) of the previous frame image is calculated to be 85, the average grayscale value of the edge region is 62, and the weighted average grayscale value is 85×60%+62×40%=75.8; the weighted average grayscale value of the corresponding region in the current frame is 112; then the grayscale average difference rate is |112|. 75.8| / 75.8=0.477, or 47.7%, which exceeds the first change rate threshold of 30%. The system determines this as a sudden change in illumination and triggers step S12.

[0042] Step S12: When a sudden change in illumination is determined, nonlinear grayscale transformation and grayscale distribution equalization processing are sequentially performed on the current frame image; In this embodiment, nonlinear grayscale transformation refers to remapping the grayscale values ​​of each pixel in the image using a nonlinear mapping function to correct the overall brightness imbalance caused by sudden changes in illumination. Grayscale distribution equalization processing refers to improving local contrast and enhancing texture discernibility by adjusting the distribution shape of the image's grayscale histogram. In this embodiment, this technical feature is used to improve image quality in stages after confirming that a sudden change in illumination has occurred: first, a reasonable brightness baseline is restored through nonlinear transformation, and then the ability to express details is enhanced through equalization. The order of the two is irreversible—if equalization is performed first and then transformation is performed, the original distortion will be amplified; if only one is performed, it is impossible to achieve both global correction and local enhancement goals.

[0043] As an example, the target transformation parameters of the nonlinear grayscale transformation in this application are dynamically adjusted according to the ambient lighting information, specifically including: Method 1: When the ambient light intensity is higher than the second intensity threshold, the transformation parameters for suppressing the highlight region are used as the target transformation parameters for the nonlinear grayscale transformation; Specifically, nonlinear grayscale transformation can be gamma correction, which can be an image enhancement method that performs nonlinear mapping of image pixel grayscale values ​​through a power function. Specifically, the transformation parameter for suppressing highlight areas can be a real number with a Gamma value in the range of 0.6 to 0.8. The smaller the Gamma value, the stronger the compression ability of the highlight area, thereby suppressing overexposure. In this embodiment, when the ambient light intensity is higher than the second intensity threshold (e.g., 500 lux), the system sets the target transformation parameter of the nonlinear grayscale transformation to the Gamma value for suppressing the highlight region. This Gamma value is used to perform Gamma correction processing on the current frame image that is determined to be a sudden change in illumination in step S12, so that the grayscale distribution of the highlight region of the image is shrunk and the details are preserved intact, providing a stable input with sufficient gradient response for subsequent feature point extraction.

[0044] Method 2: When the ambient light intensity is lower than the third intensity threshold, the transformation parameters that enhance the details in the dark areas are used as the target transformation parameters for the nonlinear grayscale transformation; It can be understood that the second intensity threshold in this embodiment is used to distinguish strong light scenes, the third intensity threshold is used to distinguish weak light scenes, and the third intensity threshold is less than the second intensity threshold.

[0045] Among them, the transformation parameter that enhances dark details can be a real number with a Gamma value in the range of 1.2 to 1.5. The larger the Gamma value, the stronger the stretching ability of low grayscale areas, thereby improving dark contrast and signal-to-noise ratio. For example, when the ambient light intensity is lower than the third intensity threshold (e.g., 100 lux), the system sets the target transformation parameter of the nonlinear grayscale transformation to a Gamma value that enhances dark details. This Gamma value is used to perform Gamma correction processing on the current frame image that is determined to be a sudden change in illumination in step S12, so that the grayscale distribution in the dark area of ​​the image is expanded and weak gradients can be distinguished, providing structural clues with sufficient response intensity for subsequent feature point extraction.

[0046] This example (by mapping the physical quantity of ambient light intensity to the target transformation parameter of nonlinear grayscale transformation, and constructing a three-level lighting scene judgment logic based on the second and third intensity thresholds) achieves coordinated adaptation of highlight suppression in strong light scenes and shadow enhancement in weak light scenes. With the help of intensity data fed back in real time by the light sensor, the system can automatically select a Gamma value that matches the current lighting level when a sudden change in lighting occurs, thereby driving the Gamma correction module to output a corrected image with higher texture fidelity. This image provides a high-quality input basis for feature point extraction in step S2, significantly improving the stability of the response value and the spatial distribution uniformity of feature points in the weak texture-sudden lighting change composite scene, thereby supporting the effective execution of subsequent cross-frame static feature point reuse and binocular geometric constraint verification, and ultimately enhancing the overall robustness of the binocular SLAM system.

[0047] As an example, the gray-level distribution equalization processing of this application can be performed using global histogram equalization to expand the overall gray-level dynamic range of the image; alternatively, it can be performed using local neighborhood histogram equalization (CLAHE) to enhance contrast while suppressing background noise amplification; furthermore, it can also be performed using a fusion-type histogram equalization strategy, i.e., applying strong equalization to high-frequency regions and weak equalization to low-frequency regions. Based on any of the above specific implementation methods, this application can achieve an improvement in the discriminability of local texture structures in the image.

[0048] The technical effect of this application embodiment is as follows: by weighted calculation of gray-level mean difference rate, the moment of illumination change is accurately identified, and on this basis, the cascade process of nonlinear gray-level transformation and gray-level distribution equalization is initiated only when necessary. This avoids invalid correction noise under stable illumination and solves the problem of single correction algorithm being ineffective in sudden change scenarios. With the help of this collaborative mechanism, the corrected image significantly improves feature discrimination while maintaining a natural appearance. This ensures that the number of effective points extracted by ORB features in S2 can still stably meet the first quantity threshold requirement in the weak texture-illumination change composite scenario, thereby providing a high-quality input foundation for the cross-frame static feature point reuse mechanism. Finally, it supports the S3 binocular geometric constraint screening to obtain a higher precision effective feature point set.

[0049] Further reference Figure 3 In one embodiment, step S2 of the foregoing embodiment may further include: Step S21: Extract feature points from the corrected image, count the number of extracted feature points, and compare it with a first number threshold; As an example, the first threshold number is, for instance, 50, used to determine whether the current scene is a weak texture scene. This threshold represents the minimum reliable feature point size required to support binocular matching and triangulation under the current image quality and illumination correction effect. When the number of feature points obtained by statistics is lower than this threshold, it indicates that the local texture information of the image is lacking and the edge and corner response is insufficient, which cannot meet the basic requirements of feature point density and distribution uniformity for subsequent binocular geometric constraint verification, thereby triggering the cross-frame static feature point reuse mechanism. This comparison action constitutes the start switch of the cross-frame reuse process, and its output directly determines whether to execute S22 and S23.

[0050] Step S22: If the number of extracted feature points is lower than the first number threshold, select feature points with inter-frame displacement less than the first displacement threshold from the cached historical keyframes and use them as static feature points. As an example, the first displacement threshold is, for example, 2 pixels, used to determine whether a feature point is a static feature point; this threshold reflects the maximum allowable projection offset of the same physical point on a fixed scene structure in the coordinate system of consecutive frames under the premise of limited camera motion; inter-frame displacement can refer to the Euclidean distance between the corresponding pixel coordinates of the same feature point in two adjacent frames. In specific implementations, one approach is to determine whether a feature point is stably within a first displacement threshold range based on the mean and standard deviation of its displacement trajectory over three consecutive frames in a historical keyframe sequence. Another approach is to inversely calculate the feature point's motion velocity in the world coordinate system based on the residual of the reprojection between the historical keyframe and the current frame after initial pose estimation, identifying feature points with velocities below 0.02 m / s as static feature points. Furthermore, this embodiment can also combine feature point response values ​​(e.g., ORB response value ≥ 100) with displacement stability for joint screening, retaining only feature points with high response and low displacement to exclude noise response or pseudo-corner interference. Based on any of the above specific implementations, this application can obtain a set of static feature points that can be reused across frames, ensuring that the selected feature points originate from real, static, rigid structures in the scene, such as walls, door frames, and fixed furniture, rather than dynamic objects, such as pedestrians, swaying curtains, or surface textures of mobile devices.

[0051] Step S23: Feature point projection fusion, projecting the selected static feature points onto the coordinate system of the current frame image, merging them with the feature points extracted from the current frame to form a supplementary feature point set; It should be noted that feature point projection fusion can refer to the process of converting the pixel positions (u_k, v_k) of static feature points in the historical keyframe image coordinate system into their equivalent pixel positions (u_c, v_c) in the current frame image coordinate system through 3D spatial mapping and reprojection operations, based on the intrinsic parameter matrix of the stereo camera, including focal lengths f_x, f_y and principal point coordinates c_x, c_y, and the relative pose transformation of historical keyframes relative to the current frame. This transformation is represented by T_{key→curr} output by the SLAM front end. In a specific implementation, one possible approach for this embodiment is to use the depth estimate of the static feature point in the historical keyframes, combined with intrinsic parameters, to inversely solve for its 3D point P_k in the camera coordinate system. This is then transformed to the current frame's camera coordinate system using T_{key→curr} to obtain P_c. Finally, the result is obtained through forward projection using intrinsic parameters (u_c, ...). (v_c) Another specific implementation of this embodiment can be to establish the epipolar geometric relationship between the historical keyframe and the current frame using the fundamental matrix F or the essential matrix E, and directly solve the corresponding matching point position of the static feature point in the current frame; In addition, the specific implementation of this embodiment can also be based on the sparse optical flow method, such as Lucas-Kanade optical flow, to perform short-range tracking of the motion trajectory of the static feature point between the historical keyframe and the current frame, and combine pose transformation for error compensation to improve the projection positioning accuracy; Based on any of the above specific implementation methods, this application can obtain the coordinates of the projected static feature points, and merge them with the feature points extracted from the current frame in the pixel coordinate space in an unordered manner, that is, without distinguishing the source, uniformly include them into the same feature point set to form a supplementary feature point set; All feature points in this set have effective pixel positions in the current frame image coordinate system, and the spatial distribution covers the entire image domain, thereby ensuring that the subsequent binocular geometric constraint verification has sufficient input cardinality and spatial representativeness.

[0052] The technical effect of this embodiment is as follows: This application sets a first quantity threshold in S21 to realize the quantitative recognition and reuse mechanism trigger control of weak texture scenes; introduces a first displacement threshold as a static criterion in S22 to separate fixed structures and dynamic interference from the kinematic dimension; and completes cross-frame spatial alignment based on the binocular geometric model in S23, so that historical static feature points have physical interpretability and coordinate consistency in the current frame. The three work together to ensure that the supplementary feature point set meets the quantity requirements and guarantees spatial reliability and matching robustness without adding hardware or modifying the SLAM backend. This supports the overall goal of outputting an effective feature point set in this application and significantly improves the tracking continuity and positioning accuracy of the SLAM system in weak texture-lighting change composite scenes.

[0053] Further reference Figure 4 In one embodiment, step S3 of the foregoing embodiment may further include: Step S31: Based on the internal parameters of the stereo camera, for each feature point in the supplementary feature point set, use its pixel coordinates in the left and right images of the current frame of the stereo camera to calculate the depth value of the feature point using the triangulation method to obtain the first depth value. It should be noted that, in this embodiment, the internal parameters of the binocular camera may include focal length, principal point coordinates, and distortion coefficients. Triangulation can refer to jointly solving the three-dimensional coordinates of the spatial point in the camera coordinate system based on the pixel coordinates of the same spatial point in the left and right eye images, the binocular baseline length, and the intrinsic parameter matrix, and extracting its Z-axis component as the first depth value. The first depth value represents the straight-line distance of the feature point from the optical center of the binocular camera in real physical space. In this embodiment, the first depth value is used to provide a basic measurement basis for subsequent multi-level geometric constraint verification and is a prerequisite input for executing S32, S33, and S34.

[0054] In a specific implementation, one approach of this embodiment is to determine the first depth value of each feature point in the supplementary feature point set using triangulation. Another approach is to determine the first depth value of each feature point in the supplementary feature point set using a triangulation method based on epipolar constraints and nonlinear optimization. Furthermore, this embodiment can also determine the first depth value of each feature point in the supplementary feature point set using a weighted least squares triangulation method that considers lens distortion compensation. Based on any of the above implementation methods, this application can obtain a first depth value to characterize the spatial reliability of the feature points, supporting subsequent three-level verification processes: depth range selection, binocular consistency verification, and reprojection error optimization.

[0055] Step S32: Remove feature points whose first depth value exceeds the range of the preset minimum depth threshold and the maximum depth threshold from the supplementary feature point set to obtain the first set of filtered points; As an example, a minimum depth threshold of 0.5 m is used to exclude feature points that are too close to the camera, causing matching ambiguity, unstable triangulation, or occlusion interference. A maximum depth threshold of 5.0 m is used to exclude feature points that are too far from the camera, causing pixel positioning error amplification, decreased depth estimation signal-to-noise ratio, or exceeding the effective mapping radius of the SLAM system. Together, they constitute a physically measurable and geometrically reliable effective depth range. In this embodiment, this depth range limits the first depth value directly applied to the output of S31, forming the first round of hard filtering to ensure that all feature points entering the subsequent verification stage are within the spatial scale supported by the system calibration and application tasks.

[0056] Step S33: For each feature point in the first set of filtering points, calculate the depth value of the feature point in the left eye image as the left eye depth value and the depth value of the feature point in the right eye image as the right eye depth value. Calculate the difference rate between the left eye depth value and the right eye depth value. Remove feature points with a difference rate greater than a preset first difference threshold from the first set of filtering points to obtain the second set of filtering points. Specifically, as an example, the left-eye depth value and the right-eye depth value refer to the two depth estimation results calculated by independently back-projecting the feature point in the left-eye image and its corresponding matching point in the right-eye image into three-dimensional space. The difference rate is defined to quantify the consistency of binocular observations. For example, the first difference threshold is 0.1, which is used to determine whether the feature point meets the observation redundancy credibility condition under binocular geometric constraints. In this embodiment, this step introduces independent depth estimation from both perspectives and a relative error criterion to identify and eliminate inconsistencies in left and right-eye depth caused by mismatches, interference from dynamic objects, or monocular imaging anomalies. This is an important supplement and enhancement to the S32 single depth threshold screening.

[0057] Step S34: Use the random sampling consensus algorithm to optimize the reprojection error of the feature points in the second selection point set, remove feature points whose reprojection error is greater than the preset second displacement threshold, and output the effective feature point set; As can be understood, the Random Sampling Consensus Algorithm (RANSAC) is a robust parameter estimation algorithm used to fit the optimal geometric model in a dataset containing a large number of outliers. In this embodiment, RANSAC takes the 3D coordinates of feature points in the second selection point set and their original pixel coordinates in the left and right eye images as input, iteratively estimates the optimal essential matrix or fundamental matrix, and calculates the reprojection error of each feature point accordingly—that is, the Euclidean distance between the pixel position obtained after the 3D coordinates of the feature point are reprojected back to the image plane by the camera model and the original matching pixel position. For example, the second displacement threshold is 2 pixels, which is used to set the acceptable upper limit of the reprojection error. In this embodiment, this step, through the robust fitting mechanism of RANSAC, further identifies and eliminates a small number of outlier matching points that still remain in the aforementioned three-step selection, ensuring that the final output effective feature point set has a high inlier rate and low noise level at the geometric modeling level.

[0058] The technical effects of this embodiment are as follows: This application obtains the first depth value of each feature point through S31, providing a unified spatial measurement benchmark for subsequent multi-level verification; with the help of S32, physically unreachable points are eliminated based on the minimum / maximum depth threshold, initially ensuring spatial rationality; further, the depth difference rate between the left and right eyes is calculated through S33 and inconsistent points are eliminated, strengthening the redundancy and reliability of binocular observation; finally, relying on the RANSAC reprojection error optimization in S34, robust removal of residual outliers is achieved, which can significantly improve the accuracy and stability of feature point matching in weak texture-lighting abrupt change composite scenes, thereby ensuring the full realization of the technical effect of outputting an effective feature point set in this application.

[0059] Further reference Figure 5 In one embodiment, step S3 is followed by steps S4 and S5 to achieve data closure of the cross-frame multiplexing mechanism: Step S4: Store the feature points that satisfy the static condition in the set of effective feature points into the key frame cache library to obtain the stored static feature points; wherein, the static condition includes that the inter-frame displacement of the feature point in multiple consecutive frames is less than the third displacement threshold.

[0060] It can be understood that the static condition (the inter-frame displacement of a feature point in multiple consecutive frames is less than the third displacement threshold) can refer to: for each feature point in the effective feature point set, calculate its pixel coordinate offset between the current frame and the previous N frames (e.g., N≥2). If the inter-frame displacement between all adjacent frame pairs is less than the third displacement threshold, then the feature point is determined to meet the static condition. The third displacement threshold is used to quantify the motion stability of the feature point in the time dimension. Its value is equal to the first displacement threshold defined in this application, for example, 2 pixels, to ensure the consistency of the static criterion across frames. This condition excludes the interference of instantaneous low displacement caused by camera micro-shakes, brief occlusions, or local deformations, and only retains stable feature points that are truly attached to the rigid scene structure.

[0061] A keyframe cache library can refer to a memory structure used to persistently store statically verified feature point data. Its organization is a keyframe queue arranged in chronological order, with each frame associated with a set of verified static feature point coordinates, descriptors, and response values. For example, the cache library in this embodiment supports a first-in-first-out update strategy, with a maximum capacity corresponding to 5-10 keyframes, ensuring that cache resources do not grow indefinitely with runtime, while maintaining the residence probability of frequently reused points. The stored static feature points are the data source on which the cross-frame static feature point reuse mechanism in subsequent frame execution steps S2 depends. Their existence enables the static point pool to have long-term availability, rather than being limited to the history of the most recent single frame.

[0062] In step S4 of this embodiment, a long-term static point pool is constructed by storing feature points that satisfy the displacement constraints of consecutive multi-frames into the keyframe cache library. This point pool forms a closed-loop feedback with the cross-frame static feature point reuse mechanism in step S2: S4 provides high-quality static point input, and S2 realizes its dynamic reuse. The two work together to improve the keyframe cache hit rate to over 85% and control the cross-frame reuse latency to within 10 ms. On this basis, even in continuous weak texture sequences, the system can still maintain a stable feature supply and avoid tracking interruptions caused by the decay of static point resources. This supports the reliable execution of reading the stored static feature points from the keyframe cache library in this application and ultimately improves the long-term robustness of stereo SLAM in complex scenes.

[0063] Step S5: For the corrected image corresponding to the subsequent frame image, repeat the step of extracting feature points from the corrected image. If the number of extracted feature points is lower than the first threshold, during the process of fusing the cached historical static feature points with the feature points extracted in the current frame (i.e., when step S2 is executed in the subsequent frame), read the stored static feature points from the key frame cache library as the cached historical static feature points, which are used to fuse with the feature points extracted in the subsequent frame (i.e., the corrected image corresponding to the subsequent frame image).

[0064] For example, in this embodiment, when the ORB-SLAM2 system is running on the NVIDIA Jetson Xavier NX embedded platform, and the robot enters a pure white wall corridor (weak texture scene) and simultaneously encounters the corridor lights turning on (sudden change in lighting), the system completes step S4 at frame 120, writing 37 feature points that satisfy the condition of displacement ≤ 2 pixels for 5 consecutive frames and response value ≥ 100 into the keyframe cache. From frame 121 onwards, each time step S2 is executed, step S5 reads all 37 stored static feature points from the cache, projects them to the current frame coordinate system in step S23, and merges them with the real-time extracted ORB feature points, so that the number of effective feature points in a single frame is stably maintained in the range of 55-62, the feature matching delay is always controlled within 28ms, and no tracking interruption occurs. The entire process does not trigger any historical keyframe re-screening action, and the cache point reuse accuracy reaches 96.2%, which is higher than the 84.1% of the temporary screening mechanism.

[0065] The technical effect of this embodiment is as follows: by using the keyframe cache library built in step S4 as the source of static feature points in step S5, a data closed loop for the cross-frame reuse mechanism is realized. With the help of this closed loop, the system no longer relies on temporary, one-time static point screening in scenarios with weak texture-lighting abrupt changes, but reuses high-quality static feature points verified by long-term displacement. On this basis, combined with the double-buffered asynchronous loading strategy, feature point reading and current frame feature extraction are executed in parallel, eliminating I / O waiting blocking. Finally, this collaborative mechanism not only compresses the cross-frame reuse time from 15 ms to <5 ms, but also enhances the convergence efficiency and mismatch rejection rate of the binocular geometric constraint verification in step S3 by improving the confidence of reused points, thereby supporting the robust, real-time, and low-power operation of the overall technical solution in a real embedded environment.

[0066] Example 2 See Figure 6 This application also proposes an effective feature point output device, which includes: The illumination adaptive correction module A is used to acquire the current frame image and corresponding ambient illumination information captured by the binocular camera, and to perform illumination correction on the current frame image based on the ambient illumination information to obtain the corrected image. The condition-triggered feature extraction and cross-frame reuse module B is used to extract feature points from the corrected image. When the number of extracted feature points is lower than the first threshold, the cross-frame static feature point reuse mechanism is triggered to fuse the cached historical static feature points with the feature points extracted in the current frame to form a supplementary feature point set. The binocular geometric constraint verification module C is used to filter the supplementary feature point set based on binocular geometric constraints, remove mismatched feature points, and output the valid feature point set.

[0067] As an example, modules A, B, and C can all be encapsulated as front-end enhancement modules. These front-end enhancement modules are independent of the SLAM system's back-end processing flow. They take the effective feature point set output by the binocular geometric constraint verification module C as input and pass it to the SLAM system's feature matching module. The SLAM system's back-end optimizer and loop closure detection algorithm receive the output of the feature matching module and perform pose optimization and loop closure correction. The core code of the back-end optimizer and loop closure detection algorithm remains unchanged. This effectively solves the problem that existing enhancement schemes require modification of the SLAM system's core algorithm, leading to poor compatibility and high integration costs. This example adopts a front-end modular enhancement approach, encapsulating the enhancement steps as independent modules without intruding on the SLAM system's core code, achieving high compatibility and low-cost robustness improvement.

[0068] Each module in the aforementioned effective feature point output device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0069] See Figure 7 This application also proposes an electronic device, including: at least one processor 1001, a communication bus 1002, a memory 1005, and a computer program 1006 stored in the memory 1005 and executable on the at least one processor 1001. When the processor 1001 executes the computer program 1006, it implements the steps in the embodiment of the effective feature point output method of this application.

[0070] The electronic device also includes a camera interface 1003 for connecting a stereo camera. The electronic device can transmit the current frame image captured by the stereo camera to the processor 1001 via the camera interface 1003. In some embodiments, the camera interface 1003 can connect to the internal image acquisition module (e.g., an image sensor) of the electronic device; in other embodiments, the camera interface can also connect to an external camera device of the electronic device. Those skilled in the art will understand that… Figure 1 This is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than shown, or a combination of certain components, or different components.

[0071] The processor 1001 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0072] In some embodiments, the memory 1005 may be an internal storage unit of the electronic device, such as a hard disk or memory. In other embodiments, the memory 1005 may be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, the memory 1005 may include both internal and external storage units of the electronic device. The memory 1005 is used to store the operating system and computer-executable programs for the method of outputting the effective feature points.

[0073] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0074] Furthermore, this application also provides a computer program product, which stores a computer program that, when executed by a processor, implements the steps of the effective feature point output method embodiment described above.

[0075] Furthermore, it should be noted that if the integrated unit described in this application is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0076] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0077] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0078] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0079] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0080] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for outputting effective feature points, characterized in that, include: The current frame image captured by the binocular camera and the corresponding ambient lighting information are acquired. The current frame image is then illuminated based on the ambient lighting information to obtain a rectified image. Feature points are extracted from the corrected image. If the number of extracted feature points is less than a first threshold, the cached historical static feature points are fused with the feature points extracted in the current frame to form a supplementary feature point set. The supplementary feature point set is filtered based on binocular geometric constraints to remove mismatched feature points and output a valid feature point set.

2. The method according to claim 1, characterized in that, The step of acquiring the current frame image captured by the binocular camera and the corresponding ambient lighting information, and performing lighting correction on the current frame image based on the ambient lighting information to obtain a corrected image, further includes: Calculate the grayscale mean difference rate between the current frame image and the previous frame image. If the grayscale mean difference rate exceeds a first change rate threshold, it is determined to be a sudden change in illumination. When a sudden change in illumination is detected, nonlinear grayscale transformation and grayscale distribution equalization are sequentially performed on the current frame image.

3. The method according to claim 2, characterized in that, In step S12, the target transformation parameters of the nonlinear grayscale transformation are dynamically adjusted according to the ambient lighting information, including: When the ambient light intensity is higher than the second intensity threshold, the transformation parameter for suppressing the highlight region is used as the target transformation parameter for the nonlinear grayscale transformation. When the ambient light intensity is below the third intensity threshold, the transformation parameters that enhance dark details are used as the target transformation parameters for the nonlinear grayscale transformation.

4. The method according to claim 1, characterized in that, The step of extracting feature points from the corrected image, where the number of extracted feature points is lower than a first threshold, involves fusing cached historical static feature points with the feature points extracted from the current frame to form a supplementary feature point set. Specifically, this includes: Feature points are extracted from the corrected image, the number of extracted feature points is counted, and compared with the first number threshold. If the number of extracted feature points is less than the first number threshold, feature points with inter-frame displacement less than the first displacement threshold are selected from the cached historical keyframes and used as static feature points. The selected static feature points are projected onto the coordinate system of the current frame image and merged with the feature points extracted from the current frame to form a supplementary feature point set.

5. The method according to claim 1, characterized in that, The step of filtering the supplementary feature point set based on binocular geometric constraints, removing mismatched feature points, and outputting a valid feature point set further includes: Based on the internal parameters of the binocular camera, for each feature point in the supplementary feature point set, the depth value of the feature point is calculated by triangulation using its pixel coordinates in the left and right images of the current frame of the binocular camera, thus obtaining the first depth value. Feature points whose first depth value exceeds the range of the preset minimum depth threshold and the maximum depth threshold are removed from the supplementary feature point set to obtain the first filter point set. For each feature point in the first set of filtering points, the depth value corresponding to the feature point in the left eye image is calculated as the left eye depth value, and the depth value corresponding to the feature point in the right eye image is calculated as the right eye depth value. The difference rate between the left eye depth value and the right eye depth value is calculated. Feature points whose difference rate is greater than a preset first difference threshold are removed from the first set of filtering points to obtain the second set of filtering points. The reprojection error of the feature points in the second set of selected points is optimized by using a random sampling consensus algorithm. Feature points with reprojection errors greater than a preset second displacement threshold are removed, and a set of valid feature points is output.

6. The method according to claim 1, characterized in that, After filtering the supplementary feature point set based on binocular geometric constraints, removing mismatched feature points, and outputting the effective feature point set, the method further includes: The feature points that satisfy the static condition in the set of effective feature points are stored in the key frame cache library to obtain the stored static feature points. The static condition includes the feature point's inter-frame displacement being less than a third displacement threshold across multiple consecutive frames.

7. The method according to claim 6, characterized in that, After storing the feature points that satisfy the static condition in the set of effective feature points into the keyframe cache library to obtain the stored static feature points, the method further includes: For the corrected image corresponding to the subsequent frame image, the process of extracting feature points from the corrected image is repeated. If the number of extracted feature points is less than a first threshold, during the process of fusing the cached historical static feature points with the feature points extracted in the current frame, the stored static feature points are read from the key frame cache library and used as the cached historical static feature points for fusing with the feature points extracted from the corrected image corresponding to the subsequent frame image.

8. A device for outputting effective feature points, characterized in that, include: An adaptive illumination correction module is used to acquire the current frame image captured by the binocular camera and the corresponding ambient illumination information, and to perform illumination correction on the current frame image according to the ambient illumination information to obtain a corrected image. The condition-triggered feature extraction and cross-frame reuse module is used to extract feature points from the corrected image. When the number of extracted feature points is lower than a first threshold, a cross-frame static feature point reuse mechanism is triggered to fuse cached historical static feature points with the feature points extracted in the current frame to form a supplementary feature point set. The binocular geometric constraint verification module is used to filter the supplementary feature point set based on binocular geometric constraints, remove mismatched feature points, and output the valid feature point set.

9. An electronic device comprising a memory, a controller, and a computer program stored in the memory and executable on the controller, characterized in that, When the controller executes the computer program, it implements the method as described in any one of claims 1 to 7.

10. A computer program product, said computer program product storing a computer program, characterized in that, When the computer program is executed by the controller, it implements the method as described in any one of claims 1 to 7.

Images