Obstacle scanning perception system for visually impaired assisted navigation

Abstract

This application relates to the field of visual impairment assistance technology, specifically to an obstacle scanning and perception system for visual impairment-assisted navigation. The system includes: a visual impairment-assisted image acquisition module, which acquires depth images of the area to be scanned in real time; a visual impairment-assisted image analysis module, which determines feature pixels, obtains distribution feature values, filters undetermined artifact pixels, compares the distribution differences of gradient magnitudes within the undetermined artifact pixel window with the differences in gradient angles between the undetermined artifact pixels and neighboring pixels, determines gradient distribution feature values, and determines the filtering correction values ​​for the undetermined artifact points; a visual impairment-assisted image processing module, which corrects the filtering window and filters the depth image; and a visual impairment-assisted obstacle recognition module, which converts the filtered depth image into three-dimensional point cloud data, performs fitting, identifies the obstacle point cloud, and provides voice feedback prompts. This application aims to improve image clarity and enhance the accuracy of obstacle recognition through filtering.

Description

Technical Field

[0001] This application relates to the field of visual impairment assistance technology, specifically to an obstacle scanning and sensing system for visual impairment-assisted navigation. Background Technology

[0002] Obstacle scanning and perception for visually impaired assistive navigation is a widely and deeply researched multimodal environmental perception technology. Its core lies in actively scanning and perceiving the physical space surrounding the user through integrated sensors to compensate for the user's lack of visual information. This technology typically uses RGB-D depth cameras, LiDAR, and infrared detection to construct a dynamic 3D point cloud model of the environment. It then identifies obstacles and acquires information about their direction and distance relative to the user, providing obstacle perception feedback through voice prompts. Through multimodal intelligent perception systems, visually impaired individuals can effectively perceive obstacles along their movement routes, significantly improving their safety.

[0003] When using binocular depth cameras for obstacle scanning and perception, several potential problems need to be considered. First, when acquiring depth images, certain weakly textured areas (such as white walls or monochromatic ground) and areas with repetitive textures may produce blocky or striped artifacts. Second, when objects such as pedestrians or vehicles move, motion blur or trailing may occur. These factors can lead to false edges, affecting the accurate identification of obstacles and the effectiveness of navigation assistance. Conventional filtering algorithms, with their fixed maximum and minimum filtering windows, cannot effectively distinguish non-uniformly distributed noise, easily resulting in blurred edges and loss of texture details. This, in turn, leads to misjudgments of obstacle details during subsequent obstacle scanning and perception in visually impaired navigation, resulting in significant deviations in obstacle location acquisition and hindering effective visually impaired navigation assistance. Summary of the Invention

[0004] In view of the above, it is necessary to provide an obstacle scanning and perception system for visually impaired assisted navigation to solve the above problems.

[0005] One embodiment of this application provides an obstacle scanning and sensing system for visually impaired assisted navigation, the system comprising:

[0006] The visually impaired assistive image acquisition module is used to acquire depth images of the area to be scanned in real time;

[0007] The visually impaired assistive image analysis module determines feature pixels based on the changes in depth values ​​within the neighborhood of each pixel. It then acquires similar neighboring pixels within a preset feature window for each feature pixel and compares the similarity of depth value distribution within the neighborhood of each feature pixel with that of its similar neighboring pixels to obtain the distribution feature value of each feature pixel. This further filters out undetermined artifact pixels. The module also compares the distribution differences of gradient magnitudes within the preset feature window of each undetermined artifact pixel and combines this with the difference in gradient angles between each undetermined artifact pixel and its similar neighboring pixels to determine the gradient distribution feature value of each undetermined artifact pixel. Finally, the module uses the gradient distribution feature value and the normalized result after forward fusion of the distribution feature values ​​as the filtering correction value for each undetermined artifact pixel.

[0008] The visually impaired assistive image processing module corrects the filtering window based on the filtering correction value to obtain the optimal filtering window for each undetermined artifact pixel and performs filtering processing on the depth image.

[0009] The visually impaired assistive obstacle recognition module is used to convert filtered depth images into three-dimensional point cloud data, fit the data, identify obstacle point clouds, and provide voice feedback prompts.

[0010] The specific steps for determining the feature pixel points include:

[0011] All pixels in the depth image are thresholded based on the dispersion of depth values ​​of all pixels within the range of each pixel to obtain the optimal threshold; pixels with a dispersion of depth values ​​greater than the optimal threshold are taken as feature pixels.

[0012] The specific process for obtaining neighboring pixels of the same type is as follows:

[0013] Cluster all pixels within the feature window of each feature pixel, and select a preset number of pixels that are in the same cluster as each feature pixel and are closest to it as the neighboring pixels of each feature pixel.

[0014] Specifically, obtaining the distribution feature value of each feature pixel is as follows:

[0015] In the feature window of each feature pixel, the similarity of the depth values ​​between the neighborhood range of each feature pixel and the neighborhood range of each neighboring pixel of the same type is obtained, and the difference between the similarity of all pairwise combinations is averaged; the overall mean and standard deviation of all average values ​​obtained from the cluster to which each feature pixel belongs are obtained, and the overall mean and standard deviation are positively fused to obtain the distribution feature value of each feature pixel.

[0016] The specific process for obtaining the undetermined artifact pixels through the screening is as follows:

[0017] All feature pixels are thresholded based on their distribution feature values ​​to obtain the distribution threshold; feature pixels with distribution feature values ​​greater than or equal to the distribution threshold are designated as undetermined artifact pixels.

[0018] The specific process for determining the gradient distribution feature value of each undetermined artifact pixel is as follows:

[0019] Based on the distribution differences of gradient magnitude between rows and columns within the preset feature window of each undetermined artifact pixel, the gradient magnitude feature value of each undetermined artifact pixel is determined.

[0020] Analyze the difference in gradient angle between each undetermined artifact pixel and its neighboring pixels of the same type to determine the gradient direction feature value of each undetermined artifact pixel;

[0021] By combining the gradient magnitude feature value and the gradient direction feature value, the gradient distribution feature value is obtained; wherein the gradient distribution feature value is positively correlated with the gradient magnitude feature value and negatively correlated with the gradient direction feature value.

[0022] Specifically, determining the gradient magnitude feature value of each undetermined artifact pixel involves:

[0023] The differences in gradient amplitude fluctuation characteristics between the row and the other rows and columns of each undetermined artifact pixel are analyzed to determine the row gradient feature value and column gradient feature value. The row gradient feature value and column gradient feature value of each undetermined artifact pixel are positively fused to obtain the gradient amplitude feature value of each undetermined artifact pixel.

[0024] Specifically, determining the gradient direction feature value of each undetermined artifact pixel involves:

[0025] The gradient angle difference between each undetermined artifact pixel and all its neighboring pixels of the same type is averaged, and the disorder of all average values ​​obtained in the cluster to which each undetermined artifact pixel belongs is calculated to obtain the gradient direction feature value of each undetermined artifact pixel.

[0026] Specifically, obtaining the optimal filtering window for each undetermined artifact pixel is as follows:

[0027] The initial filter window side length is set to 2m+1, where m has a maximum and a minimum set value.

[0028] The optimal value of m corresponding to the side length of the filtering window for the undetermined artifact pixel b is obtained by the following formula:

[0029]

[0030] In the formula, This represents the optimal value of m corresponding to the minimum filter window side length for the undetermined artifact pixel b. This is the rounding function; This represents the filter correction value for the undetermined artifact pixel b. This represents the initial value of m corresponding to the initial filter window side length.

[0031] The specific process of identifying obstacle point clouds is as follows:

[0032] Obtain the maximum fitting plane of the filtered 3D point cloud data, and treat the 3D data points other than the maximum fitting plane as obstacle point clouds.

[0033] This application has at least the following beneficial effects:

[0034] The visually impaired assistive image acquisition module of this application acquires depth images of the area to be scanned in real time, providing high-quality depth information to lay the foundation for subsequent processing and ensuring the timeliness and accuracy of the data, thereby improving the overall system response speed. The visually impaired assistive image analysis module analyzes changes in depth values, identifies feature pixels, and filters out potential artifact pixels. Based on the changes and distribution characteristics of pixel depth values, it obtains corresponding filter correction values. By comparing depth distribution feature values ​​and gradient distribution feature values, it can effectively identify possible artifact pixels, thereby improving image quality, enhancing noise processing of depth images, ensuring that subsequent recognition algorithms work more accurately, and reducing the probability of misidentification. The auxiliary image processing module corrects the filtering window based on the filtering correction value, optimizes the depth information of each undetermined artifact pixel, and provides a more accurate depth image. Filtering improves image clarity and usability, and optimizing the filtering window increases the accuracy of obstacle recognition and reduces misjudgments caused by artifacts. The visually impaired obstacle recognition module converts the filtered depth image into 3D point cloud data, identifies obstacles, and provides voice feedback. It transforms 2D depth information into 3D point cloud, making the spatial location of obstacles more intuitive and facilitating subsequent processing and analysis. Finally, voice feedback enhances the user experience, promptly notifying users of potential obstacles and improving safety and interactivity. Attached Figure Description

[0035] Figure 1 A block diagram of an obstacle scanning and sensing system for visually impaired assisted navigation provided in this application;

[0036] Figure 2 The flowchart for obstacle recognition provided in this application. Detailed Implementation

[0037] In the description of the embodiments in this application, the words "exemplary," "or," and "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary," "or," and "for example" is intended to present the relevant concepts in a specific manner.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this application's specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0039] It should also be noted that the terms "first" and "second" in this application and its accompanying drawings are used to distinguish similar objects, rather than to describe a specific order or sequence. The methods disclosed in the embodiments of this application or the methods shown in the flowcharts include one or more steps for implementing the method. Without departing from the scope of protection of this application, the execution order of multiple steps can be interchanged, and some steps can also be deleted.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0041] The following description, in conjunction with the accompanying drawings, details a specific scheme for an obstacle scanning and sensing system for visually impaired assisted navigation provided in this application.

[0042] Please see Figure 1 The diagram illustrates a block diagram of an obstacle scanning and perception system for visually impaired assisted navigation according to an embodiment of this application. The system includes: a visually impaired assisted image acquisition module, a visually impaired assisted image analysis module, a visually impaired assisted image processing module, and a visually impaired assisted obstacle recognition module.

[0043] This application first proposes an obstacle scanning and sensing system for visually impaired assisted navigation, applied in the field of visually impaired assistance technology. The system includes:

[0044] Visually impaired assistive image acquisition module: acquires depth images of the area to be scanned in real time.

[0045] In the obstacle scanning and perception process of visually impaired assisted navigation, the main focus is on detecting stationary or moving pedestrians and obstacles on the road, and providing feedback information on direction and distance. Therefore, the image acquisition device used in visually impaired assisted navigation, a binocular depth camera, acquires depth images of the area to be scanned in real time using binocular stereo vision technology. The frame rate for acquiring depth images needs to meet the frame rate requirements of real-time navigation, and the actual setting range is [10, 30] frames. In this embodiment, the frame rate is set to 20 frames, but the implementer can set it according to actual needs.

[0046] The visually impaired assistive image analysis module: Based on the changes in depth values ​​within the neighborhood of each pixel, feature pixels are identified; neighboring pixels of the same type are obtained within a preset feature window for each feature pixel, and the similarity of the depth value distribution within the neighborhood of each feature pixel and its neighboring pixels of the same type is compared to obtain the distribution feature value of each feature pixel, which is further filtered to obtain undetermined artifact pixels; the distribution differences of gradient magnitude within the preset feature window of each undetermined artifact pixel are compared, and the gradient angle differences between each undetermined artifact pixel and its neighboring pixels of the same type are combined to determine the gradient distribution feature value of each undetermined artifact pixel; the gradient distribution feature value and the normalized result after forward fusion of the distribution feature values ​​are used as the filtering correction value for each undetermined artifact pixel.

[0047] The acquired depth image is filtered and denoised. Considering the significant noise interference in depth images acquired during visually impaired assisted navigation due to equipment exposure and external environmental interference, conventional filtering algorithms often fail to effectively handle non-uniformly distributed noise, leading to blurred edges and loss of texture details. Given that adaptive median filtering can utilize windows of different sizes for filtering non-uniform noise and offers good edge protection, this application uses an adaptive median filtering algorithm to process the real-time acquired depth image. The detailed analysis steps are as follows:

[0048] When analyzing the acquired depth map, a system is constructed centered on each pixel. The window, serving as the feature window for each pixel, is used for local feature analysis of the pixel. In this embodiment, n is set to 7, but the implementer can adjust the size of the feature window according to the actual situation. It should be noted that when the pixel cannot meet the requirements... When the window size is large, the mean-compensation method is used to fill in the missing pixels within the window.

[0049] (1) In road scenes, the real edge regions of various obstacles are usually continuous curves or broken lines, which conform to the projection contour of the object in the image. At the same time, the edges of obstacles are usually more obvious, the edge transition area is relatively narrow, and the depth value distribution is more regular. However, when depth matching errors or moving target ghosting occur, these situations may form false edges. Such false edge regions are mostly irregular blocky, spot-like, or strip-like artifacts, usually showing broken pseudo-linear features. At the same time, such regions have serious noise, the noise point distribution is more discrete, and the depth value is chaotic and irregular.

[0050] Based on the above analysis, firstly, the depth values ​​of all pixels in the depth image are obtained, and then all depth values ​​are linearly mapped to... To reduce the computational complexity of subsequent calculations and avoid extreme depth values ​​that could prevent further computation, the depth dispersion of each pixel and its eight neighboring pixels is calculated first. In this embodiment, the dispersion is calculated using the standard deviation; in other embodiments, the dispersion can also be calculated using the range, coefficient of variation, etc. The dispersion of all pixels in the depth image is used as input, and the Otsu thresholding method is used to obtain the optimal threshold. Considering that pixels at textures, edges, and pseudo-edges (artifacts) often exhibit abrupt changes in depth values ​​among their eight neighboring pixels, resulting in a larger dispersion of the obtained depth values, while the overall depth values ​​in smooth internal regions are relatively flat, thus exhibiting a smaller overall dispersion of depth values. Pixels with depth dispersion greater than the optimal threshold are selected as feature pixels for further analysis.

[0051] Furthermore, taking feature pixel a as an example, based on the feature window of feature pixel a, the depth value corresponding to each pixel within the window is analyzed. All depth values ​​are used as input, and the DBSCAN clustering algorithm is used for clustering. Each cluster is output after clustering. The DBSCAN clustering algorithm is a well-known technique, and the specific steps will not be elaborated further. The cluster containing feature pixel a is analyzed, and the region containing each pixel and its eight neighboring pixels in the cluster is used as a feature matching block. Since feature pixels are usually pixels at textures, edges, or pseudo-edges, their feature range typically includes multiple pixels with similar depth values.

[0052] Within the cluster containing feature pixel a, a predetermined number of pixels closest to each pixel are identified as neighboring pixels of the same type. In this embodiment, the predetermined number is 2, but the implementer can adjust it according to actual conditions. The similarity between the feature matching block of each pixel and the feature matching blocks of its neighboring pixels of the same type is analyzed. In this embodiment, the Structural Similarity Index (SSIM) is used to calculate the similarity. In other embodiments, the similarity can also be calculated using the corresponding positions of the feature matching blocks. The calculation of the SSIM is a well-known method and will not be elaborated further. The absolute differences of the similarity between all pairwise combinations of each pixel in the cluster containing feature pixel a are averaged. In this embodiment, since only two neighboring pixels of the same type are considered, the absolute difference of the similarity between the two pixels in the cluster containing feature pixel a can be directly obtained. Furthermore, the overall mean and standard deviation of all average values ​​obtained in the cluster containing feature pixel a are calculated, and the overall mean and standard deviation are positively fused to obtain the distribution feature value corresponding to feature pixel a. In this embodiment, the forward fusion adopts the addition calculation method. In other embodiments, the forward fusion can also adopt the multiplication calculation method.

[0053] It should be understood that when the distribution feature value of feature pixel a is larger, it indicates that the pixels with similar depth values ​​in the feature window corresponding to feature pixel a are more dispersed, and the depth value distribution is chaotic. The overall similarity between the feature windows of each pixel and its neighboring pixels of the same type is relatively small, and the absolute difference between the two obtained SSIM values ​​is usually large and random. The smaller the value, the more concentrated the distribution of pixels with similar depth values ​​is within the feature window corresponding to feature pixel 'a'. This indicates that the pixel and its two nearest neighbors of the same type usually have many overlapping feature matching blocks, exhibiting high similarity. The absolute difference between the two obtained SSIM values ​​is relatively small and tends to be stable, consistent with the characteristics of key edge regions. The distribution feature values ​​of all feature pixels are obtained and used as input to the Otsu thresholding algorithm. The output is a distribution threshold. Feature pixels with distribution feature values ​​greater than or equal to the distribution threshold are designated as undetermined artifact pixels for further differentiation.

[0054] (2) In road scenarios, the texture area inside obstacles may exhibit similar local distribution characteristics to the appearance of false edges (depth matching error, blurring and trailing caused by pedestrians and vehicles moving, etc.): the internal texture area usually exhibits more detailed textures, resulting in a smaller overall depth change, and there is usually no drastic change in depth. The texture area is usually evenly diffused in space and does not have a fixed directionality. False edge areas are usually caused by texture superposition and blurring and trailing due to depth matching failure. This usually results in multiple drastic changes in depth values ​​in this area. At the same time, due to the influence of texture aliasing, blurring and trailing and the original area aliasing, the drastic change in depth values ​​is usually asymmetrically distributed, and the overall gradient direction is mainly along the broken false edge.

[0055] Based on the above analysis, using the depth values ​​of all pixels in the image as input, the Sobel operator is used to obtain the gradient magnitude and gradient direction of all pixels in the image. Based on the feature window of the undetermined artifact pixel b, the difference in the fluctuation characteristics of the gradient magnitude of the pixels in the row where the undetermined artifact pixel b is located and the pixels in the other rows is analyzed to determine the row gradient feature value; the difference in the fluctuation characteristics of the gradient magnitude of the pixels in the column where the undetermined artifact pixel b is located and the pixels in the other columns is analyzed to determine the column gradient feature value; the row gradient feature value and the column gradient feature value obtained from the undetermined artifact pixel b are positively fused to obtain the gradient magnitude feature value of the undetermined artifact pixel b; in this embodiment, the specific calculation method of the fluctuation feature is as follows: calculate the gradient of all pixels in each row of the feature window. The range and standard deviation of the amplitude are used as the fluctuation feature. The row gradient feature is calculated as follows: the absolute value of the difference between the fluctuation features of the row where the undetermined artifact pixel b is located and the other rows are calculated. The absolute values ​​of all the differences obtained in the row where the undetermined artifact pixel b is located are averaged to obtain the row gradient feature value. Based on the gradient amplitude of all pixels in each column of the feature window, the same calculation method as the row gradient feature value is used to obtain the column gradient feature value. The row gradient feature value and the column gradient feature value are positively fused by adding them together.

[0056] It should be understood that the gradient magnitude feature value is used to characterize the gradient distribution characteristics in the feature window. The larger the calculated gradient magnitude feature value, the more obvious the row and column gradient fluctuations of the undetermined artifact pixel b are, indicating that there are drastic changes in depth values ​​and possible breaks, resulting in significant differences in the intensity of depth changes in different rows and columns. Conversely, the smaller the calculated gradient magnitude feature value, the more gradual the row and column gradient fluctuations of the undetermined artifact pixel b are, indicating that there are no drastic gradient changes, and that different rows and columns show similar fluctuations and a more uniform texture distribution.

[0057] Furthermore, for the random gradient distribution in the internal texture region, and the gradient direction mainly along the broken pseudo-edge in the case of false edge artifacts, we analyze the feature. Taking the undetermined artifact pixel b as an example, we use the DBSCAN density clustering algorithm in (1) to classify the neighboring pixels of the same type of the undetermined artifact pixel b, calculate the average of the absolute difference of the gradient angle of all pairs of neighboring pixels of the same type, calculate the disorder of all average values ​​obtained in the cluster where the undetermined artifact pixel b is located, and obtain the gradient direction feature value of the undetermined artifact pixel b. In this embodiment, the disorder of multiple variables is calculated using Shannon entropy. The calculation of Shannon entropy is a well-known content and will not be elaborated in detail. When the gradient direction feature value is larger, it indicates that the gradient angle in the feature window corresponding to the undetermined artifact pixel b is more dispersed and there is no specific directional feature; while when the gradient direction feature value is smaller, it indicates that the gradient angle in the feature window corresponding to the undetermined artifact pixel b is regular and the adjacent gradients change smoothly.

[0058] Furthermore, by combining the gradient magnitude feature value and the gradient direction feature value, the gradient distribution feature value is obtained: ,in, This represents the gradient magnitude feature value of the undetermined artifact pixel b; Let b be the gradient direction feature value of the undetermined artifact pixel. This is a preset, extremely small positive number, ensuring the denominator is not 0. In this embodiment, the value is 0.1. When the obtained... The larger the value, the more drastic the gradient amplitude fluctuations and the unevenness of the features in the feature window corresponding to the pixel, and the more regular the gradient angle distribution and the smoother the gradient transformation.

[0059] The distribution feature value and gradient distribution feature value of each undetermined artifact pixel are multiplied together. The obtained product is normalized using the Min-Max normalization method, with the normalization range set to [0,1]. The normalized value obtained for each undetermined artifact pixel is used as the filtering correction value for each undetermined artifact pixel.

[0060] Visually impaired assistive image processing module: Based on the filter correction value, the filter window is corrected to obtain the optimal filter window for each undetermined artifact pixel, and the depth image is filtered.

[0061] Considering that abnormal artifacts, blurry areas, and noise interference areas may produce false edge features, the filtering effect should be strengthened for pixels in such areas. However, normal obstacle edge areas are key features for obstacle scanning, and the focus should be on preserving details, so the edge features of obstacles should be preserved in detail.

[0062] Based on the above analysis, the initial filter window side length is set to 2m+1, where the initial minimum value of m is set to 1 and the initial maximum value of m is set to 3. That is, the initial minimum filter window side length is 3 and the initial maximum filter window side length is 7, both of which are standard settings for median filter windows. For all pixels except those with undetermined artifacts, the initial minimum and maximum filter windows are used directly. For undetermined artifact pixels, taking undetermined artifact pixel b as an example, the optimal value of m corresponding to the filter window side length of undetermined artifact pixel b is obtained by combining the normalized distribution characteristic value and the gradient distribution characteristic value.

[0063]

[0064] In the formula, This represents the optimal value of m corresponding to the side length of the filtering window for the undetermined artifact pixel b. This is a rounding function that guarantees the value will be an integer. This represents the filter correction value for the undetermined artifact pixel b. This represents the initial value of m corresponding to the initial filter window side length. Since the initial minimum filter window side length is 3, it corresponds to... The initial value is 1; the initial maximum filter window side length is 7, which corresponds to The initial value is 3. By using an initial weight of 1 plus a normalized filter correction value, the minimum value of m is guaranteed to be the initial value.

[0065] Based on the above formula, the optimal minimum and maximum filtering windows for each undetermined artifact pixel are obtained.

[0066] It should be understood that in existing technologies, the minimum and maximum filtering windows of adaptive median filtering are usually fixed values. However, these do not take into account the depth matching errors that occur when there are weak or repetitive texture areas in the road depth map acquisition, as well as motion blur caused by moving objects. Such noise interference leads to false edge features in the image, resulting in significant deviations in obstacle scanning and perception due to the influence of false edges. Therefore, this application fully combines the local distribution similarity and gradient distribution features of pixels in the feature window to perform feature weighting on the side length of the adaptive median filtering window, thereby obtaining the optimal filtering window range for different pixels and reducing the error of subsequent obstacle scanning and perception information.

[0067] Furthermore, the real-time acquired depth maps are used as inputs, and the depth maps are filtered using the optimized adaptive median filtering algorithm described above to obtain the filtered depth maps. The adaptive median filtering algorithm is a well-known technique, and the specific operation steps will not be described in detail here.

[0068] The visually impaired obstacle recognition module converts the filtered depth image into 3D point cloud data, fits it, identifies obstacle point clouds, and provides voice feedback prompts.

[0069] Based on camera parameters and depth data in the depth map, the filtered depth map is converted into 3D point cloud data. This process is well known to those skilled in the art and will not be described in detail here.

[0070] Furthermore, the obtained 3D point cloud is processed using the Random Sample Consensus (RANSAC) algorithm to obtain the maximum fitting plane, outputting the parameters of the maximum fitting plane and the set of points within the maximum fitting plane. The RANSAC algorithm is a well-known technique, and its specific operation steps will not be elaborated further. The set of points within the maximum fitting plane is then removed from the 3D point cloud, and the remaining point cloud is the obstacle point cloud.

[0071] Furthermore, using obstacle point cloud data as input, the DBSCAN density clustering algorithm is used to cluster the obstacle point cloud, outputting each obstacle cluster. The DBSCAN density clustering algorithm is a well-known technology, and its specific operation steps will not be elaborated further. For each obstacle cluster, a directed bounding box is calculated, and the closest distance and orientation angle between each directed bounding box and the target are calculated using camera parameters. In this embodiment, when the closest distance to the directed bounding box of an obstacle is less than 2 meters, the distance and orientation angle data are fed back via voice feedback, with feedback occurring every 0.5 meters, completing obstacle scanning perception and feedback for visually impaired assisted navigation.

[0072] The specific flowchart for obstacle recognition is as follows: Figure 2 As shown.

[0073] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description; sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0074] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An obstacle scanning and sensing system for visually impaired assisted navigation, characterized in that, The system includes: The visually impaired assistive image acquisition module is used to acquire depth images of the area to be scanned in real time; The visually impaired assistive image analysis module determines feature pixels based on the changes in depth values ​​within the neighborhood of each pixel. It then acquires similar neighboring pixels within a preset feature window for each feature pixel and compares the similarity of depth value distribution within the neighborhood of each feature pixel with that of its similar neighboring pixels to obtain the distribution feature value of each feature pixel. This further filters out undetermined artifact pixels. The module also compares the distribution differences of gradient magnitudes within the preset feature window of each undetermined artifact pixel and combines this with the difference in gradient angles between each undetermined artifact pixel and its similar neighboring pixels to determine the gradient distribution feature value of each undetermined artifact pixel. Finally, the module uses the gradient distribution feature value and the normalized result after forward fusion of the distribution feature values ​​as the filtering correction value for each undetermined artifact pixel. The visually impaired assistive image processing module corrects the filtering window based on the filtering correction value to obtain the optimal filtering window for each undetermined artifact pixel and performs filtering processing on the depth image. The visually impaired assistive obstacle recognition module is used to convert filtered depth images into three-dimensional point cloud data, fit the data, identify obstacle point clouds, and provide voice feedback prompts.

2. The obstacle scanning and sensing system for visually impaired assisted navigation as described in claim 1, characterized in that, The specific steps for determining the feature pixel points include: All pixels in the depth image are thresholded based on the dispersion of depth values ​​of all pixels within the range of each pixel to obtain the optimal threshold; pixels with a dispersion of depth values ​​greater than the optimal threshold are taken as feature pixels.

3. The obstacle scanning and sensing system for visually impaired assisted navigation as described in claim 1, characterized in that, The specific process for obtaining neighboring pixels of the same type is as follows: Cluster all pixels within the feature window of each feature pixel, and select a preset number of pixels that are in the same cluster as each feature pixel and are closest to it as the neighboring pixels of each feature pixel.

4. The obstacle scanning and sensing system for visually impaired assisted navigation as described in claim 1, characterized in that, The specific steps for obtaining the distribution feature value of each feature pixel are as follows: In the feature window of each feature pixel, the similarity of the depth values ​​between the neighborhood range of each feature pixel and the neighborhood range of each neighboring pixel of the same type is obtained, and the difference between the similarity of all pairwise combinations is averaged; the overall mean and standard deviation of all average values ​​obtained from the cluster to which each feature pixel belongs are obtained, and the overall mean and standard deviation are positively fused to obtain the distribution feature value of each feature pixel.

5. An obstacle scanning and sensing system for visually impaired assisted navigation as described in claim 1, characterized in that, The specific process for obtaining the undetermined artifact pixels through the screening is as follows: All feature pixels are thresholded based on their distribution feature values ​​to obtain the distribution threshold; feature pixels with distribution feature values ​​greater than or equal to the distribution threshold are designated as undetermined artifact pixels.

6. An obstacle scanning and sensing system for visually impaired assisted navigation as described in claim 1, characterized in that, The specific process for determining the gradient distribution feature value of each undetermined artifact pixel is as follows: Based on the distribution differences of gradient magnitude between rows and columns within the preset feature window of each undetermined artifact pixel, the gradient magnitude feature value of each undetermined artifact pixel is determined. Analyze the difference in gradient angle between each undetermined artifact pixel and its neighboring pixels of the same type to determine the gradient direction feature value of each undetermined artifact pixel; By combining the gradient magnitude feature value and the gradient direction feature value, the gradient distribution feature value is obtained; wherein the gradient distribution feature value is positively correlated with the gradient magnitude feature value and negatively correlated with the gradient direction feature value.

7. An obstacle scanning and sensing system for visually impaired assisted navigation as described in claim 6, characterized in that, The determination of the gradient magnitude feature value of each undetermined artifact pixel is specifically as follows: Analyze the differences in the fluctuation characteristics of the gradient amplitude between the row and the other rows, and between the column and the other columns of each undetermined artifact pixel, and determine the row gradient feature value and column gradient feature value. The row gradient feature value and column gradient feature value of each undetermined artifact pixel are positively fused to obtain the gradient magnitude feature value of each undetermined artifact pixel.

8. An obstacle scanning and sensing system for visually impaired assisted navigation as described in claim 6, characterized in that, The determination of the gradient direction feature value of each undetermined artifact pixel is specifically as follows: The gradient angle difference between each undetermined artifact pixel and all its neighboring pixels of the same type is averaged, and the disorder of all average values ​​obtained in the cluster to which each undetermined artifact pixel belongs is calculated to obtain the gradient direction feature value of each undetermined artifact pixel.

9. An obstacle scanning and sensing system for visually impaired assisted navigation as described in claim 1, characterized in that, The optimal filtering window for each undetermined artifact pixel is obtained as follows: The initial filter window side length is set to 2m+1, where m has a maximum and a minimum set value. The optimal value of m corresponding to the side length of the filtering window for the undetermined artifact pixel b is obtained by the following formula: In the formula, This represents the optimal value of m corresponding to the side length of the filtering window for the undetermined artifact pixel b. This is the rounding function; This represents the filter correction value for the undetermined artifact pixel b. This represents the initial value of m corresponding to the initial filter window side length.

10. An obstacle scanning and sensing system for visually impaired assisted navigation as described in claim 1, characterized in that, The specific process for identifying obstacle point clouds is as follows: Obtain the maximum fitting plane of the filtered 3D point cloud data, and treat the 3D data points other than the maximum fitting plane as obstacle point clouds.

Images