Intelligent wheelchair adaptive positioning and obstacle avoidance method and system based on multi-sensor fusion

Abstract

The invention relates to the technical field of image recognition, in particular to an intelligent wheelchair self-adaptive positioning and obstacle avoidance method and system based on multi-sensor fusion. According to the width parameter of the chassis of the intelligent wheelchair, intercepting a driving region of interest right below the foresight road navigation image, and extracting a pixel gray level distribution matrix in the driving region of interest; according to the method, a discretized coordinate sampling sequence is operated to generate an obstacle contour curvature energy integral value, tangent angle change and a local direction sensitive item are introduced in the process, the sharpness and form risk of the edge of an obstacle can be captured, and the integral value is associated and matched to a transverse avoidance distance threshold value; avoidance space is automatically expanded for sharp high-risk obstacles, avoidance distance is properly reduced for smooth low-risk obstacles, and narrow space is utilized to the maximum extent on the premise that safety is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to an intelligent wheelchair adaptive positioning and obstacle avoidance method and system based on multi-sensor fusion. Background Technology

[0002] Image recognition technology uses computer systems to process, analyze, and understand images or video sequences in order to identify targets and objects of various patterns.

[0003] Existing image recognition technologies for smart wheelchairs typically focus on semantic classification and location selection of objects within a scene, tending to treat the road surface as a homogeneous geometric plane or only performing binary division of the drivable area. This neglects the analysis of the road surface's micro-texture and frequency domain features, making it difficult to perceive the actual roughness and friction coefficient differences of the road surface. When faced with complex and varied unstructured roads, this singular perception mode makes it difficult for the control system to adjust power output in real time according to road conditions. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and propose an intelligent wheelchair adaptive positioning and obstacle avoidance method and system based on multi-sensor fusion.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: an intelligent wheelchair adaptive positioning and obstacle avoidance method based on multi-sensor fusion, comprising the following steps: Based on the visual sensor of the intelligent wheelchair, the forward-looking road navigation image is acquired. The driving region of interest is cropped directly below the forward-looking road navigation image according to the chassis width parameter of the intelligent wheelchair. The pixel grayscale distribution matrix inside the driving region of interest is extracted. The grayscale co-occurrence contrast value between adjacent pixels in the pixel grayscale distribution matrix is ​​calculated to generate the spatial distribution feature of road texture. The data of the spatial distribution characteristics of the road surface texture are transformed from the spatial domain to the frequency domain spectral coordinate system. The spectral centroid frequency and spectral width in the frequency domain spectral coordinate system are extracted. The spectral centroid frequency and spectral width are calculated to quantify the visual roughness index of the road surface. Identify the outline of a near obstacle located outside the boundary of the area corresponding to the visual roughness index of the road surface in the forward-looking road navigation image; perform discretized coordinate sampling on the outline of the near obstacle to obtain a discretized coordinate sampling sequence; and perform calculations on the discretized coordinate sampling sequence to generate the integral value of the obstacle outline curvature energy. The road surface visual roughness index is compared and mapped to the wheelchair motor torque control table, and the obstacle contour curvature energy integral value is associated and matched to the lateral avoidance distance threshold to generate intelligent wheelchair adaptive positioning and obstacle avoidance strategy instructions.

[0006] Preferably, the step of obtaining the pixel grayscale distribution matrix is ​​as follows: Read the forward-looking road navigation image output by the intelligent wheelchair's vision sensor, call the intelligent wheelchair chassis width parameter to convert it into pixel width, locate the rectangular strip directly below the bottom edge of the forward-looking road navigation image with the center column of the imaging coordinate system as the reference, set the left and right boundaries according to the pixel width and limit the vertical height along the bottom edge, crop out the rectangular strip and correct the pixels that exceed the boundary to obtain the region of interest for driving. Based on the region of interest, the gray value of each pixel is mapped to a discrete gray level according to a fixed gray level quantization rule. The gray level is read point by point according to the row index and column index and written to the corresponding matrix position. For pixels missing at the edge, the neighboring pixel copying method is used to fill in the missing pixels to keep the rows and columns intact, thus obtaining the pixel gray level distribution matrix.

[0007] Preferably, the step of obtaining the spatial distribution features of the road surface texture is as follows: Based on the pixel grayscale distribution matrix, adjacent pixel pair index sequences are constructed in the horizontal, vertical and diagonal directions respectively. The grayscale co-occurrence contrast value of adjacent pixel pairs is calculated for each pair and the row and column coordinates are recorded. The grayscale co-occurrence contrast value is written into the corresponding pixel grid of the spatial domain texture feature layer according to the recorded coordinates and the positions of the multi-directional results are merged to obtain the spatial distribution features of the road surface texture.

[0008] Preferably, the steps for obtaining the centroid frequency and spectral width are as follows: Based on the spatial distribution characteristics of the road surface texture, the texture intensity value corresponding to each pixel position is read in row and column order. The texture intensity value sequence is transformed on the frequency axis to obtain the frequency component and the corresponding amplitude spectrum. All amplitude spectra are arranged from low to high frequency and the upper frequency limit is recorded to form the frequency domain spectrum coordinate system mapping result. Based on the frequency domain spectral coordinate system mapping results, the weighted center frequency of the spectral energy is calculated. Using the energy of each frequency point as the weight, the centroid frequency of the spectrum is obtained. Then, the standard deviation of the spectral energy distribution is calculated with the centroid frequency of the spectrum as the center, and the spectral width is obtained.

[0009] Preferably, the step of obtaining the road surface visual roughness index is as follows: Based on the centroid frequency and spectral width of the spectrum, the road surface visual roughness index is calculated using the following formula: ; in, The visual roughness index of the road surface. The centroid frequency of the spectrum is obtained by weighted averaging of frequency and energy, and the calculation formula is: ,in For the first Each frequency point, The energy at a frequency point, The spectral width reflects the degree of dispersion in the frequency distribution, and is calculated using the following formula: , This is the upper frequency limit of the frequency domain spectral coordinate system mapping result, used for normalization to eliminate sampling rate differences. It is a very small positive number, used to prevent The case where the denominator approaches zero when the value is close to zero.

[0010] Preferably, the step of obtaining the discretized coordinate sampling sequence is as follows: Based on the forward-looking road surface navigation image, the boundary coordinates of the area corresponding to the road surface visual roughness index are read, the continuous edge chain outside the boundary is located, and the region growth tracking is performed according to the pixel eight-connectivity rule to obtain the outline of the nearby obstacle. Based on the contour line of the nearby obstacle, samples are taken at fixed pixel intervals along the contour arc length, and the horizontal and vertical pixel coordinates of each sampling point are recorded to form a discretized coordinate sampling sequence.

[0011] Preferably, the step of obtaining the energy integral value of the obstacle contour curvature is as follows: Based on the discretized coordinate sampling sequence, the energy integral value of the obstacle contour curvature is calculated using the following formula: ; in, The energy integral value of the obstacle profile curvature. The number of sampling points in the discretized coordinate sampling sequence. For the first The angle between the outline of the obstacle segment and the horizontal direction is calculated using the coordinates of two adjacent sampling points, and the formula is as follows: ,in For the first The horizontal pixel coordinates of each sampling point For the first The vertical pixel coordinates of each sampling point This indicates the change in the angle between two adjacent contour lines, used to reflect the magnitude of changes in local curvature. This is a locally oriented sensitive term, and its weight increases when the contour direction is close to perpendicular.

[0012] Preferably, the step of obtaining the intelligent wheelchair adaptive positioning and obstacle avoidance strategy command is as follows: Based on the road surface visual roughness index, the index column of the wheelchair motor torque control table is read, and the segment is located according to the preset segment boundary. If it falls between adjacent segments, interpolation is performed according to the boundary ratio, and the target torque limit, torque rise slope and torque response delay are output to obtain the output parameters of the wheelchair motor torque control table. The obstacle contour curvature energy integral value is read into a level classification, and a lateral distance interval is matched according to the level of the obstacle contour curvature energy integral value. Combined with the imaging ratio, it is converted into an actual lateral distance threshold to generate a lateral avoidance distance threshold. Based on the lateral obstacle avoidance distance threshold, the system calls the target torque limit, torque rise slope, and torque response delay from the output parameters of the wheelchair motor torque control table. Depending on whether the lateral obstacle avoidance distance threshold is exceeded, the system selects to decelerate, brake, or detour. The system synthesizes the instruction sets of the longitudinal control channel and the lateral control channel to generate intelligent wheelchair adaptive positioning and obstacle avoidance strategy instructions.

[0013] The present invention also provides a system comprising: The texture feature extraction module is used to acquire a forward-looking road navigation image based on the vision sensor of the smart wheelchair, extract the driving region of interest directly below the forward-looking road navigation image based on the chassis width parameter of the smart wheelchair, extract the pixel grayscale distribution matrix inside the driving region of interest, calculate the grayscale co-occurrence contrast value between adjacent pixels in the pixel grayscale distribution matrix, and generate the road texture spatial distribution feature. The frequency domain roughness quantification module is used to convert the data of the spatial distribution characteristics of the road surface texture from the spatial domain to the frequency domain spectral coordinate system, extract the spectral centroid frequency and spectral width in the frequency domain spectral coordinate system, and perform calculations on the spectral centroid frequency and spectral width to quantify the visual roughness index of the road surface. The obstacle contour analysis module is used to identify the contour lines of nearby obstacles located outside the boundary of the area corresponding to the visual roughness index of the road surface in the forward-looking road navigation image, to perform discretized coordinate sampling on the contour lines of nearby obstacles to obtain a discretized coordinate sampling sequence, and to perform calculations on the discretized coordinate sampling sequence to generate the obstacle contour curvature energy integral value. The adaptive control strategy generation module is used to map the road surface visual roughness index to the wheelchair motor torque control table, associate and match the obstacle contour curvature energy integral value to the lateral avoidance distance threshold, and generate intelligent wheelchair adaptive positioning and obstacle avoidance strategy instructions.

[0014] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, the region of interest (ROI) directly below the forward-looking road navigation image is extracted based on the width parameter of the intelligent wheelchair chassis, and the internal pixel grayscale distribution matrix is ​​extracted. This allows for focusing on the actual road surface in contact, eliminating background interference, and ensuring efficient and targeted data processing. The grayscale co-occurrence contrast value between adjacent pixels is calculated, and the spatial distribution characteristics of the road texture are transformed into a frequency domain spectral coordinate system. The centroid frequency and spectral width of the spectrum are extracted for calculation. This frequency domain analysis method overcomes the limitations of purely spatial domain features, quantifying the visual roughness index of the road surface using spectral energy distribution characteristics, thereby objectively reflecting the road surface unevenness and friction characteristics. This index is then mapped to the wheelchair motor torque control... The system generates tables to enable the drive system to respond to road conditions in real time. On rough roads, torque is increased to prevent jamming, while on smooth roads, output is reduced to ensure stability, improving ride comfort and passability. It identifies the outline of nearby obstacles and performs discretized coordinate sampling. The discretized coordinate sampling sequence is then processed to generate the obstacle outline curvature energy integral value. This process incorporates tangent angle changes and local direction sensitivity terms to capture the sharpness and morphological risk of obstacle edges. This integral value is correlated and matched to the lateral avoidance distance threshold. For sharp, high-risk obstacles, the avoidance space is automatically expanded, while for smooth, low-risk obstacles, the avoidance distance is appropriately reduced, maximizing the use of narrow spaces while ensuring safety. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the steps of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0017] Please see Figure 1 This invention provides a technical solution: an intelligent wheelchair adaptive positioning and obstacle avoidance method based on multi-sensor fusion, comprising the following steps: Based on the visual sensor of the intelligent wheelchair, the navigation image of the road surface is collected in front. The driving region of interest is cropped directly below the navigation image of the road surface according to the width parameter of the intelligent wheelchair chassis. The pixel grayscale distribution matrix inside the driving region of interest is extracted. The grayscale co-occurrence contrast value between adjacent pixels in the pixel grayscale distribution matrix is ​​calculated to generate the spatial distribution feature of the road surface texture. The data on the spatial distribution characteristics of road surface texture are transformed from the spatial domain to the frequency domain spectral coordinate system. The centroid frequency and spectral width of the spectrum in the frequency domain spectral coordinate system are extracted. The centroid frequency and spectral width are calculated to quantify the visual roughness index of the road surface. Identify the outline of a near obstacle located outside the boundary of the area corresponding to the visual roughness index of the road surface in the forward-looking road navigation image. Discretize the coordinates of the near obstacle outline to obtain a discretized coordinate sampling sequence. Perform calculations on the discretized coordinate sampling sequence to generate the integral value of the obstacle outline curvature energy. By mapping the visual roughness index of the road surface to the torque control table of the wheelchair motor, and associating and matching the energy integral value of the obstacle contour curvature to the lateral avoidance distance threshold, intelligent wheelchair adaptive positioning and obstacle avoidance strategy instructions are generated.

[0018] The steps to obtain the pixel grayscale distribution matrix are as follows: Read the forward-looking road navigation image output by the intelligent wheelchair's vision sensor, call the intelligent wheelchair chassis width parameter to convert it into pixel width, locate the rectangular strip directly below the bottom edge of the forward-looking road navigation image with the center column of the imaging coordinate system as the reference, set the left and right boundaries according to the pixel width and limit the vertical height along the bottom edge, crop out the rectangular strip and correct the pixels that exceed the boundary to obtain the region of interest for driving. Based on the region of interest, the gray value of each pixel is mapped to a discrete gray level according to a fixed gray level quantization rule. The gray level is read point by point in the order of row index and column index and written to the corresponding matrix position. For pixels missing at the edge, the neighboring pixel copying method is used to fill in the missing pixels to keep the rows and columns intact, thus obtaining the pixel gray level distribution matrix.

[0019] Specifically, the system reads the forward-looking road navigation image output by the intelligent wheelchair's vision sensor and sets the intrinsic parameter matrix of the vision sensor. and distortion coefficient ,in Including focal length , and principal point coordinates , Set the physical width of the smart wheelchair chassis The value is 650 mm, based on the installation height of the vision sensor relative to the ground. For 800 mm and downslope angle At an angle of 35 degrees, the actual physical distance corresponding to the bottom edge of the forward-looking road navigation image is calculated using a pinhole camera imaging model. The calculation formula is: ,in The physical width is the vertical field of view corresponding to the bottom edge of the image. Mapped to pixel width on the image plane The calculation formula is With the imaging coordinate system center column Using this as a reference, calculate the coordinates of the left boundary of the rectangular strip. and right boundary coordinates Set vertical height The value is 300 pixels, set based on the braking distance required by the wheelchair at the current speed, along the lower edge of the image. Cut upwards to Iterate through the coordinates of the four vertices of the clipping region. If the coordinates exist... Then the value is forcibly assigned to 0, if Then force the assignment to the image width. ,in The original image resolution is 1920 pixels wide. After boundary checks, a matrix slicing operation is performed to obtain the region of interest for driving.

[0020] Set the total number of grayscale quantization levels based on the region of interest during driving. The system has 16 levels, which are set based on the statistical significance of road surface texture features. The system reads the original grayscale value of any pixel within the region of interest for driving. ,in and Calculate the discrete gray levels after quantization using the column and row indices of the pixels, respectively. The calculation formula is: ,in This represents the floor function, where 256 represents the grayscale depth of an 8-bit image. The matrix is ​​traversed using a double loop through the row indices. From 0 to and column indexes From 0 to , calculate Write to the newly created matrix container At the corresponding position, check if there are any missing values ​​on the matrix edges due to coordinate out-of-bounds errors. For the first row... Missing pixels, copy to the next line Fill the corresponding column with pixel values, and for the last row... For missing pixels, copy the pixel value of the corresponding column in the previous row. For missing pixels in the first and last columns, use the same neighboring column copying strategy to ensure the matrix... All elements are effectively filled without holes, resulting in a pixel grayscale distribution matrix.

[0021] The steps for obtaining the spatial distribution features of road surface texture are as follows: Based on the pixel grayscale distribution matrix, adjacent pixel pair index sequences are constructed in the horizontal, vertical and diagonal directions respectively. The grayscale co-occurrence contrast value of adjacent pixel pairs is calculated for each pair and the row and column coordinates are recorded. The grayscale co-occurrence contrast value is written into the corresponding pixel grid of the spatial domain texture feature layer according to the recorded coordinates and the positions of the multi-directional results are merged to obtain the spatial distribution features of the road surface texture.

[0022] Specifically, based on the pixel grayscale distribution matrix, a set of offsets for four spatial scanning directions is defined. Each pixel within the matrix is ​​traversed in the horizontal, vertical, right diagonal, and left diagonal directions, respectively. ,in For row coordinates, For column coordinates, with respect to the set Each direction vector in Locate adjacent pixels Calculate the local gray-level co-occurrence contrast component in the current direction. The calculation formula is: ,in Representing the One direction, Let be the numerical value of the pixel grayscale distribution matrix, and let the neighborhood coordinates be... If it exceeds the matrix range, then the component in that direction... Set to 0, the corresponding coordinates of the spatial domain texture feature layer. Establish feature vectors at the location, and calculate the features from the four directions. Perform a weighted summation and merging; the merging formula is as follows: ,in Assign weights to the horizontal and vertical directions for each direction. Diagonal weights This weight is set based on the dominance of longitudinal and lateral road surface changes during wheelchair movement. After traversing all pixels to complete the calculation, it will be... The matrix is ​​used as the final output to obtain the spatial distribution characteristics of the road surface texture.

[0023] The steps for obtaining the centroid frequency and spectral width are as follows: Based on the spatial distribution characteristics of road surface texture, the texture intensity value corresponding to each pixel position is read in row and column order. The texture intensity value sequence is transformed on the frequency axis to obtain the frequency component and the corresponding amplitude spectrum. All amplitude spectra are arranged from low to high frequency and the upper frequency limit is recorded to form the frequency domain spectrum coordinate system mapping result. Based on the frequency domain spectral coordinate system mapping results, the weighted center frequency of the spectral energy is calculated. Using the energy of each frequency point as the weight, the centroid frequency of the spectrum is obtained. Then, the standard deviation of the spectral energy distribution is calculated with the centroid frequency of the spectrum as the center, and the spectral width is obtained.

[0024] Specifically, based on the spatial distribution characteristics of the road surface texture, a complex container is initialized to store the frequency-domain transformed data, and the sampling frequency is set. The resolution should be twice the image texture detail resolution to satisfy the sampling theorem, for example, based on the number of pixels per millimeter of the camera. At a rate of 128 Hz / meter, the spatial distribution feature matrix of road surface texture is traversed in row-major order. The two-dimensional texture intensity values ​​are expanded into a one-dimensional time-series signal. To reduce the impact of spectral leakage on subsequent analysis, windowing is applied to the sequence signal. The Hanning window function is used to convolve with the signal sequence. The coefficients of this window function are generated based on the cosine function to smooth the signal edges. The windowed sequence is then input into the Fast Fourier Transform (FFT) algorithm logic, where a butterfly operation is performed to convert the time-domain signal into a frequency-domain complex signal. The real and imaginary parts are extracted from the calculation results, and the Euclidean norms of the real and imaginary parts are calculated to obtain the amplitude spectrum at each frequency point. To eliminate DC leakage... To mitigate interference, the amplitude value at the zero-frequency position is forcibly set to zero. Based on the frequency resolution, discrete frequency point indices are mapped to actual spatial frequency values. The amplitude spectrum sequence is arranged in ascending order according to the corresponding frequency values ​​from low to high. The cutoff frequency where the energy in the amplitude spectrum drops to a certain proportion of the peak energy is located, or the Nyquist frequency is directly used as the physical limit and recorded as the upper frequency limit. At the same time, a correspondence table between the frequency axis and the amplitude axis is constructed. The discrete points in the frequency interval are completed by interpolation algorithm to ensure the smoothness of the spectrum curve. Finally, the sorted frequency array and the corresponding amplitude array are combined to form the frequency domain spectrum coordinate system mapping result.

[0025] Based on the frequency domain spectral coordinate system mapping results, initialize accumulator variables to store the total energy value, frequency-weighted energy sum, and variance-weighted sum, respectively. Iterate through each frequency point and its corresponding amplitude value in the frequency domain spectral coordinate system, calculating the energy value for each frequency point. This energy value is obtained by squaring the amplitude value. Accumulate the energy values ​​of all frequency points to obtain the total spectral energy. If the total spectral energy is lower than a preset noise threshold, for example, three times the sensor's noise floor energy, it is determined to be an invalid signal and a zero value is output. Otherwise, continue calculating and accumulating the product of frequency and energy, and divide the accumulated result by the total frequency. The spectral energy is used to obtain a weighted value reflecting the dominant frequency components of the texture, which is marked as the centroid frequency of the spectrum. Then, based on this centroid frequency, the spectrum sequence is traversed again, and the square of the deviation value of each frequency point relative to the centroid frequency is calculated. The square of the deviation is multiplied by the corresponding energy value as the dispersion weight. All dispersion weight values ​​are accumulated and divided by the total spectral energy. The square root of the quotient is then performed. This operation is used to quantify the concentration of the frequency distribution. The larger the value, the more chaotic the road texture, and the smaller the value, the more uniform the texture. The calculated statistical results are output to obtain the spectral width.

[0026] The steps for obtaining the road surface visual roughness index are as follows: The visual roughness index of the road surface is calculated based on the centroid frequency and spectral width. The calculation formula is as follows: ; in, The visual roughness index of the road surface. The centroid frequency of the spectrum is obtained by weighted averaging of frequency and energy, and the calculation formula is: ,in For the first Each frequency point, The energy at a frequency point, The spectral width reflects the degree of dispersion in the frequency distribution, and is calculated using the following formula: , This is the upper frequency limit of the frequency domain spectral coordinate system mapping result, used for normalization to eliminate sampling rate differences. It is a very small positive number, used to prevent The case where the denominator approaches zero when the value is close to zero.

[0027] Specifically, the formula for calculating the visual roughness index of road surface combines the frequency center trend and the frequency dispersion to comprehensively evaluate the roughness of the road surface. The first term... The centroid frequency is normalized using the upper frequency limit, reflecting the relative fineness of the road surface texture particles. The smoother the road surface, the higher the texture frequency, and the larger this term. The second term, the logarithmic function, utilizes a variant form of the coefficient of variation. To measure the uniformity of texture distribution, when the road surface is a complex gravel road or grass, the spectral width is... A significant increase leads to an increase in the logarithmic value, thereby improving the roughness index. The output value of this nonlinear mapping design can effectively widen the feature distance between smooth road surfaces (such as tiles and cement) and rough road surfaces (such as gravel roads and grass), thereby enhancing the discrimination sensitivity of obstacle avoidance strategies.

[0028] The steps for obtaining the parameter are as follows: This parameter represents the upper frequency limit of the frequency domain analysis, and its physical meaning is the highest spatial texture frequency that the visual sensor can resolve at the current imaging distance, i.e., the Nyquist frequency. It is obtained based on the horizontal field of view of the intelligent wheelchair visual sensor. (e.g., 90 degrees), image horizontal resolution (e.g., 1920 pixels) and the physical width corresponding to the region of interest for driving. (For example, 1.5 meters), first calculate the pixel density per unit physical width. According to the sampling theorem, the highest resolvable frequency is half the pixel density, calculated using the following formula: For example, when , hour, pixels / meter The value of Hz / meter is a fixed value calculated based on the calibration parameters during the system initialization phase. It serves as the reference denominator for normalization and is used to eliminate scale differences caused by sensors with different resolutions.

[0029] The steps for obtaining the parameter are as follows: This parameter is the centroid frequency of the spectrum, which reflects the main granular characteristics of the road surface texture. Its value is obtained through real-time calculation. First, it is read from the frequency domain spectrum coordinate system mapping result generated in the previous steps. Data at each frequency point ,in For frequency values, The corresponding energy value is obtained through the formula. Calculations show that, for example, when inspecting a section of cement road surface, the low-frequency energy is relatively high, and the calculated... It could be 80 cycles per meter.

[0030] The steps for obtaining the parameter are as follows: This parameter is the spectral width, i.e., the standard deviation of the spectral distribution, reflecting the complexity and disorder of the road surface texture. Its value is also obtained through real-time calculation. Based on this, the spectrum data is traversed again to calculate the degree of deviation of each frequency point from the centroid. The calculation formula is as follows: For artificial floor tiles with regular textures, the energy is concentrated near the fundamental frequency. The numerical value is relatively small, but for messy gravel roads, the energy distribution is wide. The values ​​are relatively large; for example, the value measured here is 40 cycles per meter.

[0031] The steps for obtaining the parameter are as follows: this parameter is a minimal positive protection constant, and has the same characteristics as... Same physical dimensions This is to prevent errors caused by extremely smooth road surfaces (such as mirrored floors) during the calculation process. When the value approaches 0, the denominator becomes ineffective. To avoid inputting 0 to the logarithmic function, this parameter is set as a safe minimum for floating-point precision. This setting is determined by referring to the minimum resolution of single-precision floating-point numbers in the IEEE 754 standard. .

[0032] Calculations based on parameters: Assuming a slightly rough asphalt road surface is detected, the system obtains the following parameter values: Frequency limit (Unit: week / meter); Calculated centroid frequency of the spectrum (Unit: week / meter); Calculated spectral width (Unit: week / meter); Protection constant (Unit: (week / meter)) ); Substitute the above values ​​into the formula to perform the calculation: The first step is to calculate the normalized frequency ratio: ; The second step is to calculate the fractions within the logarithmic terms, paying attention to the fractions in the denominator. The value is extremely small and has almost no effect on the square of 80 (6400): ; The third step is to calculate the logarithmic term: ; Step 4: Calculate the final index: ; The results indicate that the current road surface has a visual roughness index of 0.0279, which is relatively low, suggesting that although the road surface has some texture (due to...), it still exhibits some degree of roughness. (embodied) and a certain roughness (by) (It is reflected), but the overall state is relatively stable, suitable for wheelchairs to travel at medium to high speeds. If the value exceeds 0.1 (for example, on a gravel road), the system will determine it as a bumpy road and trigger a deceleration command.

[0033] The steps for obtaining the discretized coordinate sampling sequence are as follows: Based on the forward-looking road surface navigation image, the boundary coordinates of the area corresponding to the road surface visual roughness index are read, the continuous edge chain outside the boundary is located, and the region growth tracking is performed according to the pixel eight-connectivity rule to obtain the outline of the nearby obstacle. Based on the contour line of the nearby obstacle, samples are taken at fixed pixel intervals along the contour arc length, and the horizontal and vertical pixel coordinates of each sampling point are recorded to form a discretized coordinate sampling sequence.

[0034] Specifically, based on the forward-looking road surface navigation image, the boundary coordinates of the area corresponding to the road surface visual roughness index are read. The Canny edge detection operator is then called to calculate the gradient of the background area outside the boundary coordinates. Two thresholds are set for the gradient magnitude to filter noise, with the higher threshold being... Set the threshold to 1.33 times the median grayscale value of the image pixels, which is the low threshold. The threshold is set to 0.5 times the high threshold. This threshold setting is based on the principle of adaptive statistics to ensure that significant structural edges can be extracted. Morphological closing operations are performed on the detected edge pixels. The edge breaks are filled with circular structural elements with a radius of 3 pixels to connect discontinuous pixel segments. In the preprocessed binary image, the search line starts from the boundary of the road area and searches for adjacent strong edge points according to the pixel 8-connectivity rule. A pixel index linked list is constructed. During the search process, the number of pixels in each connected edge chain is calculated. Short chains with fewer than 50 pixels are regarded as image noise and discarded. The longest continuous edge chain that is closest to the bottom of the image is retained as the main obstacle feature. All pixel coordinates on this edge chain are traversed and stored in a dynamic array in order to obtain the outline of the near-distance obstacle.

[0035] Based on the contour lines of nearby obstacles, a fixed pixel interval is set for discretization sampling. The value is 10 pixels. This value is set based on the real-time requirements of the visual processing unit at the highest driving speed of the intelligent wheelchair and the minimum resolution scale of obstacle features. It can effectively smooth out the small jagged noise on the contour line and preserve the overall shape features. An empty coordinate sequence container is initialized, and the coordinates of the starting point of the contour line are read. The first sampling point is stored in the container, and the cumulative arc length variable is initialized. If the value is 0, then traverse the subsequent pixels one by one according to the pixel order of the outline. Calculate the current pixel Compared to the previous pixel Euclidean distance between ,Will Accumulate to ,when Reaching or exceeding the set fixed pixel interval At that time, the current pixel Horizontal pixel coordinates with vertical pixel coordinates Record as sampling points, and Reset to 0 (or subtract) To maintain cumulative accuracy, repeat the above process until the entire contour line has been traversed. Finally, export all the coordinate points recorded in the container in order to form a discretized coordinate sampling sequence.

[0036] The steps for obtaining the energy integral value of the obstacle profile curvature are as follows: Based on the discretized coordinate sampling sequence, the energy integral value of the obstacle contour curvature is calculated using the following formula: ; in, The energy integral value of the obstacle profile curvature. The number of sampling points in the discretized coordinate sampling sequence. For the first The angle between the outline of the obstacle segment and the horizontal direction is calculated using the coordinates of two adjacent sampling points, and the formula is as follows: ,in For the first The horizontal pixel coordinates of each sampling point For the first The vertical pixel coordinates of each sampling point This indicates the change in the angle between two adjacent contour lines, used to reflect the magnitude of changes in local curvature. This is a locally oriented sensitive term, and its weight increases when the contour direction is close to perpendicular.

[0037] Specifically, in the formula for calculating the energy integral value of obstacle profile curvature, by introducing... The term serves as a weighting factor, making the curvature calculation of the obstacle contour anisotropic. Specifically, when the average direction of the contour line segment is close to the vertical direction (90 degrees or 270 degrees), the weighting factor tends to 2, thereby amplifying the curvature energy value at that point. When the contour line is close to the horizontal, the weighting factor tends to 1. This design aims to give higher obstacle avoidance sensitivity to irregular edges of vertical facades in the field of vision (such as the edges of door frames, table legs, and other hard obstacle boundaries), because vertical edges usually correspond to specific rigid structures in physical space, and drastic changes in their contours often mean complex collision risks.

[0038] The parameter acquisition steps are as follows: This parameter represents the total number of valid sampling points in the discretized coordinate sampling sequence. It is used as a normalization factor to calculate the average energy, eliminating the influence of the total contour length on the energy value. It is obtained by directly reading the length attribute of the discretized coordinate sampling sequence container. For example, in one detection loop, if the system samples and stores 52 coordinate points of the obstacle contour in front, then... It was assigned the value 52.

[0039] and The steps to obtain the parameters are as follows: these two parameters represent the first... The horizontal and vertical coordinates of each sampling point in the image coordinate system are limited by the image resolution, for example... , The method for obtaining it is to traverse the discretized coordinate sampling sequence and sort by index. Directly extract the corresponding coordinate pairs, for example, the recorded value of the 5th sampling point is ,but , These coordinates are the basic data for constructing the geometric features of the outline, directly reflecting the spatial position of the obstacle on the visual plane.

[0040] The steps to obtain the parameter are as follows: this parameter represents the first... The angle of inclination of the contour line segment relative to the horizontal axis, with the physical dimension of radians (rad), and a range of values. The method for obtaining it is to use the arctangent function to sample two adjacent points. and The coordinate difference is calculated using the following formula: ,in It is a two-parameter arctangent function, which can correctly handle the angle sign in each quadrant, for example, for sampling points. and Calculations yielded , ,but rad (approximately 45 degrees).

[0041] Calculations based on parameters: The goal is to extract a local sequence containing 5 key sampling points from an image (i.e., ), used to simulate the contour of an obstacle with curved features: The sequence of sampling point coordinates is as follows: ; (Extend horizontally to the right); (Begins to bend upwards); (The bending intensifies, tending towards verticality); (Reverse to the upper left); The first step is to calculate the angles of each line segment. (Unit: radians): Line segment 1 ( ): rad.

[0042] Line segment 2 ( ): rad.

[0043] Line segment 3 ( ): rad.

[0044] Line segment 4 ( ): rad.

[0045] The second step is to calculate the summation term ( From 1 to ): when hour: angle difference squared ; average angle rad; Weighting Items ; Sub-item energy ; when hour...... The third step is to calculate the average total energy: sum ; Normalization ; This result indicates that the currently detected obstacle profile has a value of approximately 0.8972. The curvature energy integral value is high, which reflects that there is a sharp change in direction of the outline and that some line segments are close to the vertical direction (weight close to 2). This indicates that the edge of the obstacle is complex and may be an irregular rigid object. Based on this, the system will determine that the collision risk level of the area is high and the avoidance distance needs to be increased.

[0046] The steps for obtaining adaptive positioning and obstacle avoidance strategy commands for intelligent wheelchairs are as follows: Based on the road surface visual roughness index, the index column of the wheelchair motor torque control table is read, and the segment is located according to the preset segment boundary. If it falls between adjacent segments, interpolation is performed according to the boundary ratio, and the target torque limit, torque rise slope and torque response delay are output to obtain the output parameters of the wheelchair motor torque control table. The obstacle contour curvature energy integral value is read and classified into levels. The lateral distance range is matched according to the level of the obstacle contour curvature energy integral value. Combined with the imaging ratio, it is converted into the actual lateral distance threshold to generate the lateral avoidance distance threshold. Based on the lateral avoidance distance threshold, the system calls upon the target torque limit, torque rise slope, and torque response delay from the wheelchair motor torque control table output parameters. Depending on whether the lateral avoidance distance threshold is exceeded, the system selects to decelerate, brake, or detour. The system synthesizes the instruction sets of the longitudinal control channel and the lateral control channel to generate intelligent wheelchair adaptive positioning and obstacle avoidance strategy instructions.

[0047] Specifically, based on the road surface visual roughness index, the wheelchair motor torque control table stored in non-volatile memory is retrieved. This table is a two-dimensional mapping matrix containing an index column "roughness level" and three corresponding parameter columns: "maximum output torque percentage," "torque response time constant," and "torque slope limit." The preset segmentation boundaries are set as follows: The roughness index is calculated in real time, corresponding to four working conditions: smooth road surface, slight bumps, moderate roughness, and extremely roughness. Compared with the segment boundary, if Directly call the first-level parameters, i.e., maximum torque 100%, response time 10ms. Calling the fourth-level parameters, i.e., 40% of the maximum torque and a response time of 200ms, if Falling between two boundaries, for example Then, the parameters are located between the second and third level parameters, and the output parameters are calculated using linear interpolation. The calculation formula is as follows: ,in and These are the parameter values ​​corresponding to the low boundary and the high boundary, respectively. and For the corresponding boundary values, for example when At that time, the calculated target torque limit is The same interpolation operation is performed on the torque rise slope and torque response delay, and the three calculated values ​​are encapsulated into a structure to obtain the output parameters of the wheelchair motor torque control table.

[0048] The risk level is determined by reading the energy integral value of the obstacle's contour curvature, and four risk level ranges are set: Level 1 ( Level 2 () Level 3 ) and Level 4 ( These thresholds are determined based on the statistical distribution of curvature energy from a large number of obstacle samples. For example, level one corresponds to a gentle wall surface, and level four corresponds to a sharp table corner. The value determines the risk level. A pre-defined lateral distance matching table is consulted; Level 1 risk corresponds to a basic safety distance. For each level increase, the safety distance increases by 100 millimeters. Therefore, a lateral distance of 500 millimeters is required for level four risk. After obtaining the physical safety distance corresponding to the current level, the camera's intrinsic parameter matrix and imaging scale are used to determine the physical distance. Converted to pixel distance in the image plane, the formula is: ,in The pixel / millimeter ratio is the current imaging plane. For example, at a distance of 1.5 meters from the camera, the ratio is 1.2 pixels / millimeter, so 200 millimeters corresponds to 240 pixels. The converted result is used as a dynamic threshold to generate the lateral avoidance distance threshold.

[0049] Based on the lateral avoidance distance threshold, the minimum lateral distance between the edge of the smart wheelchair's current planned path and the obstacle outline is monitored in real time. ,Will With the generated lateral avoidance distance threshold Perform a comparison, if If the condition is stable, maintain the current longitudinal speed and the original path. and This triggers the deceleration and obstacle avoidance logic, calls the target torque limit value in the wheelchair motor torque control table output parameters, multiplies the speed command in the longitudinal control channel by a coefficient of 0.6, and simultaneously superimposes a reverse repulsive force field component in the lateral control channel, causing the wheelchair to finely adjust away from the obstacle. If an emergency collision risk is identified, the longitudinal speed command is immediately set to zero and the electromagnetic braking is activated. At this time, the torque response delay and slope limit in the parameters are called to execute braking with a smooth deceleration curve to avoid discomfort to the occupants caused by sudden stops. Finally, the adjusted longitudinal speed value, lateral steering angle and braking signal are packaged to generate intelligent wheelchair adaptive positioning and obstacle avoidance strategy commands.

[0050] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for adaptive localization and obstacle avoidance of intelligent wheelchairs based on multi-sensor fusion, characterized in that, Includes the following steps: Based on the visual sensor of the intelligent wheelchair, the forward-looking road navigation image is acquired. The driving region of interest is cropped directly below the forward-looking road navigation image according to the chassis width parameter of the intelligent wheelchair. The pixel grayscale distribution matrix inside the driving region of interest is extracted. The grayscale co-occurrence contrast value between adjacent pixels in the pixel grayscale distribution matrix is ​​calculated to generate the spatial distribution feature of road texture. The data of the spatial distribution characteristics of the road surface texture are transformed from the spatial domain to the frequency domain spectral coordinate system. The spectral centroid frequency and spectral width in the frequency domain spectral coordinate system are extracted. The spectral centroid frequency and spectral width are calculated to quantify the visual roughness index of the road surface. Identify the outline of a near obstacle located outside the boundary of the area corresponding to the visual roughness index of the road surface in the forward-looking road navigation image; perform discretized coordinate sampling on the outline of the near obstacle to obtain a discretized coordinate sampling sequence; and perform calculations on the discretized coordinate sampling sequence to generate the integral value of the obstacle outline curvature energy. The road surface visual roughness index is compared and mapped to the wheelchair motor torque control table, and the obstacle contour curvature energy integral value is associated and matched to the lateral avoidance distance threshold to generate intelligent wheelchair adaptive positioning and obstacle avoidance strategy instructions.

2. The intelligent wheelchair adaptive positioning and obstacle avoidance method based on multi-sensor fusion according to claim 1, characterized in that, The steps for obtaining the pixel grayscale distribution matrix are as follows: Read the forward-looking road navigation image output by the intelligent wheelchair's vision sensor, call the intelligent wheelchair chassis width parameter to convert it into pixel width, locate the rectangular strip directly below the bottom edge of the forward-looking road navigation image with the center column of the imaging coordinate system as the reference, set the left and right boundaries according to the pixel width and limit the vertical height along the bottom edge, crop out the rectangular strip and correct the pixels that exceed the boundary to obtain the region of interest for driving. Based on the region of interest, the gray value of each pixel is mapped to a discrete gray level according to a fixed gray level quantization rule. The gray level is read point by point according to the row index and column index and written to the corresponding matrix position. For pixels missing at the edge, the neighboring pixel copying method is used to fill in the missing pixels to keep the rows and columns intact, thus obtaining the pixel gray level distribution matrix.

3. The intelligent wheelchair adaptive positioning and obstacle avoidance method based on multi-sensor fusion according to claim 1, characterized in that, The steps for obtaining the spatial distribution features of the road surface texture are as follows: Based on the pixel grayscale distribution matrix, adjacent pixel pair index sequences are constructed in the horizontal, vertical and diagonal directions respectively. The grayscale co-occurrence contrast value of adjacent pixel pairs is calculated for each pair and the row and column coordinates are recorded. The grayscale co-occurrence contrast value is written into the corresponding pixel grid of the spatial domain texture feature layer according to the recorded coordinates and the positions of the multi-directional results are merged to obtain the spatial distribution features of the road surface texture.

4. The intelligent wheelchair adaptive positioning and obstacle avoidance method based on multi-sensor fusion according to claim 1, characterized in that, The steps for obtaining the centroid frequency and bandwidth of the spectrum are as follows: Based on the spatial distribution characteristics of the road surface texture, the texture intensity value corresponding to each pixel position is read in row and column order. The texture intensity value sequence is transformed on the frequency axis to obtain the frequency component and the corresponding amplitude spectrum. All amplitude spectra are arranged from low to high frequency and the upper frequency limit is recorded to form the frequency domain spectrum coordinate system mapping result. Based on the frequency domain spectral coordinate system mapping results, the weighted center frequency of the spectral energy is calculated. Using the energy of each frequency point as the weight, the centroid frequency of the spectrum is obtained. Then, the standard deviation of the spectral energy distribution is calculated with the centroid frequency of the spectrum as the center, and the spectral width is obtained.

5. The intelligent wheelchair adaptive positioning and obstacle avoidance method based on multi-sensor fusion according to claim 1, characterized in that, The steps for obtaining the road surface visual roughness index are as follows: Based on the centroid frequency and spectral width of the spectrum, the road surface visual roughness index is calculated using the following formula: ； in, The visual roughness index of the road surface. The centroid frequency of the spectrum is obtained by weighted averaging of frequency and energy, and the calculation formula is: ,in For the first Each frequency point, The energy at a frequency point, The spectral width reflects the degree of dispersion in the frequency distribution, and is calculated using the following formula: , This is the upper frequency limit of the frequency domain spectral coordinate system mapping result, used for normalization to eliminate sampling rate differences. It is a very small positive number, used to prevent The case where the denominator approaches zero when the value is close to zero.

6. The intelligent wheelchair adaptive positioning and obstacle avoidance method based on multi-sensor fusion according to claim 1, characterized in that, The steps for obtaining the discretized coordinate sampling sequence are as follows: Based on the forward-looking road surface navigation image, the boundary coordinates of the area corresponding to the road surface visual roughness index are read, the continuous edge chain outside the boundary is located, and the region growth tracking is performed according to the pixel eight-connectivity rule to obtain the outline of the nearby obstacle. Based on the contour line of the nearby obstacle, samples are taken at fixed pixel intervals along the contour arc length, and the horizontal and vertical pixel coordinates of each sampling point are recorded to form a discretized coordinate sampling sequence.

7. The intelligent wheelchair adaptive positioning and obstacle avoidance method based on multi-sensor fusion according to claim 1, characterized in that, The steps for obtaining the energy integral value of the obstacle contour curvature are as follows: Based on the discretized coordinate sampling sequence, the energy integral value of the obstacle contour curvature is calculated using the following formula: ； in, The energy integral value of the obstacle profile curvature. The number of sampling points in the discretized coordinate sampling sequence. For the first The angle between the outline of the obstacle segment and the horizontal direction is calculated using the coordinates of two adjacent sampling points, and the formula is as follows: ,in For the first The horizontal pixel coordinates of each sampling point For the first The vertical pixel coordinates of each sampling point This indicates the change in the angle between two adjacent contour lines, used to reflect the magnitude of changes in local curvature. This is a locally oriented sensitive term, and its weight increases when the contour direction is close to perpendicular.

8. The intelligent wheelchair adaptive positioning and obstacle avoidance method based on multi-sensor fusion according to claim 1, characterized in that, The steps for obtaining the intelligent wheelchair adaptive positioning and obstacle avoidance strategy command are as follows: Based on the road surface visual roughness index, the index column of the wheelchair motor torque control table is read, and the segment is located according to the preset segment boundary. If it falls between adjacent segments, interpolation is performed according to the boundary ratio, and the target torque limit, torque rise slope and torque response delay are output to obtain the output parameters of the wheelchair motor torque control table. The obstacle contour curvature energy integral value is read into a level classification, and a lateral distance interval is matched according to the level of the obstacle contour curvature energy integral value. Combined with the imaging ratio, it is converted into an actual lateral distance threshold to generate a lateral avoidance distance threshold. Based on the lateral obstacle avoidance distance threshold, the system calls the target torque limit, torque rise slope, and torque response delay from the output parameters of the wheelchair motor torque control table. Depending on whether the lateral obstacle avoidance distance threshold is exceeded, the system selects to decelerate, brake, or detour. The system synthesizes the instruction sets of the longitudinal control channel and the lateral control channel to generate intelligent wheelchair adaptive positioning and obstacle avoidance strategy instructions.

9. The system of intelligent wheelchair adaptive positioning and obstacle avoidance method based on multi-sensor fusion according to any one of claims 1-8, characterized in that, include: The texture feature extraction module is used to acquire a forward-looking road navigation image based on the vision sensor of the smart wheelchair, extract the driving region of interest directly below the forward-looking road navigation image based on the chassis width parameter of the smart wheelchair, extract the pixel grayscale distribution matrix inside the driving region of interest, calculate the grayscale co-occurrence contrast value between adjacent pixels in the pixel grayscale distribution matrix, and generate the road texture spatial distribution feature. The frequency domain roughness quantification module is used to convert the data of the spatial distribution characteristics of the road surface texture from the spatial domain to the frequency domain spectral coordinate system, extract the spectral centroid frequency and spectral width in the frequency domain spectral coordinate system, and perform calculations on the spectral centroid frequency and spectral width to quantify the visual roughness index of the road surface. The obstacle contour analysis module is used to identify the contour lines of nearby obstacles located outside the boundary of the area corresponding to the visual roughness index of the road surface in the forward-looking road navigation image, to perform discretized coordinate sampling on the contour lines of nearby obstacles to obtain a discretized coordinate sampling sequence, and to perform calculations on the discretized coordinate sampling sequence to generate the obstacle contour curvature energy integral value. The adaptive control strategy generation module is used to map the road surface visual roughness index to the wheelchair motor torque control table, associate and match the obstacle contour curvature energy integral value to the lateral avoidance distance threshold, and generate intelligent wheelchair adaptive positioning and obstacle avoidance strategy instructions.

Images