Blind stick environment perception information determination method, device, equipment and medium

Abstract

The application provides an environment perception information determination method, device and equipment of a blind stick and a medium. The method comprises the following steps: acquiring environment images and motion information of the blind stick in different swing directions; determining a registration image in each target swing direction based on the environment images in the target swing direction and the motion information in the target swing direction; and the target swing direction is any swing direction in the different swing directions. The environment perception information is determined based on the registration images, a distance parameter of the blind stick and pixel parallax of the blind stick in different swing directions for a same road point. The application improves the spatial depth information, makes up for the deficiency of a traditional blind stick in spatial perception and improves the independent travel ability and safety of a visually impaired person in a complex environment.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, apparatus, device, and medium for determining environmental perception information using a white cane. Background Technology

[0002] Visually impaired individuals rely heavily on assistive devices to obtain environmental information and ensure their safety during daily outings. Currently, these devices primarily utilize ultrasonic, infrared, or laser rangefinders in white canes or wearable devices, or employ cameras combined with image recognition algorithms to assist visually impaired individuals.

[0003] However, solutions based on ranging sensors can only provide simple distance information and cannot obtain complex spatial structure features. Solutions combining cameras and image recognition algorithms can only collect images of the tip of the cane for recognition and cannot identify the complex spatial structure features of the entire road. Therefore, the shortcomings of existing assistive tools in spatial perception reduce the ability and safety of visually impaired people to travel independently in complex environments. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and medium for determining environmental perception information using a white cane. It addresses the shortcomings of existing assistive tools in spatial perception, which reduces the independent travel ability and safety of visually impaired individuals in complex environments. By utilizing multi-angle environmental images and motion information from the white cane's natural swing characteristics, a registration image is determined, improving the viewing angle diversity of the registration image. Then, based on the registration image with diverse viewing angles, distance parameters, and pixel parallax, environmental perception information is determined, enhancing spatial depth information. This compensates for the shortcomings of traditional white canes in spatial perception, improving the independent travel ability and safety of visually impaired individuals in complex environments.

[0005] This invention provides a method for determining environmental perception information using a white cane, comprising the following steps: Acquire environmental images and motion information of the white cane under different swing directions; Based on the environmental image and motion information under each target swing direction, a registration image is determined under each target swing direction; the target swing direction is any swing direction among the different swing directions.

[0006] Environmental perception information is determined based on the registered images, the distance parameters of the cane, and the pixel parallax of the cane for the same road point under different swing directions.

[0007] According to a method for determining environmental perception information using a white cane provided by the present invention, the method for determining environmental perception information based on each registered image, the distance parameter of the white cane, and the pixel parallax of the white cane for the same road point under different swing directions includes: determining the focal length of a monocular camera that acquires the environmental images; determining the feature vector under each target swing direction based on the focal length, the distance parameter, and the pixel parallax; and determining the environmental perception information based on each registered image and the feature vector under each target swing direction.

[0008] According to a method for determining environmental perception information of a white cane provided by the present invention, the step of determining the feature vector under the swing direction of each target based on the focal length, the distance parameter and the pixel disparity includes: determining the depth of each road point corresponding to the white cane based on the focal length, the distance parameter and the pixel disparity; and performing dimensionality reduction processing on the depth of each road point to obtain the feature vector.

[0009] According to a method for determining environmental perception information of a white cane provided by the present invention, the step of determining environmental perception information based on each of the registered images and the feature vectors under the swing direction of each target includes: performing fusion processing on each of the registered images to obtain a fused image; performing fusion processing on the feature vectors under the swing direction of each target to obtain a fused feature vector; and determining the environmental perception information based on the fused image and the fused feature vector.

[0010] According to a method for determining environmental perception information of a white cane provided by the present invention, the step of acquiring environmental images of the white cane under different swing directions includes: acquiring swing information of the white cane through a sensor in the white cane; and acquiring environmental images of the white cane under different swing directions through a monocular camera on the white cane when the swing information meets the image acquisition conditions.

[0011] According to a method for determining environmental perception information of a white cane provided by the present invention, the step of determining a registration image for each target swing direction based on the environmental image under each target swing direction and the motion information under the target swing direction includes: compressing each environmental image to obtain a target environmental image; and performing correlation processing on the target environmental image and the motion information to obtain the registration image.

[0012] According to a method for determining environmental perception information of a white cane provided by the present invention, after determining the environmental perception information, the method further includes: monitoring the environmental perception information; and, if there is a danger perception in the environmental perception information, providing a danger warning to the holder of the white cane.

[0013] The present invention also provides a device for determining environmental perception information of a white cane, comprising the following modules: The acquisition module is used to acquire environmental images and motion information of the white cane under different swing directions; The first determining module is used to determine the registration image under each target swing direction based on the environmental image under each target swing direction and the motion information under the target swing direction; the target swing direction is any swing direction among the different swing directions. The second determining module is used to determine environmental perception information based on each of the registered images, the distance parameters of the white cane, and the pixel parallax of the white cane for the same road point under different swing directions.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the environmental perception information determination method for a white cane as described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for determining environmental perception information of a cane as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the environmental perception information determination method for a white cane as described above.

[0017] This invention provides a method, apparatus, device, and medium for determining environmental perception information using a white cane. It acquires environmental images and motion information of the white cane under different swing directions; based on the environmental images and motion information under each target swing direction, it determines registration images for each target swing direction; the target swing direction can be any of the different swing directions; and based on each registration image, the distance parameter of the white cane, and the pixel parallax of the white cane for the same road point under different swing directions, it determines environmental perception information. Thus, by utilizing multi-angle environmental images and motion information from the white cane under its natural swing characteristics to determine registration images, it improves the visual diversity of the registration images. Then, based on the registration images with diverse visual perspectives, distance parameters, and pixel parallax, it determines environmental perception information, improving spatial depth information and compensating for the shortcomings of traditional white canes in spatial perception, thereby enhancing the independent travel ability and safety of visually impaired individuals in complex environments. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the method for determining environmental perception information using a white cane provided by the present invention.

[0020] Figure 2 This is a schematic diagram of the structure of the environmental perception information determination system for the white cane provided by the present invention.

[0021] Figure 3 This is a schematic diagram of the structure of the environmental perception information determination device for a white cane provided by the present invention.

[0022] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] Currently, traditional white canes provide mechanical tactile feedback to users through direct contact with the ground or obstacles, enabling them to perceive nearby ground obstacles, step edges, and curb structures, playing an irreplaceable role in preventing direct collisions.

[0025] However, with increasingly complex urban environments, the tactile information provided by traditional white canes is no longer sufficient to meet the independent travel needs of visually impaired individuals in complex spaces. The perception method of traditional white canes is essentially point- or line-based short-range tactile perception, and its effective perception range is usually limited to the length of the cane and the ground area it can reach. It cannot provide effective advance warnings for mid- to long-distance environmental information, changes in spatial structure, and potential path selection points. For example, in complex intersections, scenarios with branching paths, or in indoor corridors and elevator lobbies, users often need to make multiple attempts or even stop in place to confirm their direction, affecting both travel efficiency and safety.

[0026] To address the shortcomings of traditional white canes, a series of assistive travel solutions based on electronic sensing or computer vision have emerged in the current technology landscape. For example, ultrasonic, infrared, or laser rangefinders are incorporated into white canes or wearable devices to detect the distance to obstacles ahead; cameras combined with image recognition algorithms are used to analyze the surrounding environment and provide prompts to the user via voice or vibration. However, Solutions based on ranging sensors typically provide only simple distance information, failing to capture complex spatial features such as changes in road width, distribution of traffic space, and the existence of alternative paths. When facing intersections, branching roads, or large obstacles, single distance information is insufficient to support a user's reasonable path judgment. Vision-based assistance solutions often employ head-mounted cameras or require users to hold their phones to take pictures. These solutions often require users to actively aim at the environment or target, a method inconsistent with the natural use of a white cane, increasing operational burden and learning costs, and easily distracting the user while walking, thus affecting safety.

[0027] Furthermore, while binocular cameras or multi-sensor fusion can provide good spatial depth information in visual perception solutions, their high hardware cost, complex structure, and high power consumption make them unsuitable for large-scale application in devices like white canes, which are highly sensitive to size, weight, and energy consumption. Therefore, how to achieve effective perception of the spatial structure of the environment ahead under monocular camera conditions, combined with the movement characteristics of the white cane itself, remains an unsolved technical problem in the current technology.

[0028] Based on the aforementioned problems, this invention provides a method for determining environmental perception information using a white cane. By utilizing multi-angle environmental images and motion information from the white cane under its natural swinging characteristics, a registration image is determined, which improves the viewing angle diversity of the registration image. Then, based on the registration image with diverse viewing angles, distance parameters, and pixel parallax, environmental perception information is determined, which improves spatial depth information, compensates for the shortcomings of traditional white canes in spatial perception, and enhances the independent travel ability and safety of visually impaired individuals in complex environments.

[0029] The following is combined with Figures 1 to 2 The present invention describes a method for determining environmental perception information of a white cane. The subject executing this method can be an electronic device or a device for determining environmental perception information of a white cane installed in the electronic device. The device for determining environmental perception information of a white cane can be implemented by software, hardware, or a combination of both.

[0030] Figure 1 This is a flowchart illustrating the method for determining environmental perception information using a white cane provided by the present invention, as shown below. Figure 1 As shown, the method includes the following: Step 101: Obtain environmental images and motion information of the white cane under different swing directions.

[0031] Here, motion information can include motion information related to the swing direction, swing amplitude, and swing timing. This motion information can be determined using the acceleration, angular velocity, and attitude change information of the cane.

[0032] Optionally, the environmental image can be captured by a camera on the white cane, and the motion information can be captured by a sensor on the white cane.

[0033] Optionally, environmental images and motion information can be acquired at fixed intervals, in real time, or after certain conditions are met.

[0034] Furthermore, acquiring environmental images of the white cane under different swing directions includes: collecting swing information of the white cane through sensors in the white cane; and, when the swing information meets the image acquisition conditions, acquiring environmental images of the white cane under different swing directions through a monocular camera on the white cane.

[0035] Here, the sensor in the white cane can be understood as part of the swing sensing module. The swing information can include one or more of the swing amplitude, swing direction and swing time. For example, after the swing sensing module detects that the white cane swings to the left or right to meet the image acquisition conditions, it triggers the monocular camera to acquire environmental images, thereby obtaining environmental images with different perspectives in different swing directions.

[0036] Here, the image acquisition conditions can be different conditions designed for different road conditions. For example, in an open passage, the swing amplitude threshold can be increased to reduce the image acquisition frequency. In an environment with many obstacles or a narrow space, the threshold can be decreased to increase the acquisition frequency. In addition, the image acquisition conditions can also include filtering, integration, etc.

[0037] In this embodiment of the invention, environmental images of the white cane under different swing directions are captured only when the swing information meets the image acquisition conditions. For example, in open passages, the swing amplitude threshold can be increased to reduce the image acquisition frequency, thereby saving energy; in environments with many obstacles or narrow spaces, the threshold can be reduced to increase the acquisition frequency and ensure the accuracy of environmental perception. Furthermore, using a monocular camera avoids the cost, size, and power consumption problems associated with binocular cameras or complex multi-sensor systems. Step 102: Based on the environmental image under each target swing direction and the motion information under each target swing direction, determine the registration image under each target swing direction.

[0038] Wherein, the target swing direction is any of the different swing directions.

[0039] Here, the registered image is obtained by registering the environmental image and motion information in the same swing direction.

[0040] It should be noted that each environmental image has corresponding motion information. However, the environmental image and motion information are acquired using different acquisition devices. Therefore, the environmental image and motion image can be paired to obtain the environmental image and motion information under the target's swing direction. The pairing method can be based on determining the similarity between the features in the environmental image and the features in the motion information.

[0041] Optionally, the registration image can be determined directly based on the acquired original environmental image and motion information, or the acquired original environmental image can be compressed first, and then the registration image can be determined based on the compressed environmental image and motion information.

[0042] Furthermore, determining the registration image for each target swing direction based on the environmental image and the motion information for each target swing direction includes: compressing each environmental image to obtain a target environmental image; and performing correlation processing on the target environmental image and the motion information to obtain the registration image.

[0043] Optionally, compression processing includes, but is not limited to, image cropping, resolution compression, key region filtering, feature simplification, or frame filtering.

[0044] In this embodiment of the invention, after performing compression processing such as image cropping, resolution compression, key region filtering, feature simplification, or frame filtering on the acquired environmental image, a target environmental image is obtained. The target environmental image and motion information are then associated to obtain a registration image.

[0045] For example, the environmental image acquired by the left swing is associated with the motion information of the left swing, and the environmental image acquired by the right swing is associated with the motion information of the right swing.

[0046] Optionally, the registered image can be an image obtained by associating the target environment image with motion information, or it can be obtained by transforming the associating image.

[0047] For example, the registration image is calculated using the following formula (1): (1) in, Indicates the registered image. Represents pixel coordinates, Represents the transformation matrix. This represents an image that has been associated with environmental and operational information. Registration ensures that images with different oscillation directions are spatially aligned, providing an accurate basis for depth estimation.

[0048] In this embodiment of the invention, by compressing the environmental image, the amount of data is reduced as much as possible while ensuring the integrity of the environmental structure information, thereby improving the efficiency of subsequent processing and transmission.

[0049] Step 103: Determine environmental perception information based on each of the registered images, the distance parameters of the white cane, and the pixel parallax of the white cane for the same road point under different swing directions.

[0050] Here, the parameter distance can be the swing distance of the cane.

[0051] Here, pixel parallax refers to the horizontal pixel difference between the same road point in the left and right images.

[0052] Optionally, the environmental perception information can be calculated by substituting the registered image, distance parameters, and pixel disparity into the formula, or it can be obtained by substituting the registered image, distance parameters, and pixel disparity into the network model.

[0053] Here, environmental perception information is used to characterize the spatial features of the road ahead, including the distance to obstacles, the width of obstacles, and the width of the road.

[0054] In this embodiment of the invention, by utilizing multi-angle environmental images and motion information under the natural swing characteristics of the white cane, a registration image is determined, which improves the perspective diversity of the registration image. Then, based on the registration image with perspective diversity, distance parameters, and pixel parallax, environmental perception information is determined, which improves spatial depth information, makes up for the shortcomings of traditional white canes in spatial perception, and improves the ability and safety of visually impaired people to travel independently in complex environments.

[0055] Furthermore, determining environmental perception information based on each of the registered images, the distance parameter of the white cane, and the pixel parallax of the white cane for the same road point under different swing directions includes: determining the focal length of the monocular camera that acquires the environmental images; determining the feature vector under each of the target swing directions based on the focal length, the distance parameter, and the pixel parallax; and determining environmental perception information based on each of the registered images and the feature vector under each of the target swing directions.

[0056] Here, the feature vector is used to represent the depth, i.e., distance or width, of the road point ahead in the swing direction.

[0057] Optionally, the feature vector can be obtained by first calculating the depth matrix of road points based on the focal length, distance parameters, and pixel disparity, and then obtaining the feature vector based on the depth matrix; or it can be obtained by directly inputting the focal length, distance parameters, and pixel disparity into the network model.

[0058] Here, environmental perception information can be the spatial features of the complete road ahead obtained by fusing the registered image and the feature vectors under the swing direction of each target.

[0059] For example, environmental perception information may include road depth images, which are calculated using the following formula (2): (2) in, This indicates a road with depth in the image. The feature vector representing the leftward swing direction, The feature vector representing the direction of the rightward swing. Indicates the registered image. Let be the feature fusion function, where It can be a weighted average or a deep convolutional network.

[0060] In this embodiment of the invention, feature vectors are determined by focal length, distance parameters, and pixel parallax to reflect the spatial distribution of the roadway ahead and the location of potential obstacles.

[0061] Furthermore, determining the feature vector for each target swing direction based on the focal length, the distance parameter, and the pixel disparity includes: determining the depth of each road point corresponding to the white cane based on the focal length, the distance parameter, and the pixel disparity; and performing dimensionality reduction processing on the depth of each road point to obtain the feature vector.

[0062] Optionally, the depth of the road point can be obtained directly by inputting the focal length, distance parameters, and pixel disparity into the neural network, or it can be calculated by substituting the focal length, distance parameters, and pixel disparity into the formula.

[0063] For example, the depth of a road point is calculated using the following formula (3): (3) in, Indicates the depth of a road point. Indicates focal length. Represents the distance parameter. This represents pixel parallax.

[0064] Furthermore, by calculating the depth matrix point by point in the key regions of the image, a depth matrix can be generated. For the depth matrix Dimensionality reduction is performed to obtain feature vectors that contain motion information.

[0065] Optionally, dimensionality reduction can be achieved through downsampling, key region extraction, or principal component analysis.

[0066] For example, the feature vector is calculated using the following formula (4): (4) in, Represents the eigenvector. Represents the depth matrix of road points. This represents the dimensionality reduction function.

[0067] In this embodiment of the invention, by performing dimensionality reduction processing on the depth of each road point, the amount of wireless transmission data is reduced, the amount of data is effectively compressed, and road traffic information is retained.

[0068] Furthermore, determining the environmental perception information based on each of the registered images and the feature vectors under each of the target swing directions includes: performing fusion processing on each of the registered images to obtain a fused image; performing fusion processing on the feature vectors under each of the target swing directions to obtain a fused feature vector; and determining the environmental perception information based on the fused image and the fused feature vector.

[0069] It should be noted that the registered image includes registered images in different swing directions. Here, the registered images in different swing directions are fused to obtain the fused image of the entire road in front. Similarly, the feature vector includes feature vectors in different swing directions. The feature vectors in different swing directions are fused to obtain the fused feature vector of the entire road in front.

[0070] After obtaining the fused image and the fused feature vector, the fused image and the fused feature vector are fused together to obtain environmental perception information.

[0071] In this embodiment of the invention, environmental perception information is generated by combining feature vectors from different perspectives and environmental images for fusion analysis, ensuring the integrity and continuity of environmental spatial information.

[0072] Furthermore, after determining the environmental perception information, the method further includes: monitoring the environmental perception information; and, if there is a danger perception in the environmental perception information, providing a danger warning to the holder of the white cane.

[0073] Here, the danger warning can be an interactive warning that combines voice and vibration, or other interactive methods. For example, when the visually impaired person is at a certain distance from the danger perception point, they can be prompted by voice to indicate the current distance. When they are getting closer to the danger perception point, they can be prompted by vibration.

[0074] In this embodiment of the invention, by monitoring the danger perception in the environmental perception information, the system provides users with intuitive, continuous, and accurate spatial perception of the environment ahead, thereby improving the safety and efficiency of independent travel for visually impaired individuals.

[0075] In this embodiment of the invention, the method for determining the environmental perception information of the white cane can be implemented individually by multiple modules and processors in the white cane, or it can be implemented jointly by the white cane and the mobile terminal.

[0076] It should be noted that if a white cane and a mobile device are used... Figure 2 This is a schematic diagram of the environmental perception information determination system for a white cane provided by the present invention, as shown below. Figure 2 As shown, the environmental perception information determination system for a white cane includes a white cane 210 and a mobile terminal 220. The white cane 210 includes a monocular camera 211, a wireless communication module 212, a swing sensing module 213, a data processing module 214, and a power supply module 215. The monocular camera 211 is oriented in the same direction as the user's movement and is used to acquire images of the environment ahead. The swing sensing module 213 consists of a three-axis accelerometer and a three-axis gyroscope, used to acquire swing information. When the swing information meets the image acquisition conditions, the monocular camera is triggered to acquire an image. The data processing module 214 is used to determine the registered images and feature vectors. The power supply module 215 is used to power the white cane. The wireless communication module 212 is used to transmit the registered images and feature vectors to the mobile terminal. The mobile terminal is used to determine environmental perception information based on each registered image and the feature vectors under each target swing direction.

[0077] In practical use, visually impaired individuals naturally swing their canes left and right while walking to detect road conditions ahead. Utilizing this natural swinging behavior, when the swing sensing module detects that the cane is swinging left or right to a preset condition, it triggers a monocular camera to capture images, thus obtaining environmental images with different perspectives in different swing directions. By associating the captured images with the corresponding swing direction information, multi-view perception information reflecting the spatial distribution of the environment ahead can be obtained without increasing the user's operational burden.

[0078] To reduce the overall computing power requirements and communication burden of the system, the image data collected at the cane end is compressed. Compression processing includes, but is not limited to, image cropping, resolution compression, key region filtering, feature simplification, or frame filtering, to reduce the amount of data to be transmitted while ensuring the effectiveness of the information needed for environmental structure perception. The pre-processed image data or feature data is transmitted to the mobile terminal via a wireless communication module. The mobile terminal then combines multi-view information from different swing directions to further analyze the distribution of the passage space, changes in road width, or potential structural changes in the environment ahead.

[0079] By utilizing the multi-view information generated by the natural swinging of a white cane, this invention achieves effective perception of the spatial structure of the environment ahead under monocular camera conditions, thereby avoiding the cost, size, and power consumption problems associated with binocular cameras or complex multi-sensor systems. The perception method, based on the natural swinging sequence during continuous walking, helps improve the consistency and stability of the perception results over time, making the system more suitable for long-term operation in real-world walking scenarios.

[0080] The following describes the device for determining environmental perception information of a white cane provided by the present invention. The device for determining environmental perception information of a white cane described below and the method for determining environmental perception information of a white cane described above can be referred to in correspondence with each other.

[0081] Figure 3 This is a schematic diagram of the environmental perception information determination device for a white cane provided by the present invention, as shown below. Figure 3 As shown, the environmental perception information determination device for the white cane includes the following: The acquisition module 310 is used to acquire environmental images and motion information of the white cane under different swing directions; The first determining module 320 is used to determine a registration image for each target swing direction based on the environmental image under each target swing direction and the motion information under the target swing direction; the target swing direction is any swing direction among the different swing directions. The second determining module 330 is used to determine environmental perception information based on each of the registered images, the distance parameters of the cane, and the pixel parallax of the cane for the same road point under different swing directions.

[0082] In this embodiment of the invention, the second determining module 330 is specifically used for: determining the focal length of the monocular camera that acquires the environmental image; determining the feature vector under each target swing direction based on the focal length, the distance parameter and the pixel parallax; and determining environmental perception information based on each registered image and the feature vector under each target swing direction.

[0083] In this embodiment of the invention, the second determining module 330 is further specifically used to: determine the depth of each road point corresponding to the white cane based on the focal length, the distance parameter and the pixel parallax; and perform dimensionality reduction processing on the depth of each road point to obtain the feature vector.

[0084] In this embodiment of the invention, the second determining module 330 is further specifically used for: performing fusion processing on each of the registered images to obtain a fused image; performing fusion processing on the feature vectors under the swing direction of each target to obtain a fused feature vector; and determining the environmental perception information based on the fused image and the fused feature vector.

[0085] In this embodiment of the invention, the acquisition module 310 is specifically used to: acquire the swing information of the white cane through the sensor in the white cane; and, when the swing information meets the image acquisition conditions, acquire environmental images of the white cane in different swing directions through the monocular camera on the white cane.

[0086] In this embodiment of the invention, the first determining module 320 is specifically used for: compressing each of the environmental images to obtain a target environmental image; and performing correlation processing on the target environmental image and the motion information to obtain the registration image.

[0087] In this embodiment of the invention, after determining the environmental perception information, the environmental perception information determining device for the white cane further includes a prompting module, specifically used for: monitoring the environmental perception information; and, if there is a danger perception in the environmental perception information, providing a danger prompt to the holder of the white cane.

[0088] Figure 4 A schematic diagram of the structure of the electronic device provided by the present invention is shown below. Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a method for determining the environmental perception information of a white cane. This method includes: acquiring environmental images and motion information of the white cane under different swing directions; determining registration images under each target swing direction based on the environmental images and motion information under each target swing direction; the target swing direction being any of the different swing directions; and determining environmental perception information based on each registration image, the distance parameter of the white cane, and the pixel parallax of the white cane for the same road point under different swing directions.

[0089] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0090] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the environmental perception information determination method for a white cane provided by the above methods. The method includes: acquiring environmental images and motion information of the white cane under different swing directions; determining registration images under each target swing direction based on the environmental images and motion information under each target swing direction; the target swing direction being any of the different swing directions; and determining environmental perception information based on each registration image, the distance parameter of the white cane, and the pixel parallax of the white cane for the same road point under different swing directions.

[0091] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a method for determining environmental perception information of a white cane provided by the methods described above. This method includes: acquiring environmental images and motion information of the white cane under different swing directions; determining registration images under each target swing direction based on the environmental images and motion information under each target swing direction; wherein the target swing direction is any of the different swing directions; and determining environmental perception information based on each registration image, the distance parameter of the white cane, and the pixel parallax of the white cane for the same road point under different swing directions.

[0092] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0093] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining environmental perception information using a white cane, characterized in that, include: Acquire environmental images and motion information of the white cane under different swing directions; Based on the environmental image and motion information under each target swing direction, a registration image is determined under each target swing direction; the target swing direction is any swing direction among the different swing directions. Environmental perception information is determined based on the registered images, the distance parameters of the cane, and the pixel parallax of the cane for the same road point under different swing directions.

2. The method for determining environmental perception information using a white cane according to claim 1, characterized in that, The determination of environmental perception information based on the registered images, the distance parameters of the cane, and the pixel disparity of the cane for the same road point under different swing directions includes: Determine the focal length of the monocular camera used to capture the environmental images; Based on the focal length, the distance parameter, and the pixel disparity, determine the feature vector under each target swing direction; Environmental perception information is determined based on the registered images and the feature vectors of the target swing directions.

3. The method for determining environmental perception information using a white cane according to claim 2, characterized in that, The step of determining the feature vector for each target's swing direction based on the focal length, the distance parameter, and the pixel disparity includes: Based on the focal length, the distance parameter, and the pixel parallax, the depth of each road point corresponding to the white cane is determined; The depth of each road point is reduced in dimensionality to obtain the feature vector.

4. The method for determining environmental perception information using a white cane according to claim 2, characterized in that, The determination of environmental perception information based on the registered images and feature vectors under the swing direction of each target includes: The registered images are fused together to obtain a fused image; The feature vectors of each target swing direction are fused to obtain a fused feature vector. The environmental perception information is determined based on the fused image and the fused feature vector.

5. The method for determining environmental perception information of a white cane according to any one of claims 1 to 4, characterized in that, The acquisition of environmental images of the white cane under different swing directions includes: The swing information of the white cane is collected by the sensor in the white cane; When the swing information meets the image acquisition conditions, the environment images of the white cane under different swing directions are acquired by the monocular camera on the white cane.

6. The method for determining environmental perception information of a white cane according to any one of claims 1 to 4, characterized in that, The process of determining the registration image for each target swing direction based on the environmental image and the motion information for each target swing direction includes: The environmental images are compressed to obtain the target environment image; The target environment image and the motion information are correlated to obtain the registered image.

7. The method for determining environmental perception information using a white cane according to claim 1, characterized in that, After determining the environmental perception information, the method further includes: Monitor the environmental sensing information; If a danger is detected in the environmental perception information, a danger warning is given to the holder of the white cane.

8. A device for determining environmental perception information of a white cane, characterized in that, include: The acquisition module is used to acquire environmental images and motion information of the white cane under different swing directions; The first determining module is used to determine the registration image under each target swing direction based on the environmental image under each target swing direction and the motion information under the target swing direction; the target swing direction is any swing direction among the different swing directions. The second determining module is used to determine environmental perception information based on each of the registered images, the distance parameters of the white cane, and the pixel parallax of the white cane for the same road point under different swing directions.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method for determining environmental perception information of a white cane as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for determining environmental perception information of a white cane as described in any one of claims 1 to 7.

Images