Obstacle avoidance method and system for visually impaired persons based on stereoscopic vision

Abstract

The present application relates to a method and system for obstacle avoidance for visually impaired persons based on stereoscopic vision. The system includes: an image acquisition module for acquiring environmental image data collected by a stereoscopic vision device; a data processing module for generating obstacle information based on the environmental image data, wherein the obstacle information includes obstacle type and obstacle size; and an obstacle avoidance planning module for planning an obstacle avoidance plan based on the obstacle information. The present application can use stereoscopic vision to identify obstacles, and plan an avoidance plan based on the obstacle information, making the obstacle avoidance plan more reasonable, thereby improving the convenience of travel for visually impaired persons while ensuring the safety of their travel.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of stereoscopic vision technology, and more specifically, to a method and system for visually impaired persons to avoid obstacles based on stereoscopic vision. Background Art

[0002] Blind people's travel is a hot social issue that needs urgent attention. Visually impaired people are eager to travel independently. In order to eliminate safety hazards during their travel, they need to safely avoid obstacles during their travel. Summary of the invention

[0003] The present application provides a method, system and computer-readable storage medium for obstacle avoidance for visually impaired persons based on stereoscopic vision, which can use stereoscopic vision to identify obstacles and plan avoidance plans based on obstacle information to make the obstacle avoidance plans more reasonable, thereby improving the convenience of travel for visually impaired persons while ensuring the safety of travel for visually impaired persons.

[0004] In a first aspect, a method for visually impaired persons to avoid obstacles based on stereoscopic vision is provided, comprising:

[0005] Acquire environmental image data collected by a stereoscopic vision device;

[0006] Generate obstacle information according to the environment image data, the obstacle information including obstacle type and obstacle size; optionally, the obstacle size includes obstacle height and obstacle width;

[0007] An obstacle avoidance plan is planned according to the obstacle information.

[0008] In combination with the first aspect, in certain embodiments of the first aspect, it also includes: generating an audio signal from the obstacle information and / or obstacle avoidance plan, so that the audio playback device plays a prompt audio after receiving the audio signal.

[0009] In combination with the first aspect, in some implementations of the first aspect, the method further includes: generating a vibration signal according to the obstacle information, so that the vibration device vibrates according to the vibration intensity and vibration frequency indicated by the vibration signal.

[0010] In combination with the first aspect, in certain embodiments of the first aspect, the stereoscopic vision device includes a passive stereo camera or an active stereo camera;

[0011] Optionally, the passive stereo camera is a binocular camera;

[0012] Optionally, the active stereo camera is a binocular line laser camera.

[0013] In combination with the first aspect, in certain embodiments of the first aspect, planning an obstacle avoidance plan based on the obstacle information includes: when the obstacle type is the first type, ignoring the obstacle during walking.

[0014] In conjunction with the first aspect, in certain embodiments of the first aspect, the first type includes grasses and fallen leaves.

[0015] In combination with the first aspect, in certain embodiments of the first aspect, planning an obstacle avoidance plan based on the obstacle information includes: when the obstacle type is the second type and the obstacle height is above the first height, keeping the walking route at least a first distance from the obstacle.

[0016] In combination with the first aspect, in certain embodiments of the first aspect, planning an obstacle avoidance plan based on the obstacle information includes: when the obstacle type is the second type and the obstacle height is less than the first height and the obstacle width is less than the first width, ignoring the obstacle during walking.

[0017] In combination with the first aspect, in certain embodiments of the first aspect, the second type includes stones, bricks, pits, or branches.

[0018] In combination with the first aspect, in certain embodiments of the first aspect, planning an obstacle avoidance plan based on the obstacle information includes: when the obstacle type is the third type, stopping walking at at least a second distance from the obstacle.

[0019] In combination with the first aspect, in certain embodiments of the first aspect, the third type includes pedestrians or moving vehicles.

[0020] In combination with the first aspect, in certain embodiments of the first aspect, planning an obstacle avoidance plan based on the obstacle information includes: when the obstacle type is the fourth type, keeping the walking route at least a third distance from the obstacle.

[0021] In combination with the first aspect, in certain embodiments of the first aspect, the fourth type includes trees, walls, signs, trash cans or vehicles.

[0022] In a second aspect, a system for avoiding obstacles for visually impaired persons based on stereoscopic vision is provided, comprising:

[0023] An image acquisition module, used to acquire environmental image data collected by a stereoscopic vision device;

[0024] A data processing module, configured to generate obstacle information according to the environment image data, wherein the obstacle information includes obstacle type and obstacle size; optionally, the obstacle size includes obstacle height and obstacle width;

[0025] The obstacle avoidance planning module is used to plan an obstacle avoidance plan based on the obstacle information.

[0026] In combination with the second aspect, in certain embodiments of the second aspect, it also includes: a voice processing module, which is used to generate an audio signal from the obstacle information and / or obstacle avoidance plan, so that the audio playback device plays a prompt audio after receiving the audio signal.

[0027] In combination with the second aspect, in certain embodiments of the second aspect, it further includes: a vibration prompt module, which is used to generate a vibration signal according to the obstacle information, so that the vibration device vibrates according to the vibration intensity and vibration frequency indicated by the vibration signal.

[0028] In conjunction with the second aspect, in certain embodiments of the second aspect, the stereoscopic vision device includes a passive stereo camera or an active stereo camera;

[0029] Optionally, the passive stereo camera is a binocular camera;

[0030] Optionally, the active stereo camera is a binocular line laser camera.

[0031] In conjunction with the second aspect, in certain embodiments of the second aspect, the obstacle avoidance planning module is specifically used to:

[0032] When the obstacle type is the first type, the obstacle is ignored during walking.

[0033] In conjunction with the second aspect, in certain embodiments of the second aspect, the first type includes grasses or fallen leaves.

[0034] In conjunction with the second aspect, in certain embodiments of the second aspect, the obstacle avoidance planning module is specifically used to:

[0035] When the obstacle type is the second type and the obstacle height is above the first height, the walking route is kept at least a first distance from the obstacle.

[0036] In combination with the second aspect, in certain embodiments of the second aspect, the obstacle avoidance planning module is specifically used to: when the obstacle type is the second type and the obstacle height is less than the first height and the obstacle width is less than the first width, ignore the obstacle during walking.

[0037] In conjunction with the second aspect, in certain embodiments of the second aspect, the second type includes stones, bricks, pits, or branches.

[0038] In conjunction with the second aspect, in certain embodiments of the second aspect, the obstacle avoidance planning module is specifically used to:

[0039] When the obstacle type is the third type, the walking is stopped at a distance of at least the second distance from the obstacle.

[0040] In combination with the second aspect, in certain embodiments of the second aspect, the third type includes pedestrians or moving vehicles.

[0041] In conjunction with the second aspect, in certain embodiments of the second aspect, the obstacle avoidance planning module is specifically used to:

[0042] When the obstacle type is the fourth type, the walking route is kept at least a third distance from the obstacle.

[0043] In combination with the second aspect, in certain embodiments of the second aspect, the fourth type includes trees, walls, signs, trash cans, or vehicles.

[0044] In a third aspect, an intelligent obstacle avoidance system for visually impaired persons is provided, comprising a processor and a memory, wherein the processor and the memory are connected, wherein the memory is used to store program code, and the processor is used to call the program code to execute a method as described in any one of the first aspects.

[0045] According to a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present invention, but are not intended to limit the present invention.

[0047] Figure 1 is a flowchart of a method for avoiding obstacles for visually impaired persons based on stereoscopic vision provided by at least one embodiment of the present application;

[0048] Figure 2 is a schematic structural diagram of a visually impaired obstacle avoidance system based on stereoscopic vision provided by at least one embodiment of the present application;

[0049] Figure 3 It is a structural schematic diagram of an intelligent obstacle avoidance system for visually impaired persons provided in at least one embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0051] In the description of the present application, the terms "first", "second", "third", etc. are used to distinguish different objects and do not indicate any order or importance.

[0052] In the description of the present application, “plurality” means two or more than two, unless otherwise clearly and specifically defined.

[0053] In the description of this application, the term "and / or" is merely a term used to describe the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist at the same time, and B exists alone.

[0054] In the description of this application, words such as "connection" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0055] In the description of this application, "up", "down", "left", "right", etc. are only used to indicate relative position relationships. When the absolute position of the described object changes, the relative position relationship may also change accordingly.

[0056] It should be noted that, in the absence of conflict, the features in the following embodiments and implementations may be combined with each other.

[0057] Before introducing the solution of this application, the relevant concepts involved in this application are first introduced:

[0058] Stereo vision: A technology that imitates the human eye, perceives scene depth by acquiring and analyzing images from two or more cameras, and calculates the three-dimensional information of the target object.

[0059] Passive stereo camera: A stereo vision hardware device that uses two-dimensional images of natural light in a scene to reconstruct three-dimensional information of target objects.

[0060] Active stereo camera: A stereo vision hardware device that controls the light source or changes the angle at which the camera captures the target, and then measures the three-dimensional information of the object based on the information reflected from the target surface.

[0061] As mentioned above, visually impaired people need to safely avoid obstacles during travel. During travel, there may be various obstacles on the route. Most current obstacle avoidance solutions only determine whether there are obstacles on the road, but do not further analyze the obstacles. Therefore, when using existing solutions for obstacle avoidance, the following situations may occur:

[0062] When there are objects on the road that will not cause obstacles to the visually impaired, such as soft weeds or fallen leaves, the visually impaired do not need to avoid such obstacles and do not need to change their routes. However, when the existing solution identifies the object, it does not consider the actual impact of the object and still notifies the visually impaired to avoid it. This approach obviously affects the convenience of visually impaired people's travel.

[0063] The present application provides an obstacle avoidance method for visually impaired persons based on stereoscopic vision, the method comprising: acquiring environmental image data collected by a stereoscopic vision device; generating obstacle information based on the environmental image data, the obstacle information including obstacle type and obstacle size; and planning an obstacle avoidance plan based on the obstacle information.

[0064] In the technical solution provided in the present application, obstacles are identified using stereoscopic vision, and avoidance plans are planned based on obstacle information, making the obstacle avoidance plan more reasonable, thereby improving the convenience of travel for the visually impaired while ensuring their safety.

[0065] It should be understood that the execution subject of the obstacle avoidance method for visually impaired persons based on stereoscopic vision provided in the embodiment of the present application can be an electronic device, and the electronic device can be a computing device, a terminal device or a server, wherein the terminal device can be various personal computers, laptops, smart phones, tablet computers and portable wearable devices, etc., and the present application does not make specific limitations. The electronic device can also be a processor or a processing chip. When the electronic device is a processor or a processing chip, the electronic device can be integrated into the stereoscopic vision device.

[0066] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0067] Figure 1 1 is a flow chart of a method for avoiding obstacles for visually impaired persons based on stereoscopic vision provided by at least one embodiment of the present application. Figure 1 , the method comprises the following steps:

[0068] S11: Acquire environmental image data collected by the stereoscopic vision device.

[0069] The stereoscopic vision device can collect environmental images and generate environmental image data based on the collected environmental images. The environmental image data includes two-dimensional images, depth images and point cloud data.

[0070] The stereoscopic vision device can collect two-dimensional images and / or depth images. The above two-dimensional images can include grayscale images and can also include color images. Each pixel in the above color image can include pixel values ​​of three channels: R (Red), G (Green), and B (Blue).

[0071] The stereoscopic vision device can calculate point cloud data based on the captured two-dimensional image and / or depth image of the scene.

[0072] Point cloud data can be a collection of data of a large number of points of surface characteristics of at least one object in a scene. Each point in the point cloud data can correspond to multiple data, such as the three-dimensional coordinates, color and other data of the point. Point cloud data can reflect the real information of the surface of objects in the scene with high accuracy.

[0073] The stereo vision device may be a passive stereo camera, such as a binocular camera, or an active stereo camera, such as a binocular line laser camera.

[0074] S12: Generate obstacle information according to the environment image data, where the obstacle information includes obstacle type, obstacle height, and obstacle size.

[0075] The stereoscopic vision device may be a conventional stereo camera. At this time, the stereoscopic vision device transmits the collected environmental image data to an electronic device communicating with it for image data processing, thereby generating obstacle information.

[0076] In some embodiments, the stereoscopic vision device may be an intelligent stereo camera. Compared with a traditional stereo camera, the intelligent stereo camera can process the environmental image data at the camera end to generate obstacle information.

[0077] The above-mentioned intelligent stereo camera includes a processor or a processing chip, and the processor or the processing chip is used to generate obstacle information according to the environmental image data.

[0078] It can be understood that obstacle information is generated based on the environment image data through the existing target detection algorithm.

[0079] S13: Plan an obstacle avoidance plan based on obstacle information.

[0080] The previous steps obtain the obstacle type and size. This step plans the obstacle avoidance plan based on the obstacle information, realizing differentiated obstacle avoidance for different obstacles.

[0081] In some embodiments, when the obstacle type is the first type, the obstacle is ignored during walking.

[0082] Among them, the first type may include grass and fallen leaves. The first category may correspond to soft and short obstacles on the ground. During the movement of the visually impaired person, such obstacles on the ground will not affect the safety of the visually impaired person. Therefore, when it is determined that the obstacle type belongs to the first type, such obstacles can be directly ignored during the movement of the visually impaired person, which can improve the convenience of the visually impaired person's travel.

[0083] It should be understood that “ignore obstacles” means not changing the walking route.

[0084] In some embodiments, when the obstacle type is the second type, the obstacle size (the obstacle size includes the obstacle height and the obstacle width) is comprehensively considered to determine the obstacle avoidance solution, as follows:

[0085] When the obstacle type is the second type and the obstacle height is above the first height, the walking route is kept at least at a first distance from the obstacle.

[0086] When the obstacle type is the second type, the obstacle height is less than the first height, and the obstacle width is less than the first width, the obstacle is ignored during walking.

[0087] Among them, the second type may include stones, bricks, pits or branches. The second type may correspond to harder obstacles on the ground. If the height of such obstacles is not high and the width is not large, the visually impaired person can directly step over the obstacle during the journey without changing the walking route. However, if the height of the obstacle is high, the safety risk of the visually impaired person directly stepping over the obstacle is high. Therefore, when the obstacle type is the second type and the obstacle height is above the first height, the obstacle avoidance solution is to keep the walking route at least the first distance from the obstacle.

[0088] It should be understood that when the obstacle is a pit or other sunken obstacle, the obstacle height can be understood as the vertical height from the lowest point of the obstacle to the horizontal ground. When the obstacle is a stone or other protruding obstacle, the obstacle height can be understood as the vertical height from the highest point of the obstacle to the horizontal ground.

[0089] The first height and the first width can be determined according to the height of the visually impaired person, and more specifically, can be determined according to the pace of the visually impaired person, so as to fully ensure the safety of the visually impaired person. Therefore, the values ​​of the first height and the first width can be adjusted. For example, when the visually impaired person is taller, the values ​​of the first height and the first width can be set larger, and when the visually impaired person is shorter, the values ​​of the first height and the first width can be set smaller. For example, the first height can be 10cm, 15cm, 20cm, 25cm, 30cm, etc. For example, the first width can be 10cm, 15cm, 20cm, 25cm, 30cm, etc.

[0090] Since the second type of obstacle corresponds to a harder obstacle on the ground, when its height is high, if a certain distance is not maintained from it, the safety risk of the visually impaired person will be greatly increased. Therefore, for safety considerations, when the obstacle type is the second type and the obstacle height is above the first height, the walking route is kept at least at the first distance from the obstacle.

[0091] The first distance is a distance that allows the visually impaired person to be relatively safe from obstacles during the process of traveling, which can be preset and can also be adjusted. For example, the first distance can be 20 cm, 30 cm, 50 cm, 60 cm, etc.

[0092] In some embodiments, when the obstacle type is the third type, walking is stopped at least a second distance from the obstacle.

[0093] The third type corresponds to obstacles in motion, that is, dynamic obstacles, which may include pedestrians or moving vehicles. Since the obstacles are dynamic obstacles, at this time, based on the safety of the visually impaired, the obstacle avoidance solution is to stop walking at least the second distance from the obstacles.

[0094] In some embodiments, the obstacle information also includes obstacle distance.

[0095] In some cases, when an obstacle is detected, if the distance of the obstacle is greater than the second distance, the visually impaired person can walk to a distance at least the second distance from the obstacle and then stop walking.

[0096] In some cases, when an obstacle is detected, if the distance of the obstacle is below the second distance, the visually impaired person can stop walking.

[0097] The second distance is a distance that allows the visually impaired person to be relatively safe from obstacles during the walking process, which can be preset and can also be adjusted. For example, the second distance can be 30cm, 40cm, 50cm, 60cm, 70cm, 80cm, 90cm, 100cm, etc.

[0098] In some embodiments, when the obstacle type is the fourth type, the walking route is kept at least a third distance from the obstacle.

[0099] The fourth type includes trees, walls, signboards, trash cans or vehicles. The fourth type may correspond to relatively large and fixed obstacles.

[0100] The third distance is a distance that allows the visually impaired person to be relatively safe from obstacles during the walking process, which can be preset and can also be adjusted. For example, the third distance can be 30cm, 40cm, 50cm, 60cm, 70cm, 80cm, 90cm, 100cm, etc.

[0101] In some embodiments, the first type, second type, third type and fourth type are not limited to the objects included above, and other reasonable obstacles can be expanded into each category. In some embodiments, the obstacle types can be further expanded and refined, for example, a fifth type, a sixth type, etc. can be added.

[0102] In some embodiments, the obstacle avoidance method for visually impaired persons based on stereoscopic vision further includes the following step: generating an audio signal from obstacle information and / or obstacle avoidance scheme, so that an audio playback device plays a prompt audio after receiving the audio signal.

[0103] In some embodiments, the obstacle avoidance method for visually impaired persons based on stereoscopic vision further comprises the following steps: generating a vibration signal according to obstacle information, so that the vibration device vibrates according to the vibration intensity and vibration frequency indicated by the vibration signal. The visually impaired person can wear the vibration device to obtain obstacle information through the vibration intensity and vibration frequency.

[0104] In some embodiments, there may be only one obstacle on the ground, or there may be multiple obstacles, and these obstacles may be of different types and have different sizes, so obstacle avoidance can be performed according to one or more of the obstacle avoidance schemes determined by the above method.

[0105] In addition, at least one embodiment of the present application further provides a stereoscopic vision-based obstacle avoidance system for visually impaired persons for implementing any of the above methods. Figure 2 , the system 200 comprises:

[0106] The image acquisition module 210 is used to acquire the environmental image data collected by the stereoscopic vision device.

[0107] The stereoscopic vision device can collect environmental images and generate environmental image data based on the collected environmental images. The environmental image data includes two-dimensional images, depth images and point cloud data.

[0108] The stereoscopic vision device can collect two-dimensional images and / or depth images. The above two-dimensional images can include grayscale images and can also include color images. Each pixel in the above color image can include pixel values ​​of three channels: R (Red), G (Green), and B (Blue).

[0109] The stereoscopic vision device can generate point cloud data based on the acquired two-dimensional image and / or depth image of the scene.

[0110] Point cloud data can be a collection of data of a large number of points of surface characteristics of at least one object in a scene. Each point in the point cloud data can correspond to multiple data, such as the three-dimensional coordinates, color and other data of the point. Point cloud data can reflect the real information of the surface of objects in the scene with high accuracy.

[0111] The stereo vision device may be a passive stereo camera, such as a binocular camera, or an active stereo camera, such as a binocular line laser camera.

[0112] The data processing module 220 is used to generate obstacle information according to the environmental image data. The obstacle information includes obstacle type and obstacle size, and may also include obstacle distance.

[0113] The obstacle avoidance planning module 230 is used to plan an obstacle avoidance plan based on obstacle information.

[0114] In some embodiments, the system 200 further includes: a voice processing module for generating an audio signal from obstacle information and / or obstacle avoidance schemes, so that the audio playback device plays a prompt audio after receiving the audio signal.

[0115] In some embodiments, the system 200 further includes: a vibration prompt module, which is used to generate a vibration signal according to the obstacle information, so that the vibration device vibrates according to the vibration intensity and vibration frequency indicated by the vibration signal.

[0116] In some embodiments, the obstacle avoidance planning module is specifically used to:

[0117] When the obstacle type is the first type, the obstacle is ignored during walking.

[0118] In some embodiments, the first type includes grasses and fallen leaves.

[0119] In some embodiments, the obstacle avoidance planning module is specifically used to:

[0120] When the obstacle type is the second type and the obstacle height is above the first height, the walking route is kept at least at a first distance from the obstacle.

[0121] In some embodiments, the obstacle avoidance planning module is specifically used to: when the obstacle type is the second type, the obstacle height is less than the first height, and the obstacle width is less than the first width, ignore the obstacle during walking.

[0122] In some embodiments, the second type includes rocks, bricks, pits, or branches.

[0123] In some embodiments, the obstacle avoidance planning module is specifically used to:

[0124] When the obstacle type is the third type, the walking is stopped at a distance at least the second distance from the obstacle.

[0125] In some embodiments, the third type includes pedestrians or moving vehicles.

[0126] In some embodiments, the obstacle avoidance planning module is specifically used to:

[0127] When the obstacle type is the fourth type, the walking route is kept at least a third distance from the obstacle.

[0128] In some embodiments, the fourth type includes a tree, a wall, a sign, a trash can, or a vehicle.

[0129] At least one embodiment of the present application further provides an intelligent obstacle avoidance device for visually impaired persons based on stereoscopic vision, including a stereoscopic vision device and an electronic device connected to the stereoscopic vision device for communication;

[0130] The stereoscopic vision device is used to collect environmental images and generate environmental image data;

[0131] The electronic device is used to: obtain environmental image data; generate obstacle information based on the environmental image data, the obstacle information including obstacle category, obstacle height and obstacle distance; and plan an obstacle avoidance plan based on the obstacle information.

[0132] In some embodiments, the electronic device is further used to generate an audio signal from the obstacle information and / or obstacle avoidance scheme. At this time, the intelligent obstacle avoidance device for visually impaired persons based on stereoscopic vision also includes an audio playback device for receiving an audio signal and playing a prompt audio.

[0133] In some embodiments, the electronic device is further used to generate a vibration signal according to the obstacle information. In this case, the intelligent obstacle avoidance device for visually impaired persons based on stereoscopic vision also includes a vibration device (such as a vibration motor) for vibrating after receiving the vibration signal.

[0134] As mentioned above, the electronic device may be a processor or a processing chip. When the electronic device is a processor or a processing chip, the electronic device may be integrated into a stereoscopic vision device. In this case, the stereoscopic vision device is an intelligent stereo camera that can directly output an obstacle avoidance solution.

[0135] In the specification, multiple embodiments are described in a progressive manner, and the focus of each embodiment is different, and the same or similar parts between the embodiments can be referred to each other. For the above-mentioned system embodiment, since its overall structure and the modules involved and their functions and effects correspond to the contents in the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0136] At least one embodiment of the present application further provides an intelligent obstacle avoidance system for visually impaired persons, Figure 3 The system includes a processor 320 and a memory 310, wherein the processor 320 and the memory 310 are connected, wherein the memory 310 is used to store program codes, and the processor 320 is used to call program codes to execute a method as described in any of the above embodiments.

[0137] At least one embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. The computer program is executed by a processor to implement a method as described in any of the above embodiments.

[0138] The memory 310 may be a disk, a flash memory, or any other non-volatile storage medium. The memory 310 is used to store instructions in the corresponding embodiment of the obstacle avoidance method for visually impaired persons. The processor 320 is coupled to the memory 310 and may be implemented as one or more integrated circuits, such as a microprocessor or a microcontroller. The processor 320 is used to execute the instructions stored in the memory 310.

[0139] Another embodiment of the present application also provides a visually impaired person obstacle avoidance system. In some embodiments, the visually impaired person obstacle avoidance system includes a memory 310 and a processor 320. The processor 320 is coupled to the memory 310 via a BUS bus. The visually impaired person obstacle avoidance system can also be connected to an external storage device through a storage interface to call external data, and can also be connected to a network or another computer system (not shown) through a network interface. No further detailed description is given here.

[0140] In one aspect of the present application, a computer-readable storage medium is provided, on which computer program instructions are stored, and when the instructions are executed by a processor, the steps of the obstacle avoidance method for visually impaired persons based on stereoscopic vision described in any of the above are implemented. It should be understood by those skilled in the art that the embodiments of the present invention can be provided as methods, devices, or computer program products. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable non-transient storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0141] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of the processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0142] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0143] So far, the present application has been described in detail. In order to avoid obscuring the concept of the present application, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed herein.

[0144] The method and apparatus of the present application may be implemented in many ways. For example, the method and apparatus of the present application may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present application are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present application may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers a recording medium storing a program for executing the method according to the present application.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application rather than to limit it. Although the present application has been described in detail with reference to the preferred embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present application can still be modified or some technical features can be replaced by equivalents without departing from the spirit of the technical solution of the present application, which should be included in the scope of the technical solution for protection in the present application.

Claims

1. An obstacle avoidance system for visually impaired persons based on stereoscopic vision, characterized in that: include: An image acquisition module, used to acquire environmental image data collected by a stereoscopic vision device; A data processing module, used to generate obstacle information according to the environmental image data, wherein the obstacle information includes obstacle type and obstacle size; The obstacle avoidance planning module is used to plan an obstacle avoidance plan based on the obstacle information.

2. The obstacle avoidance system for visually impaired persons based on stereoscopic vision according to claim 1, characterized in that: Also includes: A voice processing module, used for generating an audio signal from the obstacle information and / or obstacle avoidance solution, so that the audio playback device plays a prompt audio after receiving the audio signal; and / or The vibration prompt module is used to generate a vibration signal according to the obstacle information, so that the vibration device vibrates according to the vibration intensity and vibration frequency indicated by the vibration signal.

3. The obstacle avoidance system for visually impaired persons based on stereoscopic vision according to claim 1, characterized in that: The stereoscopic vision device includes a passive stereo camera or an active stereo camera; Optionally, the passive stereo camera is a binocular camera; Optionally, the active stereo camera is a binocular line laser camera.

4. The obstacle avoidance system for visually impaired persons based on stereoscopic vision according to any one of claims 1 to 3, characterized in that: The obstacle size includes obstacle height and obstacle width, and the obstacle avoidance planning module is specifically used for: When the obstacle type is the first type, ignoring the obstacle during walking; and / or When the obstacle type is the second type and the obstacle height is above the first height, the walking route is kept at least at a first distance from the obstacle; and / or When the obstacle type is the second type and the obstacle height is less than the first height and the obstacle width is less than the first width, ignoring the obstacle during walking; and / or When the obstacle type is the third type, stopping walking at at least the second distance from the obstacle; and / or When the obstacle type is the fourth type, the walking route is kept at least a third distance from the obstacle.

5. A method for avoiding obstacles for visually impaired persons based on stereoscopic vision, characterized in that: include: Acquire environmental image data collected by a stereoscopic vision device; generating obstacle information according to the environment image data, wherein the obstacle information includes obstacle type and obstacle size; An obstacle avoidance plan is planned according to the obstacle information.

6. The obstacle avoidance method for visually impaired persons based on stereoscopic vision according to claim 5, characterized in that: Also includes: Generate an audio signal from the obstacle information and / or obstacle avoidance solution, so that an audio playback device plays a prompt audio after receiving the audio signal; and / or A vibration signal is generated according to the obstacle information, so that the vibration device vibrates according to the vibration intensity and vibration frequency indicated by the vibration signal.

7. The obstacle avoidance method for visually impaired persons based on stereoscopic vision according to claim 5, characterized in that: The stereoscopic vision device includes a passive stereo camera or an active stereo camera; Optionally, the passive stereo camera is a binocular camera; Optionally, the active stereo camera is a binocular line laser camera.

8. The method for avoiding obstacles for visually impaired persons based on stereoscopic vision according to any one of claims 5 to 7, characterized in that: The obstacle size includes obstacle height and obstacle width, and planning an obstacle avoidance scheme according to the obstacle information includes: When the obstacle type is the first type, ignoring the obstacle during walking; and / or When the obstacle type is the second type and the obstacle height is above the first height, the walking route is kept at least at a first distance from the obstacle; and / or When the obstacle type is the second type and the obstacle height is less than the first height and the obstacle width is less than the first width, ignoring the obstacle during walking; and / or When the obstacle type is the third type, stopping walking at at least the second distance from the obstacle; and / or When the obstacle type is the fourth type, the walking route is kept at least a third distance from the obstacle.

9. The obstacle avoidance method for visually impaired persons based on stereoscopic vision according to claim 8, characterized in that: The first type includes grass or fallen leaves; the second type includes stones, bricks, pits or branches; the third type includes pedestrians or moving vehicles; and the fourth type includes trees, walls, signs, trash cans or vehicles.

10. An intelligent obstacle avoidance system for visually impaired persons, characterized in that: The system comprises a processor and a memory, wherein the processor and the memory are connected, wherein the memory is used to store program codes, and the processor is used to call the program codes to execute the method as claimed in any one of claims 5 to 9.

Images