Method and device for guiding blind people to board a vehicle
The construction of a three-dimensional point cloud map through the on-board sensor group and planning the best route to board the vehicle is solved, and the difficulty of blind people and people with weak eyesight to reach the vehicle stop is achieved, and their safe boarding is achieved.
Patent Information
- Application Number
- CN202210750255.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-28
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2042-06-28
AI Technical Summary
Blind and people with weak eyesight encounter difficulties when driving to their destination. The existing technology cannot effectively assist them in reaching the vehicle stop, making it difficult to get on the car autonomously.
A three-dimensional point cloud map is constructed through a vehicle-mounted sensor group, projected into a two-dimensional grid map, evaluate the visibility and maneuverability of the walkable area, plan the best route to board the car, and detect the location and movement status of the passengers in real time, and dynamically avoid the vehicle. The barrier guides passengers to the vehicle stop.
This enables blind or weak eyes to move to the vehicle stop safely and autonomously, solving the problem of getting on the vehicle and allowing them to successfully complete the boarding process.
Smart Images

Figure CN115014379B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of vehicle navigation technology, and in particular to a method and device for guiding a blind person to get on a vehicle. Background Art
[0002] For the blind and people with low vision, driving to their destination is an almost impossible task. However, with the development of technology and the advent of the driverless era, it will soon become a reality for people with low vision to drive themselves. As a special group, the blind or people with low vision need to study how to get from their starting position to the location of the vehicle and realize their dream of driving or taking a taxi.
[0003] In existing technical solutions, the vehicle is generally navigated to the vicinity of the user first, and then the vehicle recognizes the gestures of the passenger and stops the vehicle at a position where the user can get on the vehicle.
[0004] The disadvantages of existing guidance schemes for the blind are that blind passengers cannot see, and in many places the vehicle cannot directly reach the passengers, there is a distance in between. Without auxiliary methods, it is difficult for the blind or people with poor vision to reach the boarding point. Summary of the invention
[0005] Aiming at the problem of travel difficulties for blind people or visually impaired groups, the present invention provides a method and device for guiding blind people to get on a bus.
[0006] In order to solve the above technical problems, in a first aspect, an embodiment of the present invention provides a method for guiding a blind person to board a vehicle, comprising:
[0007] Receive a passenger signal from a user terminal, obtain the initial position of the passenger, plan a driving path from the current position of the vehicle to the initial position of the passenger on a navigation map, and drive the vehicle to the vicinity of the passenger according to the driving path;
[0008] The environment between the vehicle and the passengers is scanned by a vehicle-mounted sensor group to construct a three-dimensional point cloud map, and the three-dimensional point cloud map is projected onto a plane to construct a two-dimensional grid map;
[0009] Constructing a walkable area between the passenger and the vehicle in the two-dimensional grid map;
[0010] In the walkable area of the two-dimensional grid map, the visibility and drivability of each grid in the walkable area are evaluated to find the location that can be used as a vehicle stop, and the best boarding route from the passenger's initial position to the vehicle stop is planned; visibility refers to the simultaneous perception of the camera and the lidar, and drivability refers to the ability of the vehicle to drive to the vehicle stop.
[0011] Track dynamic objects in the walkable area and mark them on the grid map to obtain the moving trajectory of dynamic objects;
[0012] Real-time detection of passengers’ current location and action status;
[0013] According to the optimal boarding route and the current position of the passenger, the passenger is guided to walk to the vehicle stop; according to the moving trajectory of the dynamic object and the action status of the passenger, the passenger is guided to dynamically avoid obstacles during the walking process until the vehicle sensor senses that the passenger has arrived within a preset distance range of the vehicle.
[0014] Preferably, the method further includes: step S8, when the passenger arrives within a preset distance range of the vehicle, the actual position of the passenger relative to the door is detected by a surround-view camera, and a voice prompt is given to the passenger to move to the door position and then open the door to get on the vehicle.
[0015] Preferably, the receiving of the boarding signal from the user terminal and obtaining the initial position of the boarding person includes:
[0016] After the user terminal with positioning function establishes a connection with the vehicle terminal, it sends a boarding signal to the vehicle terminal, and the vehicle terminal obtains the initial position of the passenger according to the boarding signal.
[0017] Preferably, the process of scanning the environment between the vehicle and the passengers by means of a vehicle-mounted sensor group to construct a three-dimensional point cloud map of the environment, and projecting the three-dimensional point cloud map onto a plane to construct a two-dimensional grid map specifically includes:
[0018] The laser point cloud is obtained by scanning the environment between the vehicle and the passengers through the laser radar; based on the sensor data collected by the vehicle sensor group, the feature point cloud is constructed by using the visual SLAM algorithm, and the laser point cloud and the feature point cloud are fused to obtain a three-dimensional point cloud map;
[0019] After semantic segmentation of the 3D point cloud map, it is projected onto a plane to construct a 2D grid map. The grids occupied by fixed obstacles in the 2D grid map are marked as occupied, and the grids without fixed obstacles are marked as unoccupied.
[0020] Preferably, the step of constructing a walkable area between the passenger and the vehicle on the two-dimensional grid map specifically includes:
[0021] On the two-dimensional grid map, starting from the passenger, a global search is performed in combination with the size that the passenger can pass through, and all searched unoccupied grids that can be connected are combined to form a walkable area.
[0022] Preferably, in the walkable area of the two-dimensional grid map, the visibility and drivability of each grid in the walkable area are evaluated to find a location that can be used as a vehicle stop, and the best boarding route from the passenger's initial position to the vehicle stop is planned, specifically including:
[0023] In the walkable area of the two-dimensional grid map, the locations that can be used as vehicle stops are found by evaluating the visibility and drivability of each grid in the walkable area, which are marked as a1, a2, ..., a n ;
[0024] The path planning algorithm is used to plan the starting point of the passenger's initial position l and the paths a1, a2...a n The boarding routes to the destination are recorded as la1, la2, ...la n ; By evaluating routes la1, la2, ...la n The best boarding route is selected based on the length and straightness of the route, and the end point of the best boarding route is the best boarding point.
[0025] In a second aspect, an embodiment of the present invention provides a device for guiding a blind person to board a vehicle, comprising:
[0026] The passenger initial position confirmation module is used to receive the passenger's boarding signal from the user terminal, obtain the passenger's initial position, plan a driving path from the vehicle's current position to the passenger's initial position on the navigation map, and the vehicle drives to the vicinity of the passenger according to the driving path;
[0027] A grid map construction module is used to scan the environment between the vehicle and the passengers through the vehicle sensor group to construct a three-dimensional point cloud map, and project the three-dimensional point cloud map onto a plane to construct a two-dimensional grid map;
[0028] A walkable area construction module, used to construct a walkable area between the passenger and the vehicle in the two-dimensional grid map;
[0029] A boarding route planning module, used to plan an optimal boarding route from the initial position of the passenger to the vehicle stop within the walkable area of the two-dimensional grid map;
[0030] The dynamic object tracking module is used to track the dynamic objects in the walkable area and obtain the moving trajectory of the dynamic objects;
[0031] The passenger behavior detection module is used to detect the current position and action status of the passenger in real time;
[0032] The module for guiding passengers to walk is used to guide passengers to walk to the vehicle stop point according to the optimal boarding route and the current position of the passengers; according to the movement trajectory of dynamic objects and the action status of the passengers, the module guides the passengers to dynamically avoid obstacles during walking until the vehicle sensor senses that the passengers have arrived within a preset distance range of the vehicle.
[0033] Preferably, the device also includes: a module for guiding passengers to get on the vehicle, which is used to detect the actual position of the passenger relative to the door through a surround-view camera when the passenger arrives within a preset distance range of the vehicle, and voice prompt the passenger to move to the door position and then open the door to get on the vehicle.
[0034] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for guiding a blind person to board a vehicle as described in the embodiment of the first aspect of the present invention are implemented.
[0035] In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for guiding a blind person to board a vehicle as described in the embodiment of the first aspect of the present invention are implemented.
[0036] The method and device for guiding the blind to board a vehicle provided by the embodiment of the present invention scans the environment between the vehicle and the passengers through the vehicle-mounted sensor group to construct a three-dimensional point cloud map, and projects the three-dimensional point cloud map onto a plane to construct a two-dimensional grid map; constructs a walkable area between the passengers and the vehicle in the two-dimensional grid map; plans the best boarding route from the passenger's initial position to the vehicle stop in the walkable area, and guides the passengers to walk to the vehicle stop; detects the movement trajectory of the passengers in real time, and senses the surrounding dynamic objects at the same time, and guides the passengers to dynamically avoid obstacles during walking until the passengers successfully board the vehicle. The present invention can guide the blind or people with poor vision to successfully and autonomously move to the vehicle stop, and guide the visually impaired people to complete the entire boarding process, solving the problem of difficulty in travel and boarding for the blind or visually impaired groups, so that the visually impaired groups can safely walk to the appropriate boarding point to board the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 A flow chart of a method for guiding a blind person to board a vehicle provided by an embodiment of the present invention;
[0038] Figure 2 A schematic diagram of a walkable area between a passenger and a vehicle constructed in a two-dimensional grid map provided in an embodiment of the present invention;
[0039] Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present invention;
[0040] Figure 4A schematic diagram of a computer-readable storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION
[0041] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0042] As a special group, the blind or people with low vision need to solve the problem of how to get from their starting position to the location of the vehicle and realize their dream of driving or taking a taxi to go out with the development of science and technology. In the existing technical solutions, the vehicle is generally navigated to the vicinity of the user first, and then the vehicle recognizes the gestures of the passengers and stops the vehicle to the position where the user can get on the vehicle. The disadvantage of the existing blind ride guidance scheme is that blind passengers cannot see, and in many places the vehicle cannot go directly to the passengers. There will be a distance in between. Without auxiliary methods, it is difficult for blind or people with low vision to reach the boarding point.
[0043] In view of the above problems in the prior art, the present invention proposes a method and device for guiding blind people to board a vehicle, which can guide blind people or people with poor vision to successfully and autonomously move to the vehicle stop, and guide the visually impaired people to complete the entire boarding process. The following will be described and introduced through multiple embodiments.
[0044] Figure 1 A method for guiding a blind person to board a vehicle is provided in accordance with an embodiment of the present invention. Figure 1 , the method comprising:
[0045] Step S1, receiving a passenger signal from a user terminal, obtaining the initial position of the passenger, planning a driving path from the current position of the vehicle to the initial position of the passenger on a navigation map, and the vehicle driving to the vicinity of the passenger according to the driving path.
[0046] After the user terminal establishes a connection with the vehicle terminal, the user uses voice or other operations to call the vehicle, and the user terminal sends a boarding signal to the vehicle terminal. The user terminal can obtain the initial position information of the passenger through the GPS positioning function. The above boarding signal contains the initial position information of the passenger. Therefore, the vehicle terminal knows the initial position of the passenger according to the boarding signal. The vehicle terminal plans a driving path on the navigation map according to the initial position of the passenger and the current position of the vehicle. The vehicle drives to the vicinity of the passenger according to the driving path. Specifically, the vicinity of the passenger refers to the position closest to the passenger that can be displayed on the navigation map.
[0047] Step S2, scanning the environment between the vehicle and the passengers through the vehicle-mounted sensor group to construct a three-dimensional point cloud map, and projecting the three-dimensional point cloud map onto a plane to construct a two-dimensional grid map.
[0048] Specifically, first, laser radar is used to scan the environment between the vehicle and the passengers to obtain a laser point cloud; based on the sensor data collected by the vehicle-mounted sensor group, a visual SLAM (Simultaneous Localization and Mapping) algorithm is used to construct a feature point cloud. The vehicle-mounted sensor group includes but is not limited to laser radar, surround view camera, IMU (Inertial Measurement Unit), GPS (Global Positioning System) and wheel speed meter.
[0049] Then, the laser point cloud and the feature point cloud are fused to obtain a three-dimensional point cloud map. Next, the three-dimensional point cloud map is semantically segmented and projected onto a plane to construct a two-dimensional grid map. Among them, three-dimensional point cloud semantic segmentation refers to assigning corresponding semantic labels to each point in the point cloud, segmenting each object, and giving each object a specific label, so that we have a more detailed understanding of the objects around us. In the field of autonomous driving, the point cloud data of the road environment is segmented through three-dimensional point cloud semantic segmentation, which can identify pedestrians, roads, cars and other objects in the driving of autonomous vehicles, so that autonomous vehicles can drive safely on the road.
[0050] Preferably, the size of each grid can be set to 50cm*50cm (size can be modified), and the grids occupied by fixed obstacles in the two-dimensional grid map are marked as occupied, and the grids without fixed obstacles are marked as unoccupied.
[0051] Step S3, constructing a walkable area between the passenger and the vehicle in the two-dimensional grid map.
[0052] Figure 2 A schematic diagram of a walkable area between a passenger and a vehicle is constructed in a two-dimensional grid map provided in an embodiment of the present invention, referring to Figure 2 On the two-dimensional grid map, starting from the passenger, a global search is performed based on the size that the passenger can pass through. All the unoccupied grids that have been searched and can be connected are combined to form a walkable area such as Figure 2 shown.
[0053] Step S4, in the walkable area of the two-dimensional grid map, by evaluating the visibility and drivability of each grid in the walkable area, a position that can be used as a vehicle stop is found, and the best boarding route from the passenger's initial position to the vehicle stop is planned; wherein visibility refers to the ability of the camera and the lidar to perceive at the same time, and drivability refers to the ability of the vehicle to drive to the stop.
[0054] The existing method of assisting the blind to board a vehicle generally involves navigating the vehicle to the vicinity of the user first, and then relying on the vehicle to recognize the passenger's gestures and stop the vehicle at a location where the user can board. However, the road between the blind person and the vehicle may be very narrow and only suitable for people to pass through, but the vehicle cannot reach it. Therefore, in many cases, the vehicle cannot directly reach the passenger. In this case, there will be a distance between the vehicle and the passenger, and the blind person needs to walk to the vehicle stop by himself. If there is no safe and reliable auxiliary navigation method for this distance, it is difficult for the blind or people with poor vision to safely reach the boarding point.
[0055] In view of the above-mentioned defects of the prior art, the present invention takes into account that the vehicle cannot directly reach the passenger in many cases, and therefore searches for the best boarding point most suitable for the vehicle to stop in the walkable area of the two-dimensional grid map constructed in step S3.
[0056] Specifically, first, the visibility and drivability of each grid in the walkable area are evaluated to find the locations that can be used as vehicle stops, which are marked as a1, a2, ... n ; Among them, visibility means that the camera and lidar can perceive it at the same time, and drivability means that the vehicle can drive to it. If a grid in the drivable area meets both visibility and drivability, it can be used as a vehicle stop.
[0057] Then, the path planning algorithm is used to plan the path from the initial position l of the passenger to the vehicle stop points a1, a2…a n The boarding routes to the destination are recorded as la1, la2, ...la n ; By evaluating routes la1, la2, ...la n The length and straightness of each boarding route are used to select the best boarding route. Specifically, the length and straightness of each boarding route are calculated, and different weights are assigned to them respectively. Then, the comprehensive score is calculated. The comprehensive score score = k1*1 / length + k2*straightness, where k1 and k2 are weight values. The boarding route with the highest comprehensive score is the best boarding route. The end point of the best boarding route is the best boarding point.
[0058] Step S5, tracking the dynamic objects in the walkable area and marking them on the grid map to obtain the moving trajectory of the dynamic objects;
[0059] It is understandable that in order to prevent visually impaired passengers from colliding with surrounding dynamic objects in the walkable area and causing losses, the present invention tracks dynamic objects in the walkable area through the on-board sensor group to help passengers avoid obstacles.
[0060] Step S6, real-time detection of the current position and action status of the passenger.
[0061] The surround-view camera, lidar and millimeter-wave radar in the vehicle-mounted sensor group are used to detect the current position and action status of the passengers in real time, where the action status includes the passenger's forward direction, movement speed, distance from the vehicle, etc.
[0062] Step S7, guiding the passenger to walk to the vehicle stop point according to the optimal boarding route and the passenger's current position; guiding the passenger to dynamically avoid obstacles during walking according to the moving trajectory of dynamic objects and the passenger's action status, until the vehicle sensor senses that the passenger has arrived within a preset distance range of the vehicle.
[0063] Specifically, after the best boarding route from the initial position of the passenger to the vehicle stop is planned in step S4, the passenger is guided to walk along the best boarding route to the vehicle stop in combination with the current position of the passenger detected in real time in step S6. In this process, in order to prevent the visually impaired passenger from colliding with the surrounding dynamic objects while walking, the passenger is guided to dynamically avoid obstacles during walking according to the moving trajectory of the dynamic objects and the action status of the passenger, until the vehicle sensor senses that the passenger has arrived within the preset distance range of the vehicle, preferably, the preset distance range can be set to 0.5m to 1m.
[0064] The method for guiding blind people to board a vehicle provided by an embodiment of the present invention uses a vehicle-mounted sensor group to scan the environment between the vehicle and the passengers to construct a three-dimensional point cloud map, and projects the three-dimensional point cloud map onto a plane to construct a two-dimensional grid map; constructs a walkable area between the passengers and the vehicle in the two-dimensional grid map; plans the best boarding route from the passenger's initial position to the vehicle stop in the walkable area, and guides the passengers to walk to the vehicle stop; detects the passenger's movement trajectory in real time, and senses the surrounding dynamic objects at the same time, and guides the passengers to dynamically avoid obstacles during walking until the passengers successfully board the vehicle. The present invention can guide blind people or people with poor vision to successfully and autonomously move to the vehicle stop, and guide people with visual impairments to complete the entire boarding process, solving the problem of difficulty in travel and boarding for blind people or people with poor vision, so that people with poor vision can safely walk to the appropriate boarding point to board the vehicle.
[0065] In the actual situation of blind people getting on the bus, when the blind people are very close to the vehicle, the map navigation ends. Due to poor eyesight, it is still difficult for the blind people to successfully find the door and open the door to get on the bus even if they are very close to the vehicle. To address this problem, the method for guiding blind people to get on the bus provided by the present invention also includes:
[0066] Step S8, when the passenger arrives within the preset distance range of the vehicle, the surround-view camera detects the passenger's actual position relative to the door, and voice prompts the passenger to move to the door position and then open the door to get on the vehicle.
[0067] In this embodiment, the surround-view camera detects the actual position of the passenger relative to the door, and can guide the passenger to successfully find the door and open the door to get on the vehicle, thereby solving the problem that it is difficult for the blind to successfully find the door.
[0068] In one embodiment, the present invention further provides a device for guiding blind people to board a vehicle, which is used to execute the method for guiding blind people to board a vehicle in the above embodiment, and the device includes:
[0069] The passenger initial position confirmation module is used to receive the passenger's boarding signal from the user terminal, obtain the passenger's initial position, plan a driving path from the vehicle's current position to the passenger's initial position on the navigation map, and the vehicle drives to the vicinity of the passenger according to the driving path;
[0070] A grid map construction module is used to scan the environment between the vehicle and the passengers through the vehicle sensor group to construct a three-dimensional point cloud map, and project the three-dimensional point cloud map onto a plane to construct a two-dimensional grid map;
[0071] A walkable area construction module, used to construct a walkable area between the passenger and the vehicle in the two-dimensional grid map;
[0072] A boarding route planning module, used to plan an optimal boarding route from the initial position of the passenger to the vehicle stop within the walkable area of the two-dimensional grid map;
[0073] The dynamic object tracking module is used to track the dynamic objects in the walkable area and obtain the moving trajectory of the dynamic objects;
[0074] The passenger behavior detection module is used to detect the current position and action status of the passenger in real time;
[0075] The module for guiding passengers to walk is used to guide passengers to walk to the vehicle stop point according to the optimal boarding route and the current position of the passengers; according to the movement trajectory of dynamic objects and the action status of the passengers, the module guides the passengers to dynamically avoid obstacles during walking until the vehicle sensor senses that the passengers have arrived within a preset distance range of the vehicle.
[0076] Preferably, the device also includes: a module for guiding passengers to get on the vehicle, which is used to detect the actual position of the passenger relative to the door through a surround-view camera when the passenger arrives within a preset distance range of the vehicle, and voice prompt the passenger to move to the door position and then open the door to get on the vehicle.
[0077] Figure 3 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 3 As shown, an embodiment of the present invention provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 520 and executable on the processor 520. When the processor 520 executes the computer program 511, the following steps are implemented:
[0078] Receive a passenger signal from a user terminal, obtain the initial position of the passenger, plan a driving path from the current position of the vehicle to the initial position of the passenger on a navigation map, and drive the vehicle to the vicinity of the passenger according to the driving path;
[0079] The environment between the vehicle and the passengers is scanned by a vehicle-mounted sensor group to construct a three-dimensional point cloud map, and the three-dimensional point cloud map is projected onto a plane to construct a two-dimensional grid map;
[0080] Constructing a walkable area between the passenger and the vehicle in the two-dimensional grid map;
[0081] In the walkable area of the two-dimensional grid map, the location that can be used as a vehicle stop is found by evaluating the visibility and drivability of each grid in the walkable area, and the best boarding route from the passenger's initial position to the vehicle stop is planned; wherein visibility refers to the simultaneous perception of the camera and the laser radar, and drivability refers to the ability of the vehicle to drive to the stop;
[0082] Track dynamic objects in the walkable area and mark them on the grid map to obtain the moving trajectory of dynamic objects;
[0083] Real-time detection of passengers’ current location and action status;
[0084] According to the optimal boarding route and the current position of the passenger, the passenger is guided to walk to the vehicle stop; according to the moving trajectory of the dynamic object and the action status of the passenger, the passenger is guided to dynamically avoid obstacles during the walking process until the vehicle sensor senses that the passenger has arrived within a preset distance range of the vehicle.
[0085] Figure 4 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention. Figure 4As shown, this embodiment provides a computer-readable storage medium 600, on which a computer program 611 is stored. When the computer program 611 is executed by a processor, the following steps are implemented:
[0086] Receive a passenger signal from a user terminal, obtain the initial position of the passenger, plan a driving path from the current position of the vehicle to the initial position of the passenger on a navigation map, and drive the vehicle to the vicinity of the passenger according to the driving path;
[0087] The environment between the vehicle and the passengers is scanned by a vehicle-mounted sensor group to construct a three-dimensional point cloud map, and the three-dimensional point cloud map is projected onto a plane to construct a two-dimensional grid map;
[0088] Constructing a walkable area between the passenger and the vehicle in the two-dimensional grid map;
[0089] In the walkable area of the two-dimensional grid map, the location that can be used as a vehicle stop is found by evaluating the visibility and drivability of each grid in the walkable area, and the best boarding route from the passenger's initial position to the vehicle stop is planned; wherein visibility refers to the simultaneous perception of the camera and the laser radar, and drivability refers to the ability of the vehicle to drive to the stop;
[0090] Track dynamic objects in the walkable area and mark them on the grid map to obtain the moving trajectory of dynamic objects;
[0091] Real-time detection of passengers’ current location and action status;
[0092] According to the optimal boarding route and the current position of the passenger, the passenger is guided to walk to the vehicle stop; according to the moving trajectory of the dynamic object and the action status of the passenger, the passenger is guided to dynamically avoid obstacles during the walking process until the vehicle sensor senses that the passenger has arrived within a preset distance range of the vehicle.
[0093] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0094] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0095] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0096] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0097] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0098] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0099] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A method for guiding a blind person to board a vehicle. It is characterized in that include: Receive a passenger signal from a user terminal, obtain the initial position of the passenger, plan a driving path from the current position of the vehicle to the initial position of the passenger on a navigation map, and drive the vehicle to the vicinity of the passenger according to the driving path; The environment between the vehicle and the passengers is scanned by a vehicle-mounted sensor group to construct a three-dimensional point cloud map, and the three-dimensional point cloud map is projected onto a plane to construct a two-dimensional grid map; Constructing a walkable area between the passenger and the vehicle in the two-dimensional grid map; In the walkable area of the two-dimensional grid map, the location that can be used as a vehicle stop is found by evaluating the visibility and drivability of each grid in the walkable area, and the best boarding route from the passenger's initial position to the vehicle stop is planned; wherein visibility refers to the simultaneous perception of the camera and the laser radar, and drivability refers to the ability of the vehicle to drive to the stop; Track dynamic objects in the walkable area and mark them on the grid map to obtain the moving trajectory of dynamic objects; Real-time detection of passengers’ current location and action status; According to the optimal boarding route and the current position of the passenger, the passenger is guided to walk to the vehicle stop; according to the moving trajectory of the dynamic object and the action status of the passenger, the passenger is guided to dynamically avoid obstacles during the walking process until the vehicle sensor senses that the passenger has arrived within a preset distance range of the vehicle.
2. The method for guiding the blind to board a vehicle according to claim 1, It is characterized in that Also includes: When the passenger arrives within a preset distance range of the vehicle, the surround-view camera detects the passenger's actual position relative to the door, and voice prompts the passenger to move to the door position and then open the door to get in.
3. The method for guiding the blind to board a vehicle according to claim 1, It is characterized in that The receiving of the boarding signal from the user terminal and obtaining the initial position of the boarding person includes: After the user terminal with positioning function establishes a connection with the vehicle terminal, it sends a boarding signal to the vehicle terminal, and the vehicle terminal obtains the initial position of the passenger according to the boarding signal.
4. The method for guiding the blind to board a vehicle according to claim 1, It is characterized in that The method comprises: scanning the environment between the vehicle and the passengers by means of the vehicle-mounted sensor group to construct a three-dimensional point cloud map of the environment, and projecting the three-dimensional point cloud map onto a plane to construct a two-dimensional grid map. The laser point cloud is obtained by scanning the environment between the vehicle and the passengers through the laser radar; based on the sensor data collected by the vehicle-mounted sensor group, the feature point cloud is constructed by using the visual SLAM algorithm, and the laser point cloud and the feature point cloud are fused to obtain a three-dimensional point cloud map; wherein the vehicle-mounted sensor group includes at least the laser radar, the surround view camera, the IMU and the GPS; After semantic segmentation of the 3D point cloud map, it is projected onto a plane to construct a 2D grid map. The grids occupied by fixed obstacles in the 2D grid map are marked as occupied, and the grids without fixed obstacles are marked as unoccupied.
5. The method for guiding the blind to board a vehicle according to claim 4, It is characterized in that The step of constructing a walkable area between a passenger and a vehicle on a two-dimensional grid map specifically includes: On the two-dimensional grid map, starting from the passenger, a global search is performed in combination with the size that the passenger can pass through, and all searched unoccupied grids that can be connected are combined to form a walkable area.
6. The method for guiding the blind to board a vehicle according to claim 5, It is characterized in that In the walkable area of the two-dimensional grid map, the visibility and drivability of each grid in the walkable area are evaluated to find a location that can be used as a vehicle stop, and the best boarding route from the passenger's initial position to the vehicle stop is planned, specifically including: In the walkable area of the two-dimensional grid map, the locations that can be used as vehicle stops are found by evaluating the visibility and drivability of each grid in the walkable area, which are marked as a 1 、a 2 …a n ; The path planning algorithm is used to plan the starting point of the passenger's initial position l and the 1 、a 2 …a n is the boarding route to the destination, respectively denoted as la 1 ,la 2 ,…la n ; By evaluating route la 1 ,la 2 ,…la n The best boarding route is selected based on the length and straightness of the route, and the end point of the best boarding route is the best boarding point.
7. A device for guiding blind people to board a vehicle. It is characterized in that include: The passenger initial position confirmation module is used to receive the passenger's boarding signal from the user terminal, obtain the passenger's initial position, plan a driving path from the vehicle's current position to the passenger's initial position on the navigation map, and the vehicle drives to the vicinity of the passenger according to the driving path; A grid map construction module is used to scan the environment between the vehicle and the passengers through the vehicle sensor group to construct a three-dimensional point cloud map, and project the three-dimensional point cloud map onto a plane to construct a two-dimensional grid map; A walkable area construction module, used to construct a walkable area between the passenger and the vehicle in the two-dimensional grid map; A boarding route planning module, used to plan an optimal boarding route from the initial position of the passenger to the vehicle stop within the walkable area of the two-dimensional grid map; The dynamic object tracking module is used to track the dynamic objects in the walkable area and obtain the moving trajectory of the dynamic objects; The passenger behavior detection module is used to detect the current position and action status of the passenger in real time; The module for guiding passengers to walk is used to guide passengers to walk to the vehicle stop point according to the optimal boarding route and the current position of the passengers; according to the movement trajectory of dynamic objects and the action status of the passengers, the module guides the passengers to dynamically avoid obstacles during walking until the vehicle sensor senses that the passengers have arrived within a preset distance range of the vehicle.
8. The vehicle boarding guidance device for the blind according to claim 7, It is characterized in that Also includes: The module for guiding passengers to get on the vehicle is used to detect the actual position of the passenger relative to the door through the surround-view camera when the passenger arrives within the preset distance range of the vehicle, and voice prompt the passenger to move to the door position and then open the door to get on the vehicle.
9. An electronic device, It is characterized in that include: Memory for storing computer software programs; A processor is used to read and execute the computer software program, thereby implementing the method for guiding the blind to board a vehicle as described in any one of claims 1 to 6.
10. A non-transitory computer-readable storage medium, It is characterized in that The storage medium stores a computer software program for implementing the method for guiding the blind to board a vehicle as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Laser radar and visual sensor fused dynamic grid map structuring method
CN109443369A
Driverless vehicle-based riding guidance system
CN110120149A
Intelligent logistics path planning method and system
CN113156886A
Method and apparatus for providing traffic information of blind persons
KR1020160033823A
