Auxiliary scanning equipment for visual impairment and application method thereof

Abstract

The invention discloses visual impairment auxiliary scanning equipment and an application method thereof, and relates to the technical field of visual impairment assistance, and the visual impairment auxiliary scanning equipment comprises an equipment housing, a laser radar scanning module, an image acquisition module, a GPS positioning module, an attitude sensor module and a data processing module. Three-dimensional point cloud data of an environment where a user is located currently, a panoramic image of the environment where the user is located currently, current position information of the user and current attitude information of the user are obtained through a laser radar scanning module, an image obtaining module, a GPS positioning module and an attitude sensor module; and a data processing module is utilized to provide various visual impairment aided navigation functions for the user according to the three-dimensional point cloud data, the panoramic image, the current position information and the current attitude information. The auxiliary scanning equipment for visual impairment obtains various data and carries out comprehensive analysis, and the problem that a single sensor and a single algorithm are insufficient in obstacle and surrounding environment recognition is solved.

Description

Technical Field

[0001] The present application relates to the field of visually impaired assistance technology, and in particular to a visually impaired assistance scanning device and an application method thereof. Background Art

[0002] As a disadvantaged group with a large population base, the visually impaired often encounter travel difficulties in their lives and usually need to use assistive devices. At present, the main assistive devices for the visually impaired are guide sticks, guide dogs or manual assistance. Guide sticks cannot avoid objects in mid-air, such as branches and isolation poles. Guide dogs have high initial investment costs and cannot be widely used. Manual assistance consumes a lot of manpower costs and cannot provide real-time navigation.

[0003] With the development of high-tech electronic technology, the types of electronic auxiliary products and the functions they can achieve are increasing. At present, most electronic auxiliary devices for the visually impaired are smart glasses, and the control system includes CCD optical and radar modules, thermal infrared sensing modules, voice and earphone modules, etc. They only provide functions such as object recognition or obstacle recognition, and the surrounding environment recognition function provided is relatively simple, and cannot provide users with detailed descriptions. Summary of the invention

[0004] The purpose of this application is to provide a visually impaired auxiliary scanning device and an application method thereof, which can overcome the problem of insufficient recognition of obstacles and surrounding environments using a single sensor and a single algorithm.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In the first aspect, the present application provides a visually impaired auxiliary scanning device, including: a device housing, a laser radar scanning module, an image acquisition module, a GPS positioning module, a posture sensor module and a data processing module. The device housing provides a hardware installation basis for the laser radar scanning module, the image acquisition module, the GPS positioning module, the posture sensor module and the data processing module. The laser radar scanning module is used to obtain the three-dimensional point cloud data of the user's current environment; the image acquisition module is used to obtain a panoramic image of the user's current environment; the GPS positioning module is used to obtain the user's current position information; the posture sensor module is used to obtain the user's current posture information; the data processing module is used to provide the user with a variety of visually impaired auxiliary navigation functions based on the three-dimensional point cloud data, the panoramic image, the current position information and the current posture information.

[0007] Optionally, the device housing is made of high-strength material that is drop-resistant, wear-resistant, and waterproof, and has an IP67 protection grade.

[0008] Optionally, the visually impaired assisting scanning device also includes: a communication module; the communication module connects the visually impaired assisting scanning device and the remote server signal, the communication module is used to send the three-dimensional point cloud data, panoramic image, current position information and current posture information of the user's current environment to the remote server for processing, and receive the processing results of the remote server, so as to provide the user with a variety of visually impaired assisting navigation functions.

[0009] Optionally, the visually impaired auxiliary scanning device also includes: a fill light; the fill light is used to supplement the lighting when the light is insufficient to ensure that the image acquisition module can acquire a valid panoramic image.

[0010] Optionally, the visually impaired auxiliary scanning device also includes: an interactive button module; the interactive button module is used to provide users with several human-computer interaction methods; the interactive button module includes a switch button, a voice interaction button, a volume adjustment button, a function selection button, an emergency call button and a flash button.

[0011] The switch button is used for the user to control the power on and off of the visually impaired auxiliary scanning device.

[0012] The voice interaction button is used for the user to control the voice input of the visually impaired auxiliary scanning device; when the voice interaction button is pressed, the user can input voice commands to the data processing module.

[0013] The volume adjustment button is used for users to adjust the volume of the visually impaired auxiliary scanning device.

[0014] The function selection button is used for the user to select a variety of visually impaired auxiliary navigation functions provided by the visually impaired auxiliary scanning device.

[0015] The emergency call button is used to provide users with a one-touch emergency call function. When the emergency call button is pressed, a call for help is sent to the surrounding people or emergency contacts.

[0016] The flash button is used for users to control the opening or closing of the fill light.

[0017] Optionally, the visually impaired assistive scanning device provides a variety of visually impaired assistive navigation functions including: walking assistive obstacle avoidance function, current position information broadcasting function and large model recognition environment function.

[0018] The walking assistance obstacle avoidance function calculates the obstacle information of the user's current environment in real time based on the three-dimensional point cloud data and provides obstacle avoidance voice navigation for the user.

[0019] The current position information broadcast function is to broadcast the user's current position information and current posture information in voice form.

[0020] The large model recognition environment function is based on the AI ​​large model. It mines information about the user's current environment and provides voice feedback based on three-dimensional point cloud data, panoramic images, current location information and current posture information.

[0021] In a second aspect, the present application provides an application method of the visually impaired auxiliary scanning device as described above, comprising the following steps:

[0022] Get the 3D point cloud data of the user's current environment.

[0023] Get a panoramic image of the user's current environment.

[0024] Get the user's current location and posture information.

[0025] Based on three-dimensional point cloud data, panoramic images, current location information and current posture information, a variety of visually impaired assisted navigation functions are provided to users.

[0026] Optionally, a plurality of visually impaired auxiliary navigation functions are provided to the user according to the three-dimensional point cloud data, the panoramic image, the current position information and the current posture information, specifically comprising the following steps:

[0027] Based on the three-dimensional point cloud data, the obstacle information of the user's current environment is calculated in real time and the user is given obstacle avoidance voice navigation.

[0028] The user's current location and posture information are reported in voice form.

[0029] Based on the AI ​​big model, information mining of the user's current environment is carried out and voice feedback is provided according to three-dimensional point cloud data, panoramic images, current location information and current posture information.

[0030] Optionally, based on the three-dimensional point cloud data, obstacle information of the user's current environment is calculated in real time and obstacle avoidance voice navigation is performed for the user, specifically including the following steps:

[0031] The 3D point cloud data is cropped and sparsely processed to reduce data redundancy and remove noise points to obtain denoised 3D point cloud data.

[0032] Through the ground filtering algorithm, the denoised 3D point cloud data is divided into a ground point set and an obstacle point set.

[0033] The points in the obstacle point set are clustered using the DBSCAN clustering algorithm to determine the number of obstacles and related information.

[0034] According to the number of obstacles and related information, the user is given obstacle avoidance voice navigation.

[0035] Optionally, based on the AI ​​big model, according to the three-dimensional point cloud data, the panoramic image, the current position information and the current posture information, information mining is performed on the user's current environment and voice feedback is performed, specifically including the following steps:

[0036] According to the three-dimensional point cloud data, the obstacle information of the user's current environment is calculated in real time to obtain the obstacle point cloud.

[0037] The three-dimensional point cloud data and the panoramic image are range matched, and the corresponding area of ​​the obstacle point cloud of the three-dimensional point cloud data in the panoramic image is marked to obtain a panoramic image with the obstacle area marked.

[0038] The panoramic image with obstacle areas marked is input into the AI ​​big model, and the AI ​​big model is used to focus on analyzing the obstacle areas in the panoramic image to generate scene analysis results.

[0039] Based on the scene analysis results, current location information and current posture information, the user's points of interest are mined in the user's current environment and voice feedback is provided.

[0040] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0041] The present application provides a visually impaired auxiliary scanning device and its application method, the visually impaired auxiliary scanning device includes: a device housing, a laser radar scanning module, an image acquisition module, a GPS positioning module, a posture sensor module and a data processing module; the three-dimensional point cloud data of the user's current environment, the panoramic image of the user's current environment, the user's current position information and the current posture information are respectively obtained through the laser radar scanning module, the image acquisition module, the GPS positioning module and the posture sensor module; and the data processing module is used to provide the user with a variety of visually impaired auxiliary navigation functions according to the three-dimensional point cloud data, the panoramic image, the current position information and the current posture information. The visually impaired auxiliary scanning device of the present application obtains a variety of data, and conducts a comprehensive analysis of the multiple data, captures the obstacle information around the user in real time and understands the surrounding environment, overcoming the problem of insufficient recognition of obstacles and the surrounding environment using a single sensor and a single algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1A functional module diagram of a visually impaired auxiliary scanning device provided in one embodiment of the present application.

[0044] Figure 2 A conceptual schematic diagram of a visually impaired auxiliary scanning device provided in one embodiment of the present application.

[0045] Figure 3 A flowchart of an application method of a visually impaired auxiliary scanning device provided in one embodiment of the present application.

[0046] Figure 4 A detailed flow chart of step A4 in an application method of a visually impaired auxiliary scanning device provided in one embodiment of the present application.

[0047] Figure 5 A detailed flow chart of step A41 in an application method of a visually impaired auxiliary scanning device provided in one embodiment of the present application.

[0048] Figure 6 A detailed flow chart of step A43 in an application method of a visually impaired auxiliary scanning device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0051] In an exemplary embodiment, Figure 1 As shown, a visually impaired auxiliary scanning device is provided, including: a device housing, a laser radar scanning module, an image acquisition module, a GPS positioning module, a posture sensor module and a data processing module. The device housing provides a hardware installation basis for the laser radar scanning module, the image acquisition module, the GPS positioning module, the posture sensor module and the data processing module.

[0052] The laser radar scanning module is used to obtain three-dimensional point cloud data of the user's current environment; in this embodiment, the laser radar scanning module uses a laser radar scanner.

[0053] The image acquisition module is used to acquire a panoramic image of the user's current environment; in this embodiment, the image acquisition module uses an RGB panoramic camera. In an exemplary embodiment, the visually impaired auxiliary scanning device also includes: a fill light; the fill light is used to supplement the light when the light is insufficient to ensure that the image acquisition module can acquire a valid panoramic image.

[0054] The GPS positioning module is used to obtain the user's current location information; the posture sensor module is used to obtain the user's current posture information. The data processing module is used to provide users with a variety of visually impaired auxiliary navigation functions based on three-dimensional point cloud data, panoramic images, current location information and current posture information.

[0055] As an improved embodiment, the device housing is made of high-strength material that is drop-resistant, wear-resistant, and waterproof, and has an IP67 protection grade. Figure 2 What is shown is a conceptual schematic diagram of the visually impaired auxiliary scanning device proposed in this embodiment.

[0056] In an exemplary embodiment, the visually impaired assisting scanning device also includes: a communication module; the communication module connects the visually impaired assisting scanning device and the remote server signal, and the communication module is used to send the three-dimensional point cloud data, panoramic image, current position information and current posture information of the user's current environment to the remote server for processing, and receive the processing results of the remote server to provide the user with a variety of visually impaired assisting navigation functions.

[0057] In an exemplary embodiment, the visually impaired auxiliary scanning device also includes: an interactive button module; the interactive button module is used to provide users with several human-computer interaction methods; the interactive button module includes a switch button, a voice interaction button, a volume adjustment button, a function selection button, an emergency call button and a flash button.

[0058] The switch button is used for the user to control the power on and off of the visually impaired auxiliary scanning device.

[0059] The voice interaction button is used for the user to control the voice input of the visually impaired auxiliary scanning device; when the voice interaction button is pressed, the user can input voice commands to the data processing module.

[0060] The volume adjustment button is used for users to adjust the volume of the visually impaired auxiliary scanning device.

[0061] The function selection button is used for the user to select a variety of visually impaired auxiliary navigation functions provided by the visually impaired auxiliary scanning device.

[0062] The emergency call button is used to provide users with a one-touch emergency call function. When the emergency call button is pressed, a call for help is sent to the surrounding people or emergency contacts.

[0063] The flash button is used for users to control the opening or closing of the fill light.

[0064] In an exemplary embodiment, the multiple visually impaired assisted navigation functions provided by the visually impaired assisted scanning device include: walking assisted obstacle avoidance function, current position information broadcasting function and large model recognition environment function.

[0065] The walking assistance obstacle avoidance function calculates the obstacle information of the user's current environment in real time based on the three-dimensional point cloud data and provides obstacle avoidance voice navigation for the user.

[0066] The current position information broadcast function is to broadcast the user's current position information and current posture information in voice form.

[0067] The large model recognition environment function is based on the AI ​​large model. It mines information about the user's current environment and provides voice feedback based on three-dimensional point cloud data, panoramic images, current location information and current posture information.

[0068] certainly, Figure 1 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different functions. Figure 1 One or at least two components of the system shown.

[0069] The usage process of the above-mentioned visually impaired assistive scanning device proposed in this embodiment is: the visually impaired person presses the power button of the device for three seconds, and the device is turned on. At this time, the walking assistive obstacle avoidance function can be turned on through the function selection button, or the voice interaction button can be long pressed to turn on the voice assistant to interact and turn on the walking assistive obstacle avoidance function.

[0070] When the user walks, the device's camera points toward the forward direction, and the device collects the panoramic image and 3D point cloud data, GPS location information and posture information in real time. The device analyzes the 3D point cloud data through the point cloud algorithm to determine whether there are obstacles in front. If there are obstacles, the user is informed of the distance and height of the obstacles through voice and tactile vibration.

[0071] After the user selects the large model recognition environment function through the function selection button and clicks the OK button (the function selection button consists of the upper, lower, left, and right selection buttons and the OK button in the center), the AI ​​large model is used to further analyze the panoramic image, identify obstacles and information about the surrounding environment, summarize them into short sentences and provide feedback through voice playback. When the user needs to know more about the current environment, he can press the voice interaction button to ask questions about the recognition results. The user continues to double-click and press the OK button, and the device obtains the current location information, direction, and information about surrounding points of interest.

[0072] In another exemplary embodiment of the present application, based on the same inventive concept, a method for applying the above-mentioned visually impaired auxiliary scanning device is also provided. Figure 3 As shown, a method for applying a visually impaired auxiliary scanning device is provided, comprising the following steps:

[0073] A1. Obtain the three-dimensional point cloud data of the user's current environment.

[0074] A2. Obtain a panoramic image of the user's current environment.

[0075] A3. Obtain the user's current location information and current posture information.

[0076] A4. Provide users with a variety of visually impaired assisted navigation functions based on the three-dimensional point cloud data, panoramic image, current position information and current posture information. Figure 4 As shown, step A4 specifically includes the following steps:

[0077] A41. Based on the three-dimensional point cloud data, the obstacle information of the user's current environment is calculated in real time and obstacle avoidance voice navigation is provided to the user.

[0078] A42. Report the user's current location information and current posture information in voice form.

[0079] A43, based on the AI ​​big model, according to the three-dimensional point cloud data, panoramic image, current location information and current posture information, the user's current environment is mined and voice feedback is provided.

[0080] As an optional implementation, Figure 5 In the flowchart shown, step A41 calculates obstacle information of the user's current environment in real time based on the three-dimensional point cloud data and performs obstacle avoidance voice navigation for the user, specifically including the following steps:

[0081] A411. The three-dimensional point cloud data is cropped and sparsely denoised through preprocessing to reduce data redundancy and remove noise points to obtain denoised three-dimensional point cloud data.

[0082] In this embodiment, preprocessing includes: point cloud cropping, which sets the point cloud data in a rectangular area with a length of 5 meters, a width of 2 meters, and a height of 2 meters in the direction of the user's position, retains the point cloud, and deletes the rest; point cloud thinning, using a grid thinning method, using a 1.5 cm cube grid based on empirical values ​​to reduce the redundancy of the point cloud; point cloud denoising, in order to delete noise points that appear under strong light or hardware, the point cloud is denoised by using a radius filtering algorithm.

[0083] A412. Through the ground filtering algorithm, the denoised 3D point cloud data is divided into a ground point set and an obstacle point set.

[0084] In this embodiment, the ground filtering algorithm is divided into two steps, namely, height coarse filtering and RANSAC fine filtering. First, through the height threshold method, the point with the lowest z value of the point cloud is taken as the target, and the points within 4 cm higher than it are screened out and regarded as ground points, otherwise they are obstacle point clouds. When the obstacle point cloud exceeds 50, it enters the next step of fine filtering, otherwise it returns the result that there is no obstacle ahead.

[0085] When the coarse filter detects an obstacle, it will be suspected that the point cloud in front may be a slope point cloud that causes misidentification, so the RANSAC filter algorithm is used to detect the plane. When there are more than fifty remaining point clouds after the plane is detected, the result that there is an obstacle in front is returned, otherwise further judgment is made. The plane detected by the RANSAC filter algorithm has a normal vector value. According to the empirical value, the normal vector value of the plane parallel to the ground is between 0.95-1, and the normal vector value of the plane perpendicular to the ground and with a larger slope angle is between 0-0.85, and the slope with a smaller angle is between 0.85-0.95. According to the above relationship between the normal vector value and the plane, when the value is between 0.85-0.95, a passable slope result is returned, when the value is between 0.95-1, the ground ahead is passable, and when the value is between 0-0.85, the result that there is an obstacle in front is returned.

[0086] A413. Cluster the points in the obstacle point set using the DBSCAN clustering algorithm to determine the number of obstacles and related information. The obstacle related information specifically includes the height information and distance information of the obstacles.

[0087] A414. Provide obstacle avoidance voice navigation to users based on the number of obstacles and related information.

[0088] As an optional implementation, Figure 6 In the flowchart shown, step A43 is based on the AI ​​big model, and according to the three-dimensional point cloud data, the panoramic image, the current position information and the current posture information, information mining is performed on the user's current environment and voice feedback is performed, which specifically includes the following steps:

[0089] A431. According to the three-dimensional point cloud data, the obstacle information of the user's current environment is calculated in real time to obtain the obstacle point cloud.

[0090] A432. Perform range matching between the three-dimensional point cloud data and the panoramic image, mark the area corresponding to the obstacle point cloud of the three-dimensional point cloud data in the panoramic image, and obtain a panoramic image with the obstacle area marked.

[0091] A433. Input the panoramic image with the obstacle area marked into the AI ​​big model, and use the AI ​​big model to focus on analyzing the obstacle area in the panoramic image to generate a scene analysis result. In this embodiment, the panoramic image with the obstacle area marked is sent to the AI ​​big model for analysis, and a prompt word is given at the same time, so that the model focuses on analyzing the obstacle frame area, describes the semantic information of the obstacle, and analyzes the overall scene in the image, and gives a brief image description.

[0092] During this step, users can ask follow-up questions through voice interaction. When the user is not satisfied with the brief image description, they input voice, the large model recognizes the voice information, and asks follow-up questions about the image. For example, if the panoramic image is a beach scene, the initial prompt description may be: "This is a scene on the beach, with several people on the beach." If you ask follow-up questions about this picture, you may ask: "What activities are these people doing?" or "Are there any other items on the beach, such as parasols or beach chairs?" Through these further questions, the model can obtain more details and provide a more accurate description.

[0093] A434. Based on the scene analysis results, current location information and current posture information, the user's points of interest are mined in the user's current environment and voice feedback is provided. After the scene analysis results are generated, the current location information and current posture information are used to obtain detailed place name information of the user's current location, as well as surrounding road names and points of interest based on this information when connected to the Internet. Points of interest refer to places with specific meaning or value in geographic space. These places are generally considered to be useful or worthy of attention to users, such as restaurants, shops, attractions, gas stations, banks, etc.

[0094] The above-mentioned scheme of the present application receives the three-dimensional point cloud obtained by the lidar scanner, the panoramic image captured by the RGB panoramic camera, the position information obtained by the GPS positioning module, and the posture information obtained by the posture sensing module, verifies and integrates each other, analyzes the data through machine learning methods and AI large model algorithms, captures obstacle information in real time and understands the surrounding environment, and overcomes the shortcomings of using a single sensor and a traditional single algorithm to identify obstacles and the surrounding environment.

[0095] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0096] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A visually impaired auxiliary scanning device, characterized in that: include: Equipment housing, laser radar scanning module, image acquisition module, GPS positioning module, attitude sensor module and data processing module; the equipment housing provides a hardware installation basis for the laser radar scanning module, the image acquisition module, the GPS positioning module, the attitude sensor module and the data processing module; the laser radar scanning module is used to obtain three-dimensional point cloud data of the user's current environment; the image acquisition module is used to obtain a panoramic image of the user's current environment; the GPS positioning module is used to obtain the user's current location information; the attitude sensor module is used to obtain the user's current attitude information; The data processing module is used to provide a variety of visually impaired auxiliary navigation functions to the user based on the three-dimensional point cloud data, the panoramic image, the current position information and the current posture information.

2. The visually impaired auxiliary scanning device according to claim 1, characterized in that: The device housing is made of high-strength material that is drop-resistant, wear-resistant, and waterproof, and has an IP67 protection grade.

3. The visually impaired auxiliary scanning device according to claim 1, characterized in that: Also includes: Communication module; The communication module connects the visually impaired auxiliary scanning device and the remote server signal. The communication module is used to send the three-dimensional point cloud data, the panoramic image, the current position information and the current posture information of the user's current environment to the remote server for processing, and receive the processing results of the remote server to provide the user with a variety of visually impaired auxiliary navigation functions.

4. The visually impaired auxiliary scanning device according to claim 1, characterized in that: Also includes: Fill light; The fill light is used to provide supplementary lighting when light is insufficient, so as to ensure that the image acquisition module can acquire a valid panoramic image.

5. The visually impaired auxiliary scanning device according to claim 4, characterized in that: Also includes: Interactive key module; the interactive key module is used to provide users with several human-computer interaction methods; the interactive key module includes a switch key, a voice interaction key, a volume adjustment key, a function selection key, an emergency call key and a flash light key; The switch button is used for the user to control the power on and off of the visually impaired auxiliary scanning device; The voice interaction button is used for the user to control the voice input of the visually impaired auxiliary scanning device; when the voice interaction button is pressed, the user can input a voice command to the data processing module; The volume adjustment button is used for the user to adjust the volume of the visually impaired auxiliary scanning device; The function selection button is used for the user to select a variety of visually impaired auxiliary navigation functions provided by the visually impaired auxiliary scanning device; The emergency call button is used to provide the user with a one-touch emergency call function. When the emergency call button is pressed, a call for help is sent to the surrounding people or emergency contacts; The flash button is used for the user to control the on or off of the fill light.

6. The visually impaired auxiliary scanning device according to claim 5, characterized in that: The visually impaired auxiliary scanning device provides a variety of visually impaired auxiliary navigation functions including: walking auxiliary obstacle avoidance function, current position information broadcasting function and large model recognition environment function; The walking assistance obstacle avoidance function is to calculate the obstacle information of the user's current environment in real time based on the three-dimensional point cloud data and provide the user with obstacle avoidance voice navigation; The current position information broadcasting function is to broadcast the user's current position information and current posture information in voice form; The large model recognition environment function is based on the AI ​​large model, and according to the three-dimensional point cloud data, the panoramic image, the current position information and the current posture information, it mines information about the user's current environment and provides voice feedback.

7. An application method of the visually impaired auxiliary scanning device according to any one of claims 1 to 6, characterized in that: include: Obtain the three-dimensional point cloud data of the user's current environment; Obtain a panoramic image of the user's current environment; Get the user's current location information and current posture information; Based on the three-dimensional point cloud data, the panoramic image, the current position information and the current posture information, a variety of visually impaired auxiliary navigation functions are provided to the user.

8. The application method of the visually impaired auxiliary scanning device according to claim 7, characterized in that: According to the three-dimensional point cloud data, the panoramic image, the current position information and the current posture information, a plurality of visually impaired auxiliary navigation functions are provided to the user, specifically including: According to the three-dimensional point cloud data, the obstacle information of the user's current environment is calculated in real time and the user is given obstacle avoidance voice navigation; Reporting the current position information and the current posture information of the user in the form of voice; Based on the AI ​​big model, information mining is performed on the user's current environment and voice feedback is provided according to the three-dimensional point cloud data, the panoramic image, the current position information and the current posture information.

9. The application method of the visually impaired auxiliary scanning device according to claim 8, characterized in that: According to the three-dimensional point cloud data, the obstacle information of the user's current environment is calculated in real time and the user is given obstacle avoidance voice navigation, specifically including: The three-dimensional point cloud data is cropped and sparsely processed to remove noise through preprocessing, so as to reduce data redundancy and remove noise points, thereby obtaining denoised three-dimensional point cloud data; By using a ground filtering algorithm, the denoised three-dimensional point cloud data is divided into a ground point set and an obstacle point set; Using the DBSCAN clustering algorithm, clustering the points in the obstacle point set to determine the number of obstacles and related information; According to the number of obstacles and related information, the user is given obstacle avoidance voice navigation.

10. The application method of the visually impaired auxiliary scanning device according to claim 8, characterized in that: Based on the AI ​​big model, according to the three-dimensional point cloud data, the panoramic image, the current position information and the current posture information, information mining is performed on the user's current environment and voice feedback is performed, specifically including: According to the three-dimensional point cloud data, obstacle information of the user's current environment is calculated in real time to obtain an obstacle point cloud; Performing range matching between the three-dimensional point cloud data and the panoramic image, marking the area corresponding to the obstacle point cloud of the three-dimensional point cloud data in the panoramic image, and obtaining a panoramic image with the obstacle area marked; Inputting the panoramic image with the obstacle area marked into the AI ​​big model, using the AI ​​big model to focus on analyzing the obstacle area in the panoramic image to generate a scene analysis result; According to the scene analysis result, the current position information and the current posture information, the user's interest points are mined in the user's current environment, and voice feedback is provided.

Images