Obstacle object determination method, wearable smart glasses and smart guide stick

Abstract

The embodiment of the application discloses a kind of obstacle object determination method, wearable smart glasses and smart guide stick, applied to guide blind technology field, can solve how to accurately detect the problem of obstacle object.The camera is arranged in wearable smart glasses, and the method comprises the following steps: wearable smart glasses obtain the target position data of smart guide stick;Wearable smart glasses obtain three-dimensional picture in real time by camera;Wearable smart glasses identify the object corresponding to target position data in three-dimensional picture, and obtain the target obstacle object contacted by smart guide stick.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of guide technology, and in particular to a method for determining obstacles, wearable smart glasses, and a smart guide cane. Background Technology

[0002] People with disabilities, as a special group in society, often need to rely on assistive devices to detect obstacles when traveling. However, existing obstacle detection methods are mostly affected by external factors, resulting in low accuracy. For example, using ultrasonic pulse waves to detect obstacles. Therefore, how to accurately detect obstacles has become an urgent problem to be solved. Summary of the Invention

[0003] This invention provides a method for determining obstacles, wearable smart glasses, and a smart guide cane to solve the problem of how to accurately detect obstacles in the prior art.

[0004] In a first aspect, an obstacle detection method is provided, applied to wearable smart glasses, wherein the wearable smart glasses are equipped with a camera, and the method includes:

[0005] Acquire target location data for the smart guide cane;

[0006] The camera captures 3D images in real time.

[0007] The object corresponding to the target location data in the three-dimensional image is identified to obtain the target obstacle object that the smart guide cane is in contact with.

[0008] As an optional implementation, in a first aspect of the present invention, the wearable smart glasses are provided with a first ultra-wideband (UWB) module, and the smart guide cane is provided with a second UWB module; acquiring the target position data of the smart guide cane includes:

[0009] The first UWB module receives pulse data sent by the smart guide cane through the second UWB module, and determines the target position data of the smart guide cane based on the pulse data.

[0010] As an optional implementation, in a first aspect of the present invention, the first UWB module includes at least three signal receivers, and the second UWB module includes a signal transmitter; the step of receiving pulse data transmitted by the smart guide cane through the second UWB module via the first UWB module, and determining the target position data of the smart guide cane based on the pulse data, includes:

[0011] The at least three signal receivers respectively receive the pulse data transmitted by the intelligent guide cane through the signal transmitter;

[0012] The signal transmission duration corresponding to each pulse data is obtained based on the signal transmission time included in the pulse data received by each of the at least three signal receivers, and the signal reception time corresponding to the time when each signal receiver receives the pulse data.

[0013] Based on the transmission duration of each signal, the distance between each signal receiver and the signal transmitter in the smart guide cane is determined, resulting in at least three distances;

[0014] The target position data of the smart guide cane relative to the wearable smart glasses is determined based on the at least three distances.

[0015] As an optional implementation, in a first aspect of the present invention, identifying the object in the three-dimensional image corresponding to the target location data to obtain the target obstacle object contacted by the smart guide cane includes:

[0016] Based on the aforementioned three-dimensional image, establish a three-dimensional coordinate system;

[0017] Determine the coordinate point corresponding to the target position data in the three-dimensional coordinate system;

[0018] The object corresponding to the coordinate point in the three-dimensional image is identified to determine the target obstacle object that the smart guide cane is in contact with.

[0019] As an optional implementation, in a first aspect of the present invention, after identifying the object corresponding to the target location data in the three-dimensional image to obtain the target obstacle object contacted by the smart guide cane, the method further includes:

[0020] The system outputs prompts in a non-visual manner, which are used to alert the user to the target obstacle object that the smart guide cane is in contact with.

[0021] As an optional implementation, in a first aspect of the present invention, after identifying the object corresponding to the target location data in the three-dimensional image to obtain the target obstacle object contacted by the smart guide cane, the method further includes:

[0022] Based on the three-dimensional image and the target location data, an obstacle avoidance route is determined;

[0023] The obstacle avoidance route is output in a non-visual manner.

[0024] Secondly, a method for determining obstacles is provided, applied to a smart guide cane, wherein the smart guide cane is equipped with a second UWB module, and the method includes:

[0025] When the smart guide cane is detected to be in a gripped state, pulse data is sent from the second UWB module to the first UWB module in the wearable smart glasses. This allows the wearable smart glasses to determine the target position data of the smart guide cane based on the pulse data, and to identify the object corresponding to the target position data in the three-dimensional image, thus obtaining the target obstacle object that the smart guide cane is in contact with. The three-dimensional image is acquired in real time by the camera in the wearable smart glasses.

[0026] As an optional implementation, in a second aspect of the present invention, the step of sending pulse data to a first UWB module installed in wearable smart glasses via the second UWB module when the smart guide cane is detected to be in a gripped state includes:

[0027] When the smart guide cane is detected to be in the gripping state, the environmental information of the smart guide cane is detected in real time.

[0028] If an obstacle is detected within the target distance of the smart guide cane, the pulse data is sent from the second UWB module to the first UWB module installed in the wearable smart glasses.

[0029] As an optional implementation, in a second aspect of the present invention, the smart guide cane is equipped with a distance sensor, and the real-time detection of the environmental information of the smart guide cane includes:

[0030] The distance sensor transmits a ranging signal;

[0031] If a reflected signal corresponding to the ranging signal is received within a preset time period, it is determined that there is an obstacle within the target distance of the smart guide cane.

[0032] The preset duration corresponds to the target distance.

[0033] Thirdly, a wearable smart glasses is provided, wherein a camera is provided in the wearable smart glasses, and the wearable smart glasses include:

[0034] The acquisition module is used to acquire the target location data of the smart guide cane;

[0035] The acquisition module is also used to acquire three-dimensional images in real time through the camera;

[0036] The processing module is used to identify the object in the three-dimensional image that corresponds to the target position data, and to obtain the target obstacle object that the smart guide cane is in contact with.

[0037] Fourthly, a smart guide cane is provided, wherein a second UWB module is provided, and the smart guide cane includes:

[0038] The transceiver module is used to send pulse data to the first UWB module set in the wearable smart glasses through the second UWB module when the smart guide cane is detected to be in a gripped state. This allows the wearable smart glasses to determine the target position data of the smart guide cane based on the pulse data, and to identify the object corresponding to the target position data in the three-dimensional image, thereby obtaining the target obstacle object that the smart guide cane is in contact with. The three-dimensional image is acquired in real time by the camera in the wearable smart glasses.

[0039] Fifthly, a wearable smart glasses is provided, wherein a camera is provided in the wearable smart glasses, and the wearable smart glasses include:

[0040] Memory containing executable program code;

[0041] A processor coupled to the memory;

[0042] The processor calls the executable program code stored in the memory to execute the obstacle determination method in the first aspect of the present invention.

[0043] Sixthly, a smart guide cane is provided, wherein a second UWB module is provided in the smart guide cane, and the smart guide cane includes:

[0044] Memory containing executable program code;

[0045] A processor coupled to the memory;

[0046] The processor calls the executable program code stored in the memory to execute the obstacle determination method in the second aspect of the present invention.

[0047] In a seventh aspect, a computer-readable storage medium is provided, which stores a computer program that causes a computer to execute the obstacle determination method of the first aspect of the present invention, or the obstacle determination method of the second aspect of the present invention. The computer-readable storage medium includes ROM / RAM, a magnetic disk, or an optical disk, etc.

[0048] Eighthly, a computer program product is provided that, when the computer program product is run on a computer, causes the computer to perform some or all of the steps of any method of the first aspect, or some or all of the steps of any method of the second aspect.

[0049] Ninth aspect, an application publishing platform is provided for publishing computer program products, wherein when the computer program product is run on a computer, the computer performs some or all of the steps of any method of the first aspect, or some or all of the steps of any method of the second aspect.

[0050] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0051] In this embodiment of the invention, wearable smart glasses can acquire target location data of the smart guide cane, and then, through a camera installed in the wearable smart glasses, acquire a three-dimensional image in real time. The glasses then identify objects in the three-dimensional image that correspond to the target location data, thereby determining the target obstacle object contacted by the smart guide cane. In this solution, the wearable smart glasses can determine the obstacle object corresponding to the smart guide cane in the three-dimensional image based on the cane's location data. This allows for more accurate obstacle detection results, unaffected by various factors in the current environment. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a scene illustration of an obstacle determination method provided in an embodiment of the present invention. Figure 1 ;

[0054] Figure 2 This is a flowchart illustrating an obstacle determination method provided in an embodiment of the present invention. Figure 1 ;

[0055] Figure 3 This is a flowchart illustrating an obstacle determination method provided in an embodiment of the present invention. Figure 2 ;

[0056] Figure 4 This is a flowchart illustrating an obstacle determination method provided in an embodiment of the present invention. Figure 3 ;

[0057] Figure 5This is a scene illustration of an obstacle determination method provided in an embodiment of the present invention. Figure 2 ;

[0058] Figure 6 This is a schematic diagram of the structure of a wearable smart glasses provided in an embodiment of the present invention. Figure 1 ;

[0059] Figure 7 This is a schematic diagram of the structure of a smart guide cane provided in an embodiment of the present invention. Figure 1 ;

[0060] Figure 8 This is a schematic diagram of the structure of a wearable smart glasses provided in an embodiment of the present invention. Figure 2 ;

[0061] Figure 9 This is a schematic diagram of the structure of a smart guide cane provided in an embodiment of the present invention. Figure 2 . Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] The terms "first" and "second," etc., used in the specification and claims of this invention are used to distinguish different objects, not to describe a specific order of objects. For example, "first UWB module" and "second UWB module," etc., are used to distinguish different UWB modules, not to describe a specific order of UWB modules.

[0064] The terms “comprising” and “having” and any variations thereof in this invention are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product or device.

[0065] It should be noted that in the embodiments of the present invention, the terms "exemplary" or "for example" are used to indicate that they are examples, illustrations, or descriptions. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or design options. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0066] Embodiments of the present invention can be achieved through the interaction between wearable smart glasses and a smart guide cane, such as... Figure 1 As shown, the user can wear wearable smart glasses 11 and hold a smart guide cane 12 in their hand. The wearable smart glasses 11 and the smart guide cane 12 can communicate with each other.

[0067] It should be noted that the wearable smart glasses are smart glasses that can be worn like ordinary glasses. The wearable smart glasses are equipped with a memory and processor, which can store and process images, data, etc.; the smart guide cane is also similar to an ordinary guide cane, and users can hold the smart guide cane to explore obstacles in front of them.

[0068] The embodiments of the present invention can be applied to scenarios where blind users are wearing wearable smart glasses and holding smart guide canes while walking on the road, and the smart guide canes come into contact with obstacles on the road.

[0069] like Figure 2 As shown, this embodiment of the invention provides a method for determining obstacles, which may include the following steps:

[0070] 201. Wearable smart glasses acquire target location data of smart guide canes.

[0071] In this embodiment of the invention, wearable smart glasses can communicate with smart guide canes, thus enabling wearable smart glasses to acquire target location data of the smart guide canes.

[0072] The target location data can be the relative position of the smart guide cane with respect to the wearable smart glasses, or it can be the absolute position of the smart guide cane. This embodiment of the invention does not limit the specific location data.

[0073] It should be noted that the wearable smart glasses can acquire the target location data of the smart guide cane in various ways, such as satellite positioning and infrared positioning.

[0074] 202. Wearable smart glasses acquire 3D images in real time through a camera.

[0075] In this embodiment of the invention, the wearable smart glasses are equipped with a camera, so that the wearable smart glasses can acquire three-dimensional images through the camera.

[0076] It should be noted that the three-dimensional image is the image within the shooting range of the camera, and the three-dimensional image may include factors such as the distance of objects.

[0077] Optionally, the camera in the wearable smart glasses can be a depth camera. In addition to acquiring planar images, the depth camera can also obtain the depth information of the object being photographed, that is, the three-dimensional position and size information, so that the wearable smart glasses can acquire the three-dimensional position data of the current environment and objects in the environment, thereby obtaining a three-dimensional image.

[0078] Optionally, the wearable smart glasses can also be equipped with at least two cameras, which simultaneously capture images. The processor in the wearable smart glasses then processes these two images to recover the depth information of the captured object, thereby obtaining a three-dimensional image of the current scene.

[0079] 203. Wearable smart glasses identify objects in a 3D image that correspond to the target location data, thus identifying the target obstacle object that the smart guide cane is in contact with.

[0080] In this embodiment of the invention, wearable smart glasses can first determine the object in the three-dimensional image that corresponds to the target location data, and then identify the object to determine the target obstacle object that the smart guide cane is in contact with.

[0081] It should be noted that since the target obstacle is in contact with the smart guide cane, wearable smart glasses can determine the position data of the smart guide cane as the position data of the target obstacle.

[0082] Optionally, wearable smart glasses can first capture an image of the target obstacle, and then use image recognition technology to identify the image and determine parameters such as the type, color, size, and whether the target obstacle is movable.

[0083] This invention provides a method for determining obstacles. Wearable smart glasses can acquire target position data of a smart guide cane, and then a camera installed in the wearable smart glasses can acquire a three-dimensional image in real time. The method then identifies objects in the three-dimensional image that correspond to the target position data, thereby determining the target obstacle object that the smart guide cane is in contact with. In this solution, the wearable smart glasses can determine the obstacle object corresponding to the smart guide cane in the three-dimensional image based on the position data of the smart guide cane. This method is unaffected by various factors in the current environment, resulting in a more accurate obstacle detection result.

[0084] like Figure 3 As shown in the figure, an embodiment of the present invention provides a method for determining an obstacle, which may further include the following steps:

[0085] 301. When the smart guide cane detects that it is in a gripped state, it sends pulse data to the first UWB module set in the wearable smart glasses through the second UWB module.

[0086] In this embodiment of the invention, when the smart guide cane detects that it is in a gripping state, it means that the user is holding the smart guide cane. At this time, the wearable smart glasses need to know the target position data of the smart guide cane. Therefore, the smart guide cane can send pulse data to the first UWB module set in the wearable smart glasses through the second UWB module set therein.

[0087] It should be noted that Ultra Wideband (UWB) is a carrier-free communication technology that uses narrow, non-sinusoidal pulses in the nanosecond to microsecond range to transmit data. The most basic working principle of UWB is to transmit and receive Gaussian single-cycle ultra-short-time pulses with strictly controlled pulse intervals. The ultra-short-time single-cycle pulse determines the wide bandwidth of the signal. The receiver directly uses a single-stage front-end cross-correlator to convert the pulse sequence into a baseband signal, eliminating the intermediate frequency stage in traditional communication equipment and greatly reducing the complexity of the equipment.

[0088] The wearable smart glasses are equipped with a first UWB module, and the smart guide cane is equipped with a second UWB module. Both the first and second UWB modules can transmit and receive pulse data. That is, the smart guide cane can send pulse data to the first UWB module in the wearable smart glasses through the second UWB module, and similarly, the wearable smart glasses can send pulse data to the second UWB module in the smart guide cane through the first UWB module.

[0089] 302. The wearable smart glasses receive pulse data sent by the smart guide cane through the second UWB module via the first UWB module.

[0090] 303. Wearable smart glasses determine the target location data of the smart guide cane based on pulse data.

[0091] In this embodiment of the invention, after receiving pulse data, the wearable smart glasses can analyze and process the pulse data to obtain the target position data of the smart guide cane.

[0092] Optionally, the pulse data sent by the smart guide cane includes both the pulse signal and the transmission time. This allows the wearable smart glasses to determine the transmission duration of the pulse signal between the wearable smart glasses and the smart guide cane based on the transmission and reception times. Combined with the signal transmission speed, the distance between the wearable smart glasses and the smart guide cane can be obtained, thereby determining the target position data of the smart guide cane.

[0093] 304. Wearable smart glasses acquire 3D images in real time through a camera.

[0094] 305. Wearable smart glasses establish a three-dimensional coordinate system based on the three-dimensional image.

[0095] In this embodiment of the invention, wearable smart glasses can establish a three-dimensional coordinate system in a three-dimensional image. Specifically, wearable smart glasses can establish a three-dimensional coordinate system with their own position as the origin, the direction of forward movement as the x-axis, the direction parallel to the sea surface and perpendicular to the forward movement as the y-axis, and the direction perpendicular to the sea surface as the z-axis.

[0096] 306. Wearable smart glasses determine the coordinates of the target location data in a three-dimensional coordinate system.

[0097] In this embodiment of the invention, after the wearable smart glasses establish a three-dimensional coordinate system, since the target position data includes: the direction of the smart guide cane relative to the wearable smart glasses, and the distance between the wearable smart glasses and the smart guide cane, the wearable smart glasses can bring the target position data into the three-dimensional coordinate system. In this way, the wearable smart glasses can determine the coordinate point corresponding to the target position data in the three-dimensional coordinate system, and there is only one such coordinate point.

[0098] 307. Wearable smart glasses identify objects corresponding to coordinate points in a 3D image and determine the target obstacle object that the smart guide cane is in contact with.

[0099] 308. Wearable smart glasses output prompts in a non-visual form, which are used to prompt the user to contact the target obstacle object with the smart guide cane.

[0100] In this embodiment of the invention, after the wearable smart glasses identify the target obstacle, they need to output a reminder to the user. However, since the user is blind, the wearable smart glasses can output the prompt information in a non-visual form to remind the user of the target obstacle that the smart guide cane is touching.

[0101] Optionally, non-visual methods can primarily be voice-based, meaning that wearable smart glasses are equipped with microphones. Once the wearable smart glasses identify a target obstacle, they can use the microphone to announce parameters such as the object's type, color, size, and whether it is movable.

[0102] This invention provides a method for determining obstacles. Wearable smart glasses can acquire target position data of a smart guide cane, and then a camera installed in the wearable smart glasses can acquire a three-dimensional image in real time. The method then identifies objects in the three-dimensional image that correspond to the target position data, thereby determining the target obstacle object that the smart guide cane is in contact with. In this solution, the wearable smart glasses can determine the obstacle object corresponding to the smart guide cane in the three-dimensional image based on the position data of the smart guide cane. This method is unaffected by various factors in the current environment, resulting in a more accurate obstacle detection result.

[0103] Furthermore, when determining the target location data of the smart guide cane, the wearable smart glasses can transmit pulse data through the first UWB module in the wearable smart glasses and the second UWB module in the smart guide cane to determine the target location data of the smart guide cane. A three-dimensional coordinate system is established in the three-dimensional image, and the target location data is input into this three-dimensional coordinate system to obtain the coordinate points corresponding to the target location data in the three-dimensional coordinate system to determine the target obstacle. Then, a non-visual prompt is output to alert the blind user to the target obstacle ahead. Through the above scheme, the obstacle determination method is unaffected by various environmental factors, effectively improving the accuracy of obstacle detection. Moreover, timely non-visual reminders to the user enhance the intelligence level of both the wearable smart glasses and the smart guide cane.

[0104] like Figure 4 As shown in the figure, an embodiment of the present invention provides a method for determining an obstacle, which may further include the following steps:

[0105] 401. When the smart guide cane detects that it is being held, it monitors the environment around the smart guide cane in real time.

[0106] In this embodiment of the invention, when the smart guide cane detects that it is in a gripping state, it means that the user is holding the smart guide cane. At this time, the smart guide cane can detect the surrounding environmental information.

[0107] Optionally, the environmental information may include whether there are obstacles or objects in the vicinity.

[0108] Optionally, the environment information of the smart guide cane can be detected in real time, which may include: transmitting a ranging signal through a distance sensor; if a reflected signal corresponding to the ranging signal is received within a preset time, it is determined that there is an obstacle within the target distance of the smart guide cane.

[0109] In this implementation, a distance sensor can be installed in the smart guide cane, which can measure the distance between the smart guide cane and surrounding objects.

[0110] It should be noted that this distance sensor can emit ranging signals in all directions. If there is an obstacle in a certain direction, the ranging signal emitted in that direction will bounce back to the distance sensor after hitting the obstacle. The distance sensor can then receive the reflected signal corresponding to the ranging signal. The distance sensor can determine the distance between the distance sensor and the obstacle based on the total time of the ranging signal bounce.

[0111] In this implementation, in order to detect whether there are obstacles within the target distance of the smart guide cane, the smart guide cane can be preset with a preset time based on the target distance. That is, if the reflected signal corresponding to the ranging signal is received within the preset time, it can be concluded that there are obstacles within the target distance of the smart guide cane. The preset time corresponds to the target distance and is calculated based on the target distance and the signal transmission speed.

[0112] 402. If the smart guide cane detects an obstacle within the target distance of the smart guide cane, it sends pulse data to the first UWB module set in the wearable smart glasses through the second UWB module.

[0113] 403. Wearable smart glasses receive pulse data transmitted by a smart guide cane via a signal transmitter through at least three signal receivers.

[0114] In this embodiment of the invention, the first UWB module of the wearable smart glasses may be equipped with at least three signal receivers, and the second UWB module of the smart guide cane may be equipped with a signal transmitter. The signal transmitter can simultaneously send pulse data to at least three signal receivers. Since the distance between each signal receiver and the signal transmitter is different, the time at which each signal receiver receives the pulse data is different.

[0115] 404. The wearable smart glasses obtain the signal transmission duration corresponding to each pulse data based on the signal transmission time included in the pulse data received by each of the at least three signal receivers, and the signal reception time corresponding to the pulse data received by each signal receiver.

[0116] In this embodiment of the invention, the pulse data sent by the signal transmitter in the smart guide cane includes the signal transmission time. In this way, after each signal receiver receives the pulse data, it can know the signal transmission time when the signal transmitter sent the pulse data to the signal receiver. By combining the signal reception time of the pulse data it receives, the signal transmission time of each pulse data from the signal transmitter to the signal receiver can be obtained.

[0117] 405. Wearable smart glasses determine the distance between each signal receiver and the signal transmitter in the smart guide cane based on the transmission duration of each signal, obtaining at least three distances.

[0118] In this embodiment of the invention, the wearable smart glasses can determine the distance between the signal receiver and the signal transmitter corresponding to each pulse data based on the signal transmission duration corresponding to each pulse data. The wearable smart glasses perform the same operation on each pulse data to obtain the distance between each signal receiver and the signal transmitter, i.e., at least three distances.

[0119] 406. Wearable smart glasses determine the target position data of the smart guide cane relative to the wearable smart glasses based on at least three distances.

[0120] In this embodiment of the invention, after the wearable smart glasses determine the distance between each signal receiver and signal transmitter, the at least three distances can be processed. Since the position of the wearable smart glasses is determined, that is, the positions of at least three signal receivers are determined, the wearable smart glasses can draw circles with the position of each signal receiver as the center and the distance between each signal receiver and signal transmitter as the radius, thus obtaining at least three circles. At this time, the wearable smart glasses can determine the intersection of the at least three circles as the target position data of the smart guide cane.

[0121] For example, such as Figure 5 As shown, assume that the wearable smart glasses 51 has three signal receivers: a first signal receiver 511 located on the left temple, a second signal receiver 512 located on the right temple, and a third signal receiver 513 located on the nose pad. The signal transmitter 521 in the smart guide cane 52 is located at the bottom of the smart guide cane 52. After the wearable smart glasses 51 acquires the three distances, a circle can be drawn with the first signal receiver 511 as the center and the distance r1 between the first signal receiver 511 and the signal transmitter 521 as the radius; then a circle can be drawn with the second signal receiver 512 as the center and the distance r2 between the second signal receiver 512 and the signal transmitter 521 as the radius; and then a circle can be drawn with the third signal receiver 513 as the center and the distance r3 between the third signal receiver 513 and the signal transmitter 521 as the radius. This will result in three circles, which have exactly one intersection point P. This intersection point P is the position of the signal transmitter 521, which is also the position of the smart guide cane 52.

[0122] 407. Wearable smart glasses acquire 3D images in real time through a camera.

[0123] 408. Wearable smart glasses identify objects in a 3D image that correspond to the target location data, thus identifying the target obstacle object that the smart guide cane is in contact with.

[0124] 409. Wearable smart glasses determine obstacle avoidance routes based on 3D images and target location data.

[0125] In practical applications, wearable smart glasses not only need to show blind users what the obstacles in front of them are, but also need to tell them how to avoid the obstacles. Therefore, in this embodiment of the invention, wearable smart glasses can determine the obstacle avoidance route that the user can take to avoid the target obstacle based on the three-dimensional image and the target position data.

[0126] 410. Wearable smart glasses output obstacle avoidance routes through non-visual means.

[0127] In this embodiment of the invention, after the wearable smart glasses determine the obstacle avoidance route, it is necessary to output a reminder to the user. However, since the user is blind, the wearable smart glasses can output the obstacle avoidance route in a non-visual form to prompt the user on how to avoid the target obstacle ahead.

[0128] Optionally, the non-visual form can be mainly voice-based, meaning that the wearable smart glasses are equipped with a microphone, and once the wearable smart glasses determine the obstacle avoidance route, the obstacle avoidance route is broadcast aloud through the microphone.

[0129] This invention provides a method for determining obstacles. Wearable smart glasses can acquire target position data of a smart guide cane, and then a camera installed in the wearable smart glasses can acquire a three-dimensional image in real time. The method then identifies objects in the three-dimensional image that correspond to the target position data, thereby determining the target obstacle object that the smart guide cane is in contact with. In this solution, the wearable smart glasses can determine the obstacle object corresponding to the smart guide cane in the three-dimensional image based on the position data of the smart guide cane. This method is unaffected by various factors in the current environment, resulting in a more accurate obstacle detection result.

[0130] Furthermore, when the smart guide cane is held and an obstacle is present within the target distance, it can send pulse data via a UWB module to at least three signal receivers within the first UWB module of the wearable smart glasses. The wearable smart glasses can then determine the target location data of the smart guide cane using the pulse data received by the at least three signal receivers. Additionally, the wearable smart glasses can provide the user with a non-visual route to avoid the target obstacle. This approach makes the obstacle determination method unaffected by environmental factors, effectively improving the accuracy of obstacle detection. Moreover, it can promptly remind the user of obstacle avoidance routes through non-visual means, enhancing the intelligence level of both the wearable smart glasses and the smart guide cane.

[0131] As an alternative implementation, wearable smart glasses can receive voice commands from users, determine the target path based on the destination included in the voice command, and output the target path in a non-visual form.

[0132] In real-world applications, blind users may sometimes actively want to go to a certain place, such as a roadside bench, shop, or bus stop. In this case, the blind user only needs to say where they want to go, and the microphone in the wearable smart glasses can collect the user's voice command. After determining the destination in a 3D image, the glasses can determine the target path based on the current location and the location of the destination. Finally, the wearable smart glasses can output the target path in a non-visual form to guide the user to the destination.

[0133] With this optional implementation, wearable smart glasses can determine the path to the user's desired destination based on the user's voice commands, thereby guiding the user there. This allows for real-time response to user needs and improves the intelligence level of wearable smart glasses.

[0134] like Figure 6 As shown, this embodiment of the invention provides wearable smart glasses, which include a camera and include:

[0135] The acquisition module 601 is used to acquire target location data of the smart guide cane;

[0136] The acquisition module 601 is also used to acquire three-dimensional images in real time via a camera;

[0137] The processing module 602 is used to identify the object in the three-dimensional image that corresponds to the target position data, and to obtain the target obstacle object that the smart guide cane is in contact with.

[0138] Optionally, the wearable smart glasses may include a first ultra-wideband (UWB) module.

[0139] The wearable smart glasses also include:

[0140] The transceiver module 603 is used to receive pulse data sent by the smart guide cane through the second UWB module via the first UWB module;

[0141] The processing module 602 is specifically used to determine the target position data of the smart guide cane based on the pulse data.

[0142] Optionally, the first UWB module includes at least three signal receivers, and the second UWB module includes a signal transmitter;

[0143] The transceiver module 603 is specifically used to receive pulse data transmitted by the smart guide cane through the signal transmitter via at least three signal receivers;

[0144] The processing module 602 is specifically used to obtain the signal transmission duration corresponding to each pulse data based on the signal transmission time included in the pulse data received by each of the at least three signal receivers and the signal reception time corresponding to the pulse data received by each signal receiver.

[0145] The processing module 602 is specifically used to determine the distance between each signal receiver and the signal transmitter in the smart guide cane based on the transmission duration of each signal, and obtain at least three distances;

[0146] The processing module 602 is specifically used to determine the target position data of the smart guide cane relative to the wearable smart glasses based on at least three distances.

[0147] Optionally, the processing module 602 is specifically used to establish a three-dimensional coordinate system based on the three-dimensional image;

[0148] Processing module 602 is specifically used to determine the coordinate points corresponding to the target position data in the three-dimensional coordinate system;

[0149] The processing module 602 is specifically used to identify objects corresponding to coordinate points in the three-dimensional image and determine the target obstacle object that the smart guide cane is in contact with.

[0150] Optionally, the processing module 602 is also used to output prompt information in a non-visual form, the prompt information being used to prompt the user about the target obstacle object that the smart guide cane is in contact with.

[0151] Optionally, the processing module 602 is also used to determine the obstacle avoidance route based on the 3D image and the target position data;

[0152] The processing module 602 is also used to output obstacle avoidance routes in a non-visual manner.

[0153] In this embodiment of the invention, each module can implement the obstacle determination method provided in the above method embodiment and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0154] like Figure 7 As shown, an embodiment of the present invention provides a smart guide cane, which includes a second UWB module. The smart guide cane includes:

[0155] The transceiver module 701 is used to send pulse data to the first UWB module set in the wearable smart glasses through the second UWB module when the smart guide cane is detected to be in a gripped state. This allows the wearable smart glasses to determine the target position data of the smart guide cane based on the pulse data, and to identify the object corresponding to the target position data in the three-dimensional image, thereby obtaining the target obstacle object that the smart guide cane is in contact with. The three-dimensional image is acquired in real time by the camera in the wearable smart glasses.

[0156] Optionally, the smart guide cane also includes:

[0157] The processing module 702 is used to detect the environmental information of the smart guide cane in real time when it is detected that the smart guide cane is in a gripping state.

[0158] The transceiver module 701 is specifically used to send pulse data to the first UWB module set in the wearable smart glasses through the second UWB module if an obstacle is detected within the target distance of the smart guide cane.

[0159] Optionally, the transceiver module 701 is specifically used to transmit ranging signals via a distance sensor;

[0160] The processing module 702 is specifically used to determine that there is an obstacle within the target distance of the smart guide cane if a reflected signal corresponding to the ranging signal is received within a preset time period.

[0161] The preset duration corresponds to the target distance.

[0162] In this embodiment of the invention, each module can implement the obstacle determination method provided in the above method embodiment and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0163] like Figure 8 As shown, this embodiment of the invention also provides wearable smart glasses, which are equipped with a camera. The wearable smart glasses include:

[0164] Memory 801 storing executable program code;

[0165] Processor 802 coupled to memory 801;

[0166] Specifically, the processor 802 calls the executable program code stored in the memory 801 to execute the obstacle determination method performed by the wearable smart glasses in the above method embodiments.

[0167] like Figure 9 As shown, this embodiment of the invention also provides a smart guide cane, which includes a second UWB module. The smart guide cane includes:

[0168] Memory 901 storing executable program code;

[0169] Processor 902 coupled to memory 901;

[0170] In this process, the processor 902 calls the executable program code stored in the memory 901 to execute the obstacle determination method of the smart guide cane in the above method embodiments.

[0171] This invention provides a computer-readable storage medium storing a computer program that causes a computer to perform some or all of the steps of the methods described in the above embodiments.

[0172] This invention also provides a computer program product, wherein when the computer program product is run on a computer, the computer performs some or all of the steps of the methods described in the above method embodiments.

[0173] This invention also provides an application publishing platform, which is used to publish computer program products. When the computer program products are run on a computer, the computer performs some or all of the steps of the methods described in the above method embodiments.

[0174] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also recognize that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to the invention.

[0175] In various embodiments of the present invention, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0176] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they can be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0177] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0178] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several requests to cause a computer device (which can be a personal computer, server, or network device, specifically a processor in the computer device) to execute some or all of the steps of the methods described in the various embodiments of the present invention.

[0179] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

Claims

1. A method for determining obstacles, characterized in that, Applied to wearable smart glasses, the wearable smart glasses being equipped with a camera and a first UWB module, the method includes: Obtain target position data of the smart guide cane; the target position data is the relative position of the smart guide cane with respect to the wearable smart glasses, or the target position data is the absolute position of the smart guide cane; The camera captures 3D images in real time. The object corresponding to the target location data in the three-dimensional image is identified to obtain the target obstacle object that the smart guide cane is in contact with; The smart guide cane is equipped with a second UWB module. The process of acquiring the target position data of the smart guide cane includes: The first UWB module receives pulse data sent by the smart guide cane through the second UWB module, and determines the target position data of the smart guide cane based on the pulse data. The step of identifying the object in the three-dimensional image corresponding to the target location data to obtain the target obstacle object contacted by the smart guide cane includes: Based on the aforementioned three-dimensional image, establish a three-dimensional coordinate system; Determine the coordinate point corresponding to the target position data in the three-dimensional coordinate system; The object corresponding to the coordinate point in the three-dimensional image is identified to determine the target obstacle object that the smart guide cane is in contact with.

2. The method according to claim 1, characterized in that, The first UWB module includes at least three signal receivers, and the second UWB module includes a signal transmitter; the step of receiving pulse data transmitted by the smart guide cane through the second UWB module via the first UWB module, and determining the target position data of the smart guide cane based on the pulse data, includes: The at least three signal receivers respectively receive the pulse data transmitted by the intelligent guide cane through the signal transmitter; The signal transmission duration corresponding to each pulse data is obtained based on the signal transmission time included in the pulse data received by each of the at least three signal receivers, and the signal reception time corresponding to the time when each signal receiver receives the pulse data. Based on the transmission duration of each signal, the distance between each signal receiver and the signal transmitter in the smart guide cane is determined, resulting in at least three distances; The target position data of the smart guide cane relative to the wearable smart glasses is determined based on the at least three distances.

3. The method according to claim 1 or 2, characterized in that, After identifying the object corresponding to the target location data in the three-dimensional image and obtaining the target obstacle object contacted by the smart guide cane, the method further includes: The system outputs prompts in a non-visual manner, which are used to alert the user to the target obstacle object that the smart guide cane is in contact with.

4. The method according to claim 1 or 2, characterized in that, After identifying the object corresponding to the target location data in the three-dimensional image and obtaining the target obstacle object contacted by the smart guide cane, the method further includes: Based on the three-dimensional image and the target location data, an obstacle avoidance route is determined; The obstacle avoidance route is output in a non-visual manner.

5. A method for determining an obstacle, characterized in that, The method, applied to a smart guide cane for the visually impaired, wherein the smart guide cane is equipped with a second UWB module, includes: When the smart guide cane is detected to be in a gripped state, pulse data is sent from the second UWB module to the first UWB module in the wearable smart glasses. This allows the wearable smart glasses to determine the target position data of the smart guide cane based on the pulse data, and to establish a three-dimensional coordinate system based on the three-dimensional image. The wearable smart glasses then determine the coordinate point corresponding to the target position data in the three-dimensional coordinate system, identify the object corresponding to the coordinate point in the three-dimensional image, and determine the target obstacle object that the smart guide cane is in contact with. The three-dimensional image is acquired in real time by the camera in the wearable smart glasses. The target position data is the relative position of the smart guide cane with respect to the wearable smart glasses, or the target position data is the absolute position of the smart guide cane.

6. The method according to claim 5, characterized in that, When the smart guide cane is detected to be in a gripped state, pulse data is sent from the second UWB module to the first UWB module in the wearable smart glasses, including: When the smart guide cane is detected to be in the gripping state, the environmental information of the smart guide cane is detected in real time. If an obstacle is detected within the target distance of the smart guide cane, the pulse data is sent from the second UWB module to the first UWB module installed in the wearable smart glasses.

7. The method according to claim 6, characterized in that, The smart guide cane is equipped with a distance sensor, and the real-time detection of the environmental information of the smart guide cane includes: The distance sensor transmits a ranging signal; If a reflected signal corresponding to the ranging signal is received within a preset time period, it is determined that there is an obstacle within the target distance of the smart guide cane. The preset duration corresponds to the target distance.

8. A wearable smart glasses, characterized in that, The wearable smart glasses include a camera and a first UWB module, and the wearable smart glasses include: The acquisition module is used to acquire target position data of the smart guide cane; the target position data is the relative position of the smart guide cane with respect to the wearable smart glasses, or the target position data is the absolute position of the smart guide cane. The acquisition module is also used to acquire three-dimensional images in real time through the camera; The processing module is used to identify the object in the three-dimensional image that corresponds to the target position data, and to obtain the target obstacle object that the smart guide cane is in contact with; The smart guide cane is equipped with a second UWB module. The acquisition module is also used to receive pulse data sent by the smart guide cane through the second UWB module via the first UWB module, and determine the target position data of the smart guide cane based on the pulse data. The processing module is further configured to establish a three-dimensional coordinate system based on the three-dimensional image; determine the coordinate point corresponding to the target position data in the three-dimensional coordinate system; identify the object corresponding to the coordinate point in the three-dimensional image; and determine the target obstacle object that the smart guide cane is in contact with.

9. A smart guide cane, characterized in that, The smart guide cane is equipped with a second UWB module, and the smart guide cane includes: The transceiver module is used to send pulse data to the first UWB module in the wearable smart glasses via the second UWB module when the smart guide cane is detected to be in a gripped state. This allows the wearable smart glasses to determine the target position data of the smart guide cane based on the pulse data, and to establish a three-dimensional coordinate system based on a three-dimensional image. The module then determines the coordinate point corresponding to the target position data in the three-dimensional coordinate system and identifies the object corresponding to the coordinate point in the three-dimensional image to determine the target obstacle object contacted by the smart guide cane. The three-dimensional image is acquired in real time by a camera in the wearable smart glasses. The target position data is either the relative position of the smart guide cane with respect to the wearable smart glasses, or the absolute position of the smart guide cane.

10. A computer-readable storage medium, characterized in that, include: The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the obstacle determination method as described in any one of claims 1 to 4, or the obstacle determination method as described in any one of claims 5 to 7.

Images