Visual impairment auxiliary perception method and device, electronic equipment and storage medium

Abstract

The application provides a visual impairment auxiliary perception method and device, electronic equipment and storage medium. The method comprises: acquiring position information of a target obstacle; wherein the position information of the target obstacle comprises distance information of the target obstacle, angle information of the target obstacle and direction information of the target obstacle; controlling a first stimulation device to output a first stimulation signal and a second stimulation device to output a second stimulation signal according to the position information of the target obstacle; wherein the stimulation intensity of the first stimulation signal and the second stimulation signal is negatively correlated with the distance information, the sequence relationship between the starting time of the first stimulation signal and the starting time of the second stimulation signal represents the direction information, and the time difference between the starting time of the first stimulation signal and the starting time of the second stimulation signal is positively correlated with the angle information. The application improves the accuracy of visual impairment auxiliary perception.

Description

Technical Field

[0001] This application relates to communication technology, and more particularly to a visually impaired assistive sensing method, device, electronic device, and storage medium. Background Technology

[0002] Visually impaired individuals have partially or completely impaired visual function and are unable to perceive obstacles independently. Currently, they primarily rely on white canes or electronic guide devices to perceive obstacles.

[0003] When visually impaired people use a white cane to perceive obstacles, the length of the cane limits their ability to accurately perceive obstacles at different distances. When visually impaired people use electronic guide devices to perceive obstacles, these devices mostly use a single information source to determine the distance to obstacles, which also limits their ability to accurately perceive obstacles in different directions.

[0004] The aforementioned existing technologies cannot achieve accurate perception of the location of obstacles for visually impaired individuals, resulting in a low accuracy rate for visually impaired assisted perception. Summary of the Invention

[0005] This application provides a visually impaired assistive sensing method, device, electronic device, and storage medium, aiming to solve the problem of low accuracy in visually impaired assistive sensing.

[0006] In a first aspect, this application provides a visually impaired assistive perception method applied to a visually impaired assistive perception device. The visually impaired assistive perception device includes a first stimulation device representing a first direction and a second stimulation device representing a second direction, wherein the first direction is opposite to the second direction. The method includes: acquiring position information of a target obstacle; wherein the position information of the target obstacle includes distance information, angle information, and orientation information of the target obstacle; the angle information of the target obstacle is the offset angle of the target obstacle's location relative to the user's direct front; the orientation information of the target obstacle is the offset direction of the target obstacle's location relative to the user's direct front; and, based on the position information of the target obstacle, controlling the first stimulation device to output a first stimulation signal and controlling the second stimulation device to output a second stimulation signal; wherein the stimulation intensity of the first stimulation signal and the second stimulation signal is negatively correlated with the distance information, the sequential relationship between the start time of the first stimulation signal and the start time of the second stimulation signal represents the orientation information, and the time difference between the start time of the first stimulation signal and the start time of the second stimulation signal is positively correlated with the angle information.

[0007] Optionally, before controlling the first stimulation device to output a first stimulation signal and the second stimulation device to output a second stimulation signal based on the position information of the target obstacle, the method further includes: performing a correlation calculation on the angle information of the target obstacle to obtain the time difference.

[0008] Optionally, before controlling the first stimulation device to output a first stimulation signal and the second stimulation device to output a second stimulation signal based on the position information of the target obstacle, the method further includes: if the orientation information is the first direction, determining the start time of the first stimulation signal; and taking the time corresponding to the sum of the start time of the first stimulation signal and the time difference as the start time of the second stimulation signal; if the orientation information is the second direction, determining the start time of the second stimulation signal; and taking the time corresponding to the sum of the start time of the second stimulation signal and the time difference as the start time of the first stimulation signal; if the orientation information is without offset, determining the start times of the first stimulation signal and the second stimulation signal, and the start times of the first stimulation signal and the second stimulation signal are the same.

[0009] Optionally, before controlling the first stimulation device to output a first stimulation signal and the second stimulation device to output a second stimulation signal based on the location information of the target obstacle, the method further includes: calculating the product of the reciprocal of the distance information and a predetermined intensity coefficient to obtain a calculation result; and obtaining the intensity amplitude of the first stimulation signal and the intensity amplitude of the second stimulation signal based on the calculation result.

[0010] Optionally, obtaining the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal based on the calculation result includes: if the orientation information is the first direction, then using the calculation result as the intensity amplitude of the second stimulus signal; and calculating the sum of the calculation result and a preset amplitude increment to obtain the intensity amplitude of the first stimulus signal; if the orientation information is the second direction, then using the calculation result as the intensity amplitude of the first stimulus signal; and calculating the sum of the calculation result and a preset amplitude increment to obtain the intensity amplitude of the second stimulus signal; if the orientation information is that there is no offset, then using the calculation result as the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal.

[0011] Optionally, obtaining the location information of the target obstacle includes: detecting obstacles within a predetermined range near the user; if multiple obstacles are detected, obtaining the distance information of each obstacle, and taking the obstacle with the smallest distance information as the target obstacle, and obtaining the orientation information and angle information of the target obstacle.

[0012] Optionally, the first stimulus signal and the second stimulus signal are digital signals; the method further includes: acquiring the moving speed of the target obstacle; determining the stimulation frequency of the first stimulus signal and the second stimulus signal based on the moving speed, wherein the stimulation frequency is positively correlated with the moving speed.

[0013] Secondly, this application provides a visually impaired assistive sensing device, the device comprising: a first stimulation device representing a first direction, a second stimulation device representing a second direction, a position acquisition module, and a control module, wherein the first direction is opposite to the second direction; the position acquisition module is used to acquire position information of a target obstacle; wherein the position information of the target obstacle includes distance information of the target obstacle, angle information of the target obstacle, and orientation information of the target obstacle; the angle information of the target obstacle is the offset angle of the target obstacle's location relative to the user's direct front; the orientation information of the target obstacle is the offset direction of the target obstacle's location relative to the user's direct front; the control module is used to control the first stimulation device to output a first stimulation signal and control the second stimulation device to output a second stimulation signal according to the position information of the target obstacle; wherein the stimulation intensity of the first stimulation signal and the second stimulation signal is negatively correlated with the distance information, the sequential relationship between the start time of the first stimulation signal and the start time of the second stimulation signal represents the orientation information, and the time difference between the start time of the first stimulation signal and the start time of the second stimulation signal is positively correlated with the angle information.

[0014] Optionally, the device further includes: a first calculation module, used to perform correlation calculation on the angle information of the target obstacle to obtain the time difference.

[0015] Optionally, the device further includes: a time determination module; the time determination module is configured to: if the orientation information is the first direction, determine the start time of the first stimulus signal; and take the time corresponding to the sum of the start time of the first stimulus signal and the time difference as the start time of the second stimulus signal; if the orientation information is the second direction, determine the start time of the second stimulus signal; and take the time corresponding to the sum of the start time of the second stimulus signal and the time difference as the start time of the first stimulus signal; if the orientation information is without offset, determine the start times of the first stimulus signal and the second stimulus signal, and the start times of the first stimulus signal and the second stimulus signal are the same.

[0016] Optionally, the device further includes: a second calculation module, configured to calculate the product of the reciprocal of the distance information and a predetermined intensity coefficient to obtain a calculation result; and an intensity determination module, configured to obtain the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal based on the calculation result.

[0017] Optionally, the intensity determination module is specifically used for: if the orientation information is the first direction, then using the calculation result as the intensity amplitude of the second stimulus signal; and calculating the sum of the calculation result and a preset amplitude increment to obtain the intensity amplitude of the first stimulus signal; if the orientation information is the second direction, then using the calculation result as the intensity amplitude of the first stimulus signal; and calculating the sum of the calculation result and a preset amplitude increment to obtain the intensity amplitude of the second stimulus signal; if the orientation information is without offset, then using the calculation result as the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal.

[0018] Optionally, the location acquisition module includes: a detection unit, used to detect obstacles within a predetermined range near the user; and an acquisition unit, used to acquire distance information of each obstacle if multiple obstacles are detected, and to take the obstacle with the smallest distance information as the target obstacle, and to acquire the orientation information and angle information of the target obstacle.

[0019] Optionally, the first stimulus signal and the second stimulus signal are digital signals; the device further includes: a speed acquisition module for acquiring the moving speed of the target obstacle; and a frequency determination module for determining the stimulation frequency of the first stimulus signal and the second stimulus signal based on the moving speed, wherein the stimulation frequency is positively correlated with the moving speed.

[0020] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method described above.

[0021] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described above.

[0022] The visually impaired assistive sensing method, device, electronic device, and storage medium provided in this application acquire the position information of a target obstacle; wherein, the position information of the target obstacle includes the distance information of the target obstacle, the angle information of the target obstacle, and the orientation information of the target obstacle; based on the position information of the target obstacle, a first stimulation device is controlled to output a first stimulation signal and a second stimulation device is controlled to output a second stimulation signal; wherein, the stimulation intensity of the first stimulation signal and the second stimulation signal are negatively correlated with the aforementioned distance information, the sequential relationship between the start time of the first stimulation signal and the start time of the second stimulation signal represents the aforementioned orientation information, and the time difference between the start time of the first stimulation signal and the start time of the second stimulation signal is positively correlated with the aforementioned angle information. The solution of this application uses two stimulation devices representing opposite directions to output a first stimulation signal and a second stimulation signal to the user. The user can determine the orientation information of the target obstacle by the sequence of the start time of the first stimulation signal and the start time of the second stimulation signal, determine the angle information of the target obstacle by the time difference between the start times of the first stimulation signal and the second stimulation signal, and determine the distance information of the target obstacle by the stimulation intensity of the first stimulation signal and the second stimulation signal. This enables visually impaired people to accurately perceive the location of the target obstacle and improves the accuracy of visually impaired assisted perception. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0024] Figure 1 This is a flowchart illustrating a visually impaired assistive perception method provided in Embodiment 1 of this application;

[0025] Figure 2 This is a flowchart illustrating a visually impaired assistive perception method provided in Embodiment 2 of this application;

[0026] Figure 3 This is a flowchart illustrating a visually impaired assistive perception method provided in Embodiment 3 of this application;

[0027] Figure 4 This is a spatial diagram illustrating four positional scenarios provided in Embodiment 4 of this application;

[0028] Figure 5 This is a schematic diagram of the stimulation device output for four different positional scenarios provided in Embodiment 4 of this application;

[0029] Figure 6 This is a schematic diagram of the structure of a visually impaired assistive sensing device provided in Embodiment 5 of this application;

[0030] Figure 7This is a schematic diagram of the structure of the electronic device provided in Embodiment Six of this application.

[0031] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0032] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0033] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0034] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities and do not necessarily imply a specific order or sequence, unless otherwise indicated. It should be understood that such terms can be used interchangeably where appropriate, for example, to implement the application in a sequence other than those given in the embodiments illustrated or described herein.

[0035] Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover but not exclude inclusion. For example, a product or device that includes a series of components is not necessarily limited to those explicitly listed, but may include other components not explicitly listed or inherent to such product or device. As used in this application, the term "module" means any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0036] Because visually impaired individuals suffer partial or complete impairment of their visual function, they are unable to perceive obstacles independently, leading to numerous inconveniences in their travel and daily lives. Ensuring the safety and convenience of visually impaired individuals has become a focus of social concern. Currently, visually impaired individuals primarily rely on white canes or electronic guide devices to perceive obstacles.

[0037] When visually impaired individuals use a white cane to perceive obstacles, the cane's length limitation forces them to use a three-point tapping method, limiting their ability to perceive obstacles within a fixed distance. This fails to assist them in accurately perceiving obstacles at varying distances. Similarly, when using electronic navigation devices, these devices typically rely on a single information source to determine obstacle distances, hindering their ability to accurately perceive obstacles in different directions. Understandably, neither cane nor electronic navigation devices provide a reliable method for visually impaired individuals to accurately pinpoint obstacle locations, resulting in a low accuracy rate for assisted perception.

[0038] The visually impaired assistive perception method provided in this application involves two stimulation devices in opposite directions outputting a first stimulation signal and a second stimulation signal to the user. The user can determine the orientation information of the target obstacle by the sequence of the start times of the first and second stimulation signals, the angle information of the target obstacle by the time difference between the start times of the first and second stimulation signals, and the distance information of the target obstacle by the stimulation intensity of the first and second stimulation signals. This enables visually impaired individuals to accurately perceive the location of the target obstacle, improving the accuracy of visually impaired assistive perception.

[0039] The technical solutions of this application will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. In the description of this application, unless otherwise expressly specified and limited, the terms should be broadly understood within the art. The embodiments of this application will now be described with reference to the accompanying drawings.

[0040] Example 1

[0041] Figure 1 This is a flowchart illustrating a visually impaired assistive sensing method according to Embodiment 1 of this application. The method in this embodiment is applied to a visually impaired assistive sensing device, which includes a first stimulation device representing a first direction and a second stimulation device representing a second direction. The first direction is opposite to the second direction, as shown below. Figure 1 As shown, the method includes the following steps:

[0042] S101. Obtain the location information of the target obstacle;

[0043] S102. Based on the location information of the target obstacle, control the first stimulation device to output a first stimulation signal and control the second stimulation device to output a second stimulation signal.

[0044] In practical applications, the executing entity of this visually impaired assistive sensing method can be a visually impaired assistive sensing device. There are various ways to implement a visually impaired assistive sensing device. For example, it can be implemented through a computer program, such as application software; or, for example, a chip. It can also be implemented as a medium storing the relevant computer program, such as a USB flash drive or cloud storage; or, it can be implemented through a physical device that integrates or installs the relevant computer program, such as a server.

[0045] The location information of the target obstacle refers to the position of the target obstacle relative to the user; the location information of the target obstacle includes the distance information of the target obstacle, the angle information of the target obstacle, and the orientation information of the target obstacle; the angle information of the target obstacle is the offset angle of the target obstacle's position relative to the front of the user; the orientation information of the target obstacle is the offset direction of the target obstacle's position relative to the front of the user.

[0046] In this embodiment, the location information of the target obstacle can be one of the following: directly in front of the user, to the left of the user's direct front, or to the right of the user's direct front. In one example, an obstacle can be detected using a target detection algorithm. Optionally, a deep learning-based target detection method can acquire images of the areas directly in front of the user, to the left of the user's direct front, and to the right of the user's direct front, respectively. By recognizing the features of the acquired images, the existence of an obstacle and its location information can be determined, and the detected obstacle can be taken as the target obstacle.

[0047] The distance information of the target obstacle refers to the straight-line distance between the target obstacle and the user. For example, target information of the target obstacle can be obtained through methods such as monocular ranging, binocular ranging, and radar ranging. For instance, a radar signal is sent to the target obstacle, and the radar signal reflected from the target obstacle is received. The transmission time of the radar signal is determined, and the product of the radar signal transmission speed and the transmission time is calculated. The result of dividing the product by 2 is used as the distance information of the target obstacle.

[0048] For example, a Cartesian coordinate system is established by taking the user's location as the origin, the direction directly in front of the user as the positive direction of the X-axis, and the direction offset 90° to the right of the direction directly in front of the user as the positive direction of the Y-axis. In this embodiment, the angle information of the target obstacle ranges from 0° to 90°.

[0049] Specifically, in S101, the location information of the target obstacle is acquired. For example, the visually impaired assistive sensing device also includes a first transceiver representing the first direction and a second transceiver representing the second direction. The first and second transceivers simultaneously send measurement signals to the target obstacle. The first and second transceivers receive the measurement signals reflected by the target obstacle. The direction represented by the transceiver that first receives the measurement signal reflected by the target obstacle is taken as the orientation information of the target obstacle. Let the coordinates of the target obstacle's location be (x, y), the relative positions of the first and second transceivers with the user be fixed, and the coordinates of the first and second transceivers be (x1, y1) and (x2, y2) respectively. Let the velocity of the measurement signal be v, the time difference between the first transceiver receiving and sending the measurement signal be t1, and the time difference between the first transceiver receiving and sending the measurement signal be t2. Then, the equation 1 corresponding to the distance between the location of the first transceiver and the location of the target obstacle is:

[0050]

[0051] Similarly, Equation 2 corresponding to the distance between the location of the first transceiver and the location of the target obstacle is:

[0052]

[0053] Establish a system of equations 1 and 2, solve for the coordinates of the target obstacle, use trigonometric formulas to find the angle of the target obstacle, and use the distance formula between two points to determine the distance of the target obstacle.

[0054] In this embodiment, the stimulation intensity of the first and second stimulus signals is negatively correlated with the distance information. The sequential relationship between the start times of the first and second stimulus signals represents the orientation information. The time difference between the start times of the first and second stimulus signals is positively correlated with the angle information. The first and second stimulus signals uniquely represent the location information of the target obstacle.

[0055] It is understandable that the first and second directions are opposite; the first stimulation device represents the first direction, and the second stimulation device represents the second direction. Therefore, the positions of the first and second stimulation devices are opposite. For example, if the first direction is the left side directly in front of the user, and the second direction is the right side directly in front of the user, the first stimulation device is located on the left side directly in front of the user, and the second stimulation device is located on the right side directly in front of the user. Conversely, if the first direction is the right side directly in front of the user, and the second direction is the left side directly in front of the user, the first stimulation device is located on the right side directly in front of the user, and the second stimulation device is located on the left side directly in front of the user. In practical applications, the first and second stimulation devices can be located on sensitive areas on either side of the user's body. For example, if the first direction is the right side directly in front of the user, and the second direction is the left side directly in front of the user, the first stimulation device is worn on the user's right wrist, and the second stimulation device is worn on the user's left wrist.

[0056] Specifically, after acquiring the location information of the target obstacle, the system controls a first stimulation device and a second stimulation device representing opposite directions to output a first stimulation signal and a second stimulation signal to the user, respectively. Based on the stimulation intensity of the first and second stimulation signals, the user can determine the distance information of the target obstacle; based on the sequential relationship between the start times of the first and second stimulation signals, the user can determine the orientation information of the target obstacle; and based on the time difference between the start times of the first and second stimulation signals, the user can determine the angle information of the target obstacle. In other words, based on the orientation information, angle information, and distance information of the target obstacle, the user can perceive the precise position of the target obstacle relative to the user.

[0057] Furthermore, when there may be multiple obstacles in the area in front of the user, the obstacle closest to the user can be used as the target obstacle, and the location information of the target obstacle can be obtained. In one possible implementation, the above S101 includes:

[0058] Detect obstacles within a predetermined range near the user;

[0059] If multiple obstacles are detected, the distance information of each obstacle is obtained, and the obstacle with the smallest distance information is taken as the target obstacle. The orientation information and angle information of the target obstacle are then obtained.

[0060] As an example, obstacles within a predetermined range in front of a user can be detected using computer vision algorithms. For instance, based on the salientity detection (SD) method, images of a predetermined range near the user are acquired, and obstacles within that range are detected by extracting salient features from the acquired images.

[0061] In practical applications, multiple obstacles may exist within a predetermined range near the user, and the positions of these obstacles relative to the user change as the user moves. When multiple obstacles exist within this range, to ensure the user's safety, the user needs to perceive the obstacle with the greatest impact and avoid collisions. It's understandable that the smaller the distance to an obstacle, the greater its impact on the user. Therefore, identifying the obstacle with the smallest distance as the target obstacle, obtaining its orientation and angle information, and perceiving it can effectively improve the accuracy of visually impaired assisted perception.

[0062] In this embodiment, distance information of multiple obstacles is obtained, and the obstacle with the smallest distance information is selected as the target obstacle. Then, the orientation information and angle information of the target obstacle are obtained. This allows the user to perceive the obstacle with the greatest impact, thus improving the accuracy of visual impairment assisted perception.

[0063] In practical applications, the target obstacle may be moving. Users can perceive the obstacle's speed in time and take timely action to ensure their safety. In one possible implementation, the first and second stimulus signals are digital signals. The method further includes:

[0064] Obtain the movement speed of the aforementioned target obstacle;

[0065] Based on the aforementioned movement speed, the stimulation frequencies of the first and second stimulation signals are determined.

[0066] The target obstacle's movement speed includes both its speed and direction of movement. As an example, an acquisition period is set. When the acquisition period arrives, the target obstacle's current position information is acquired. Based on the target obstacle's current position information and the position information obtained in the most recent acquisition period, the target obstacle's movement speed within the acquisition period is determined.

[0067] In practical applications, the direction of movement of a target obstacle can determine whether it is approaching or moving away from the user. The closer the obstacle gets to the user, the higher its speed, and the more likely a collision is to occur. Therefore, by sensing the speed of the obstacle in time, users can avoid collisions and ensure their safety.

[0068] The stimulation frequencies of the first and second stimulus signals are positively correlated with the moving speed of the target obstacle. For example, the first and second stimulus signals can be pulse signals with a 50% duty cycle, and the stimulation frequency refers to the number of pulses generated per unit time. The greater the moving speed of the target obstacle, the higher the stimulation frequency, and the more pulses generated per unit time. Therefore, the user can determine the moving speed of the target obstacle based on the number of pulses generated per unit time by the first and second stimulus signals.

[0069] In this embodiment, the moving speed of the target obstacle is obtained, and the stimulation frequency of the first stimulus signal and the second stimulus signal is determined based on the moving speed of the target obstacle. The user can perceive the moving speed of the target obstacle based on the stimulation frequency of the first stimulus signal and the second stimulus signal, thereby improving the safety of the user's travel.

[0070] The visually impaired assistive perception method provided in this application acquires the location information of a target obstacle; wherein, the location information of the target obstacle includes the distance information, the angle information, and the orientation information of the target obstacle; based on the location information of the target obstacle, a first stimulation device is controlled to output a first stimulation signal and a second stimulation device is controlled to output a second stimulation signal; wherein, the stimulation intensity of the first stimulation signal and the second stimulation signal are negatively correlated with the aforementioned distance information, the sequential relationship between the start time of the first stimulation signal and the start time of the second stimulation signal represents the aforementioned orientation information, and the time difference between the start time of the first stimulation signal and the start time of the second stimulation signal is positively correlated with the aforementioned angle information. In the process of assistive perception for the visually impaired, two stimulation devices representing opposite directions output a first stimulation signal and a second stimulation signal to the user, respectively. The user can determine the orientation information of the target obstacle by the sequential relationship between the start time of the first stimulation signal and the start time of the second stimulation signal, determine the angle information of the target obstacle by the time difference between the start time of the first stimulation signal and the start time of the second stimulation signal, and determine the distance information of the target obstacle by the stimulation intensity of the first stimulation signal and the second stimulation signal. This enables the visually impaired to accurately perceive the location of the obstacle and improves the accuracy of assistive perception for the visually impaired.

[0071] Example 2

[0072] Figure 2 This is a flowchart illustrating a visually impaired assistive perception method provided in Embodiment 2 of this application, as shown below. Figure 2 As shown, based on the above embodiment, before S102, the method further includes:

[0073] S201. Perform correlation calculation on the angle information of the above-mentioned target obstacle to obtain the above-mentioned time difference;

[0074] S202. Determine whether the above orientation information is offset;

[0075] S203. If the above orientation information is without offset, then determine the start time of the first stimulus signal and the second stimulus signal, and the start time of the first stimulus signal and the second stimulus signal are the same.

[0076] S204. If the above directional information is offset, determine whether the above directional information is the first direction.

[0077] S205. If the aforementioned orientation information is the aforementioned first direction, then the start time of the aforementioned first stimulus signal is determined; and the time corresponding to the sum of the start time of the aforementioned first stimulus signal and the aforementioned time difference is taken as the start time of the aforementioned second stimulus signal.

[0078] S206. If the above-mentioned orientation information is not the first direction, then the start time of the second stimulus signal is determined; and the time corresponding to the sum of the start time of the second stimulus signal and the time difference is taken as the start time of the first stimulus signal.

[0079] In practical applications, the aforementioned time difference refers to the time difference between the start times of the first stimulus signal and the second stimulus signal. This time difference is positively correlated with the angle information. Specifically, the start time of the first stimulus signal refers to the moment when the first stimulation device begins outputting its first stimulus signal, and the start time of the second stimulus signal refers to the moment when the second stimulation device begins outputting its second stimulus signal. For example, the larger the angle information of the target obstacle, the larger the time difference. Users can determine the angle information of the target obstacle by using the time difference between the start times of the first and second stimulus signals.

[0080] Specifically, in step S201, correlation calculations are performed on the angle information of the target obstacle to obtain the aforementioned time difference. As an example, relevant parameters are set, and the product of the angle information of the target obstacle and the relevant parameters is used as the time difference. For instance, if the time difference is T, the relevant parameter is M, and the angle information of the target obstacle is Y, then the time difference can be expressed as: T = M * Y.

[0081] It can be understood that when there is no offset in the orientation information, the angle information is 0°, and the time difference between the start time of the first stimulus signal and the second stimulus signal is 0, meaning the start time of the first stimulus signal and the start time of the second stimulus signal are the same. When there is an offset in the orientation information, the angle information is greater than 0°, and the time difference between the start time of the first stimulus signal and the second stimulus signal is not 0.

[0082] Based on the above example, the order of the start times of the first stimulus signal and the second stimulus signal represents the orientation information of the target obstacle. In this embodiment, the stimulation device corresponding to the orientation information of the target obstacle outputs the stimulus signal first; specifically, if the orientation information is a first orientation, then the start time of the first stimulus signal is earlier than the start time of the second stimulus signal; if the second orientation information is a second orientation, then the start time of the second stimulus signal is earlier than the first stimulus signal.

[0083] Optionally, the stimulation device corresponding to the orientation information of the target obstacle outputs a stimulation signal; specifically, if the orientation information is a first orientation, the start time of the first stimulation signal is later than the start time of the second stimulation signal; if the second orientation information is a second orientation, the start time of the second stimulation signal is later than the first stimulation time.

[0084] In one example, the time corresponding to the acquisition time of the target obstacle's location information is delayed by a predetermined time period can be used as the stimulation time of the first stimulus signal and the stimulation time of the second stimulus signal, with the earlier stimulation time being the stimulation time of the stimulus signal; the time corresponding to the sum of the stimulation time of the earlier stimulus signal and the above time difference can be used as the stimulation time of the later stimulus signal.

[0085] Specifically, in S203, if the aforementioned orientation information has no offset, then the start times of the first stimulus signal and the second stimulus signal are determined, and the start times of the first stimulus signal and the second stimulus signal are the same. Referring to the above example, if the orientation information has no offset, the time difference T is 0, and the start time T of the first stimulus signal is... a The start time T of the second stimulus signal b Similarly, if the location information of the target obstacle is acquired at time T1 and the predetermined time period is T2, then T... a =T b =T1+T2.

[0086] Specifically, in S205, if the aforementioned directional information corresponds to the first direction, then the start time of the first stimulus signal is determined; and the time corresponding to the sum of the start time of the first stimulus signal and the time difference is taken as the start time of the second stimulus signal. Referring to the above example, if the directional information has an offset, the time difference T is not zero, the directional information is the first direction, and the start time T of the first stimulus signal... a Earlier than the onset time T of the second stimulus signal b If the location information of the target obstacle is acquired at time T1, and the predetermined time period is T2, then T a =T1+T2, T b =T a +T.

[0087] Specifically, in S206, if the aforementioned azimuth information is not the first direction, then the start time of the second stimulus signal is determined; and the time corresponding to the sum of the start time of the second stimulus signal and the aforementioned time difference is taken as the start time of the first stimulus signal. Referring to the above example, if the azimuth information has an offset, the time difference T is not 0, the azimuth information is the second direction, and the start time T of the second stimulus signal... b Earlier than the onset time T of the first stimulus signal a If the location information of the target obstacle is acquired at time T1, and the predetermined time period is T2, then T b =T1+T2, T a =T b +T.

[0088] Optionally, the time difference between the start time of the first stimulus signal and the start time of the second stimulus signal can be preset. Steps S202 to S206 can be implemented independently to determine the start time of the first stimulus signal and the start time of the second stimulus signal. For example, based on the above-mentioned directional information, the order of the start times of the first stimulus signal and the second stimulus signal can be determined. The stimulus time of the stimulus signal with the earlier stimulus time can be determined, and the sum of the stimulus time of the earlier stimulus signal and the time difference can be used to determine the stimulus time of the later stimulus signal.

[0089] In the visually impaired assistive perception method provided in this application, correlation calculation is performed on the angle information of the target obstacle to obtain the aforementioned time difference; it is determined whether the aforementioned orientation information has an offset; if the aforementioned orientation information has no offset, the start time of the aforementioned first stimulus signal and the aforementioned second stimulus signal is determined, and the start time of the aforementioned first stimulus signal and the aforementioned second stimulus signal are the same; if the aforementioned orientation information has an offset, it is determined whether the aforementioned orientation information is the aforementioned first direction; if the aforementioned orientation information is the aforementioned first direction, the start time of the aforementioned first stimulus signal is determined; and the time corresponding to the sum of the start time of the aforementioned first stimulus signal and the aforementioned time difference is taken as the start time of the aforementioned second stimulus signal; if the aforementioned orientation information is not the aforementioned first direction, the start time of the aforementioned second stimulus signal is determined; and the time corresponding to the sum of the aforementioned second stimulus signal and the aforementioned time difference is taken as the start time of the aforementioned first stimulus signal. In the process of visually impaired assisted perception, the starting time of the first stimulus signal and the starting time of the second stimulus signal are determined based on the angle signal and the orientation information of the target obstacle. The user can determine the angle information of the target obstacle based on the time difference between the starting time of the first stimulus signal and the starting time of the second stimulus signal, and can determine the orientation information of the target obstacle based on the order of the starting time of the first stimulus signal and the second stimulus signal, thereby improving the accuracy of visually impaired assisted perception.

[0090] Example 3

[0091] Figure 3 This is a flowchart illustrating a visually impaired assistive perception method provided in Embodiment 3 of this application, as shown below. Figure 3 As shown, prior to S102 in the above embodiment, the method further includes:

[0092] S301. Calculate the product of the reciprocal of the above distance information and the predetermined intensity coefficient to obtain the calculation result;

[0093] S302. Based on the above calculation results, obtain the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal.

[0094] The predetermined intensity coefficient is a positive integer; therefore, the calculation result is negatively correlated with the distance information—the smaller the distance information, the larger the calculation result. Referring to the example above, the smaller the distance information of the target obstacle, the greater its impact on the user; the calculation result is positively correlated with the impact on the user.

[0095] Specifically, in S301, the product of the reciprocal of the aforementioned distance information and the predetermined intensity coefficient is calculated to obtain the calculation result; for example, the distance information of the target obstacle can be represented as X, the intensity coefficient is N, and the calculation result is N*1 / X.

[0096] Intensity amplitude refers to the maximum value of the stimulus signal intensity. For example, when the stimulus signal is an electric current signal, the intensity amplitude can be the maximum current value of the stimulus signal. For instance, the intensity amplitudes of the first and second stimulus signals are negatively correlated with the distance information of the target obstacle; that is, the smaller the distance information of the target obstacle, the larger the intensity amplitudes of the first and second stimulus signals.

[0097] In conjunction with the above embodiments, steps S202 to S206 can determine the start time of the first stimulus signal and the start time of the second stimulus signal, which characterize the aforementioned directional information. Therefore, in one example, the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal can be the same, equal to the calculated result above. For example, the intensity amplitude of the first stimulus signal is S... a The intensity amplitude of the first stimulus signal is S b The intensity amplitudes of the first stimulus signal and the second stimulus signal can be expressed as S. a =S b =N*1 / X.

[0098] In this embodiment, a calculation result negatively correlated with the distance information of the target obstacle is obtained, and the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal are obtained based on the calculation result. The user can perceive the distance information of the target obstacle based on the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal.

[0099] Optionally, the orientation information can be characterized by the relationship between the intensity amplitudes of the first stimulus signal and the intensity amplitudes of the second stimulus signal. In one possible implementation, S302 includes:

[0100] If the above orientation information is the first direction, then the above calculation result is used as the intensity amplitude of the second stimulus signal; and the sum of the above calculation result and the preset amplitude increment is calculated to obtain the intensity amplitude of the first stimulus signal.

[0101] If the above orientation information is the second direction, then the above calculation result is used as the intensity amplitude of the first stimulus signal; and the sum of the above calculation result and the preset amplitude increment is calculated to obtain the intensity amplitude of the second stimulus signal.

[0102] If the above orientation information has no offset, then the above calculation results are used as the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal.

[0103] Referring to the above example, the intensity amplitudes of the first and second stimulus signals are negatively correlated with the distance information of the target obstacle; that is, the smaller the location information of the target obstacle, the larger the intensity amplitudes of the first and second stimulus signals. In this embodiment, the relationship between the intensity amplitudes of the first and second stimulus signals is related to the orientation information of the target obstacle.

[0104] In this embodiment, when the aforementioned directional information is offset, the intensity amplitude of the stimulation signal output by the stimulation device corresponding to the directional information is greater than the intensity amplitude of the stimulation signal output by the other stimulation device; when the aforementioned directional information is not offset, the intensity amplitudes of the first stimulation signal and the second stimulation signal are the same. It can be understood that the user can determine the aforementioned directional information based on the magnitude relationship between the intensity amplitudes of the first stimulation signal and the second stimulation signal.

[0105] Specifically, if the aforementioned directional information corresponds to the first direction, the calculated result is used as the intensity amplitude of the second stimulus signal; and the sum of the calculated result and a preset amplitude increment is calculated to obtain the intensity amplitude of the first stimulus signal. For example, if the aforementioned directional information corresponds to the first direction, and the intensity amplitude of the first stimulus signal is S... a The intensity amplitude of the first stimulus signal is S b The preset amplitude increment is p, and the intensity amplitude of the second stimulus signal can be expressed as S. b =N*1 / X, the intensity amplitude of the first stimulus signal can be expressed as S a =S b +p.

[0106] Correspondingly, if the aforementioned directional information corresponds to the second direction, the calculated result is used as the intensity amplitude of the first stimulus signal; and the sum of the calculated result and a preset amplitude increment is calculated to obtain the intensity amplitude of the second stimulus signal. For example, if the aforementioned directional information corresponds to the second direction, and the intensity amplitude of the first stimulus signal is S... a The intensity amplitude of the first stimulus signal is S b The preset amplitude increment is p, and the intensity amplitude of the first stimulus signal can be expressed as S. a =N*1 / X, the intensity amplitude of the second stimulus signal can be expressed as S b =S a +p.

[0107] Correspondingly, if the aforementioned directional information has no offset, then the calculation result is used as the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal. For example, if the aforementioned directional information has no offset, the intensity amplitude of the first stimulus signal is S. a The intensity amplitude of the first stimulus signal is Sb The preset amplitude increment is p, and the intensity amplitudes of the first stimulus signal and the second stimulus signal can be expressed as S. a =S b =N*1 / X.

[0108] Optionally, if the aforementioned azimuth information has no offset, the sum of the calculated result and a preset amplitude increment is used as the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal. For example, if the aforementioned azimuth information has no offset, and the intensity amplitude of the first stimulus signal is S... a The intensity amplitude of the first stimulus signal is S b The preset amplitude increment is p, and the intensity amplitudes of the first stimulus signal and the second stimulus signal can be expressed as S. a =S b =N*1 / X+p.

[0109] Optionally, when the above-mentioned orientation information is offset, the intensity amplitude of the stimulation signal output by the stimulation device corresponding to the above-mentioned orientation information is less than the intensity amplitude of the stimulation signal output by the other stimulation device.

[0110] In this embodiment, the intensity amplitudes of the first stimulus signal and the second stimulus signal are determined based on the aforementioned orientation information and calculation results. The intensity amplitudes of the first and second stimulus signals are negatively correlated with the distance information of the target obstacle. The relationship between the magnitudes of the first and second stimulus signal intensity amplitudes characterizes the aforementioned orientation information. Based on the intensity amplitudes of the first and second stimulus signals, the user can determine the distance and orientation information of the target obstacle, thus improving the accuracy of visually impaired assisted perception.

[0111] In the visually impaired assisted perception method provided in this application, the product of the reciprocal of the aforementioned distance information and a predetermined intensity coefficient is calculated to obtain a calculation result; based on the aforementioned calculation result, the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal are obtained. In the process of visually impaired assisted perception, this application's embodiment calculates a result negatively correlated with the distance information of the target obstacle, and obtains the intensity amplitudes of the first stimulus signal and the second stimulus signal based on the calculation result. This facilitates the user's perception of the distance information of the target obstacle based on the intensity amplitudes of the first and second stimulus signals, improving the accuracy of visually impaired assisted perception.

[0112] Example 4

[0113] To facilitate understanding of the solution, the following will provide an exemplary description of Embodiment 4 of this application, with examples. The first direction is the right side directly in front of the user, and the second direction is the left side directly in front of the user. This embodiment illustrates obstacles in four different positions. The flow of the visually impaired assistive perception method of Embodiment 4 of this application includes:

[0114] Step 1: Obtain the location information of the target obstacle; wherein, the location information of the target obstacle includes the distance information of the target obstacle, the angle information of the target obstacle, and the orientation information of the target obstacle; the angle information of the target obstacle is the offset angle of the target obstacle's location relative to the front of the user; the orientation information of the target obstacle is the offset direction of the target obstacle's location relative to the front of the user.

[0115] Example, Figure 4 The following is a spatial diagram of four positional scenarios provided in Embodiment 4 of this application. The positional information of the four obstacles is shown in Table 1, where X1 = X2 = X3 > X4, Y4 = Y1 = Y3, and Y2 = 0.

[0116] Table 1

[0117] Case number Location information Distance information Angle information 1 The user's left side directly in front <![CDATA[X1]]> <![CDATA[Y1]]> 2 User in front <![CDATA[X2]]> <![CDATA[Y2]]> 3 The user's right side <![CDATA[X3]]> <![CDATA[Y3]]> 4 The user's right side <![CDATA[X4]]> <![CDATA[Y4]]>

[0118] Step 2: Determine the start time and intensity amplitude of the first stimulus signal corresponding to the location information of the obstacle, as well as the start time and intensity amplitude of the second stimulus signal.

[0119] Step 21: Calculate the correlation of the angle information of the target obstacle to obtain the time difference mentioned above.

[0120] Step 22: If the above orientation information is the right side directly in front of the user, then determine the start time of the first stimulus signal; and take the time corresponding to the sum of the start time of the first stimulus signal and the time difference as the start time of the second stimulus signal.

[0121] If the above orientation information is to the left of the user's front, then the start time of the second stimulus signal is determined; and the time corresponding to the sum of the start time of the second stimulus signal and the time difference is taken as the start time of the first stimulus signal.

[0122] If the above orientation information is without offset, then the start time of the first stimulus signal and the second stimulus signal is determined, and the start time of the first stimulus signal and the second stimulus signal is the same.

[0123] Step 23: Calculate the product of the reciprocal of the above distance information and the predetermined intensity coefficient to obtain the calculation result;

[0124] Step 24: If the above orientation information is the right side directly in front of the user, then the above calculation result is used as the intensity amplitude of the second stimulus signal; and the sum of the above calculation result and the preset amplitude increment is calculated to obtain the intensity amplitude of the first stimulus signal.

[0125] If the above orientation information is directly in front of the user on the left, then the above calculation result is used as the intensity amplitude of the first stimulus signal; and the sum of the above calculation result and the preset amplitude increment is calculated to obtain the intensity amplitude of the second stimulus signal.

[0126] If the above orientation information has no offset, then the above calculation result is used as the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal.

[0127] Step 3: Based on the location information of the target obstacle, control the first stimulation device to output a first stimulation signal and control the second stimulation device to output a second stimulation signal.

[0128] Example, Figure 5 This is a schematic diagram illustrating the output of the stimulation device under four different positional conditions provided in Embodiment 4 of this application. Figure 5 As shown, in case 1, the start time T of the first stimulus signal a =5, the start time T of the second stimulus signal b =2, time difference T=3, the intensity amplitude of the first stimulus signal is S a =4, the amplitude of the first stimulus signal is S b =6; In case 2, the start time T of the first stimulus signal a =2, the start time T of the second stimulus signal b =2, time difference T=0, the intensity amplitude of the first stimulus signal is S a =6, the intensity amplitude of the first stimulus signal is S b =6; In case 3, the start time T of the first stimulus signal a =2, the start time T of the second stimulus signal b =5, time difference T=3, the intensity amplitude of the first stimulus signal is S a =6, the intensity amplitude of the first stimulus signal is S b =4; In case 4, the start time T of the first stimulus signal a =2, the start time T of the second stimulus signal b =12, time difference T=10, the intensity amplitude of the first stimulus signal is S a =8, the intensity amplitude of the first stimulus signal is S b =6.

[0129] Figure 5In the above, for scenarios 1, 2, 3, and 4, based on the order of the start times of the first and second stimulus signals, the user can determine the location information of the target obstacle in each of the four scenarios as being to the left, directly in front of the user, to the right, and to the right of the user's direct front, respectively. Based on the time differences between the four scenarios, the user can determine that the angle information of the target obstacle is the same for scenarios 1 and 3, the angle information of the target obstacle in scenario 2 is 0, and the angle information of the target obstacle in scenario 4 is the largest. Based on the magnitude of the intensity amplitude of the first and second stimulus signals for each scenario, the user can determine the location information of the target obstacle in each of the four scenarios as being to the left, directly in front of the user, to the right, and to the right of the user's direct front, respectively. Based on the intensity amplitude of the stimulus signal with the smaller intensity amplitude in each of the four scenarios, the user can determine that the distance information of the target obstacle in scenarios 1, 2, and 3 is the same and greater than the distance information of the target obstacle in scenario 4.

[0130] The visually impaired assistive perception method provided in this application acquires the location information of a target obstacle; wherein, the location information of the target obstacle includes the distance information, the angle information, and the orientation information of the target obstacle; based on the location information of the target obstacle, a first stimulation device is controlled to output a first stimulation signal and a second stimulation device is controlled to output a second stimulation signal; wherein, the stimulation intensity of the first stimulation signal and the second stimulation signal are negatively correlated with the aforementioned distance information, the sequential relationship between the start time of the first stimulation signal and the start time of the second stimulation signal represents the aforementioned orientation information, and the time difference between the start time of the first stimulation signal and the start time of the second stimulation signal is positively correlated with the aforementioned angle information. In the process of assistive perception for the visually impaired, two stimulation devices representing opposite directions output a first stimulation signal and a second stimulation signal to the user, respectively. The user can determine the orientation information of the target obstacle by the sequential relationship between the start time of the first stimulation signal and the start time of the second stimulation signal, determine the angle information of the target obstacle by the time difference between the start time of the first stimulation signal and the start time of the second stimulation signal, and determine the distance information of the target obstacle by the stimulation intensity of the first stimulation signal and the second stimulation signal. This enables the visually impaired to accurately perceive the location of the obstacle and improves the accuracy of assistive perception for the visually impaired.

[0131] Example 5

[0132] Figure 6 This is a schematic diagram of the structure of a visually impaired assistive sensing device provided in Embodiment 5 of this application, as shown below. Figure 6As shown, the device includes: a first stimulation device 61 representing a first direction, a second stimulation device 62 representing a second direction, a position acquisition module 63, and a control module 64, wherein the first direction is opposite to the second direction.

[0133] Location acquisition module 63 is used to acquire the location information of the target obstacle;

[0134] The control module 64 is used to control the first stimulation device 61 to output a first stimulation signal and the second stimulation device 62 to output a second stimulation signal based on the position information of the target obstacle.

[0135] The location information of the target obstacle refers to the position of the target obstacle relative to the user; the location information of the target obstacle includes the distance information of the target obstacle, the angle information of the target obstacle, and the orientation information of the target obstacle; the angle information of the target obstacle is the offset angle of the target obstacle's position relative to the front of the user; the orientation information of the target obstacle is the offset direction of the target obstacle's position relative to the front of the user.

[0136] In this embodiment, the location information of the target obstacle can be one of the following: directly in front of the user, to the left of the user's direct front, or to the right of the user's direct front. In one example, an obstacle can be detected using a target detection algorithm. Optionally, a deep learning-based target detection method can acquire images of the areas directly in front of the user, to the left of the user's direct front, and to the right of the user's direct front, respectively. By recognizing the features of the acquired images, the existence of an obstacle and its location information can be determined, and the detected obstacle can be taken as the target obstacle.

[0137] The distance information of the target obstacle refers to the straight-line distance between the target obstacle and the user. For example, target information of the target obstacle can be obtained through methods such as monocular ranging, binocular ranging, and radar ranging.

[0138] In this embodiment, the stimulation intensity of the first and second stimulus signals is negatively correlated with the distance information. The sequential relationship between the start times of the first and second stimulus signals represents the orientation information. The time difference between the start times of the first and second stimulus signals is positively correlated with the angle information. The first and second stimulus signals uniquely represent the location information of the target obstacle.

[0139] It is understandable that the first and second directions are opposite; the first stimulation device represents the first direction, and the second stimulation device represents the second direction. Therefore, the locations of the first and second stimulation devices are in opposite directions. In practical applications, the first and second stimulation devices can be located on sensitive areas on either side of the user's body.

[0140] Specifically, after acquiring the location information of the target obstacle, the system controls a first stimulation device and a second stimulation device representing opposite directions to output a first stimulation signal and a second stimulation signal to the user, respectively. Based on the stimulation intensity of the first and second stimulation signals, the user can determine the distance information of the target obstacle; based on the sequential relationship between the start times of the first and second stimulation signals, the user can determine the orientation information of the target obstacle; and based on the time difference between the start times of the first and second stimulation signals, the user can determine the angle information of the target obstacle. In other words, based on the orientation information, angle information, and distance information of the target obstacle, the user can perceive the precise position of the target obstacle relative to the user.

[0141] Optionally, in one possible implementation, the device further includes:

[0142] The first calculation module is used to perform correlation calculations on the angle information of the aforementioned target obstacle to obtain the aforementioned time difference.

[0143] In practical applications, the aforementioned time difference refers to the time difference between the start times of the first stimulus signal and the second stimulus signal. This time difference is positively correlated with the angle information. Specifically, the start time of the first stimulus signal refers to the moment when the first stimulation device begins outputting its first stimulus signal, and the start time of the second stimulus signal refers to the moment when the second stimulation device begins outputting its second stimulus signal. For example, the larger the angle information of the target obstacle, the larger the time difference. Users can determine the angle information of the target obstacle by using the time difference between the start times of the first and second stimulus signals.

[0144] As an example, relevant parameters are set, and the product of the angle information of the target obstacle and the relevant parameters is used as the time difference.

[0145] In this embodiment, the first calculation module performs correlation calculation on the angle information of the target obstacle to obtain the time difference between the start time of the first stimulus signal and the start time of the second stimulus signal. Based on the time difference, the user can perceive the angle information of the target obstacle.

[0146] Optionally, in one possible implementation, the device further includes: a time determination module; the time determination module is used for:

[0147] If the above-mentioned orientation information is the first direction, then the start time of the first stimulus signal is determined; and the time corresponding to the sum of the start time of the first stimulus signal and the time difference is taken as the start time of the second stimulus signal.

[0148] If the above-mentioned orientation information is the second direction, then the start time of the second stimulus signal is determined; and the time corresponding to the sum of the start time of the second stimulus signal and the time difference is taken as the start time of the first stimulus signal.

[0149] If the above orientation information is without offset, then the start time of the first stimulus signal and the second stimulus signal is determined, and the start time of the first stimulus signal and the second stimulus signal is the same.

[0150] It can be understood that when there is no offset in the orientation information, the angle information is 0°, and the time difference between the start time of the first stimulus signal and the second stimulus signal is 0, meaning the start time of the first stimulus signal and the start time of the second stimulus signal are the same. When there is an offset in the orientation information, the angle information is greater than 0°, and the time difference between the start time of the first stimulus signal and the second stimulus signal is not 0.

[0151] Based on the above example, the order of the start times of the first stimulus signal and the second stimulus signal represents the orientation information of the target obstacle. In this embodiment, the stimulation device corresponding to the orientation information of the target obstacle outputs the stimulus signal first; specifically, if the orientation information is a first orientation, then the start time of the first stimulus signal is earlier than the start time of the second stimulus signal; if the second orientation information is a second orientation, then the start time of the second stimulus signal is earlier than the first stimulus signal.

[0152] Optionally, the stimulation device corresponding to the orientation information of the target obstacle outputs a stimulation signal; specifically, if the orientation information is a first orientation, the start time of the first stimulation signal is later than the start time of the second stimulation signal; if the second orientation information is a second orientation, the start time of the second stimulation signal is later than the first stimulation time.

[0153] In one example, the time corresponding to the acquisition time of the target obstacle's location information is delayed by a predetermined time period can be used as the stimulation time of the first stimulus signal and the stimulation time of the second stimulus signal, with the earlier stimulation time being the stimulation time of the stimulus signal; the time corresponding to the sum of the stimulation time of the earlier stimulus signal and the above time difference can be used as the stimulation time of the later stimulus signal.

[0154] Optionally, the time difference between the start time of the first stimulus signal and the start time of the second stimulus signal can be preset. For example, based on the above-mentioned directional information, the order of the start times of the first stimulus signal and the second stimulus signal can be determined. The stimulus time of the stimulus signal with the earlier stimulus time can be determined, and the sum of the stimulus time of the earlier stimulus signal and the time difference can be used to determine the stimulus time of the later stimulus signal.

[0155] In this embodiment, the time determination module determines the start time of the first stimulus signal and the start time of the second stimulus signal based on the time difference and the orientation information of the target obstacle. The user can determine the angle information of the target obstacle based on the time difference between the start time of the first stimulus signal and the start time of the second stimulus signal, and can determine the orientation information of the target obstacle based on the order of the start time of the first stimulus signal and the second stimulus signal, thereby improving the accuracy of visual impairment assisted perception.

[0156] Optionally, in one possible implementation, the device further includes:

[0157] The second calculation module is used to multiply the reciprocal of the above distance information by the predetermined intensity coefficient to obtain the calculation result;

[0158] The intensity determination module is used to obtain the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal based on the above calculation results.

[0159] The predetermined intensity coefficient is a positive integer; therefore, the calculation result is negatively correlated with the distance information—the smaller the distance information, the larger the calculation result. Referring to the example above, the smaller the distance information of the target obstacle, the greater its impact on the user; the calculation result is positively correlated with the impact on the user.

[0160] Intensity amplitude refers to the maximum value of the stimulus signal intensity. For example, when the stimulus signal is an electric current signal, the intensity amplitude can be the maximum current value of the stimulus signal. For instance, the intensity amplitudes of the first and second stimulus signals are negatively correlated with the distance information of the target obstacle; that is, the smaller the distance information of the target obstacle, the larger the intensity amplitudes of the first and second stimulus signals.

[0161] Based on the above example, the time determination module can determine the start time of the first stimulus signal and the start time of the second stimulus signal, which characterize the aforementioned directional information. Therefore, in one example, the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal can be the same, equal to the calculation result described above.

[0162] In this embodiment, the second calculation module calculates a result that is negatively correlated with the distance information of the target obstacle. The intensity determination module obtains the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal based on the calculation result. Based on the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal, the user can perceive the distance information of the target obstacle.

[0163] Optionally, in one possible implementation, the above-mentioned intensity determination module is specifically used for:

[0164] If the above orientation information is the first direction, then the above calculation result is used as the intensity amplitude of the second stimulus signal; and the sum of the above calculation result and the preset amplitude increment is calculated to obtain the intensity amplitude of the first stimulus signal.

[0165] If the above orientation information is the second direction, then the above calculation result is used as the intensity amplitude of the first stimulus signal; and the sum of the above calculation result and the preset amplitude increment is calculated to obtain the intensity amplitude of the second stimulus signal.

[0166] If the above orientation information has no offset, then the above calculation results are used as the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal.

[0167] Referring to the above example, the intensity amplitudes of the first and second stimulus signals are negatively correlated with the distance information of the target obstacle; that is, the smaller the location information of the target obstacle, the larger the intensity amplitudes of the first and second stimulus signals. In this embodiment, the relationship between the intensity amplitudes of the first and second stimulus signals is related to the orientation information of the target obstacle.

[0168] In this embodiment, when the aforementioned directional information is offset, the intensity amplitude of the stimulation signal output by the stimulation device corresponding to the directional information is greater than the intensity amplitude of the stimulation signal output by the other stimulation device; when the aforementioned directional information is not offset, the intensity amplitudes of the first stimulation signal and the second stimulation signal are the same. It can be understood that the user can determine the aforementioned directional information based on the magnitude relationship between the intensity amplitudes of the first stimulation signal and the second stimulation signal.

[0169] Optionally, if the above orientation information has no offset, the sum of the above calculation result and the preset amplitude increment is used as the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal.

[0170] In this embodiment, the intensity determination module determines the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal based on the aforementioned orientation information and calculation results. The intensity amplitudes of the first and second stimulus signals are negatively correlated with the distance information of the target obstacle. The relationship between the magnitudes of the first and second stimulus signal intensity amplitudes characterizes the aforementioned orientation information. Based on the intensity amplitudes of the first and second stimulus signals, the user can determine the distance and orientation information of the target obstacle, thus improving the accuracy of visually impaired assisted perception.

[0171] Optionally, in one possible implementation, the location acquisition module 63 includes:

[0172] The detection unit is used to detect obstacles within a predetermined range near the user;

[0173] The acquisition unit is used to acquire distance information of each obstacle if multiple obstacles are detected, and to take the obstacle with the smallest distance information as the target obstacle, and to acquire the orientation information and angle information of the target obstacle.

[0174] As an example, obstacles within a predetermined range in front of a user can be detected using computer vision algorithms. For instance, based on the salientity detection (SD) method, images of a predetermined range near the user are acquired, and obstacles within that range are detected by extracting salient features from the acquired images.

[0175] In practical applications, multiple obstacles may exist within a predetermined range near the user, and the positions of these obstacles relative to the user change as the user moves. When multiple obstacles exist within this range, to ensure the user's safety, the user needs to perceive the obstacle with the greatest impact and avoid collisions. It's understandable that the smaller the distance to an obstacle, the greater its impact on the user. Therefore, identifying the obstacle with the smallest distance as the target obstacle, obtaining its orientation and angle information, and perceiving it can effectively improve the accuracy of visually impaired assisted perception.

[0176] In this embodiment, the location acquisition module acquires distance information of multiple obstacles, selects the obstacle with the smallest distance information as the target obstacle, and then acquires the orientation information and angle information of the target obstacle. This allows the user to perceive the obstacle with the greatest impact, thus improving the accuracy of visually impaired assisted perception.

[0177] Optionally, in one possible implementation, the first stimulation signal and the second stimulation signal are digital signals; the device further includes:

[0178] The speed acquisition module is used to acquire the moving speed of the aforementioned target obstacle.

[0179] The frequency determination module is used to determine the stimulation frequencies of the first stimulation signal and the second stimulation signal based on the aforementioned movement speed.

[0180] The target obstacle's movement speed includes both its speed and direction of movement. As an example, an acquisition period is set. When the acquisition period arrives, the target obstacle's current position information is acquired. Based on the target obstacle's current position information and the position information obtained in the most recent acquisition period, the target obstacle's movement speed within the acquisition period is determined.

[0181] In practical applications, the direction of movement of a target obstacle can determine whether it is approaching or moving away from the user. The closer the obstacle gets to the user, the higher its speed, and the more likely a collision is to occur. Therefore, by sensing the speed of the obstacle in time, users can avoid collisions and ensure their safety.

[0182] The stimulation frequencies of the first and second stimulus signals are positively correlated with the moving speed of the target obstacle. For example, the first and second stimulus signals can be pulse signals with a 50% duty cycle, and the stimulation frequency refers to the number of pulses generated per unit time. The greater the moving speed of the target obstacle, the higher the stimulation frequency, and the more pulses generated per unit time. Therefore, the user can determine the moving speed of the target obstacle based on the number of pulses generated per unit time by the first and second stimulus signals.

[0183] In this embodiment, the speed acquisition module acquires the moving speed of the target obstacle; the frequency determination module determines the stimulation frequency of the first stimulus signal and the second stimulus signal based on the moving speed of the target obstacle. The user can perceive the moving speed of the target obstacle based on the stimulation frequency of the first stimulus signal and the second stimulus signal, thereby improving the safety of the user's travel.

[0184] In the visually impaired assistive sensing device provided in this application, the position acquisition module acquires the position information of the target obstacle; wherein, the position information of the target obstacle includes the distance information of the target obstacle, the angle information of the target obstacle, and the orientation information of the target obstacle; the control module controls the first stimulation device to output a first stimulation signal and controls the second stimulation device to output a second stimulation signal according to the position information of the target obstacle; wherein, the stimulation intensity of the first stimulation signal and the second stimulation signal are negatively correlated with the aforementioned distance information, the sequential relationship between the start time of the first stimulation signal and the start time of the second stimulation signal represents the aforementioned orientation information, and the time difference between the start time of the first stimulation signal and the start time of the second stimulation signal is positively correlated with the aforementioned angle information. In the process of assistive perception for the visually impaired, two stimulation devices representing opposite directions output a first stimulation signal and a second stimulation signal to the user, respectively. The user can determine the orientation information of the target obstacle by the sequential relationship between the start time of the first stimulation signal and the start time of the second stimulation signal, determine the angle information of the target obstacle by the time difference between the start time of the first stimulation signal and the start time of the second stimulation signal, and determine the distance information of the target obstacle by the stimulation intensity of the first stimulation signal and the second stimulation signal. This enables the visually impaired to accurately perceive the location of the obstacle and improves the accuracy of assistive perception for the visually impaired.

[0185] Example 6

[0186] Figure 7 This is a schematic diagram of the structure of the electronic device provided in Embodiment Six of this application, as shown below. Figure 7 As shown, the electronic device includes:

[0187] The processor 71 and the main control device also include a memory 72; it may also include a communication interface 73 and a bus 74. The processor 71, memory 72, and communication interface 73 can communicate with each other via the bus 74. The communication interface 73 can be used for information transmission. The processor 71 can call logical instructions stored in the memory 72 to execute the methods of the above embodiments.

[0188] Furthermore, the logic instructions in the aforementioned memory 72 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0189] The memory 72, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this application. The processor 71 executes functional applications and data processing by running the software programs, instructions, and modules stored in the memory 72, thereby implementing the methods in the above-described method embodiments.

[0190] The memory 72 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 72 may include high-speed random access memory and may also include non-volatile memory.

[0191] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method in any of the embodiments. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0192] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0193] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A visually impaired assistive perception method, characterized in that, An assistive sensing device for the visually impaired, the assistive sensing device for the visually impaired including a first stimulation device representing a first direction and a second stimulation device representing a second direction, wherein the first direction is opposite to the second direction, the method includes: Obtain the location information of the target obstacle; wherein, the location information of the target obstacle includes the distance information of the target obstacle, the angle information of the target obstacle, and the orientation information of the target obstacle; the angle information of the target obstacle is the offset angle of the target obstacle's location relative to the front of the user; the orientation information of the target obstacle is the offset direction of the target obstacle's location relative to the front of the user; Based on the location information of the target obstacle, the first stimulation device is controlled to output a first stimulation signal and the second stimulation device is controlled to output a second stimulation signal; the angle information of the target obstacle is correlated to obtain a time difference; if the orientation information is the first direction, the start time of the first stimulation signal is determined; and the time corresponding to the sum of the start time of the first stimulation signal and the time difference is taken as the start time of the second stimulation signal; if the orientation information is the second direction, the start time of the second stimulation signal is determined; and the time corresponding to the sum of the start time of the second stimulation signal and the time difference is taken as the start time of the first stimulation signal; if the orientation information has no offset, the start times of the first stimulation signal and the second stimulation signal are determined, and the start times of the first stimulation signal and the second stimulation signal are the same; wherein, the stimulation intensity of the first stimulation signal and the second stimulation signal is negatively correlated with the distance information, the sequential relationship between the start times of the first stimulation signal and the start times of the second stimulation signal characterizes the orientation information, and the time difference between the start times of the first stimulation signal and the start times of the second stimulation signal is positively correlated with the angle information.

2. The method according to claim 1, characterized in that, Before controlling the first stimulation device to output a first stimulation signal and the second stimulation device to output a second stimulation signal based on the location information of the target obstacle, the method further includes: Calculate the product of the reciprocal of the distance information and the predetermined intensity coefficient to obtain the calculation result; Based on the calculation results, the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal are obtained.

3. The method according to claim 2, characterized in that, The step of obtaining the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal based on the calculation results includes: If the orientation information is the first direction, then the calculation result is used as the intensity amplitude of the second stimulus signal; and the sum of the calculation result and the preset amplitude increment is calculated to obtain the intensity amplitude of the first stimulus signal. If the orientation information is the second direction, then the calculation result is used as the intensity amplitude of the first stimulus signal; and the sum of the calculation result and the preset amplitude increment is calculated to obtain the intensity amplitude of the second stimulus signal. If the orientation information is non-offset, the calculation result is used as the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal.

4. The method according to claim 1, characterized in that, The acquisition of the location information of the target obstacle includes: Detect obstacles within a predetermined range near the user; If multiple obstacles are detected, the distance information of each obstacle is obtained, and the obstacle with the smallest distance information is taken as the target obstacle. The orientation information and angle information of the target obstacle are then obtained.

5. The method according to any one of claims 1-4, characterized in that, The first stimulus signal and the second stimulus signal are digital signals; the method further includes: Obtain the moving speed of the target obstacle; Based on the movement speed, the stimulation frequencies of the first stimulation signal and the second stimulation signal are determined, and the stimulation frequencies are positively correlated with the movement speed.

6. A visually impaired assistive sensing device, characterized in that, The device includes: a first stimulation device representing a first direction, a second stimulation device representing a second direction, a position acquisition module, a control module, a first calculation module, and a time determination module, wherein the first direction is opposite to the second direction; The location acquisition module is used to acquire the location information of the target obstacle; wherein, the location information of the target obstacle includes the distance information of the target obstacle, the angle information of the target obstacle, and the orientation information of the target obstacle; the angle information of the target obstacle is the offset angle of the target obstacle's location relative to the front of the user; the orientation information of the target obstacle is the offset direction of the target obstacle's location relative to the front of the user. The control module is used to control the first stimulation device to output a first stimulation signal and control the second stimulation device to output a second stimulation signal based on the position information of the target obstacle. Wherein, the stimulation intensity of the first stimulation signal and the second stimulation signal is negatively correlated with the distance information, the sequential relationship between the start time of the first stimulation signal and the start time of the second stimulation signal represents the orientation information, and the time difference between the start time of the first stimulation signal and the start time of the second stimulation signal is positively correlated with the angle information. The first calculation module is used to perform correlation calculation on the angle information of the target obstacle to obtain the time difference; The time determination module is configured to: if the orientation information is the first direction, determine the start time of the first stimulus signal; and take the time corresponding to the sum of the start time of the first stimulus signal and the time difference as the start time of the second stimulus signal; if the orientation information is the second direction, determine the start time of the second stimulus signal; and take the time corresponding to the sum of the start time of the second stimulus signal and the time difference as the start time of the first stimulus signal; if the orientation information is without offset, determine the start times of the first stimulus signal and the second stimulus signal, and the start times of the first stimulus signal and the second stimulus signal are the same.

7. The apparatus according to claim 6, characterized in that, The device further includes: The second calculation module is used to calculate the product of the reciprocal of the distance information and the predetermined intensity coefficient to obtain the calculation result; An intensity determination module is used to obtain the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal based on the calculation results.

8. The apparatus according to claim 7, characterized in that, The intensity determination module is specifically used for: If the orientation information is the first direction, then the calculation result is used as the intensity amplitude of the second stimulus signal; and the sum of the calculation result and the preset amplitude increment is calculated to obtain the intensity amplitude of the first stimulus signal. If the orientation information is the second direction, then the calculation result is used as the intensity amplitude of the first stimulus signal; And, the sum of the calculation result and the preset amplitude increment is calculated to obtain the intensity amplitude of the second stimulus signal; If the orientation information is non-offset, the calculation result is used as the intensity amplitude of the first stimulus signal and the intensity amplitude of the second stimulus signal.

9. The apparatus according to claim 6, characterized in that, The location acquisition module includes: The detection unit is used to detect obstacles within a predetermined range near the user; The acquisition unit is used to acquire distance information of each obstacle if multiple obstacles are detected, and to take the obstacle with the smallest distance information as the target obstacle, and to acquire the orientation information and angle information of the target obstacle.

10. The apparatus according to any one of claims 6-9, characterized in that, The first stimulation signal and the second stimulation signal are digital signals; the device further includes: The speed acquisition module is used to acquire the moving speed of the target obstacle; A frequency determination module is used to determine the stimulation frequencies of the first stimulation signal and the second stimulation signal based on the movement speed, wherein the stimulation frequencies are positively correlated with the movement speed.

11. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-5.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-5.

Images