Auxiliary method, device and equipment for space perception teaching and storage medium

Abstract

The application provides a kind of space perception teaching auxiliary method, device, equipment and storage medium, wherein, the method comprises: determining target value for training, and recording the first starting point of first user action;When triggering end instruction, determine the critical point of first user action;The difference between first starting point and critical point is used as measurement value;According to the comparison result between target value and measurement value, determine the guidance information, feedback guidance information in the form that visual impaired person can perceive.The space perception teaching auxiliary method, device, equipment and storage medium provided by the embodiment of the application, the user makes first user action with the help of mobile terminal, realizes the perception of specified length or angle, and does not need teaching aid throughout;And, each time triggering end instruction, mobile terminal will compare the result of measurement value and target value to user in real time, and does not need the feedback of naked eye person.Provide the efficiency of teaching, also reduce economic cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human-computer interaction, in particular to a space perception teaching auxiliary method and device, equipment and storage medium. BACKGROUND

[0002] The visually impaired generally perceive the external environment by touching objects. This ability to perceive space can be improved through small-scale task teaching. Current teaching methods mainly provide a variety of different lengths of teaching aids in a classroom, and the visually impaired perceive the difference in length by touching the teaching aids. For example, a sighted person tells the visually impaired person the length of the teaching aid, or the visually impaired person estimates the length of the teaching aid while touching it, and the sighted person gives guidance. The current teaching method is inefficient because the visually impaired person is subject to the teaching aids and feedback from the sighted person. SUMMARY

[0003] To solve the existing technical problems, the embodiments of the present application provide a space perception teaching auxiliary method, device, equipment and storage medium.

[0004] In a first aspect, the embodiments of the present application provide a space perception teaching auxiliary method, comprising:

[0005] determining a target value for training, and recording a first starting point of a first user action.

[0006] determining a critical point of the first user action when a end instruction is triggered.

[0007] taking the difference between the first starting point and the critical point as a measurement value; wherein the target value and the measurement value are both distances or angles.

[0008] determining guidance information according to the comparison result between the target value and the measurement value, and feeding back the guidance information in a form that can be perceived by the visually impaired.

[0009] In a second aspect, the embodiments of the present application also provide a space perception teaching auxiliary device, comprising:

[0010] an initialization module for determining a target value for training, and recording a first starting point of a first user action.

[0011] a critical point determination module for determining a critical point of the first user action when a end instruction is triggered.

[0012] a measurement value determination module for taking the difference between the first starting point and the critical point as a measurement value; wherein the target value and the measurement value are both distances or angles.

[0013] The first feedback module is configured to determine guidance information according to a comparison result between the target value and the measured value, and feed back the guidance information in a form that can be perceived by the visually impaired.

[0014] In a third aspect, an apparatus is provided, which includes a processor and a memory. The memory stores a computer program. The processor executes the computer program stored in the memory. The computer program, when executed by the processor, implements the method for assisting spatial perception teaching according to the first aspect.

[0015] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program. The computer program, when executed by a processor, implements the method for assisting spatial perception teaching according to the first aspect.

[0016] In a fifth aspect, a computer program product is provided, which includes a computer program. The computer program, when executed, implements the method for assisting spatial perception teaching according to the first aspect or any possible design of the first aspect.

[0017] The method, device, apparatus and storage medium for assisting spatial perception teaching provided by the embodiments of the present application can pre-determine a target value for training, monitor the first user action by the mobile terminal in each training process, compare the size relationship between the measured value and the target value, and feed back the comparison result to the user by the mobile terminal in a form that can be perceived by the visually impaired. The user makes the first user action with the help of the mobile terminal, realizes the perception of the specified length or angle, and does not need the teaching aid throughout the process. Moreover, the mobile terminal feeds back the comparison result between the measured value and the target value to the user in real time each time the end instruction is triggered, and does not need the feedback of the sighted person. Therefore, the teaching aid and the feedback of the sighted person are not needed in the training process, the efficiency of teaching is improved, and the economic cost is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background art, the drawings needed to be used in the embodiments of the present application or the background art will be described below.

[0019] Figure 1 A flow chart of a method for assisting spatial perception teaching provided by an embodiment of the present application is shown;

[0020] Figure 2 A schematic diagram of five measurement methods provided by an embodiment of the present application is shown;

[0021] Figure 3 A schematic diagram of a teaching process of spatial perception teaching provided by an embodiment of the present application is shown;

[0022] Figure 4 Fig. 1 shows a structural schematic diagram of an auxiliary device for spatial perception teaching according to an embodiment of the present application;

[0023] Figure 5 Fig. 1 shows a structural schematic diagram of an auxiliary device for spatial perception teaching according to an embodiment of the present application; DETAILED DESCRIPTION

[0024] The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0025] The embodiments of the present application provide an auxiliary method for spatial perception teaching, which can be applied to a mobile terminal, for example, a mobile phone. The spatial perception teaching can be realized by using the mobile terminal. Figure 1 Fig. 2 shows a flow chart of an auxiliary method for spatial perception teaching according to an embodiment of the present application. As shown in the figure, the method comprises the following steps. Figure 1

[0026] Step 102: determining a target value for training, and recording a first starting point of a first user action.

[0027] The spatial perception teaching refers to cultivating the user's perception ability of space, for example, cultivating the user's perception ability of different lengths or angles. The embodiments of the present application can help the user to perceive different lengths or angles without the help of teaching aids, and play an auxiliary role in the spatial perception teaching of the user. The embodiments of the present application can consolidate the user's perception of a specified length or a specified angle by training the user to repeatedly practice a specific interactive action. For example, the perception of a specified length can be consolidated by repeatedly practicing the sliding of a single finger on the screen of the mobile terminal for a specified length. Alternatively, the user holds the mobile phone, which can be kept in a stable orientation, for example, the mobile phone is kept horizontally upward, the user's hand action remains unchanged, and the user's body or arm rotates a specified angle. The perception of a specified angle can be consolidated by repeatedly practicing this interactive action. The specified length and the specified angle are the target value of the training, for example, 30 cm or 45 degrees; and the first user action is the specific interactive action practiced by the user in each training process. After the target value is determined, the mobile terminal can feed back the target value to the user in the form of voice broadcast, for example, the mobile terminal says "the target value we are about to train is 30 cm" in the form of voice broadcast.

[0028] ​The user uses the mobile terminal to perform an interactive action, and a starting point, i.e., a first starting point, can be detected. The first starting point of the first user action, i.e., the position at which the first user action starts, is recorded. For example, the first starting point can be the position of a single finger when the finger starts to slide on the screen of the mobile terminal, which can be represented by the pixel coordinates of the finger on the touch screen. Alternatively, the first starting point can be the angle of the mobile terminal when the user starts to turn the body while holding the mobile terminal horizontally and keeping the hand motion unchanged.

[0029] Step 104: When the end instruction is triggered, the critical point of the first user action is determined.

[0030] The end instruction is triggered when it is not necessary to continue to monitor the first user action. For example, when the mobile terminal detects that the user has stopped the first user action or needs to remind the user to stop the first user action, it is considered that it is not necessary to continue to monitor the first user action, i.e., the end instruction is triggered. The end instruction can be an instruction to immediately end or to soon end the monitoring of the first user action. When the end instruction is triggered, the critical point of the first user action is determined, i.e., the position at which the user ends the first user action or the position of the first user action when the user is reminded to stop the first user action.

[0031] For example, after a single finger slides on the screen of the mobile terminal for a certain distance, the user stops the sliding, the mobile terminal detects that the user has stopped the first user action, the end instruction is triggered, the mobile terminal stops monitoring the first user action, and the position at which the user stops the sliding is the critical point of the first user action. Alternatively, the mobile terminal detects that the distance moved or the angle turned by the user approaches a target value, the user can be reminded to stop the first user action, the end instruction is triggered, i.e., the mobile terminal soon ends the monitoring of the first user action, and the position at which the user is reminded to stop the first user action is the critical point of the first user action.

[0032] Step 106: The difference between the first starting point and the critical point is taken as a measurement value; and the target value and the measurement value are distances or angles.

[0033] The difference between the first starting point and the critical point is calculated, and the difference between the two is the measured value. For example, the first user action trained is that a single finger slides a specified length on the screen of the mobile terminal, and at this time, the target value is the specified length, and the measured value is the distance that the single finger of the user actually slides on the screen of the mobile terminal during the training, that is, the difference between the first starting point and the critical point; at this time, both the target value and the measured value are distances. Or the interactive action trained is that the user holds the mobile phone, the mobile phone is kept horizontally upward, the action of the hand is kept unchanged, and the body of the user rotates a specified angle, and at this time, the target value is the specified angle, and the measured value is the angle that the body of the user actually rotates during the training, that is, the difference between the angle of the first starting point and the critical point; at this time, both the target value and the measured value are angles.

[0034] Step 108: determining the guidance information according to the comparison result between the target value and the measured value, and feeding back the guidance information in a form that can be perceived by the visually impaired.

[0035] In order to help the user clearly know the training situation, the measured value can be compared with the target value, that is, the distance actually reached by the user during the practice is compared with the set distance, or the angle actually reached during the practice is compared with the set angle, and the guidance information is generated based on the comparison result. The guidance information is fed back in a form that can be perceived by the visually impaired, such as voice broadcast or vibration, so that the visually impaired can perceive the guidance information when using the mobile terminal for training, thereby clearly perceiving the difference between the distance actually reached during the practice and the set distance, or the difference between the angle actually reached during the practice and the set angle.

[0036] Regarding the feedback of the guidance information, two cases in which the first user action does not need to be continuously monitored are exemplified.

[0037] The first case is that when the mobile terminal detects that the user has stopped the first user action, the size relationship between the target value and the measured value can be determined based on the comparison result between the target value and the measured value, and the size relationship between the target value and the measured value is taken as the guidance information that needs to be fed back. For example, it is assumed that the target value is 30 cm, and in the case where the user has ended the first user action, the measured value is calculated to be 35 cm, and the guidance information can be "please shorten the distance a little", to indicate that the measured value is greater than the target value, wherein the guidance information can be told to the user in the form of voice broadcast. The guidance information can also include the measured value, that is, the size relationship between the target value and the measured value and the measured value can be taken as the guidance information; for example, it is assumed that the target value is 30 cm, and in the case where the user has ended the first user action, the measured value is calculated to be 35 cm, and the guidance information can be "the distance you currently operate is 35 cm, please shorten the distance a little".

[0038] The second case is that when the mobile terminal detects that the user needs to be reminded to stop the first user action, for example, the mobile terminal detects that the distance or the angle of rotation that the user has moved approaches a target value, the user can be reminded to stop the first user action, and the user can be reminded that the distance or the angle of rotation that the user has moved approaches the target value in the form of vibration. The vibration reminder signal can represent the comparison result between the target value and the measured value, that is, the target value is substantially equal to the measured value, and the vibration signal can be used as a guide information. Alternatively, the guide information can be "the distance of your current operation is 29 cm, which approaches the target value", and the guide information is told to the user in the form of voice broadcast. For the second case, the vibration and voice broadcast can also be combined for feedback.

[0039] The method provided by the embodiment of the application for assisting spatial perception teaching comprises the following steps: determining a target value in advance; monitoring a first user action in each training process; comparing the size relationship between the target value and a measured value; and feeding back the comparison result to a user in a form that can be perceived by a visually impaired person. The user makes the first user action with the aid of the mobile terminal to realize perception of a specified length or angle, and the whole process does not need a teaching aid. Moreover, the mobile terminal feeds back the comparison result between the measured value and the target value to the user in real time each time an end instruction is triggered, and does not need feedback from a sighted person. Therefore, the teaching efficiency is improved and the economic cost is reduced in the training process without the teaching aid and the feedback from the sighted person.

[0040] In the embodiment of the application, the measurement mode that can be detected by the mobile terminal comprises a distance measurement mode or an angle measurement mode, and the distance measurement mode or the angle measurement mode has multiple types. The method further comprises step A1.

[0041] Step A1: receiving a selection instruction of a user to determine a current measurement mode of a corresponding type. The first starting point and the critical point are determined based on the current measurement mode.

[0042] The visually impaired person can generally perceive the external environment by touching the surface of an object. According to the touch habits of the visually impaired person, two types of interactive actions are summarized, that is, a distance-based interactive action and an angle-based interactive action. The distance-based interactive action refers to an interactive action that can be detected by the mobile terminal to change the distance, for example, a finger slides from one point to another point on the screen of the mobile terminal. Alternatively, in the case where the position of the mobile terminal is fixed, a palm is placed in front of the camera and moves away from the camera from the current position. The angle-based interactive action refers to an interactive action that can be detected by the mobile terminal to change the angle, for example, the user holds the mobile terminal, and the body of the user rotates to rotate the mobile terminal by a certain angle from the current position.

[0043] For the two types of interactive actions, the mobile terminal provides two types of corresponding measurement methods, a distance measurement method for detecting distance-based interactive actions, and an angle measurement method for detecting angle-based interactive actions.

[0044] Distance-based interactive actions can have various types, such as a finger sliding on a screen, or a palm moving away from the position of a camera, etc. The corresponding measurement methods for different types of distance-based interactive actions can also be different. According to the different types of interactive actions, the distance measurement methods can also have various types, such as a single-finger measurement method, a double-finger measurement method, a single-hand measurement method, and a double-hand measurement method, etc. Specifically, as shown in Figure 2 a single-finger measurement method is used to detect the distance of a single finger sliding on the screen of the mobile terminal from one point to another point; a double-finger measurement method is used to detect the distance change between two fingers when the two fingers slide on the screen of the mobile terminal; a single-hand measurement method is used to detect the distance change of a hand from one position to another position when the hand holds the mobile terminal; and a double-hand measurement method is used to detect the distance change of a hand from one position to another position in front of the camera of the mobile terminal when the other hand holds the mobile terminal.

[0045] Angle-based interactive actions can also have various types, such as an arm turning from a current position to another position, or a body turning from a current position to another position. Accordingly, the angle measurement methods can also have various types, for example, when the angle-based interactive action is an arm turning from a current position to another position, the corresponding angle measurement method can be an arm measurement method; when the angle-based interactive action is a body turning from a current position to another position, the corresponding angle measurement method can be a body measurement method. Taking the body measurement method as an example, specifically, as shown in Figure 2 a user holds a mobile phone, the mobile phone is parallel to the ground and kept horizontally upward, and the hand action remains unchanged, and the angle change of the body of the user from a current position to another position is monitored.

[0046] For different types of measurement methods, the mobile terminal may implement monitoring in different ways, i.e., the determination of the first starting point and the critical point may be different. For example, for the single-hand measurement method, the mobile terminal may implement monitoring in the following way: the mobile terminal obtains the displacement of the rear camera coordinate system relative to the world coordinate system by using the motion tracking function provided by the EasyAR framework through the rear camera and the IMU (inertial sensor), and then calculates the distance between the positions of the palm at different times. The first starting point can be the position of the origin of the rear camera coordinate system relative to the world coordinate system when monitoring starts, and the critical point can be the position of the origin of the rear camera coordinate system relative to the world coordinate system when the end instruction is triggered. For the double-hand measurement method, the mobile terminal may implement monitoring in the following way: the actual width of the palm is pre-recorded, the mobile terminal identifies the palm in the field of view of the front camera by using a hand recognition framework such as 3D human skeleton detection (Mediapipe), and estimates the distance of the palm relative to the front camera according to the relative size between the displayed palm width and the pre-recorded actual width. The first starting point can be the position of the palm in the front camera when monitoring starts, i.e., the distance between the palm and the front camera, and the critical point can be the position of the palm in the front camera when the end instruction is triggered.

[0047] In order to more accurately and quickly monitor the interactive action, before starting the training, the mobile terminal provides options for the measurement method, and the user can make a selection. Based on the selection made by the user, the current measurement method is determined. For example, before starting the training, the mobile terminal provides five options: single-finger measurement method, double-finger measurement method, single-hand measurement method, double-hand measurement method, and body measurement method. The user can select one of them. Assuming that the user selects the single-hand measurement method, the mobile terminal calls the interface corresponding to the single-hand measurement method to monitor the sliding of the single finger of the user on the screen of the mobile terminal. After determining the target value and the current measurement method, the mobile terminal can also feed back the target value and the current measurement method to the user in the form of voice broadcast, as shown in FIG. 8. After determining the target value and the current measurement method, the mobile terminal says, in the form of voice broadcast, “30 cm will be practiced, try the double-hand measurement method”. Figure 3

[0048] In the embodiment of the present application, step 106 comprises steps B1 to B3.

[0049] Step B1: in the case where the current measurement method is the angle measurement method, the rotation angle from the current frame to the adjacent next frame in the process of performing the first user action is recorded frame by frame.

[0050] Step B2: the rotation angles between the adjacent two frames are accumulated to determine the measured angle.

[0051] ​Step B3: when the end instruction is triggered, the measurement angle is taken as the measurement value.

[0052] In the case that the current measurement mode is the angle measurement mode, the motion sensor of the mobile terminal, such as the gravity acceleration sensor and the magnetic field sensor, can calculate the corresponding angle of the mobile terminal at different time. Each time corresponds to a frame, and a frame corresponds to an angle. In the process of monitoring the first user action, the mobile terminal records the rotation angle between the current frame (i.e. the frame corresponding to the current time) and the adjacent next frame (i.e. the frame corresponding to the next adjacent time) when rotating from the current frame to the adjacent next frame. The time interval between the adjacent two frames is very short. In the complete monitoring process of the first user action, there are multiple rotation angles between the adjacent two frames. The sum of the rotation angles between all adjacent two frames is calculated, and the sum obtained is the measurement angle. It can be understood that the measurement angle changes with the execution of the first user action. When the end instruction is triggered, the measurement angle at this time is taken as the measurement value.

[0053] In the traditional space perception teaching, the user trains the perception of the angle through the teaching aid. Due to the limitation of the teaching aid itself, only the angle within 360 degrees can be trained. In the embodiment of the present application, the angle (i.e. the measurement value) rotated by the first user action is the sum of the rotation angles between all adjacent two frames, and the sum of the rotation angles between all adjacent two frames can be greater than 360 degrees. Therefore, the angle that can be trained in the embodiment of the present application can be within 360 degrees, or can be greater than 360 degrees, and the applicability is wider.

[0054] In the embodiment of the present application, step B1 comprises step B11 to step B13.

[0055] Step B11: in the process of executing the first user action, the rotation direction from the current frame to the adjacent next frame is determined.

[0056] Step B12: the angle rotated from the current frame to the adjacent next frame along the rotation direction is taken as the rotation angle; wherein, if the rotation direction is consistent with the preset direction, the rotation angle is positive, and if the rotation direction is opposite to the preset direction, the rotation angle is negative.

[0057] Step B13: the rotation angle from the current frame to the adjacent next frame is recorded frame by frame.

[0058] In the case that the current measurement mode is the angle measurement mode, there are two directions in which the mobile terminal can rotate from the current frame to the adjacent next frame in the process of monitoring the first user action, and the two directions of rotation specifically refer to rotating from the current frame to the adjacent next frame in a clockwise direction or rotating from the current frame to the adjacent next frame in an anticlockwise direction. The embodiment of the present application determines the direction in which the angle of rotation is smaller as the direction of rotation from the current frame to the adjacent next frame. For example, if the angle of rotation from the current frame to the adjacent next frame in the clockwise direction is smaller than the angle of rotation from the current frame to the adjacent next frame in the anticlockwise direction, the clockwise direction is determined as the direction of rotation from the current frame to the adjacent next frame; otherwise, the anticlockwise direction is determined as the direction of rotation.

[0059] Before the training starts, a direction is preset, and the direction is the preset direction. The angle of rotation in the preset direction is positive, and the angle of rotation in the direction opposite to the preset direction is negative. The preset direction can be the clockwise direction or the anticlockwise direction. The angle of rotation from the current frame to the adjacent next frame in the direction of rotation is taken as the angle of rotation, that is, in addition to determining the size of the angle of rotation, the direction of rotation of the angle of rotation also needs to be determined. Specifically, according to whether the direction of rotation is consistent with the preset direction, the angle of rotation between each pair of adjacent frames is determined as positive or negative. For example, if the preset direction is the clockwise direction, if the direction of rotation between the adjacent frames is also the clockwise direction, the angle of rotation between the adjacent frames is positive; if the direction of rotation between the adjacent frames is the anticlockwise direction, the angle of rotation between the adjacent frames is negative.

[0060] By determining the preset direction, the angle of rotation is given a corresponding positive value or negative value according to whether the direction of rotation is the same as the preset direction, so that the scenarios to which the embodiment of the present application is applicable are more extensive. For example, when a user first contacts spatial perception teaching, the user can not easily make an accurate first interaction action. For example, in a training process, the user can rotate the body in one direction at the beginning, and can rotate the body in the opposite direction at the next time. That is, in a training process, the user can rotate the body in different directions at different times. In this case, the embodiment of the present application gives the angle of rotation a corresponding positive value or negative value according to whether the direction of rotation is the same as the preset direction, and the sum of the angles of rotation between all pairs of adjacent frames calculated is still the angle of rotation of the user's body from the first starting point to the critical point, so the embodiment of the present application is also applicable to the scenario in which the user randomly rotates the body in different directions in a training process.

[0061] In the embodiment of the present application, optionally, the method further includes steps C1 to C3.

[0062] Step C1: record the second starting point of the second user action.

[0063] Step C2: determining an end point of the second user action when detecting that the user ends the second user action.

[0064] Step C3: feeding back the difference between the second start point and the end point of the second user action in a form that can be perceived by the visually impaired.

[0065] After the user repeatedly practices the specific interactive action and the perception of the specified length or the specified angle is consolidated, the embodiment of the present application can monitor the interactive action of the user without setting the target value in advance and inform the user of the length or the angle corresponding to the detected interactive action of the user. The method provided by the embodiment of the present application is not only suitable for spatial perception teaching but also can realize the function of measurement in daily life, and is more practical. For example, the user wants to measure the length of a table, can hold the mobile phone on one side of the table and keep still, and the other hand faces the camera of the mobile phone and moves to the other side of the table, and stops moving when reaching the other side of the table. After the distance of the movement of the other hand is obtained, the length of the table to be measured can be known.

[0066] Specifically, the user uses the mobile terminal to perform the interactive action, and the mobile terminal starts to monitor the interactive action of the user. Here, the interactive action of the user is the second user action, and the mobile terminal can detect the start point of the interactive action, that is, the second start point. When detecting that the user ends the second user action, the mobile terminal can detect the end point of the second user action. The difference between the second start point and the end point of the second user action is calculated, and the difference is fed back to the user in a form that can be perceived by the visually impaired.

[0067] The above embodiment of the present application mentions that for different types of measurement methods, the monitoring manner implemented by the mobile terminal can be different. Before monitoring the second user action, the mobile terminal can provide the option of the measurement method, or can not provide the option of the measurement method. If the mobile terminal provides the option of the measurement method, based on the selection made by the user, the mobile terminal determines the monitoring manner corresponding to the measurement method selected by the user. If the mobile terminal does not provide the option of the measurement method, at this time, the mobile terminal can determine which measurement method needs to be adopted according to the priority determination condition in turn. For example, if there are two fingers on the screen, it is determined as the double-finger measurement method; if there is only a single finger on the screen and obvious movement occurs, it is determined as the single-finger measurement method; if the front camera detects a palm, it is determined as the double-hand measurement method; if the mobile phone is parallel to the ground, it is determined as the body measurement method; if the mobile phone is not parallel to the ground, and there is no single finger or double finger detected on the screen, and the front camera does not detect a palm, it can be determined as the single-hand measurement method. The priority determination condition can be adaptively modified according to actual needs.

[0068] Optionally, step 104 can include steps D1 to D2 in the embodiment of the present application.

[0069] Step D1: calculate the range of each trajectory point of the first user action in a preset time interval.

[0070] Step D2: in the case that the range is less than or equal to a first preset threshold, determine to trigger the end instruction, and select one trajectory point in the preset time interval as the critical point of the first user action.

[0071] In the monitoring process of the mobile terminal to the first user action, the movement trajectory of the first user action is recorded, which includes a plurality of trajectory points. In a certain time interval, if the position change of the plurality of trajectory points in the time interval is small, it can be considered that the user has stopped the first user action.

[0072] Specifically, a small time interval, for example, 1 second, is set in advance, which is the preset time interval. The positions of each trajectory point of the first user action in the preset time interval are compared. For example, for the distance measurement mode, the coordinates of each trajectory point are compared, and for the angle measurement mode, the Euler angles of each trajectory point are compared. Moreover, a threshold based on the position change is set in advance, for example, 0.1 cm, which is the first preset threshold. After determining the trajectory points of the first user action, the range of the positions of each trajectory point in the preset time interval is calculated, that is, the difference between the maximum value and the minimum value in each trajectory point. In the case that the range is less than or equal to the first preset threshold, it can be considered that the user has stopped the first user action, thereby triggering the end instruction.

[0073] Moreover, when selecting one trajectory point in the preset time interval, a plurality of selection methods can be used. For example, any one trajectory point in the preset time interval can be selected as the critical point of the first user action; or the average value of each trajectory point in the preset time interval can be calculated, and the trajectory point closest to the average value is selected as the critical point of the first user action. The embodiment of the present application does not limit this.

[0074] Optionally, step 104 can also include steps E1 to E2 in the embodiment of the present application.

[0075] Step E1: take the difference between the current trajectory point of the first user action and the first starting point as the current difference.

[0076] Step E2: in the case that the difference between the current difference and the target value does not exceed a second preset threshold, determine to trigger the end instruction, and determine the current trajectory point as the critical point of the first user action.

[0077] In the monitoring of the first user action by the mobile terminal, the trajectory point corresponding to the current time is taken as the current trajectory point. In the current time, the change of the position of the first user action from the position of the first starting point to the position of the current time can be obtained by calculating the difference between the current trajectory point of the first user action and the first starting point, and the difference is the current difference. If the mobile terminal detects that the current difference (i.e., the distance or angle of rotation that the user has moved) is close to the target value, it means that the user can be prompted to stop the first user action, so as to trigger the end instruction; by pre-setting a threshold value, for example, 2 cm, it can be used to judge whether the distance or angle of rotation that the user has moved is close to the target value, and the threshold value is the second preset threshold value. In the case where the difference between the current difference and the target value does not exceed the second preset threshold value, it is determined that the end instruction is triggered, and the current trajectory point is determined as the critical point of the first user action.

[0078] Figure 3 A teaching process display diagram of the spatial perception teaching provided by the embodiment of the application is shown. In the multiple links of the first user action, for example, the target value and measurement method link, the independent operation link, and the correction link, information feedback is provided. Specifically as follows:

[0079] The target value and measurement method link: after determining the target value and the current measurement method as the double-hand measurement method, the mobile terminal speaks out "30 cm will be practiced, try to use the double-hand measurement method" in the form of voice broadcast.

[0080] The independent operation link: after determining the target value and the current measurement method, the user can start the first user action with the help of the mobile terminal, and when the mobile terminal judges that the user has stopped the first user action, the mobile terminal speaks out "the distance of your current operation is 35 cm, please shorten the distance a little" in the form of voice broadcast.

[0081] The correction link: when the user shortens the distance to approach the target value according to the feedback information in the independent operation link, the mobile terminal gives a vibration prompt to remind the user that the target value has been approached, and speaks out "do you remember this length? Try again" in the form of voice broadcast. The user can start the first user action again according to the prompt, that is, enter the independent operation link again.

[0082] The feedback information provided by the embodiment of the application can accompany each link of the start, process, and end of the first user action. Similarly, each link of the second user action can also provide similar feedback information, which is not described here. In the process of perceiving space, the user can get feedback at any time, so that the efficiency of spatial perception teaching is higher.

[0083] The auxiliary method for space perception teaching provided by the embodiment of the application is described in detail above, and the auxiliary method can also be implemented by a corresponding device. The auxiliary device for space perception teaching provided by the embodiment of the application is described in detail below.

[0084] Figure 4 A structure schematic diagram of an auxiliary device for space perception teaching provided by the embodiment of the application is shown. As shown in the figure, Figure 4 The auxiliary device for space perception teaching comprises:

[0085] The initialization module 41 is configured to determine a target value for training and record a first starting point of the first user action.

[0086] The critical point determination module 42 is configured to determine a critical point of the first user action when the end instruction is triggered.

[0087] The measurement value determination module 43 is configured to take a difference between the first starting point and the critical point as a measurement value; wherein the target value and the measurement value are distances or angles.

[0088] The first feedback module 44 is configured to determine guidance information according to a comparison result between the target value and the measurement value, and feed back the guidance information in a form that can be perceived by the visually impaired.

[0089] In the embodiment of the application, optionally, the measurement manner that can be detected by the mobile terminal comprises a distance measurement manner and an angle measurement manner, and the type of the distance measurement manner or the angle measurement manner is multiple; the device further comprises:

[0090] The measurement manner determination module is configured to receive a selection instruction of the user and determine a current measurement manner of a corresponding type; wherein the first starting point and the critical point are determined based on the current measurement manner.

[0091] In the embodiment of the application, optionally, the measurement value determination module 43 comprises:

[0092] The frame recording sub-module is configured to, in a case where the current measurement manner is the angle measurement manner, record a rotation angle from a current frame to an adjacent next frame in a process of performing the first user action frame by frame.

[0093] The measurement angle determination sub-module is configured to accumulate the rotation angle between the adjacent two frames to determine a measurement angle.

[0094] The measurement value determination sub-module is configured to, when the end instruction is triggered, take the measurement angle as the measurement value.

[0095] In the embodiment of the application, optionally, the frame recording sub-module comprises:

[0096] The rotation direction determining unit is configured to determine a rotation direction from a current frame to a next adjacent frame during execution of the first user action.

[0097] The rotation angle determining unit determines an angle from the current frame to the next adjacent frame along the rotation direction as a rotation angle, wherein the rotation angle is positive if the rotation direction is consistent with the preset direction, and the rotation angle is negative if the rotation direction is opposite to the preset direction.

[0098] The recording unit is configured to record the rotation angle from the current frame to the next adjacent frame frame by frame.

[0099] In the embodiments of the present application, the device optionally further comprises:

[0100] The second starting point recording module is configured to record a second starting point of the second user action.

[0101] The end point determining module is configured to determine an end point of the second user action when detecting that the user ends the second user action.

[0102] The second feedback module is configured to feed back a difference value between the second starting point and the end point of the second user action in a form that can be perceived by the visually impaired.

[0103] In the embodiments of the present application, the critical point determining module 42 optionally comprises:

[0104] The range determining submodule is configured to calculate a range of each trajectory point of the first user action within a preset time interval.

[0105] The first critical point determining submodule is configured to determine a trigger end instruction and select one trajectory point within the preset time interval as a critical point of the first user action if the range is less than or equal to a first preset threshold.

[0106] In the embodiments of the present application, the critical point determining module 42 optionally comprises:

[0107] The current difference determining submodule is configured to determine a difference value between the current trajectory point of the first user action and the first starting point as a current difference value.

[0108] The second critical point determining submodule is configured to determine a trigger end instruction and determine the current trajectory point as a critical point of the first user action if a difference value between the current difference value and a target value does not exceed a second preset threshold.

[0109] It should be noted that the space perception teaching auxiliary device provided by the above-mentioned embodiments is only taken as an example for dividing the above-mentioned functional modules when realizing the corresponding functions, and in actual application, the above-mentioned functions can be completed by different functional modules according to the needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above-mentioned functions. In addition, the space perception teaching auxiliary device and the space perception teaching auxiliary method provided by the above-mentioned embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0110] According to an aspect of the present application, the embodiments of the present application further provide a computer program product, which comprises a computer program containing program codes for executing the method shown in the flow chart. In such embodiments, the computer program can be downloaded and installed from the network through the communication part. When the computer program is executed by the processor, the space perception teaching auxiliary method provided by the embodiments of the present application is executed.

[0111] In addition, the embodiments of the present application further provide a space perception teaching auxiliary device, which comprises a processor and a memory, the memory stores a computer program, the processor can execute the computer program stored in the memory, and when the computer program is executed by the processor, the space perception teaching auxiliary method provided by any of the above-mentioned embodiments can be realized.

[0112] For example, Figure 5 A space perception teaching auxiliary device provided by the embodiments of the present application is shown, which comprises a bus 1110, a processor 1120, a transceiver 1130, a bus interface 1140, a memory 1150 and a user interface 1160.

[0113] In the embodiments of the present application, the device further comprises a computer program stored on the memory 1150 and executable on the processor 1120, and when the computer program is executed by the processor 1120, the processes of the above-mentioned space perception teaching auxiliary method embodiments are realized.

[0114] The transceiver 1130 is used for receiving and sending data under the control of the processor 1120.

[0115] In the embodiments of the present application, the bus architecture (represented by the bus 1110) can include any number of interconnected buses and bridges, and the bus 1110 connects various circuits including one or more processors represented by the processor 1120 and the memory represented by the memory 1150 together.

[0116] Bus 1110 represents one or more of any of several types of bus structures, including an address bus, a data bus, a control bus, a memory bus, and a storage bus, each of which can be implemented using various technologies and standards. Bus 1110 can include one or more buses implementing various bus standards, such as an Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Extended ISA (EISA) bus, Advanced Graphics Port (AGP), and / or a Peripheral Component Interconnect (PCI) bus.

[0117] Processor 1120 can be an integrated circuit chip located on a bus, or a plurality of chips configured to be communicatively coupled to a bus. Processor 1120 can be a general purpose processor, a central processing unit (CPU), a network processing unit (NPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a complex programmable logic device (CPLD), a programmable logic array (PLA), a microcontroller unit (MCU), or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed by the processor. For example, the processor can be a single-core processor or a multi-core processor, and the processor can be integrated into a single chip or located in multiple different chips.

[0118] The processor 1120 can be a microprocessor or any conventional processor. The disclosed method steps in conjunction with the embodiments of the present application can be directly executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can be located in a readable storage medium known in the art, such as a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), a register, etc. The readable storage medium is located in a memory, and the processor reads the information in the memory to complete the steps of the above method in conjunction with the hardware thereof.

[0119] The bus 1110 can also connect various other circuits or devices, such as peripheral devices, voltage stabilizers, or power management circuits, together. The bus interface 1140 provides an interface between the bus 1110 and the transceiver 1130, which are well known in the art. Therefore, the embodiments of the present application will not be further described.

[0120] The transceiver 1130 can be one element or multiple elements, such as multiple receivers and transmitters, which provide a unit for communicating with various other devices on a transmission medium. For example, the transceiver 1130 receives external data from other devices, and the transceiver 1130 is used to send data processed by the processor 1120 to other devices. Depending on the nature of the computer system, a user interface 1160 can also be provided, such as a touch screen, a physical keyboard, a display, a mouse, a speaker, a microphone, a trackball, a joystick, a stylus.

[0121] It should be understood that the memory 1150 in the embodiments of the present application can further include a memory remotely disposed with respect to the processor 1120, which can be connected to a server through a network. One or more portions of the above-mentioned network can be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless wide area network (WWAN), a metropolitan area network (MAN), the Internet, a public switched telephone network (PSTN), a public or private telephone network, a wireless phone network, a wireless network, a Wi-Fi network, and a combination of two or more of the above-mentioned networks. For example, the cellular phone network and the wireless network can be a global system for mobile communications (GSM) system, a code division multiple access (CDMA) system, a worldwide interoperability for microwave access (WiMAX) system, a general packet radio service (GPRS) system, a wideband code division multiple access (WCDMA) system, a long term evolution (LTE) system, an LTE frequency division duplex (FDD) system, an LTE time division duplex (TDD) system, an LTE-advanced (LTE-A) system, a universal mobile telecommunications system (UMTS) system, an enhanced mobile broadband (eMBB) system, a massive machine type of communication (mMTC) system, an ultra-reliable low latency communications (uRLLC) system, and the like.

[0122] It should be understood that the memory 1150 in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory includes a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM), or a flash memory.

[0123] The volatile memory includes random access memory (RAM), which acts to provide external cache to the processor. By way of example, and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), SynchBurst DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The memory 1150 of the subject embodiments includes, but is not limited to, forms of RAM.

[0124] In the embodiments of the present application, the memory 1150 stores an operating system 1151 and an application program 1152, including executable modules, data structures, or subsets thereof, or expanded sets thereof.

[0125] In particular, the operating system 1151 includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks. The application program 1152 includes various application programs, such as a media player (Media Player) and a browser (Browser), for implementing various application services. The program for implementing the method of the embodiments of the present application can be included in the application program 1152. The application program 1152 includes applets, objects, components, logic, data structures, and other computer system executable instructions for performing specific tasks or implementing specific abstract data types.

[0126] In addition, the embodiments of the present application also provide a computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements each process of the above-mentioned auxiliary method for space perception teaching embodiment and can achieve the same technical effects. To avoid repetition, it will not be repeated here.

[0127] Computer-readable storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer-readable storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, memory cards, solid-state RAM, or any other medium which can be used to store the desired information in a non-transitory fashion. According to specific embodiments of the present application, computer-readable storage media does not include transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0128] In several embodiments provided in the present application, it should be understood that the disclosed apparatus, device and method can be implemented in other manners. For example, the division of the above-described apparatus embodiments is only a logical function division, and there can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be electric, mechanical or other forms of connections.

[0129] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, and can be located in one position, or can be distributed on multiple network units. Some or all of the units can be selected according to actual needs to solve the problems to be solved by the embodiments of the present application.

[0130] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can be a physically separate unit, or two or more units can be integrated into one unit. The integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0131] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (including a personal computer, a server, a data center, or other network devices) to execute all or part of the steps of the methods described in the various embodiments of the present application. The storage medium includes various media listed above.

[0132] In the description of the embodiments of the present application, those skilled in the art should know that the embodiments of the present application can be implemented as methods, devices, apparatuses and storage media. Therefore, the embodiments of the present application can be embodied in the following forms: complete hardware, complete software (including firmware, resident software, microcode, etc.), and a combination of hardware and software. In addition, in some embodiments, the embodiments of the present application can also be embodied in the form of a computer program product in one or more computer readable storage media, which includes computer program code.

[0133] The above computer readable storage medium can adopt any combination of one or more computer readable storage media. The computer readable storage medium includes an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or apparatus, or any combination thereof. More specific examples of the computer readable storage medium include a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory (Flash Memory), an optical fiber, an optical disk only read memory (CD-ROM), an optical storage device, a magnetic storage device, or any combination thereof. In the embodiments of the present application, the computer readable storage medium can be any tangible medium containing or storing a program that can be used or combined with an instruction execution system, device or apparatus.

[0134] The computer program code contained in the above computer readable storage medium can be transmitted by any appropriate medium, including wireless, wire, optical cable, radio frequency (Radio Frequency, RF) or any appropriate combination thereof.

[0135] Computer program code for carrying out operations of embodiments of the present application can be written in an assembly language, an instruction-set-architecture (ISA) language, machine language, machine dependent language, microcode, firmware, state-setting data, integrated circuit configuration data, or in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer program code can execute entirely on a user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0136] The methods, apparatus, and / or devices described in this specification can be presented using flow diagrams and / or block diagrams.

[0137] It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions. These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0138] These computer readable program instructions can also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other

[0139] These computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0140] The above merely describes specific implementation of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the embodiments of the present application, which should be covered in the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application should be subject to the protection scope of the claims.

Claims

1. An auxiliary method for teaching spatial perception, characterized in that, Applied to a mobile terminal, the method includes: Determine the target value for training and record the first starting point of the first user action; When the end command is triggered, determine the critical point of the first user action; The difference between the first starting point and the critical point is used as the measured value; wherein, both the target value and the measured value are distances or angles; Guidance information is determined based on the comparison between the target value and the measured value, and the guidance information is fed back in a form that can be perceived by the visually impaired person. The step of determining the critical point of the first user action when triggering the end command includes: Within a preset time interval, calculate the range of each trajectory point of the first user's action; If the range is less than or equal to a first preset threshold, the termination instruction is triggered, and a trajectory point within the preset time interval is selected as the critical point of the first user action.

2. The method according to claim 1, characterized in that, The measurement methods that the mobile terminal can detect include: distance measurement method and angle measurement method, and the types of the distance measurement method or the angle measurement method are multiple; The method further includes: Receive the user's selection instruction and determine the current measurement method of the corresponding type; The first starting point and the critical point are determined based on the current measurement method.

3. The method according to claim 2, characterized in that, The step of using the difference between the first starting point and the critical point as a measurement value includes: When the current measurement method is the angle measurement method, the rotation angle from the current frame to the next adjacent frame is recorded frame by frame during the execution of the first user action; The rotation angles between two adjacent frames are summed to determine the measurement angle; When the termination command is triggered, the measured angle is used as the measured value.

4. The method according to claim 3, characterized in that, The rotation angle from the current frame to the next adjacent frame is recorded frame by frame during the execution of the first user action, including: During the execution of the first user action, the rotation direction from the current frame to the next adjacent frame is determined; The angle at which the frame rotates from the current frame to the next adjacent frame along the rotation direction is taken as the rotation angle; wherein, if the rotation direction is consistent with the preset direction, the rotation angle is a positive value, and if the rotation direction is opposite to the preset direction, the rotation angle is a negative value. Record the rotation angle from the current frame to the next adjacent frame frame by frame.

5. The method according to claim 1, characterized in that, Also includes: Record the second starting point of the second user's action; When the user is detected to have ended the second user action, the endpoint of the second user action is determined; Feedback is provided in a form that the visually impaired person can perceive, showing the difference between the second starting point and the endpoint of the second user's action.

6. The method according to claim 1, characterized in that, The step of determining the critical point of the first user action when triggering the end command includes: The difference between the current trajectory point of the first user action and the first starting point is taken as the current difference. If the difference between the current difference and the target value does not exceed a second preset threshold, the termination command is triggered, and the current trajectory point is determined as the critical point of the first user action.

7. An auxiliary device for teaching spatial perception, characterized in that, include: The initialization module is used to determine the target value for training and record the first starting point of the first user action; The critical point determination module is used to determine the critical point of the first user action when the end command is triggered; The measurement value determination module is used to take the difference between the first starting point and the critical point as the measurement value; wherein, both the target value and the measurement value are distance or angle. The first feedback module is used to determine guidance information based on the comparison result between the target value and the measured value, and to provide feedback on the guidance information in a form that can be perceived by the visually impaired person. The critical point determination module is specifically used for: Within a preset time interval, calculate the range of each trajectory point of the first user's action; If the range is less than or equal to a first preset threshold, the termination instruction is triggered, and a trajectory point within the preset time interval is selected as the critical point of the first user action.

8. A device comprising a processor and a memory, the memory storing a computer program, characterized in that, The processor executes the computer program stored in the memory to implement the auxiliary method for spatial perception teaching as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the auxiliary method for spatial perception teaching as described in any one of claims 1 to 6.

Images