Control method and device of intelligent tactile stick, intelligent tactile stick and readable medium

Abstract

The invention relates to a control method and device of an intelligent tactile stick, the intelligent tactile stick and a readable medium. The method comprises the steps of obtaining knocking data of the intelligent tactile stick; performing feature recognition on the knock data based on a pre-trained hybrid neural network recognition model, extracting local features and time features of the knock data, and determining knock times and knock frequency of the intelligent tactile stick according to the local features and the time features; the number of knocking times and the knocking frequency of the intelligent tactile stick are matched with an instruction database of an intelligent home system, a target execution instruction is determined, the instruction database comprises the number of knocking times and the incidence relation between the knocking frequency and the execution instruction, and the target execution instruction is determined; the target execution instruction comprises a target control object and a control instruction; and controlling the smart home system to execute the control instruction on the target control object. According to the invention, the linkage control accuracy of the smart home system and the smart tactile stick can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and particularly to a control method, device, intelligent blind cane, and readable medium for an intelligent blind cane. Background Art

[0002] Visually impaired people face many inconveniences in their daily lives. As an important navigation and assistance tool for them, traditional intelligent blind canes mainly rely on physical touch to detect the surrounding environment, with relatively single functions. With the development of technology, although some intelligent blind canes have tried to improve the obstacle avoidance ability by adding sensors such as ultrasonic and infrared sensors, these sensors can detect obstacles at a farther distance and provide sound or vibration feedback to help users better perceive the surrounding environment. However, these improvements mainly focus on the obstacle avoidance function, and there are limitations such as complex operations, low recognition accuracy, and poor user experience in the interaction with the smart home system, making it difficult to meet the actual needs of visually impaired people.

[0003] There are many problems with existing intelligent blind canes when interacting with these systems. For example, many smart home devices need to be controlled through a mobile application or a voice assistant, which is not friendly to visually impaired people. Although some intelligent blind canes have tried to integrate voice control functions, due to low recognition accuracy and complex operations, the user experience is poor, making it difficult to meet the actual needs of visually impaired people. In addition, existing intelligent blind canes lack the ability to intelligently recognize the user's operation intention and cannot automatically adjust functions or interact with smart home devices according to the user's behavior to achieve linkage with smart home devices in a simple and intuitive way.

[0004] In summary, existing intelligent blind canes have problems of complex operations and low recognition accuracy in the linkage control with the smart home system. Summary of the Invention

[0005] This application provides a control method, device, intelligent blind cane, and readable medium for an intelligent blind cane to solve the technical problems of complex operations and low recognition accuracy in the linkage control of existing intelligent blind canes with the smart home system in the above-mentioned prior art.

[0006] According to one aspect of the embodiments of the present application, the present application provides a control method for a smart blind stick, which is applicable to the linkage control between a smart home system and the smart blind stick. The method includes: obtaining the tapping data of the smart blind stick; performing feature recognition on the tapping data based on a pre-trained hybrid neural network recognition model, extracting the local features and time features of the tapping data, and determining the tapping times and tapping frequency of the smart blind stick according to the local features and the time features; matching the tapping times and the tapping frequency of the smart blind stick with an instruction database of the smart home system to determine a target execution instruction, where the instruction database includes the association relationship between the tapping times and the tapping frequency and the execution instruction, and the target execution instruction includes a target control object and a control instruction; controlling the smart home system to execute the control instruction on the target control object.

[0007] Optionally, the tapping data includes vibration data and time series data. The obtaining of the tapping data of the smart blind stick includes: collecting vibration data through a vibration sensor disposed in the smart blind stick within a preset collection time range based on a preset sampling frequency; obtaining the time series data corresponding to the vibration data.

[0008] Optionally, the time series data includes the timestamps of the vibration data. The obtaining of the time series data corresponding to the vibration data includes: determining whether there is a time series in which each time difference satisfies different preset time difference intervals according to the timestamps of the vibration data collected within the preset collection time range; if there is no time series in which each time difference satisfies different preset time difference intervals, determining that there is a unique time series data for the continuously collected vibration data; if there is a time series in which each time difference satisfies different preset time difference intervals, dividing the time series data according to the time series that satisfies different preset time difference intervals to obtain multiple sub-time series data corresponding to the continuously collected vibration data.

[0009] Optionally, the local feature includes a tapping spike, the time feature includes the timestamp corresponding to each tapping spike, and the pre-trained hybrid neural network recognition model performs feature recognition on the tapping data, extracts the local feature and time feature of the tapping data, and determines the tapping times and tapping frequency of the intelligent blind stick according to the local feature and the time feature, including: performing feature recognition on the continuously collected vibration data through the first neural network in the pre-trained hybrid neural network recognition model, and extracting the tapping spikes of the vibration data, wherein the pre-trained hybrid neural network recognition model is trained according to the vibration data samples and time series data samples in the pre-constructed multi-source data samples; performing feature recognition on the time series data corresponding to the continuously collected vibration data through the second neural network in the pre-trained hybrid neural network recognition model, and extracting the timestamps corresponding to each tapping spike in the time series data; determining the tapping times and the tapping frequency of the intelligent blind stick according to the tapping spikes extracted from the vibration data and the timestamps corresponding to each tapping spike in the time series data.

[0010] Optionally, determining the tapping times and the tapping frequency of the intelligent blind stick according to the tapping spikes extracted from the vibration data and the timestamps corresponding to each tapping spike in the time series data includes: calculating based on the peak values of the tapping spikes to determine whether there is a tapping spike whose peak value does not meet the preset peak condition; if there is a tapping spike whose peak value does not meet the preset peak condition, determining the tapping spike whose peak value does not meet the preset peak condition as an abnormal tapping spike, and performing data cleaning on the abnormal tapping spike and the timestamp corresponding to the abnormal tapping spike, and updating the tapping spikes of the vibration data and the time series data; determining the tapping times of the intelligent blind stick based on the updated tapping spikes, and determining the tapping frequency of the intelligent blind stick based on the timestamps corresponding to each tapping spike in the updated time series data.

[0011] Optionally, after matching the tapping times and the tapping frequency of the intelligent blind stick with the instruction database of the smart home system to determine the target execution instruction, the method further includes: obtaining the feedback tapping data of the intelligent blind stick for the target execution instruction; determining whether the feedback tapping data meets a preset specific tapping sequence, wherein the preset specific tapping sequence includes a result confirmation sequence and an action pause sequence; if the feedback tapping data meets the result confirmation sequence, confirming that the target execution instruction is executable; if the feedback tapping data meets the action pause sequence, pausing the execution of the target execution instruction.

[0012] Optionally, after controlling the smart home system to execute the control instruction on the target control object, the method further includes: obtaining an execution result of executing the control instruction on the target control object; obtaining result feedback data of the user based on the execution result, and determining whether the result feedback data meets the target control expectation; if the result feedback data meets the target control expectation, maintaining the state of the execution result; if the result feedback data does not meet the target control expectation, readjusting the target control object and the control instruction according to the number of taps and the tap frequency of the smart blind stick.

[0013] According to another aspect of the embodiments of the present application, the present application provides a control device for a smart blind stick, the device includes: a data acquisition module, configured to acquire tap data of the smart blind stick; a feature recognition module, configured to perform feature recognition on the tap data based on a pre-trained hybrid neural network recognition model, extract local features and time features of the tap data, and determine the number of taps and the tap frequency of the smart blind stick according to the local features and the time features; a data matching module, configured to match the number of taps and the tap frequency of the smart blind stick with an instruction database of the smart home system to determine a target execution instruction, where the instruction database includes an association relationship between the number of taps and the tap frequency and an execution instruction, and the target execution instruction includes a target control object and a control instruction; a control module, configured to control the smart home system to execute the control instruction on the target control object.

[0014] According to another aspect of the embodiments of the present application, the present application provides a smart blind stick, and the smart blind stick realizes the linkage control between the smart blind stick and the smart home system according to the above-mentioned control method of the smart blind stick.

[0015] According to another aspect of the embodiments of the present application, the present application provides a computer-readable medium having non-volatile program code executable by a processor, and the program code causes the processor to execute the steps of the above-mentioned control method of the smart blind stick.

[0016] The above technical solutions provided by the embodiments of the present application have the following advantages compared with the related technologies:

[0017] The present application provides a control method for a smart blind cane, which is applicable to the linkage control scenario between a smart home system and the smart blind cane. Based on the tapping data of the smart blind cane, by obtaining the tapping data of the smart blind cane and inputting it into a pre-trained hybrid neural network recognition model for feature recognition, and then extracting the local features and time features in the tapping data, the tapping times and tapping frequencies of the smart blind cane are analyzed according to the local features and time features, and further matched with the instruction database of the smart home system, so as to obtain the target execution instruction for execution. Among them, tapping is the most basic behavior of the smart blind cane. By analyzing the data generated by the most basic behavior of the smart blind cane, the operation is more convenient; and by using the pre-trained hybrid neural network recognition model to recognize the local features and time features of the tapping data to analyze the tapping times and tapping frequencies of the smart blind cane, it is more conducive to improving the accuracy of data recognition, realizing accurate recognition, and further conducive to accurate matching with the smart home system to obtain the target control instruction and realizing the accurate linkage and interaction ability between the smart blind cane and the smart home system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with the present application and used together with the specification to explain the principles of the present application.

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or related technologies. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 It is a schematic diagram of the hardware environment of an optional control method for a smart blind cane according to an embodiment of the present application;

[0021] Figure 2 It is a schematic diagram of the flow of an optional control method for a smart blind cane according to an embodiment of the present application;

[0022] Figure 3 It is a schematic diagram of the flow of another optional control method for a smart blind cane according to an embodiment of the present application;

[0023] Figure 4 It is a schematic diagram of the flow of an optional step S202 according to an embodiment of the present application;

[0024] Figure 5 It is a schematic diagram of the structure of an optional control device for a smart blind cane according to an embodiment of the present application;

[0025] Figure 6A schematic diagram of an optional computer device structure provided by an embodiment of the present application. Detailed implementation manners

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are some but not all of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0027] To solve the problems mentioned in the background art, according to one aspect of the embodiments of the present application, an embodiment of a control method for an intelligent blind stick is provided.

[0028] As Figure 1 shown, the control method for the intelligent blind stick is applicable to Figure 1 the hardware system environment shown. The system architecture of the hardware system environment includes a terminal device 101, a server 103, an intelligent blind stick 105, and a smart home system 107. Among them, the terminal device 101 includes a client, and the client may include an intelligent blind stick APP. The intelligent blind stick APP, the intelligent blind stick 105, and the smart home system 107 can achieve joint control through wireless communication and network technologies, and jointly provide users with an intelligent and convenient life experience.

[0029] In the intelligent blind stick APP on the terminal device 101, operations such as network configuration for the intelligent blind stick 105, access to the smart home system 107, and instruction setting can be performed, allowing visually impaired users to set the corresponding control of the smart home device status according to personal habits. It supports voice input and voice feedback functions to ensure that users can easily complete settings and adjustments. At the same time, the intelligent blind stick APP also provides detailed device status query and log recording functions to help users understand the operation status and historical operation records of home devices at any time.

[0030] The server 103 is connected to the terminal device 101, the intelligent blind stick 105, and the smart home system 107 through a network, and can be used to provide services for the terminal or the client installed on the terminal, the intelligent blind stick 105, and the smart home system 107, including processing data requests and instructions from the terminal device 101, the intelligent blind stick 105, and the smart home system 107 through the server 103 to achieve communication and data exchange between the two. A database 109 can be set on the server 103 or independently of the server to provide data storage services for the server 103. Among them, the network can include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0031] Among them, the terminal device 101 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, etc. The server 103 can be a server that provides various services. For example, it is a background server that provides support for the pages displayed on the terminal device 101.

[0032] Among them, the smart home system 107 integrates facilities related to home life by using comprehensive wiring technology, network communication technology, security prevention technology, automatic control technology, and audio and video technology to build an efficient management system for residential facilities and family daily affairs. Through the linkage control between the smart blind stick 105 and the smart home system 107, the user can control the smart home system 107 by controlling the smart blind stick 105, so as to realize operation controls such as starting, pausing, and shutting down various smart devices in the home system. The smart home system 107 can monitor the running states of various smart devices in real time, including working hours, current power, state, mode, energy consumption data, etc. The smart home system 107 can record these information on the terminal device 101 through the network in the smart blind stick APP and feedback it to the user in real time, so that the user can always master the usage conditions of various smart home devices.

[0033] It should be noted that the control method of the smart blind stick provided by the embodiments of the present application is generally executed by a server, a terminal device, and / or a smart blind stick. Correspondingly, the control device of the smart blind stick is generally set in the server, the terminal device, or the smart blind stick. And it should be understood that Figure 1 the numbers of smart blind sticks, terminal devices, networks, and servers in

[0034] are only illustrative. According to the implementation requirements, there can be any number of smart blind sticks, terminal devices, networks, and servers. Figure 2 As Figure 2 shown,

[0035] Step S202, obtain the tapping data of the smart blind stick.

[0036] In this embodiment, the control method of the intelligent blind stick is applicable to the scenario of linkage control between the intelligent blind stick and the smart home system, and can provide more convenient auxiliary functions for visually impaired people in the control of the smart home system. A multi-device compatible interface can be provided in the intelligent blind stick to facilitate the design of a unified and standardized communication protocol, enabling the intelligent blind stick to easily access various smart home systems and achieve cross-brand and cross-category device control.

[0037] Among them, a vibration sensor, a sound sensor, etc. can be provided in the intelligent blind stick. When the user uses the intelligent blind stick, the vibration sensor can generate tapping data by tapping the ground. Therefore, the tapping data generated by the vibration sensor can be collected, that is, the tapping data generated by the intelligent blind stick can be obtained. Among them, the tapping data can refer to the data used to reflect the corresponding control actions to be performed on the smart home device. Based on the tapping data collected by different sensors, the tapping times, tapping frequencies, tapping forces, tapping directions, tapping moving distances, etc. of the intelligent blind stick can be identified. For example, the tapping times and tapping frequencies are identified according to the data collected by the vibration sensor to judge the instructions issued by the user or the user's feedback; the tapping force can be identified according to the voiceprint data collected by the sound sensor, and the ground material can be judged. Combining the analysis of the ground material can reduce the vibration interference caused by non-tapping actions; the tapping direction can be identified through the built-in gyroscope sensor / acceleration sensor, and the moving distance of each tap can be identified through the built-in ultrasonic sensor / infrared sensor, which can be used to judge whether the vibration sensor collects non-instruction data due to the user walking normally with the blind stick.

[0038] Step S204: Based on the pre-trained hybrid neural network recognition model, perform feature recognition on the tapping data, extract the local features and time features of the tapping data, and determine the tapping times and tapping frequencies of the intelligent blind stick according to the local features and the time features.

[0039] Among them, the pre-trained hybrid neural network recognition model can be trained by at least two neural networks. In this embodiment, it can be a CNN (Convolutional Neural Network) and an LSTM (Long Short-Term Memory). Among them, through convolutional operations, the CNN can capture local patterns in the data, such as edges, textures, etc. The CNN generally includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The convolutional layer uses a convolutional kernel to slide on the input data to extract local features, and the pooling layer is used to reduce the spatial size of the features, reduce the amount of computation, and prevent overfitting. The LSTM is a special recurrent neural network that can be used to process and predict time series data. Through the gating mechanism (input gate, forget gate, and output gate), the LSTM can capture long-term dependencies in the sequence, including an input layer, an LSTM layer (composed of multiple LSTM units), and an output layer. Each LSTM unit internally contains a forget gate, an input gate, an output gate, and a memory unit, which are used to control the flow and storage of information.

[0040] In this embodiment, by combining the CNN and the LSTM to construct a hybrid neural network recognition model, the local features of the tapping data with time series can be extracted by the CNN therein, and then these features are input into the LSTM to capture long-term dependencies and extract the time dependencies between these local features, that is, time features. This combination method can make full use of the advantages of the CNN in feature extraction and the ability of the LSTM in capturing long-term dependencies, thus being more conducive to accurately identifying the key features in the tapping data sent by the intelligent blind stick, so as to achieve more accurate control of each intelligent device and meet the user's expectations.

[0041] Furthermore, this embodiment can be described based on the number of taps and the tapping frequency of the intelligent blind stick as user instructions. Among them, the local features can be used to reflect the features of the number of taps of the intelligent blind stick, and the time features can be used to reflect the tapping frequency corresponding to the number of taps of the intelligent blind stick, that is, the interval time between multiple tapping actions in a single instruction. After the local features and time features of the tapping data are recognized by the pre-trained hybrid neural network recognition model, the number of taps and the tapping frequency of the intelligent blind stick can be analyzed based on the recognized local features and time features.

[0042] Step S206, match the number of taps and the tapping frequency of the intelligent blind stick with the instruction database of the smart home system to determine the target execution instruction, where the instruction database includes the association relationship between the number of taps and the tapping frequency and the execution instruction, and the target execution instruction includes the target control object and the control instruction.

[0043] CombineFigure 3 As shown, as a possible implementation, after completing the hardware design of the sensor for the intelligent blind stick to count the number of taps and tap frequency, a matching mobile phone APP can be developed for the intelligent blind stick. The APP is mainly used for operations such as network configuration of the intelligent blind stick, access to third-party systems, and instruction settings. It allows visually impaired users to set the corresponding control of smart home device states according to personal habits, and the data of the preset tap patterns corresponding to the control of smart home device states can be stored in the intelligent blind stick and the smart home system respectively. Among them, the APP supports voice input and voice feedback functions to ensure that users can conveniently complete settings and adjustments. At the same time, the APP also provides detailed device status query and log recording functions to help users understand the operation status and historical operation records of home devices at any time. For example: if a user wants to set a scene where the air conditioner is adjusted to the sleep mode and the electrical appliances in the living room are turned off before going to bed. To do this, the user opens the APP, adds the sleep scene based on voice or screen operations, sets the air conditioner sleep mode + turning off other home appliances through voice or screen operations, and then the APP prompts the user to set the corresponding action. At this time, the user taps the blind stick 4 times with a tap frequency of 0.2 seconds. The vibration sensor of the blind stick receives the action for recognition and transmits the action of tapping 4 times with a tap frequency of 0.2 seconds to the smart home system. The smart home system transmits the action to the mobile phone APP, and the APP prompts that the setting is completed; simultaneously, the action of tapping the blind stick 4 times with a tap frequency of 0.2 seconds is associated with the action of air conditioner sleep mode + turning off other home appliances and saved in the smart home system and the intelligent blind stick. The next time the intelligent blind stick operates with the action of tapping 4 times with a tap frequency of 0.2 seconds, the smart home system will quickly locally link the device to execute the action of air conditioner sleep mode + turning off other home appliances. In this way, it will not be necessary to go through the server and the mobile phone, etc., ensuring fast and stable operation.

[0044] Furthermore, both the smart cane and the smart home control system may be provided with a memory for data storage, including storage of preset tapping patterns and the corresponding smart home device states, data generated by each control, device data, etc. In the smart home system, a command database may be constructed to store the mapping relationship between the number of tapping times and the tapping frequency and the corresponding control instructions. By matching the number of tapping times and the tapping frequency of the identified smart cane with the mapping relationship between the number of tapping times and the tapping frequency and the control instructions in the command database of the smart home system, the target execution instruction may be determined, that is, the control instruction and the target control object that the smart cane wants to execute may be determined. Among them, the target execution instruction may be to control a single device, or to control multiple devices in a certain scenario. For example: tap 3 times continuously, the tapping frequency between the two intervals is between 0.5 seconds and 1 second, the target control object in the target execution instruction is the TV, and the control instruction is to increase the volume; or tap 5 times continuously, the tapping frequency is 0.3 seconds, the target control objects in the target execution instruction are the TV and the bedroom light, and the control instruction is to turn off the TV and the bedroom light at the same time.

[0045] Step S208: controlling the smart home system to execute the control instruction on the target control object.

[0046] Among them, after data matching is performed in the smart home system, the smart home system can be controlled to execute control instructions on the target control object. For example, if the smart cane is matched and tapped 4 times with a tapping frequency of 0.3 seconds, the matching result is that the air conditioner switches to sleep mode and controls the smart electric curtains to close. For this, the smart home system will quickly link the air conditioner and the smart electric curtains locally to execute: air conditioner sleep mode + smart electric curtain closing action. Another example: when the visually impaired person A returns home at night, he only needs to tap the ground twice with a cane with a tapping frequency of 0.1 seconds. The smart home system can recognize it as an instruction to turn on the living room lights. At this time, the lights in the living room will automatically light up to illuminate his way home. Another example: The visually impaired person B feels that the indoor temperature is too high, and taps the ground four times with a smart cane. The system recognizes it as an instruction to turn on the air conditioner and adjust it to the cooling mode. At this time, the air conditioner will automatically start and adjust according to the preset temperature and wind speed, bringing a cool and comfortable environment to Xiao Li. If you want to adjust the temperature or wind speed of the air conditioner, just tap the cane again and select the corresponding option according to the voice prompt in the APP.

[0047] In the embodiments of the present invention, based on the tapping data of the intelligent blind stick, by acquiring the tapping data of the intelligent blind stick and inputting it into a pre-trained hybrid neural network recognition model for feature recognition, and then extracting the local features and time features in the tapping data, the tapping times and tapping frequencies of the intelligent blind stick are analyzed based on the local features and time features, and further matched with the instruction database of the smart home system, so as to obtain the target execution instruction for execution. Among them, tapping is the most basic behavior of the intelligent blind stick. By analyzing the data generated by combining the most basic behavior of the intelligent blind stick, the operation is more convenient; and by using the pre-trained hybrid neural network recognition model to identify the local features and time features of the tapping data to analyze the tapping times and tapping frequencies of the intelligent blind stick, it is more conducive to improving the accuracy of data recognition, realizing accurate recognition, and further conducive to accurately matching with the smart home system to obtain the target control instruction and realizing the accurate linkage and interaction ability between the intelligent blind stick and the smart home system.

[0048] In some alternative embodiments, in combination with Figure 4 as shown, the tapping data includes vibration data and time series data, and the above step S202 includes:

[0049] S2021, based on a preset sampling frequency, within a preset acquisition time range, collect vibration data through a vibration sensor provided in the intelligent blind stick;

[0050] S2023, obtain the time series data corresponding to the vibration data.

[0051] Among them, the tapping data may include vibration data and the time series data associated with the process of generating the vibration data. The above preset sampling frequency can be used as a minimum frequency limit. After the generated vibration is higher than this minimum frequency, the vibration data can be collected through the vibration sensor in the intelligent blind stick. In this way, it is more conducive to avoiding the influence of vibrations generated during the normal use of the user or vibrations generated by environmental noise on the accuracy of vibration data collection. Among them, the vibration data can be represented in the form of a vibration signal, for example, a vibration waveform.

[0052] In some possible embodiments, the acquisition time range can be preset. The preset acquisition time range can be the maximum data acquisition time, and the maximum data acquisition time can be used as the time limit for collecting vibration data. When the acquisition time reaches the maximum data acquisition time, the current acquisition can be stopped, avoiding the situation where the vibration sensor is in the data acquisition stage for a long time but no vibration data can be collected subsequently, reducing the response duration and improving the response efficiency.

[0053] Further, the above time series data is also the time series that generates vibration data. The time series includes timestamps for generating each vibration peak, and the time difference can also be obtained based on the timestamps of adjacent vibration peaks in the time series. After collecting vibration data through a vibration sensor disposed in the intelligent blind stick within a preset sampling frequency and a preset acquisition time range, the corresponding time series data of the vibration data can be obtained synchronously.

[0054] In this embodiment, preliminary limitation is performed through the preset sampling frequency, which is more conducive to avoiding the influence of vibrations generated during the normal use of the user or vibrations generated by environmental noise on the accuracy of vibration data collection; by setting the preset acquisition time range, it is possible to avoid the situation where the vibration sensor is in the data collection stage for a long time but no vibration data can be collected subsequently, reducing the response duration and being beneficial to improving the response efficiency.

[0055] In some alternative embodiments, the time series data includes timestamps of each vibration data, and the above step S2024 includes:

[0056] Judging whether there is a time series in which each time difference respectively satisfies different preset time difference intervals according to the timestamps of the vibration data collected within the preset acquisition time range;

[0057] If there is no time series in which each time difference respectively satisfies different preset time difference intervals, it is determined that there is a unique time series data for the continuously collected vibration data;

[0058] If there is a time series in which each time difference respectively satisfies different preset time difference intervals, the time series data is divided according to the time series that satisfies different preset time difference intervals, and a plurality of sub-time series data corresponding to the continuously collected vibration data are obtained.

[0059] In some examples, since the tapping actions generated by the user are not mechanically generated and the tapping frequencies of different users are different, time difference intervals can be preset for each vibration data respectively. The preset time difference intervals can be determined based on a reference value and an allowable error range. The reference value can be determined according to the timestamps corresponding to the same tapping instruction during the user's multiple historical uses, including determining the average value, median value, variance, etc. of multiple tapping frequencies collected by issuing the same tapping instruction. For example: the user issues a tapping instruction A through the intelligent blind stick and taps continuously 3 times. The tapping frequency of the first time is 0.5 seconds, the tapping frequency of the second time is 0.6 seconds, and the tapping frequency of the third time is 0.4 seconds. If the reference value is determined by the average value, the reference value is 0.5 seconds. When the error range is ±0.1 seconds, the corresponding preset time difference interval is 0.4 - 0.6 seconds.

[0060] Further, based on the timestamps generated for each vibration data, the time differences between the vibration data can be calculated. If all the time differences satisfy the same preset time difference interval, it can be determined that there is only one time series data among the vibration data continuously collected within the preset acquisition time range. If among all the time differences, there are different time differences that respectively satisfy different preset time difference intervals, it can be determined that there are multiple time series data in the continuously collected vibration data. In this case, the timestamps corresponding to the different time differences before and after can be used as nodes to divide the time series data, obtaining multiple sub-time series data. For example, if there are 5 knocks, the knocking frequency of the first 3 knocks is 0.3 seconds, satisfying the preset time difference interval of 0.28 - 0.35 seconds, and the knocking frequency of the last 2 knocks is 1 second, satisfying the preset time difference interval of 0.9 - 1.1 seconds, then it can be considered that there are 2 sub-time series; if there are 5 knocks and the knocking frequency of all 5 knocks is 0.5 seconds, satisfying the preset time difference interval of 0.45 - 0.55 seconds, then it is determined that there is only one time series.

[0061] As a possible implementation, considering the limitation on the number of knocks of the knocking action for visually impaired persons, it is usually limited to a relatively short number of knocks. If a longer number of knocks is set, it will limit the convenience of the knocking action. Therefore, in order to achieve control in more scenarios or modes, by dividing the knocking frequency corresponding to the number of knocks of the knocking action into multiple sub-time series, the control instructions for visually impaired persons can be increased, realizing control of more functions. For example, for the case of only one time series data, the corresponding control action can be set as single-device control; for the case of multiple sub-time series data, the corresponding control action can be set as multi-device joint control.

[0062] In this embodiment, by identifying and dividing the time series, it is more convenient for visually impaired persons to make operation distinctions, and it is also convenient for the home intelligent system to quickly identify whether it is single-device control or multi-device joint control, which not only improves the convenience of use but also facilitates the setting of more control functions.

[0063] In some alternative embodiments, the local feature includes a knocking peak, and the time feature includes the timestamp corresponding to each knocking peak. The above step S204 includes:

[0064] S2041, using the first neural network in the pre-trained hybrid neural network recognition model to perform feature recognition on the continuously collected vibration data, and extracting the knocking peak of the vibration data, where the pre-trained hybrid neural network recognition model is trained according to the vibration data samples and time series data samples in the pre-constructed multi-source data samples;

[0065] S2042. Use the second neural network in the pre-trained hybrid neural network recognition model to perform feature recognition on the time series data corresponding to the continuously collected vibration data, and extract the timestamps corresponding to each tapping spike in the time series data;

[0066] S2043. Determine the number of taps and the tapping frequency of the intelligent blind cane according to the tapping spikes extracted from the vibration data and the timestamps corresponding to each tapping spike in the time series data.

[0067] In some possible implementation manners, to achieve feature recognition, model training can be performed in advance. The algorithm can use a hybrid neural network architecture (CNN + LSTM) to build the model. The first neural network CNN is used to extract local features of the vibration sensor signal, such as the tapping spikes in the vibration waveform, and the second neural network LSTM is used to capture the tapping time dependence relationship, such as the interval frequency between two taps.

[0068] The hybrid neural network recognition model can be trained through vibration data samples and time series data samples in a pre-constructed multi-source data sample. Among them, the multi-source data sample can include multiple sample data of different types and sources. Before training the hybrid neural network recognition model, first input the data: vibration data samples and time series data samples, which are input in real time with a sliding window, such as a 0.5-second window. Output data: the number of taps and the tapping frequency. Among them, the number of taps belongs to a classification task, and the tapping frequency belongs to a regression task. When training the model, first perform multi-user pre-training. By collecting diverse samples, including different holding angles, tapping forces, environmental interference data, etc., and annotating the data, that is, marking the exact timestamps and intensities of each tap. Further through transfer learning, use large-scale user data to train the basic network. For example, the dataset can include 10 visually impaired users, with 200 tapping samples for each person. Optionally, when a new user uses it for the first time, a personalized dataset can be generated through 10 standard taps. Incremental learning mechanism: By continuously recording user operations, when a mis-trigger is detected, automatically store the current sensor data in the buffer as mis-trigger data for avoidance.

[0069] In this embodiment, before feature recognition and extraction, the data can be preprocessed, including cleaning, denoising, and normalizing the collected vibration data to ensure the accuracy and consistency of the data. The features (knocking peaks) of the vibration data of the blind stick are extracted through a CNN, and determining the number of knocks specifically includes: the trained CNN model can extract key features such as knocking peaks from the vibration data, and these features are usually located in the output of the convolutional layer. Further, by applying a classifier or a regressor to the output of the fully connected layer, the number of knocks is determined according to the extracted knocking peaks, including classifying the number of knocks using a softmax classifier or performing regression prediction on the number of knocks using a linear regressor. For example, if the output of the classifier is [0.1, 0.8, 0.1], the number of knocks can be considered to be 2.

[0070] Further, in the trained second neural network, the LSTM network, the output of the LSTM layer before the fully connected layer can be regarded as the feature representation of the vibration data, and these features contain key information in the time series data, such as vibration frequency, amplitude, timestamp, etc., corresponding to each knocking peak. Therefore, to analyze the extracted knocking peaks, a visualization tool can be used to plot the curve of the knocking peaks changing with time, so as to obtain the vibration frequency. For example, the curve is plotted through the matplotlib tool.

[0071] In some possible implementation manners, in order to distinguish user actions and habits, model training can also be performed through multi-modal feature fusion, including extracting time-domain features in the vibration data: knocking peak value, waveform integral (energy), zero-crossing rate, obtaining frequency-domain features in the time series data: FFT (Fast Fourier Transform) to extract the knocking frequency, and collecting voiceprint data through a voice sensor. For example, the voiceprint features of the sound collected by a microphone are used for multi-feature fusion analysis, which is more conducive to excluding non-knocking vibration interference. The hybrid neural network recognition model trained in this way can also be more refined to match user actions and habits.

[0072] In this embodiment, a hybrid neural network recognition model is constructed based on a CNN and an LSTM. Not only can the CNN in it extract the knocking peaks of the knocking data with a time series, but also based on the knocking peaks, the LSTM can capture the long-term dependence relationship to extract the time dependence relationship between the knocking peaks, that is, the time feature. Finally, the knocking frequency and the number of knocks are obtained. This combination method can make full use of the advantages of the CNN in feature extraction and the ability of the LSTM in capturing long-term dependence relationships, so as to be more conducive to accurately identifying the key features in the knocking data sent by the intelligent blind stick, so as to achieve more accurate control of each intelligent device and meet the user's expectations.

[0073] In some alternative embodiments, S2043 includes:

[0074] Calculating based on the peak values of each of the tapping spikes to determine whether there is a tapping spike whose peak value does not meet the preset peak condition;

[0075] If there is a tapping spike whose peak value does not meet the preset peak condition, then determine the tapping spike that does not meet the preset peak condition as an abnormal tapping spike, and perform data cleaning on the abnormal tapping spike and the time stamp corresponding to the abnormal tapping spike, and update the tapping spikes and the time series data of the vibration data;

[0076] Determine the number of taps of the intelligent blind cane based on the updated tapping spikes, and determine the tapping frequency of the intelligent blind cane based on the time stamps corresponding to each tapping spike in the updated time series data.

[0077] In this embodiment, for the case of multiple taps, in order to improve the accurate recognition of the number of taps and the tapping frequency, further recognition can be combined with the effectiveness of the tapping spikes. The above peak values can be represented by tapping intensity or force, so that each tapping spike can be numerically processed. In order to judge the effectiveness of the tapping spikes, a preset peak condition can be set. The preset peak condition can be a range from a minimum effective peak value to a maximum effective peak value. For example, when the tapping force or intensity satisfies the range of M2 to M1, the corresponding tapping spike is judged to meet the preset peak condition.

[0078] Further, if there is a tapping spike whose peak value does not meet the preset peak condition, it can be determined as an abnormal tapping spike, which is not generated by the user's tapping action. For example, the preset minimum effective peak value is M1, the preset maximum effective peak value is M2, the user only taps 4 times b1, b2, b3, b4, and the peak values B1 of the 4 times satisfy M1 ≤ B1 ≤ M2, but there are 5 tapping spikes in the waveform. In addition to b1, b2, b3, b4, it also includes b5 arranged at the end, and the peak value is B2 < M1. In this case, judge the tapping spike b5 as an abnormal tapping spike, which is not caused by the user's tapping action.

[0079] Further, for the case where there are abnormal tapping spikes, data cleaning can be performed on the abnormal tapping spikes and their corresponding time stamps, for example, deleting them. In this way, the update of the tapping spikes and the corresponding time series data can be achieved, and then the number of the updated tapping spikes can be used as the number of taps, and the tapping frequency of the intelligent blind cane can be calculated according to the time stamps corresponding to each updated tapping spike in the updated time series data.

[0080] In this embodiment, by analyzing the tapping spikes, data cleaning is performed on abnormal tapping spikes that do not meet the preset peak effective conditions and the corresponding timestamps, enabling the update of the tapping spikes and the corresponding time series data. This is beneficial for more accurately filtering out tapping spikes caused by factors such as non-tapping actions of the user, environmental noise, and floor materials, reducing the misjudgment rate, and improving the accuracy of the data and the accuracy of data matching with the smart home system.

[0081] In some alternative embodiments, after step S206, the method further includes:

[0082] Obtaining feedback tapping data of the intelligent blind cane for the target execution instruction;

[0083] Determining whether the feedback tapping data meets a preset specific tapping sequence, where the preset specific tapping sequence includes a result confirmation sequence and an action pause sequence;

[0084] If the feedback tapping data meets the result confirmation sequence, it is confirmed that the target execution instruction is executable;

[0085] If the feedback tapping data meets the action pause sequence, the execution of the target execution instruction is paused.

[0086] In this embodiment, in order to prevent misoperations and ensure the security of user sensitive data, a security verification link can be set. For example, changing device settings or deleting user information, etc. When a target execution instruction is matched in the smart home system, the target execution instruction can be fed back to the intelligent blind cane. In the intelligent blind cane, the specific content of the currently matched target execution instruction can be reminded to the user through voice broadcast or other means, and then the user can send feedback tapping data to the smart home system by tapping.

[0087] In some examples, when the feedback tapping data meets the result confirmation sequence, it is considered that the target execution instruction can be executed. For example: The feedback tapping data of the intelligent blind cane is: tap 2 times, the tapping frequency is 1 second, which meets the result confirmation sequence, and it is confirmed to execute the target execution instruction; when the intelligent blind cane taps 4 times, the tapping frequency is 0.2 seconds, the voice will prompt whether to confirm the execution of the sleep mode, and when the intelligent blind cane taps 2 times, the tapping frequency is 0.2 seconds, it means confirmation of execution, and the sleep mode action will be executed.

[0088] In some other examples, when the feedback tapping data meets the action pause sequence, it is considered that the target execution instruction cannot be executed and the execution is paused. For example, the action pause sequence corresponding to the emergency stop function is set as follows: tap 2 times first, with a tapping frequency of 0.2 seconds, then tap 2 times again, with a tapping frequency of 1 second. When the feedback tapping data meets this sequence, it is judged as an emergency stop. After triggering the emergency stop, a voice prompt will ask whether it is an emergency stop. After the user taps 2 times with the intelligent blind stick, with a tapping frequency of 2 seconds to confirm, the scene will execute a stop action, and at the same time, the controlled intelligent device will resume the previous state to avoid misoperation. Among them, whether it is paused or can be executed, a voice prompt can be given through the intelligent blind stick to announce the execution situation.

[0089] In this embodiment, in order to ensure the safety and privacy of users as well as prevent misoperation, a security verification process is set during the linkage control process between the intelligent blind stick and the smart home system. Whether the target execution instruction can be executed or whether the execution of the target execution instruction needs to be paused is judged according to the feedback tapping data of the intelligent blind stick on the target execution instruction. This not only improves the user's security protection of sensitive data but also reduces the possibility of misoperation.

[0090] In some alternative embodiments, after step S208, the method further includes:

[0091] Obtain the execution result of executing the control instruction on the target control object;

[0092] Obtain the result feedback data of the user based on the execution result, and judge whether the result feedback data meets the target control expectation;

[0093] If the result feedback data meets the target control expectation, maintain the state of the execution result;

[0094] If the result feedback data does not meet the target control expectation, readjust the target control object and the control instruction according to the number of taps and the tapping frequency of the intelligent blind stick again.

[0095] In this embodiment, in combination with Figure 3As shown, after the control instruction is executed on the target control object, the execution result can be obtained first, and then the execution result can be sent to the intelligent blind stick. The execution result can be prompted to the user in the form of voice prompts. The user feeds back the execution feedback data through the intelligent blind stick. Among them, the execution feedback data can be fed back by means of tapping or pressing buttons to indicate whether it meets the target control expectation. Among them, the target control expectation can represent the control effect that the user himself wants to achieve. If the result feedback data meets the target control expectation, the state of the execution result is maintained. For example, if it meets the target control expectation, the intelligent blind stick is tapped once and slides continuously for 1 second. If the result feedback data executes this action, the result feedback data meets the target control expectation. On the contrary, if the result feedback data does not meet the target control expectation, it can be considered that there is an error in the initial setting process or an error occurs when matching with the smart home system. To improve the accuracy of the target execution instruction, the target control object and the control instruction can be adjusted again according to the tapping times and tapping frequencies of the intelligent blind stick. For example, the user can set different tapping times and tapping frequencies of the intelligent blind stick through voice / mobile APP to trigger different scenarios.

[0096] In this embodiment, after the target execution instruction is executed, the result feedback data of the user on the execution result is obtained to determine whether it meets the target control expectation. In the case of non-compliance, the target control object and the control instruction are adjusted again according to the tapping times and tapping frequencies of the intelligent blind stick, which can improve the accuracy of the execution of the target execution instruction and meet the personalized adjustment of the user.

[0097] According to another aspect of the embodiments of the present application, as Figure 5 shown, corresponding to the control method of the intelligent blind stick in the above embodiments, this embodiment provides a control device for the intelligent blind stick. The device includes:

[0098] A data acquisition module 501, configured to acquire the tapping data of the intelligent blind stick;

[0099] A feature recognition module 503, configured to perform feature recognition on the tapping data based on a pre-trained hybrid neural network recognition model, extract the local features and time features of the tapping data, and determine the tapping times and tapping frequencies of the intelligent blind stick according to the local features and the time features;

[0100] A data matching module 505, configured to match the tapping times and the tapping frequencies of the intelligent blind stick with an instruction database of the smart home system to determine a target execution instruction, where the instruction database includes the association relationship between the tapping times and the tapping frequencies and the execution instruction, and the target execution instruction includes a target control object and a control instruction;

[0101] A control module 507, configured to control the smart home system to execute the control instruction on the target control object.

[0102] It should be noted that in this embodiment, the data acquisition module 501 may be used to execute step S202 in the embodiment of the present application, the feature recognition module 503 in this embodiment may be used to execute step S204 in the embodiment of the present application, the data matching module 505 in this embodiment may be used to execute step S206 in the embodiment of the present application, and the control module 507 in this embodiment may be used to execute step S208 in the embodiment of the present application.

[0103] It should be noted here that the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules, as part of the device, can run in a hardware environment as shown in Figure 1 and can be implemented by software or by hardware.

[0104] According to another aspect of the embodiments of the present application, the present application provides a smart blind stick. The smart blind stick realizes the linkage control between the smart blind stick and the smart home system according to the control method of the smart blind stick, and can implement each embodiment in the control method of the smart blind stick and achieve the corresponding technical effects, which will not be elaborated here.

[0105] Optionally, a camera, a sound sensor, a vibration sensor, etc. that can implement functions such as data acquisition and recognition can be integrated in the smart blind stick.

[0106] According to another aspect of the embodiments of the present application, the present application provides a computer device, as shown in Figure 6 , including a memory 601, a processor 603, a communication interface 605 and a communication bus 607. A computer program that can run on the processor 603 is stored in the memory 601. The memory 601 and the processor 603 communicate through the communication interface 605 and the communication bus 607. When the processor 803 executes the computer program, the steps of the above control method of the smart blind stick are implemented.

[0107] The memory and the processor in the above computer device communicate through the communication bus and the communication interface. The communication bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0108] The above-mentioned memory may include a Random Access Memory (RAM), or may also include a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0109] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0110] According to another aspect of the embodiments of the present application, there is also provided a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps of the control method of the intelligent blind stick in any of the above embodiments.

[0111] Optionally, in the embodiments of the present application, the computer-readable medium is set to store program codes for the processor to execute the steps of the control method of the intelligent blind stick in the above embodiments.

[0112] Optionally, specific examples in this embodiment may refer to the examples described in the above embodiments, and details are not repeated here. When the embodiments of the present application are specifically implemented, reference may be made to the above various embodiments, and corresponding technical effects are achieved.

[0113] It will be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For a hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or a combination thereof. For a software implementation, the techniques described herein can be implemented by units that execute the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented within the processor or external to the processor.

[0114] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0115] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the division of the modules is only a logical function division, and there can be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0116] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, ROMs, RAMs, magnetic disks, or optical discs that can store program codes.

[0117] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the said element.

[0118] The above are only specific embodiments of the present application, enabling those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but rather will conform to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A control method for an intelligent blind stick, applicable to the linkage control between a smart home system and an intelligent blind stick, characterized in that, The method includes: Obtaining the tapping data of the intelligent blind cane; Performing feature recognition on the tapping data based on a pre-trained hybrid neural network recognition model, extracting the local features and temporal features of the tapping data, and determining the tapping times and tapping frequency of the intelligent blind cane according to the local features and the temporal features; Matching the tapping times and the tapping frequency of the intelligent blind cane with the instruction database of the smart home system to determine a target execution instruction, wherein the instruction database includes the association relationship between the tapping times and the tapping frequency and the execution instruction, and the target execution instruction includes a target control object and a control instruction; Controlling the smart home system to execute the control instruction on the target control object.

2. The control method of the intelligent blind stick according to claim 1, characterized in that The tapping data includes vibration data and time series data. The obtaining of the tapping data of the intelligent blind cane includes: Collecting vibration data through a vibration sensor disposed in the intelligent blind cane within a preset acquisition time range based on a preset sampling frequency; Obtaining the time series data corresponding to the vibration data.

3. The control method of the intelligent blind stick according to claim 2, wherein The time series data includes the timestamps of the respective vibration data. The obtaining of the time series data corresponding to the vibration data includes: Judging whether there is a time series in which each time difference respectively satisfies different preset time difference intervals according to the timestamps of the vibration data collected within the preset acquisition time range; If there is no time series in which each time difference respectively satisfies different preset time difference intervals, it is determined that there is a unique time series data in the continuously collected vibration data; If there is a time series in which each time difference respectively satisfies different preset time difference intervals, the time series data is divided according to the time series that satisfies different preset time difference intervals to obtain a plurality of sub-time series data corresponding to the continuously collected vibration data.

4. The control method of the intelligent blind stick according to claim 2, wherein, The local features include tapping spikes, and the temporal features include the timestamps corresponding to each tapping spike. The performing of feature recognition on the tapping data based on a pre-trained hybrid neural network recognition model, extracting the local features and temporal features of the tapping data, and determining the tapping times and tapping frequency of the intelligent blind cane according to the local features and the temporal features includes: Performing feature recognition on the continuously collected vibration data through a first neural network in the pre-trained hybrid neural network recognition model to extract the tapping spikes of the vibration data, wherein the pre-trained hybrid neural network recognition model is trained according to the vibration data samples and time series data samples in a pre-constructed multi-source data sample; Performing feature recognition on the time series data corresponding to the continuously collected vibration data through a second neural network in the pre-trained hybrid neural network recognition model to extract the timestamps corresponding to each tapping spike in the time series data; Determining the tapping times and the tapping frequency of the intelligent blind cane according to the tapping spikes extracted from the vibration data and the timestamps corresponding to each tapping spike in the time series data.

5. The control method of the intelligent blind stick according to claim 4, characterized in that, Determining the number of taps and the tapping frequency of the intelligent blind cane based on the tapping spikes extracted from the vibration data and the timestamps corresponding to each tapping spike in the time series data includes: Calculating based on the peak values of each of the tapping spikes to determine whether there is a tapping spike whose peak value does not meet the preset peak condition; If there is a tapping spike whose peak value does not meet the preset peak condition, determine the tapping spike that does not meet the preset peak condition as an abnormal tapping spike, and perform data cleaning on the abnormal tapping spike and the timestamp corresponding to the abnormal tapping spike, and update the tapping spikes of the vibration data and the time series data; Determine the number of taps of the intelligent blind cane based on the updated tapping spikes, and determine the tapping frequency of the intelligent blind cane based on the timestamps corresponding to each tapping spike in the updated time series data.

6. The control method of the intelligent blind stick according to claim 1, wherein After matching the number of taps and the tapping frequency of the intelligent blind cane with the instruction database of the smart home system to determine the target execution instruction, the method further includes: Obtain the feedback tapping data of the intelligent blind cane for the target execution instruction; Determine whether the feedback tapping data meets a preset specific tapping sequence, where the preset specific tapping sequence includes a result confirmation sequence and an action pause sequence; If the feedback tapping data meets the result confirmation sequence, confirm that the target execution instruction is executable; If the feedback tapping data meets the action pause sequence, pause the execution of the target execution instruction.

7. The control method of the intelligent blind stick according to any one of claims 1 to 6, characterized in that, After controlling the smart home system to execute the control instruction on the target control object, the method further includes: Obtain the execution result of executing the control instruction on the target control object; Obtain the result feedback data of the user based on the execution result, and determine whether the result feedback data meets the target control expectation; If the result feedback data meets the target control expectation, maintain the state of the execution result; If the result feedback data does not meet the target control expectation, readjust the target control object and the control instruction according to the number of taps and the tapping frequency of the intelligent blind cane again.

8. A control device for an intelligent blind stick, characterized in that, The device includes: A data acquisition module for acquiring the tapping data of the intelligent blind cane; A feature recognition module for performing feature recognition on the tapping data based on a pre-trained hybrid neural network recognition model, extracting the local features and time features of the tapping data, and determining the number of taps and the tapping frequency of the intelligent blind cane according to the local features and the time features; A data matching module for matching the number of taps and the tapping frequency of the intelligent blind cane with the instruction database of the smart home system to determine the target execution instruction, where the instruction database includes the association relationship between the number of taps and the tapping frequency and the execution instruction, and the target execution instruction includes the target control object and the control instruction; A control module for controlling the smart home system to execute the control instruction on the target control object.

9. An intelligent blind stick, characterized in that, The intelligent blind stick realizes the linkage control between the intelligent blind stick and the smart home system according to the control method of the intelligent blind stick described in any one of claims 1 to 7 above.

10. A computer-readable medium having non-volatile program code executable by a processor, characterized in that, The program code causes the processor to execute the steps of the control method of the intelligent blind stick described in any one of claims 1 to 7.

Images