Wearable device-based prompting method, wearable device, and storage medium

Abstract

The application discloses a prompt method based on a wearable device, the wearable device and a storage medium, relates to the technical field of data processing, and the method comprises the following steps: when a user wears or stretches the wearable device, real-time monitoring of multi-modal data of the wearable device is performed, wherein the multi-modal data comprises strain sensor data, acceleration data and image data representing deformation of a connecting part; whether the wearable device has an over-deformation risk is determined according to the multi-modal data; and if it is determined that the wearable device has the over-deformation risk, prompt information representing that the wearable device has the over-deformation risk is output. The application can give an early warning on the over-deformation risk of the wearable device, and support the improvement of the service life of the wearable device.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a prompting method, wearable device, and storage medium based on wearable devices. Background Technology

[0002] Wearable devices (such as headphones) are widely used in consumer electronics, gaming, and professional audio fields due to their good sound quality and wearing stability. However, with the increase in market share, the issue of "headband breakage" has become increasingly prominent in user complaints.

[0003] Analysis revealed that the main cause of headband breakage was the habitual overstretching of the headband (the connecting part of the wearable device) during use or storage. This caused plastic deformation of the headband material (especially plastic or lightweight alloys). When the deformation exceeded the material's yield strength, cracks or even breakage occurred. However, there is currently no solution to provide early warning for excessive deformation of wearable devices and reduce the risk of such deformation.

[0004] Therefore, how to provide early warnings of the risk of excessive deformation of wearable devices has become an urgent problem to be solved.

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main objective of this application is to provide a prompting method, wearable device, and storage medium based on wearable devices, aiming to solve the technical problem of how to provide early warning of the risk of excessive deformation of wearable devices.

[0007] To achieve the above objectives, this application proposes a prompting method based on a wearable device. The wearable device includes a connecting part and functional components connected to both ends of the connecting part. The prompting method based on the wearable device includes the following steps: When a user wears or stretches the wearable device, the multimodal data of the wearable device is monitored in real time. The multimodal data includes strain sensor data, acceleration data and image data that characterize the deformation of the connection. Determine whether wearable devices are at risk of excessive deformation based on multimodal data; If it is determined that the wearable device is at risk of excessive deformation, a warning message indicating that the wearable device is at risk of excessive deformation will be output.

[0008] Optionally, a strain sensor may be installed at the connection point. The steps for determining whether a wearable device is at risk of excessive deformation based on multimodal data include: Acquire strain sensor data collected when the joint deforms; Detect whether the strain sensor data is greater than a first preset threshold; If the strain sensor data is greater than the first preset threshold, it is determined that the wearable device is at risk of excessive deformation.

[0009] Optionally, the prompting method based on wearable devices also includes: Determine the index corresponding to the material used in the connection part, and enter the index into a lookup table containing the mapping relationship between materials and yield limit strain values ​​to obtain the first yield limit strain value; The difference between the first yield limit strain value and the preset safety margin is determined, and the difference is used as the first preset threshold.

[0010] Optionally, an acceleration sensor is provided at the connection point. The steps for determining whether wearable devices are at risk of excessive deformation based on multimodal data also include: When deformation occurs at the connection point, acceleration data collected by multiple acceleration sensors at different locations of the connection point is obtained; The first opening angle of the wearable device is determined based on multiple acceleration data. Detect whether the first opening angle is greater than a preset angle threshold; If the first opening angle is greater than the preset angle threshold, it is determined that the wearable device is at risk of excessive deformation.

[0011] Optionally, the step of determining the first opening angle of the wearable device based on multiple acceleration data includes: Among multiple acceleration data, the first acceleration data collected by the acceleration sensor located in the middle of the connection part, and the second acceleration data collected by the acceleration sensor located in a non-middle position of the connection part are identified; The first opening angle is obtained by calculating the angle based on the first acceleration data and the second acceleration data.

[0012] Optionally, the step of determining whether a wearable device is at risk of excessive deformation based on multimodal data further includes: Determine the external image acquisition device in the environment where the wearable device is located, and collect multiple frames of image data from the wearable device; Image recognition is performed on each frame of image data to obtain the second opening angle of the wearable device; Detect whether the second opening angle is greater than a preset angle threshold; If the second opening angle is greater than the preset angle threshold, it is determined that the wearable device is at risk of excessive deformation.

[0013] Optionally, after the step of real-time monitoring of multimodal data from wearable devices, the method further includes: Based on multimodal data, the deformation direction corresponding to the deformation of the connection part of the wearable device is determined, wherein the deformation direction includes the tensile direction and the torsional direction; Based on the deformation direction and multimodal data, determine whether wearable devices are at risk of excessive deformation.

[0014] Optionally, the step of determining whether the wearable device is at risk of excessive deformation based on the deformation direction and multimodal data includes: The first preset threshold and / or preset angle threshold corresponding to the wearable device are updated according to the deformation direction, and the step of determining whether the wearable device has excessive deformation risk based on multimodal data is performed based on the updated first preset threshold and / or preset angle threshold.

[0015] In addition, to achieve the above objectives, this application also proposes a wearable device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the prompting method based on the wearable device as described above.

[0016] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the above-described prompting method based on a wearable device.

[0017] In this application, by monitoring the multimodal data of the wearable device in real time when the user wears or stretches the device, namely strain sensor data, acceleration data, and image data characterizing the deformation of the connection part of the wearable device, it is possible to obtain sensor data characterizing the deformation information of the connection part from multiple dimensions. After obtaining the multimodal data, it is determined whether there is a risk of excessive deformation of the wearable device. When an excessive deformation risk is determined, a warning message characterizing the excessive deformation risk of the wearable device is output. This enables early warning of deformation or breakage of the wearable device, reduces the risk of wearable device deformation, and improves the service life of the wearable device. Furthermore, since the excessive deformation risk prediction of the wearable device is performed using multimodal data, it can be adapted to the excessive deformation risk prediction of wearable devices in different scenarios, thereby improving the accuracy of excessive deformation risk prediction. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a prompting method based on a wearable device in this application; Figure 2 This is a schematic diagram of a scenario in the wearable device-based prompting method of this application, where the wearable device is worn on the user's head; Figure 3 This is a schematic diagram of a scenario in which a strain sensor is set in a wearable device in the prompting method based on a wearable device in this application; Figure 4 This is a schematic diagram of different opening angles in the prompting method based on wearable devices in this application; Figure 5 This is a schematic diagram illustrating a scenario of deformation in the stretching direction of the wearable device in the notification method based on wearable devices in this application; Figure 6 This is a schematic diagram illustrating a scenario where the wearable device undergoes torsional deformation in the notification method based on the wearable device in this application. Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the prompting method based on wearable devices in the embodiments of this application.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] Optionally, the implementing entity in this application embodiment may be a wearable device.

[0025] Alternatively, users may habitually overstretch the connecting parts during the use or storage of wearable devices (such as headphones), causing plastic deformation of the connecting material, or even cracks or breaks.

[0026] Optionally, embodiments of this application can avoid the phenomenon of wearable devices deforming or breaking due to excessive stretching by the user, resulting in a lack of proactive warnings. This proactively prevents the risk of excessive deformation by intervening at the source of user operation, rather than remedying the situation afterward, effectively reducing the risk of connection breakage. It also achieves low cost and lightweight design, requiring only a small number of additional sensors and chips, without replacing high-strength materials or increasing the structural size of the wearable device, thus having minimal impact on its weight and cost. Furthermore, it enhances the intelligence of the wearable device by setting different thresholds (such as a first preset threshold and a preset angle threshold) based on different connection materials (e.g., plastic, metal, carbon fiber) to adapt to different product models. Intuitive audio-visual prompts can also help users develop correct usage habits, thereby extending the lifespan of the wearable device.

[0027] Alternatively, wearable devices can be electronic devices that need to be worn on the human body, such as headphones, smart glasses, head massagers, etc.

[0028] Optionally, the wearable device includes a connecting part and functional components connected to both ends of the connecting part. Taking a headset as an example, the connecting part is the headband, and the functional components are the earcups of the headset.

[0029] Optionally, the connecting part can be a structural layout that spans the human head, which serves to connect two functional layouts at both ends of the wearable device and provide support and clamping force for the wearable device when worn.

[0030] Optionally, the functional component may be a structural component connected to both ends of the connector to realize the core functions of the wearable device.

[0031] Optionally, the wearable device may include a sensor module, a main control chip, and a prompting module. The sensor module may include a strain sensor, an acceleration sensor, and other types of sensors.

[0032] Optionally, the main control chip can have controller functions and can be a processor such as an MCU.

[0033] Optionally, the sensor modules can be evenly arranged along the length direction on the inner or outer side of the connecting part (such as the head beam), for example, 3-5 modules can be arranged.

[0034] Optionally, the main control chip can be electrically connected to the sensor module. The main control chip has a built-in storage unit and a comparison and judgment unit. The storage unit is used to store various types of data, such as data collected by the sensor module, a first preset threshold, a preset angle threshold, etc. The main control chip can execute the various steps of the prompting method based on the wearable device.

[0035] Optionally, the alert module may include an alert unit, such as a buzzer, and may also include an LED indicator. When the main control chip determines that the wearable device has a risk of excessive deformation, the alert module may output corresponding warning information, such as a continuous buzzer or a rapidly flashing LED.

[0036] Based on this, the embodiments of this application provide a prompting method based on a wearable device, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the prompting method based on wearable devices in this application.

[0037] In this embodiment, the prompting method based on wearable devices is applied to wearable devices. Wearable devices include a connecting part and functional components connected to both ends of the connecting part. The prompting method based on wearable devices includes steps S10 to S30.

[0038] Step S10: Monitor the multimodal data of the wearable device in real time when the user wears or stretches the wearable device; It should be noted that the multimodal data includes strain sensor data, acceleration data, and image data characterizing the deformation of the connection.

[0039] Alternatively, sensor data collected by different sensor modules when the connection part of the wearable device deforms can be used as multimodal data.

[0040] Optionally, the strain sensor data can be sensor data collected by the strain sensor on the connection part of the wearable device, which can characterize the micro-deformation and / or stress concentration at a local location of the connection part.

[0041] Optionally, the strain sensor data can be electrical or optical signal data that reflects the degree of local strain on or inside the connection surface.

[0042] Optionally, the acceleration data can be sensor data collected by an accelerometer from the connection part of the wearable device. This data can characterize the attitude changes of each node in the chain in space, thereby calculating the opening angle of the connection part of the wearable device and determining whether there is a risk of excessive deformation.

[0043] Optionally, the image data can be the external geometry representing the entire connection part, or it can be image data of the entire protective wearable device captured by an external image acquisition device (such as a mobile phone camera, webcam, etc.).

[0044] Optionally, the status of the wearable device can be monitored in real time. When the wearable device is detected to be worn or stretched by the user, the sensor module can monitor the sensor data of the wearable device when it is deformed due to the user wearing or stretching, and use it as multimodal data.

[0045] Optionally, when the main control chip in the wearable device detects that the wearable device is being worn or stretched by the user, it can activate the sensor module to collect data from the connection part of the wearable device. This data can be obtained when the connection part is deformed due to the user wearing or stretching the wearable device, such as strain sensor data and acceleration data. The collected sensor data can be used as multimodal data.

[0046] Alternatively, image data containing the wearable device can be obtained by capturing images of the wearable device, such as taking photos or recording videos, and the image data can be sent to the wearable device, which stores the image data as multimodal data in one dimension.

[0047] Reference Figure 2 The user wears a wearable device equipped with three sensors. At this time, the wearable device's sensors collect multimodal data in real time and perform subsequent operations.

[0048] Step S20: Determine whether the wearable device is at risk of excessive deformation based on multimodal data; Step S30: If it is determined that the wearable device has a risk of excessive deformation, output a prompt message indicating that the wearable device has a risk of excessive deformation.

[0049] Optionally, the risk of excessive deformation can be that the connection part of the wearable device is approaching or has already approached the material mechanical performance limit in its current deformed state. If the deformation is maintained or increased, it will lead to irreversible plastic deformation (i.e., it cannot return to its original state on its own) or breakage damage of the connection part.

[0050] Optionally, the prompt message can be a warning signal used to attract the user's attention, which can be conveyed through sound, image display, vibration, or other means.

[0051] Optionally, the main control chip can filter and amplify the data collected by the sensor to obtain multimodal data, and then determine whether the wearable device has the risk of excessive deformation based on the multimodal data.

[0052] Optionally, after the wearable device is powered on, the main control chip can perform a self-test to confirm that the communication between each sensor (such as a strain sensor and an accelerometer) and the main control chip is normal. The main control chip can also establish a communication connection with external image acquisition devices, such as Bluetooth or Wi-Fi. After completing the self-test, the main control chip can enter a real-time monitoring mode to detect and determine whether the wearable device is at risk of deformation or breakage.

[0053] Optionally, during the process of the user wearing or stretching the wearable device, the main control chip controls each sensor to collect data at a preset sampling frequency (e.g., 50Hz). For example, the strain sensor collects the strain signal of the connection part, the accelerometer collects the spatial attitude signal of the connection part, and the external image acquisition device captures an image sequence containing the overall shape of the wearable device and sends it to the main control chip.

[0054] Optionally, the main control chip can preprocess the sensor data collected by each sensor and use the preprocessed sensor data as multimodal data. Data preprocessing may include signal amplification and noise filtering.

[0055] Optionally, the main control chip can determine whether the wearable device has a risk of excessive deformation based on multimodal data. For example, multimodal data can be input into a preset neural network model, and the output can provide specific conclusions about whether the wearable device has a risk of excessive deformation, such as whether there is a risk of excessive deformation or not. Alternatively, other judgment methods can be used to determine whether the wearable device has a risk of excessive deformation, such as comparing it with a pre-set threshold.

[0056] Optionally, if it is determined that the wearable device does not pose a risk of excessive deformation, the wearable device can continue to be monitored to obtain new multimodal data, and the step of determining whether the wearable device poses a risk of excessive deformation based on the new multimodal data can be performed.

[0057] Optionally, if it is determined that the wearable device poses a risk of excessive deformation, a warning message indicating this risk can be output. For example, an LED indicator or a buzzer can be used in the warning module to provide the user with the corresponding warning message.

[0058] In this embodiment, by monitoring the multimodal data of the wearable device in real time when the user wears or stretches the device, namely strain sensor data, acceleration data, and image data characterizing the deformation of the connection part of the wearable device, it is possible to obtain sensor data characterizing the deformation information of the connection part from multiple dimensions. After obtaining the multimodal data, it is determined whether there is a risk of excessive deformation of the wearable device. When an excessive deformation risk is determined, a prompt message characterizing the excessive deformation risk of the wearable device is output. This enables early warning of deformation or breakage of the wearable device, reduces the risk of wearable device deformation, and improves the service life of the wearable device. Furthermore, since the excessive deformation risk prediction of the wearable device is performed using multimodal data, it can adapt to the excessive deformation risk prediction of wearable devices in different scenarios, thereby improving the accuracy of excessive deformation risk prediction.

[0059] Based on the first embodiment of this application, a second embodiment of this application is proposed. In the second embodiment of this application, the same or similar contents as those in the above embodiment can be referred to the above description, and will not be repeated hereafter.

[0060] Optionally, a strain sensor may be installed at the connection point.

[0061] Alternatively, the strain sensor can be a sensing element that can convert the strain (i.e., the amount of deformation per unit length) on the surface of the joint into a measurable electrical or optical signal.

[0062] Alternatively, strain sensors can be evenly distributed along the length of the connection (such as the head beam).

[0063] Alternatively, the strain sensor may be a resistive strain gauge or a piezoelectric sensor.

[0064] Optionally, if the strain sensor uses a resistive strain gauge, when the resistive strain gauge deforms with the connection, the length and cross-sectional area of ​​its internal metal sensitive grid or semiconductor material change, resulting in a change in resistance value. The change in resistance is then converted into a voltage signal to obtain the corresponding strain sensor data.

[0065] For example, such as Figure 3 As shown, the user configures a wearable device equipped with a strain sensor and collects strain sensor data. The strain sensor can be placed in the stress concentration area of ​​the connection or the weakest area of ​​the structure. By detecting the strain data in the stress concentration area or the weakest area of ​​the structure and comparing it with the strain sensor data at the corresponding location when failure occurs, it can be determined whether the wearable device has a risk of excessive deformation. If it exceeds the threshold, an excessive deformation risk is determined. The first preset threshold can be the strain sensor data at the time of failure, or it can be set based on the strain sensor data at the time of failure.

[0066] Optionally, step S20, which involves determining whether the wearable device is at risk of excessive deformation based on multimodal data, includes steps a10-a30.

[0067] Step a10: Obtain strain sensor data collected by the strain sensor when the connection deforms; Optionally, multiple strain sensors can be set at different parts of the connection. For example, a strain sensor can be set in the middle of the connection, and a strain sensor can be set on each of the left and right sides of the connection at a certain distance from the middle.

[0068] Optionally, strain sensor data collected by strain sensors at different locations of the connection when the connection deforms can be obtained and sent to the main control chip.

[0069] Step a20: Detect whether the strain sensor data is greater than the first preset threshold; Optionally, the first preset threshold can be a pre-set critical value of the strain sensor data.

[0070] Optionally, step a20 can be performed for each strain sensor data point acquired.

[0071] Optionally, in an implementable scenario, the wearable device-based prompting method further includes steps b10-b20.

[0072] Step b10: Determine the index corresponding to the material used in the connection part, and input the index into a lookup table containing the mapping relationship between materials and yield limit strain values ​​to obtain the first yield limit strain value; Step b20: Determine the difference between the first yield limit strain value and the preset safety margin, and use the difference as the first preset threshold.

[0073] Optionally, the yield limit strain value can be the critical strain point at which the material transitions from the elastic deformation stage to the plastic deformation stage during a tensile or bending test.

[0074] Optionally, the preset safety margin can be a safety buffer value reserved below the material's yield limit strain value to ensure that the wearable device can maintain its structural and functional integrity during long-term repeated use. For example, the preset safety margin can be set to 10% or 20% of the yield limit value.

[0075] Alternatively, different models and batches of wearable devices may use different materials for the connectors, such as titanium alloy or plastic.

[0076] Optionally, different first preset thresholds can be set for different wearable devices.

[0077] Optionally, the material used for the connection part of the wearable device can be determined first, and the index corresponding to that material can be read from the storage unit. The storage unit pre-stores the indexes corresponding to different materials.

[0078] Alternatively, the index can be entered into a lookup table containing the mapping relationship between materials and yield limit strain values ​​to obtain the corresponding yield limit strain value, which can then be used as the first yield limit strain value.

[0079] Optionally, a preset safety margin can be determined based on the first yield strength strain value. For example, 10% of the first yield strength strain value can be used as the preset safety margin.

[0080] Optionally, the difference between the first yield limit strain value and the preset safety margin can be calculated, and the difference can be used as the first preset threshold.

[0081] In this embodiment, the accuracy of the first preset threshold can be ensured by determining the first preset threshold based on the first yield strength strain value corresponding to the material used in the connection part and in combination with a preset safety margin.

[0082] Step a30: If the strain sensor data is greater than the first preset threshold, it is determined that the wearable device has a risk of excessive deformation.

[0083] Optionally, if the strain sensor data is determined to be greater than a first preset threshold, it can be preliminarily determined that the wearable device has a risk of excessive deformation. If the strain sensor data is determined to be less than or equal to the first preset threshold, other multimodal data can be used to further determine whether the wearable device has a risk of excessive deformation.

[0084] Optionally, the location area with the risk of excessive deformation can be determined based on strain sensor data that is greater than a first preset threshold, and a prompt message can be output for that location area.

[0085] For example, if a strain sensor and a corresponding indicator module (such as an indicator light) are installed in the middle of the connection part, and a strain sensor is installed on each of the left and right sides of the connection part at a certain distance from the middle part, and a corresponding indicator module (such as an indicator light) is installed on each of the left and right sides.

[0086] If the strain sensor data collected by the strain sensor located in the middle of the connection part exceeds a first preset threshold, it can be determined that there is a risk of excessive deformation at the middle of the connection part of the wearable device. A warning module located in the middle of the connection part can then be activated, such as an indicator light, to remind the user that the wearable device is at risk of excessive deformation in its current state, and that there is a risk of deformation or breakage at the middle of the connection part.

[0087] In this embodiment, by collecting strain sensor data, and determining that the wearable device is at risk of excessive deformation when the strain sensor data is greater than a first preset threshold, a prompt message is output to remind the user that the wearable device is at risk of excessive deformation, thereby improving the service life of the wearable device.

[0088] Optionally, an acceleration sensor may be provided at the connection point.

[0089] Alternatively, the acceleration sensors can be evenly distributed along the length of the connection (such as the head beam).

[0090] Reference Figure 4 Since different users wear wearable devices at different opening angles, the accelerometers installed in the connection part of the wearable device can be used to detect whether the wearable device is at risk of excessive deformation.

[0091] Optionally, the accelerometer can detect its own pitch and roll degrees of freedom in real time. By comparing it with the accelerometer in the middle position, the opening angle of the wearable device can be calculated, and the deformation of the connecting parts such as the headband can be obtained. If the degree of deformation exceeds the limit deformation condition (such as greater than 170°), an alarm will be triggered.

[0092] Optionally, step S20, which involves determining whether the wearable device is at risk of excessive deformation based on multimodal data, includes steps c10-c40.

[0093] Step c10: When the connecting part deforms, acquire acceleration data collected by multiple acceleration sensors at different positions of the connecting part; Step c20: Determine the first opening angle of the wearable device based on multiple acceleration data; Step c30: Detect whether the first opening angle is greater than the preset angle threshold; Step c40: If the first opening angle is greater than the preset angle threshold, it is determined that the wearable device has a risk of excessive deformation.

[0094] Optionally, the first opening angle can be the degree of opening of the two functional components of the wearable device relative to the wearable device in its naturally closed state when stretched by the user, and can be measured in terms of angle values, i.e., the opening angle.

[0095] Optionally, the larger the first opening angle, the more the connecting part is stretched and the greater the degree of deformation.

[0096] Optionally, the preset angle threshold can be a critical angle value representing the opening angle of the wearable device that is stored in the memory unit within the main control chip, such as 170°.

[0097] Optionally, multiple acceleration sensors can be set at different parts of the connection. For example, an acceleration sensor can be set in the middle of the connection, and an acceleration sensor can be set on each of the left and right sides of the connection at a certain distance from the middle.

[0098] Optionally, acceleration data collected by acceleration sensors at different locations of the connection part when the connection part deforms can be obtained and sent to the main control chip.

[0099] Optionally, when the user wears or stretches the wearable device, the connecting part may deform. At this time, sensor data, such as acceleration data, can be collected by accelerometers located at various positions on the connecting part. The first opening angle of the wearable device can be determined based on the acceleration data. For example, the acceleration data can be input into a neural network to output the first opening angle. Alternatively, the index corresponding to the acceleration data can be input into a lookup table containing the opening angle to obtain the first opening angle.

[0100] Optionally, an example is given by setting an acceleration sensor at the middle, left, and right ends of the connection part.

[0101] When the user stretches the wearable device, the connecting part bends and deforms, and the accelerometers at the left and right ends of the connecting part will undergo spatial displacement as the functional components open.

[0102] Optionally, taking the left-hand accelerometer as an example (the same applies to the right-hand one), the main control chip extracts the gravitational acceleration from the X, Y, and Z-axis data corresponding to the acceleration data collected by the accelerometer. Since the output vector of the accelerometer mainly reflects the projection of gravitational acceleration onto the sensor's local coordinate system when in a static or slowly changing motion state, the pitch and roll angles of the accelerometer relative to the horizontal plane are calculated using arctangent trigonometric functions.

[0103] Optionally, the main control chip performs a differential operation (i.e., subtraction) between the pitch / roll angle calculated by the left-end accelerometer and the corresponding angle calculated by the reference accelerometer at the middle position to obtain the relative pitch and roll angles of the left end relative to the middle position. The key function of this differential operation is to eliminate common-mode interference caused by the overall tilt of the user's head (such as looking down, up, or tilting the head), and retain only the actual attitude change caused by the bending deformation of the connection part itself.

[0104] Optionally, the main control chip calculates the spatial angle between the left-end functional component and the right-end functional component based on the structural geometric parameters of the connecting part (including the length L of the connecting part half arm and the static angle in the initial closed state), combined with the relative attitude angles corresponding to the left-end accelerometer and the right-end accelerometer respectively, through spatial vector rotation matrix operation, and uses it as the first opening angle.

[0105] Optionally, after determining the first opening angle, the first opening angle is compared with a preset angle threshold (e.g., 170°). If the first opening angle is greater than the preset angle threshold, it is determined that the wearable device has a risk of excessive deformation. If the first opening angle is less than or equal to the preset angle threshold, other multimodal data can be used to determine whether the wearable device has a risk of excessive deformation.

[0106] In this embodiment, by collecting acceleration data and determining the corresponding first opening angle, when the first opening angle is determined to be greater than a preset angle threshold, it is determined that the wearable device has a risk of excessive deformation, and then a prompt message is output to remind the user that the wearable device has a risk of excessive deformation, thereby improving the service life of the wearable device.

[0107] Optionally, step c20, which involves determining the first opening angle of the wearable device based on multiple acceleration data, includes steps d10-d20.

[0108] Step d10: Determine the first acceleration data collected by the acceleration sensor located in the middle of the connection part and the second acceleration data collected by the acceleration sensor located in a non-middle position of the connection part from among the multiple acceleration data. Step d20: Calculate the angle based on the first acceleration data and the second acceleration data to obtain the first opening angle.

[0109] Alternatively, the intermediate position can be the location of the geometric center point of the connecting part along its length. For example, if the wearable device is a pair of headphones and the connecting part is a headband, then the intermediate position is the center point of the top of the headband.

[0110] Optionally, the non-intermediate position can be a gas position on the connector other than the intermediate position, including positions near both ends of the connector.

[0111] Optionally, since the functional components expand outward along both sides of the user's head during the stretching process, the displacement and posture changes at non-central positions are significantly greater than those at the central positions.

[0112] Optionally, the first acceleration data may be acceleration data collected by an acceleration sensor located in the middle of the connection, and may include component values ​​of the three axes X, Y, and Z.

[0113] Optionally, the second acceleration data may be acceleration data collected by an acceleration sensor located at a non-middle position of the connection, and may include component values ​​of the X, Y, and Z axes.

[0114] Optionally, the second acceleration data may include acceleration data collected by the left-end acceleration sensor (hereinafter referred to as the third acceleration data) and acceleration data collected by the right-end acceleration sensor (hereinafter referred to as the fourth acceleration data).

[0115] Optionally, the first acceleration data and the second acceleration data can be input into the neural network to output the first opening angle.

[0116] Optionally, the spatial attitude angles corresponding to the first acceleration data and the second acceleration data can be calculated. Alternatively, the spatial attitude angles can be determined by looking up a table or by using a neural network.

[0117] Optionally, the spatial attitude angles include the pitch angle, roll angle, and yaw angle of the accelerometer, which correspond to rotation about the longitudinal axis, lateral axis, and vertical axis, respectively.

[0118] Optionally, the relative attitude angle between the non-intermediate position and the intermediate position can be determined based on the spatial attitude angle corresponding to the second acceleration data and the spatial attitude angle corresponding to the first acceleration data. For example, the absolute value of the difference between the spatial attitude angle corresponding to the second acceleration data and the spatial attitude angle corresponding to the first acceleration data can be used as the relative attitude angle, and the first opening angle can be calculated based on the relative attitude angle. For example, the index corresponding to the relative attitude angle can be entered into a lookup table to obtain the first opening angle.

[0119] In this embodiment, the first opening angle is calculated and determined based on the first acceleration data corresponding to the middle position of the connecting part and the second acceleration data corresponding to the non-middle position, thereby ensuring the accuracy and effectiveness of the obtained first opening angle.

[0120] Optionally, step S20, which determines whether the wearable device has a risk of excessive deformation based on multimodal data, further includes steps e10-e40.

[0121] Step e10: Determine the external image acquisition device of the environment where the wearable device is located, and collect multiple frames of image data from the wearable device. Step e20: Perform image recognition on each frame of image data to obtain the second opening angle of the wearable device; Step e30: Detect whether the second opening angle is greater than the preset angle threshold; Step e40: If the second opening angle is greater than the preset angle threshold, it is determined that the wearable device has a risk of excessive deformation.

[0122] Alternatively, the external image acquisition device can be an electronic device with image / video capture capabilities that is independent of the wearable device, such as a mobile phone or camera.

[0123] Optionally, the multi-frame image data can be multiple images of the wearable device captured by an external image acquisition device in a continuous time series.

[0124] Optionally, the second opening angle can be the opening angle between two functional components of the wearable device, obtained by recognizing and measuring each frame of image data using image recognition technology.

[0125] Optionally, when a user wears or stretches the wearable device, an external image acquisition device can capture image data containing the wearable device at a preset frame rate (e.g., 15 frames per second) and send it to the wearable device.

[0126] The wearable device preprocesses multiple frames of image data, such as through filtering. Then, it performs image recognition on each frame to identify the wearable device contained within the image data, and calculates the opening angle of the identified wearable device, using it as the second opening angle.

[0127] Optionally, the second opening angle is compared with a preset angle threshold. If the second opening angle is greater than the preset angle threshold, it is determined that the wearable device has a risk of excessive deformation. If the second opening angle is less than or equal to the preset angle threshold, other multimodal data can be used to further determine whether the wearable device has a risk of excessive deformation.

[0128] Optionally, in one scenario embodiment, a tiered warning process can also be implemented. If simultaneously the strain sensor data is detected to be greater than a first preset threshold, the first opening angle is greater than a preset angle threshold, and the second opening angle is greater than a preset angle threshold, then it is determined that the wearable device has a high risk of excessive deformation. At this time, a buzzer and an indicator light can be used together to remind the user, for example, the buzzer will beep continuously and the indicator light will flash, to remind the user that the wearable device has a risk of deformation or breakage in the current state, until the user turns off the buzzer and indicator light.

[0129] If the strain sensor reading is greater than the first preset threshold, the first opening angle is greater than the preset angle threshold, and the second opening angle is less than or equal to the preset angle threshold; or, the strain sensor reading is greater than the first preset threshold, the first opening angle is less than or equal to the preset angle threshold, and the second opening angle is greater than the preset angle threshold; or, the strain sensor reading is less than or equal to the first preset threshold, the first opening angle is greater than the preset angle threshold, and the second opening angle is greater than the preset angle threshold, then it is determined that the wearable device has a medium-risk excessive deformation risk. At this time, the user can be reminded that the wearable device has a risk of deformation or breakage in the current state by continuously sounding a buzzer or continuously flashing an indicator light until the user turns off the buzzer and indicator light.

[0130] If the strain sensor reading is greater than the first preset threshold, the first opening angle is less than or equal to the preset angle threshold, and the second opening angle is less than or equal to the preset angle threshold; or, the strain sensor reading is less than or equal to the first preset threshold, the first opening angle is greater than the preset angle threshold, and the second opening angle is less than or equal to the preset angle threshold; or, the strain sensor reading is less than or equal to the first preset threshold, the first opening angle is less than or equal to the preset angle threshold, and the second opening angle is greater than the preset angle threshold, then it is determined that the wearable device has a low-risk risk of excessive deformation. At this time, a short-term reminder can be given through the indicator light, such as the indicator light flashing for 1 minute or 5 minutes, to remind the user that the wearable device has a risk of excessive deformation in the current state.

[0131] In this embodiment, by combining image data acquired by an external image acquisition device, the second opening angle of the wearable device is determined. When the second opening angle is greater than a preset angle threshold, it is determined that the wearable device has a risk of excessive deformation, and then a prompt message is output to remind the user that the wearable device has a risk of excessive deformation, thereby improving the service life of the wearable device.

[0132] Based on the first or second embodiment of this application, a third embodiment of this application is proposed. In the third embodiment of this application, the same or similar content as the above embodiment can be referred to the above description, and will not be repeated hereafter.

[0133] Optionally, after step S10, which involves real-time monitoring of the multimodal data of the wearable device, steps f10-f20 are also included.

[0134] Step f10: Determine the deformation direction corresponding to the deformation of the connection part of the wearable device based on the multimodal data, wherein the deformation direction includes the tensile direction and the torsional direction; Step f20: Determine whether the wearable device is at risk of excessive deformation based on the deformation direction and multimodal data.

[0135] Optionally, the stretching direction can be the direction in which the two functional components at both ends of the connector move away from each other, that is, the direction in which the connector as a whole is stretched or stretched.

[0136] Optionally, the deformation in the stretching direction of the wearable device can be referenced. Figure 5 That is, the direction of deformation is the direction of tension.

[0137] Optionally, the direction of torsion can be the direction in which the functional components at both ends of the connector rotate relative to each other about the length axis of the connector. For example, when a user removes the headphones with one hand, a torsional force is applied to one earcup, and the direction of the deformation of the headband is the direction of torsion.

[0138] Optionally, the torsional deformation of the wearable device is referenced. Figure 6 That is, the direction of deformation is the direction of torsion.

[0139] Optionally, the deformation direction can be determined based on strain sensor data, acceleration data, and image data.

[0140] Optionally, when the connection primarily undergoes tensile deformation (i.e., the corresponding deformation direction is the tensile direction), the accelerometers located on both sides of the connection (left and right ends) will detect symmetrical rotation about the pitch axis, i.e., the left end rotates to the upper left and the right end rotates to the upper right, and the strain sensor detects a significant increase in the normal strain along the length of the connection. When the connection primarily undergoes torsional deformation (i.e., the corresponding deformation direction is the torsional direction), the accelerometers at the left and right ends will detect opposite rotation about the yaw axis (one end clockwise and the other end counterclockwise), and the strain sensor detects a significant increase in the shear strain component.

[0141] Alternatively, image recognition can be performed directly on the image data to determine its deformation direction.

[0142] Optionally, it can be determined whether there is a risk of excessive deformation of the wearable device based on the deformation direction and multimodal data, and when it is determined that there is a risk of excessive deformation, corresponding prompt information can be output.

[0143] Optionally, the deformation direction and multimodal data can be input into the neural network to output the result of whether the wearable device is at risk of excessive deformation.

[0144] In this embodiment, the deformation direction is determined based on multimodal data, and the wearable device is determined to have excessive deformation risk based on the deformation direction and multimodal data. Once the excessive deformation risk is determined, a prompt message is output to remind the user that the wearable device has excessive deformation risk, thereby improving the service life of the wearable device.

[0145] Optionally, step f20, determining whether the wearable device has a risk of excessive deformation based on the deformation direction and multimodal data, includes step g10.

[0146] Step g10: Update the first preset threshold and / or preset angle threshold corresponding to the wearable device according to the deformation direction, and perform the step of determining whether the wearable device has excessive deformation risk based on multimodal data according to the updated first preset threshold and / or preset angle threshold.

[0147] Optionally, different deformation directions correspond to different first preset thresholds, or they can correspond to different preset angle thresholds.

[0148] Optionally, the first preset threshold corresponding to the stretching direction is greater than the first preset threshold corresponding to the torsion direction, and the preset angle threshold corresponding to the stretching direction is greater than the preset angle threshold corresponding to the torsion direction.

[0149] Optionally, when determining whether there is a risk of excessive deformation in the wearable device based on multimodal data, the obtained first preset threshold and / or preset angle threshold can be updated according to the direction of deformation, such as reducing or increasing the first preset threshold and / or preset angle threshold.

[0150] Optionally, strain sensor data can be compared with an updated first preset threshold. If the strain sensor data is greater than the updated first preset threshold, it is determined that the wearable device has a risk of excessive deformation. A first opening angle and / or a second opening angle can be compared with an updated preset angle threshold. If the first opening angle and / or the second opening angle is greater than the updated preset angle threshold, it is determined that the wearable device has a risk of excessive deformation.

[0151] In this embodiment, by updating the first preset threshold and / or the preset angle threshold according to the deformation direction, and then using this to determine whether the wearable device has an excessive deformation risk, the accuracy of the judgment on the excessive deformation risk of the wearable device is improved.

[0152] Furthermore, this application provides a wearable device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the prompting method based on the wearable device in Embodiment 1 above.

[0153] The following is for reference. Figure 7 The figure illustrates a structural schematic diagram suitable for implementing wearable devices according to embodiments of this application. Wearable devices in embodiments of this application may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The devices shown in the figure are merely examples and should not impose any limitations on the functionality and scope of use of embodiments of this application.

[0154] Wearable devices may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for device operation. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows wearable devices to communicate wirelessly or wiredly with other devices to exchange data. While wearable devices with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0155] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0156] The wearable device provided in this application, employing the wearable device-based prompting method described in the above embodiments, can solve the technical problem of how to provide early warning of the risk of excessive deformation of the wearable device. Compared with the prior art, the beneficial effects of the wearable device provided in this application are the same as those of the wearable device-based prompting method provided in the above embodiments, and other technical features of the wearable device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0157] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0158] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0159] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the prompting method based on a wearable device in the above embodiments.

[0160] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0161] The aforementioned computer-readable storage medium may be included in the wearable device; or it may exist independently and not assembled into the wearable device.

[0162] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the wearable device, enable the wearable device to perform the steps in the aforementioned prompting method based on the wearable device.

[0163] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0164] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0165] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0166] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned wearable device-based prompting method, thereby solving the technical problem of how to provide early warning of the risk of excessive deformation of wearable devices. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the wearable device-based prompting method provided in the above embodiments, and will not be repeated here.

[0167] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described prompting method based on a wearable device.

[0168] The computer program product provided in this application can solve the technical problem of how to provide early warning of the risk of excessive deformation of wearable devices. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the wearable device-based prompting method provided in the above embodiments, and will not be repeated here.

[0169] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A notification method based on wearable devices, characterized in that, The method for prompting based on a wearable device, the wearable device including a connecting part and functional components connected to both ends of the connecting part, includes the following steps: When a user wears or stretches the wearable device, the multimodal data of the wearable device is monitored in real time, wherein the multimodal data includes strain sensor data, acceleration data and image data characterizing the deformation of the connecting part; Based on the multimodal data, determine whether the wearable device is at risk of excessive deformation; If it is determined that the wearable device has a risk of excessive deformation, a prompt message indicating that the wearable device has a risk of excessive deformation will be output.

2. The prompting method based on wearable devices as described in claim 1, characterized in that, A strain sensor is installed in the connecting part. The step of determining whether the wearable device has a risk of excessive deformation based on the multimodal data includes: Acquire strain sensor data when the strain sensor deforms in response to the connection portion; Detect whether the strain sensor data is greater than a first preset threshold; If the strain sensor data is greater than a first preset threshold, it is determined that the wearable device is at risk of excessive deformation.

3. The prompting method based on wearable devices as described in claim 2, characterized in that, The notification method based on wearable devices further includes: Determine the index corresponding to the material used in the connection part, and input the index into a lookup table containing the mapping relationship between materials and yield limit strain values ​​to obtain the first yield limit strain value; The difference between the first yield strength strain value and the preset safety margin is determined, and the difference is used as the first preset threshold.

4. The prompting method based on wearable devices as described in claim 1, characterized in that, An acceleration sensor is installed in the connecting part. The step of determining whether the wearable device has a risk of excessive deformation based on the multimodal data further includes: When the connecting part deforms, acceleration data collected by multiple acceleration sensors at different positions of the connecting part are obtained; The first opening angle of the wearable device is determined based on multiple acceleration data. Detect whether the first opening angle is greater than a preset angle threshold; If the first opening angle is greater than a preset angle threshold, it is determined that the wearable device is at risk of excessive deformation.

5. The prompting method based on wearable devices as described in claim 4, characterized in that, The step of determining the first opening angle of the wearable device based on multiple acceleration data includes: Among the multiple acceleration data, the first acceleration data collected by the acceleration sensor located in the middle of the connection part, and the second acceleration data collected by the acceleration sensor located in a non-middle position of the connection part are determined; The first opening angle is obtained by calculating the angle based on the first acceleration data and the second acceleration data.

6. The prompting method based on wearable devices as described in claim 1, characterized in that, The step of determining whether the wearable device has a risk of excessive deformation based on the multimodal data further includes: Determine the external image acquisition device of the environment in which the wearable device is located, and collect multiple frames of image data from the wearable device; Image recognition is performed on each frame of image data to obtain the second opening angle of the wearable device; Detect whether the second opening angle is greater than a preset angle threshold; If the second opening angle is greater than a preset angle threshold, it is determined that the wearable device is at risk of excessive deformation.

7. The prompting method based on wearable devices as described in claim 1, characterized in that, Following the step of real-time monitoring of the multimodal data of the wearable device, the method further includes: Based on the multimodal data, the deformation direction corresponding to the deformation of the connection part of the wearable device is determined, wherein the deformation direction includes the tensile direction and the torsional direction; Based on the deformation direction and the multimodal data, it is determined whether the wearable device is at risk of excessive deformation.

8. The prompting method based on wearable devices as described in claim 7, characterized in that, The step of determining whether the wearable device has a risk of excessive deformation based on the deformation direction and the multimodal data includes: The first preset threshold and / or preset angle threshold corresponding to the wearable device are updated according to the deformation direction, and the step of determining whether the wearable device has excessive deformation risk based on the multimodal data is performed based on the updated first preset threshold and / or preset angle threshold.

9. A wearable device, characterized in that, The wearable device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the prompting method based on the wearable device as described in any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the prompting method based on a wearable device as described in any one of claims 1 to 8.

Images