Method, device and equipment for detecting health based on sound and vibration and storage medium

Abstract

The embodiment of the invention provides a method, device and equipment for detecting health based on sound and vibration and a storage medium. The method comprises: in response to an information acquisition request for a detection position, acquiring test information acquired by an electronic device from the detection position of an object, the test information including sound test information and tremor test information; in response to the fact that the test information meets the preset test requirement, detection information collected by the electronic equipment from the detection position of the object is obtained, and the detection information comprises sound detection information and tremor detection information; and determining a first health prediction result related to the detection position based on the detection information in response to the condition that the acquisition time of the detection information exceeds a predetermined threshold value and the detection information meets a predetermined detection requirement. Therefore, the accuracy of detecting the object data by the electronic equipment can be improved.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly to a method, apparatus, device, and storage medium for detecting health based on sound and vibration. Background Art

[0002] Health monitoring equipment can help people monitor their health status. For example, the stethoscope has been an important screening and monitoring tool for many diseases such as cardiopulmonary diseases for many years, and there are also many electronic stethoscopes that can assist in recording and analyzing auscultation signals. However, existing electronic stethoscopes are expensive and have barriers to use. They can only be used in medical institutions or by medical professionals and cannot be used in a wider range of disease screening scenarios. In addition, the scope of use of electronic stethoscopes is highly limited. Summary of the invention

[0003] In the first aspect of the present disclosure, a method for detecting health based on sound and vibration is provided. The method includes: in response to an information acquisition request for a detection position, acquiring test information collected by an electronic device from the detection position of an object, the test information including sound test information and tremor test information; in response to the test information meeting a predetermined test requirement, acquiring detection information collected by the electronic device from the detection position of the object, the detection information including sound detection information and tremor detection information; and in response to the acquisition time of the detection information exceeding a predetermined threshold and the detection information meeting a predetermined detection requirement, determining a first health prediction result related to the detection position based on the detection information.

[0004] In the second aspect of the present disclosure, a device for detecting health based on sound and vibration is provided. The device includes: a test information acquisition unit, configured to obtain test information collected by an electronic device from the detection position of the object in response to an information acquisition request for the detection position, the test information including sound test information and tremor test information; a detection information acquisition unit, configured to obtain detection information collected by the electronic device from the detection position of the object in response to the test information meeting a predetermined test requirement, the detection information including sound detection information and tremor detection information; a first health prediction result determination unit, configured to determine a first health prediction result related to the detection position based on the detection information in response to the acquisition time of the detection information exceeding a predetermined threshold and the detection information meeting a predetermined detection requirement.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of the present disclosure, there is provided a computer program product, which is tangibly stored in a computer storage medium and comprises computer executable instructions, which when executed by a device cause the device to perform the method of the first aspect.

[0008] It should be understood that the contents described in this content section are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram of an application scenario according to some embodiments of the present disclosure is shown;

[0011] Figure 2 A schematic block diagram of a method for detecting health based on sound and vibration according to some embodiments of the present disclosure is shown;

[0012] Figure 3 A flowchart of a method for detecting health based on sound and vibration according to some embodiments of the present disclosure is shown;

[0013] Figure 4 shows an overall schematic diagram of a device for detecting health based on sound and vibration according to an embodiment of the present disclosure; and

[0014] Figure 5 A block diagram of an electronic device is shown in which one or more embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION

[0015] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0016] It should be noted that the titles of any sections / subsections provided herein are not restrictive. Various embodiments are described throughout this article, and any type of embodiment may be included under any section / subsection. In addition, the embodiments described in any section / subsection may be combined in any manner with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0017] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below. The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may be included below.

[0018] As used herein, the term "model" can learn the association between the corresponding input and output from the training data, so that after the training is completed, the corresponding output can be generated for a given input. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multiple layers of processing units. In this article, "model" may also be referred to as "machine learning model", "machine learning network" or "network", which are used interchangeably in this article. A model may also include different types of processing units or networks.

[0019] As used herein, a "unit", "operating unit" or "subunit" may consist of a machine learning model or network of any suitable structure. As used herein, a group of elements or similar expressions may include one or more such elements. For example, a "group of convolutional units" may include one or more convolutional units.

[0020] As briefly mentioned above, traditional electronic stethoscopes have the disadvantages of high cost and limited use range. In addition, since the microphone units integrated in electronic devices such as smartphones can collect sound wave signals, the collection of body data using electronic devices has attracted more and more attention.

[0021] In the current solution of using electronic devices as electronic stethoscopes, since sound wave signals are only collected through microphone units, the accuracy of the information obtained is low.

[0022] The embodiments of the present disclosure propose a scheme for detecting health based on sound and vibration. According to various embodiments of the present disclosure, by bringing an electronic device close to or in contact with a detection position, the electronic device can obtain test information of the detection position, and the test information includes sound test information and tremor test information. In response to the test information meeting the predetermined test requirements, the detection information is collected from the detection position. The detection information includes sound detection information and tremor detection information. The first health prediction result related to the detection position is determined according to the detection information. In this way, on the basis of the microphone unit collecting the sound wave signal, the electronic device can collect the tremor signal of the detection position based on the inertial measurement unit, so as to collaboratively determine the condition of the detection position based on the sound wave signal and the tremor signal. Thereby improving the accuracy of the first health prediction result related to the detection position. In addition, before collecting the detection information, the electronic device will also test the signal quality of the detection position, thereby improving the accuracy of the electronic device in obtaining the detection information. By using the inertial measurement unit to collect the tremor information, the mechanical vibration signal and the low-frequency (for example, less than 20HZ) sound signal of the target part of the object can be more accurately identified. Thereby, the accuracy of information recognition can be further improved.

[0023] Figure 1 A schematic diagram of an application scenario 100 according to some embodiments of the present disclosure is shown. Figure 1 As shown, the application scenario 100 includes an object 110 and an electronic device 120 coupled to the detection point of the object 110. The object 110 may be a living being, such as a person, an animal, etc. In some other embodiments, the object 110 may also be a device suitable for emitting sound and / or vibration during operation, such as an engine, a reducer, etc. The electronic device 120 includes a microphone unit suitable for detecting sound and an inertial measurement unit suitable for detecting mechanical vibration. The electronic device 120 is arranged at the detection position of the object 110, so that the electronic device 120 can collect the sound signal at the detection position of the object 110 through the microphone unit, and the inertial measurement unit can collect the vibration signal at the detection position of the object 110. In some embodiments, the inertial measurement unit may include an accelerometer, a gyroscope sensor, etc.

[0024] It should be understood that the electronic device 120 is arranged at the detection position of the object 110, including the electronic device 120 being close to the detection position of the object 110 or the electronic device 120 being in contact with the detection position of the object 110. For example, in some embodiments, the electronic device 120 can be attached to the surface of the object 110, for example, the electronic device 120 can be attached to the skin of a living being, and through the contact between the electronic device 120 and the skin, the electronic device 120 can more accurately collect the tremor information of the skin. In some other embodiments, the electronic device 120 can also maintain a predetermined distance from the surface of the object 110, for example, there can be clothing, hair, etc. between the electronic device 120 and the skin of the living being.

[0025] The electronic device 120 can be any type of mobile terminal, fixed terminal or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication systems (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio broadcast receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the electronic device 120 can also support any type of interface for the user (such as "wearable" circuits, etc.).

[0026] In some embodiments, the electronic device 120 may process the collected sound signal and tremor signal based on its own computing power to output a prediction result. For example, the electronic device 120 may process the sound signal and tremor signal of a living being to determine the health status of the living being.

[0027] In some embodiments, the application scenario 100 further includes a server 130, and the electronic device 120 can send the acquired sound signal and tremor signal to the server 130, and the server 130 processes the above signals, and the server 130 can return the processing results of the sound signal and tremor signal to the electronic device 120. The electronic device 120 can also display the processing results to the user.

[0028] The server 130 may be various types of computing systems / servers 130 that can provide computing capabilities, including but not limited to mainframes, edge computing nodes, computing devices in a cloud environment, and the like.

[0029] Figure 2 FIG. 2 shows a schematic block diagram 200 of a method for detecting health based on sound and vibration according to some embodiments of the present disclosure. Figure 2As shown, if the user uses the electronic device 120 to obtain the first health prediction result of the detection location, in box 210, the user first needs to open the application in the electronic device 120 and select the function to be used in the application. In box 220, the user places the electronic device 120 at the detection location to be detected according to the prompts in the application. In box 230, the electronic device 120 collects a test signal at the detection location, and if the collected test signal meets the predetermined requirements, it starts to collect the detection signal. In box 240, during the period when the electronic device 120 collects the detection signal, the microphone unit of the electronic device 120 collects the sound detection signal. In box 250, the inertial measurement unit collects the tremor detection signal. In box 241, the sound detection signal and the tremor detection signal are denoised and preprocessed. In box 251, each pair of tremor detection signals is denoised and preprocessed. In box 260, the sound detection signal and the tremor detection signal are multimodal signal processed, and in box 270, the first health prediction result for the detection location is output. The following will specifically describe how the sound and vibration based health detection method of the present disclosure processes the detected position of the object 110 .

[0030] Figure 3 FIG. 3 is a flow chart showing a method 300 for detecting health based on sound and vibration according to some embodiments of the present disclosure. Figure 3 As shown, method 300 can be implemented by server 130 in the application scenario, and the test information and / or detection information (to be described in detail below) collected by electronic device 120 can be sent to server 130 via a data network, and server 130 processes the test information and / or detection information. The following mainly describes the execution of method 300 by server 130. In addition, in some cases, for example, when the computing power of electronic device 120 allows, method 300 can also be implemented by electronic device 120 in application scenario 100, and electronic device 120 can process test information and / or detection information while collecting test information and / or detection information, and the present disclosure will not go into details. It should be understood that method 300 may also include additional boxes not shown and / or may omit a certain (or some) box shown, and the scope of the present disclosure is not limited in this respect.

[0031] In block 310 , the server 130 obtains test information collected by the electronic device 120 from the detection position of the object 110 in response to the information acquisition request for the detection position, where the test information includes sound test information and tremor test information.

[0032] In some embodiments, while a user is sending an information acquisition request for a detection position to the electronic device 120, the electronic device 120 may present introductory information through a display module (e.g., a display or a speaker) based on the user's information acquisition request, thereby assisting the user in moving the electronic device 120 to a detection position close to or proximate to the object 110.

[0033] In some embodiments, the electronic device 120 displays introductory information related to one or more detection positions in the target part of the object 110 in response to the information acquisition request for the target part, and the introductory information includes at least one of image information, sound information and text information.

[0034] For example, if the user needs to obtain information about the lungs of the object 110, the user can send an information acquisition request for the target part of the lungs to the electronic device 120, and the electronic device 120 displays multiple position information related to the target part based on the information acquisition request for the target part. For example, the electronic device 120 can display the introductory information of the detection position related to the lungs (for example, the left upper lung, left lower lung, right upper lung, right lower lung of the front chest of the object, and the left lower lung, right lower lung of the back of the object, etc.) to the user through the output module of the display screen or the speaker. In this way, the user can move the electronic device 120 to the detection position according to the prompt of the electronic device 120. In some other examples, for the inspection of the target part to the heart, the electronic device can prompt the user to the detection position related to the heart (for example, the first auscultation area of ​​the aortic valve, the second auscultation area of ​​the aortic valve, the mitral valve area, the tricuspid valve area, and the pulmonary valve area). Thus, it is convenient for the user to accurately move the electronic device to the detection position related to the heart. By displaying the detection position related to the target part by the electronic device 120, the accuracy of arranging the electronic device 120 at the detection position related to the target part can be improved, thereby improving the detection efficiency and the accuracy of outputting the second health prediction result for the target part.

[0035] In some embodiments, the introduction information may include at least one of a voice introduction, a text introduction, a picture introduction, and a video introduction.

[0036] If the electronic device 120 moves to the detection position of the object 110, based on the information acquisition request for the detection position, the signal of the detection position is collected to determine whether the signal of the detection position meets the detection requirements. For example, the microphone unit of the electronic device 120 can collect sound test information of the detection position, and the inertial measurement unit of the electronic device 120 can collect vibration test information of the detection position.

[0037] In some embodiments, if the electronic device 120 moves to the detection position, the user can send an information acquisition request for the detection position to the electronic device 120 through the input module of the electronic device 120 (for example, a button, a touch screen, etc.). After receiving the information acquisition request for the detection position, the electronic device 120 collects a test signal of the detection position. In some embodiments, the electronic device 120 can also determine whether the electronic device 120 has moved to the detection position through its own sensor and other modules. For example, the electronic device 120 can determine whether the predetermined position of the electronic device 120 is in contact with the object 110 through its own distance sensor, etc., and then start collecting the test signal of the detection position after the electronic device 120 contacts the detection position of the object 110.

[0038] In some embodiments, the user may place the electronic device 120 at the detection position of the object 110, and issue an information acquisition request to the electronic device 120 through an input module (e.g., a keyboard, a touch screen, etc.) of the electronic device 120. After obtaining the information acquisition request for the detection position, the electronic device 120 may collect test information from the detection position.

[0039] In block 320 , in response to the test information satisfying a predetermined test requirement, the server 130 acquires detection information collected by the electronic device 120 from a detection position of the object 110 , the detection information including sound detection information and tremor detection information.

[0040] In some embodiments, the server 130 detects the collected sound test information and tremor test information. If the sound test information and the tremor test information both meet the predetermined conditions, it means that the signal quality of the detection position meets the requirements. Therefore, the server 130 can control the electronic device 120 to continue to collect detection information at the detection position.

[0041] In some embodiments, the server 130 can determine whether the test information meets the predetermined conditions based on the signal-to-noise ratio of the test information. For example, the server 130 can determine that the sound test information meets the predetermined requirements based on that the signal-to-noise ratio of the sound test information exceeds the first predetermined threshold within the first predetermined time. For another example, the server 130 can also determine that the tremor test information meets the predetermined requirements based on that the signal-to-noise ratio of the tremor test information exceeds the second predetermined threshold within the second predetermined time. It should be understood that in some other embodiments, the server 130 can also determine whether the test information meets the requirements based on other appropriate physical parameters, which will not be repeated in this disclosure.

[0042] In some embodiments, the server 130 sends a prompt message in response to the test information meeting the predetermined test requirements, and the prompt message includes at least one of visual prompt information, auditory prompt information, and tactile prompt information. After receiving the prompt message, the electronic device 120 sends a corresponding visual prompt, such as a flashing indicator light, a display screen showing relevant content, etc., an auditory prompt, such as a speaker or buzzer emitting a prompt sound, a tactile prompt, a vibration motor vibrating, etc. Through the prompt message, the electronic device 120 can inform the user of the current detection position of the electronic device 120.

[0043] In some other embodiments, if the test information deviates from the predetermined detection requirements, the electronic device 120 may issue an alarm message to prompt the user that the current detection position is abnormal. For example, if the signal-to-noise ratio of the sound test information acquired by the electronic device 120 is less than the first predetermined threshold value within a first predetermined time (e.g., 5s, 10s, 15s, 20s, etc.), or the duration of the sound test information acquired by the electronic device 120 is less than the first predetermined time, the electronic device 120 may send an alarm message to the user. For another example, if the signal-to-noise ratio of the tremor test information collected by the electronic device 120 is less than the second predetermined threshold value within a second predetermined time (e.g., 15s, 10s, 15s, 20s, etc.), or the duration of the tremor test information collected by the electronic device 120 is less than the second predetermined time, the electronic device 120 may also send an alarm message to the user.

[0044] The warning information includes at least one of auditory information, visual information, and tactile information. For example, the electronic device 120 can warn the user based on the warning information by sound, flashing lights, text or video reminders, vibration, etc., to prompt the user to correct the detection position or the wearing method of the electronic device 120.

[0045] In block 330 , in response to the acquisition time of the detection information exceeding a predetermined threshold and the detection information satisfying a predetermined detection requirement, the server 130 determines a first health prediction result associated with the detection location based on the detection information.

[0046] Similar to the above determination of whether the test information meets the predetermined condition, the server 130 can determine whether the detection information is available by determining whether the detection information meets the predetermined requirement. Specifically, in some embodiments, the server 130 can confirm that the sound detection information meets the predetermined requirement based on that the signal-to-noise ratio of the sound detection information exceeds the third predetermined threshold within a third predetermined time (e.g., 5s, 10s, 15s, 20s, etc.). In some other embodiments, the server 130 can also confirm that the tremor detection information meets the predetermined requirement based on that the signal-to-noise ratio of the tremor detection information exceeds the fourth predetermined threshold within a fourth predetermined time (e.g., 5s, 10s, 15s, 20s, etc.).

[0047] In some embodiments, for different target parts, the time for the electronic device 120 to collect test information and detection information is also different. For example, if the target part is the heart, the time for the electronic device 120 to collect sound test information, tremor test information, sound detection information, and tremor detection information can be 10 seconds. That is, the first predetermined time, the second predetermined time, the third predetermined time, and the fourth predetermined time can be arbitrarily selected as 10 seconds. If the target part is the lungs, the time for the electronic device 120 to collect sound test information, tremor test information, sound detection information, and tremor detection information can be 20 seconds. That is, the first predetermined time, the second predetermined time, the third predetermined time, and the fourth predetermined time can be arbitrarily selected as 20 seconds.

[0048] During use, in order to ensure the accuracy of signal (test signal and / or detection signal) collection, the electronic device 120 can also prompt the user to remain still, and during the test or detection, if the user's movement amplitude is too large (that is, the inertial measurement unit of the electronic device 120 detects that the vibration exceeds the predetermined range), the electronic device 120 will also remind the user to remain still through voice, video, etc.

[0049] If the detection information collected by the electronic device 120 does not meet the predetermined requirements, for example, the signal-to-noise ratio of the sound detection information is less than a predetermined threshold, or the signal-to-noise ratio of the tremor detection information is less than a predetermined threshold, the server 130 can provide warning information to the electronic device 120, and the electronic device 120 warns the user based on the warning information by sound, light, text or video reminder, vibration, etc., to prompt the user to correct the detection position or the wearing method of the electronic device 120.

[0050] If the server 130 determines that both the sound detection information and the tremor detection information meet the predetermined requirements, the server 130 may output a first health prediction result related to the detection location based on the sound detection information and the tremor detection information.

[0051] In some embodiments, if the electronic device 120 is arranged in the aortic valve first auscultation area of ​​the subject's chest, the server can determine that a murmur occurs in the aortic valve second auscultation area based on detection information of the aortic valve second auscultation area.

[0052] In some embodiments, in response to obtaining the detection information, the server 130 performs an adjustment operation on the detection information to obtain adjusted detection information, and the adjustment operation includes at least one of the following: denoising the detection information based on a filter, and aligning the detection information; and determining whether the adjusted detection information meets predetermined detection requirements.

[0053] In some embodiments, after the electronic device 120 collects the test information or the detection information, the test information or the detection information can be uploaded to the server 130, and the server 130 uses a Butterworth filter to reduce noise for the sound information (sound test information or sound detection information) and the tremor information (tremor test information or tremor detection information). Subsequently, an automatic alignment algorithm is used to align multiple cycles (e.g., multiple respiratory cycles or multiple cardiac cycles) to finally obtain a median signal of one cycle.

[0054] In some embodiments, the server 130 determines the first health prediction result related to the detection location based on the detection information further includes: the server 130 extracts feature representation information of the detection information. The server 130 applies a cross-attention mechanism and a bimodal fusion mechanism to the feature representation information to determine the first health prediction result related to the detection location.

[0055] In some embodiments, the server 130 may use a convolutional neural network to process and extract features from the sound detection information and the tremor detection information, and perform cross-attention and bimodal fusion to form a fusion layer, and finally output a first health prediction result related to the detection location.

[0056] In some embodiments, the method 300 further includes the server 110 acquiring first health prediction results of multiple detection positions of the object 110. The server 110 determines second health prediction results of target parts corresponding to the multiple detection positions based on the first health prediction results of the multiple detection positions.

[0057] In some embodiments, the user may send an information acquisition request for the target part to the electronic device 120, and the electronic device 120 determines multiple detection positions based on the information acquisition request for the target part. Thus, the user may make information acquisition requests for multiple detection positions to the electronic device 120 respectively, and the server 110 may determine multiple first health prediction results related to the multiple detection positions in response to the information acquisition requests for the multiple detection positions. Then the server 110 determines the second health prediction result based on the multiple first health prediction results.

[0058] For example, if the electronic device 120 detects the target part of the heart of the subject under test, the server can determine that the heart function of the subject under test is normal based on the first health prediction result of multiple detection positions related to the heart (for example, the first auscultation area of ​​the aortic valve, the second auscultation area of ​​the aortic valve, the mitral valve area, the tricuspid valve area, the pulmonary valve area, etc.).

[0059] In some embodiments, the server 130 may also determine the second health prediction result of the target part based on the detection information of multiple detection positions related to the target part. For example, the server 130 may directly filter and reduce noise of the detection information of multiple detection positions, and then align multiple cycles of the multiple detection information using an automatic alignment algorithm to obtain the median signal of one cycle of the target part. Finally, a convolutional neural network is used to process and extract features, and a fusion layer is formed after cross-attention and bimodal fusion, and finally the second health prediction result related to the target part is output.

[0060] By using a microphone unit to obtain a sound detection signal and an inertial measurement unit to obtain a vibration detection signal, the accuracy of the electronic device in obtaining detection information can be improved. In particular, by using an inertial measurement unit to collect vibration information, the mechanical vibration signal and low-frequency sound signal of the target part of the object can be more accurately identified. This can further improve the accuracy of information recognition.

[0061] Figure 4 FIG. 4 is a schematic diagram showing an overall diagram of a device 400 for detecting health based on sound and vibration according to an embodiment of the present disclosure. Figure 4 As shown, the apparatus 400 includes a test information acquisition unit 410, which is configured to obtain test information collected by the electronic device 120 from the detection position of the object 110 in response to an information acquisition request for the detection position, wherein the test information includes sound test information and tremor test information.

[0062] The detection information acquisition unit 420 is configured to acquire detection information collected by the electronic device 120 from the detection position of the object 110 in response to the test information satisfying the predetermined test requirement, where the detection information includes sound detection information and tremor detection information.

[0063] The first health prediction result determining unit 430 is configured to determine a first health prediction result related to the detection location based on the detection information in response to the acquisition time of the detection information exceeding a predetermined threshold and the detection information meeting a predetermined detection requirement.

[0064] In some embodiments, the apparatus 400 further includes a second health prediction result determination module configured to determine a second health prediction result of a target part corresponding to the plurality of detection positions based on the first health prediction results of each of the plurality of detection positions.

[0065] In some embodiments, the device 400 also includes an introductory information display module, which is configured to display introductory information related to one or more detection positions in the target part of the object 110 in response to an information acquisition request for the target part, and the introductory information includes at least one of image information, sound information and text information.

[0066] In some embodiments, the device 400 also includes a test information comparison module, which is configured to compare test information based on predetermined test requirements, and the test information meets the predetermined test requirements including: within a first predetermined time, the signal-to-noise ratio of the sound test information exceeds a first predetermined threshold; and within a second predetermined time, the signal-to-noise ratio of the tremor test information exceeds a second predetermined threshold.

[0067] In some embodiments, the device 400 also includes a detection information comparison module, which is configured to compare the detection information based on predetermined detection requirements, and the detection information meets the predetermined detection requirements including: within a third predetermined time, the signal-to-noise ratio of the sound detection information exceeds a third predetermined threshold; and within a fourth predetermined time, the signal-to-noise ratio of the tremor detection information exceeds a fourth predetermined threshold.

[0068] In some embodiments, the apparatus 400 further comprises an alarm information providing module configured to provide an alarm information in response to a deviation between the test information and a predetermined test requirement, or a deviation between the detection information and a predetermined detection requirement.

[0069] In some implementations, the first health prediction result determination module is further configured to extract feature representation information of the detection information; and apply a cross-attention mechanism and a bimodal fusion mechanism to the feature representation information to determine a first health prediction result related to the detection location.

[0070] In some embodiments, the device 400 also includes an adjustment module, which is configured to perform an adjustment operation on the detection information in response to acquiring the detection information to obtain adjusted detection information, and the adjustment operation includes at least one of the following: denoising the detection information based on a filter, and aligning the detection information; and determining whether the adjusted detection information meets the predetermined detection requirements.

[0071] In some embodiments, the device 400 further includes a prompt information module configured to send prompt information in response to the test information satisfying a predetermined test requirement, wherein the prompt information includes at least one of visual prompt information, auditory prompt information, and tactile prompt information.

[0072] Figure 5 1 is a block diagram of an electronic device 500 in which one or more embodiments of the present disclosure may be implemented. The electronic device 500 may be used to implement, for example, Figure 1 The server 130 shown. It should be understood that Figure 5 The electronic device 500 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein.

[0073] like Figure 5As shown, the electronic device 500 is in the form of a general electronic device. The components of the electronic device 500 may include, but are not limited to, one or more processors 510 or processing units, a memory 520, a storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. The processing unit may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 520. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 500.

[0074] The electronic device 500 typically includes a plurality of computer storage media. Such media may be any available media accessible to the electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 520 may be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 530 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data (e.g., training data for training) and may be accessed within the electronic device 500.

[0075] The electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 5 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 520 may include a computer program product having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0076] The communication unit 540 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 500 can be implemented in a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 500 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0077] The input device 550 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 560 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 500 may also communicate with one or more external devices (not shown) through the communication unit 540 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 500, or communicate with any device that allows the electronic device 500 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0078] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which one or more computer instructions are stored, wherein the one or more computer instructions are executed by a processor to implement the method described above.

[0079] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0080] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0081] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0082] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0083] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A method for detecting health based on sound and vibration, comprising: In response to an information acquisition request for a detection position, acquiring test information collected by the electronic device from the detection position of the object, the test information including sound test information and tremor test information; In response to the test information satisfying a predetermined test requirement, acquiring detection information collected by the electronic device from the detection position of the object, the detection information including sound detection information and tremor detection information; as well as In response to the acquisition time of the detection information exceeding a predetermined threshold and the detection information meeting a predetermined detection requirement, a first health prediction result related to the detection location is determined based on the detection information. 2 . The method according to claim 1 , wherein the sound test information and / or the sound detection information is collected by at least utilizing a microphone unit of the electronic device.

3. The method according to claim 1, wherein the tremor test information and / or the tremor detection information is collected by at least utilizing an inertial measurement unit of the electronic device.

4. The method according to claim 1, further comprising: Acquire a first health prediction result of a plurality of detection positions of the object; A second health prediction result of the target part corresponding to the plurality of detection positions is determined based on the first health prediction results of each of the plurality of detection positions.

5. The method according to claim 4, further comprising: In response to an information acquisition request for the target part, introduction information related to one or more detection positions in the target part of the object is displayed, and the introduction information includes at least one of image information, sound information and text information.

6. The method according to claim 1, wherein the test information satisfies the predetermined test requirements, comprising: Within a first predetermined time, the signal-to-noise ratio of the sound test information exceeds a first predetermined threshold; as well as Within a second predetermined time, the signal-to-noise ratio of the tremor test information exceeds a second predetermined threshold.

7. The method according to claim 1, wherein the detection information satisfies the predetermined detection requirements comprises: Within a third predetermined time, the signal-to-noise ratio of the sound detection information exceeds a third predetermined threshold; as well as Within a fourth predetermined time, a signal-to-noise ratio of the tremor detection information exceeds a fourth predetermined threshold.

8. The method according to claim 6 or 7, further comprising: In response to a deviation between the test information and the predetermined test requirement, or a deviation between the detection information and the predetermined detection requirement, an alarm message is provided.

9. The method according to claim 1, wherein determining a first health prediction result related to the detection location based on the detection information comprises: Extracting feature representation information of the detection information; as well as A cross-attention mechanism and a bimodal fusion mechanism are applied to the feature representation information to determine a first health prediction result associated with the detection location.

10. The method according to claim 1, further comprising: In response to acquiring the detection information, performing an adjustment operation on the detection information to obtain adjusted detection information, wherein the adjustment operation includes at least one of the following: performing noise reduction processing on the detection information based on a filter, and performing alignment processing on the detection information; as well as Determine whether the adjusted detection information meets the predetermined detection requirement.

11. The method according to claim 1, further comprising: In response to the test information satisfying the predetermined test requirement, prompt information is sent, where the prompt information includes at least one of visual prompt information, auditory prompt information, and tactile prompt information.

12. A device for detecting health based on sound and vibration, comprising: a test information acquisition unit configured to acquire test information collected by the electronic device from the detection position of the object in response to an information acquisition request for the detection position, wherein the test information includes sound test information and tremor test information; a detection information acquisition unit, configured to acquire detection information collected by the electronic device from the detection position of the object in response to the test information satisfying a predetermined test requirement, the detection information including sound detection information and tremor detection information; The first health prediction result determining unit is configured to determine a first health prediction result related to the detection location based on the detection information in response to the acquisition time of the detection information exceeding a predetermined threshold and the detection information meeting a predetermined detection requirement.

13. An electronic device comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 11 when executed by the at least one processing unit.

14. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 11.

15. A computer program product tangibly stored in a computer storage medium and comprising computer executable instructions which, when executed by a device, cause the device to perform the method according to any one of claims 1 to 11.

Images