Knee joint health status early screening system and method based on multi-channel acoustic information and improved SRU

Abstract

The invention belongs to the technical field of medical equipment and artificial intelligence, and provides a knee joint health state early screening system and method based on multichannel acoustic information and improved SRU, and the system comprises knee joint acoustic signal collection equipment, a signal preprocessing module, a feature extraction module and an improved SRU neural network model. The knee joint acoustic signal acquisition equipment is worn on a knee joint and is used for acquiring a knee joint acoustic signal; the signal preprocessing module is used for filtering the acquired knee joint acoustic signals; the feature extraction module is used for extracting a plurality of spectral features from the filtered knee joint acoustic signal and then constructing an acoustic signal feature matrix of the knee joint; the improved SRU neural network model is used for reconstructing an acoustic signal feature matrix of the knee joint and introducing positioning information so as to identify the health state of the knee joint. The device is used for prevention or early screening of knee osteoarthritis, and has the advantages of high precision, noninvasiveness, portability, intelligence and the like.

Description

Technical Field

[0001] The present invention belongs to the technical fields of medical devices and artificial intelligence, and relates to a knee joint health status detection system, and particularly to a knee joint health status early screening system and method based on multi-channel acoustic information and an improved SRU. Background Art

[0002] With the aggravation of population aging and the frequent occurrence of sports injuries, knee joint injuries have become common clinical problems. Common knee joint injuries include cartilage wear, meniscus injury, synovitis, etc. If these injuries are not diagnosed and treated in time, they may lead to serious functional disorders. However, traditional knee joint injury detection methods rely on imaging tools such as X-rays and magnetic resonance imaging (MRI), but these methods require expensive equipment and are difficult to achieve real-time monitoring, and are usually only used after the injury occurs.

[0003] Knee joint detection methods based on acoustic data have gradually become a promising research direction due to their non-invasive, low-cost and portability. However, in the prior art, there are still technical bottlenecks in accurately identifying the types and degrees of knee joint injuries through acoustic signals. The existing knee joint acoustic detection technologies mainly focus on using a single sensor to collect acoustic data, resulting in limited data dimensions, unable to fully capture the complex movements and physical changes within the knee joint, and difficult to accurately identify complex knee joint injuries. In addition, single-channel data is vulnerable to noise interference, and the accuracy of data processing is insufficient.

[0004] Therefore, how to use multi-channel acoustic data combined with advanced machine learning algorithms for knee joint health status recognition is the main technical problem studied in the present invention. Summary of the Invention

[0005] The purpose of the present invention aims to solve the above-mentioned technical problems existing in the prior art, and provides a knee joint health status early screening system and method based on multi-channel acoustic information and an improved SRU, which can not only process high-dimensional time series data, but also achieve efficient and accurate recognition of the knee joint health status, and can also locate the injury position.

[0006] The inventive concept of the present invention is: to establish a portable and non-invasive knee joint acoustic signal acquisition device to collect input signals for algorithm analysis. Due to the complex composition of the bones and ligaments of the knee joint, in the form of an array, microphones are placed at designated positions near the knee joint to achieve multi-channel and multi-position sound signal acquisition. Then use the beamforming algorithm to enhance the acoustic signal in a specific direction by controlling the phases of multiple sensor arrays. Then classify the acoustic signals generated by the friction between the bearing surfaces during knee joint movement through the SRU neural network.

[0007] Based on the above invention concept, the present invention provides a knee joint health status early screening system based on multi-channel acoustic information and an improved SRU, which includes:

[0008] A knee joint acoustic signal acquisition device, which is used to be worn on the knee joint to acquire knee joint acoustic signals; the knee joint acoustic signal acquisition device includes a wearable carrier, and a sensor array composed of a plurality of acoustic sensor units installed on the wearable carrier;

[0009] A signal preprocessing module, which is used to perform filtering processing on the acquired knee joint acoustic signals;

[0010] A feature extraction module, which is used to extract a plurality of spectral features from the filtered knee joint acoustic signals, and then construct an acoustic signal feature matrix of the knee joint;

[0011] An improved SRU neural network model, which is used to reconstruct the acoustic signal feature matrix of the knee joint, introduce positioning information, and then identify the health status of the knee joint; the improved SRU neural network model includes a delay summation module, a time difference positioning module, and an SRU neural network; the delay summation module is used to perform time delay compensation and weighted summation on the signals in the acoustic signal feature matrix of the knee joint, and perform the first reconstruction on the feature matrix; the time difference positioning module is used to determine the sound source position according to the time difference of the signals arriving at multiple acoustic sensor units, obtain the positioning information, and add the positioning information to the feature matrix after the first reconstruction, and perform the second reconstruction on the feature matrix; the SRU neural network is used to identify the health status of the knee joint according to the feature matrix after the second reconstruction, and output the identification result and the positioning information.

[0012] In an implementable manner, the acoustic sensor units in the sensor array are evenly distributed around the knee joint, and are used to acquire acoustic signals of the knee joint in different activity states. In a preferred implementation manner, a plurality of acoustic sensor units provided in the knee joint acoustic signal acquisition device are respectively fixed on the upper and lower sides, left and right sides of the patella, on both sides of the tibiofemoral joint, on the front side of the tibial head, on the left and right sides of the tibia and at a position 5-10 cm away from the tibiofemoral joint, and / or on the left and right sides of the femur and at a position 10-15 cm away from the tibiofemoral joint.

[0013] Further, the knee joint acoustic signal acquisition device includes a first acoustic sensor unit and a second acoustic sensor unit fixed on the upper and lower sides of the patella, a third acoustic sensor unit and a fourth acoustic sensor unit fixed on the left and right sides of the patella, a fifth acoustic sensor unit and a sixth acoustic sensor unit fixed on both sides of the tibiofemoral joint, a seventh acoustic sensor unit fixed on the front side of the tibial head, an eighth acoustic sensor unit and a ninth acoustic sensor unit fixed on the left and right sides of the tibia at a position 5-10 cm away from the tibiofemoral joint, and / or a tenth acoustic sensor unit and an eleventh acoustic sensor unit fixed on the left and right sides of the femur at a position 10-15 cm away from the tibiofemoral joint; the distance between the first acoustic sensor unit and the second acoustic sensor unit, the distance between the third acoustic sensor unit and the fourth acoustic sensor unit, and the distance between the fifth acoustic sensor unit and the sixth acoustic sensor unit are all 6-8 cm; the distance between the seventh acoustic sensor unit and the second acoustic sensor unit is 6-8 cm; the distance between the eighth acoustic sensor unit and the ninth acoustic sensor unit and the distance between the tenth acoustic sensor unit and the eleventh acoustic sensor unit are 10-15 cm.

[0014] In an implementable manner, each acoustic sensor unit can capture the acoustic changes inside the knee joint at different angles, including the acoustic characteristics in the bending, stretching, and loading states. The acoustic sensor unit includes a flexible circuit board and two microphones mounted on the flexible circuit board. The flexible circuit board has good bending performance, facilitating the subject to move according to the given exercise plan. Both microphones on the flexible circuit board are connected to the main control unit; the main control unit uses programmable logic controllers such as PAL, GAL, and FPGA. The main control unit is used to control the data acquisition of each microphone according to the commands from the server side and send the acquired acoustic signals to the server side.

[0015] Further, the knee joint acoustic signal acquisition device further includes sensors (such as inertial sensors, angle sensors, etc.) for obtaining the motion physical quantities to assist subsequent analysis, facilitating the resolution and extraction of the acoustic signals during exercise when processing data subsequently.

[0016] In an implementable manner, the signal preprocessing module, the feature extraction module, and the improved SRU neural network model are all integrated on the server side (here referring to a PC) and run on the server side.

[0017] The signal preprocessing module is used to filter the collected knee joint acoustic signals to eliminate external environmental noise and other interferences, and obtain clean and reliable knee joint acoustic signal data. In a specific implementation manner, the signal preprocessing module can use a band-pass filter to filter the collected knee joint acoustic signals. Then, the filtered knee joint acoustic signals are divided according to a set periodic window.

[0018] In an implementable manner, the feature extraction module is used to extract several key spectral features of the acoustic signal, including time-domain features or / and frequency-domain features; the time-domain features include at least one of time-domain amplitude, short-time energy, zero-crossing rate, etc.; the frequency-domain features include at least one of spectral centroid, spectral spread, spectral peak, Mel-frequency cepstral coefficients, etc.

[0019] Then, the feature extraction module averages the windowed features within each period to create a feature matrix with m rows of moving periods × n columns of features for each microphone (m represents the number of windows, and n represents the number of features), and connects the feature matrices of all microphones to obtain the acoustic signal feature matrix of the knee joint.

[0020] In an implementable manner, when performing sound source localization in human bones, beamforming can enhance the signals from specific parts while reducing other noise interferences. The present invention combines the Delay and Sum (DS) algorithm with the Time Difference of Arrival (TDOA) to construct an end-to-end improved SRU neural network model based on DS-TDOASRU.

[0021] In the improved SRU neural network model, the delay summation module is based on the Delay and Sum (DS) algorithm, which is a beamforming technology with good anti-incoherent noise performance. By precisely compensating the time delays of the signals of the acoustic sensor units at different positions in the acoustic signal feature matrix of the knee joint, the signals from a specific direction are aligned in time, and then the corresponding feature signals in each acoustic sensor unit after time delay compensation are weighted and summed to reconstruct the feature matrix with the obtained signals, completing the first reconstruction of the feature matrix, thereby enhancing the target signal and suppressing the interference signal.

[0022] In the improved SRU neural network model, the time difference of arrival (TDOA) positioning module is based on the TDOA algorithm, which is a method for determining the sound source position by using the time difference of the signal arriving at multiple receivers. It can achieve high-precision sound source positioning through simple hardware configuration and is applicable to complex environments. The TDOA algorithm can calculate the positions of more than one sound source, that is, obtain the positioning information of more than one sound source, by measuring the time difference of the sound wave arriving at different acoustic sensor units from the sound source and combining the position information of each acoustic sensor unit; the positioning information is added as a new feature to the feature matrix after the first reconstruction to complete the second reconstruction of the feature matrix.

[0023] The SRU (Simple Recurrent Unit) neural network is an efficient variant of the recurrent neural network and has the advantage of processing long-time series data. In the present invention, the SRU neural network takes the feature matrix after the second reconstruction as the input data and the knee joint health status recognition result and the positioning information as the output. By training a large amount of knee joint acoustic data, the SRU neural network can classify and identify different types of knee joint injuries. Compared with the traditional LSTM (Long Short-Term Memory) and GRU (Gated Recurrent Unit), the SRU has higher computational efficiency and can process acoustic signals faster and identify the patterns of knee joint injuries.

[0024] The present invention also provides a method for early screening of knee joint health status based on multi-channel acoustic information and improved SRU, which is characterized in that the above knee joint health status early screening system is used according to the following steps:

[0025] S1 Wear the knee joint acoustic signal acquisition device on the knee joint of the subject; and collect the knee joint acoustic signal through a plurality of acoustic sensor units installed on the wearable carrier.

[0026] S2 Filter the collected knee joint acoustic signal.

[0027] S3 Extract a plurality of spectral features from the filtered knee joint acoustic signal, and then construct an acoustic signal feature matrix of the knee joint.

[0028] S4 Reconstruct the constructed acoustic signal feature matrix of the knee joint through the delay summation module and the TDOA positioning module, and based on the reconstructed feature matrix, complete the recognition of the knee joint health status of the subject through the SRU neural network to obtain the recognition result and the positioning information.

[0029] The above recognition results include two levels of output: the first-level output types include normal knee joint, early knee osteoarthritis, and severe knee osteoarthritis; the second-level output types include cartilage wear, cruciate ligament injury, synovitis, loose bodies in the knee joint, etc. Only one type is output at the first level, and at least one of them can be included in the second-level output; because in severe patients, there may be multiple knee joint injuries simultaneously in the later stage. When the first-level output is a normal knee joint, there is no output at the second level.

[0030] Therefore, through the knee joint health status early screening system and method based on multi-channel acoustic information and improved SRU provided by the present invention, normal knee joints, early-stage knee joints, and severe knee joints are distinguished, and at the same time, normal knee joints, cartilage wear, cruciate ligament injury, synovitis, loose bodies in the knee joint, etc. are also identified. The acoustic data of knee joint movement of the masses is collected and tested on a large scale within the community to achieve the purpose of early screening, early intervention, and early treatment of knee joint diseases.

[0031] Compared with the prior art, the knee joint health status early screening system and method based on multi-channel acoustic information and improved SRU provided by the present invention have the following beneficial effects:

[0032] (1) In the present invention, acoustic sensors in the form of an array are fixed at points around the knee joint for multi-channel and multi-position acoustic signals; then the improved SRU neural network automatically analyzes the acoustic signals of the knee joint and accurately identifies different types of knee joint health status; and the location of knee joint injuries can be located; this intelligent diagnosis reduces the workload of doctors and can provide effective data support for personalized diagnosis and treatment suggestions for patients;

[0033] (2) The present invention uses a common microphone array to collect acoustic signals from the knee joint for the prevention or early screening of knee osteoarthritis, and has the advantages of high precision, non-invasiveness, portability, and intelligence. It can be used as a non-invasive and wearable device at locations outside professional environments such as hospitals (such as communities) to achieve the purpose of large-scale screening and prevention of knee osteoarthritis;

[0034] (3) The present invention integrates the DS and TDOA positioning algorithms into the SRU neural network to enhance the collected acoustic signals and determine the sound source location, thereby realizing the effective identification of the knee joint health status; the system of the present invention is an efficient, non-invasive, and real-time knee joint injury detection system, which can diagnose in the early stage of knee joint injury and has wide clinical application value;

[0035] (4) The present invention can detect knee joint injuries at the early stage through changes in acoustic signals, which helps patients take rehabilitation measures as early as possible to avoid the deterioration of the condition. In addition, personalized health advice and exercise guidance can be provided based on the user's historical data to help the user better maintain knee joint health; BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 FIG. is a schematic structural diagram of a knee joint health status early screening system based on multi-channel acoustic information and an improved SRU provided in Embodiment 1 of the present invention;

[0037] Figure 2 FIG. is a schematic diagram of a knee joint acoustic signal acquisition device and its assembly structure provided in Embodiment 1 of the present invention; wherein, (a) corresponds to the structural diagram of the knee joint acoustic signal acquisition device in the working state; (b) is the unfolded diagram of the wearable carrier; (c) corresponds to the schematic diagram of the acoustic sensor unit; (d) corresponds to the principle framework diagram of the knee joint acoustic signal acquisition device; in the figure, 1 - wearable carrier; 11 - defined area of the wearable carrier; 2 - sensor array; 21 - first acoustic sensor unit; 22 - second acoustic sensor unit; 23 - third acoustic sensor unit; 24 - fourth acoustic sensor unit; 25 - fifth acoustic sensor unit; 26 - sixth acoustic sensor unit; 27 - seventh acoustic sensor unit; 28 - eighth acoustic sensor unit; 29 - ninth acoustic sensor unit; 210 - tenth acoustic sensor unit; 211 - eleventh acoustic sensor unit; 3 - first inertial sensor; 4 - second inertial sensor;

[0038] Figure 3 FIG. is a schematic flowchart of a knee joint health status early screening method based on multi-channel acoustic information and an improved SRU provided in Embodiment 1 of the present invention;

[0039] Figure 4 FIG. is a schematic diagram of the display of acoustic signals in different channels.

[0040] Figure 5 FIG. shows the signals of two channels on one acoustic sensor unit. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope protected by the present invention.

[0042] Embodiment 1

[0043] The present embodiment provides a knee joint health status early screening system based on multi-channel acoustic information and improved SRU, which includes a knee joint acoustic signal acquisition device, a signal preprocessing module, a feature extraction module and an improved SRU neural network model. The signal preprocessing module, the feature extraction module and the improved SRU neural network model are all integrated in the server side (here refers to the PC).

[0044] (1) Knee joint acoustic signal acquisition equipment

[0045] The knee joint acoustic signal acquisition device is used to be worn on the knee joint to collect the knee joint acoustic signal. The knee joint acoustic signal acquisition device includes a wearable carrier 1 and a sensor array 2 composed of 11 acoustic sensor units installed on the wearable carrier. The wearable carrier 1 is set on the subject's knee joint.

[0046] Due to the complex structure of the bones and ligaments of the knee joint, in this embodiment, the knee joint acoustic signal acquisition device adopts an array form, and 11 acoustic sensor units are placed at designated positions near the knee joint to achieve multi-channel, multi-position sound signal acquisition. Compared with ultrasonic devices, the array microphone form adopted is small in size and the use conditions and placement methods are not strict, which is convenient for use in complex environments. In addition to helping to increase the amount of training data for the model, the multi-channel data collected by the array microphone also helps to mine spatial information, achieve further positioning of knee joint lesions, and thereby achieve classification detection of various knee joint injuries (including cartilage wear, cruciate ligament injury, synovitis, loose bodies of the knee joint, etc.). The sensor array uses a structure similar to a knee pad / elastic band as a wearable carrier to achieve the initial fixation of the array, and each microphone is fixed to a designated position by double-sided tape.

[0047] The acoustic sensor units in the sensor array are evenly distributed around the knee joint and are used to collect acoustic signals of the knee joint in different activity states. The 11 acoustic sensor units are fixed on the upper and lower sides of the patella, the left and right sides, the sides of the tibiofemoral joint, the front side of the tibia head, the left and right sides of the tibia and 5 to 10 cm away from the tibiofemoral joint, and the left and right sides of the femur and 10 to 15 cm away from the tibiofemoral joint. Specifically, Figure 2As shown in (a), the knee joint acoustic signal acquisition device includes a first acoustic sensor unit 21 and a second acoustic sensor unit 22 fixed on the upper and lower sides of the patella, a third acoustic sensor unit 23 and a fourth acoustic sensor unit 24 fixed on the left and right sides of the patella, a fifth acoustic sensor unit 25 and a sixth acoustic sensor unit 26 fixed on both sides of the tibiofemoral joint, a seventh acoustic sensor unit 27 fixed on the front side of the tibial head, an eighth acoustic sensor unit 28 and a ninth acoustic sensor unit 29 fixed on the left and right sides of the tibia at a position 5 - 10 cm away from the tibiofemoral joint, and a tenth acoustic sensor unit 210 and an eleventh acoustic sensor unit 211 fixed on the left and right sides of the femur at a position 10 - 15 cm away from the tibiofemoral joint.

[0048] Furthermore, as Figure 2 shown in (b), when the wearable carrier is unfolded, the first acoustic sensor unit 21, the second acoustic sensor unit 22, and the seventh acoustic sensor unit 27 are arranged in sequence from top to bottom along the vertical symmetry axis of the wearable carrier within the defined area 11 of the wearable carrier 1; the third acoustic sensor unit 23 and the fourth acoustic sensor unit 24 are symmetrically arranged with respect to the vertical symmetry axis; the fifth acoustic sensor unit 25 and the sixth acoustic sensor unit 26 are symmetrically arranged with respect to the vertical symmetry axis; the fifth acoustic sensor unit 25 and the sixth acoustic sensor unit are respectively located below the third acoustic sensor unit 23 and the fourth acoustic sensor unit 24; the eighth acoustic sensor unit 28, the ninth acoustic sensor unit 29, the tenth acoustic sensor unit 210, and the eleventh acoustic sensor unit 211 are respectively located at the four corners of the defined area 11 of the wearable carrier. And the distance between the first acoustic sensor unit 21 and the second acoustic sensor unit 22, the distance between the third acoustic sensor unit 23 and the fourth acoustic sensor unit 24, and the distance between the fifth acoustic sensor unit 25 and the sixth acoustic sensor unit 26 are 6 cm; the distance between the seventh acoustic sensor unit 27 and the second acoustic sensor unit 22 is 6 cm; the distance between the eighth acoustic sensor unit 28 and the ninth acoustic sensor unit 29 and the distance between the tenth acoustic sensor unit 210 and the eleventh acoustic sensor unit 211 are 12 cm.

[0049] Each acoustic sensor unit can capture the acoustic changes inside the knee joint at different angles, including the acoustic characteristics in the bending, stretching, and loading states. As Figure 2As shown in (c), the acoustic sensor unit includes a flexible printed circuit board and two microphones mounted on the flexible printed circuit board. The flexible printed circuit board has good bending performance, facilitating the subject to move according to the given movement plan. The two microphones on the flexible printed circuit board are both connected to the main control unit. The main control unit uses an FPGA programmable logic controller to collect, store the acoustic signals collected by each microphone, and transmit them to the server side for further analysis through the improved SRU neural network model provided in this embodiment. The circuit board carrying the main control unit can be fixedly separated from the wearable carrier 1, for example, fixed on the upper part of the front or outer side of the thigh.

[0050] The knee joint acoustic signal acquisition device further includes a first inertial sensor 3 and a second inertial sensor 4 connected to the main control unit, which are respectively located at the upper and lower edge positions of the symmetry axis of the defined area 11 of the wearable carrier. The two inertial sensors (IMUs) are used to obtain the motion physical quantities for assisting subsequent analysis, facilitating the resolution and extraction of the acoustic signals during movement when processing data subsequently. In this embodiment, only the acoustic signals during movement are analyzed and processed.

[0051] (2) Signal preprocessing module

[0052] The signal preprocessing module is used to filter the collected knee joint acoustic signals to eliminate external environmental noise and other interferences, and obtain clean and reliable knee joint acoustic signal data.

[0053] In a specific implementation manner, the signal preprocessing module can use a band-pass filter to filter the collected knee joint acoustic signals. In this embodiment, according to the angle information collected by the first inertial sensor 3 and the second inertial sensor 4, the starting and ending moments of the movement time are judged, so as to extract the acoustic signals during joint movement from the original audio data of each microphone, and perform preliminary band-pass filtering (150 Hz - 5 kHz) through the signal preprocessing module. Subsequently, the signal preprocessing module divides the band-pass filtered acoustic signals into windows of 30 ms, with an overlap of 20 ms between adjacent window data.

[0054] (3) Feature extraction module

[0055] The feature extraction module is used to extract several spectral features from the filtered knee joint acoustic signals, and then construct an acoustic signal feature matrix of the knee joint.

[0056] In this embodiment, the feature extraction module extracts key spectral features from the acoustic signal data of each window, including time-domain features and frequency-domain features; the time-domain features include time-domain amplitude, short-time energy, and zero-crossing rate; the frequency-domain features include Spectral Centroid, Spectral Spread, Spectral Crest, and Mel Frequency Cepstral Coefficient. The extraction of these features all adopts the conventional methods disclosed in the art and will not be explained in detail here.

[0057] Then, the feature extraction module averages the windowed features within each period to create a feature matrix with m rows of moving periods × n columns of features for each microphone, and concatenates the feature matrices of all microphones to obtain the acoustic signal feature matrix of the knee joint.

[0058] (4) Improved SRU neural network model

[0059] An improved SRU neural network model is used to reconstruct the acoustic signal feature matrix of the knee joint and introduce positioning information, and then identify the health status of the knee joint. When performing motion sound localization in human bones, beamforming can enhance the signals from specific parts while reducing other noise interferences. The present invention combines the Delay and Sum (DS) algorithm and the Time Difference of Arrival (TDOA) to construct an end-to-end improved SRU neural network model based on DS-TDOASRU. Therefore, the improved SRU neural network model includes a delay summation module, a time difference positioning module, and an SRU neural network.

[0060] The delay summation module is used to perform time delay compensation and weighted summation on the signals in the acoustic signal feature matrix of the knee joint to perform the first reconstruction of the feature matrix. The delay summation module is based on the Delay and Sum (DS) algorithm, which is a beamforming technology with good anti-incoherent noise performance. By performing precise time delay compensation on the acoustic sensor unit signals at different positions in the acoustic signal feature matrix of the knee joint (in this embodiment, precise time delay compensation is performed on 22 microphone signals), the signals from a specific direction are aligned in time, and then the corresponding feature signals in each acoustic sensor unit (i.e., 22 microphones) after time delay compensation are weighted and summed to reconstruct the feature matrix with the obtained signals, completing the first reconstruction of the feature matrix, thereby enhancing the target signal and suppressing the interference signal.

[0061] The time difference of arrival (TDOA) positioning module is used to determine the sound source position based on the time differences of the signals arriving at multiple acoustic sensor units, obtain the positioning information, and add the positioning information to the feature matrix after the first reconstruction, and then perform the second reconstruction on the feature matrix. The TDOA positioning module is based on the Time Difference of Arrival (TDOA) algorithm, which is a method of using the time differences of the signals arriving at multiple receivers to determine the sound source position. It can achieve high-precision sound source positioning through simple hardware configuration and is applicable to complex environments. It can be solved by the least squares method, Chan's method, or the bi-spherical solution method, etc. The TDOA algorithm measures the time differences of the sound waves arriving at different acoustic sensor units from the sound source, and combines the position information of each acoustic sensor unit to calculate the positions of more than one sound source, that is, obtain the positioning information of more than one sound source. The positioning information is added as new features to the feature matrix after the first reconstruction to complete the second reconstruction of the feature matrix.

[0062] The SRU neural network is used to identify the knee joint health status based on the feature matrix after the second reconstruction and output the identification result and the positioning information. The SRU (Simple Recurrent Unit) neural network is an efficient variant of the recurrent neural network and has the advantage of processing long time series data. In this embodiment, the SRU neural network takes the feature matrix after the second reconstruction as the input data and the knee joint health status identification result and the positioning information as the output. The identification result includes two levels: the first-level output types include normal knee joint, early knee osteoarthritis, and severe knee osteoarthritis; the second-level output types include cartilage wear, cruciate ligament injury, synovitis, and knee joint loose body. The first-level output only outputs one type, and the second-level output can include at least one of them; because for severe patients, there may be multiple knee joint injuries at the later stage. When the first-level output is a normal knee joint, there is no output at the second level.

[0063] This embodiment also provides a method for early screening of knee joint health status based on multi-channel acoustic information and improved SRU, as Figure 3 shown, which is carried out according to the following steps using the above-mentioned knee joint health status early screening system:

[0064] S1 Wear the knee joint acoustic signal acquisition device on the knee joint of the subject; and collect the knee joint acoustic signals through several acoustic sensor units installed on the wearable carrier.

[0065] In this embodiment, the knee joint acoustic signal acquisition device is worn on the knee joint of the subject; and the knee joint acoustic signals are acquired by 11 acoustic sensor units installed on the wearable carrier, and are synchronously transmitted to the server side through the main control unit together with the motion information acquired by the first inertial sensor and the second inertial sensor, and the acquired acoustic signals are analyzed and processed.

[0066] In order to improve the detection effect, a specified motion paradigm can be provided to the subject. The motion paradigm should include specified motion actions involving knee joint activities (such as sitting up and walking), the duration of each action, the rest interval between each action, and the number and duration of actions required for one round of data collection, etc.

[0067] S2 Filter the acquired knee joint acoustic signals.

[0068] According to the motion information acquired by the first inertial sensor and the second inertial sensor, the knee joint acoustic signals during the motion of each subject are screened out, the acquired knee joint acoustic signals are filtered by the signal preprocessing module, and the filtered acoustic signals are divided according to a 30 ms cycle window, with 20 ms data overlap between adjacent windows.

[0069] S3 Extract several spectral features from the filtered knee joint acoustic signals, and then construct an acoustic signal feature matrix of the knee joint.

[0070] Extract key spectral features from the acoustic signal data of each window, including time domain features and frequency domain features; the time domain features include time domain amplitude, short-time energy, and zero-crossing rate; the frequency domain features include spectral centroid, spectral spread, spectral peak, and Mel frequency cepstral coefficients.

[0071] Then the feature extraction module averages the windowed features in each period, creates a feature matrix with m rows of moving periods × n columns of features for each microphone, and connects the feature matrices of all microphones to obtain the acoustic signal feature matrix of the knee joint.

[0072] S4 Reconstruct the constructed acoustic signal feature matrix of the knee joint through the delay summation module and the time difference localization module, and based on the reconstructed feature matrix, complete the identification of the knee joint health status of the subject through the SRU neural network to obtain the identification result and the localization information.

[0073] First, the signals in the acoustic signal feature matrix of the knee joint are subjected to time-delay compensation and weighted summation through a delay summation module to perform the first reconstruction of the feature matrix. Then, based on the time difference of arrival of the signals at multiple acoustic sensor units, the time difference positioning module determines the sound source position to obtain positioning information, and adds the positioning information to the feature matrix after the first reconstruction to perform the second reconstruction of the feature matrix. The feature matrix after the second reconstruction is expressed as {time amplitude, short-time energy, zero-crossing rate, spectral centroid, spectral spread, spectral peak, Mel frequency cepstral coefficient, sound source position 1, sound source position 2,... sound source position p}.

[0074] Then, the feature matrix after the second reconstruction is input into the trained SRU neural network to obtain the recognition result of the knee joint health state of the subject and the positioning information. The recognition result of the knee joint health state of the subject and the positioning information can be visually displayed through the server side. In particular, the positioning information can be mapped onto the knee joint image for graphical display.

[0075] Figure 4 The visualization interface shows the acoustic signal information collected from different channels.

[0076] In this embodiment, for subjects with different knee joint health states such as normal knee joints, cartilage wear, cruciate ligament injury, synovitis, and knee joint loose bodies, the knee joint acoustic signal acquisition device is worn on the knee joints of the subjects; and the knee joint acoustic signals are collected through 11 acoustic sensor units installed on the wearable carrier, and are synchronously transmitted to the server side together with the motion information collected by the first inertial sensor and the second inertial sensor. The knee joint acoustic signals of different channels collected are as Figure 5 shown; Figure 5 The signals on two channels of the same acoustic sensor unit are given. Suspected synovitis and cartilage wear acoustic signals are shown in both channels. Among them, synovitis is manifested as a vibration signal, while cartilage wear is manifested as a pulse signal.

[0077] Then, filter the collected knee joint acoustic signals and extract spectral features according to the above steps S2 - S3, and construct an acoustic signal feature matrix of the knee joint. Then, reconstruct the constructed feature matrix through a delay summation module and a time difference positioning module. At the same time, label the knee joint health status of each subject to obtain corresponding knee joint health status labels. The labels include two levels. The first - level types include normal knee joint, early knee osteoarthritis, and severe knee osteoarthritis; the second - level types include cartilage wear, cruciate ligament injury, synovitis, and knee joint loose body. For example, for subject 1, his label annotation includes early knee osteoarthritis (output of the first level) and cartilage wear (output of the second level). Use the reconstructed feature matrix and the corresponding labels to construct a data set, and divide the data set into a training set and a test set.

[0078] First, use the training set to train the SRU neural network. The training method can adopt the conventional training method of the SRU neural network (see Tao Lei and Yu Zhang. Training RNNs as Fast as CNNs. arXiv:1709.02755, 2017). Then, use the test set to test the trained SRU neural network.

[0079] Table 1 shows the diagnostic results of using the trained SRU neural network for subjects with different knee joint health statuses. It can be seen from Table 1 that the present invention can effectively identify the knee joint injury status.

[0080] Table 1 Diagnostic Results of Subjects with Different Knee Joint Health Statuses

[0081] The first level type Accuracy rate The second level type Accuracy rate Normal knee joint (30) 86.67％ - - Early knee osteoarthritis (16) 81.25％ Cartilage wear (17) 76.47％ Severe knee osteoarthritis (12) 83.33％ Cruciate ligament injury (20) 75％ - - Synovitis (14) 71.43％ - - Knee joint loose body (9) 77.78％

[0082] Note: Here, 58 subjects are tested. For the second - level types, some subjects may have multiple knee joint injuries at the same time.

[0083] Those of ordinary skill in the art will realize that the embodiments here are for helping readers understand the principles of the present invention. It should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those of ordinary skill in the art can make various other specific deformations and combinations that do not depart from the essence of the present invention according to these technical revelations disclosed in the present invention, and these deformations and combinations are still within the protection scope of the present invention.

Claims

1. An early screening system for knee joint health status based on multi-channel acoustic information and improved SRU, characterized in that: include: A knee joint acoustic signal acquisition device, which is worn on the knee joint to collect the knee joint acoustic signal; The knee joint acoustic signal acquisition device comprises a wearable carrier, and a sensor array composed of a plurality of acoustic sensor units installed on the wearable carrier; A signal preprocessing module is used to filter the collected knee joint acoustic signals; A feature extraction module is used to extract a number of spectral features from the filtered knee joint acoustic signal, and then construct a feature matrix of the acoustic signal of the knee joint; An improved SRU neural network model is used to reconstruct the acoustic signal feature matrix of the knee joint and introduce positioning information to further identify the health status of the knee joint; the improved SRU neural network model includes a delay summation module, a time difference positioning module and an SRU neural network; the delay summation module is used to perform time delay compensation and weighted summation on the signals in the acoustic signal feature matrix of the knee joint, and perform a first reconstruction of the feature matrix; the time difference positioning module is used to determine the location of the sound source according to the time difference between the signals reaching multiple acoustic sensor units, obtain positioning information, and add the positioning information to the feature matrix after the first reconstruction, and perform a second reconstruction of the feature matrix; the SRU neural network is used to identify the health status of the knee joint according to the feature matrix after the second reconstruction, and output the recognition result and positioning information.

2. The knee joint health status early screening system based on multi-channel acoustic information and improved SRU according to claim 1, characterized in that: The several acoustic sensor units arranged in the knee joint acoustic signal acquisition device are respectively fixed on the upper and lower sides, left and right sides of the patella, both sides of the tibiofemoral joint, the front side of the tibia head, the left and right sides of the tibia at a distance of 5 to 10 cm from the tibiofemoral joint, or / and the left and right sides of the femur at a distance of 10 to 15 cm from the tibiofemoral joint.

3. The knee joint health status early screening system based on multi-channel acoustic information and improved SRU according to claim 2, characterized in that: The knee joint acoustic signal acquisition device comprises a first acoustic sensor unit and a second acoustic sensor unit fixed on the upper and lower sides of the patella, a third acoustic sensor unit and a fourth acoustic sensor unit fixed on the left and right sides of the patella, a fifth acoustic sensor unit and a sixth acoustic sensor unit fixed on the two sides of the tibiofemoral joint, a seventh acoustic sensor unit fixed on the front side of the tibia head, an eighth acoustic sensor unit and a ninth acoustic sensor unit fixed on the left and right sides of the tibia and 5 to 10 cm away from the tibiofemoral joint, and / or a third acoustic sensor unit and a fourth acoustic sensor unit fixed on the left and right sides of the femur and 10 to 15 cm away from the tibiofemoral joint. The tenth acoustic sensor unit and the eleventh acoustic sensor unit are arranged; the spacing between the first acoustic sensor unit and the second acoustic sensor unit, the spacing between the third acoustic sensor unit and the fourth acoustic sensor unit, and the spacing between the fifth acoustic sensor unit and the sixth acoustic sensor unit are 6 to 8 cm; the spacing between the seventh acoustic sensor unit and the second acoustic sensor unit is 6 to 8 cm; the spacing between the eighth acoustic sensor unit and the ninth acoustic sensor unit, and the spacing between the tenth acoustic sensor unit and the eleventh acoustic sensor unit are 10 to 15 cm.

4. The early screening system for knee joint health status based on multi-channel acoustic information and improved SRU according to any one of claims 1 to 3, characterized in that: The acoustic sensor unit includes a circuit board and two microphones installed on the circuit board; the two microphones on the circuit board are connected to a main control unit; the main control unit adopts a PAL, GAL or FPGA programmable logic controller.

5. The knee joint health status early screening system based on multi-channel acoustic information and improved SRU according to claim 4, characterized in that: The knee joint acoustic signal acquisition device also includes an inertial sensor or an angle sensor.

6. The knee joint health status early screening system based on multi-channel acoustic information and improved SRU according to claim 4, characterized in that: The signal preprocessing module uses a bandpass filter to filter the collected knee joint acoustic signals; and then divides the filtered knee joint acoustic signals according to a set period window.

7. The knee joint health status early screening system based on multi-channel acoustic information and improved SRU according to claim 4, characterized in that: The feature extraction module is used to extract several key spectral features of the acoustic signal, including time domain features and / or frequency domain features; the time domain features include at least one of time domain amplitude, short-time energy, and zero-crossing rate; the frequency domain features include at least one of spectrum centroid, spectrum extension, spectrum peak, and Mel-frequency cepstrum coefficient; Then the feature extraction module averages the windowed features in each cycle, creates a feature matrix of m rows of moving cycles × n columns of features for each microphone, and concatenates the feature matrices of all microphones to obtain the acoustic signal feature matrix of the knee joint.

8. A method for early screening of knee joint health status based on multi-channel acoustic information and improved SRU, characterized in that: The knee joint health status early screening system according to any one of claims 1 to 7 is used in accordance with the following steps: S1 wears a knee joint acoustic signal acquisition device on the subject's knee joint; and collects knee joint acoustic signals through a plurality of acoustic sensor units installed on a wearable carrier; S2 performs filtering processing on the collected acoustic signals of the knee joint; S3 extracts several spectral features from the filtered acoustic signal of the knee joint, and then constructs a feature matrix of the acoustic signal of the knee joint; S4 reconstructs the constructed acoustic signal feature matrix of the knee joint through the delayed sum module and the time difference positioning module, and based on the reconstructed feature matrix, completes the recognition of the health status of the subject's knee joint through the SRU neural network to obtain the recognition result and positioning information.

Images