Traditional Chinese medicine intelligent auxiliary diagnosis system based on autism

Abstract

The invention relates to the technical field of intelligent diagnosis, in particular to a traditional Chinese medicine intelligent auxiliary diagnosis system based on autism, and the system comprises a data collection module which collects daily behavior data of a patient, detects the integrity of the daily behavior data, and carries out the definition judgment of audio data; the posture analysis module is used for extracting and preprocessing the collected video frame by frame, normalizing original coordinates of limb joint points, inputting the normalized original coordinates into a human body posture estimation library, obtaining a multi-frame joint point position data set, and carrying out comprehensive quantitative analysis on patient postures by utilizing the joint point position data set to provide a basis for intelligent auxiliary diagnosis; and the voice analysis module is used for deeply analyzing the change condition of the voice rhythm characteristics so as to judge whether the sound production control is influenced or not, assisting illness state diagnosis and providing acoustic dimension evidence for diagnosis. Rich data such as videos and audios are fused, limb joint point coordinates of a patient are accurately analyzed, voice rhythm features are extracted and the like, and a solid and reliable foundation is provided for diagnosis.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent diagnosis, and in particular to a Chinese medicine intelligent auxiliary diagnosis system based on autism. Background Art

[0002] Autism, as a common mental developmental disorder, has increased dramatically in the past few decades. The clinical cause of the disease is mostly abnormal brain activity. Most clinical diagnosis methods are based on symptomatic behavioral observation, which usually requires a large amount of high-quality medical resources and long-term observation, monitoring and screening to conclude whether the disease is present and to initiate subsequent long-term treatment. Autism consumes a lot of time for evaluation and treatment, which is long and consumes a lot of medical resources.

[0003] By combining traditional Chinese medicine with artificial intelligence, we will create digital famous Chinese medicine doctors to assist in the diagnosis and treatment of autism, optimize and improve the diagnosis and treatment process of traditional Chinese medicine, improve the prevention and treatment level of autism, expand the diagnosis and treatment capabilities of famous Chinese medicine doctors, and demonstrate the unique advantages of traditional Chinese medicine in health services. Summary of the invention

[0004] The present invention aims at the technical problems existing in the prior art and provides a TCM intelligent auxiliary diagnosis system based on autism.

[0005] The technical solution of the present invention to solve the above technical problems is as follows: A TCM intelligent auxiliary diagnosis system based on autism, comprising:

[0006] Data collection module: collects patients' daily behavior data, detects its integrity, and determines the clarity of audio data;

[0007] Posture analysis module: extract and preprocess the collected video frame by frame, normalize the original coordinates of the limb joints and input them into the human posture estimation library, obtain multi-frame joint position data sets, and use the joint position data sets to perform comprehensive quantitative analysis of the patient's posture, providing a basis for intelligent auxiliary diagnosis;

[0008] Speech analysis module: Through in-depth analysis of changes in speech rhythmic features, it can determine whether vocal control is affected, assist in disease diagnosis, and provide acoustic dimension evidence for diagnosis.

[0009] In a preferred embodiment, the data acquisition and processing module collects the patient's daily behavior video and audio samples, performs integrity and clarity detection on the collected data, traverses the video and audio file storage directory, obtains the file header information of each file, including file size, duration and encoding format data, and immediately marks the group of data as incomplete if the video and audio file headers show insufficient duration, evaluates the clarity of the collected audio, and performs frame processing on the audio, assuming that the length of each frame is L frame sampling points, the total number of frames is Nframes , calculate the energy of each frame, the specific calculation formula is as follows:

[0010]

[0011] in, represents the audio time domain sampling value, i represents the frame number, E i Represents the energy of each frame. Based on the energy of each frame, the total energy of the audio is calculated. The specific calculation formula is as follows:

[0012]

[0013] Calculate the average energy of the audio. The specific calculation formula is as follows:

[0014]

[0015] Set the average energy lower limit threshold E min , to determine the clarity of the audio, the judgment method is as follows:

[0016]

[0017] If the audio clarity is judged as 0, it means that the audio has clarity problems and does not meet the requirements, and needs to be re-collected.

[0018] In a preferred embodiment, the posture analysis module extracts the collected video of the patient's daily activities frame by frame to obtain an image frame sequence, normalizes the original coordinates of each limb joint point in the image frame in pixels, and marks the horizontal coordinate of a joint point in the original image frame as x. original , the vertical axis is marked as y original , the width of the image is W pixels and the height is H pixels, then the normalized horizontal coordinate x of the joint point is normalized and the ordinate y normalized They are:

[0019]

[0020] The normalized image frames are input into the pre-trained human posture estimation library. The human posture estimation library outputs the position coordinate information of each key joint point of the patient's limbs for each frame of the image. The skeletal motion model is constructed based on the output position coordinate information. The skeletal model is used to quantify the patient's limb posture and motion characteristics by calculating the distance between joint nodes, joint angles, bone length ratios, motion amplitudes, and motion continuity. The relevant theories of traditional Chinese medicine are combined to assist in the diagnosis of the condition.

[0021] Calculation of distance between joint nodes When analyzing the patient's limb extension and range of motion using the coordinate information of the patient's limb joint points in each frame of the output image, it is necessary to calculate the distance between different joint points. For the analysis of the arm extension state, the shoulder joint point is selected and its coordinate is set as A(x A ,y A ) and the wrist joint point, let its coordinates be B(x B ,y B ), calculate the Euclidean distance between them, the specific calculation formula is as follows:

[0022]

[0023] By observing the change of the distance value in multiple frames of images, the extension degree of the patient's arm at different times can be understood, the amplitude characteristics of the arm movement can be analyzed, and the patient's movement incoordination and physical weakness that cause abnormal arm extension can be determined.

[0024] The calculation of joint angle is done by analyzing the patient's arm bending angle and calculating the shoulder joint point coordinates S(x s ,y s ), elbow joint coordinates E(x E ,y E ), wrist joint coordinates W(x W ,y W ), use the vector to calculate the angle at the elbow joint. First calculate the vector and vector Use the vector dot product formula to determine the cosine value cosθ of their angle. The calculation formula is as follows:

[0025]

[0026] Among them, the vector dot product vector Model vector Model The angle value θ is obtained by the arc cosine function. The specific calculation formula is as follows:

[0027]

[0028] θ=arccos(cosθ)

[0029] in, represents the vector formed by the shoulder joint point pointing to the elbow joint point, represents the vector pointing from the elbow joint point to the wrist joint point, It reflects the position change vector from elbow to wrist, cosθ represents the vector and The cosine of the angle, It reflects the actual length of the vector from the shoulder to the elbow. It reflects the actual length of the vector from the elbow to the wrist. By calculating the angles at different joints and comparing the normal angle ranges of the corresponding joints of normal patients of the same age group, it can determine whether the patient's bending degree and overall posture are abnormal, and assist in determining the patient's uncoordinated movements.

[0030] The bone length ratio calculation is based on the fixed ratio between the lengths of the bones in the normal human bone structure. The length ratio of the femur and shin bones of the patient is calculated based on the coordinates of the joint points to assist in determining whether the patient's physical development meets the normal standard. The coordinates of the hip joint point are H(x H ,y H ), the coordinates of the knee joint are K(x K ,y K ), the ankle joint coordinates are A(x A ,y A ), first calculate the length of the femur, the specific calculation formula is as follows:

[0031]

[0032] Among them, L femur Indicates the length of the thigh bone, and then calculates the length of the calf bone. The specific calculation formula is as follows:

[0033]

[0034] Among them, L tibia represents the length of the tibia, then the length ratio of the femur to the tibia is:

[0035]

[0036] Among them, P ft Indicates the length ratio of the thigh bone to the calf bone. Compare the patient's ratio with the normal average bone length ratio range of the same age group. When the difference exceeds the reasonable range, it can help determine whether there is abnormal physical development;

[0037] The motion amplitude is calculated based on the output joint point coordinate information of the patient's limb movement. The motion amplitude is calculated by analyzing the unique situation of the relevant joint points during the action. The coordinates of the joint nodes involved in the action in the starting frame are (x start ,y start ), when the action is completed, the coordinates of the joint point in the end frame become (x end ,y end ), then the range of motion A of this joint point in this action is calculated according to the following formula:

[0038]

[0039] Compare the patient's range of motion with that of the same age group to help determine whether the patient has characteristic differences in motion range due to physical function problems;

[0040] The action continuity is judged by analyzing the position change of the same joint point in adjacent frames. Suppose the coordinates of the elbow joint in the i-th frame are (x i ,y i ), the coordinates in the i+1th frame are (x i+1 ,y i+1 ), calculate the difference between the position coordinates of the joint point between two adjacent frames, which are Δx=x i+1 -x i , Δy=y i+1 -y i , according to the fluctuation range of the normal corresponding action joint node position change, set the threshold value of the coordinate change difference |Δx|≤x threshold And |Δy|≤y threshold If the difference in the position change of the joint point in multiple consecutive frames is within the set threshold range, the action is coherent, otherwise it indicates that the action coherence is poor, x threshold Indicates the threshold of the difference in the horizontal coordinate change, y threshold Indicates the threshold of the vertical coordinate change difference.

[0041] In a preferred embodiment, the speech analysis module quantitatively analyzes the abnormal condition of the patient by analyzing the patient's fundamental frequency changes, intonation fluctuations, speech speed stability and abnormal vocalization behavior, and performs frame analysis on the audio through an audio processing tool;

[0042] The fundamental frequency analysis is performed by extracting the fundamental frequency value f of each frame. i (1,2,..,n), n represents the total number of frames, calculate the fundamental frequency mean:

[0043]

[0044] in, Represents the fundamental frequency mean, and then calculates the fundamental frequency standard deviation SD f , used to measure the change of fundamental frequency:

[0045]

[0046] If the patient's SD f If the fundamental frequency standard deviation is beyond the normal range, it means that the patient's fundamental frequency changes do not conform to the normal pattern;

[0047] The intonation fluctuation is achieved by dividing the semantic unit into K semantic units, and the maximum fundamental frequency in the kth semantic unit is f max , the minimum value is f min , then the specific calculation formula of intonation change rate is as follows:

[0048]

[0049] If R intonation If the value is close to 0, it means that the patient's tone is monotonous and stereotyped;

[0050] To measure the stability of speech rate, we need to first determine the duration of each sentence. j , and count the number of words n contained in each sentence j , calculate the speaking speed v of each sentence j , mean speech rate and the standard deviation of speech rate SD v , the specific calculation formula of speaking speed is as follows:

[0051]

[0052] The specific calculation formula for the mean speech speed is as follows:

[0053]

[0054] The specific calculation formula for the standard deviation of speaking speed is as follows:

[0055]

[0056] If the value of the speech rate standard deviation is too large, it indicates that the patient's speech rate is unstable;

[0057] The frequency of abnormal vocalization behavior is determined by observing the number of times N abnormal vocalization behaviors occur in the total length of the video. abnormal , then the specific calculation formula for the average frequency of abnormal vocalization behavior is as follows:

[0058]

[0059] Among them, F abnormal represents the average frequency of abnormal vocalization behavior, T total Indicates the total duration of the observed video. If the average frequency of abnormal vocalization behavior is high, it indicates that there is abnormal vocalization in this aspect. For abnormal vocalization behavior, the duration of its occurrence is counted as T abnormal , then the specific calculation formula for the abnormal vocalization behavior duration ratio is as follows:

[0060]

[0061] If the duration of abnormal vocal behavior is high, it means that the abnormal vocal behavior accounts for a large proportion in the overall vocalization process. Based on the quantification of the patient's language characteristics and vocal abnormalities, it provides accurate data basis for the diagnosis of autism based on TCM theory.

[0062] The beneficial effects of the present invention are: the present invention provides effective suggestions for some doctors, assists them in evaluating the physical condition of patients and improving their diagnostic ability, takes more comprehensive considerations, and breaks through regional limitations. The distribution of medical resources in my country is uneven in regions, and most of them are distributed in coastal areas, resulting in many autistic patients in remote areas being unable to regularly go to tertiary hospitals for timely and effective diagnosis due to the long distance. The emergence of this system provides convenience for patients. They only need to go to local hospitals for treatment and input data into the system to get corresponding suggestions. It integrates rich data such as video and audio, accurately analyzes the coordinates of patients' limb joints, extracts speech rhythm features, etc., captures details in multiple dimensions, and processes data rigorously and meticulously. The quality is ensured through normalization, cleaning and other operations, providing a solid and reliable foundation for diagnosis and greatly reducing misdiagnosis and missed diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 is a flow chart of the present invention;

[0064] Figure 2 It is a system block diagram of the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0066] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise clearly and specifically defined.

[0067] In the description of the present application, the term "for example" is used to mean "used as an example, illustration or description". Any embodiment described as "for example" in the present application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid unnecessary details to obscure the description of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0068] like Figure 1-2 This embodiment provides: a TCM intelligent auxiliary diagnosis system based on autism, comprising:

[0069] Data collection module: collects patients' daily behavior data, detects its integrity, and determines the clarity of audio data;

[0070] In this embodiment, what needs to be specifically explained is the data acquisition and processing module, which collects the patient's daily behavior video and audio samples, performs integrity and clarity detection on the collected data, traverses the video and audio file storage directory, obtains the file header information of each file, including file size, duration and encoding format data, and immediately marks the group of data as incomplete if the video and audio file headers show insufficient duration, evaluates the clarity of the collected audio, and performs frame processing on the audio, assuming that the length of each frame is L frame sampling points, the total number of frames is N frames , calculate the energy of each frame, the specific calculation formula is as follows:

[0071]

[0072] in, represents the audio time domain sampling value, i represents the frame number, E i Represents the energy of each frame. Based on the energy of each frame, the total energy of the audio is calculated. The specific calculation formula is as follows:

[0073]

[0074] Calculate the average energy of the audio. The specific calculation formula is as follows:

[0075]

[0076] Set the average energy lower limit threshold E min , to determine the clarity of the audio, the judgment method is as follows:

[0077]

[0078] If the audio clarity is judged as 0, it means that the audio has clarity problems and does not meet the requirements, and needs to be re-collected.

[0079] It should be noted that the daily behavior video is a multi-angle shot of life clips, with a total duration of no less than 30 minutes. The audio samples need to collect the child's speech, distress and laughter, with a total duration of no less than 15 minutes. The average energy of the audio is an important indicator for measuring the strength of the audio signal. Average energy plays a key role in evaluating audio clarity. Clear audio usually has a certain intensity of signal energy. If the average energy is too low, it may mean that the audio signal is very weak. This may be due to the recording device being too far away from the sound source, the noisy recording environment causing the effective signal to be masked, or the gain setting of the recording device itself is improper. For example, when recording the patient's voice for autism diagnosis, if the average energy is insufficient, the sound details may be lost, and details such as the child's weak language expression and emotional vocalization are difficult to accurately capture and analyze. By comparing with a reasonable average energy lower limit threshold, it is possible to preliminarily determine whether the audio meets the quality requirements, which helps to screen out audio that may have clarity problems.

[0080] Posture analysis module: extract and preprocess the collected video frame by frame, normalize the original coordinates of the limb joints and input them into the human posture estimation library, obtain multi-frame joint position data sets, and use the joint position data sets to perform comprehensive quantitative analysis of the patient's posture, providing a basis for intelligent auxiliary diagnosis;

[0081] In this embodiment, the posture analysis module needs to be specifically explained. The posture analysis module extracts the collected video of the patient's daily activities frame by frame to obtain an image frame sequence, normalizes the original coordinates of each limb joint point in the image frame in pixels, and marks the horizontal coordinate of a joint point in the original image frame as x. original , the vertical axis is marked as y original , the width of the image is W pixels and the height is H pixels, then the normalized horizontal coordinate x of the joint point is normalized and the ordinate y normalized They are:

[0082]

[0083] The normalized image frames are input into the pre-trained human posture estimation library. The human posture estimation library outputs the position coordinate information of each key joint point of the patient's limbs for each frame of the image. The skeletal motion model is constructed based on the output position coordinate information. The skeletal model is used to quantify the patient's limb posture and motion characteristics by calculating the distance between joint nodes, joint angles, bone length ratios, motion amplitudes, and motion continuity. The relevant theories of traditional Chinese medicine are combined to assist in the diagnosis of the condition.

[0084] Calculation of distance between joint nodes When analyzing the patient's limb extension and range of motion using the coordinate information of the patient's limb joint points in each frame of the output image, it is necessary to calculate the distance between different joint points. For the analysis of the arm extension state, the shoulder joint point is selected and its coordinate is set as A(x A ,y A ) and the wrist joint point, let its coordinates be B(x B ,y B ), calculate the Euclidean distance between them, the specific calculation formula is as follows:

[0085]

[0086] By observing the change of the distance value in multiple frames of images, the extension degree of the patient's arm at different times can be understood, the amplitude characteristics of the arm movement can be analyzed, and the patient's movement incoordination and physical weakness that cause abnormal arm extension can be determined.

[0087] The calculation of joint angle is done by analyzing the patient's arm bending angle and calculating the shoulder joint point coordinates S(x s ,y s ), elbow joint coordinates E(x E ,y E ), wrist joint coordinates W(x W ,y W ), use the vector to calculate the angle at the elbow joint. First calculate the vector and vector Use the vector dot product formula to determine the cosine value cosθ of their angle. The calculation formula is as follows:

[0088]

[0089] Among them, the vector dot product vector Model vector Model The angle value θ is obtained by the arc cosine function. The specific calculation formula is as follows:

[0090]

[0091] θ=arccos(cosθ)

[0092] in, represents the vector formed by the shoulder joint point pointing to the elbow joint point, represents the vector pointing from the elbow joint point to the wrist joint point, It reflects the position change vector from the elbow to the wrist. cosθ represents the vector and The cosine of the angle, It reflects the actual length of the vector from the shoulder to the elbow. It reflects the actual length of the vector from the elbow to the wrist. By calculating the angles at different joints and comparing the normal angle ranges of the corresponding joints of normal patients of the same age group, it can determine whether the patient's bending degree and overall posture are abnormal, and assist in determining the patient's uncoordinated movements.

[0093] The bone length ratio calculation is based on the fixed ratio between the lengths of the bones in the normal human bone structure. The length ratio of the femur and shin bones of the patient is calculated based on the coordinates of the joint points to assist in determining whether the patient's physical development meets the normal standard. The coordinates of the hip joint point are H(x H ,y H ), the coordinates of the knee joint are K(x K ,y K ), the ankle joint coordinates are A(x A ,y A ), first calculate the length of the femur, the specific calculation formula is as follows:

[0094]

[0095] Among them, L femur Indicates the length of the thigh bone, and then calculates the length of the calf bone. The specific calculation formula is as follows:

[0096]

[0097] Among them, L tibia represents the length of the tibia, then the length ratio of the femur to the tibia is:

[0098]

[0099] Among them, P ft Indicates the length ratio of the thigh bone to the calf bone. Compare the patient's ratio with the normal average bone length ratio range of the same age group. When the difference exceeds the reasonable range, it can help determine whether there is abnormal physical development;

[0100] The motion amplitude is calculated based on the output joint point coordinate information of the patient's limb movement. The motion amplitude is calculated by analyzing the unique situation of the relevant joint points during the action. The coordinates of the joint nodes involved in the action in the starting frame are (x start ,y start ), when the action is completed, the coordinates of the joint point in the end frame become (x end ,y end ), then the range of motion A of this joint point in this action is calculated according to the following formula:

[0101]

[0102] Compare the patient's range of motion with that of the same age group to help determine whether the patient has characteristic differences in motion range due to physical function problems;

[0103] The action continuity is judged by analyzing the position change of the same joint point in adjacent frames. Suppose the coordinates of the elbow joint in the i-th frame are (x i ,y i ), the coordinates in the i+1th frame are (x i+1 ,y i+1 ), calculate the difference between the position coordinates of the joint point between two adjacent frames, which are Δx=x i+1 -x i , Δy=y i+1 -y i , according to the fluctuation range of the normal corresponding action joint node position change, set the threshold value of the coordinate change difference |Δx|≤x threshold And |Δy|≤y threshold If the difference in the position change of the joint point in multiple consecutive frames is within the set threshold range, the action is coherent, otherwise it indicates that the action coherence is poor, x threshold Indicates the threshold of the difference in the horizontal coordinate change, y threshold Indicates the threshold of the vertical coordinate change difference.

[0104] It should be noted that the length of the femur is calculated by using the coordinates of the hip joint and the knee joint, and the length of the tibia is calculated by using the coordinates of the knee joint and the ankle joint.

[0105] It should be noted that the specific steps for the human body pose estimation library to output the coordinate information of the human body joints are as follows: initialize the human body pose estimation library and configure relevant parameters, use the image processing library to read the collected video frames or image frames, and input the read video frames or image frames into the human body pose estimation library to identify the positions of various key points of the human body. After the pose estimation is completed, the data of the key points is obtained by accessing the interface provided by the human body pose estimation library, and the specific coordinate information is extracted from the data. Extracting the coordinates of the joints through the human body pose estimation library is an existing technology and will not be elaborated here.

[0106] Speech analysis module: Through in-depth analysis of changes in speech rhythmic features, it can determine whether vocal control is affected, assist in disease diagnosis, and provide acoustic dimension evidence for diagnosis.

[0107] In this embodiment, it is necessary to specifically explain the speech analysis module, which quantitatively analyzes the abnormal condition of the patient by analyzing the patient's fundamental frequency changes, intonation fluctuations, speech speed stability and abnormal vocalization behavior, and performs frame analysis on the audio through an audio processing tool;

[0108] The fundamental frequency analysis is performed by extracting the fundamental frequency value f of each frame. i (1,2,..,n), n represents the total number of frames, calculate the fundamental frequency mean:

[0109]

[0110] in, Represents the fundamental frequency mean, and then calculates the fundamental frequency standard deviation SD f , used to measure the change of fundamental frequency:

[0111]

[0112] If the patient's SD f If the fundamental frequency standard deviation is beyond the normal range, it means that the patient's fundamental frequency changes do not conform to the normal pattern;

[0113] The intonation fluctuation is achieved by dividing the semantic unit into K semantic units, and the maximum fundamental frequency in the kth semantic unit is f max , the minimum value is f min , then the specific calculation formula of intonation change rate is as follows:

[0114]

[0115] If R intonation If the value is close to 0, it means that the patient's tone is monotonous and stereotyped;

[0116] To measure the stability of speech rate, we need to first determine the duration of each sentence. j, and count the number of words n contained in each sentence j , calculate the speaking speed v of each sentence j , mean speech rate and the standard deviation of speech rate SD v , the specific calculation formula of speaking speed is as follows:

[0117]

[0118] The specific calculation formula for the mean speech speed is as follows:

[0119]

[0120] The specific calculation formula for the standard deviation of speaking speed is as follows:

[0121]

[0122] If the value of the speech rate standard deviation is too large, it indicates that the patient's speech rate is unstable;

[0123] The frequency of abnormal vocalization behavior is determined by observing the number of times N abnormal vocalization behaviors occur in the total length of the video. abnormal , then the specific calculation formula for the average frequency of abnormal vocalization behavior is as follows:

[0124]

[0125] Among them, F abnormal represents the average frequency of abnormal vocalization behavior, T total Indicates the total duration of the observed video. If the average frequency of abnormal vocalization behavior is high, it indicates that there is abnormal vocalization in this aspect. For abnormal vocalization behavior, the duration of its occurrence is counted as T abnormal , then the specific calculation formula for the abnormal vocalization behavior duration ratio is as follows:

[0126]

[0127] If the duration of abnormal vocal behavior is high, it means that the abnormal vocal behavior accounts for a large proportion in the overall vocalization process. Based on the quantification of the patient's language characteristics and vocal abnormalities, it provides accurate data basis for the diagnosis of autism based on TCM theory.

[0128] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0129] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0130] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0131] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0132] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0133] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0134] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A TCM intelligent auxiliary diagnosis system based on autism, characterized in that: include: Data collection module: collects patients' daily behavior data, detects its integrity, and determines the clarity of audio data; Posture analysis module: extract and preprocess the collected video frame by frame, normalize the original coordinates of the limb joints and input them into the human posture estimation library, obtain multi-frame joint position data sets, and use the joint position data sets to perform comprehensive quantitative analysis of the patient's posture, providing a basis for intelligent auxiliary diagnosis; Speech analysis module: Through in-depth analysis of changes in speech rhythmic features, it can determine whether vocal control is affected, assist in disease diagnosis, and provide acoustic dimension evidence for diagnosis.

2. According to claim 1, a TCM intelligent auxiliary diagnosis system based on autism is characterized in that: The data acquisition and processing module collects the patient's daily behavior video and audio samples, performs integrity and clarity detection on the collected data, traverses the video and audio file storage directory, obtains the file header information of each file, including file size, duration and encoding format data, and immediately marks the group of data as incomplete if the video and audio file headers show insufficient duration. It evaluates the clarity of the collected audio and performs frame processing on the audio. Suppose the length of each frame is L frame sampling points, the total number of frames is N frames , calculate the energy of each frame, the specific calculation formula is as follows: in, represents the audio time domain sampling value, i represents the frame number, E i Represents the energy of each frame. Based on the energy of each frame, the total energy of the audio is calculated. The specific calculation formula is as follows: Calculate the average energy of the audio. The specific calculation formula is as follows: Set the average energy lower limit threshold E min , to determine the clarity of the audio, the judgment method is as follows: If the audio clarity is judged as 0, it means that the audio has clarity problems and does not meet the requirements, and needs to be re-collected.

3. According to claim 1, a TCM intelligent auxiliary diagnosis system based on autism is characterized in that: The posture analysis module extracts the collected video of the patient's daily activities frame by frame to obtain an image frame sequence, normalizes the original coordinates of each limb joint point in the image frame in pixels, and marks the horizontal coordinate of a joint point in the original image frame as x. original , the vertical axis is marked as y original , the width of the image is W pixels and the height is H pixels, then the normalized horizontal coordinate x of the joint point is normalized and the ordinate y normalized They are: The normalized image frames are input into the pre-trained human posture estimation library, which outputs the position coordinate information of the key joints of the patient's limbs for each frame of the image, and builds a skeletal motion model based on the output position coordinate information. The skeletal model is used to quantify the patient's limb posture and motion characteristics by calculating the distance between joint nodes, joint angles, bone length ratios, motion amplitudes, and motion continuity, and combined with relevant theories of traditional Chinese medicine to assist in judging the condition.

4. According to claim 3, a TCM intelligent auxiliary diagnosis system based on autism is characterized in that: The distance calculation between joint nodes uses the coordinate information of the patient's limb joint points in each frame of the output image to analyze the patient's limb extension and movement range. It is necessary to calculate the distance between different joint points. For the analysis of the arm extension state, the shoulder joint point is selected and its coordinate is set as A(x A ,y A ) and the wrist joint point, let its coordinates be B(x B ,y B ), calculate the Euclidean distance between them, the specific calculation formula is as follows: By observing the changes in the distance value in multiple frames of images, we can understand the degree of extension of the patient's arm at different times, analyze the amplitude characteristics of the arm movement, and assist in judging whether the patient has incoordination of movements and physical weakness that causes abnormal arm extension.

5. According to claim 3, a TCM intelligent auxiliary diagnosis system based on autism is characterized in that: The calculation of the joint angle is performed by analyzing the patient's arm bending angle and according to the shoulder joint point coordinates S(x s ,y s ), elbow joint coordinates E(x E ,y E ), wrist joint coordinates W(x W ,y W ), use the vector to calculate the angle at the elbow joint. First calculate the vector and vector Use the vector dot product formula to determine the cosine value of their angle, cosθ, as follows: Among them, the vector dot product vector Model vector Model The angle value θ is obtained by the arc cosine function. The specific calculation formula is as follows: θ=arccos(cosθ) in, represents the vector formed by the shoulder joint point pointing to the elbow joint point, represents the vector pointing from the elbow joint point to the wrist joint point, It reflects the position change vector from elbow to wrist, cosθ represents the vector and The cosine of the angle, It reflects the actual length of the vector from the shoulder to the elbow. It reflects the actual length of the vector from the elbow to the wrist. By calculating the angles at different joints and comparing them with the normal angle ranges of the corresponding joints of normal patients of the same age, it can determine the patient's degree of bending and whether there are any abnormalities in the overall movement posture, and assist in determining the patient's movement incoordination.

6. The TCM intelligent auxiliary diagnosis system based on autism according to claim 3 is characterized in that: The bone length ratio calculation is based on the fixed ratio relationship between the lengths of the bones of each part in the normal human bone structure. The length ratio of the femur and shin bone of the patient is calculated based on the joint point coordinates to assist in judging whether the patient's physical development meets the normal standard. Suppose the hip joint point coordinates are H(x H ,y H ), the coordinates of the knee joint are K(x K ,y K ), the ankle joint coordinates are A(x A ,y A ), first calculate the length of the femur, the specific calculation formula is as follows: Among them, L femur Indicates the length of the thigh bone, and then calculates the length of the calf bone. The specific calculation formula is as follows: Among them, L tibia represents the length of the tibia, then the length ratio of the femur to the tibia is: Among them, P ft It indicates the length ratio of the thigh bone to the calf bone. The patient's ratio is compared with the normal average bone length ratio range of the same age group. When the difference exceeds a reasonable range, it can help determine whether there is any physical development abnormality.

7. The TCM intelligent auxiliary diagnosis system based on autism according to claim 3 is characterized in that: The motion amplitude is calculated by analyzing the unique situation of the relevant joint points during the motion process based on the output joint point coordinate information of the patient's limb motion. The coordinates of the joint nodes involved in the motion in the starting frame are (x start ,y start ), when the action is completed, the coordinates of the joint point in the end frame become (x end ,y end ), then the range of motion A of this joint point in this action is calculated according to the following formula: Compare the patient's range of motion with that of people of the same age group to help determine whether the patient has characteristic differences in motion range due to physical function problems.

8. The TCM intelligent auxiliary diagnosis system based on autism according to claim 3 is characterized in that: The action continuity judgment is performed by analyzing the position change of the same joint point in adjacent frames. Assume that the coordinates of the elbow joint in the i-th frame are (x i ,y i ), the coordinates in the i+1th frame are (x i+1 ,y i+1 ), calculate the difference between the position coordinates of the joint point between two adjacent frames, which are Δx=x i+1 -x i , Δy=y i+1 -y i , according to the fluctuation range of the normal corresponding action joint node position change, set the threshold value of the coordinate change difference |Δx|≤x threshold And |Δy|≤y threshold If the difference in the position change of the joint point in multiple consecutive frames is within the set threshold range, the action is coherent, otherwise it indicates that the action coherence is poor, x threshold Indicates the threshold of the difference in the horizontal coordinate change, y threshold Indicates the threshold of the vertical coordinate change difference.

9. The TCM intelligent auxiliary diagnosis system based on autism according to claim 1, characterized in that: The speech analysis module quantitatively analyzes the abnormal condition of the patient by analyzing the patient's fundamental frequency changes, intonation fluctuations, speech speed stability and abnormal vocalization behavior, and performs frame analysis on the audio through an audio processing tool; The fundamental frequency analysis is performed by extracting the fundamental frequency value f of each frame. i (1,2,..,n), n represents the total number of frames, calculate the fundamental frequency mean: in, Represents the fundamental frequency mean, and then calculates the fundamental frequency standard deviation SD f , used to measure the change of fundamental frequency: If the patient's SD f If the fundamental frequency standard deviation is beyond the normal range, it means that the patient's fundamental frequency changes do not conform to the normal pattern; The intonation fluctuation is achieved by dividing the semantic unit into K semantic units, and the maximum fundamental frequency in the kth semantic unit is f max , the minimum value is f min , then the specific calculation formula of intonation change rate is as follows: If R intonation If the value is close to 0, it means that the patient's tone is monotonous and stereotyped; To measure the stability of speech rate, we need to first determine the duration of each sentence. j , and count the number of words n contained in each sentence j , calculate the speaking speed v of each sentence j , mean speech rate and the standard deviation of speech rate SD v , the specific calculation formula of speaking speed is as follows: The specific calculation formula for the mean speech speed is as follows: The specific calculation formula for the standard deviation of speaking speed is as follows: If the value of the speech rate standard deviation is too large, it indicates that the patient's speech rate is unstable; The frequency of abnormal vocalization behavior is determined by observing the number of times N abnormal vocalization behaviors occur in the total length of the video. abnormal , then the specific calculation formula for the average frequency of abnormal vocalization behavior is as follows: Among them, F abnormal represents the average frequency of abnormal vocalization behavior, T total Indicates the total duration of the observed video. If the average frequency of abnormal vocalization behavior is high, it indicates that there is abnormal vocalization in this aspect. For abnormal vocalization behavior, the duration of its occurrence is counted as T abnormal , then the specific calculation formula for the abnormal vocalization behavior duration ratio is as follows: If the duration of abnormal vocal behavior is high, it means that the abnormal vocal behavior accounts for a large proportion in the overall vocalization process. Based on the quantification of the patient's language characteristics and vocal abnormalities, it provides accurate data basis for the diagnosis of autism based on TCM theory.

Images