Body position recognition method, device and system based on artificial intelligence

Abstract

The invention discloses a body position recognition device, system and method based on artificial intelligence. The artificial intelligence-based body position recognition method comprises the following steps of: acquiring a body position data sequence of a subject during a BPPV (Brain Per Position Virus) displacement test; preprocessing the body position data sequence to obtain a standard data sequence to be input; inputting the standard data sequence into the trained body position recognition model; performing body position recognition through a body position recognition model, and recognizing each specific body position contained in the body position data sequence and the starting and ending time points of each specific body position; wherein the body position recognition model is a deep learning network model based on a time sequence. Based on the artificial intelligence technology, the body position information in the BPPV displacement test can be quickly and accurately recognized, and subsequent accurate diagnosis or resetting of the BPPV is facilitated.

Description

Technical Field

[0001] The present invention relates to the field of BPPV diagnosis and treatment, and in particular to a body position recognition method, device and system based on artificial intelligence. Background Art

[0002] In the diagnosis of benign paroxysmal positional vertigo (BPPV), observing the patient's eye movements in a specific position after a specific action is a necessary step in the diagnosis. The diagnosis and reduction of BPPV involves numerous body position actions. Each different method has a different specific position. Different methods may have the same specific position actions, making it a complex issue to accurately identify the diagnosis and reduction methods, as well as the detailed branch actions in each method and the starting and ending points of each branch action.

[0003] During the actual examination and reduction process, doctors often use head-mounted devices (with built-in gyroscopes) or swivel chairs to diagnose and reduce the patient's position. These devices will record the position of the head, but the axis data of the gyroscope and swivel chair, as well as the spatial correspondence, are inconsistent, making it difficult to directly identify the patient's specific body position information during diagnosis and reduction. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a body position recognition method, body position recognition device and system based on artificial intelligence. Specifically, the technical solution of the present invention is as follows:

[0005] In the first aspect, the present application discloses a body position recognition method based on artificial intelligence, comprising: obtaining a body position data sequence of a subject during a BPPV position change test; preprocessing the body position data sequence to obtain a standard data sequence to be input; inputting the standard data sequence into a trained body position recognition model; performing body position recognition through the body position recognition model to identify each specific body position contained in the body position data sequence, and the start and end time points of each specific body position; wherein the body position recognition model is a deep learning network model based on time series; and the body position recognition model uses a training data set composed of test data samples of different types of BPPV position change tests; wherein each test data sample in the training data set is a body position data sequence with timestamp information, and the body position data sequence with timestamp information is also marked with a body position label, and the start and end time points of each specific body position.

[0006] Preferably, the body position data sequence includes a head position data sequence of the subject; the preprocessing of the body position data sequence to obtain a standard data sequence to be input specifically includes: performing data conversion on the head position data sequence to obtain an Euler angle data sequence; performing isometric extraction on the Euler angle data sequence at set intervals to obtain a streamlined Euler angle data sequence; and normalizing or standardizing each data in the streamlined Euler angle data sequence to obtain a standard data sequence in a format acceptable to a body position recognition model.

[0007] Preferably, before the body position data sequence is preprocessed, the method further includes: performing change detection on the body position data sequence, and dividing the body position data sequence into a plurality of body position holding time periods and a plurality of body position change time periods according to the detection result; wherein the data points in each body position holding time period are equal, and the data points in each body position change data segment are not equal; and marking the subsequences corresponding to the body position holding time periods in the body position data sequence as specific body positions to be identified;

[0008] Or after preprocessing the body position data sequence, the method further includes:

[0009] Perform change detection on the standard data sequence, and divide the standard data sequence into a plurality of body position holding time periods and a plurality of body position change time periods according to the detection results; wherein the data points in each body position holding time period are the same, and the data points in each body position change data segment are different; and the subsequences corresponding to each specific body position holding time period in the standard data sequence are respectively marked as specific body positions to be identified.

[0010] Preferably, the model architecture of the body position recognition model includes: a number of convolutional neural networks, a time series neural network and / or an attention mechanism network, and a transposed output layer; wherein: the number of convolutional neural networks are used to extract the features of the input body position data sequence; the time series neural network and / or the attention mechanism network are used to perform front-to-back time association based on the features extracted by the number of convolutional neural networks, and further extract the time association features of the body position data sequence; the output layer is used to output the structured body position data corresponding to each data point in the body position data sequence.

[0011] Further preferably, the time series neural network includes any one of a bidirectional long short-term memory network, a recurrent convolutional neural network, a recurrent neural network, and a gated recurrent unit.

[0012] Preferably, after identifying each specific body position contained in the body position data sequence, and the start and end time points of each specific body position, it also includes: arranging each specific body position in sequence according to the start and end time of each specific body position to obtain a first body position change sequence; based on the body position change reference sequence of each type of BPPV position change test, determining the BPPV test type of the first body position change sequence.

[0013] Preferably, the body position recognition method further comprises: collecting video data of the subject's body position changes when doing the BPPV position change test through a camera; and after preprocessing the body position data sequence to obtain a standard data sequence to be input, further comprising: obtaining timestamp information of each data point of the standard data sequence; and based on the timestamp information, extracting video frames at corresponding time points from the video data to obtain a video frame sequence; and performing time alignment on the video frame sequence and the standard data sequence;

[0014] The model architecture of the body position recognition model includes: a multimodal input layer, which is used to receive the aligned standard data sequence and video frame sequence respectively and extract the features of the standard data sequence through a number of one-dimensional convolutional neural networks, and extract the features of the video frame sequence through a number of two-dimensional convolutional neural networks; a feature fusion layer, which is used to fuse the feature vector extracted from the standard data sequence and the feature vector extracted from the video frame sequence; wherein the feature fusion layer adopts an attention mechanism to dynamically adjust the feature weights of the standard data sequence and the video frame sequence; an output layer, which is used to output the structured body position data corresponding to each data point in the body position data sequence according to the fusion features output by the feature fusion layer.

[0015] Preferably, the body position recognition method also includes: collecting video data of the subject's body position changes when doing the BPPV position change test through a camera; and the body position recognition model includes a first recognition model and a second recognition model; the inputting of the body position data sequence into the trained body position recognition model for recognition specifically includes: inputting the standard data sequence into the first recognition model, identifying and outputting the body position label corresponding to each data point in the standard data sequence; and the start and end time points of each specific body position; the first recognition model is a deep learning network model based on time series; based on each specific body position and its corresponding start and end time points, extracting key video frame data from the body position video frame sequence; wherein the key video frame data carries timestamp information; inputting the key video frame data into the second recognition model, identifying and outputting the body position information of the key video frame data; the second recognition model is trained by a large number of picture samples labeled with different specific body positions; according to the recognition results of the image recognition model and the recognition results of the body position recognition model, determining each specific body position and the corresponding start and end time points included in the BPPV position change test.

[0016] In the second aspect, the present application also discloses an artificial intelligence-based body posture recognition device, comprising: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement any of the above-mentioned body posture recognition methods.

[0017] In the third aspect, the present application also discloses a body position recognition system based on artificial intelligence, including the above-mentioned body position recognition device; and: a body position data acquisition device, used to collect the body position data sequence of the subject during the BPPV position change test; the body position recognition device is configured to execute the steps of: receiving the body position data sequence; preprocessing the body position data sequence to obtain a standard data sequence to be input; inputting the standard data sequence into a trained body position recognition model; performing body position recognition through the body position recognition model to identify each specific body position contained in the body position data sequence, and the start and end time points of each specific body position; wherein the body position recognition model is a deep learning network model based on time series; and the body position recognition model uses a training data set composed of test data samples of different types of BPPV position change tests; wherein each test data sample in the training data set is a body position data sequence with timestamp information, and the body position data sequence with timestamp information is also marked with a body position label, and the start and end time points of each specific body position.

[0018] Preferably, the body position data acquisition device is a head-mounted device, and a motion sensor is provided in the head-mounted device.

[0019] Preferably, the body position data acquisition device is a BPPV swivel chair, which is used to perform a BPPV position change experiment on the subject according to the operator's instructions, and transmit the swivel chair rotation information to the body position recognition device in real time during the position change experiment.

[0020] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0021] 1. In the posture recognition scheme of the present application, a deep learning network model based on time series, the posture recognition model, is used for posture recognition. In addition to extracting the characteristics of the posture data itself, the front and back time correlation characteristics of each posture data in the posture data sequence can also be captured, so that each specific posture included in each BPPV position change test and its corresponding start and end time can be accurately identified. This provides an important reference for doctors to diagnose or reposition BPPV.

[0022] 2. In the posture recognition scheme of the present application, the data of the input model used is a preprocessed posture data sequence, which is not image data or video data, but comes from non-visual sensors, such as gyroscopes, accelerometers, etc. The amount of data in the posture data sequence is much smaller than that of conventional video data, and in the process of BPPV diagnosis or resetting, multiple BPPV displacement tests may be involved, and each BPPV test needs to observe the nystagmus in different specific postures, so the entire duration is relatively long. If the entire process is collected through video, the amount of data is even more amazing, and regardless of the accuracy of the recognition, the recognition efficiency alone is significantly lower than the scheme of the present application.

[0023] 3. The posture recognition scheme of the present application can be used to perform visualization after the posture is recognized. When displaying nystagmus information, the patient's posture change information based on the time dimension can also be displayed synchronously, which is more helpful for doctors to diagnose BPPV. Compared with the method of displaying the position curve, this scheme can intuitively reflect the specific posture of the head at each moment. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The preferred implementation modes will be described below in a clear and understandable manner with reference to the accompanying drawings to further illustrate the above-mentioned characteristics, technical features, advantages and implementation methods of the present invention.

[0025] Figure 1 This is a flow chart of a body position recognition method based on artificial intelligence provided in an embodiment of the present application;

[0026] Figure 2 is a schematic diagram of a right rear Dix-Hallpike test shown in an embodiment of the present application;

[0027] Figure 3 It is a schematic diagram of a model architecture of a body position recognition model based on artificial intelligence provided in an embodiment of the present application;

[0028] Figure 4 It is a schematic diagram showing an output result of body position recognition based on artificial intelligence provided in an embodiment of the present application;

[0029] Figure 5 It is a schematic diagram of a body position recognition process based on a multimodal data fusion model provided in an embodiment of the present application;

[0030] Figure 6 It is a schematic diagram of a multi-model based body posture recognition process provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings and other implementation methods can be obtained based on these drawings without creative work.

[0032] In order to simplify the drawings, only the parts related to the invention are schematically shown in each figure, and they do not represent the actual structure of the product. In addition, in order to simplify the drawings and facilitate understanding, in some figures, only one of the parts with the same structure or function is schematically drawn or marked. In this article, "one" not only means "only one", but also means "more than one".

[0033] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0034] In addition, in the description of the present application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0035] BPPV position test, also known as BPPV position test, is mainly used for the diagnosis or reduction of BPPV. Conventional BPPV position test include Roll-test, Dix-Hallpike test, EPLEY test, Bow and Leantest, Side-lying test, etc. Of course, the BPPV position test of this application is not limited to the above-mentioned tests. Any position change scheme used for BPPV diagnosis or reduction can be regarded as BPPV position test.

[0036] Usually, the BPPV displacement test can be diagnosed or relocated by manual diagnosis or relocation, or by using a BPPV swivel chair. In conventional manual diagnosis or relocation, the doctor manually controls the patient's head movement and observes the patient's nystagmus in a specific position to determine the type of BPPV, or after knowing which semicircular canal the patient has otolithiasis, performs the corresponding BPPV displacement test to relocate the patient's otoliths. This conventional manual diagnosis and relocation relies entirely on the doctor's naked eye observation and experience to make judgments. Sometimes subtle nystagmus or short-term nystagmus may be difficult to identify, and reference data such as nystagmus latency is also difficult to accurately obtain, so it requires a relatively high level of professionalism from the doctor. Therefore, on the basis of conventional manual diagnosis or relocation, if combined with a head-mounted video nystagmus-oculography device, the accuracy of diagnosis or relocation is greatly improved. Specifically, the video nystagmus-oculography device has a built-in camera that can be used to collect the patient's eye movement video, so that the doctor can repeatedly watch the nystagmus. However, the diagnosis or reduction of BPPV is not simply determined by the nystagmus collected by the video nystagmusgraph, but also needs to be combined with a specific body position. Therefore, the recognition and judgment of body position is also particularly important. When displaying nystagmus information, the patient's body position information is displayed synchronously, which is more helpful for doctors to diagnose BPPV. The present application aims to provide a body position recognition method based on artificial intelligence, a device for body position recognition, and a system.

[0037] In one embodiment, the reference specification Figure 1 The method for body position recognition based on artificial intelligence provided by the present invention comprises:

[0038] S100, obtaining the subject's body position data sequence during the BPPV position change test;

[0039] Specifically, the "acquisition" in this step can be understood as "receiving" or "collecting". This embodiment does not limit this, as long as the subject's body position data sequence during the BPPV position change test is obtained. Since the subject's body position will change with the change of time in the BPPV position change test, the acquired body position data is a time-based body position data sequence, and further, the body position data sequence is a user's head position data sequence. It is worth noting that the body position data sequence is derived from a non-visual sensor and does not include any video or image data collected by a camera or other visual acquisition device. That is to say, the body position data sequence here is not the video data or image data of the subject's body position collected by a visual sensor. The body position data sequence can be the wearer's body position information collected by a motion sensor, such as the user's head motion sensor data collected by a head-mounted device with a built-in gyroscope worn on the user's head, or the user's head position change data collected by a wearable device equipped with a three-axis / six-axis angle sensor. Or the swivel chair axis data transmitted by the BPPV swivel chair (the subject sits on the swivel chair, is relatively fixed to the swivel chair, and the rotation of the swivel chair drives the user's body position change).

[0040] S200, preprocessing the body position data sequence to obtain a standard data sequence to be input;

[0041] Specifically, after obtaining the body position data sequence during BPPV diagnosis and treatment, it is necessary to perform some preprocessing on these body position data sequences. For example, some models have requirements for the input data format, so the collected body position data sequence needs to be processed during preprocessing to meet the format requirements of the model input. In addition, in order to improve the recognition ability of the model, after the body position data sequence is collected, some preprocessing can also be performed on it to improve the later recognition accuracy or speed, such as denoising, such as streamlining data, etc. This embodiment does not limit this.

[0042] S300, inputting the standard data sequence into the trained body position recognition model;

[0043] S400, performing body position recognition through a body position recognition model to recognize each specific body position included in the body position data sequence, and the start and end time points of each specific body position;

[0044] Among them, the body position recognition model is a deep learning network model based on time series; and the body position recognition model adopts a training data set composed of test data samples of different types of BPPV position change tests; wherein each test data sample in the training data set is a body position data sequence with timestamp information, and the body position data sequence with timestamp information is also marked with a body position label, as well as the start and end time points of each specific body position.

[0045] Specifically, since the diagnosis and reduction of BPPV is a continuous process, the change of body position is not achieved overnight, and there are connections between the before and after. Therefore, the body position recognition model used in this embodiment is a deep learning network model based on time series. The time series deep learning network model is a model that uses deep learning technology to process time series data. A time series is a series of data points arranged in chronological order, and a deep learning model can effectively capture the associations or trends in time series data. When training the body position recognition model, the training samples used are body position data sequences with timestamps and marked with specific body position labels and the start and end time points of each specific body position.

[0046] After the body position recognition model recognizes the input body position data sequence, it outputs the specific body positions contained in the sequence, as well as the start and end time points of each specific body position. When diagnosing BPPV, it is necessary to observe nystagmus in a specific body position after a specific action to assist in diagnosis, and which semicircular canal has otoliths also needs to be comprehensively judged based on the specific body position and nystagmus. Therefore, when diagnosing or resetting BPPV, in addition to the collection and recognition of nystagmus data, the recognition of body position is also very important.

[0047] The holding time of a specific body position must reach a set time threshold. The time thresholds corresponding to different specific body positions can be the same or different, and can be set specifically according to actual conditions. A specific body position is generally a body position maintained after a specific action. There are also different types of specific body positions, such as supine position, left side position, right side position, etc., and even the same specific body position may appear multiple times in a BPPV test, and the body position duration may be the same or different. Therefore, when performing body position recognition, it is necessary not only to identify the body position type of a specific body position, but also to identify the start time and end time of the specific body position, that is, the start and end time points.

[0048] Another method embodiment of the present application, based on the above embodiment, the body position recognition model includes: a number of convolutional neural networks, time series neural networks and / or attention mechanism networks, and a transposed output layer; wherein:

[0049] Several convolutional neural networks are used to extract features of the input body position data sequence;

[0050] A time series neural network and / or an attention mechanism network is used to perform temporal association based on the features extracted by several convolutional neural networks, and further extract the temporal association features of the body position data sequence;

[0051] The output layer is used to output the structured body position data corresponding to each data point in the body position data sequence.

[0052] Self-attention networks, such as Transformer, can also be used for sequence data processing. They use self-attention to capture the dependencies between different positions in the input sequence.

[0053] The time series neural network in the above embodiments includes but is not limited to any one of a bidirectional long short-term memory network, a recurrent convolutional neural network, a recurrent neural network, and a gated recurrent unit.

[0054] Specifically, Recurrent Neural Networks (RNNs): RNNs are a special class of neural networks designed to process sequence data. RNNs have a cyclic structure that allows information to be passed from one time step to the next. This allows RNNs to take into account previous information when processing time series data.

[0055] Bidirectional Long Short-Term Memory Network (LSTM): LSTM is an improved RNN variant. The LSTM unit includes an input gate, a forget gate, and an output gate, which can effectively capture long-term dependencies.

[0056] Gated Recurrent Unit (GRU): GRU is another improved RNN variant, similar to LSTM but with a more simplified structure.

[0057] Recurrent Neural Network (RNN): Recurrent Neural Networks are a type of deep learning model that works on tree-like structures or graph data. They are suitable for certain time series data with hierarchical structures.

[0058] Another method embodiment of the present application, based on any of the above embodiments, further includes before or after preprocessing the body position data:

[0059] Perform change detection on the data sequence before or after preprocessing, and divide the data sequence into multiple body position holding time periods and multiple body position change time periods according to the detection results; wherein the data points in each body position holding time period are equal, and the data points in each body position change data segment are not equal; and mark the subsequences corresponding to the body position holding time periods in the data sequence as specific body positions to be identified. Preferably, mark the subsequences corresponding to the body position holding time periods whose holding time reaches a set time threshold as specific body positions to be identified.

[0060] In this embodiment, whether it is change detection and marking of specific body positions to be identified on the data sequence before preprocessing, or change detection and marking of specific body positions to be identified on the data sequence after preprocessing, it is for the purpose of dividing the data sequence of the input model into body positions, and marking the body positions that are helpful for the diagnosis and treatment of BPPV (the body positions corresponding to the body position holding stage) as specific body positions, so that the body position recognition model can focus on the data of these specific body positions during recognition, thereby accurately identifying the specific body positions in the BPPV position change test and their corresponding start and end time points, providing a reference for the diagnosis or reduction of BPPV.

[0061] The specific body position in this embodiment refers to a maintained body position, further, a body position maintained for a set time threshold, and more preferably, a body position maintained after a specific action. Taking the right posterior Dix-Hallpike test as an example, the specific steps of the manual diagnosis or reduction operation are as follows:

[0062] like Figure 2 As shown, doctors usually ask patients to sit on the examination bed. The doctor stands in front of or behind the patient, holds the patient's head with both hands, rotates it 45 degrees to the right, and maintains this position (hereinafter referred to as the 45-degree right turn position). Then the patient is quickly changed to a supine position, with the head hanging back outside the bed, 30 degrees from the horizontal plane, and the head position is maintained at 45 degrees to observe whether there is vertigo or nystagmus. At this time, this position just makes the posterior semicircular canal in the plane pulled by gravity, and the otoliths attached to the top of the ampulla ridge or floating on the long arm of the semicircular canal will move and cause vertigo and nystagmus. Because there is a latent period for the occurrence of vertigo and nystagmus, this position should be maintained for 30 seconds during the examination. Patients with posterior semicircular canal BPPV often induce vertigo and nystagmus when the affected ear is facing downward, and patients with anterior semicircular canal BPPV can induce vertigo and nystagmus when the affected ear is facing upward. Then the patient slowly returns to a sitting position.

[0063] During the above test, the patient's head rotated 45 degrees to the right and the position remained unchanged. Therefore, the 45-degree right position is a specific position. More importantly, the dizziness and nystagmus are mainly observed in the hanging-supine position (the head hangs backward outside the bed, 30 degrees from the horizontal plane, and the head position must be maintained at 45 degrees). The patient needs to maintain this position for a certain period of time. Therefore, the hanging-supine position is a specific position. In addition, the 45-degree right position is a position maintained after the specific action of rotating the head to the right. The hanging-supine position is also a position maintained after lying down quickly on the basis of the 45-degree right position. Therefore, the 45-degree right position and the hanging-supine position are both specific positions, and there is a time sequence between these two specific positions. The latter is formed after a specific action on the basis of the former. This further illustrates that when using a neural network model, a deep learning network model based on time series must be used, so that time series can be processed and the association of time features before and after in the data series can be identified.

[0064] Therefore, in the above-mentioned right posterior Dix-Hallpike test, the 45-degree right turn position and the hanging-supine position can both be regarded as specific positions; the left posterior Dix-Hallpike is similar to it. Unlike the right posterior Dix-Hallpike, the doctor first rotates the patient's head 45 degrees to the left, and then maintains this position for a period of time (this position is called the 45-degree left turn position). The subsequent operations are similar to the right posterior. The specific positions of the left posterior Dix-Hallpike are the 45-degree left turn position and the hanging-supine position.

[0065] Another method embodiment of the present application, based on any of the above embodiments, further includes the steps of:

[0066] S500, post-processing the recognition result output by the body position recognition model to obtain body position change information of the body position data sequence in the time dimension.

[0067] After obtaining the specific body position label output by the body position recognition model, it is necessary to perform post-processing based on the recognition output results to obtain complete body position change information. Specifically, for example, each specific body position is arranged in order according to the start and end time of each specific body position to obtain the first body position change sequence; then, based on the body position change reference sequence of each type of BPPV position change test, the BPPV test type of the first body position change sequence is determined.

[0068] Another example is to visualize the recognition results output by the posture recognition model to show the posture changes of the posture data sequence in the time dimension. The specific implementation steps are as follows:

[0069] Identify the specific body position label output by the body position recognition model, and obtain the body position label corresponding to each data point in the standard data sequence;

[0070] After the specific body position recognition of all data points is completed, the standard data sequence for the specific body position recognition is converted back to the body position data sequence of the original time length by reverse data simplification;

[0071] According to the start and end time points of each specific body position identified by the body position recognition model, the body position labels corresponding to the unidentified data points located between the start and end time points of each specific body position in the restored body position data sequence are determined; the body positions of the data points located between the start and end time points of each specific body position are all corresponding specific body positions.

[0072] The body position recognition results of the body position data sequence are visualized, and the body position change information of the motion data in the time dimension is displayed according to the identified specific body positions and their start and end time points.

[0073] In addition, in another embodiment, the body position label may include a change body position label in addition to the labels of each specific body position (each specific body position corresponds to a body position label). Since a specific body position is generally a maintained body position formed after a specific action, the change body position label can be used to mark the body position data points corresponding to the body position change stage, indicating that the body position corresponding to these data points is a change body position, rather than a maintained body position (specific body position). In this way, after the standard data is restored to the body position data sequence of the original time length, the body position corresponding to the data points within the start and end time points of each identified specific body position is the corresponding specific body position, and the body position corresponding to the data points outside the start and end time points of the specific body position is a change body position.

[0074] Another embodiment of the body position recognition method of the present application includes the following steps:

[0075] 1. Data acquisition:

[0076] There are many ways to obtain the body position data sequence. For example, the head position data of the subject can be collected through a head-mounted device with a built-in motion sensor worn by the subject, such as a video nystagmus with a built-in gyroscope. When the subject wears the video nystagmus during the BPPV test, the head position data sequence based on time changes can be obtained as the body position data sequence.

[0077] Of course, the rotation information of the BPPV swivel chair can also be obtained through the BPPV swivel chair. The rotation information includes the rotation axis information of the BPPV swivel chair, that is, the position information of the swivel chair. Since the subject sits on the swivel chair and the subject's head is kept still by the head holding device of the swivel chair, the subject's position information can be obtained through the position information of the swivel chair.

[0078] This embodiment does not limit the method for acquiring the body position data sequence.

[0079] 2. Data preprocessing:

[0080] First, the received posture data sequence needs to be converted into a format acceptable to the model. Let's take gyroscope data as an example. Gyroscope data is usually time series data that contains information about the head position. The preprocessing steps include but are not limited to:

[0081] ①Format conversion: Convert the body position data in the body position data sequence into Euler angles. Take the quaternion data obtained by the gyroscope as an example. A quaternion has four components (x, y, z and w). Convert the quaternion into Euler angles (also called attitude angles) and display them using pitch, yaw and roll:

[0082] roll(X-axis rotation) = atan2(2*(wx+yz),1-2*(xx+yy))

[0083] pitch(Y-axis rotation) = asin(2*(wy-zx))

[0084] yaw (Z-axis rotation) = atan2 (2*(wz+xy), 1-2*(yy+zz))

[0085] Among them, atan2() is the four-quadrant inverse tangent function (arc tangent function), which can return the angle value between -pi and pi, and takes into account the signs of the denominator and numerator. In addition, asin is the inverse sine function.

[0086] ② Data simplification: This data simplification is not a necessary preprocessing step, but if the sampling rate of the body position acquisition device is high, it is a more optimized solution to simplify the collected body position data during preprocessing. Specifically, the method of equal interval extraction is adopted to perform equal interval sampling from the body position data sequence (or Euler angle data sequence), simplify the data, but will not lose key information, and obtain the simplified data sequence.

[0087] Of course, if data simplification is not performed in data preprocessing, the sliding window technology of the model can also be used to perform time window sliding cutting on the input data sequence. Specifically, the sliding window technology is used to create overlapping windows in the input standard data sequence, and each time window is continuously identified, thereby achieving continuous body position recognition.

[0088] ③Standardization or normalization processing: Normalize or standardize the data so that the data is returned to the range of 0 to 1 or -1 to 1 to improve the model training effect.

[0089] X_norm = (X - X_min) / (X_max - X_min), where X is the original data, X_min and X_max are the minimum and maximum values ​​of the data respectively, and X_norm is the processed data.

[0090] X_std=(X-μ) / σ, where X is the original data, μ and σ are the mean and standard deviation of the data respectively, and X_std is the processed data.

[0091] 3. Model Identification

[0092] After preprocessing the body position data sequence, a standard data sequence that conforms to the model input format can be obtained, and then input into the trained body position recognition model to perform specific body position recognition, as well as the start and end time points corresponding to each specific body position. When it comes to the body position recognition model, it is necessary to briefly explain the architecture design and model training of the body position recognition model, as follows:

[0093] ① Regarding the architectural design of the posture recognition model, the posture recognition model needs to process time series data, so it must be a deep learning network model based on time series, and it needs to output multiple types of posture labels and the start and end time information of specific posture labels. Therefore, the loss function used by the posture recognition model includes a multi-label classification loss function and a timestamp loss function to accurately predict the posture label and time information. The following is an example of the architecture of a posture recognition model:

[0094] The body position recognition model includes three one-dimensional convolutional neural networks, a bidirectional LSTM layer, and a one-dimensional convolutional transpose layer. Figure 3 As shown in the figure, N one-dimensional convolution blocks are mainly used to extract features from the input data sequence, and the Bi-LSTM layer is mainly used to capture the time correlation in the data sequence. N deconvolution blocks are used to reversely restore the original data volume, and the final fully connected layer FC is used to output the recognition results.

[0095] ② Training and evaluation of posture recognition model

[0096] Sample collection: First, we need to collect data samples of various types of BPVV position change tests to form a training data set; each test data sample in the training data set is a position data sequence with timestamp information, and the position data sequence with timestamp information is also marked with a position label and the start and end time points of each specific position. Different specific positions correspond to different position labels.

[0097] Model training and validation: Use samples in the training dataset to train the body position recognition model, and use the validation set to adjust hyperparameters to avoid overfitting.

[0098] ③About the deployment and application of the model

[0099] Once training and validation are completed and the performance of the posture recognition model reaches a satisfactory level, such as the accuracy reaches a set threshold, the model can be deployed to actual applications or systems for real-time recognition of the patient's BPPV posture.

[0100] 4. Post-processing of output results

[0101] After obtaining the output results, you can further convert the output results into interpretable content for visual display. The specific examples are as follows:

[0102] ①Convert the model prediction result label to the corresponding specific body position:

[0103] The body position classification result label of each time node is converted into the corresponding body position classification;

[0104] Each segment of the same category is a same specific body position, and the corresponding starting and ending points are the starting and ending points of the specific body position.

[0105] ②Integrate all specific posture results.

[0106] ③According to the correspondence between different inspection and reduction actions and specific body positions, obtain the specific method of the current inspection or reduction.

[0107] ④ Convert the data back to the original data time dimension: Restore the data back to the original time length by reversing the aforementioned data reduction steps.

[0108] ⑤Show the final prediction results. Taking the supine rolling test as an example, the output results after recognition are shown as follows Figure 4 As shown. The horizontal axis represents time, and the vertical axis represents the deflection angle; the blue line represents the deflection angle component of the subject on the swivel chair axis 1 (pitch axis of the swivel chair, horizontal direction) that changes with time; the green line represents the up and down flip angle component of the subject on the swivel chair axis 2 (Yaw axis of the swivel chair, vertical direction) that changes with time. In addition, different specific body positions can not only be displayed through text, but also divided into intervals through different colors, intuitively displaying different specific body positions and corresponding time ranges.

[0109] Another embodiment of the body posture recognition method embodiment of the present application, based on any of the above embodiments, adds video or image data collected by a visual sensor, and performs body posture recognition through a multimodal fusion model to improve the accuracy and efficiency of recognition.

[0110] Specifically, in the data acquisition stage, in addition to acquiring the body position data sequence acquired by the non-visual sensor, the visual sensor (camera) is also used to collect video or image data of the subject's body position changes, and after preprocessing the body position data sequence to obtain the standard data sequence to be input, it also includes:

[0111] Obtaining timestamp information of each data point of the standard data sequence; and based on the timestamp information, extracting the video frame of the corresponding time point from the video data to obtain a video frame sequence;

[0112] Temporally align a video frame sequence with a standard data sequence.

[0113] Finally, the time-aligned video frame sequence and the standard data sequence are taken as input together and input into the trained body position recognition model to identify the specific body positions and corresponding start and end time points contained in the BPPV test, and further identify the type of BPPV test.

[0114] It is worth noting that the posture recognition model in this embodiment is different from the posture recognition model in any of the previous embodiments. The posture recognition model in this embodiment is actually a multimodal fusion model. The input data of this model includes two types of data, one is the posture data sequence collected by non-visual sensors, and the other is the video or image data collected by visual sensors. These two types of data are different, so the architecture of this posture recognition model is different from the posture recognition model architecture in any of the previous embodiments. Specifically, the model architecture of the posture recognition model in this embodiment includes:

[0115] A multimodal input layer, for respectively receiving the aligned standard data sequence and the video frame sequence and extracting features of the standard data sequence through a plurality of one-dimensional convolutional neural networks, and extracting features of the video frame sequence through a plurality of two-dimensional convolutional neural networks;

[0116] The feature fusion layer is used to fuse the feature vectors extracted from the standard data sequence and the feature vectors extracted from the video frame sequence. The feature fusion layer uses an attention mechanism to dynamically adjust the feature weights of the standard data sequence and the video frame sequence.

[0117] The output layer is used to output the structured body position data corresponding to each data point in the body position data sequence according to the fusion features output by the feature fusion layer.

[0118] In another embodiment of the body position recognition method of the present application, it is also necessary to collect video or image data of the subject's body position changes through a visual sensor (such as a camera), but the difference between this embodiment and the previous embodiment is that the body position recognition model in this embodiment is not a multimodal fusion model, but is divided into two sub-body position recognition models: a first recognition model and a second recognition model. Among them, the first recognition model is used to identify the body position in the body position data sequence. After the specific body position and the start and end time points in the body position data sequence are recognized by the first recognition model, the specific body position and its start and end time points are determined based on the recognition result of the first recognition model. The relevant content of the judgment of the specific body position has been described in the previous embodiment, and it will not be repeated here. Then, based on each specific body position and its corresponding start and end time points, the key video frame data is extracted from the body position video frame sequence; wherein, the key video frame data has timestamp information; then, the key video frame data is input into the second recognition model to identify the body position information of the key video frame data; the second recognition model is trained by a large number of picture samples labeled with different specific body positions. Finally, the specific body positions and corresponding start and end time points included in the BPPV position change test are determined based on the recognition results of the image recognition model and the body position recognition model. In other words, the accuracy of the recognition results of the first recognition model is verified by the recognition results of the second recognition model.

[0119] Based on the same technical concept, the present application also discloses a body position recognition device based on artificial intelligence, comprising:

[0120] one or more processors;

[0121] A storage device for storing one or more programs;

[0122] When one or more programs are executed by one or more processors, the one or more processors implement the steps of the body position recognition method of any of the above method embodiments.

[0123] For example, the one or more processors implement the following steps of the body position recognition method:

[0124] Obtaining the subject's body position data sequence during the BPPV position change test;

[0125] Preprocessing the body position data sequence to obtain a standard data sequence to be input;

[0126] Input the standard data sequence into the trained body position recognition model;

[0127] Performing body position recognition through a body position recognition model to recognize each specific body position contained in the body position data sequence, as well as the start and end time points of each specific body position;

[0128] Among them, the body position recognition model is a deep learning network model based on time series; and the body position recognition model adopts a training data set composed of test data samples of different types of BPPV position change tests; wherein each test data sample in the training data set is a body position data sequence with timestamp information, and the body position data sequence with timestamp information is also marked with a body position label, as well as the start and end time points of each specific body position.

[0129] In another embodiment of the body position recognition device of the present application, the steps of the body position recognition method implemented by the processor can be specifically referred to any of the aforementioned method embodiments, and will not be described again here to reduce repetition.

[0130] In addition, the present application also discloses a body position recognition system based on artificial intelligence, which includes a body position data acquisition device and a body position recognition device in any of the above embodiments.

[0131] The body position recognition device includes at least one data processor and is configured to perform the following steps:

[0132] receiving a body position data sequence;

[0133] Preprocessing the body position data sequence to obtain a standard data sequence to be input;

[0134] The standard data sequence is identified by the trained body position recognition model, and each specific body position contained in the body position data sequence is output; as well as the start and end time points of each body position;

[0135] Among them: the body position recognition model is a deep learning network model based on time series; and the body position recognition model uses a training data set composed of test data samples of different types of BPPV position change tests; wherein each test data sample in the training data set includes at least: a body position data sequence, each specific body position contained in the body position data sequence, and the start and end time points of each specific body position.

[0136] Specifically, in one embodiment of a body position recognition system, the body position data acquisition device in the body position recognition system is a head-mounted device, and a motion sensor, such as a gyroscope, is provided in the head-mounted device to collect the head position information of the wearer. After the subject wears the head-mounted device, the BPPV displacement test is performed. Then, the head-mounted device is used to collect the subject's head position information based on time changes during the BPPV displacement test, thereby obtaining a body position data sequence (i.e., a gyroscope data sequence). Of course, the subject's body position data sequence can also be collected by other means, such as a VR head-mounted device. Preferably, the head-mounted device is a video nystagmus, which has a built-in gyroscope and a camera, wherein the camera is used to collect the subject's eye movement video, and the gyroscope is used to collect the subject's head body position data, including but not limited to position change data, acceleration, angular velocity, etc.

[0137] In another implementation, the body position data acquisition device in the body position recognition system is a BPPV swivel chair, which is used to perform a BPPV position change experiment on the subject according to the operator's instructions, and transmit the swivel chair position information to the body position recognition device in real time during the position change experiment. The BPPV swivel chair is provided with a head position holding device, which is used to keep the head of the subject sitting on the swivel chair fixed during the BPPV position change experiment, that is, the subject's head and the swivel chair do not produce relative movement, and the subject's entire body, especially the head, moves with the movement of the swivel chair, and the subject's head is fixed at a fixed position on the swivel chair by the head holding device. Therefore, since the relative position of the head position and the swivel chair is fixed, the subject's body position information, or the subject's head position information, can be obtained by obtaining the swivel chair position information transmitted by the BPPV swivel chair.

[0138] After obtaining the body position data sequence, the body position recognition device needs to preprocess the body position data sequence to make it a data sequence that conforms to the model input format. Specifically, the preprocessing of the body position data sequence includes but is not limited to the following processing:

[0139] (1) Data format conversion processing: The body position data sequence is converted to obtain the Euler angle data sequence. Specifically, the body position data collected by the body position data acquisition device is generally position data. For example, the position information format collected by the gyroscope is in the quaternion data format. Because the BPPV position change test involves many body position (mainly head position) deflection changes, the Euler angle data format is more conducive to body position recognition.

[0140] (2) Data simplification and extraction processing: extract the Euler angle data sequence after format conversion at equal intervals. Specifically, since the sampling frequency of the body position data acquisition equipment is generally high and the body position does not change much in a short period of time, if all the collected body position data are used as input, too much redundant and repeated data may be input, which is not conducive to the training or recognition of the neural network model.

[0141] (3) Data normalization: The Euler angle data sequence is normalized or standardized so that the data falls within the range of 0 to 1, or -1 to 1, thereby improving the training effect of the neural network model.

[0142] It is worth noting that the above-mentioned data simplification and extraction preprocessing is not necessary, but an optimization solution. It is also possible to perform data format conversion and data standardization processing instead of data simplification and extraction processing.

[0143] In another embodiment of the body position recognition system, in order to further improve the training effect of the body position recognition model and improve the recognition accuracy and efficiency of the body position recognition model, the input data of the model can be further marked, so that the model pays special attention to the marked data subsequences to improve the recognition accuracy. Specifically, any of the following implementation methods can be adopted:

[0144] (1) Implementation before preprocessing: Perform change detection on the body position data sequence, divide the body position data sequence into multiple body position holding time periods and multiple body position change time periods according to the detection results, and then mark the subsequences corresponding to the body position holding time periods in the body position data sequence as specific body positions to be identified; wherein the data points in each body position holding time period are equal, and the data points in each body position change data segment are not equal;

[0145] (2) Implementation after preprocessing: Perform change detection on the standard data sequence, divide the standard data sequence into multiple body position holding time periods and multiple body position change time periods according to the detection results, and mark the subsequences corresponding to each specific body position holding time period in the standard data sequence as specific body positions to be identified; wherein the data points in each body position holding time period are the same, and the data points in each body position change data segment are different.

[0146] Regardless of whether the change detection is performed before or after preprocessing, the marked posture holding time period is a specific posture to be identified, so that the posture recognition model can pay more attention to the marked specific posture for identification. It should be noted that when the standard data sequence or the standard data sequence is divided into multiple posture holding time periods and posture change time periods, the judgment is mainly based on the data values ​​of adjacent data points. Specifically, in the posture holding time period, the posture of the subject theoretically remains unchanged (if the posture data sequence collected by the head-mounted device, to be precise, the subject's head position remains unchanged). Of course, in actual situations, even in the posture holding stage, the subject cannot remain completely still, and there may be some very slight displacement changes. Therefore, the equality of each data point in the posture holding time period means that the difference between any adjacent data points in these data points is within the allowable set error range. The posture change stage refers to the stage when the user changes from one specific posture to another specific posture, such as the stage of lying down, the stage of turning to the left, etc. In the posture change stage, theoretically, the data points in this stage are different (the difference between adjacent data points is greater than the set error range) and change. Therefore, the above scheme can be used to divide the posture data sequence or the standard data sequence into subsequences to obtain several posture maintenance stages and several posture change stages, and the posture maintenance stages and the posture change stages are interspersed and staggered, and each posture maintenance stage is followed by a posture change stage.

[0147] Since the body position maintaining stage and the body position changing stage are relatively easy to distinguish, a simple judgment can be made from the difference between adjacent data points. The body position changing stage is not a special focus of BPPV body position recognition. The specific body position that is focused on mainly refers to the body position corresponding to the body position maintaining stage. Therefore, before the model is input, a simple change detection process can be performed to divide the body position data sequence or standard data sequence into stages, and the body position maintaining stage that needs to be identified is marked as the specific body position to be identified.

[0148] After the body position data sequence is preprocessed as above, it is input into the trained body position recognition model to recognize and output the specific body position and its corresponding start and end time points.

[0149] For the description of the training and architecture of the body position recognition model, please refer to the description of the body position recognition model in the implementation of the previous method. In order to reduce repetition, it will not be repeated here.

[0150] The body position recognition device recognizes each specific body position contained in the body position data sequence and the start and end time points of each specific body position through a body position recognition model.

[0151] In another embodiment of the body position recognition system, based on any of the above system embodiments, after the body position recognition device recognizes each specific body position and its corresponding start and end time points through the body position recognition model, it will further post-process the output results. Including but not limited to any one or more of the following:

[0152] (1) Identification of BPPV position test type.

[0153] After obtaining each specific body position and its corresponding start and end time points, each specific body position is arranged in the order of the start and end time points to obtain a first body position change sequence; then based on the pre-stored body position change reference sequences of various types of BPPV position change tests, the BPPV test type of the first body position change sequence is determined, so as to know which position test or tests of the BPPV test currently being performed.

[0154] (2) Visualization processing to display the posture change information of the posture data series in the time dimension. Specifically including:

[0155] Identify the body position label output by the body position recognition model, and obtain the body position corresponding to each data point in the standard data sequence;

[0156] After completing the body position recognition of all data points of the standard data sequence, the body position recognition result of the Euler angle data sequence corresponding to the standard data sequence is obtained, and the Euler angle data sequence after recognition is converted back to the Euler angle data sequence of the original time length by reverse data simplification;

[0157] According to the start and end time points of each specific body position identified by the body position recognition model, the specific body position corresponding to the data point that has not been identified after restoration is determined;

[0158] The restored Euler angle data sequence is visualized, and the subject's posture change information in the time dimension is displayed based on the identified start and end time points of each specific posture.

[0159] Figure 4The output of the model is visualized to show a schematic diagram of the body position change information in the time dimension when the subject is doing a rolling test. In contrast, the three-dimensional head model needs to be played on the time axis to show the head position of the entire inspection or reset process. If there is no serial action, it is difficult to make an accurate interpretation of the head position of the entire inspection or reset sequence at a specific moment. By adopting the solution of this embodiment, an inspection or reset action sequence can be provided at any specified time or different time intervals, and the specific actions / positions in the action sequence can be specifically given. The demarcation of the specific time intervals for all body positions provides an important basis for doctors to quantitatively analyze the characteristics of the subject's nystagmus latency and so on, thereby providing doctors with more clues for disease diagnosis.

[0160] In another embodiment of the body position recognition system, in order to make the recognition more accurate, different types of data can be input into the body position recognition model, thereby providing the model with more comprehensive information, which helps to more accurately identify the BPPV body position. Specifically, the input body position data sequence is not the body position data acquired by a single type of body position acquisition device, but the body position data acquired by different body position acquisition devices. Specifically, for example, in addition to collecting the subject's head position data sequence through the gyroscope in the head-mounted device, the camera is also used to collect a video frame sequence of the subject's body position changes during the BPPV position change experiment. After obtaining the two-dimensional body position data information, the body position recognition model is used for recognition processing. The following are two examples:

[0161] (1) Multimodal data fusion.

[0162] Specifically, Figure 5 As shown, the user's head position data sequence is collected through the built-in motion sensor of the head-mounted device (such as a gyroscope or accelerometer, etc.), and the head position data sequence is preprocessed (the preprocessing process can be referred to the content in the previous embodiment), so that the reference time point of each data point in the streamlined Euler angle data sequence or the standard data sequence can be obtained, and then the corresponding video frame is extracted from the video collected by the camera based on the reference time point to form a corresponding video frame sequence, and then the video frame sequence and the standard data sequence are time-aligned and input into the multimodal body position recognition model. The model architecture of the multimodal body position recognition model includes: a multimodal input layer, a feature fusion layer, and an output layer. Specifically as follows:

[0163] A multimodal input layer, for respectively receiving the aligned standard data sequence and the video frame sequence and extracting features of the standard data sequence through a plurality of one-dimensional convolutional neural networks, and extracting features of the video frame sequence through a plurality of two-dimensional convolutional neural networks;

[0164] The feature fusion layer is used to fuse the feature vectors extracted from the standard data sequence and the feature vectors extracted from the video frame sequence. The feature fusion layer uses an attention mechanism to dynamically adjust the feature weights of the standard data sequence and the video frame sequence.

[0165] The model output layer is used to output the structured body position label corresponding to each data point, as well as the start and end time points of each specific body position.

[0166] The multimodal body position recognition model in this embodiment can extract the features of the standard data sequence through a one-dimensional convolutional neural network, extract the features of the video frame sequence through a two-dimensional convolutional neural network, and then perform feature vector fusion through a feature fusion layer. More importantly, the feature fusion layer also uses an attention mechanism to adjust the feature weights, thereby improving the efficiency and accuracy of recognition. For example, in this embodiment, the input gyroscope data and visual image data are fused together to provide multimodal information, which helps to increase the accuracy of the model. By combining different data sources, possible errors or deficiencies in gyroscope data can be compensated.

[0167] (2) Multiple model evidence

[0168] Specifically, Figure 6 As shown, the body position recognition model in this embodiment includes a first recognition model and a second recognition model; wherein the first recognition model is used to identify the body position label corresponding to each data point and the start and end time points of a specific body position according to an input head position sequence, and the first recognition model is a deep learning network model based on time series.

[0169] In this embodiment, based on the start and end time points of the specific body position output by the first recognition model, the corresponding video frames in the body position video can be extracted based on the start and end time points of each specific body position to form key video frame data with timestamp information, and then the key video frame data is input into the second recognition model to identify the specific body position information and the corresponding start and end time points contained in the key video frame data. Finally, based on the output results of the first recognition model and the second recognition model, the specific body position and the corresponding start and end time points contained in this BPPV position change test are determined.

[0170] In this embodiment, although the body position recognition result can be obtained only by the first recognition model, in order to verify the accuracy of the recognition result of the first recognition model, the second recognition model is used to recognize the body position video of the subject taken during the BPPV position change test, and the body position information recognized by the second recognition model is used to determine whether the recognition result of the first recognition model is accurate. Specifically, if the recognition results of the first recognition model and the second recognition model are consistent, it means that the body position recognition result is accurate. If the two results are inconsistent, the inconsistent body position time period is further determined and sent to the user end (such as the doctor end) for judgment. It is worth noting that in this implementation, not all the video frame data taken are input into the second recognition model, but based on the start and end time points of each specific body position in the output result of the first recognition model, the video frame data in the corresponding time period is locked to obtain the key video frame data with timestamp information, and finally all the key video frame data with timestamp information are input into the second recognition model for recognition output. In this embodiment, the input video frame of the second recognition model is determined based on the output result of the first recognition model, thereby removing a lot of redundant video information. The second recognition model is mainly used to verify whether the specific body position recognized by the first recognition model is accurate. Therefore, it is only necessary to use the video frame data within the specific body position time period for image recognition for verification. In addition, the first recognition model and the second recognition model process different types of data, so the model architectures of the two models are also different. In this embodiment, the model architectures of the first recognition model and the second recognition model are not limited. The neural network model (first recognition model) that can be used to process time series data in the prior art, and the neural network model (second recognition model) that can be used to process image data are all acceptable.

[0171] The system embodiments, device embodiments and method embodiments in the present application correspond to each other, and the technical details of the method embodiments in the present application are also applicable to the device embodiments and system embodiments, and vice versa. To reduce repetition, they are not described again.

[0172] It should be noted that the above embodiments can be freely combined as needed. The above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A body position recognition method based on artificial intelligence, characterized in that: include: Obtaining the subject's body position data sequence during the BPPV position change test; Preprocessing the body position data sequence to obtain a standard data sequence to be input; Inputting the standard data sequence into a trained body position recognition model; Performing body position recognition by using the body position recognition model to recognize each specific body position included in the body position data sequence, and the start and end time points of each specific body position; Among them, the body position recognition model is a deep learning network model based on time series; and the body position recognition model adopts a training data set composed of test data samples of different types of BPPV position change tests; each test data sample in the training data set is a body position data sequence with timestamp information, and the body position data sequence with timestamp information is also marked with a body position label, as well as the start and end time points of each specific body position.

2. The body position recognition method according to claim 1, characterized in that: The body position data sequence includes a head position data sequence of the subject; The preprocessing of the body position data sequence to obtain a standard data sequence to be input specifically includes: Performing data conversion on the head position data sequence to obtain an Euler angle data sequence; Extracting the Euler angle data sequence in equal length according to a set interval to obtain a streamlined Euler angle data sequence; Each data in the simplified Euler angle data sequence is normalized or standardized to obtain a standard data sequence in a format acceptable to the body position recognition model.

3. The body position recognition method according to claim 1, characterized in that: Before preprocessing the body position data sequence, the method further includes: Performing change detection on the body position data sequence, and dividing the body position data sequence into a plurality of body position holding time periods and a plurality of body position change time periods according to the detection result; wherein the data points in each body position holding time period are equal, and the data points in each body position change data segment are not equal; Marking the subsequences corresponding to the body position holding time periods in the body position data sequence as specific body positions to be identified; or After preprocessing the body position data sequence, the method further includes: Performing change detection on the standard data sequence, and dividing the standard data sequence into a plurality of body position holding time periods and a plurality of body position change time periods according to the detection result; wherein the data points in each body position holding time period are the same, and the data points in each body position change data segment are different; The subsequences corresponding to the specific body position holding time periods in the standard data sequence are respectively marked as specific body positions to be identified.

4. The body position recognition method according to claim 1, characterized in that: The model architecture of the body position recognition model includes: a number of convolutional neural networks, time series neural networks and / or attention mechanism networks, and a transposed output layer; wherein: The plurality of convolutional neural networks are used to extract features of the input body position data sequence; The time series neural network and / or attention mechanism network is used to perform temporal association based on the features extracted by the plurality of convolutional neural networks, and further extract the temporal association features of the body position data sequence; The output layer is used to output the structured body position data corresponding to each data point in the body position data sequence.

5. The body position recognition method according to claim 3, characterized in that: The time series neural network includes any one of a bidirectional long short-term memory network, a recurrent convolutional neural network, a recurrent neural network, and a gated recurrent unit.

6. The body position recognition method according to claim 1, characterized in that: After identifying each specific body position included in the body position data sequence and the start and end time points of each specific body position, the method further includes: Arranging the specific body positions in sequence according to the start and end time of each specific body position to obtain a first body position change sequence; Based on the body position change reference sequences of various types of BPPV position change tests, the BPPV test type of the first body position change sequence is determined.

7. The body position recognition method according to claim 2, characterized in that: Also includes: The camera is used to collect video data of the subject's body position changes when doing the BPPV position change test; After preprocessing the body position data sequence to obtain a standard data sequence to be input, the method further includes: Acquire timestamp information of each data point of the standard data sequence; and based on the timestamp information, extract the video frame of the corresponding time point from the video data to obtain a video frame sequence; Temporally aligning the video frame sequence with the standard data sequence; The model architecture of the body position recognition model includes: A multimodal input layer, used to respectively receive the aligned standard data sequence and video frame sequence and extract features of the standard data sequence through a plurality of one-dimensional convolutional neural networks, and to extract features of the video frame sequence through a plurality of two-dimensional convolutional neural networks; A feature fusion layer, used to fuse the feature vector extracted from the standard data sequence and the feature vector extracted from the video frame sequence; wherein the feature fusion layer uses an attention mechanism to dynamically adjust the feature weights of the standard data sequence and the video frame sequence; The output layer is used to output structured body position data corresponding to each data point in the body position data sequence according to the fusion features output by the feature fusion layer.

8. The body position recognition method according to claim 1, characterized in that: Also includes: The video data of the changes in the body position of the subject when doing the BPPV position change test is collected by a camera; and the body position recognition model includes a first recognition model and a second recognition model; the inputting of the body position data sequence into the trained body position recognition model for recognition specifically includes: Input the standard data sequence into the first recognition model, identify and output the body position label corresponding to each data point in the standard data sequence; and the start and end time points of each specific body position; the first recognition model is a deep learning network model based on time series; Based on each specific body position and its corresponding start and end time points, extract key video frame data from the body position video frame sequence; wherein the key video frame data carries time stamp information; Inputting the key video frame data into a second recognition model to recognize and output the body position information of the key video frame data; the second recognition model is trained by a large number of image samples marked with different specific body positions; According to the recognition results of the image recognition model and the recognition results of the body position recognition model, the specific body positions included in the BPPV position change test and the corresponding start and end time points are determined.

9. A body position recognition device based on artificial intelligence, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the body position recognition method as described in any one of claims 1-8.

10. A body position recognition system based on artificial intelligence, characterized in that: include: The body position recognition device according to claim 9; as well as: Body position data acquisition equipment, used to collect body position data sequence of subjects during BPPV position change test; The body position recognition device is configured to perform the following steps: receiving the body position data sequence; Preprocessing the body position data sequence to obtain a standard data sequence to be input; Inputting the standard data sequence into a trained body position recognition model; Performing body position recognition by using the body position recognition model to recognize each specific body position included in the body position data sequence, and the start and end time points of each specific body position; Among them, the body position recognition model is a deep learning network model based on time series; and the body position recognition model adopts a training data set composed of test data samples of different types of BPPV position change tests; wherein each test data sample in the training data set is a body position data sequence with timestamp information, and the body position data sequence with timestamp information is also marked with a body position label, and the start and end time points of each specific body position.

11. The body position recognition system according to claim 10, characterized in that: The body position data acquisition device is a head-mounted device, and a motion sensor is arranged in the head-mounted device.

12. The body position recognition system according to claim 10, characterized in that: The body position data acquisition device is a BPPV swivel chair, which is used to perform a BPPV position change experiment on a subject according to an operator's instructions, and transmits the swivel chair rotation information to the body position recognition device in real time during the position change experiment.

Images