Method and electronic apparatus for automatic tracking of movements of the fascial system

Abstract

Electronic method and apparatus for automatic tracking of the movements of a patient's fascial system comprising the following steps: a) detecting a sequence of ultrasound images (frames) during a movement protocol on a region of interest of said fascial system; b) segmenting the images by identifying in each image of the sequence a surface layer, an intermediate layer and a lower layer, in order to create a sequence of segmented images, c) identifying the movement of the pixels belonging to a first image, in at least a second subsequent image, in order to create a matrix of movements, d) associating the segmented images with said matrix of movements, in order to identify the movement of the surface, intermediate and lower layers in this sequence of images.

Description

[0001] METHOD AND ELECTRONIC APPARATUS FOR AUTOMATIC TRACKING OF MOVEMENTS OF THE FASCIAL SYSTEM

[0002] The present invention relates to a method and an electronic apparatus for automatic tracking of movements of the fascial system, carried out by analysing videos and ultrasound images. The result of the tracking can be used for evaluating the fascial system with various clinical implications, such as the planning and monitoring of personalized treatments and therapies.

[0003] The lack of or reduced sliding between the various substrates characterizing the fascial system occurs in numerous clinical conditions such as traumas, post- surgical outcomes, musculoskeletal dysfunctions such as lumbago, cervicalgia, nerve compressions in all topographic regions, etc. Such conditions can induce local inflammations, pains, peripheral sensitization and dysfunctions .

[0004] Therefore, the evaluation of the mechanical properties of the fascial tissues becomes of paramount importance, including therein their mobility and in particular their sliding; the latter can be evaluated in vivo by using the ultrasound.

[0005] Video-based studies of ultrasound examinations that analysed and quantified fascial mobility are known to the state of the art.

[0006] They mainly describe programmes to improve the quality of the ultrasound findings or to analyse some specific parts thereof.

[0007] For example, patent application US20050143655 describes such a type of software for improving the quality of ultrasound scans. The scientific publication article Langevin, H. M. , Fox, J. R. , Koptiuch, C. , Badger, G. J. , Greenan-Naumann, A. C. , Bouffard, N. A. , Konofagou, E. E. , Lee, W. N. , Triano, J. J. , & Henry, S. M. (2011) . Reduced thoracolumbar fascia shear strain in human chronic low back pain. BMC musculoskeletal disorders , 12, 203. https : / / doi . org / 10.1186 / 1471-2474-12-203 illustrates the use of cross-correlation software through automatic tracking algorithms for the evaluation of the thoracolumbar fascia sliding.

[0008] Nowadays, the analysis of the fascial dynamics is clinically possible only by using non-specific and not fully automatable approaches. This entails a considerable expenditure in terms of time of the operator and above all an impossibility to standardize, reproduce the method and to have a portability of the result.

[0009] Normally, through a visual evaluation by the experienced sonographer, the comparison between the images of the ultrasound videos is used, during the performance of a protocol by the subject under investigation (for example, the repetition of a state of muscle contraction with the performance of a specific movement task) , to measure the sliding at the level of the different layers of the fascial system. In addition to the well-known limitation of the algorithms based on the intensity of grey scales of pixels to interpret the structures represented without running the risk of confusing them with artefacts (typically present in the investigation carried out on the ultrasound findings) , the proposed analysis methods are not suitable for a precise and fast evaluation in real time to be implemented in everyday clinical activity. Operationally, in fact, their use consists in implementing numerous and different steps, carried out at different times and all that makes them difficult to use. Actually, to date there are no programs able to automatically identify the different substrates of the fascial system and follow their movement in the different ultrasound frames in a time accessible to clinical practice. Rather, in order to obtain a complete characterization of the sliding it is often necessary to use different programs with prohibitive costs and time losses, as well as risks in the compatibility and integration between the programs themselves and consequent limitation about the interpretation and comparison of the results.

[0010] This means that the fascia cannot be analysed dynamically in clinical practice, limiting the patient's immediate or medium / long-term diagnostic and follow-up process.

[0011] In addition, a technical difficulty in the analysis of this type of (ultrasound) videos is often represented by the poor quality of the images and the high level of noise that reduces the definition of the structures whose shape and speed are to be evaluated. Consequently, identifying the contours of anatomical structures and of clear references to be tracked through the frames becomes problematic .

[0012] The scientific publication article by Wu et al, entitled: "Characterization of facial tissue sliding by ultrasound" on behalf of the University of Auckland describes a method for identifying movement data of the structures of the facial soft tissues by ultrasound acquisition. An optical flow algorithm is implemented to visualise the deformation field and segment the discontinuity region. Finite element tracking meshes with cubic Hermite bases were used to measure movement and deformation at the discontinuous interface.

[0013] The Applicant notes that this article suggests the segmentation of two layers, also referred to as regions (composite + muscle) (Section II. B. ) . In addition, this article does not explain in detail the segmentation method and consequently the fact that it may have been performed automatically.

[0014] Furthermore, in this article, in section I. A. a generic anatomical analysis is carried out for the layers of the structures, which are then not identified individually. In the present invention the subdivision into layers (surface, intermediate and deep) is uniquely defined and realized automatically by a segmentation algorithm as indicated in claim 1.

[0015] The present invention aims to achieve the automatic tracking of movements through a program based on a combination of modules that allow the integration of the different steps clinically used, in the investigation of the identification and sliding of the structures of the fascial system.

[0016] The method is fully automatic, precise and immediate and can be clinically validated.

[0017] One aspect of the present invention relates to a method for automatic tracking of movements of the fascial system, having the characteristics of the appended claim 1.

[0018] A further aspect of the present invention relates to an apparatus for automatic tracking of the movements of the fascial system, having the characteristics of the appended claim 7. Further characteristics of the present invention are contained in the dependent claims.

[0019] The characteristics and advantages of the present invention will become more apparent from the following description of an embodiment of the invention, provided by way of non-limiting example, with reference to the schematic attached drawings, wherein:

[0020] • figure la illustrates an ultrasound image in which a window of interest is highlighted;

[0021] • figure lb illustrates the identification of the structures of interest, extrapolated from the compressive image of figure la;

[0022] • figure 2 illustrates four graphs of the movement of the fasciae in an area of interest carried out for a movement protocol; a graph of the movement of the structures of the fascial system identified in figure lb along the X axis is displayed in the upper left, a graph of the movement of the same along the Y axis is displayed in the lower left, a graph of the cumulative movement of the same along the X axis is displayed in the upper right, a graph of the cumulative movement of the same along the Y axis is displayed in the lower right, respectively;

[0023] • figure 3 illustrates a comparison between the four graphs of figure 2 detected before a treatment and four graphs of the same area of interest detected after a treatment;

[0024] • figures 4a and 4b are two flow algorithms relating to two embodiments of the method according to the present invention. A fascial system consists of a plurality of substrates superimposed on each other which for the purposes of the present invention are grouped into three distinct groups :

[0025] • surface layer comprising epidermis, surface adipose tissue, surface fascia, deep adipose tissue;

[0026] • intermediate layer formed by the deep muscle fasciae ;

[0027] • lower layer comprising the muscles.

[0028] The present invention is adapted to evaluate, in a region of interest extrapolated from ultrasound findings from a patient, the movements between the aforementioned three layers, for example to verify pathologies, dysfunctions thereof or traumas suffered.

[0029] The region of interest is a part of the body to be analysed, such as a region of the trunk, limbs, etc....

[0030] This patient carries out a movement protocol of the region of interest (i.e. performing a specific movement task adapted to bring out any sliding alterations in the investigated region of interest) during which a plurality of ultrasound images (subsequent frames of a video) are acquired over time by means of an ultrasound scanner which are analysed by an electronic processing apparatus to reveal the movements between these layers. The electronic processing apparatus can potentially be integrated into the same ultrasound instrument if suitably equipped with the necessary software.

[0031] This acquisition and subsequent analysis can be repeated at different times. For example, a first movement protocol can be carried out on the patient, before a treatment (for example pharmacological, physical therapy, manual one, etc...) , during which images are acquired and an analysis of the movements of the structures belonging to the pre-t reatment fascial system is carried out.

[0032] Subsequently, the movement protocol is repeated, after this treatment, during which images are acquired and an analysis of the movements of the structures belonging to the post-treatment fascial system is carried out.

[0033] The comparison between the two analyses can reveal the effectiveness of the treatment, as well as, in the absence of intermediate treatments, the follow-up, intended as a re-evaluation of the patient over time, in the medium / long term. Furthermore, the analyses can also be done at intermediate moments of the treatment, in order to have more comparisons. In fact, this method makes it possible to objectively evaluate anatomical / functional alterations based on the detected speeds, as a further step towards an evidence-based medicine .

[0034] The method and the apparatus according to the present invention comprises a first ultrasound image processing block, which in the window of interest and for each ultrasound image (frame) belonging to a set of subsequent images, acquired during the protocol, identifies the three layers or segments (surface, intermediate and lower) generating a segmented image. Identification takes place, for example, by means of known automatic image segmentation techniques (e.g. artificial intelligence or classical image processing systems) . The result obtained from this first step are binary masks, each of which identifies the pixels belonging to a certain layer analysed. Such a first block preferably uses a segmentation algorithm (e.g. a neural network) . Segmentation is formally defined as "dividing an image into a set of non-superimposable regions that together give the entire image" . That is, it is a set of techniques to convey and extract information from the images through some properties, including morphological ones, assigning each pixel a unique label, associated with a class. There are several segmentation methods and principles, according to the present invention preferably the segmentation block uses a neural network to perform such segmentation. Still according to the present invention, a second processing block, starting from said sequence of segmented images, in which the layers have been identified, determines the movement of said layers between one image and the subsequent ones, generating for example a matrix of movements.

[0035] This second block uses motion estimation algorithms, such as "dense motion estimation algorithms", to estimate the movement of each pixel from one frame to the subsequent one.

[0036] To detect this movement, it is proceeded with identifying a set of structures in a certain frame, at determined pixels and, in the subsequent frames, by looking for similar characteristics thereof. By thus associating a speed to each pixel in the frame, the movement is determined by the ratio between the distance in pixels of the movement of the characteristic in question possibly normalized for the time elapsed between one frame and the other. Advantageously, the movements can be mapped in any direction on a plane, identified by a pair of Cartesian X and Y axes .

[0037] Still according to the present invention, a third processing block combines the information processed by the first and second block by associating the movements of the pixels between the various images with the layers present in the segmented images.

[0038] The result of the processings can be displayed in graphs such as those, for example, shown in figures 3 and 4.

[0039] The identification data that originates the movement graphs are generated by this third processing block in tabular form (for example according to a .csv format) . In particular, figure 3 in the upper left graph identifies the speed along the X axis of the movement of each layer in a frame with respect to the previous frame, while in the graph next to it the movements are accumulated starting from the first frame, in order to make the information more readable for a technician in the field.

[0040] Similarly, the two graphs below show instead the movements (average and cumulated) of the layers along the Y axis .

[0041] The method according to the present invention, therefore, provides the following steps: a) detecting a sequence of ultrasound images (frames) during a movement protocol on a region of interest of a patient's fascial system; b) segmenting the images by identifying in each image of the sequence a surface layer, an intermediate layer and a lower layer, in order to create a sequence of segmented images, c) identifying the movement of the pixels belonging to a first image, in at least a second subsequent image, in order to create a matrix of movements, d) associating the segmented images with said matrix of movements, in order to identify the movement of the surface, intermediate and lower layers in this sequence of images.

[0042] According to a first embodiment, step c) provides that the movement of the pixels is identified by analysing the image as a whole (e.g. from the first up to the last image) , substantially generating overall matrices of movements among the frames and step d) combines the movement information of the pixels obtaining the movements of the individual layers, based on the segmentation of the layers performed in step b) . This embodiment is illustrated in the algorithm of figure 5a and in practice the movement segmentation and identification processings are performed in parallel. According to a second embodiment, step c) provides that the movement of the pixels is analysed in a first and a subsequent image, separately for each structure (layer) identified in point b) . The operation is then repeated for the subsequent images of the sequence, in pairs of images; for example, the second with a third, the third with a fourth and so on up to the last image of the sequence, adding the movements of the layers each time. The first embodiment potentially guarantees greater processing speed, while the second embodiment can instead guarantee greater accuracy in identifying the movements .

[0043] The method presented represents a step forward in the simplification of the procedure for the evaluation of the sliding of the different layers of the fascial system, potentially also using different ultrasound scans, despite some limitations are still present. One of these is linked to the importance of an optimal acquisition of the ultrasound videos.

[0044] The method, according to the present invention, introduces a new method that can be clinically validated for the tracking (i.e. estimation of the movement) of the substrates of the fascial system in ultrasound videos, providing an objective quantity that allows correlation with the pathology for site-specific and personalized treatments.

[0045] Clinical validity benefits can be identified as:

[0046] - study of the movement of the structures of interest after identification;

[0047] - limitation of artefacts linked to the intensity of the structures represented;

[0048] - automation of processes usually dependent on an operator;

[0049] - precise and specific identification of structures whose nomenclature is still not common and therefore for the benefit of their standardization.

[0050] The results of the processing are obtained in real time and are applicable to daily clinical activity.

Claims

CLAIMS1. Electronic method for automatic tracking of the movements of a patient's fascial system comprising the following steps: a) detecting a sequence of ultrasound images (frames) during a movement protocol on a region of interest of said fascial system; b) segmenting the images by identifying in each image of the sequence a surface layer, an intermediate layer and a lower layer, in order to create a sequence of segmented images, c) identifying the movement of the pixels belonging to a first image, in at least a second subsequent image, in order to create a matrix of movements, d) associating the segmented images with said matrix of movements, in order to identify the movement of the surface, intermediate and lower layers in this sequence of images, wherein in this step b) a surface layer comprising the epidermis, surface adipose tissue, surface fascia, deep adipose tissue, an intermediate layer comprising the deep fascia and a lower layer comprising the muscles are identified .

2. Electronic method according to claim 1 wherein step c) provides that the movement of the pixels is identified by analysing from the first up to the last image for all the images of the sequence, substantially generating an overall matrix of movements and step d) identifies the movements of the layers starting from the first sequence up to the last.

3. Electronic method according to claim 1 wherein step c) provides that the movement of the pixels isidentified by analysing a first and a subsequent image and comparing in step d) the matrix of movement of the fasciae between this first and this subsequent image, the operation being repeated for the subsequent images of the image pair sequence up to the last image of the sequence .

4. Electronic method according to claim 1 wherein said segmentation step b) is carried out by means of a neural network, for example a convolution one.

5. Electronic method according to claim 1 wherein said movement identification step c) is carried out by means of a motion estimation algorithm, for example of the "dense motion estimation algorithms" type.

6. Electronic processing apparatus for automatic tracking of the movements of the fascial system of a patient carrying out the method according to claim 1, wherein said apparatus comprises a first processing block in which the segmentation of the images is carried out, a second processing block, starting from said sequence of segmented images, in which the layers have been identified, determines the movement of said layers between one image and the subsequent ones, generating the matrix of movements and a third processing block which combines the information processed by the first and second block associating the movements of the pixels between the various images to the layers present in the segmented images.

7. Apparatus according to claim 6, characterized in that it is integrated in an ultrasound instrument.

Images