Device for assisting guiding to capture an ultrasound image and associated method

Abstract

The invention relates to a device (1) for assisting guiding for capturing ultrasound images, said device being configured to receive, in real time, ultrasound images (11) captured by an ultrasound probe connected to said device (1). The invention also relates to an associated method.

Description

[0001] Description

[0002] Title: DEVICE FOR ASSISTANCE IN GUIDANCE FOR ACQUIRING AN ULTRASOUND IMAGE AND ASSOCIATED METHOD

[0003] Field of invention

[0004] The present invention relates to a device and a method for assisting the user in the acquisition of ultrasound images.

[0005] State of the art

[0006] Ultrasound imaging, also known as sonography, is a medical imaging technique that uses high-frequency waves to visualize a two-dimensional structure inside the body of a living subject. Since ultrasound images are taken in real time, these ultrasound images show the movement of internal organs within the body, such as the movements of the heart during its beat.

[0007] To acquire such images, an ultrasound probe is placed directly on the subject's skin. A thin layer of gel may be applied to the skin to allow ultrasound waves to travel from the probe into the subject's body. Ultrasound images are produced by measuring the reflection of ultrasound waves from the subject's organs. The amplitude of the measured reflected waves and the time of flight of this wave provide the information needed to reconstruct the ultrasound image.

[0008] During an ultrasound examination, the operator must know where to place the probe and how to orient it to obtain the desired image. Typically, the desired images from an ultrasound are images that allow certain measurements to be made.

[0009] In order to increase the ultrasound examination capacity, it was thought to relieve doctors (general practitioners, cardiologists, radiologists, etc.) or health professionals from taking images. For example, the view images can be taken by an operator who will record the images and then transmit them to the doctor who will then only have to interpret them. Such an organization makes it possible to increase the number of ultrasound examinations that can be analyzed by a doctor. However, it is then necessary to be certain that the ultrasound images are of sufficient quality for their interpretation by a doctor. The operator may have knowledge of ultrasound, but when he will not perform the analysis of the images himself, there is a need for a device to assist in taking such ultrasound images.

[0010] This acquisition can be difficult for an unfamiliar operator.

[0011] Software-implemented methods are known to guide the operator in taking ultrasound images. These methods include detecting the position of the probe via position sensors and accelerometers. The software can display a directional arrow on the screen, indicating to the user where to move the probe to improve the quality of the acquired ultrasound image.

[0012] A disadvantage of these methods is that it remains difficult for the operator to assess the path to be taken to obtain a satisfactory ultrasound image. Another disadvantage is that the user can hardly anticipate the path and sequence of actions to be taken before obtaining a satisfactory ultrasound image. Another disadvantage is that the operator has no idea, when an arrow is displayed, whether the distance to be covered is short or long. The operator must therefore grope for the correct probe position. Another disadvantage is that the examiner's gaze remains focused on the arrow and thus loses sight of the ultrasound image he is acquiring.

[0013] Also known is document EP3868303 describing a method based on the quality trend of a series of images to provide an indication to the user.

[0014] A disadvantage of this method is that it may fail to guide the user to a target position, for example if the user encounters a local extremum. Another disadvantage is that this method does not generate real-time guidance: enough images must have been acquired at different positions to detect a quality trend.

[0015] Document US20154 / 327838 describing the detection of regions of interest on the ultrasound image is also known.

[0016] A disadvantage is that this method requires that such areas of interest be present on the acquired ultrasound image, which is not the case if the position of the probe is too far from a target position. There is therefore a need for a new device to facilitate the operator in taking ultrasound images.

[0017] The invention therefore aims to provide a device and an associated method allowing operators to acquire more easily and more quickly sequences of ultrasound images of sufficient quality.

[0018] Summary of the invention

[0019] According to one aspect, the invention relates to a guidance assistance device for the acquisition of ultrasound images, in particular ultrasound images of the heart, configured to receive in real time ultrasound images (in particular of the heart) acquired by an ultrasound probe connected to said device, and comprising:

[0020] ■ a first computer, associated with a first memory, for implementing a classifier configured to classify in real time the received ultrasound images and associate a class with each image to generate a quality indicator based on said associated class;

[0021] ■ a display to, in real time, display the last quality indicator generated;

[0022] ■ a second computer, associated with a second memory, and configured to perform the following steps:

[0023] ■ determining a relative position of the ultrasound probe with respect to a predefined probe target position, said relative position being determined from the, or at least one of the, received ultrasound images; and

[0024] ■ the generation and display on the display in real time of a position indicator based on the determined relative position.

[0025] In one embodiment, the display is configured to display the received ultrasound images in real time.

[0026] One advantage is that it allows the user to visualize the difference between the position of their probe and a position that allows them to acquire a desired ultrasound image. In addition, such a position indicator, generated based on the relative position, allows the user to both estimate a distance to travel and the direction to take with a single indicator. The user can thus more quickly and easily acquire the desired ultrasound image.

[0027] According to one embodiment, the first and the second are the same memory. Thus, in the present application, a memory is indifferently designated as a physical device for storing data or an available memory space of a physical device.

[0028] According to one embodiment, the first calculator and / or the second calculator is / are included locally in the device of the invention. According to another embodiment, the first calculator and / or the second calculator is / are included in a remote device such as a data server. In the latter case, the device of the invention is understood as a system comprising several devices. In one embodiment, the first and second calculators are combined or implemented in the same computing device.

[0029] In one embodiment, the classifier is configured to classify an image into a plurality of classes, each representing a particular view of the organ, and at least one class representing a view of insufficient quality.

[0030] The advantage is that it can detect both a particular view and images that would not be usable by the doctor to establish a diagnosis.

[0031] In one embodiment, the at least one class representing a view of insufficient quality comprises at least one class of ultrasound images comprising enough information (e.g., visible anatomical areas) to allow determination of the relative position of the probe with respect to a target position or a reference position.

[0032] In one embodiment, the classifier is implemented using a learning function configured from supervised machine learning.

[0033] In one embodiment, the step of determining the relative position of the ultrasound probe with respect to a predefined target position of the probe from the received ultrasound images is implemented by means of a learning function configured from supervised automated learning. In one embodiment, said determining step is implemented by means of a neural network trained by a series of labeled ultrasound images, in particular of the heart. The advantage of such a classifier is that it allows reliable classification of the ultrasound images from training using labeled images.

[0034] For example, the neural network is a convolutional neural network, called CNN or Convet.

[0035] According to one embodiment, the configuration of the CNN neural network may comprise:

[0036] ■ Convolutions or neural network layers comprising a plurality of matrix multiplications comprising weighting coefficients obtained from a learning process;

[0037] ■ non-linear operations.

[0038] According to one embodiment, the configuration of the CNN neural network comprises as input(s) images acquired or received or stored in a memory.

[0039] The CNN neural network can include convolutions in its first layers and then fully connected layers of neurons, called "fully connected layers" at the end of the model. In the latter case, these are neurons connected to all the neurons of the previous layer and connected to all those of the following layer.

[0040] Convolution layers may involve scanning an input matrix to produce a series of matrix calculations. Other layers of the neural network typically involve matrix calculations on the size of the input matrix.

[0041] In one example, each convolution includes a matrix product between an input matrix and a weight matrix and the consideration of an additional bias.

[0042] Applying layered processing within the CNN neural network includes applying a series of matrix multiplications followed by a non-linear function to produce an output of said layer. The succession of these operations defines the depth of the neural network. According to an exemplary embodiment, the neural network is a multi-layer perceptron, known by the acronym MLP and in English terminology as “multi-layer perceptron”. According to an example, the neural network may be a network equivalent to the MLP.

[0043] In one embodiment, said labels comprise relative spatial coordinates of the position of the probe relative to a target position and / or information of relative orientations of the probe relative to target orientations in an orthonormal reference frame.

[0044] In one embodiment, the predefined probe target position is pre-stored, preferably in the second memory. In one embodiment, the target position is associated with a predefined image class.

[0045] In one embodiment, the displayed position indicator comprises a first visual element and a target marker; the position of said first element displayed on the display relative to the target marker being a function of the relative position of the determined probe.

[0046] In one embodiment, the determined relative position comprises at least one piece of information on the relative orientation of the probe relative to a target orientation relative to a predetermined axis of an orthonormal reference frame; and in which the orientation of said first visual element relative to the orientation of the target reference frame is a function of said relative orientation information.

[0047] One advantage is that it allows the user to visualize in a single visual element the probe placement deviation and a probe orientation deviation to acquire a desired ultrasound image. This allows the user to acquire the desired ultrasound image more quickly and easily.

[0048] In one embodiment, the determined relative position reaches a predefined threshold value, the first visual element is confused with the target marker.

[0049] In one embodiment, when the acquired image is classified into the predefined image class, the first visual element is confused with the target marker. In one embodiment,

[0050] In one embodiment, the target marker is arranged on the real-time displayed ultrasound image. An advantage is to allow the user to view both the acquired ultrasound image and the position indicator at the same location.

[0051] In one embodiment, the target marker comprises the contours of the displayed ultrasound image; the dimensions of the first visual element being similar to the dimensions of the contours of the displayed ultrasound image.

[0052] In one embodiment, the target cue comprises displaying a target and wherein the position of the first visual element relative to the center of the target is a function of the determined relative position.

[0053] In one embodiment, the position indicator further comprises a 3-dimensional view of an organ; the target mark comprising a first target plane passing through said displayed organ and the first visual element comprising a second plane passing through said displayed organ.

[0054] According to another aspect, the invention relates to a system comprising a device according to the invention and an ultrasound probe. The ultrasound probe is preferably capable of being connected to said device so as to transmit the ultrasound images acquired by the probe to the device.

[0055] According to another aspect, the invention relates to a method for assisting with guidance for the acquisition of ultrasound images, in particular ultrasound images of the heart. Said method is implemented by computer.

[0056] The said method comprises the following steps:

[0057] ■ receiving ultrasound images (11) from an ultrasound probe;

[0058] ■ real-time classification of received ultrasound images, and association of each image with a class;

[0059] ■ the generation and display (300) of a quality indicator as a function of said class associated with the classified ultrasound image;

[0060] ■ detecting a relative position of the ultrasound probe with respect to a predefined probe target position from the received ultrasound images; and ■ generating and displaying on the display in real time a position indicator as a function of the determined relative position.

[0061] In one embodiment, the method includes continuously or in real time displaying the received ultrasound images on the display.

[0062] In one embodiment, the position indicator is displayed overlaid on the displayed ultrasound image. In one embodiment, the position indicator is displayed overlaid on a portion of a human body displayed on the display.

[0063] According to another aspect, the invention relates to a device comprising software and hardware means for executing the method according to the invention. The hardware means may comprise a display, a receiver, an ultrasound probe, one or more processors or computers associated with memories, one or more memories for storing the ultrasound image sequences, and / or a transmitter.

[0064] According to another aspect, the invention relates to a computer program product comprising instructions which cause the device according to the invention to execute the steps of the method according to the invention. According to one aspect, the invention relates to a computer-readable medium, on which the computer program according to the invention is recorded.

[0065] Brief description of the figures

[0066] Other characteristics and advantages of the invention will emerge on reading the detailed description which follows, with reference to the appended figures, which illustrate:

[0067] Figure 1: a schematic view of the image displayed by the display of the device according to a first embodiment of the invention.

[0068] Figure 2: a schematic view of a device according to one embodiment of the invention.

[0069] Figure 3A: a schematic view of the image displayed by the display of the device according to a second embodiment of the invention.

[0070] Figure 3B: a schematic view of the image displayed by the display of the device according to a third embodiment of the invention.

[0071] Figure 3C: a schematic view of the image displayed by the display of the device according to a fourth embodiment of the invention Figure 3D: a schematic view of the image displayed by the display of the device according to a fifth embodiment.

[0072] Figure 4: a flowchart representing the different stages of a method according to an embodiment of the invention.

[0073] Detailed description

[0074] The invention relates to a method 1000 for assisting in the generation of a sequence of ultrasound images.

[0075] The invention also relates to an associated device. An example of such a device 1 is illustrated in Figure 2.

[0076] The device 1 may comprise a tablet, a smartphone, a computer, or any other device comprising at least one display and a processor associated with a memory. According to another aspect, the invention also relates to a system 2 comprising a device 1 according to the invention and a SECH ultrasound probe connected to said device 1.

[0077] In one embodiment, the device comprises software and hardware means for implementing the method according to the invention described below. Said device preferably comprises at least one REC receiver, a classifier and a display.

[0078] RECEPTION

[0079] The method comprises a step 100 of receiving ultrasound images 11 in real time. Said ultrasound images are received by a receiver REC of the device. In one embodiment, the receiver REC may comprise a buffer memory in which the images received 11 by the receiver REC are temporarily stored before their transmission.

[0080] The received ultrasound images 11 are displayed in real time, preferably on an AFF display of the device. The received ultrasound images can therefore be transmitted to a display for their display in real time.

[0081] The device comprises an AFF display. The AFF display may comprise a screen such as the screen of a monitor, a touchscreen tablet or a smartphone. The AFF display is connected to the REC receiver and / or to the CLASS classifier and / or to the processor. The AFF display is configured to display in real time the ultrasound images received by the REC receiver. The AFF display advantageously allows the operator to have real-time feedback on the ultrasound images that he is currently acquiring. The AFF display is configured to display indicators or additional data that will be described later in this description. The AFF display displays an image 1 comprising an ultrasound image received 11 by the REC receiver. Preferably, the ultrasound image displayed is the last ultrasound image received by the REC receiver.The AFF display is thus configured to display in real time the images captured by the SECH ultrasound probe connected to the device according to the invention.

[0082] CLASSIFICATION

[0083] The method comprises a step 200 of classifying the ultrasound images 11 received by the receiver REC in real time. The classification is carried out by a classifier CLASS of the device.

[0084] The classification of an ultrasound image comprises the association of said ultrasound image with a class 14, preferably a class from a group of predefined classes.

[0085] In one embodiment, the classification further comprises the generation 300 of a quality indicator 13 based on said class associated with the ultrasound image. The quality indicator may be representative of a class or a predefined set of classes.

[0086] In one execution mode, the classification step further includes generating a confidence value. The confidence value may represent the confidence rate of the classification performed. The confidence value is generated by the CLASS classifier.

[0087] The classification step is preferably executed by a CLASS classifier of the device. By classifier, we mean an algorithmic function executed by a processor or a computer associated with a memory.

[0088] The classifier CLASS is configured to receive images received by the receiver REC. The classifier is configured to classify in real time at least a portion of the images received by the receiver REC. The classifier is configured to associate a class with an ultrasound image. The classifier generates a quality indicator 13 based on the associated class. The quality indicator 13 is preferentially associated with said classified ultrasound image.

[0089] Preferably, the device of image processing means for processing the received images before providing them to the classifier CLASS. These processing means may comprise image filters or contrast functions.

[0090] Preferably, the classifier CLASS is configured to classify an ultrasound image from among a plurality of classes comprising, on the one hand, at least one class each representing a particular view of the organ and on the other hand, at least one class representing a view of insufficient quality. In one embodiment, the at least one class representing a view of insufficient quality comprises a class representing a view whose quality allows the determination of the relative position of the probe as described later.

[0091] Thus, when an image is classified into a class representing a particular view of the organ, this image is considered to be of sufficient quality to advantageously allow the clinician to carry out predefined measurements from these images.

[0092] In one embodiment, each class representing a particular view of the organ represents a particular view of the heart from among:

[0093] * The view of a parasternal long axis section

[0094] * The view of a parasternal short axis section (aortic, mitral, pillar)

[0095] * The view of an apical section (2, 3, 4 and 5 cavities)

[0096] • View of a subcostal section (4 cavities, inferior vena cava)

[0097] • The view of a suprasternal section

[0098] • The view of a straight paraphernal section

[0099] Each of these views is well known to cardiologists and is used to visualize different parts of the heart, calculate or measure specific data, and identify certain pathologies. Each of these views can be characterized by the presence of one or more specific parts of the heart. For example, a parasternal short-axis view is characterized by the presence of the left ventricular and right ventricular sections on the image. From this view, the cardiologist can calculate the shortening fraction and calculate pulmonary pressures.

[0100] In one embodiment, the class representing a view of insufficient quality represents ultrasound views of the heart not belonging to one of the specific views of the other classes or specific ultrasound views listed above, but whose image quality does not allow the necessary measurements related to this specific view to be carried out.

[0101] The invention is particularly advantageous in the context of acquiring ultrasound images of the heart since the heart requires the taking of particular images well known to practitioners as explained previously.

[0102] However, the invention may also find advantages in the ultrasound acquisition of other organs requiring user guidance assistance to obtain a characteristic view of said organ. For example, the invention also finds its interest in the acquisition of ultrasound images of the lungs, the liver, the uterus or in the case of an abdominal ultrasound or an obstetric ultrasound. In one embodiment, the "organ" comprises a human or animal fetus or an organ of such a fetus.

[0103] In one embodiment, the CLASS classifier is implemented using a learning function trained using supervised and / or machine learning. The learning function preferably comprises a neural network. The learning function is preferably trained using a series of labeled ultrasound images of an organ.

[0104] In one example, the learning function was configured from a training comprising submitting to the classifier a plurality of ultrasound images each associated with a label. In one embodiment, the classifier was trained with ultrasound images of a specific view of an organ of sufficient quality, each image being associated with a label representing said specific view. The classifier was also trained with ultrasound images of views of the organ other than those cited above or not having sufficient quality allowing the physician to perform measurements from these images, each of these images being each associated with a quality label representing a view of insufficient quality.The labels may also include information characteristic of a shot of an image of an organ having a characteristic viewing angle and / or size, image quality, the presence of a specific portion of an image of the organ, the presence of a measurable anatomical shape contour. Preferably, the labels include the name of the particular view of the organ or a name associated with a poor quality view.

[0105] In another embodiment, the ultrasound images used for training may include ultrasound images depicting particular views of another organ.

[0106] The CLASS classifier may be executed in a processor itself associated with a memory. The CLASS classifier may be recorded in a computer-readable medium such as a memory associated with said processor.

[0107] In one embodiment, the step of classifying an ultrasound image comprises generating a correspondence score for each class. The correspondence score may comprise a probability that the ultrasound image belongs to each class. The method then comprises selecting the class with the highest score. Preferably, the confidence value is generated from said correspondence score of the class associated with said ultrasound image.

[0108] In one embodiment, the CLASS classifier classifies 100% of the images received in real time.

[0109] In an alternative embodiment, the classifier CLASS is configured to classify only a rate of received images. The classifier is then configured to classify a portion of the ultrasound images received in real time. This embodiment is particularly advantageous when the classification rate is lower than the image acquisition rate (in images per unit of time).

[0110] In one embodiment, the classification of an ultrasound image comprises the classification of the last received ultrasound image. When said image has been classified, the method again takes the last received ultrasound image. Thus, it is possible that some ultrasound images are not classified. However, this embodiment advantageously allows for real-time classification of ultrasound images, regardless of the speed of the classifier or the frequency of reception of the ultrasound images.

[0111] QUALITY INDICATOR

[0112] The method according to the invention comprises the generation and display in real time on the AFF display of the generated quality indicator 13. Preferably, the displayed quality indicator 13 corresponds to the last generated quality indicator 13.

[0113] In one embodiment, this quality indicator 13 is generated by the classifier CLASS. Such a classifier is configured to, when it receives an ultrasound image of the organ, classify said image into one of the classes mentioned above. The method then comprises the generation of a quality indicator representative of the classification of this image. Said quality indicator can be associated with said classified ultrasound image.

[0114] Alternatively, the quality indicator 13 may be generated by a remote processor connected to the AFF display and receiving information from the classifier.

[0115] The display of the quality indicator 13 may comprise the display of a colored indicator whose color depends on the class associated with the classified ultrasound image. The color of the indicator may be representative of a single class. Preferably, the color of the colored indicator may take two distinct colors, a first color representative of a group of classes including the classes representing a particular view of the organ such as those stated above, and a second color representative of a class representing a view of insufficient quality. Thus, the operator advantageously visualizes more quickly whether the ultrasound image that he is taking is of sufficient quality or not.

[0116] In one embodiment, the classifier is also configured to, when receiving an ultrasound image of the organ, generate a confidence value 21. The confidence value 21 may be representative of a level of certainty that the image has been well classified. In one embodiment, the quality indicator may comprise said confidence value. The confidence indicator 21 is preferably displayed by the display AFF. The quality indicator 13 associated with the classified image may include said confidence indicator.

[0117] In a first example, the displayed quality indicator 13 comprises a colored frame, which preferably extends around the displayed ultrasound image. The operator can then advantageously visualize whether the image he is taking is indeed the one expected without having to look away from the image. The confidence indicator 21 can be displayed in the form of a numerical value as illustrated in FIG. 1.

[0118] In a second example, the quality indicator 13 comprises a colored indicator. The color of the colored indicator is representative of the classification of the last ultrasound image. At least one of the dimensions of said colored indicator is a function of the confidence value. In the example illustrated in FIG. 1, the quality indicator is a bar 17 whose length varies according to the confidence value and whose color depends on the class associated with the last ultrasound image.

[0119] Such an indicator advantageously allows the operator to visualize whether the image he is taking is of sufficient quality and also allows him to visualize without looking away whether the indicator is degraded or improved depending on the movement of the SECH ultrasound probe. The advantage of such an indicator is to allow the operator, when he slightly moves the probe, to visualize whether this movement increases or reduces the confidence value, and consequently increases or reduces his chances of obtaining a validated image sequence described below.

[0120] SEQUENCE

[0121] The method may comprise the automatic recording 400 of a sequence 20 of images in a memory MEM. This sequence 20 is recorded automatically when a sequence of a predefined number of received or classified images comprises a rate of images associated with the same class 14 greater than a predefined rate. Preferably, said class is a class representing a particular view of the organ.

[0122] An "image sequence" means a series of ultrasound images that follow one another in chronological order of acquisition. Therefore, the term "video sequence" will also be used to refer to such an image sequence.

[0123] Thus, when a sufficiently long video includes, for example, a majority of ultrasound images classified in the same class representing a particular view of the organ, the video sequence is automatically recorded in a memory. This automatic recording advantageously automatically generates videos of a particular view of the organ which can be analyzed by a cardiologist without validation from the operator.

[0124] The predefined number of images received or classified can be understood as a predefined minimum duration, as long as the image acquisition frequency of the SECH ultrasound probe is constant. The advantage of this threshold is to ensure that the video sequence is long enough to be analyzed by a physician. In another execution mode, the predefined number of images can be replaced by a number of images received or classified. For cardiology, the predefined number can be set to correspond to a predefined number of cardiac cycles of the subject's heart.

[0125] The rate of images associated with the same class in said video sequence higher than a predefined rate advantageously allows automatic recording despite a negligible number of images not having sufficient quality in said sequence. Indeed, this negligible number of images in a video sequence can come from a classification error, or from noise linked to a particular image. This tolerance advantageously consists of a good compromise between the ease of automatic generation of a sequence and the quality of said sequence.

[0126] In one execution mode, the preset rate is at least greater than 50%. In another execution mode, the preset rate may be set by the operator, for example, through a device user interface.

[0127] In one embodiment, the automatic recording step comprises buffering the continuously received ultrasound images as well as the generated quality indices associated with said images. In this way, when a video sequence meeting the criteria set out above is detected, the images belonging to this sequence can be transferred from the buffer memory to another memory, and / or grouped into a single file to generate a video sequence.

[0128] DETERMINATION OF POSITION

[0129] According to one embodiment, the method according to the invention comprises the display of at least one position indicator 12. The position indicator 12 is intended to assist the user in guiding the probe to obtain ultrasound images which can be classified into a predetermined class.

[0130] Generating a position indicator 12 includes detecting the real-time relative position of the ultrasound probe relative to a pre-recorded target position.

[0131] Said target position is preferentially associated with a class of ultrasound images or with a quality indicator as described above.

[0132] In a first embodiment, the relative position of the probe is obtained by means of detection of the probe, such as an optical detection device and / or acceleration sensors embedded in the probe.

[0133] In a second preferred embodiment, the relative position of the probe is determined in real time from the acquired images 11 received from the SECH ultrasound probe. In one embodiment, a computer is configured to execute a learning function such as a deep learning program. The learning function may be similar to that described above.

[0134] The learning function is trained from labeled ultrasound images, each training ultrasound image being associated with a relative position of the probe in which said image was acquired relative to the target position. Said function is configured to receive ultrasound images as input and to generate as output a relative position of the probe relative to said target position, preferably by a regression operation. Preferably, the function is configured to receive ultrasound images and to generate, for each ultrasound image received, a relative position of the probe relative to a target position.Preferably, said learning function implements a classifier configured to classify in real time the received ultrasound images and to associate with each image a relative position with respect to a predefined target position and to generate a position indicator as a function of said associated relative position.

[0135] In one embodiment, the ultrasound images labeled for training the learning function were generated by performing the acquisition of ultrasound images while calculating the real-time position of the ultrasound probe, for example with an optical camera.

[0136] Said relative position may comprise two spatial coordinates of positions X, Y representing the relative position of the probe head with respect to the target position on the surface of the subject's body.

[0137] Said relative position may further comprise relative orientation information T1, T2, T3 of the probe with respect to target orientations. Preferably, the relative position comprises 3 relative orientation information T1, T2, T3. Each relative information represents an angular deviation between the orientation of the probe with respect to the target orientation along an axis of an orthogonal reference frame.

[0138] In one embodiment, the learning function is therefore configured to, from each received ultrasound image, associate with said image spatial position coordinates and / or 3 pieces of information relating to the relative orientation of the probe, each piece of relative information representing an angular deviation between the orientation of the probe relative to the target orientation along an axis of an orthogonal reference frame. In this embodiment, said function has preferably been previously trained by labeled training ultrasound images where each training ultrasound image was associated with two spatial position coordinates and / or 3 pieces of information relating to the relative orientation of the probe.

[0139] The generation of the position indicator includes the real-time display of the position indicator based on the relative position determined in real time.

[0140] In one embodiment, the first visual element 31 is displayed on the screen and the position of this visual element on the display relative to a target point of the display is a function of the determined relative position. More precisely, a reference position is determined on the display AFF (preferably displayed) and the spatial coordinates of the first visual indicator relative to the reference position are a function of the spatial position coordinates of the determined relative position. For example, the distance between the first visual element 31 and the target position is a function of the relative position.

[0141] This advantageously allows the user not only to know in which direction to point his probe, but also to give him an idea of ​​the distance to travel to reach the position allowing him to obtain the desired ultrasound image.

[0142] In a first example illustrated in Figure 1, the visual element comprises a geometric shape and the target position is visualized by the contours 32 of the ultrasound image. This example is particularly advantageous since in addition to giving a good idea of ​​the relative position to the user, the latter can more easily visualize both the relative position indicator and the ultrasound image that he is acquiring.

[0143] Preferably, said geometric shape of the visual indicator is similar to that of the contours 32 of the ultrasound image 11 displayed in real time. Thus, the user knows that the probe has reached the target position when the visual element 31 is confused with the contours 32 of the ultrasound image 11.

[0144] In one embodiment, the orientation of the visual element 32 is a function of relative orientation information T1 along a first axis, preferably relative to an axis normal to the surface of the subject's body. This advantageously allows the user to simultaneously view two indications on the same visual element 31. In this mode, the geometric shape is not circular. For this purpose, the pie chart shape characteristic of the ultrasound image is particularly advantageous.

[0145] In one embodiment, the position indicator comprises one or two other visual elements 33, 34 representing the relative orientation information T2, T3.

[0146] In a second example illustrated in FIG. 3A, the visual indicator comprises a target whose center represents the target position and an element whose position relative to the center depends on the relative position of the determined probe. Said element may comprise a virtual probe 41. The virtual probe may comprise an orientation indicator 44 to allow the user to visualize the orientation of the virtual probe along an axis normal to the displayed target 42.

[0147] Preferably, the top of the virtual probe shifts relative to its base according to the relative orientation information T2, T3.

[0148] In a third example illustrated in Figure 3B, the visual indicator comprises a view of a member 53 in 3 dimensions and a target plane 52 passing through said member and further comprises a movable plane 51 whose position relative to the target plane is a function of the determined relative position.

[0149] In a fourth example illustrated in Figure 3C, the visual indicator comprises a target 62 and a marker 61. The target comprises a first visual cue 622 configured to indicate a position and comprises a second visual cue 621 configured to indicate a direction. In the example illustrated in Figure 3C, the first visual cue comprises a cross and the second visual cue comprises a direction.

[0150] The marker 61 comprises a first visual marker intended to indicate the relative position of the probe with respect to the position of the target according to two degrees of freedom in translation. In the illustrated example, the first visual marker 611 is formed by the intersection of two lines and / or is formed by the center of a geometric shape representing the sensor of the head of the ultrasound probe.

[0151] The marker 61 further comprises a second visual cue intended to indicate a direction such as an arrow 612. This second visual cue is intended to indicate a direction of the probe. In one embodiment, the first visual indicator, the geometric shape used, if it is not circular, can also serve to indicate said direction. The second visual cue and the first visual cue are preferably displayed fixed relative to each other.

[0152] The marker further comprises a third visual cue 613 for indicating an orientation of the probe.

[0153] In the embodiment illustrated in Figure 3C, the target 62 is displayed on the display in a fixed manner.

[0154] The marker 61 is displayed as movable by the display. The coordinates of the first visual marker 611 relative to the first visual marker 622 of the target 62 are a function of the relative position of the probe relative to the predefined position of the probe to obtain a good quality image.

[0155] This second visual cue thus advantageously makes it possible to indicate to the user the orientation to give to the probe along an axis perpendicular to the plane of the sensor of the probe head to obtain an image considered to be of sufficient quality.

[0156] Finally, the position of the third visual indicator 613 of the marker is displayed as movable relative to the first visual indicator 611 and the second visual indicator 612 of the marker 61. The third visual indicator 613 is designed to display a position on the display. The position of the third visual indicator 613 of the marker relative to the position of the first visual indicator 611 of the marker 61 is a function of a difference between the inclination of the probe relative to the surface of the skin according to two degrees of freedom.

[0157] To summarize, the probe on the body can be moved according to 2 degrees of freedom in translation and according to 3 degrees of freedom in rotation. Each target position of the probe allowing the acquisition of an image of satisfactory quality is therefore predefined by 2 coordinates and 3 orientations.

[0158] The relative position of the probe therefore includes 2 coordinate values ​​and 3 orientation values ​​according to the 3 degrees of freedom in rotation.

[0159] The relative position on the display of the first visual reference 611 of the marker 61 relative to the first visual reference 622 of the target 62 is a function of the two translation coordinates of the relative position of the probe.

[0160] The angle formed between the direction of the second visual reference mark 621 of the target 62 and the direction of the second visual reference mark 612 of the marker 61 is a function of a first orientation value according to a first degree of freedom in rotation of the relative position of the probe, preferably according to an axis parallel to the longitudinal axis of the probe.

[0161] Finally, the relative position on the display of the third visual reference mark 613 of the marker 61 with respect to the first visual reference mark 611 of the marker 61 is a function of the value of the other two orientations according to the other two degrees of freedom in rotation, preferably according to the two axes orthogonal to the axis parallel to the longitudinal axis of the probe. In the example illustrated in FIG. 3C, the user will move the probe so as to superimpose the positions of the first visual reference mark 622 of target 62 and the first visual reference mark 611 of the marker 61, will rotate the probe clockwise or counterclockwise to align the arrow 621 of the target with the arrow 612 of the marker, and tilt the probe to superimpose the first visual reference mark 611 of the marker 61 with the third visual reference mark 613 of the marker 61.

[0162] Preferably, the marker 61 represents an oval representing the sensor of the probe head and the third indicator 613 represents the extension of the longitudinal axis of the probe.

[0163] In one embodiment shown in the various figures 3A, 3B, 3C, the position indicator 12 is displayed next to the acquired ultrasound image 11.

[0164] In another embodiment not shown, the position indicator 12 is displayed superimposed on the acquired ultrasound image 11.

[0165] In another embodiment shown in Figure 3D, the position indicator 12 is superimposed on a representation 141 of a human body. In this embodiment, the display of the ultrasound image on the display is optional.

[0166] PROGRAM

[0167] According to one embodiment, the method according to the invention comprises the selection of a first class from among the classes representing a particular view of the organ, for example one of the views of the heart mentioned above.

[0168] The selection of the first class generates the display on the AFF display of a first pre-recorded ultrasound image 15 of the organ representing such a view. Preferably, the label 17 which may include the name of said particular view is also displayed on the AFF display. The selection of the first class also generates the display of a first setpoint image 14. The first setpoint image 14 is preferably pre-recorded. The setpoint image illustrates a position and / or orientation setpoint for the SECH ultrasound probe on a patient to obtain said view corresponding to the selected class. The operator is thus advantageously guided and assisted in taking such a view. He can thus, on the same AFF display, view an example of the view he must take, the position and orientation of the probe to achieve it.Finally, the operator can visualize in real time whether the acquired view is of sufficiently good quality using the quality indicator 13, and can advantageously visualize the influence of the movements he makes on the quality of the acquired image using the confidence indicator.

[0169] Finally, once the operator has found an image of satisfactory quality, he is assisted in real time by the time indicator representing the time for which he must maintain the probe to acquire images of sufficient quality. The operator is also assured, by viewing the time indicator, that a first sequence 20 will be recorded while maintaining his position.

[0170] Once the first sequence is recorded or validated, the method may comprise the automatic selection of a second class from among the classes representing a particular view of the organ. Once again, the selection of said second class automatically generates the display of a second pre-recorded reference image and the display of a second pre-recorded ultrasound image illustrating an ultrasound view.

[0171] According to one embodiment, the method further comprises generating and displaying a progress indicator. The progress indicator may be representative of the number of sequences that have been recorded. For example, the progress indicator is representative of the number of classes representing a particular view of the organ for which an image sequence 20 has been generated and / or recorded automatically according to the method of the invention.

[0172] According to one embodiment, the method comprises displaying the recorded sequence and further comprises a second manual validation by the operator after displaying said sequence.

[0173] DEVICE

[0174] According to one aspect, the device according to the invention comprises software and hardware means for implementing the method as described above.

[0175] An embodiment of the device according to the invention is now described with reference to Figure 2. The device comprises a receiver REC. The receiver REC is intended to be connected to an SECH ultrasound probe so as to continuously and in real time receive ultrasound images 11 acquired by said SECH ultrasound probe.

[0176] The REC receiver can be connected to the SECH ultrasound probe by a wired connection or a wireless connection, for example by a Bluetooth connection or a WI-FI connection or any other data exchange protocol known to those skilled in the art.

[0177] The REC receiver may comprise or be associated with one or more memories for temporarily storing the received images. The REC receiver is directly or indirectly connected to the AFF display to transmit the acquired ultrasound images 11 to the AFF display.

[0178] The device further comprises means for implementing a classifier CLASS as described above. The classifier is configured to receive the ultrasound images 11, associate a class 14 with the ultrasound images in real time, and generate the quality indicator 13 in real time. The classifier is directly or indirectly connected to the AFF display to transmit the generated indicators to the AFF display.

[0179] Alternatively, the classifier is implemented by remote electronic equipment, such as a remote server. In this case, the device comprises an interface for exchanging data with the remote equipment in order to transmit data and retrieve the result of the processed, i.e. classified, data.

[0180] Finally, in the present invention, it is understood that when the classification is implemented entirely or partly by remote equipment, the device of the invention can be interpreted as being the system comprising on the one hand the local device described in the present application and the remote means making it possible to implement the classification function.

[0181] The device further comprises at least one processor or a CALC calculator associated with a memory to execute at least part of the steps of the method according to the invention. For example, said processor can be configured to execute the steps of classification 200, generation of display of the quality indicator 300 and of the position indicator 500 and / or automatic recording 300 of the sequence 20 of images. Said processor can be connected to the AFF display to transmit the position indicator 12 to the AFF display.

[0182] The device 1 may also comprise a plurality of processors, each associated with one or more memories, and configured to together execute such steps. In one embodiment, the processor(s) may be remote and connected to the display by a data network.

[0183] The device further comprises a memory for storing or recording the sequences generated by the method according to the invention. In one embodiment, the device further comprises an EMM transmitter connected to said MEM memory for transmitting said sequences recorded on said MEM memory to a data network.

[0184] The AFF display may comprise means for receiving the various information 11, 13, 21, 12 received by the various REC, CLASS, PROC means of the device to generate the final image to be displayed.

Claims

CLAIMS A guidance assistance device (1) for the acquisition of ultrasound images, configured to receive in real time ultrasound images (11) acquired by an ultrasound probe (SECH) connected to said device (1), and comprising: ■ a first computer, associated with a first memory, to implement a classifier (CLASS) configured to classify in real time the received ultrasound images (11) and associate each image with a class (14) to generate a quality indicator (13) according to said class (14) associated; ■ a display (AFF) to display, in real time, the last quality indicator (13) generated; ■ a second calculator (CALC2), associated with a second memory, and configured to execute the following steps: ■ the determination of a relative position of the ultrasound probe (SECH) with respect to a predefined target probe position; said relative position being determined from at least one of the received ultrasound images (11); and ■ the generation and real-time display on the display of a position indicator (12) as a function of the determined relative position. Device according to the preceding claim, wherein the displayed position indicator (12) comprises a first visual element (31, 41, 51) and a target marker (32, 42, 52); the position of said first element displayed on the display relative to the target marker being a function of the determined relative position of the probe. Device according to the preceding claim, wherein the determined relative position comprises at least one relative orientation information of the probe (SECH) with respect to a target orientation with respect to a predetermined axis of an orthonormal coordinate system; and wherein the orientation of said first visual element (31, 41, 51) with respect to the orientation of the target reference frame (32, 42, 52) is a function of said relative orientation information.

4. Device according to claim 2 or claim 3, wherein, when the determined relative position reaches a predefined threshold value, the first visual element (31, 41, 51) is confused with the target marker (32, 42, 52).

5. Device according to claim 2 or claim 3, wherein, when the acquired image is classified into the predefined image class, the first visual element (31, 41, 51) is confused with the target marker (32, 42, 52).

6. Device according to any one of claims 2 to 5, wherein the display is configured to display in real time the received ultrasound images (11) and wherein the target marker (32) is arranged on the ultrasound image displayed (11) in real time.

7. Device according to claim 6, wherein the target marker (32) comprises the contours of the displayed ultrasound image (11); the dimensions of the first visual element (31) being similar to the dimensions of the contours of the displayed ultrasound image.

8. Device according to claim any one of claims 2 to 5, wherein the target marker (42) comprises the display of a target and wherein the position of the first visual element (41) relative to the center of the target is a function of the determined relative position.

9. Device according to claim any one of claims 2 to 5, wherein the position indicator (12) further comprises a 3-dimensional view of a component (53); the target marker comprising a first target plane (52) passing through said component (53) displayed and the first visual element comprising a second plane (51) passing through said component (53) displayed.

10. Device according to any one of the preceding claims, configured to display the received ultrasound image in real time on the display and configured to display the position indicator (12) superimposed on the received ultrasound image. A device according to any one of the preceding claims, configured to display a representation of at least a portion of a body model on the display and configured to display the position indicator (12) superimposed on said portion of the body model. A device according to any one of the preceding claims, characterized in that said relative position comprises at least two spatial position coordinates representing the relative position of the probe head with respect to the predefined target position on the body surface of a subject.A device according to any one of the preceding claims, characterized in that the second computer is associated with a second memory to implement a learning function configured to associate in real time with each image a relative position of the ultrasound probe with respect to a target position and to generate a position indicator as a function of said associated relative position. A device according to claim 13, characterized in that the learning function is trained from labeled training ultrasound images, each training ultrasound image being associated with a relative position of the ultrasound probe with respect to the predefined target position. A method (1000) for computer-implemented guidance assistance for ultrasound image acquisition comprising: ■ the reception (100) of ultrasound images (11) from an ultrasound probe (SEC); ■ the real-time classification (200) of received ultrasound images (11), and the association of each image with a class; ■ the generation and display (300) of a quality indicator (13) according to said class (14) associated with the classified ultrasound image; ■ the real-time determination, for each received ultrasound image (11), of a relative position of the ultrasound probe with respect to a predefined target probe position; and ■ the generation and display (500) on the real-time display of a position indicator (12) as a function of the determined relative position. Method (1000) according to the preceding claim, further comprising the continuous display on a display of the received ultrasound images (11). Product: computer program comprising instructions that lead the device according to any one of claims 1 to 14 to perform the steps of the method according to any one of claims 15 to 16. Computer-readable medium (MEM) on which the computer program according to claim 17 is recorded.

Images