User interaction method based on ultrasonic image and ultrasonic imaging system

Abstract

The user interaction method based on the ultrasonic image and the ultrasonic imaging system are used for a mobile terminal and handheld ultrasonic equipment and comprise the steps that the ultrasonic image is displayed in a display interface of the mobile terminal; generating a selection area identifier on the ultrasonic image, wherein the boundary or the boundary band of the selection area identifier defines at least two touch areas on the ultrasonic image; receiving a touch operation of a user through the display interface; according to a touch area corresponding to the operation position of the touch operation, activating a function for adjusting the position of the selected area identifier, a function for adjusting the size of the selected area identifier or a function for adjusting the shape of the selected area identifier; according to an adjusting function activated by the operation position of the touch operation, adjusting the selected area identifier to obtain an adjusted selected area identifier; identifying the target tissue in an area of the ultrasonic image corresponding to the adjusted selected area identifier to obtain an identification result; and displaying the identification result. According to the invention, the operation process of adjusting the selected area identifier is simplified.

Description

Technical Field

[0001] The present invention relates to the technical field of ultrasonic imaging, and more particularly to a user interaction method based on ultrasonic images and an ultrasonic imaging system. Background Art

[0002] Ultrasonic imaging systems usually have intelligent analysis functions, that is, users set selection marks in the ultrasound image, and the ultrasound imaging system performs intelligent analysis on the area defined by the selection marks in the ultrasound image, such as automatically calculating the size of target tissues such as lesions. In order to accurately analyze the target tissue, the user needs to accurately set the size and position of the selection mark. Conventional ultrasound imaging systems usually obtain user operations based on trackballs, which makes it difficult to accurately judge the user's operating intentions. Therefore, the operation is cumbersome when adjusting the selection mark. Summary of the invention

[0003] A series of simplified concepts are introduced in the Summary of the Invention, which will be further described in detail in the Detailed Description of the Invention. The Summary of the Invention does not mean to attempt to define the key features and essential technical features of the claimed technical solution, nor does it mean to attempt to determine the scope of protection of the claimed technical solution.

[0004] A first aspect of an embodiment of the present invention provides a user interaction method based on ultrasonic images, which is applied to a mobile terminal and a handheld ultrasonic device, wherein a communication connection is established between the mobile terminal and the handheld ultrasonic device, the handheld ultrasonic device is used to transmit ultrasonic waves to a target object, receive ultrasonic echoes returned by the target object and obtain ultrasonic echo signals, and process the ultrasonic echo signals to obtain ultrasonic image data; the mobile terminal receives the ultrasonic echo signals sent by the handheld ultrasonic device or receives the ultrasonic image data processed by the handheld ultrasonic device through the communication connection; the mobile terminal is installed with an application, and the application is used to provide a display interface, through which human-computer interaction can be performed; the method comprises:

[0005] After the mobile terminal runs the application, an ultrasonic image obtained based on the ultrasonic echo signal or the ultrasonic image data is displayed on a display interface of the mobile terminal;

[0006] Generating a selection area mark on the ultrasound image, wherein a boundary or a boundary band of the selection area mark defines a first area and a second area on the ultrasound image, and the second area surrounds the first area;

[0007] receiving a user's touch operation through the display interface;

[0008] If the operation position of the touch operation is located in the first area, the function for adjusting the position of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the size of the selection area mark is activated; or, if the operation position of the touch operation is located in the first area, the function for adjusting the size of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the position of the selection area mark is activated;

[0009] According to the adjustment function activated by the operation position of the touch operation, the selection area identifier is adjusted to obtain an adjusted selection area identifier;

[0010] Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result;

[0011] The recognition result is displayed.

[0012] A second aspect of an embodiment of the present invention provides a user interaction method based on ultrasonic images, which is applied to a mobile terminal and a handheld ultrasonic device, wherein a communication connection is established between the mobile terminal and the handheld ultrasonic device, the handheld ultrasonic device is used to transmit ultrasonic waves to a target object, receive ultrasonic echoes returned by the target object and obtain ultrasonic echo signals, and process the ultrasonic echo signals to obtain ultrasonic image data; the mobile terminal receives the ultrasonic echo signals sent by the handheld ultrasonic device or receives the ultrasonic image data processed by the handheld ultrasonic device through the communication connection; the mobile terminal is installed with an application, and the application is used to provide a display interface, through which human-computer interaction can be performed; the method comprises:

[0013] After the mobile terminal runs the application, an ultrasonic image obtained based on the ultrasonic echo signal or the ultrasonic image data is displayed on a display interface of the mobile terminal;

[0014] Generating a selection area mark on the ultrasound image, wherein the boundary of the selection area mark includes an outer boundary and an inner boundary, the outer boundary of the selection area mark defines a first area and a second area on the ultrasound image, and the second area surrounds the first area;

[0015] receiving a user's touch operation through the display interface;

[0016] If the operation position of the touch operation is located in the first area, the function for adjusting the position of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the size of the selection area mark is activated; or, if the operation position of the touch operation is located in the first area, the function for adjusting the size of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the position of the selection area mark is activated;

[0017] According to the adjustment function activated by the operation position of the touch operation, the selection area identifier is adjusted to obtain an adjusted selection area identifier;

[0018] Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result;

[0019] The recognition result is displayed.

[0020] A third aspect of an embodiment of the present invention provides a user interaction method based on ultrasonic images, which is applied to a mobile terminal and a handheld ultrasonic device, wherein a communication connection is established between the mobile terminal and the handheld ultrasonic device, the handheld ultrasonic device is used to transmit ultrasonic waves to a target object, receive ultrasonic echoes returned by the target object and obtain ultrasonic echo signals, and process the ultrasonic echo signals to obtain ultrasonic image data; the mobile terminal receives the ultrasonic echo signals sent by the handheld ultrasonic device or receives the ultrasonic image data processed by the handheld ultrasonic device through the communication connection; the mobile terminal is installed with an application, and the application is used to provide a display interface, through which human-computer interaction can be performed; the method comprises:

[0021] After the mobile terminal runs the application, an ultrasonic image obtained based on the ultrasonic echo signal or the ultrasonic image data is displayed on a display interface of the mobile terminal;

[0022] Generate a selection mark on the ultrasound image, take the boundary of the selection mark as a reference and an area with a preset width surrounding the boundary of the selection mark as the second area, or take the boundary band of the selection mark as a reference and an area with a preset width surrounding the boundary band of the selection mark as the second area, and the area surrounded by the second area is the first area;

[0023] receiving a user's touch operation through the display interface;

[0024] If the operation position of the touch operation is located in the first area, the function for adjusting the position of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the size of the selection area mark is activated; or, if the operation position of the touch operation is located in the first area, the function for adjusting the size of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the position of the selection area mark is activated;

[0025] According to the adjustment function activated by the operation position of the touch operation, the selection area identifier is adjusted to obtain an adjusted selection area identifier;

[0026] Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result;

[0027] The recognition result is displayed.

[0028] In one embodiment, the touch operation is a click operation.

[0029] In one embodiment, generating a selection area mark on the ultrasound image includes:

[0030] The selection area identifier is generated based on a default size, a default position and / or a default shape.

[0031] In one embodiment, generating a selection area mark on the ultrasound image includes:

[0032] In response to a drawing operation performed by a user on the ultrasound image, the selection area mark is generated on the ultrasound image.

[0033] In one embodiment, the recognition result includes at least one of the following:

[0034] The outline of the target tissue, the radial position of the target tissue, the radial length of the target tissue, and the area of ​​the target tissue.

[0035] In one embodiment, the method further comprises:

[0036] If the target tissue is not identified in the ultrasound image region corresponding to the adjusted selection mark, a prompt message about the identification failure is displayed.

[0037] In one embodiment, the method further includes: adjusting the shape of the selection area mark based on the touch operation, wherein:

[0038] Based on the touch operation on the first operation control, cropping the selection area mark;

[0039] Based on the touch operation on the second operation control, the selection area mark is filled.

[0040] A fourth aspect of an embodiment of the present invention provides a user interaction method based on ultrasonic images, which is applied to a mobile terminal and a handheld ultrasonic device, wherein a communication connection is established between the mobile terminal and the handheld ultrasonic device, the handheld ultrasonic device is used to transmit ultrasonic waves to a target object, receive ultrasonic echoes returned by the target object and obtain ultrasonic echo signals, and process the ultrasonic echo signals to obtain ultrasonic image data; the mobile terminal receives the ultrasonic echo signals sent by the handheld ultrasonic device or receives the ultrasonic image data processed by the handheld ultrasonic device through the communication connection; the mobile terminal is installed with an application, and the application is used to provide a display interface, through which human-computer interaction can be performed; the method comprises:

[0041] After the mobile terminal runs the application, an ultrasonic image obtained based on the ultrasonic echo signal or the ultrasonic image data is displayed on a display interface of the mobile terminal;

[0042] Generating a selection area mark on the ultrasound image, wherein the selection area mark defines at least two touch control areas on the ultrasound image;

[0043] receiving a user's touch operation through the display interface;

[0044] Based on the touch area corresponding to the touch operation, activating a target function among at least two functions for adjusting the selection area marker, the at least two functions comprising at least two of the following: a function of adjusting a position of the selection area marker, a function of adjusting a size of the selection area marker, and a function of adjusting a shape of the selection area marker;

[0045] According to the target adjustment function, the selection area identifier is adjusted to obtain an adjusted selection area identifier;

[0046] Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result;

[0047] The recognition result is displayed.

[0048] In one embodiment, the selection area mark includes a frame representing a boundary or a boundary band of the selection area mark and an auxiliary mark located near the frame, and the method further includes:

[0049] When a touch operation on the auxiliary identifier is received, a target function corresponding to the auxiliary identifier is activated.

[0050] In one embodiment, generating a selection area mark on the ultrasound image includes:

[0051] The selection area identifier is generated based on a default size, a default position and / or a default shape.

[0052] In one embodiment, generating a selection area mark on the ultrasound image includes:

[0053] In response to a drawing operation performed by a user on the ultrasound image, the selection area mark is generated on the ultrasound image.

[0054] In one embodiment, the recognition result includes at least one of the following:

[0055] The outline of the target tissue, the radial position of the target tissue, the radial length of the target tissue, and the area of ​​the target tissue.

[0056] In one embodiment, the method further comprises:

[0057] If the target tissue is not identified in the ultrasound image region corresponding to the adjusted selection mark, a prompt message about the identification failure is displayed.

[0058] A fifth aspect of an embodiment of the present invention provides a user interaction method based on ultrasonic images, which is applied to a mobile terminal and a handheld ultrasonic device, wherein a communication connection is established between the mobile terminal and the handheld ultrasonic device, the handheld ultrasonic device is used to transmit ultrasonic waves to a target object, receive ultrasonic echoes returned by the target object and obtain ultrasonic echo signals, and process the ultrasonic echo signals to obtain ultrasonic image data; the mobile terminal receives the ultrasonic echo signals sent by the handheld ultrasonic device or receives the ultrasonic image data processed by the handheld ultrasonic device through the communication connection; the mobile terminal is installed with an application, and the application is used to provide a display interface, through which human-computer interaction can be performed; the method comprises:

[0059] After the mobile terminal runs the application, an ultrasonic image obtained based on the ultrasonic echo signal or the ultrasonic image data is displayed on a display interface of the mobile terminal;

[0060] Generating a selection area mark on the ultrasound image;

[0061] receiving a user's touch operation through the display interface;

[0062] Based on the touch operation, a target function is activated in at least two functions for adjusting the selection area mark, wherein the at least two functions include at least two of the following: a function of adjusting the position of the selection area mark, a function of adjusting the size of the selection area mark, and a function of adjusting the shape of the selection area mark; wherein,

[0063] If the touch operation is a single-finger operation, activating the function of adjusting the position of the selection area mark;

[0064] If the touch operation is a two-finger operation, activating the function of adjusting the size of the selection area mark;

[0065] If the touch operation is a touch operation on a preset operation control, activating the function of adjusting the shape of the selection area mark;

[0066] According to the target adjustment function, the selection area identifier is adjusted to obtain an adjusted selection area identifier;

[0067] Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result;

[0068] The recognition result is displayed.

[0069] A sixth aspect of an embodiment of the present invention provides a user interaction method based on ultrasonic images, which is applied to a mobile terminal and a handheld ultrasonic device, and is characterized in that a communication connection is established between the mobile terminal and the handheld ultrasonic device, the handheld ultrasonic device is used to transmit ultrasonic waves to a target object, receive ultrasonic echoes returned by the target object and obtain ultrasonic echo signals, and process the ultrasonic echo signals to obtain ultrasonic image data; the mobile terminal receives the ultrasonic echo signals sent by the handheld ultrasonic device or receives the ultrasonic image data processed by the handheld ultrasonic device through the communication connection; the mobile terminal is installed with an application, and the application is used to provide a display interface, through which human-computer interaction can be performed; the method comprises:

[0070] After the mobile terminal runs the application, an ultrasonic image obtained based on the ultrasonic echo signal or the ultrasonic image data is displayed on a display interface of the mobile terminal;

[0071] In response to a user's drawing operation on the ultrasound image, generating a selection area mark on the ultrasound image;

[0072] receiving a touch operation of the user based on the display interface;

[0073] Based on the touch operation, adjusting the shape or size of the selection area mark to obtain an adjusted selection area mark;

[0074] Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result;

[0075] The recognition result is displayed.

[0076] In one embodiment, adjusting the shape of the selection area mark based on the touch operation includes:

[0077] Based on the touch operation on the first operation control, cropping the selection area mark;

[0078] Based on the touch operation on the second operation control, the selection area mark is filled.

[0079] In one embodiment, the drawing operation includes a drawing operation on a border of the selection area marker.

[0080] In one embodiment, the drawing operation of the border of the selection area mark includes a line drawing operation, and the method further includes:

[0081] The line is automatically closed to obtain a border of the selection area mark.

[0082] In one embodiment, the drawing operation of the border of the selection area mark includes a point drawing operation, and the method further includes:

[0083] A border of the selection area mark is generated based on the multiple points drawn by the point drawing operation.

[0084] In one embodiment, the drawing operation includes a drawing operation on an internal structure of the selection area identifier, and the method further includes:

[0085] The preset width is expanded outward with the internal structure as the center to obtain the border of the selection area mark.

[0086] In one embodiment, the method further comprises:

[0087] Based on the touch operation, the position of the selection area marker is adjusted to obtain the adjusted selection area marker.

[0088] A seventh aspect of an embodiment of the present invention provides a user interaction method based on an ultrasound image, the method comprising:

[0089] Displaying the ultrasound image in the display interface;

[0090] Generating a selection area mark on the ultrasound image, wherein the boundary of the selection area mark defines a first area and a second area on the ultrasound image, and the second area surrounds the first area;

[0091] receiving a user's touch operation through the display interface;

[0092] If the operation position of the touch operation is located in the first area, the function for adjusting the position of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the size of the selection area mark is activated; or, if the operation position of the touch operation is located in the first area, the function for adjusting the size of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the position of the selection area mark is activated;

[0093] According to the adjustment function activated by the operation position of the touch operation, the selection area identifier is adjusted to obtain an adjusted selection area identifier;

[0094] Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result;

[0095] The recognition result is displayed.

[0096] An eighth aspect of the embodiments of the present invention provides a user interaction method based on ultrasound images, the method comprising:

[0097] Displaying the ultrasound image in the display interface;

[0098] In response to a user's drawing operation on the ultrasound image, generating a selection area mark in the ultrasound image;

[0099] receiving a touch operation of the user based on the display interface;

[0100] Based on the touch operation, adjusting the shape or size of the selection area mark to obtain an adjusted selection area mark;

[0101] Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result;

[0102] The recognition result is displayed.

[0103] A ninth aspect of an embodiment of the present invention provides a user interaction method based on an ultrasound image, the method comprising:

[0104] Displaying the ultra-ultrasound image in the display interface;

[0105] Generating a selection area mark on the ultrasound image, wherein the boundary of the selection area mark includes an outer boundary and an inner boundary, the outer boundary of the selection area mark defines a first area and a second area on the ultrasound image, and the second area surrounds the first area;

[0106] receiving a user's touch operation through the display interface;

[0107] If the operation position of the touch operation is located in the first area, the function for adjusting the position of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the size of the selection area mark is activated; or, if the operation position of the touch operation is located in the first area, the function for adjusting the size of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the position of the selection area mark is activated;

[0108] According to the adjustment function activated by the operation position of the touch operation, the selection area identifier is adjusted to obtain an adjusted selection area identifier;

[0109] Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result;

[0110] The recognition result is displayed.

[0111] A tenth aspect of the embodiments of the present invention provides a user interaction method based on ultrasound images, the method comprising:

[0112] Displaying the ultrasound image in the display interface;

[0113] Generate a selection mark on the ultrasound image, take the boundary of the selection mark as a reference and an area with a preset width surrounding the boundary of the selection mark as the second area, and the area surrounded by the second area as the first area;

[0114] receiving a user's touch operation through the display interface;

[0115] If the operation position of the touch operation is located in the first area, the function for adjusting the position of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the size of the selection area mark is activated; or, if the operation position of the touch operation is located in the first area, the function for adjusting the size of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the position of the selection area mark is activated;

[0116] According to the adjustment function activated by the operation position of the touch operation, the selection area identifier is adjusted to obtain an adjusted selection area identifier;

[0117] Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result;

[0118] The recognition result is displayed.

[0119] In one embodiment, the method is used in a mobile terminal.

[0120] An eleventh aspect of the embodiments of the present invention provides an ultrasound imaging system, including a mobile terminal and a handheld ultrasound device, wherein a communication connection is established between the mobile terminal and the handheld ultrasound device;

[0121] The handheld ultrasonic device is used to transmit ultrasonic waves to a target object, receive ultrasonic echoes returned by the target object and obtain ultrasonic echo signals, and process the ultrasonic echo signals to obtain ultrasonic image data;

[0122] The mobile terminal receives the ultrasonic echo signal or the ultrasonic image data processed by the handheld ultrasonic device through the communication connection; the mobile terminal is installed with an application, which is used to provide a display interface through which human-computer interaction can be performed; the mobile terminal is also used to execute the user interaction method as described above.

[0123] The ultrasonic image-based user interaction method and ultrasonic imaging system of the embodiments of the present invention make full use of the touch screen of the mobile terminal to adjust the selection area mark according to the user's touch operation, simplifying the operation process of adjusting the selection area mark and making the operation method more suitable for the operation interface of the mobile terminal. BRIEF DESCRIPTION OF THE DRAWINGS

[0124] The above and other purposes, features and advantages of the present invention will become more apparent by describing the embodiments of the present invention in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0125] Figure 1 A structural block diagram of an ultrasound imaging system according to an embodiment of the present invention is shown;

[0126] Figure 2 A schematic flow chart showing a user interaction method based on ultrasound images according to an embodiment of the present invention;

[0127] Figure 3 A schematic diagram showing displaying an ultrasound image in a display interface according to an embodiment of the present invention;

[0128] Figure 4 A schematic diagram showing a menu bar displayed in a display interface according to an embodiment of the present invention;

[0129] Figure 5 A schematic diagram showing a selection area identification according to an embodiment of the present invention;

[0130] Figure 6A schematic diagram showing activation of a function for adjusting the size of a selection area marker based on a touch operation on a first area according to an embodiment of the present invention;

[0131] Figure 7 A schematic diagram showing activation of a function for adjusting the position of a selection area marker based on a touch operation on a second area according to an embodiment of the present invention;

[0132] Figure 8 A schematic diagram showing an adjusted selection area identification according to an embodiment of the present invention;

[0133] Fig. 9 A schematic diagram showing an auxiliary identification according to an embodiment of the present invention;

[0134] Fig.10 A schematic diagram showing a selection area identification according to another embodiment of the present invention;

[0135] Fig.11 A schematic diagram showing a display of recognition results according to an embodiment of the present invention;

[0136] Fig.12 A schematic diagram showing a prompt message about recognition failure according to an embodiment of the present invention;

[0137] Fig.13 A schematic flow chart showing a user interaction method based on ultrasound images according to another embodiment of the present invention;

[0138] Fig.14 A schematic flow chart showing a user interaction method based on ultrasound images according to yet another embodiment of the present invention;

[0139] Figure 15-Figure 22 A schematic diagram showing generation of a selection area mark on an ultrasound image in response to a user's drawing operation is shown. DETAILED DESCRIPTION

[0140] In order to make the purpose, technical scheme and advantages of the present invention more obvious, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the protection scope of the present invention.

[0141] In the following description, a large number of specific details are provided to provide a more thorough understanding of the present invention. However, it is apparent to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present invention, some technical features well known in the art are not described.

[0142] It should be understood that the present invention can be implemented in different forms and should not be interpreted as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to make the disclosure thorough and complete and to fully convey the scope of the present invention to those skilled in the art.

[0143] The purpose of the terms used herein is only to describe specific embodiments and is not intended to be limiting of the present invention. When used herein, the singular forms "one", "an" and "said / the" are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "consisting of" and / or "comprising", when used in this specification, determine the presence of the features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or groups. When used herein, the term "and / or" includes any and all combinations of the relevant listed items.

[0144] In order to fully understand the present invention, a detailed structure will be proposed in the following description to illustrate the technical solution proposed by the present invention. The optional embodiments of the present invention are described in detail as follows, but in addition to these detailed descriptions, the present invention may also have other implementations.

[0145] Next, first refer to Figure 1 An ultrasound imaging system 100 according to an embodiment of the present invention is described. Figure 1 FIG. 1 is a schematic structural block diagram of an ultrasound imaging system 100 according to an embodiment of the present invention.

[0146] like Figure 1 As shown, the ultrasound imaging system includes a handheld ultrasound device 110 and a mobile terminal 120. A communication connection is established between the mobile terminal 120 and the handheld ultrasound device 110. The handheld ultrasound device 110 is used to transmit ultrasound to a target object, receive ultrasound echoes returned by the target object and obtain ultrasound echo signals, and process the ultrasound echo signals to obtain ultrasound image data. The mobile terminal 120 receives ultrasound echo signals sent by the handheld ultrasound device or receives ultrasound image data processed by the handheld ultrasound device through a communication connection with the handheld ultrasound device 110. The mobile terminal 120 is installed with an application, which is used to provide a display interface, and human-computer interaction can be performed through the display interface provided by the application.

[0147] Exemplarily, the mobile terminal 120 may include a mobile electronic device such as a mobile phone and a tablet computer. The mobile terminal 120 has a touch screen, which can be used to obtain a user's touch operation. The user can input the touch operation through fingers, gloves, or a touch pen. The touch screen can provide a display interface, which can be used for human-computer interaction, specifically including displaying an ultrasound image to the user, receiving a touch operation input by the user and performing a specific function based on the user operation, such as drawing a selection area mark on the ultrasound image. The communication connection between the mobile terminal 120 and the handheld ultrasound device 110 can be a wired connection or a wireless connection. The handheld ultrasound device 110 uses the display interface of the mobile terminal for human-computer interaction, so the handheld ultrasound device 110 itself may not be equipped with an output device such as a display and an input device such as a mouse or a trackball, so it is smaller in size and more portable.

[0148] Exemplarily, the handheld ultrasound device 110 includes an ultrasound probe, a transmitting circuit, a receiving circuit, and a processor. Further, the handheld ultrasound device 110 may also include a transmit / receive selection switch and a beamforming module, and the transmitting circuit and the receiving circuit may be connected to the ultrasound probe through the transmit / receive selection switch. In some embodiments, the handheld ultrasound device 110 may also be configured with a display.

[0149] The ultrasonic probe includes a plurality of transducer array elements, which can be arranged in a row to form a linear array, or arranged in a two-dimensional matrix to form a surface array, or a plurality of transducer array elements can also form a convex array. The transducer array element is used to transmit ultrasonic waves according to the excitation electrical signal, or to convert the received ultrasonic waves into electrical signals. Therefore, each transducer array element can be used to realize the mutual conversion between the electrical pulse signal and the ultrasonic wave, so as to transmit ultrasonic waves to the tissue of the target area of ​​the object to be tested, and can also be used to receive ultrasonic waves reflected back by the tissue. When performing ultrasonic testing, the transmitting sequence and the receiving sequence can be used to control which transducer array elements are used to transmit ultrasonic waves and which transducer array elements are used to receive ultrasonic waves, or control the transducer array elements to transmit ultrasonic waves or receive ultrasonic waves in time slots. The transducer array elements participating in ultrasonic wave transmission can be excited by electrical signals at the same time, so as to transmit ultrasonic waves at the same time; or, the transducer array elements participating in ultrasonic wave beam transmission can also be excited by several electrical signals with a certain time interval, so as to continuously transmit ultrasonic waves with a certain time interval.

[0150] During the ultrasonic imaging process, the processor controls the transmitting circuit to send the delayed focused transmitting pulse to the ultrasonic probe through the transmitting / receiving selection switch. The ultrasonic probe is stimulated by the transmitting pulse to transmit an ultrasonic beam to the tissue of the target area of ​​the object to be measured, and receives the ultrasonic echo with tissue information reflected from the tissue of the target area after a certain delay, and reconverts the ultrasonic echo into an electrical signal. The receiving circuit receives the electrical signal converted and generated by the ultrasonic probe, obtains the ultrasonic echo signal, and sends these ultrasonic echo signals to the beamforming module, which performs focusing delay, weighting and channel summing on the ultrasonic echo data, and then sends them to the processor. The processor performs signal detection, signal enhancement, data conversion, logarithmic compression and other processing on the ultrasonic echo signal to form ultrasonic image data. The ultrasonic image data obtained by the processor can be displayed on a display or stored in a memory. The processor can send the ultrasonic image data to the mobile terminal 120 for display, or send the unprocessed ultrasonic echo signal to the mobile terminal 120 for image processing and display.

[0151] Optionally, the processor may be implemented as software, hardware, firmware or any combination thereof, and may use a single or multiple application specific integrated circuits (ASICs), a single or multiple general purpose integrated circuits, a single or multiple microprocessors, a single or multiple programmable logic devices, or any combination of the aforementioned circuits and / or devices, or other suitable circuits or devices. Furthermore, the processor may control other components in the handheld ultrasound device 110 to perform the corresponding steps of the methods in various embodiments of the present specification.

[0152] The handheld ultrasound device 110 may also include a memory for storing instructions executed by the processor, storing received ultrasound echoes, storing ultrasound images, etc. The memory may be a flash memory card, a solid-state memory, a hard disk, etc. It may be a volatile memory and / or a non-volatile memory, a removable memory and / or a non-removable memory, etc.

[0153] It should be understood that Figure 1 The components included in the ultrasonic imaging system 100 are only exemplary, and the ultrasonic imaging system 100 may include more or fewer components, and the present invention is not limited thereto.

[0154] Next, we will refer to Figure 2 The user interaction method based on ultrasound image according to an embodiment of the present invention is described. The method can be implemented in Figure 1 An ultrasound imaging system 100 is shown. Figure 2 is a schematic flow chart of a user interaction method 200 based on ultrasound images according to an embodiment of the present invention.

[0155] like Figure 2 As shown, a user interaction method 200 based on ultrasound images according to an embodiment of the present invention includes the following steps:

[0156] In step S210, an ultrasound image is displayed on a display interface;

[0157] In step S220, a selection area mark is generated on the ultrasonic image, wherein the selection area mark defines at least two touch control areas on the ultrasonic image;

[0158] In step S230, a touch operation of the user is received through the display interface;

[0159] In step S240, based on the touch area corresponding to the touch operation, a target function is activated from among at least two functions for adjusting the selection area marker, wherein the at least two functions include at least two of the following: a function for adjusting the position of the selection area marker, a function for adjusting the size of the selection area marker, and a function for adjusting the shape of the selection area marker;

[0160] In step S250, the selection area identifier is adjusted according to the target adjustment function to obtain an adjusted selection area identifier;

[0161] In step S260, the target tissue is identified in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result;

[0162] In step S270, the recognition result is displayed.

[0163] The ultrasound image-based user interaction method 200 of the embodiment of the present invention activates a corresponding function for adjusting a selection area identifier according to an operation position of a touch operation, thereby simplifying the operation process.

[0164] Specifically, the handheld ultrasound device transmits ultrasound to the target object, receives the ultrasound echo returned by the target object to obtain an ultrasound echo signal, and processes the ultrasound echo signal to obtain ultrasound image data. During the scanning process, the transmitting circuit sends a set of delayed and focused transmission pulses to the ultrasound probe to stimulate the ultrasound probe to transmit ultrasound to the target tissue. The receiving circuit controls the ultrasound probe to receive the ultrasound echo reflected by the target tissue and converts it into an electrical signal. The beamforming module performs corresponding delay and weighted summation processing on the ultrasound echo signals obtained by multiple transmissions and receptions to achieve beamforming, and then sends them to the processor for subsequent signal processing to obtain ultrasound image data.

[0165] The mobile terminal is installed with an application, which is used to provide a display interface through which human-computer interaction can be performed. After the mobile terminal runs the application, an ultrasonic image obtained based on an ultrasonic echo signal or ultrasonic image data is displayed in the display interface. Specifically, the mobile terminal receives an ultrasonic echo signal through a communication connection with a handheld ultrasonic device, performs signal processing on the ultrasonic echo signal to obtain ultrasonic image data, and displays the ultrasonic image in the display interface. Alternatively, the mobile terminal can directly receive the ultrasonic image data obtained by processing the handheld ultrasonic device, and display the ultrasonic image obtained based on the ultrasonic image data in the display interface.

[0166] Reference Figure 3 , which shows a display interface when the handheld ultrasound device is a mobile phone. The ultrasound image is displayed at the center of the display interface, and a freeze button is displayed below the ultrasound image. The mobile terminal can freeze the ultrasound image in response to the selection operation of the freeze button during real-time imaging, and perform subsequent operations on the frozen ultrasound image, or the mobile terminal can also retrieve an offline ultrasound image from the memory and perform subsequent operations on the offline ultrasound image.

[0167] For example, the mobile terminal may also display a menu bar in the display interface, and display more operation controls in the menu bar. In a normal state, the mobile terminal may hide the menu bar to avoid occupying the display space of the ultrasound image. When a user operation of calling out the menu bar is detected, the mobile terminal displays the menu bar in the display interface. Figure 4 In the example, the user operation of calling out the menu bar can be a touch operation of sliding from the left side of the display interface to the right. The user operation of calling out the menu bar can also be a selection operation of a specific mark, etc. A "Smart clac" button is displayed in the menu bar. When a selection operation is received for the button, the mobile terminal activates the smart analysis function, and subsequently determines the selection mark in combination with user interaction to perform smart analysis on the ultrasound image.

[0168] After the ultrasound image to be analyzed is selected, a selection area mark is generated on the ultrasound image. The selection area mark is used to select a corresponding area in the ultrasound image, and the mobile terminal performs intelligent analysis on the ultrasound image based on the selection area mark.

[0169] Exemplarily, the selection area mark may include a border representing the selection area mark boundary or border zone, and the interior of the border may be filled with no color / texture or may be filled with color / texture. Figure 5 In the example of , the selection area mark includes a border without color filling, and the border is a circular dotted line. The selection area mark may also have no border and only consist of a color block or a texture block. In this case, the outer contour of the color block or the texture block is the boundary or boundary band of the selection area mark.

[0170] Exemplarily, when the intelligent analysis function is started, the mobile terminal generates a selection mark based on a default size and / or a default position. For example, the mobile terminal may generate a selection mark of a default size at the center of the ultrasound image. The selection mark may have a default shape, which may be a circle, a square, or any shape. The user may select a default shape of the selection mark from a variety of preset shapes. For example, Figure 5 In the example, the default shape of the selection marker is a circle. Fig.10 In the example shown in the figure, the default shape of the selection marker is a square.

[0171] Optionally, a selection mark can be generated on the ultrasound image in response to a user's drawing operation on the ultrasound image. Based on the touch screen of the mobile terminal, the user can flexibly draw a selection mark of any size and shape, so that the selection mark has a higher matching degree with the target tissue such as the lesion to be analyzed, the analysis result is more accurate, and unnecessary calculations are reduced.

[0172] At least one of the size, shape and position of the selection mark is adjustable, so that the user can accurately select the target area in the ultrasound image for intelligent analysis. When adjusting the selection mark, conventional ultrasound imaging systems usually need to move the selection mark with a trackball, determine the position of the selection mark, adjust the size of the selection mark with a trackball, and determine the size of the selection mark, which is cumbersome and complicated. In order to simplify the adjustment operation of the selection mark, the embodiment of the present invention makes full use of the touch-sensitive feature of the mobile terminal, and adjusts the size, position or shape of the selection mark based on the operation position of the touch operation.

[0173] Specifically, the selection area mark defines at least two touch areas on the ultrasound image. When a user's touch operation is received through the display interface, a target function is activated in at least two functions for adjusting the selection area mark based on the touch area corresponding to the touch operation. The at least two functions include at least two of the following: a function of adjusting the position of the selection area mark, a function of adjusting the size of the selection area mark, and a function of adjusting the shape of the selection area mark. Afterwards, the selection area mark is adjusted according to the adjustment function activated by the operation position of the touch operation to obtain an adjusted selection area mark.

[0174] In one embodiment, the touch area defined by the selection area identifier includes at least a first area and a second area, and the second area surrounds the first area. If the operation position of the touch operation is located in the first area, the function for adjusting the position of the selection area identifier is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the size of the selection area identifier is activated, or if the operation position of the touch operation is located in the first area, the function for adjusting the size of the selection area identifier is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the position of the selection area identifier is activated.

[0175] Among them, the first area and the second area can be defined by the border or border band of the selection area mark. For example, the area extending from the border or border band of the selection area mark to the inside of the selection area mark by a preset width is the second area, and the area located inside the second area is the first area. Alternatively, the area surrounding the border or border band of the selection area mark and extending to the outside of the selection area mark by a preset width is the second area, and the area located inside the second area is the first area. Alternatively, the area extending to the inside and outside of the selection area mark by a preset distance with the border or border band of the selection area mark as the center is the second area, and the area located inside the second area is the first area. Optionally, the border band of the selection area mark can be directly used as the second area, and the area inside the border band can be the first area. Exemplarily, the first area can be the internal area of ​​the selection area mark, and the second area can be the peripheral area of ​​the selection area mark.

[0176] When the user performs a touch operation on the ultrasound image, if the operation position of the touch operation is located in the first area, the function for adjusting the position of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the size of the selection area mark is activated. The touch operation can be a click operation, for example, referring to Figure 6 If it is recognized that the user clicks on the second area outside the selection mark, the function for adjusting the size of the selection mark is activated. At this time, the user can perform a drag operation to adjust the size of the selection mark. During the adjustment process, the size of the selection mark changes with the user's drag operation. For example, when the drag operation is directed toward the outside of the selection mark, the selection mark is enlarged, and when the drag operation is directed toward the inside of the selection mark, the selection mark is reduced. Figure 7 As shown, if it is recognized that the user clicks on the first area inside the selection marker, the function for adjusting the position of the selection marker is activated. At this time, the user can drag the inside of the selection marker to adjust the position of the selection marker. During the dragging process, the position of the selection marker moves synchronously with the user's dragging operation.

[0177] As another implementation, if the operation position of the touch operation is located in the first area, the function for adjusting the size of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the position of the selection area mark is activated. For example, the user can drag the outer area of ​​the selection area mark to adjust its position, and drag the inner area of ​​the selection area mark to adjust its size.

[0178] In another embodiment, the selection area identifier may define the first area and the second area in the following manner: the boundary of the selection area identifier includes an outer boundary and an inner boundary, and the area between the outer boundary and the inner boundary is the target area of ​​the intelligent analysis. The outer boundary of the selection area identifier defines the first area and the second area on the ultrasound image, and the second area surrounds the first area. For example, the selection area identifier may be a circular ring-shaped identifier, which is composed of an inner boundary and an outer boundary, and its outer boundary defines the first area and the second area on the ultrasound image, and the second area surrounds the first area. For example, the area extending from the outer boundary of the selection area identifier to the inside of the selection area identifier by a preset distance is the second area, or the area extending from the outer boundary of the selection area identifier to the outside of the selection area identifier by a preset distance is the second area, or the area extending from the outer boundary of the selection area identifier to the inside and outside of the selection area identifier by a preset distance with the outer boundary of the selection area identifier as the center is the second area. The first area is located inside the second area and outside the inner boundary of the selection area identifier.

[0179] Optionally, the selection area marker further includes an auxiliary marker located near an edge of the frame, and when a touch operation on the auxiliary marker is received, a target function corresponding to the auxiliary marker is activated. Fig. 9 An example of an auxiliary mark is shown, which is a direction icon to prompt the user to drag the auxiliary mark in various directions. The target function corresponding to the auxiliary mark can be a function of adjusting the position of the selection mark, that is, the user can drag the auxiliary mark to adjust the position of the selection mark; the target function corresponding to the auxiliary mark can also be a function of adjusting the size of the selection mark, that is, the user can drag the auxiliary mark to adjust the size of the selection mark. The logic of adjusting the selection mark by the auxiliary mark can reuse the adjustment logic of the sampling frame in C mode (color mode) to reduce the user's learning cost.

[0180] Exemplarily, the mobile terminal can adjust the selection mark multiple times based on the touch operation. For example, the user can drag the second area of ​​the selection mark multiple times until the selection mark reaches the target position; or the user can drag the first area of ​​the selection mark multiple times until the selection mark has the target size. After completing the adjustment of the selection mark, the user can perform a determination operation to instruct the mobile terminal to perform intelligent analysis on the ultrasound image based on the current selection mark. The determination operation can be a selection operation of the determination option in the display interface, for example, referring to Figure 8 , the determination option may be an "update" option in the display interface. The determination operation may also be a touch operation in a preset form, such as double-clicking the display interface, etc. In response to the determination operation, the mobile terminal turns off the adjustment function for the selection area identifier and starts to perform intelligent analysis. Exemplarily, when the touch operation for the selection area identifier is received again, the adjustment function for the selection area identifier is restarted.

[0181] After the selection mark is adjusted, the target tissue is identified in the region of the ultrasound image corresponding to the adjusted selection mark to obtain an identification result. The region of the ultrasound image corresponding to the selection mark may be the region surrounded by the boundary of the selection mark.

[0182] Identifying the target tissue includes at least determining the target tissue region in the area of ​​the ultrasound image corresponding to the selection mark. In one example, an edge detection image segmentation method can be used to identify and segment the target tissue region in the area of ​​the ultrasound image corresponding to the selection mark. For example, the discontinuity in the ultrasound image can be detected by edge detection algorithms including but not limited to edge detection algorithms such as differential operators, thereby achieving identification and segmentation of the target tissue region. In another example, a machine learning model can be used for image segmentation. For example, a trained deep learning neural network model can be used to segment the target tissue region in the area of ​​the ultrasound image corresponding to the selection mark. This type of method learns the features of a pre-constructed database by stacking convolutional layers and fully connected layers, thereby directly obtaining the area to be segmented of the input image and its corresponding category. Alternatively, an image feature recognition algorithm can be used to segment the target tissue region. For example, feature extraction can be performed on image blocks in a predetermined neighborhood around each pixel point in the ultrasound image corresponding to the selection mark, and then the extracted features can be matched with features in a pre-constructed database, and the extracted features can be classified using a classifier to determine the category of the image block corresponding to the feature, that is, the image block can be divided into image blocks of the target tissue area and image blocks of other areas according to the features of the image block, thereby achieving segmentation of the target tissue area.

[0183] After the target tissue region is determined, the target tissue region is further analyzed to obtain a recognition result, which is displayed in the display interface. Exemplarily, the recognition result includes the contour of the target tissue, that is, the precise location of the target tissue. The recognition result may also include the radial position of the target tissue and / or the radial length of the target tissue, such as Fig.11 As shown, the outline of the target tissue can be drawn in the ultrasound image, and the long diameter, short diameter and length of the target tissue can be marked inside the outline of the target tissue. The recognition result of the target tissue can also include the area of ​​the target tissue, the type of the target tissue, the deterioration level of the target tissue, etc. The recognition result for the target tissue may vary depending on the type of the target tissue, and the embodiment of the present invention does not limit the type of specific recognition result.

[0184] For example, if the target tissue is not identified in the ultrasound image area corresponding to the adjusted selection mark, a prompt message about the identification failure is displayed. Fig.12As shown, the prompt information about the recognition failure can be displayed in a pop-up window superimposed on the ultrasound image. In previous ultrasound devices, when the target tissue cannot be recognized in the target area, no feedback is usually given, which causes user confusion. In comparison, the embodiment of the present invention displays the prompt information about the recognition failure when the target tissue is not recognized in the selection mark, which can prompt the user to adjust the selection mark or re-acquire the ultrasound image in time.

[0185] In some embodiments, Figure 2 The ultrasound image-based user interaction method 200 can also be used for Figure 1 Other ultrasound imaging systems besides the ultrasound imaging system 100 shown in the figure can be used as long as the ultrasound imaging system can receive the user's touch operation through the display interface. For example, the ultrasound imaging system used to implement the user interaction method 200 can be a portable ultrasound imaging system with a touch screen or a non-portable ultrasound imaging system.

[0186] To summarize, the ultrasound image-based user interaction method 200 of an embodiment of the present invention activates a function for adjusting the size or position of a selection area marker according to the operation position of a touch operation, and adjusts the selection area marker according to the activated function. There is no need to switch buttons to select the size or position of the selection area marker, which saves operation time and operation steps and is more suitable for one-handed operation on a small-size screen.

[0187] like Fig.13 As shown, another embodiment of the present invention provides a user interaction method 1300 based on ultrasound images, which can be used Figure 1 The ultrasonic imaging system shown, or other ultrasonic imaging systems capable of receiving touch operations; the method comprises the following steps:

[0188] In step S1310, an ultrasound image is displayed on a display interface;

[0189] In step S1320, a selection area mark is generated on the ultrasound image;

[0190] In step S1330, receiving a user's touch operation through the display interface;

[0191] In step S1340, based on the touch operation, a target function is activated in at least two functions for adjusting the selection area mark, wherein the at least two functions include at least two of the following: a function for adjusting the position of the selection area mark, a function for adjusting the size of the selection area mark, and a function for adjusting the shape of the selection area mark.

[0192] In step S1350, the selection area identifier is adjusted according to the target adjustment function to obtain an adjusted selection area identifier;

[0193] In step S1360, the target tissue is identified in the region of the ultrasound image corresponding to the adjusted selection mark to obtain an identification result;

[0194] In step S1370, the recognition result is displayed.

[0195] The user interaction method 1300 based on ultrasound images of an embodiment of the present invention activates a target function based on the user's touch operation. For example, if the user's touch operation is a single-finger operation, the function of adjusting the position of the selection mark is activated, and the user can drag the selection mark through a single-finger operation to adjust the position of the selection mark. If the touch operation is a two-finger operation, the function of adjusting the size of the selection mark is activated, and the user can drag the selection mark through two fingers to adjust the size of the selection mark. Exemplarily, when the two fingers are dragged toward each other, the selection mark can be reduced; when the two fingers are dragged in the opposite direction, the selection mark can be enlarged. Among them, the operation position of the single-finger operation and the two-finger operation can be located inside the selection mark or outside the selection mark.

[0196] Exemplarily, if the touch operation is a touch operation on a preset operation control, the function of adjusting the shape of the selection area mark is activated. If the touch operation is a touch operation on a first operation control, the function of cropping the selection area mark is activated. If the touch operation is a touch operation on a second operation control, the function of filling the selection area mark is activated.

[0197] The ultrasound image-based user interaction method 1300 of the embodiment of the present invention activates a function for adjusting the size, position or shape of a selection area marker according to a touch operation, and adjusts the selection area marker according to the activated function, thereby simplifying the operation process.

[0198] like Fig.14 As shown, another embodiment of the present invention provides a user interaction method 1400 based on ultrasound images, which can be used Figure 1 The ultrasonic imaging system shown, or other ultrasonic imaging systems capable of receiving touch operations; the method comprises the following steps:

[0199] In step S1410, an ultrasound image is displayed on a display interface;

[0200] In step S1420, in response to a user's drawing operation on the ultrasound image, a selection area mark is generated on the ultrasound image;

[0201] In step S1430, receiving a touch operation of the user based on the display interface;

[0202] In step S1440, based on the touch operation, the shape or size of the selection area mark is adjusted to obtain an adjusted selection area mark;

[0203] In step S1450, the target tissue is identified in the region of the ultrasound image corresponding to the adjusted selection mark to obtain an identification result;

[0204] In step S1460, the recognition result is displayed.

[0205] The ultrasound image-based user interaction method 1400 of this embodiment generates a selection mark on the ultrasound image in response to the user's drawing operation on the ultrasound image, and adjusts the shape or size of the selection mark in combination with the user's touch operation. Based on the touch screen of the mobile terminal, the user can flexibly draw a selection mark of any size and shape, so that the selection mark has a higher matching degree with the target tissue such as the lesion to be analyzed, the analysis result is more accurate, and unnecessary calculations are reduced; by adjusting the selection mark, the selection mark can be flexibly modified after it is generated without the user having to redraw the selection mark, thereby improving operation efficiency and user experience.

[0206] In one embodiment, the user's drawing operation includes a drawing operation on a border of the selection area mark, that is, the graphic drawn by the user is determined as the border of the selection area mark, and the area inside the border is the target area of ​​the intelligent analysis.

[0207] In one example, if Fig.15 and Fig.16 As shown, the drawing operation of the border of the selection area mark includes a line drawing operation, that is, the line drawn by the user in the ultrasound image can be used as the border of the selection area mark. The user can draw most of the border of the selection area mark, and after the drawing is completed, the system automatically closes the line to obtain a closed border of the selection area mark. Automatically closing the line may include connecting a straight line or a smooth curve between the starting point and the end point of the line drawn by the user to close the line drawn by the user.

[0208] In another embodiment, Fig.19 , Fig. 20 , Fig.21 and Fig. 22 As shown, the drawing operation of the border of the selection area mark includes a point drawing operation, and the border of the selection area mark can be generated based on multiple points drawn by the point drawing operation. Fig.19 As shown in , the location where the user first clicks is taken as the first point of the border; then Fig. 20 As shown in , the location where the user clicks for the second time is taken as the second point of the border, and a dotted line is used to connect the first point and the second point; Fig.21 As shown in the figure, the user stops drawing after drawing the fourth point and connects the fourth point with the first point to obtain the following Fig. 22 The closed bounding box is shown.

[0209] In yet another embodiment, Fig.18 and Fig.19 As shown, the drawing operation of the selection mark includes the drawing operation of the internal structure of the selection mark. Specifically, the preset width is expanded outward with the internal structure as the center to obtain the border of the selection mark. The line drawn by the user is the internal structure of the selection mark, and the preset width can be expanded outward with the line drawn by the user as the center, so that the area within the expansion range is used as the area corresponding to the selection mark.

[0210] Exemplarily, the shape of the selection marker is adjusted based on the touch operation, including: cropping the selection marker based on the touch operation on the first operation control; filling the selection marker based on the touch operation on the second operation control. The first operation control can be a cropping marker displayed on the ultrasound image or other areas of the display interface. When the touch operation on the first operation control is received, the function of cropping the selection marker is activated, and the selection marker is cropped based on the next touch operation received. For example, if the trajectory of the touch operation is located inside the selection marker and connects two points on the boundary of the selection marker, the portion between the trajectory of the touch operation and the boundary of the selection marker is cropped and removed from the original selection marker. In this way, non-target areas can be removed from the selection marker to prevent non-target areas from affecting subsequent intelligent analysis.

[0211] Similarly, the second operation control may be a filling mark displayed on the ultrasound image or other areas of the display interface. When a touch operation on the second operation control is received, the function of filling the selection mark is activated, and the selection mark is filled based on the next touch operation received. For example, if the track of the touch operation is outside the selection mark and connects two points on the boundary of the selection mark, the part between the track of the touch operation and the boundary of the selection mark is added to the original selection mark, thereby ensuring that the selection mark completely covers the entire target area.

[0212] In some embodiments, the selection mark can also be cropped or filled according to the position of the touch operation. For example, if the track of the touch operation is inside the selection mark, the function of cropping the selection mark is activated; if the track of the touch operation is outside the selection mark, the function of filling the selection mark is activated.

[0213] In some embodiments, the size or position of the selection area mark can also be adjusted based on the touch operation. For example, the selection area mark defines at least two touch areas on the ultrasound image, and the function of adjusting the size of the selection area mark or the function of adjusting the position of the selection area mark can be activated according to the touch area corresponding to the touch operation.

[0214] The ultrasound image-based user interaction method 1400 of an embodiment of the present invention generates a selection mark on the ultrasound image in response to the user's drawing operation on the ultrasound image, and adjusts the shape or size of the selection mark in combination with the user's touch operation, so that the selection mark can be drawn and adjusted more flexibly.

[0215] The embodiment of the present invention further provides an ultrasound imaging system for implementing the above-mentioned ultrasound image-based user interaction method. Figure 1 , the ultrasound imaging system can be implemented as follows Figure 1 The ultrasound imaging system 100 shown may include a mobile terminal 120 and a handheld ultrasound device 110, wherein a communication connection is established between the mobile terminal 120 and the handheld ultrasound device 110; the handheld ultrasound device 110 is used to transmit ultrasound to a target object, receive ultrasound echoes returned by the target object, so as to obtain ultrasound echo signals, and process the ultrasound echo signals to obtain ultrasound image data; the mobile terminal 120 receives ultrasound echo signals or ultrasound image data processed by the handheld ultrasound device through a communication connection; the mobile terminal 120 is installed with an application, which is used to provide a display interface, through which human-computer interaction can be performed; the mobile terminal is also used to execute the user interaction method described above. The above only describes the main functions of the various components of the ultrasound imaging system. For more details, please refer to the relevant description of the user interaction method based on ultrasound images, which will not be repeated here.

[0216] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present invention thereto. Various changes and modifications may be made therein by one of ordinary skill in the art without departing from the scope and spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as required by the appended claims.

[0217] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0218] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0219] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0220] Similarly, it should be understood that in order to streamline the present invention and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present invention should not be interpreted as reflecting the following intention: the claimed invention requires more features than the features explicitly stated in each claim. More specifically, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present invention.

[0221] It will be understood by those skilled in the art that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0222] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0223] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functions of some modules according to embodiments of the present invention. The present invention can also be implemented as a device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0224] It should be noted that the above embodiments illustrate rather than limit the invention, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of means, several of these means may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0225] The above is only a specific embodiment of the present invention or an explanation of a specific embodiment. The protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A user interaction method based on ultrasound images, applied to a mobile terminal and a handheld ultrasound device, characterized in that: A communication connection is established between the mobile terminal and the handheld ultrasonic device, the handheld ultrasonic device is used to transmit ultrasonic waves to a target object, receive ultrasonic echoes returned by the target object and obtain ultrasonic echo signals, and process the ultrasonic echo signals to obtain ultrasonic image data; the mobile terminal receives the ultrasonic echo signals sent by the handheld ultrasonic device or receives the ultrasonic image data processed by the handheld ultrasonic device through the communication connection; The mobile terminal is installed with an application, and the application is used to provide a display interface, through which human-computer interaction can be performed; the method includes: After the mobile terminal runs the application, an ultrasonic image obtained based on the ultrasonic echo signal or the ultrasonic image data is displayed on a display interface of the mobile terminal; Generating a selection area mark on the ultrasound image, wherein a boundary or a boundary band of the selection area mark defines a first area and a second area on the ultrasound image, and the second area surrounds the first area; receiving a user's touch operation through the display interface; If the operation position of the touch operation is located in the first area, the function for adjusting the position of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the size of the selection area mark is activated; or, if the operation position of the touch operation is located in the first area, the function for adjusting the size of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the position of the selection area mark is activated; According to the adjustment function activated by the operation position of the touch operation, the selection area identifier is adjusted to obtain an adjusted selection area identifier; Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result; The recognition result is displayed.

2. A user interaction method based on ultrasound images, applied to a mobile terminal and a handheld ultrasound device, characterized in that: A communication connection is established between the mobile terminal and the handheld ultrasonic device, the handheld ultrasonic device is used to transmit ultrasonic waves to a target object, receive ultrasonic echoes returned by the target object and obtain ultrasonic echo signals, and process the ultrasonic echo signals to obtain ultrasonic image data; the mobile terminal receives the ultrasonic echo signals sent by the handheld ultrasonic device or receives the ultrasonic image data processed by the handheld ultrasonic device through the communication connection; The mobile terminal is installed with an application, and the application is used to provide a display interface, through which human-computer interaction can be performed; the method includes: After the mobile terminal runs the application, an ultrasonic image obtained based on the ultrasonic echo signal or the ultrasonic image data is displayed on a display interface of the mobile terminal; Generating a selection area mark on the ultrasound image, wherein the boundary of the selection area mark includes an outer boundary and an inner boundary, the outer boundary of the selection area mark defines a first area and a second area on the ultrasound image, and the second area surrounds the first area; receiving a user's touch operation through the display interface; If the operation position of the touch operation is located in the first area, the function for adjusting the position of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the size of the selection area mark is activated; or, if the operation position of the touch operation is located in the first area, the function for adjusting the size of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the position of the selection area mark is activated; According to the adjustment function activated by the operation position of the touch operation, the selection area identifier is adjusted to obtain an adjusted selection area identifier; Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result; The recognition result is displayed.

3. A user interaction method based on ultrasound images, applied to a mobile terminal and a handheld ultrasound device, characterized in that: A communication connection is established between the mobile terminal and the handheld ultrasonic device, the handheld ultrasonic device is used to transmit ultrasonic waves to a target object, receive ultrasonic echoes returned by the target object and obtain ultrasonic echo signals, and process the ultrasonic echo signals to obtain ultrasonic image data; the mobile terminal receives the ultrasonic echo signals sent by the handheld ultrasonic device or receives the ultrasonic image data processed by the handheld ultrasonic device through the communication connection; The mobile terminal is installed with an application, and the application is used to provide a display interface, through which human-computer interaction can be performed; the method includes: After the mobile terminal runs the application, an ultrasonic image obtained based on the ultrasonic echo signal or the ultrasonic image data is displayed on a display interface of the mobile terminal; Generate a selection mark on the ultrasound image, take the boundary of the selection mark as a reference and an area with a preset width surrounding the boundary of the selection mark as the second area, or take the boundary band of the selection mark as a reference and an area with a preset width surrounding the boundary band of the selection mark as the second area, and the area surrounded by the second area is the first area; receiving a user's touch operation through the display interface; If the operation position of the touch operation is located in the first area, the function for adjusting the position of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the size of the selection area mark is activated; or, if the operation position of the touch operation is located in the first area, the function for adjusting the size of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the position of the selection area mark is activated; According to the adjustment function activated by the operation position of the touch operation, the selection area identifier is adjusted to obtain an adjusted selection area identifier; Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result; The recognition result is displayed.

4. The user interaction method according to any one of claims 1 to 3, characterized in that: The touch operation is a click operation.

5. The user interaction method according to any one of claims 1 to 3, characterized in that: The step of generating a selection area mark on the ultrasound image comprises: The selection area identifier is generated based on a default size, a default position and / or a default shape.

6. The user interaction method according to any one of claims 1 to 3, characterized in that: The step of generating a selection area mark on the ultrasound image comprises: In response to a drawing operation performed by a user on the ultrasound image, the selection area mark is generated on the ultrasound image.

7. The user interaction method according to any one of claims 1 to 3, characterized in that: The recognition result includes at least one of the following: The outline of the target tissue, the radial position of the target tissue, the radial length of the target tissue, and the area of ​​the target tissue.

8. The user interaction method according to any one of claims 1 to 3, characterized in that: The method further comprises: If the target tissue is not identified in the ultrasound image region corresponding to the adjusted selection mark, a prompt message about the identification failure is displayed.

9. The user interaction method according to any one of claims 1 to 3, characterized in that: The method further includes: adjusting the shape of the selection area mark based on the touch operation, wherein: Based on the touch operation on the first operation control, cropping the selection area mark; Based on the touch operation on the second operation control, the selection area mark is filled.

10. A user interaction method based on ultrasound images, applied to a mobile terminal and a handheld ultrasound device, characterized in that: A communication connection is established between the mobile terminal and the handheld ultrasonic device, the handheld ultrasonic device is used to transmit ultrasonic waves to a target object, receive ultrasonic echoes returned by the target object and obtain ultrasonic echo signals, and process the ultrasonic echo signals to obtain ultrasonic image data; the mobile terminal receives the ultrasonic echo signals sent by the handheld ultrasonic device or receives the ultrasonic image data processed by the handheld ultrasonic device through the communication connection; The mobile terminal is installed with an application, and the application is used to provide a display interface, through which human-computer interaction can be performed; the method includes: After the mobile terminal runs the application, an ultrasonic image obtained based on the ultrasonic echo signal or the ultrasonic image data is displayed on a display interface of the mobile terminal; Generating a selection area mark on the ultrasound image, wherein the selection area mark defines at least two touch control areas on the ultrasound image; receiving a user's touch operation through the display interface; Based on the touch area corresponding to the touch operation, activating a target function among at least two functions for adjusting the selection area marker, the at least two functions comprising at least two of the following: a function of adjusting a position of the selection area marker, a function of adjusting a size of the selection area marker, and a function of adjusting a shape of the selection area marker; According to the target adjustment function, the selection area identifier is adjusted to obtain an adjusted selection area identifier; Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result; The recognition result is displayed.

11. The user interaction method according to claim 10, characterized in that: The selection area mark includes a frame representing the boundary or boundary zone of the selection area mark and an auxiliary mark located near the frame, and the method further includes: When a touch operation on the auxiliary identifier is received, a target function corresponding to the auxiliary identifier is activated.

12. The user interaction method according to claim 9, characterized in that: The step of generating a selection area mark on the ultrasound image comprises: The selection area identifier is generated based on a default size, a default position and / or a default shape.

13. The user interaction method according to claim 9, characterized in that: The step of generating a selection area mark on the ultrasound image comprises: In response to a drawing operation performed by a user on the ultrasound image, the selection area mark is generated on the ultrasound image.

14. The user interaction method according to claim 9, characterized in that: The recognition result includes at least one of the following: The outline of the target tissue, the radial position of the target tissue, the radial length of the target tissue, and the area of ​​the target tissue.

15. The user interaction method according to claim 9, characterized in that: The method further comprises: If the target tissue is not identified in the ultrasound image region corresponding to the adjusted selection mark, a prompt message about the identification failure is displayed.

16. A user interaction method based on ultrasound images, applied to a mobile terminal and a handheld ultrasound device, characterized in that: A communication connection is established between the mobile terminal and the handheld ultrasonic device, the handheld ultrasonic device is used to transmit ultrasonic waves to a target object, receive ultrasonic echoes returned by the target object and obtain ultrasonic echo signals, and process the ultrasonic echo signals to obtain ultrasonic image data; the mobile terminal receives the ultrasonic echo signals sent by the handheld ultrasonic device or receives the ultrasonic image data processed by the handheld ultrasonic device through the communication connection; The mobile terminal is installed with an application, and the application is used to provide a display interface, through which human-computer interaction can be performed; the method includes: After the mobile terminal runs the application, an ultrasonic image obtained based on the ultrasonic echo signal or the ultrasonic image data is displayed on a display interface of the mobile terminal; Generating a selection area mark on the ultrasound image; receiving a user's touch operation through the display interface; Based on the touch operation, a target function is activated in at least two functions for adjusting the selection area mark, wherein the at least two functions include at least two of the following: a function of adjusting the position of the selection area mark, a function of adjusting the size of the selection area mark, and a function of adjusting the shape of the selection area mark; wherein, If the touch operation is a single-finger operation, activating the function of adjusting the position of the selection area mark; If the touch operation is a two-finger operation, activating the function of adjusting the size of the selection area mark; If the touch operation is a touch operation on a preset operation control, activating the function of adjusting the shape of the selection area mark; According to the target adjustment function, the selection area identifier is adjusted to obtain an adjusted selection area identifier; Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result; The recognition result is displayed.

17. A user interaction method based on ultrasound images, applied to a mobile terminal and a handheld ultrasound device, characterized in that: A communication connection is established between the mobile terminal and the handheld ultrasonic device, the handheld ultrasonic device is used to transmit ultrasonic waves to a target object, receive ultrasonic echoes returned by the target object and obtain ultrasonic echo signals, and process the ultrasonic echo signals to obtain ultrasonic image data; the mobile terminal receives the ultrasonic echo signals sent by the handheld ultrasonic device or receives the ultrasonic image data processed by the handheld ultrasonic device through the communication connection; The mobile terminal is installed with an application, and the application is used to provide a display interface, through which human-computer interaction can be performed; the method includes: After the mobile terminal runs the application, an ultrasonic image obtained based on the ultrasonic echo signal or the ultrasonic image data is displayed on a display interface of the mobile terminal; In response to a user's drawing operation on the ultrasound image, generating a selection area mark on the ultrasound image; receiving a touch operation of the user based on the display interface; Based on the touch operation, adjusting the shape or size of the selection area mark to obtain an adjusted selection area mark; Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result; The recognition result is displayed.

18. The user interaction method according to claim 17, characterized in that: The adjusting the shape of the selection area mark based on the touch operation includes: Based on the touch operation on the first operation control, cropping the selection area mark; Based on the touch operation on the second operation control, the selection area mark is filled.

19. The user interaction method according to claim 17, characterized in that: The drawing operation includes a drawing operation on a border of the selection area mark.

20. The user interaction method according to claim 19, characterized in that: The drawing operation of the border of the selection area mark includes a line drawing operation, and the method further includes: The line is automatically closed to obtain a border of the selection area mark.

21. The user interaction method according to claim 19, characterized in that: The drawing operation of the border of the selection area mark includes a point drawing operation, and the method further includes: A border of the selection area mark is generated based on the multiple points drawn by the point drawing operation.

22. The user interaction method according to claim 17, characterized in that: The drawing operation includes a drawing operation on the internal structure of the selection area identifier, and the method further includes: The preset width is expanded outward with the internal structure as the center to obtain the border of the selection area mark.

23. The user interaction method according to claim 17, characterized in that: The method further comprises: Based on the touch operation, the position of the selection area marker is adjusted to obtain the adjusted selection area marker.

24. A user interaction method based on ultrasound images, characterized in that: The method comprises: Displaying the ultrasound image in a display interface; Generating a selection area mark on the ultrasound image, wherein the boundary of the selection area mark defines a first area and a second area on the ultrasound image, and the second area surrounds the first area; receiving a user's touch operation through the display interface; If the operation position of the touch operation is located in the first area, the function for adjusting the position of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the size of the selection area mark is activated; or, if the operation position of the touch operation is located in the first area, the function for adjusting the size of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the position of the selection area mark is activated; According to the adjustment function activated by the operation position of the touch operation, the selection area identifier is adjusted to obtain an adjusted selection area identifier; Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result; The recognition result is displayed.

25. A user interaction method based on ultrasound images, characterized in that: The method comprises: Displaying the ultrasound image in a display interface; In response to a user's drawing operation on the ultrasound image, generating a selection area mark in the ultrasound image; receiving a touch operation of the user based on the display interface; Based on the touch operation, adjusting the shape or size of the selection area mark to obtain an adjusted selection area mark; Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result; The recognition result is displayed.

26. A user interaction method based on ultrasound images, characterized in that: The method comprises: Displaying the ultra-sonic image in the display interface; Generating a selection area mark on the ultrasound image, wherein the boundary of the selection area mark includes an outer boundary and an inner boundary, the outer boundary of the selection area mark defines a first area and a second area on the ultrasound image, and the second area surrounds the first area; receiving a user's touch operation through the display interface; If the operation position of the touch operation is located in the first area, the function for adjusting the position of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the size of the selection area mark is activated; or, if the operation position of the touch operation is located in the first area, the function for adjusting the size of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the position of the selection area mark is activated; According to the adjustment function activated by the operation position of the touch operation, the selection area identifier is adjusted to obtain an adjusted selection area identifier; Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result; The recognition result is displayed.

27. A user interaction method based on ultrasound images, characterized in that: The method comprises: Displaying the ultrasound image in a display interface; Generate a selection mark on the ultrasound image, take the boundary of the selection mark as a reference and an area with a preset width surrounding the boundary of the selection mark as the second area, and the area surrounded by the second area as the first area; receiving a user's touch operation through the display interface; If the operation position of the touch operation is located in the first area, the function for adjusting the position of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the size of the selection area mark is activated; or, if the operation position of the touch operation is located in the first area, the function for adjusting the size of the selection area mark is activated, and if the operation position of the touch operation is located in the second area, the function for adjusting the position of the selection area mark is activated; According to the adjustment function activated by the operation position of the touch operation, the selection area identifier is adjusted to obtain an adjusted selection area identifier; Identifying the target tissue in the area of ​​the ultrasound image corresponding to the adjusted selection mark to obtain an identification result; The recognition result is displayed.

28. The user interaction method according to any one of claims 24-27, characterized in that: The method is used in a mobile terminal.

29. An ultrasonic imaging system, characterized in that: It includes a mobile terminal and a handheld ultrasound device, wherein a communication connection is established between the mobile terminal and the handheld ultrasound device; The handheld ultrasonic device is used to transmit ultrasonic waves to a target object, receive ultrasonic echoes returned by the target object and obtain ultrasonic echo signals, and process the ultrasonic echo signals to obtain ultrasonic image data; The mobile terminal receives the ultrasonic echo signal or the ultrasonic image data processed by the handheld ultrasonic device through the communication connection; the mobile terminal is installed with an application, which is used to provide a display interface through which human-computer interaction can be performed; the mobile terminal is also used to execute the user interaction method described in any one of claims 1-28.

Images