Display method for endoscope system, endoscope system, and ultrasound imaging system

Abstract

The invention discloses a display method for an endoscope system and the endoscope system, and the method comprises the steps: obtaining first content related to ultrasonic imaging from an ultrasonic imaging system in communication connection with the endoscope system, wherein an interface of an ultrasonic display of the ultrasonic imaging system at least has a menu area for displaying an interaction menu and an image area for displaying an ultrasonic image, and the first content related to the ultrasonic imaging comprises content of one or more selected from the menu area and the image area based on a user selection instruction. According to the present invention, contents of one or more selected from a menu area and an image area based on a user selection instruction are selected on an ultrasonic display and displayed on an endoscope display. The endoscope display not only displays the endoscope image, but also displays the first content related to the ultrasonic imaging selected by the user from the ultrasonic display on the same interface, so that the user does not need to switch the sight between the endoscope display and the ultrasonic display back and forth, and the operation is very convenient.

Description

Technical Field

[0001] The present invention relates to the field of medical equipment, and more particularly to a display method for an endoscope system, an endoscope system, and an ultrasonic imaging system. Background Art

[0002] Due to surgical requirements, a wide variety of medical devices are used in operating rooms, such as surgical operating rooms. Endoscopes are a key component of these devices. They can visualize the morphology of a patient's internal organs and the condition of internal lesions during minimally invasive surgery, facilitating diagnosis and surgical intervention. They are a crucial tool for diagnosis and treatment in modern medicine. In addition to viewing images on an endoscope display using an endoscope system, doctors often need to view images captured by other medical devices, such as ultrasound images on an ultrasound display using an ultrasound imaging system. This can be inconvenient for doctors in real-world scenarios. Summary of the Invention

[0003] To solve the above problems, the present invention provides a display method for an endoscope system, an endoscope system, and an ultrasonic imaging system, which are described in detail below.

[0004] According to a first aspect, a display method for an endoscope system includes:

[0005] Acquiring endoscopic image signals of the part to be observed;

[0006] generating an endoscopic image based on the endoscopic image signal;

[0007] acquiring first content related to ultrasound imaging from an ultrasound imaging system communicatively connected to the endoscope system, wherein an interface of an ultrasound display of the ultrasound imaging system has at least a menu area for displaying an interactive menu and an image area for displaying an ultrasound image, and the first content related to ultrasound imaging includes one or more contents selected from the menu area and the image area based on a user selection instruction;

[0008] The endoscopic image and the first content are displayed on an interface of an endoscope display of the endoscope system.

[0009] In one embodiment, the interface of the endoscope display includes a first area and a second area, the endoscope image is displayed in the first area, and the first content is displayed in the second area.

[0010] In one embodiment, displaying the endoscopic image and the first content on the interface of the endoscope display of the endoscope system includes: positionally aligning the endoscopic image and the ultrasound image, and fusing the endoscopic image and the ultrasound image after position alignment to generate and display a fused image.

[0011] In one embodiment, the first operating component of the ultrasound probe of the ultrasound imaging system can be operated by a user to generate a display parameter instruction, wherein the display parameter instruction is used to set the display parameters of the endoscope display; and / or, the first operating component of the ultrasound probe of the ultrasound imaging system can be operated by a user to generate an imaging parameter instruction, wherein the imaging parameter instruction is used to set the imaging parameters of the ultrasound imaging system; and / or, the first operating component of the ultrasound probe of the ultrasound imaging system can be operated by a user to generate an ultrasound operation instruction, wherein the ultrasound operation instruction is used to execute a freeze, save, or unfreeze function on the ultrasound imaging system;

[0012] and / or,

[0013] The handheld operating part of the endoscope system has a second operating component; the second operating component can be operated by the user to generate display parameter instructions, and the display parameter instructions are used to set the display parameters of the endoscope display; and / or, the second operating component can be operated by the user to generate imaging parameter instructions, and the imaging parameter instructions are used to set the imaging parameters of the ultrasound imaging system; and / or, the second operating component can be operated by the user to generate ultrasound operation instructions, and the ultrasound operation instructions are used to perform a freeze or unfreeze function on the ultrasound imaging system.

[0014] In one embodiment, when a first operating component of the ultrasound probe of the ultrasound imaging system is operated by a user, the first content further includes an interactive menu generated when the first operating component is operated; and / or, the handheld operating portion of the endoscope system has a second operating component, and when the second operating component is operated by the user, the endoscope display 270 displays the interactive menu generated when the second operating component is operated;

[0015] The interactive menu is used for the user to set the display parameters of the endoscope display through the first operating component; and / or, the interactive menu is used for the user to set the imaging parameters of the ultrasound imaging system through the first operating component; and / or, the interactive menu is used for the user to perform a freeze, save or unfreeze function on the ultrasound imaging system through the first operating component.

[0016] In one embodiment, the display parameters include one or more of the following:

[0017] a ratio of the first area to the second area;

[0018] the size and / or location of the first area;

[0019] the size and / or location of the second area;

[0020] The resolution, refresh rate or brightness of the interface of the endoscope display.

[0021] In one embodiment, the imaging parameters include one or more of the following:

[0022] one or more imaging modalities;

[0023] Imaging sub-parameters corresponding to each imaging mode;

[0024] Navigation function parameters.

[0025] In one embodiment, the imaging mode includes one or more of a B mode, a C mode, a PW mode, a CW mode, a Power mode, a contrast imaging mode, and an elastic imaging mode;

[0026] The imaging sub-parameters corresponding to the B-mode include: one or more of gain, frequency, focus, depth and dynamic range;

[0027] The imaging sub-parameters corresponding to the C-mode include one or more of gain, frequency, pulse repetition frequency, sampling frame size, sampling line angle, inversion color, wall filtering, smoothing, persistence, and color map;

[0028] The imaging sub-parameters corresponding to the PW mode include one or more of gain, frequency, pulse repetition frequency, sample volume, sampling line angle, wall filter and baseline position;

[0029] The imaging sub-parameters corresponding to the CW mode include one or more of gain, frequency, sampling line angle, and wall filtering;

[0030] The imaging sub-parameters corresponding to the Power mode include one or more of gain, frequency, pulse repetition frequency, sampling frame size, sampling line angle, wall filtering, smoothing, persistence, dynamic range and energy spectrum;

[0031] The imaging sub-parameters corresponding to the contrast imaging mode include one or more of focus, frequency, acoustic output power, and MI mechanical index;

[0032] The imaging sub-parameters corresponding to the elastic imaging mode include one or more of shear wave elasticity, strain elasticity, transient elasticity and viscoelasticity;

[0033] The navigation function parameters include one or more of tumor parameters, guide line parameters, and guide region parameters.

[0034] In one embodiment, the first operating component of the ultrasound probe of the ultrasound imaging system can be operated by a user to generate a close instruction, wherein the close instruction is used to close the display of the first content on the interface of the endoscope display; and / or the first operating component of the ultrasound probe of the ultrasound imaging system can be operated by a user to generate a start instruction, wherein the start instruction is used to start the display of the first content on the interface of the endoscope display;

[0035] And / or, the second operating component of the handheld operating part of the endoscope system can be operated by the user to generate a close instruction, and the close instruction is used to turn off the display of the first content on the interface of the endoscope display; and / or, the second operating component of the handheld operating part of the endoscope system can be operated by the user to generate an open instruction, and the open instruction is used to turn on the display of the first content on the interface of the endoscope display.

[0036] In one embodiment, the endoscope system and the ultrasound imaging system are communicated via a wired or wireless method; the wireless method includes a Wi-Fi connection, a WHDI connection, or a WiHD connection.

[0037] According to a second aspect, a display method for an endoscope system includes:

[0038] Acquiring endoscopic image signals of the part to be observed;

[0039] generating an endoscopic image based on the endoscopic image signal;

[0040] When the first operating component of the ultrasound probe of the ultrasound imaging system is operated by the user to generate an opening instruction, the endoscopic image and the first content related to the ultrasound imaging are displayed on the interface of the endoscope display of the endoscope system; wherein, the first content related to the ultrasound imaging is obtained from the ultrasound imaging system that is communicatively connected to the endoscope system.

[0041] In one embodiment, when the first operating component of the ultrasound probe is operated by the user to generate a closing instruction, the first content displayed on the interface of the endoscope display is closed.

[0042] In one embodiment, the interface of the ultrasound display of the ultrasound imaging system has at least a menu area for displaying an interactive menu and an image area for displaying an ultrasound image, and the first content related to ultrasound imaging includes one or more contents selected from the menu area and the image area based on a user selection instruction.

[0043] In one embodiment, the interface of the endoscope display includes a first area and a second area, the endoscope image is displayed in the first area, and the first content is displayed in the second area.

[0044] In one embodiment, displaying the endoscopic image and the first content on the interface of the endoscope display of the endoscope system includes: positionally aligning the endoscopic image and the ultrasound image, and fusing the endoscopic image and the ultrasound image after position alignment to generate and display a fused image.

[0045] In one embodiment, the first operating component of the ultrasound probe of the ultrasound imaging system can be operated by a user to generate a display parameter instruction, wherein the display parameter instruction is used to set the display parameters of the endoscope display; and / or, the first operating component of the ultrasound probe of the ultrasound imaging system can be operated by a user to generate an imaging parameter instruction, wherein the imaging parameter instruction is used to set the imaging parameters of the ultrasound imaging system; and / or, the first operating component of the ultrasound probe of the ultrasound imaging system can be operated by a user to generate an ultrasound operation instruction, wherein the ultrasound operation instruction is used to execute a freeze, save, or unfreeze function on the ultrasound imaging system;

[0046] and / or,

[0047] The handheld operating part of the endoscope system has a second operating component; the second operating component can be operated by the user to generate display parameter instructions, and the display parameter instructions are used to set the display parameters of the endoscope display; and / or, the second operating component can be operated by the user to generate imaging parameter instructions, and the imaging parameter instructions are used to set the imaging parameters of the ultrasound imaging system; and / or, the second operating component can be operated by the user to generate ultrasound operation instructions, and the ultrasound operation instructions are used to perform a freeze or unfreeze function on the ultrasound imaging system.

[0048] In one embodiment, when a first operating component of the ultrasound probe of the ultrasound imaging system is operated by a user, the first content further includes an interactive menu generated when the first operating component is operated; and / or, the handheld operating portion of the endoscope system has a second operating component, and when the second operating component is operated by the user, the endoscope display 270 displays the interactive menu generated when the second operating component is operated;

[0049] The interactive menu is used for the user to set the display parameters of the endoscope display through the first operating component; and / or, the interactive menu is used for the user to set the imaging parameters of the ultrasound imaging system through the first operating component; and / or, the interactive menu is used for the user to perform a freeze, save or unfreeze function on the ultrasound imaging system through the first operating component.

[0050] In one embodiment, the display parameters include one or more of the following:

[0051] a ratio of the first area to the second area;

[0052] the size and / or location of the first area;

[0053] the size and / or location of the second area;

[0054] The resolution, refresh rate or brightness of the interface of the endoscope display.

[0055] In one embodiment, the imaging parameters include one or more of the following:

[0056] one or more imaging modalities;

[0057] Imaging sub-parameters corresponding to each imaging mode;

[0058] Navigation function parameters.

[0059] In one embodiment, the imaging mode includes one or more of a B mode, a C mode, a PW mode, a CW mode, a Power mode, a contrast imaging mode, and an elastic imaging mode;

[0060] The imaging sub-parameters corresponding to the B-mode include: one or more of gain, frequency, focus, depth and dynamic range;

[0061] The imaging sub-parameters corresponding to the C-mode include one or more of gain, frequency, pulse repetition frequency, sampling frame size, sampling line angle, inversion color, wall filtering, smoothing, persistence, and color map;

[0062] The imaging sub-parameters corresponding to the PW mode include one or more of gain, frequency, pulse repetition frequency, sample volume, sampling line angle, wall filter and baseline position;

[0063] The imaging sub-parameters corresponding to the CW mode include one or more of gain, frequency, sampling line angle, and wall filtering;

[0064] The imaging sub-parameters corresponding to the Power mode include one or more of gain, frequency, pulse repetition frequency, sampling frame size, sampling line angle, wall filtering, smoothing, persistence, dynamic range and energy spectrum;

[0065] The imaging sub-parameters corresponding to the contrast imaging mode include one or more of focus, frequency, acoustic output power, and MI mechanical index;

[0066] The imaging sub-parameters corresponding to the elastic imaging mode include one or more of shear wave elasticity, strain elasticity, transient elasticity and viscoelasticity;

[0067] The navigation function parameters include one or more of tumor parameters, guide line parameters, and guide region parameters.

[0068] In one embodiment, the first operating component of the ultrasound probe of the ultrasound imaging system can be operated by a user to generate a close instruction, wherein the close instruction is used to close the display of the first content on the interface of the endoscope display; and / or the first operating component of the ultrasound probe of the ultrasound imaging system can be operated by a user to generate a start instruction, wherein the start instruction is used to start the display of the first content on the interface of the endoscope display;

[0069] And / or, the second operating component of the handheld operating part of the endoscope system can be operated by the user to generate a close instruction, and the close instruction is used to turn off the display of the first content on the interface of the endoscope display; and / or, the second operating component of the handheld operating part of the endoscope system can be operated by the user to generate an open instruction, and the open instruction is used to turn on the display of the first content on the interface of the endoscope display.

[0070] In one embodiment, the endoscope system and the ultrasound imaging system are communicated via a wired or wireless method; the wireless method includes a Wi-Fi connection, a WHDI connection, or a WiHD connection.

[0071] According to a third aspect, an embodiment provides an endoscope system, comprising an endoscope, a light source, a camera host connected to the endoscope, and an endoscope display for displaying;

[0072] The light source is used to provide illumination to the patient's part to be observed;

[0073] The endoscope includes an insertion portion and a handheld operation portion, wherein the insertion portion is used to be inserted into a patient to observe the part to be observed; the handheld operation portion is used for handheld operation by a user; the endoscope is provided with one or more image sensors, and the image sensors are used to collect endoscopic image signals of the part to be observed;

[0074] The camera host is used in the method described in any embodiment of this document.

[0075] According to a fourth aspect, an embodiment provides an ultrasound imaging system, comprising:

[0076] An ultrasonic probe comprises a housing, an array element group, and a first operating component disposed on the housing; the array element group is used to transmit ultrasonic waves and receive echo signals of the ultrasonic waves; the first operating component is used for user operation;

[0077] an ultrasound host, configured to generate an ultrasound image according to the echo signal;

[0078] an ultrasound display, configured to display the ultrasound image;

[0079] in:

[0080] The first operating component of the ultrasound probe is used for a user to operate to generate an interactive menu, and the interactive menu is used for the user to control ultrasound imaging through the first operating component; the ultrasound display displays the interactive menu.

[0081] In one embodiment, the interactive menu is used for the user to set imaging parameters of the ultrasound image through the first operating component; and / or, the interactive menu is used for the user to freeze, save or unfreeze the ultrasound image through the first operating component.

[0082] In one embodiment, the imaging parameters include one or more of the following:

[0083] one or more imaging modalities;

[0084] Imaging sub-parameters corresponding to each imaging mode;

[0085] Navigation function parameters.

[0086] In one embodiment, the imaging mode includes one or more of a B mode, a C mode, a PW mode, a CW mode, a Power mode, a contrast imaging mode, and an elastic imaging mode;

[0087] The imaging sub-parameters corresponding to the B-mode include: one or more of gain, frequency, focus, depth and dynamic range;

[0088] The imaging sub-parameters corresponding to the C-mode include one or more of gain, frequency, pulse repetition frequency, sampling frame size, sampling line angle, inversion color, wall filtering, smoothing, persistence, and color map;

[0089] The imaging sub-parameters corresponding to the PW mode include one or more of gain, frequency, pulse repetition frequency, sample volume, sampling line angle, wall filter and baseline position;

[0090] The imaging sub-parameters corresponding to the CW mode include one or more of gain, frequency, sampling line angle, and wall filtering;

[0091] The imaging sub-parameters corresponding to the Power mode include one or more of gain, frequency, pulse repetition frequency, sampling frame size, sampling line angle, wall filtering, smoothing, persistence, dynamic range and energy spectrum;

[0092] The imaging sub-parameters corresponding to the contrast imaging mode include one or more of focus, frequency, acoustic output power, and MI mechanical index;

[0093] The imaging sub-parameters corresponding to the elastic imaging mode include one or more of shear wave elasticity, strain elasticity, transient elasticity and viscoelasticity;

[0094] The navigation function parameters include one or more of tumor parameters, guide line parameters, and guide region parameters.

[0095] In one embodiment, the interactive menu includes a first-level menu and a second-level menu, and the first-level menu has first-level options for users to select imaging modes and / or navigation functions; after any first-level option of the interactive menu is selected, the interactive menu will expand the second-level menu associated with the selected first-level option; wherein, the second-level menu associated with the first-level option of the imaging mode is a setting option for the imaging sub-parameters corresponding to the imaging mode, and the second-level menu associated with the first-level option of the navigation function is a setting option for the navigation function parameters.

[0096] In one embodiment, the first operating component includes a menu button and direction keys arranged on both sides of the menu button, which are used to generate or close the interactive menu when the menu button is triggered, and the direction keys are used to select the first-level options of the first-level menu and the setting options of the second-level menu.

[0097] In one embodiment, the ultrasound imaging system further includes a first communication component, which is used to communicate with an endoscope system; the ultrasound host sends first content related to ultrasound imaging to the endoscope system through the first communication component, so that the endoscope system displays the endoscope image and the first content on the interface of its endoscope display; wherein the first content includes the ultrasound image and / or the interactive menu.

[0098] In one embodiment, the interface of the ultrasound display of the ultrasound imaging system has at least a menu area for displaying an interactive menu and an image area for displaying an ultrasound image, and the first content related to ultrasound imaging includes one or more contents selected from the menu area and the image area based on a user selection instruction.

[0099] In one embodiment, the ultrasound host sends the first content related to ultrasound imaging to the endoscope system through the first communication component, so that the endoscope system displays the endoscopic image and the first content on the interface of its endoscope display, including: the ultrasound host sends the first content related to ultrasound imaging to the endoscope system through the first communication component, so that the endoscope system displays the endoscopic image in the first area of ​​the interface of its endoscope display, and displays the ultrasound image and / or the interactive menu in the second area.

[0100] In one embodiment, the interactive menu is used for the user to set display parameters of the endoscope display through the first operating component; the display parameters include one or more of the following:

[0101] a ratio of the first area to the second area;

[0102] the size and / or location of the first area;

[0103] the size and / or location of the second area;

[0104] The resolution, refresh rate or brightness of the interface of the endoscope display.

[0105] According to the display method and the endoscope system for an endoscope system of the above-mentioned embodiment, by selecting one or more contents selected from the menu area and the image area based on a user selection instruction on the ultrasound display and displaying them on the endoscope display, the endoscope display can display not only the endoscopic image but also the first content related to ultrasound imaging selected by the user on the ultrasound display on the same interface, so that the user does not need to switch his or her line of sight back and forth between the endoscope display and the ultrasound display, which is very convenient.

[0106] According to the display method for an endoscope system and the endoscope system of the above-mentioned embodiment, an opening instruction is generated based on the first operating component of the ultrasound probe, so that an endoscopic image and first content related to ultrasound imaging acquired from the ultrasound imaging system communicatively connected to the endoscope system can be displayed on the same interface through the endoscope display, thereby allowing the user to control whether to view the endoscopic image and the first content related to ultrasound imaging simultaneously on the endoscope display as needed;

[0107] According to the ultrasound imaging system of the above embodiment, the first operating component of the ultrasound probe is used for the user to operate to generate an interactive menu, and the interactive menu is used for the user to control the ultrasound imaging through the first operating component; considering that the ultrasound probe itself has a small area for setting function keys, an interactive menu is specially designed for the first operating component of the ultrasound probe, so that the number of function keys on the ultrasound probe can be reduced, and more functions can be achieved with as few function keys as possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0108] Figure 1 is a schematic structural diagram of an ultrasound imaging system according to an embodiment;

[0109] Figure 2 1 is a schematic structural diagram of an ultrasonic probe according to an embodiment;

[0110] Figure 3 is a schematic structural diagram of a first operating component according to an embodiment;

[0111] Figure 4 A schematic diagram of an interactive menu according to an embodiment;

[0112] FIG5(a) is a schematic diagram showing an example of performing data rearrangement; FIG5(b) is a schematic diagram showing another example of performing data rearrangement; FIG5(c) is a schematic diagram showing yet another example of performing data rearrangement;

[0113] Figure 6 1 is a schematic structural diagram of an ultrasound host according to an embodiment;

[0114] Figure 7 An exemplary diagram of an interface of an ultrasound display according to an embodiment;

[0115] Figure 8 is a schematic structural diagram of an endoscope system according to an embodiment;

[0116] Figure 9 1 is a flow chart of a display method for an endoscope system according to an embodiment;

[0117] Figure 10 An exemplary diagram of an interface of an endoscope display according to an embodiment;

[0118] Figure 11 is a schematic diagram of an embodiment in which an ultrasound imaging system communicates with an endoscope system via a first communication component;

[0119] Figure 12 is a schematic diagram of an endoscope system communicating with an ultrasound imaging system via a second communication component in one embodiment;

[0120] Figure 13 The figure is a schematic structural diagram of an imaging system for an operating room according to an embodiment. DETAILED DESCRIPTION

[0121] The present invention will be further described in detail below by means of specific embodiments in conjunction with the accompanying drawings. Similar elements in different embodiments are numbered with associated similar elements. In the following embodiments, many detailed descriptions are provided to enable the present application to be better understood. However, those skilled in the art will readily appreciate that some of the features may be omitted in different circumstances, or may be replaced by other elements, materials, or methods. In some cases, some operations related to the present application are not shown or described in the specification. This is to avoid the core portion of the present application being overwhelmed by excessive descriptions, and for those skilled in the art, it is not necessary to describe these related operations in detail. They will fully understand the related operations based on the description in the specification and the general technical knowledge in the art.

[0122] In addition, the features, operations, or characteristics described in the specification may be combined in any appropriate manner to form various embodiments. Furthermore, the steps or actions in the method description may be reordered or adjusted in a manner readily apparent to those skilled in the art. Therefore, the various sequences in the specification and drawings are provided solely for the purpose of clearly describing a particular embodiment and are not intended to be mandatory, unless otherwise specified.

[0123] The serial numbers assigned to components herein, such as "first," "second," etc., are used solely to distinguish the objects being described and do not convey any sequential or technical meaning. References to "connection" and "coupling" herein, unless otherwise specified, include both direct and indirect connections (couplings).

[0124] Different medical devices display information, content, and images on their respective displays. However, in special scenarios such as operating rooms, users (e.g., doctors) need to use different medical devices and view images captured by multiple different medical devices. This requires users to switch their vision back and forth between the display screens of various medical devices. This is also very inconvenient for users when they need to carefully compare the images captured by different medical devices.

[0125] The following is an example of an ultrasonic imaging system and an endoscope system commonly used in an operating room.

[0126] Figure 1 is an example diagram of an ultrasound imaging system 100 ; in some embodiments, the ultrasound imaging system 100 includes an ultrasound probe 10 , an ultrasound host 30 , and an ultrasound display 50 , which are described in detail below.

[0127] The ultrasonic probe 10 is used to transmit ultrasonic waves and receive echo signals of the ultrasonic waves. Specifically, the ultrasonic probe 10 is used to transmit ultrasonic waves to a region of interest, receive ultrasonic echoes returned from the region of interest, and convert the ultrasonic echoes into electrical signals to obtain ultrasonic echo signals.

[0128] In some embodiments, the ultrasound probe 10 includes a housing 10 and an array element group 11; Figure 2 As an example, the following is a detailed description.

[0129] In some embodiments, the housing 10 is formed with a hand-grip area for holding. A user can hold the ultrasound probe 10 through the hand-grip area of ​​the housing 10 .

[0130] In some embodiments, the array element group 11 is used to transmit ultrasound waves toward the region of interest and receive ultrasound echoes reflected from the region of interest to obtain ultrasound echo signals. In some embodiments, the array element group 11 includes multiple array elements, which are used to convert electrical pulse signals into ultrasound waves, thereby transmitting ultrasound waves toward the region of interest and receiving ultrasound echoes reflected from tissue. The ultrasound echoes are then converted into electrical signals to obtain ultrasound echo signals. In some embodiments, the multiple array elements included in the array element group 11 can be arranged in a row to form a linear array; in some embodiments, the multiple array elements included in the array element group 11 can be arranged in a two-dimensional matrix to form a planar array. The array elements can transmit ultrasound waves based on an excitation electrical signal or convert received ultrasound waves into electrical signals. Therefore, each array element can be used to transmit ultrasound waves toward the region of interest or to receive ultrasound echoes returned from the region of interest. During ultrasound testing, a transmit sequence and a receive sequence can be used to control which array elements are used for transmitting and which are used for receiving ultrasound waves, or to control the time slots in which the array elements are used for transmitting and receiving ultrasound echoes. All array elements involved in ultrasonic emission can be excited by electrical signals simultaneously, thereby emitting ultrasonic waves simultaneously; or the array elements involved in ultrasonic emission can also be excited by several electrical signals with a certain time interval, thereby continuously emitting ultrasonic waves with a certain time interval.

[0131] In some embodiments, the ultrasound probe 10 may further include a first operating component 13, which is configured for user operation, such as for user control of ultrasound imaging. For example, the user may use the first operating component 13 to set imaging parameters, or to freeze, save, and / or unfreeze a displayed ultrasound image.

[0132] Through the first operating component 13 on the ultrasound probe 10, the user does not need to operate the operation panel of the ultrasound host 30. Instead, the user can directly control the ultrasound imaging by operating the first operating component 13 of the ultrasound probe 10 while holding the ultrasound probe 10 for ultrasound imaging, which is very convenient.

[0133] The first operating component 13 may include one or more physical function keys. Since the ultrasound probe 10 itself has a relatively small area for setting function keys, in order to reduce the number of function keys and achieve more functions with as few function keys as possible, an interactive menu may be designed specifically for the first operating component 13.

[0134] In some embodiments, the first operating component 13 of the ultrasound probe 10 is configured to be operated by a user to generate an interactive menu, so that the ultrasound display 50 displays the interactive menu.

[0135] In some embodiments, the interactive menu is used for the user to control ultrasound imaging through the first operating component 13 , wherein the interactive menu may be displayed on the ultrasound display 50 .

[0136] In some embodiments, the interactive menu is used for the user to set imaging parameters through the first operating component 13. In some embodiments, the interactive menu is used for the user to freeze, save, or unfreeze the ultrasound image through the first operating component 13.

[0137] In some embodiments, the imaging parameters include one or more of the following:

[0138] 1) One or more imaging modes, such as one or more of B-mode, C-mode, PW mode, CW mode, Power mode, contrast imaging mode, and elastography mode.

[0139] 2) Imaging sub-parameters corresponding to each imaging mode;

[0140] For example, the imaging sub-parameters corresponding to the B-mode include: one or more of gain, frequency, focus, depth, and dynamic range;

[0141] For example, the imaging sub-parameters corresponding to the C mode include one or more of gain, frequency, pulse repetition frequency, sampling frame size, sampling line angle, inversion color, wall filtering, smoothing, persistence, and color map;

[0142] For example, the imaging sub-parameters corresponding to the PW mode include one or more of gain, frequency, pulse repetition frequency, sampling volume, sampling line angle, wall filter, and baseline position;

[0143] For example, the imaging sub-parameters corresponding to the CW mode include one or more of gain, frequency, sampling line angle, and wall filtering;

[0144] For example, the imaging sub-parameters corresponding to the Power mode include one or more of gain, frequency, pulse repetition frequency, sampling frame size, sampling line angle, wall filtering, smoothing, persistence, dynamic range, and energy spectrum;

[0145] For example, the imaging sub-parameters corresponding to the contrast imaging mode include one or more of focus, frequency, acoustic output power, and MI mechanical index;

[0146] For example, the imaging sub-parameters corresponding to the elastic imaging mode include one or more of shear wave elasticity, strain elasticity, transient elasticity, and viscoelasticity.

[0147] 3) Navigation function parameters; for example, navigation function parameters include one or more of tumor parameters, guide line parameters, and guide region parameters.

[0148] In some embodiments, the interactive menu includes a first-level menu and a second-level menu. Imaging modes and / or navigation functions can be arranged in the first-level menu, so that the first-level menu includes first-level options for the user to select an imaging mode and / or navigation function. After any first-level option in the interactive menu is selected, the interactive menu will expand the second-level menu associated with the selected first-level option; wherein, the second-level menu associated with the first-level option of the imaging mode contains setting options for the imaging sub-parameters corresponding to the imaging mode, and the second-level menu associated with the first-level option of the navigation function contains setting options for the navigation function parameters.

[0149] In some embodiments, the first operating component 13 includes a menu button 13a and direction keys 13b provided on both sides of the menu button 13a. There may be one or more direction keys 13b, for example Figure 3 This is an example of four direction keys 13b, which respectively represent up, down, left and right movements on the interface; when the menu button 13a is triggered, it is used to generate or close the interactive menu - for example, by double-clicking the menu button 13a to generate the interactive menu, and double-clicking the menu button 13a again to close the interactive menu; the direction key 13b is used to select the first-level options of the first-level menu and the setting options of the second-level menu.

[0150] Figure 4 It is an example of an interactive menu; in some examples, the user generates or calls out the interactive menu by double-clicking the menu button 13a, and the user moves between the first-level options in the first-level menu by using the direction key 13b; after staying on any first-level option for more than a preset time, the interactive menu automatically expands the second-level menu associated with the first-level option, or the user selects the first-level option by clicking the menu button 13a, and after the first-level option is selected, the interactive menu expands the second-level menu associated with the first-level option, or the user selects the first-level option by using the right direction key 13b, and after the first-level option is selected, the interactive menu expands the second-level menu associated with the first-level option. After entering the secondary menu, the user can use the arrow keys 13b to move between the various setting options in the second-level menu. For example, after entering the second-level menu of the B mode, if the second-level menu of the B mode includes setting options for gain, frequency, focus, depth, and dynamic range, the arrow keys 13b can be used to move between the various setting options in the B mode; the user selects the setting options of the second-level menu by clicking the menu button 13a, or the user selects the setting options of the second-level menu by using the right arrow key 13b; if the setting option requires a numerical value, the setting option of the second-level menu can have a default numerical value or a numerical value corresponding to the current ultrasound image after it is selected, and the numerical value can be adjusted by, for example, the up and down arrow keys 13b. It should be noted that in Figure 4In the example of the interactive menu, C mode is selected and the second-level menu corresponding to C mode is expanded.

[0151] It should be noted that the above are only some examples of interaction between buttons and interactive menus. In specific embodiments, technicians can also design specific interaction logic between buttons and interactive menus based on needs.

[0152] In some embodiments, the options and settings of the interactive menu can also be configured based on user needs. For example, if the user does not need all imaging modes, but only needs a few imaging modes that the user frequently uses, the first level of the interactive menu can be configured, for example, through the ultrasound host 30, to retain only the user's frequently used imaging modes. Similarly, if the user only needs to adjust a few of the user's frequently used imaging sub-parameters in a certain imaging mode, the user can configure, for example, through the ultrasound host 30, the second level of the menu in the corresponding imaging mode to retain only the setting options corresponding to the user's frequently used imaging sub-parameters.

[0153] The above is a description of the first operating component 13 on the ultrasound probe 10. As can be seen, in some examples, the user can switch imaging modes and adjust and set imaging sub-parameters in specific imaging modes by operating the first operating component 13 on the ultrasound probe 10. In some examples, the user can freeze, save, and / or unfreeze ultrasound images by operating the first operating component 13 on the ultrasound probe 10.

[0154] In some embodiments, the ultrasound host 30 is used to process the ultrasonic echo signal to obtain ultrasound data. The ultrasound host 30 and the ultrasound probe can be connected by wire or wirelessly.

[0155] In some embodiments, the ultrasound host 30 processes the ultrasound echo signals and generates ultrasound images, which involves one or more processing steps. For example, the processing of the ultrasound echo signals may include analog-to-digital conversion, signal demodulation, amplification, filtering, downsampling, beamforming, modulo extraction, logarithmic compression, and grayscale conversion. The following describes each data processing step.

[0156] The ultrasonic echo signal received by the array element group 11 of the ultrasonic probe 10 is an analog signal, and digital ultrasonic data is obtained after the analog-to-digital conversion link. The signal demodulation link refers to the signal demodulation of the input ultrasonic data, wherein the input ultrasonic data can be a digital signal obtained after the analog-to-digital conversion link; the demodulation method can include: simple demodulation, orthogonal demodulation, Hilbert transform demodulation, secondary sampling demodulation, multiple sampling demodulation or baseband sampling demodulation, etc. The commonly used demodulation method is orthogonal demodulation. That is, the received echo signal is divided into two paths, and multiplied by cos(ωnTs ) and sin(ωnT s The amplification processing step includes: amplifying the ultrasonic data with different amplification factors according to the different reception times of the ultrasonic data to compensate for data signal attenuation; or providing different amplification factors according to the different locations of the ultrasonic data to compensate for data signal attenuation; the amplification processing step may be performed after the signal demodulation step.

[0157] Filtering typically improves signal quality after signal demodulation, using, for example, a low-pass filter. Downsampling lowers the sampling rate to reduce computational complexity. Data normalization, which can include scaling or standardization, limits data to a specific range, eliminating the negative effects of outliers.

[0158] Principal component analysis includes: centering the ultrasound data to obtain features, solving the covariance matrix of the features, solving the eigenvalues ​​of the covariance matrix, selecting the largest eigenvalue to form the eigenvector, and projecting the ultrasound data onto the eigenvector; principal component analysis mainly plays the role of reducing the feature dimension of the data.

[0159] Data augmentation involves shifting and / or adding noise to ultrasound data to improve the accuracy of neural network processing. For example, when training a neural network, operations such as shifting and adding noise to the limited training data can expand the dataset size, thereby enhancing the accuracy of the neural network.

[0160] The data rearrangement includes rearranging the ultrasonic data in at least one of the following ways: arranging the ultrasonic data received by each array element of the ultrasonic probe 10 into two columns after demodulation (one column is I data and the other column is Q data. Assuming that the data received by a certain array element is (Npoint*1), it is arranged into two columns as shown in FIG5(a) I1Q1 two columns of data) or arranging it into one column before demodulation; arranging the ultrasonic data (Npoint*2n) received by all effective array elements of the ultrasonic probe 10 after the ultrasonic probe 10 transmits the ultrasonic wave at the same time into a matrix (as shown in FIG5(b) I1Q1...I n Q n The ultrasonic data (Npoint*1) received by each array element of the ultrasonic probe 10 is divided into multiple (for example, m) and then arranged into a matrix (Npoint / m, 2m, as shown in FIG5(c)). 1-1 Q 1-1 , I 1-2 Q 1-2 Figure 5(c) shows an example where m is 2. It should be noted that if the data is before demodulation, it is no longer arranged into two columns, i.e. Figure 5(a) to Figure 5(c)The number of columns in is reduced by half. In addition, in other examples, Figure 5(a) to Figure 5(c) The rearranged data can be combined into three-dimensional or even higher-dimensional data inputs. The rearranged data is input into the neural network as input data, which can improve the accuracy of the neural network.

[0161] The beamforming step refers to the reconstruction and conversion of ultrasonic data (which can be the RF signal before demodulation or the baseband signal after demodulation) from the channel domain (for example, the data dimension is: time direction * number of channels * number of transmissions) into beam domain data (i.e., beamformed data, for example, the data dimension is: number of vertical points * number of horizontal points, which are points in the actual physical space); beamforming can adopt a variety of beamforming methods, including but not limited to the delay apodization sum (DAS) method, adaptive beamforming method, coherence factor beamforming method and / or incoherent beamforming method or frequency domain beamforming method, etc.

[0162] Modulo, logarithmic compression and grayscale conversion are the processing steps for ultrasound data in the image domain. These three items can also be collectively referred to as scan conversion, thereby obtaining an ultrasound image for display on the ultrasound display 50.

[0163] In some embodiments, please refer to Figure 6 The ultrasound host 30 includes a transmitting and receiving control circuit 31 and a processor 33, which will be described in detail below.

[0164] In some embodiments, the transmit and receive control circuit 31 is used to control the ultrasound probe 10 to transmit ultrasound waves and receive ultrasound echo signals. For example, the transmit and receive control circuit 20 is used to control the ultrasound probe 10 to transmit ultrasound waves toward a region of interest and to control the ultrasound probe 10 to receive ultrasound echoes reflected by tissue. In some specific embodiments, the transmit and receive control circuit 20 is used to generate a transmit sequence and a receive sequence and output them to the ultrasound probe 10. The transmit sequence is used to control some or all of the multiple array elements in the ultrasound probe 10 to transmit ultrasound waves toward the region of interest. The parameters of the transmit sequence include the number of array elements used for transmission and ultrasound transmission parameters (e.g., amplitude, frequency, number of transmissions, transmission interval, transmission angle, waveform, and / or focus position). The receive sequence is used to control some or all of the multiple array elements to receive ultrasound echoes after passing through tissue. The parameters of the receive sequence include the number of array elements used for reception and echo reception parameters (e.g., reception angle, depth, etc.). Depending on the application of the ultrasound echo or the image generated based on the ultrasound echo, the ultrasound parameters in the transmit sequence and the echo parameters in the receive sequence may vary. For example, different working modes, such as B mode, C mode, M mode and D mode (Doppler mode), may have different transmission sequence parameters. After the echo signal is received by the ultrasound probe 10 under the control of the transmission and reception control circuit 20 and processed by subsequent modules and corresponding algorithms, a B image reflecting the tissue anatomical structure, a C image reflecting the blood flow information, and a D image reflecting the Doppler spectrum image can be generated.

[0165] In some embodiments, the processor 33 can perform one or more steps of the above-mentioned processing of the ultrasonic echo signal, such as receiving and forming channel data, analog-to-digital conversion, signal demodulation, amplification, filtering, downsampling, beamforming, modulo, logarithmic compression and grayscale conversion.

[0166] In some embodiments, the processor 33 includes but is not limited to a central processing unit (CPU), a microcontroller unit (MCU), a field-programmable gate array (FPGA), and a digital signal processing (DSP), etc., which are used to interpret computer instructions and process data in computer software.

[0167] In some embodiments, the processor 33 is used to execute each computer application in the non-transitory computer-readable storage medium, thereby performing the corresponding steps and methods. For example, the processor 30 can be implemented by software, hardware, firmware, or a combination thereof, and can use at least one of a circuit, a single or multiple application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor, so that the processor 20 can execute some or all of the steps in the ultrasonic imaging method in various embodiments of the present application, or any combination of the steps therein.

[0168] The ultrasound display 50 is used to display information, such as content related to ultrasound imaging. Figure 7 This is an example of the interface A of the ultrasound display 50; in some embodiments, the interface A of the ultrasound display 50 may have at least a menu area 51 and an image area 52, the menu area 51 is used to display an interactive menu, and the image area 52 is used to display an ultrasound image.

[0169] The above is some description of the ultrasound imaging system 100 .

[0170] Figure 8 FIG2 is an example diagram of an endoscope system 200. In some embodiments, the endoscope system 200 includes an endoscope and a camera host 250 connected to the endoscope. The endoscope may include an insertion portion 230 and a handheld operation portion 260. The insertion portion 230 is used to be inserted into a patient to observe the part to be observed, and the handheld operation portion 260 is used for handheld operation by a user. In some embodiments, the insertion portion 230 and the handheld operation portion 260 may be an integral structure or a detachable structure. In some embodiments, the endoscope further includes at least one image sensor (not shown). For example, the image sensor may be provided at the front end of the insertion portion 230 of the endoscope. The camera host 250 is used to obtain image signals from the endoscope for signal processing.

[0171] In some embodiments, the endoscope system 200 further includes a light source 210; the light source 210 is used to provide illumination to the patient's area to be observed. For example, the light source 210 may include a visible light source and a special light source. Exemplarily, the visible light source is an LED light source, which can provide multiple monochromatic lights of different wavelength ranges, a combination of multiple monochromatic lights, or a wide-spectrum white light source. The special light source can be a laser light source corresponding to a fluorescent agent, such as near-infrared light. In some embodiments, before imaging using the endoscope system 200, a fluorescent agent is injected into the area to be observed, and the fluorescent agent can generate fluorescence after absorbing the laser light generated by the laser light source.

[0172] In some embodiments, the endoscope system 200 may include a light guide 220 and a cable 240 , etc.; the light source 210 is connected to the endoscope through the light guide 220 , the handheld operating unit 260 is connected to the camera host 250 through the cable 240 , and is connected to the light source 210 through the light guide 220 .

[0173] In some embodiments, the insertion portion 230 of the endoscope includes a tube, an image sensor, and an illumination optical path. The front end of the tube is intended for insertion into the human body and probing deeply into the area to be examined. The illumination optical path interfaces with the light guide 220 and is used to illuminate the light generated by the light source 210 onto the area to be examined / observed. The image sensor, which may include but is not limited to a CCD sensor or a CMOS sensor, is used to convert optical signals into electrical signals. The image signal captured by the image sensor undergoes preliminary signal processing at the handheld operating unit 260 and is then transmitted to the camera host 250 for subsequent image processing. This preliminary signal processing includes amplification and filtering, ultimately generating an endoscopic image of the area to be examined / observed. Furthermore, in some embodiments, the image sensor may include a first image sensor and a second image sensor, each configured to convert optical signals into electrical signals. These sensors may include but are not limited to a CCD sensor or a CMOS sensor. The image signal captured by the image sensor undergoes preliminary signal processing at the handheld operating unit 260 and is then transmitted to the camera host 250 for subsequent image processing. This preliminary signal processing includes amplification and filtering. The optical axes of the first and second image sensors may be parallel or at an angle. The first image signal and the second image signal collected by the first image sensor and the second image sensor can correspond to a stereo pair of images observed by the left and right eyes of a person, thereby simulating binocular stereo vision of the human eye.

[0174] The other end of the handheld operating unit 260 is connected to the camera host 250 via a cable 240, and transmits the image signal to the camera host 250 for processing via the cable 240. In some embodiments, the handheld operating unit 260 can also send image data to the camera host 250 by wireless transmission.

[0175] In some embodiments, a processor is provided in the camera host 250 , and the processor obtains the image signal output by the handheld operation unit 260 , processes the image signal, and outputs the processed image data.

[0176] In some embodiments, the endoscope system 200 further includes an endoscope display 270, and the camera host 250 can be connected to the endoscope display 270 via a wired connection such as a video cable. For example, the camera host 250 can send an endoscopic image to the endoscope display 270 for display.

[0177] In some embodiments, the handheld operating unit 260 includes a second operating component 261. The second operating component 261 may include one or more physical function keys.

[0178] In some embodiments, the user controls the endoscopic imaging through the second operating component 261 ; in some embodiments, the user controls the display of the endoscopic display 270 through the second operating component 261 .

[0179] It should be noted that Figure 8 The endoscope system 200 is merely an example and does not limit the endoscope system 200. The endoscope system 200 may include Figure 8 More or fewer components, or combinations of certain components, or different components may be shown. For example, the endoscope system 200 may also include a dilator, a smoke control device, input and output devices, a network access device, etc.

[0180] The above is some description of the endoscope system 200.

[0181] As described above, different medical devices display information, content, and images on their respective displays. For example, the ultrasound imaging system 100 displays ultrasound images through the interface of its ultrasound display 50, and the endoscope system 200 displays endoscopic images through the interface of its endoscope display 270. When a user needs to view endoscopic images and ultrasound images, or even needs to compare the two images, the user needs to frequently switch his or her line of sight between the interfaces of different displays, which affects the smooth progress of, for example, surgery and is very inconvenient.

[0182] To this end, some embodiments provide a display method, which can be used in an endoscope system 200 such as disclosed in some embodiments of this document; in some specific embodiments, the display method can be implemented by the camera host 250 of the endoscope system 200 by controlling other components. The display method for the endoscope system 200 is described below.

[0183] Please refer to Figure 9 In some embodiments, a display method for the endoscope system 200 includes the following steps:

[0184] Step 1000: Acquire an endoscopic image signal of a part to be observed.

[0185] Step 1100: Generate an endoscopic image based on the endoscopic image signal.

[0186] Step 1200 : Acquire first content related to ultrasound imaging from the ultrasound imaging system 100 that is communicatively connected to the endoscope system 200 .

[0187] In some embodiments, the first content related to ultrasound imaging is the content (eg, ultrasound image) in the image area 52 of the interface of the ultrasound display 50 .

[0188] In some embodiments, the first content related to ultrasound imaging is the content of the menu area 51 of the interface of the ultrasound display 50 (eg, an interactive menu).

[0189] In some embodiments, the first content related to ultrasound imaging includes the content of the image area 52 (eg, ultrasound image) and the content of the menu area 51 (eg, interactive menu) of the interface of the ultrasound display 50 .

[0190] In some embodiments, the first content related to ultrasound imaging includes one or more contents selected from the menu area 51 and the image area 52 based on a user selection instruction. For example, the first content related to ultrasound imaging includes the content of the menu area 51 (for example, an interactive menu) selected from the menu area 51 and the image area 52 based on a user selection instruction. For another example, the first content related to ultrasound imaging includes the content of the image area 52 (for example, an ultrasound image) selected from the menu area 51 and the image area 52 based on a user selection instruction. For another example, the first content related to ultrasound imaging includes the content of both the menu area 51 and the image area 52 (for example, an ultrasound image and an interactive menu) selected from the menu area 51 and the image area 52 based on a user selection instruction.

[0191] Step 1300 : Displaying the endoscopic image and the first content on the interface of the endoscope display 270 of the endoscope system 200 .

[0192] In some embodiments, the interface of the endoscope display 270 includes a first area 271 and a second area 272, wherein the endoscopic image is displayed in the first area 271 and the first content is displayed in the second area 272. That is, step 1300 displays the endoscopic image in the first area 271 of the interface of the endoscope display 270 and displays the first content in the second area 272. Figure 10 FIG. 2 is an example of an interface B of an endoscope display 270 .

[0193] In some embodiments, the area of ​​the first region 271 is larger than the area of ​​the second region 272 .

[0194] In some embodiments, the area of ​​the first region 271 is smaller than the area of ​​the second region 272 .

[0195] In some embodiments, the area of ​​the first region 271 is equal to the area of ​​the second region 272 .

[0196] In some embodiments, step 1300 performs positional registration on the endoscopic image and the ultrasound image, fuses the registered endoscopic image and the ultrasound image to generate a fused image, and displays the fused image on the interface of the endoscope display 270 .

[0197] In some embodiments, the user can perform control via the first operating component 13 of the ultrasound probe 10 .

[0198] In some embodiments, the first operating component 13 of the ultrasound probe 10 can be operated by the user to generate a display parameter instruction, which is used to set the display parameters of the endoscope display 270. In some embodiments, the display parameters include one or more of the following:

[0199] a ratio of the first region 271 to the second region 272;

[0200] the size and / or location of the first region 271;

[0201] the size and / or location of the second region 272;

[0202] The resolution, refresh rate, or brightness of the interface of the endoscope display 270 .

[0203] Therefore, the user can adjust the display parameters of the endoscope display 270 by operating the first operating component 13 of the ultrasound probe 10, such as adjusting the ratio of the first area 271 and the second area 272, adjusting the size and / or position of the first area 271, adjusting the size and / or position of the second area 272, and so on.

[0204] In some embodiments, the first operating component 13 of the ultrasound probe 10 can be operated by the user to generate imaging parameter instructions, which are used to set the imaging parameters of the ultrasound imaging system; the imaging parameters of the ultrasound imaging system can be found in the above description and will not be repeated here.

[0205] Therefore, the user can switch the image imaging mode and adjust and set the imaging sub-parameters under the specific imaging mode by operating the first operating component 13 of the ultrasound probe 10. In this way, when the user switches from one imaging mode to another by operating the ultrasound probe 10, the ultrasound image displayed on the endoscope display 270 will also be switched. When the user adjusts and sets the imaging sub-parameters by operating the ultrasound probe 10, the ultrasound image displayed on the endoscope display 270 will be updated accordingly.

[0206] In some embodiments, the first operating component 13 of the ultrasound probe 10 can be operated by a user to generate ultrasound operation instructions, which are used to perform a freeze, save, or unfreeze function on the ultrasound imaging system 100, that is, to freeze, save, or unfreeze the ultrasound image.

[0207] In some embodiments, when the first operating component 13 of the ultrasound probe 10 is operated by the user, the first content also includes an interactive menu generated / produced when the first operating component 13 is operated. In some embodiments, the interactive menu is used for the user to set the display parameters of the endoscope display 270 through the first operating component 13. The display parameters of the endoscope display 270 can be found in the description above and will not be repeated here. In some embodiments, the interactive menu is used for the user to set the imaging parameters of the ultrasound imaging system through the first operating component 13. The imaging parameters can be found in the description above and will not be repeated here. In some embodiments, the interactive menu is used for the user to perform a freeze, save, or unfreeze function on the ultrasound imaging system 100 through the first operating component 13, that is, to freeze, save, or unfreeze the ultrasound image.

[0208] As described above, in some embodiments, the handheld operating unit 260 includes a second operating component 261 .

[0209] In some embodiments, the second operating component 261 of the handheld operating unit 260 can be operated by the user to generate a display parameter instruction, which is used to set the display parameters of the endoscope display 270. In some embodiments, the display parameters include one or more of the following:

[0210] a ratio of the first region 271 to the second region 272;

[0211] the size and / or location of the first region 271;

[0212] the size and / or location of the second region 272;

[0213] The resolution, refresh rate, or brightness of the interface of the endoscope display 270 .

[0214] Therefore, the user can adjust the display parameters of the endoscope display 270 by operating the second operating component 261 of the handheld operating unit 260, such as adjusting the ratio of the first area 271 and the second area 272, adjusting the size and / or position of the first area 271, adjusting the size and / or position of the second area 272, and so on.

[0215] In some examples, the second operating component 261 of the handheld operating unit 260 can be operated by the user to generate imaging parameter instructions, which are used to set the imaging parameters of the ultrasound imaging system; the imaging parameters of the ultrasound imaging system can be found in the above description and will not be repeated here.

[0216] Therefore, the user can switch the imaging mode of the ultrasound imaging system 100 by operating the second operating component 261 of the handheld operating unit 260, and adjust and set the imaging sub-parameters under the specific imaging mode. In this way, when the user switches from one imaging mode to another by operating the handheld operating unit 260, the ultrasound image displayed on the endoscope display 270 will also be switched. When the user adjusts and sets the imaging sub-parameters by operating the handheld operating unit 260, the ultrasound image displayed on the endoscope display 270 will be updated accordingly.

[0217] In some examples, the second operating component 261 of the handheld operating unit 260 can be operated by the user to generate ultrasound operation instructions, which are used to perform a freeze, save, or unfreeze function on the ultrasound imaging system 100, that is, to freeze, save, or unfreeze the ultrasound image.

[0218] In some embodiments, when the second operating component 261 of the handheld operating unit 260 is operated by the user, the endoscope display 270 also displays an interactive menu generated / produced when the second operating component 261 is operated. In some embodiments, the interactive menu of the second operating component 261 is used to allow the user to set the display parameters of the endoscope display 270 through the second operating component 261. For details on the interactive menu display parameters of the endoscope display 270, please refer to the description above and will not be repeated here. In some embodiments, the interactive menu of the second operating component 261 is used to allow the user to set the imaging parameters of the ultrasound imaging system through the second operating component 261. For details on the imaging parameters, please refer to the description above and will not be repeated here. In some embodiments, the interactive menu of the second operating component 261 is used to allow the user to perform a freeze, save, or unfreeze function on the ultrasound imaging system 100 through the second operating component 261, that is, to freeze, save, or unfreeze the ultrasound image.

[0219] In some examples, the user can turn on or off the display of the first content (eg, ultrasound image) on the interface of the endoscope display 270 by operating the first operating component 13 of the ultrasound probe 10 and / or operating the handheld operating unit 260 .

[0220] In some embodiments, the first operating component 13 of the ultrasound probe 10 can be operated by a user to generate a close instruction, where the close instruction is used to close the display of the first content on the interface of the endoscope display 270 .

[0221] In some embodiments, the first operating component 13 of the ultrasound probe 10 can be operated by a user to generate an opening instruction, where the opening instruction is used to start displaying the first content on the interface of the endoscope display 270 .

[0222] In some embodiments, the second operating component 261 of the handheld operating unit 260 can be operated by the user to generate a close instruction, where the close instruction is used to close the display of the first content on the interface of the endoscope display 270 .

[0223] In some embodiments, the second operating component 261 of the handheld operating unit 260 can be operated by the user to generate an opening instruction, and the opening instruction is used to start displaying the first content on the interface of the endoscope display 270.

[0224] By displaying the endoscopic image and the ultrasound image on the endoscope display 270, the user can view the endoscopic image and the ultrasound image on the same screen on a display interface; at the same time, through the first operating component 13 of the ultrasound probe 10 or the second operating component 261 of the handheld operating part 260, parameters such as display parameters and ultrasound imaging parameters can be adjusted and set, which makes the user's operation process more convenient.

[0225] It can be understood that when adjusting factors such as the ratio of the first area 271 and the second area 272, the size of the first area 271 and the size of the second area 272, etc., the content displayed in the corresponding area will also change size. For example, if the first area 271 becomes larger, the endoscopic image will also become larger; if the first area 271 becomes smaller, the endoscopic image will also become smaller; if the second area 272 becomes larger, the first content (such as an ultrasound image) will also become larger; if the second area 272 becomes smaller, the first content (such as an ultrasound image) will also become smaller.

[0226] It can be understood that the interactive menu displayed on the interface of the endoscope display 270 can automatically disappear or become a menu icon after the operation is completed, and can also automatically disappear or become a menu icon if it is not operated for a period of time; similarly, the interactive menu displayed on the interface of the ultrasound display 50 can automatically disappear or become a menu icon after the operation is completed, and can also automatically disappear or become a menu icon if it is not operated for a period of time.

[0227] In order to display the endoscopic image and the ultrasound image on the endoscope display 270 , the ultrasound imaging system 100 and the endoscope system 200 need to establish a communication connection.

[0228] In some embodiments, a wired communication connection is established between the ultrasound imaging system 100 and the endoscope system 200 , for example, a wired communication connection is established between the ultrasound host 30 and the camera host 250 .

[0229] In the operating room, there are many kinds of medical equipment, such as not only the ultrasound imaging system 100 and the endoscope system 200, but also equipment such as a pendant, a shadowless lamp, a ventilator and monitoring equipment. Therefore, the space in the operating room is limited, and the wired connection must take into account the positions of the ultrasound imaging system 100 and the endoscope system 200, which limits the placement of the equipment. In addition, there are many people during the operation (such as the surgeon, assistants, nurses, etc.), the space is limited, and the personnel also have to operate various equipment, which increases the risk of wired connection. Moreover, the wired connection needs to be set up in advance, which is relatively cumbersome and complicated, especially for medical staff without IT technology background.

[0230] In some embodiments, the ultrasound imaging system 100 and the endoscope system 200 establish a wireless communication connection, for example, the ultrasound host 30 establishes a wireless communication connection with the camera host 250 .

[0231] Wireless communication can reduce the number of wires between devices, making the placement of machines more flexible.

[0232] In some embodiments, the wireless communication connection includes a Wi-Fi connection, a WHDI connection, a WiHD connection, or the like.

[0233] In some embodiments, please refer to Figure 11 The ultrasound imaging system 100 communicates with the endoscope system 200 via the first communication component 01, for example, transmitting data to the endoscope system 200 or acquiring data from the endoscope system 200. In some embodiments, the ultrasound imaging system 100 may include the first communication component 01, for example, the first communication component 01 is integrated in the ultrasound host 30, and for another example, the ultrasound host 30 has a first communication interface 01a, and the first communication interface 01a is used to plug in the first communication component 01, that is, the first communication component 01 is pluggable and disposed on the first communication interface 01a of the ultrasound host 30.

[0234] In some embodiments, please refer to Figure 12The endoscope system 200 communicates with the ultrasound imaging system 100 via the second communication component 02, for example, transmitting data to the ultrasound imaging system 100 or acquiring data from the ultrasound imaging system 100. In some embodiments, the endoscope system 200 may include the second communication component 02, for example, the second communication component 02 may be integrated into the camera host 250, or the camera host 250 or the endoscope display 270 may have a second communication interface 02a, and the second communication interface 02a is used to plug in the second communication component 02, that is, the second communication component 02 may be pluggable and disposed on the second communication interface 02a.

[0235] In some embodiments, the first communication component 01 and the second communication component 02 may also constitute a separate product, a communication component, which is independent of the ultrasound imaging system 100 and the endoscope system 200. The communication component is used to establish a communication connection between the endoscope system 200 and the ultrasound imaging system 100. For example, the endoscope system 200 and the ultrasound imaging system 100 perform data communication through the communication component. Furthermore, the ultrasound host 30 has a first communication interface 01a, which is used to plug in the first communication component 01. The camera host 250 or the endoscope display 270 has a second communication interface 02a, which is used to plug in the second communication component 02.

[0236] In some embodiments, the first communication component 01 is a wifi communication component, a WHDI communication component or a WiHD communication component.

[0237] In some embodiments, the second communication component 02 is a wifi communication component, a WHDI communication component or a WiHD communication component.

[0238] Please refer to Figure 13 In some embodiments, an imaging system 300 for an operating room is also disclosed. The imaging system 300 may include an ultrasound imaging system 100, an endoscope system 200, and a communication component 310. The communication component 310 is used to establish a communication connection between the endoscope system 200 and the ultrasound imaging system 100.

[0239] In some embodiments, the communication component 310 includes a first communication component 01 and a second communication component 02; the ultrasound host 30 has a first communication interface 01a, which is used to plug in the first communication component 01; the camera host 250 or the endoscope display 270 has a second communication interface 02a, which is used to plug in the second communication component 02.

[0240] This document is described with reference to various exemplary embodiments. However, those skilled in the art will recognize that changes and modifications may be made to the exemplary embodiments without departing from the scope of this document. For example, the various operational steps and components used to perform the operational steps may be implemented in different ways (e.g., one or more steps may be deleted, modified, or incorporated into other steps) depending on the specific application or considering any number of cost functions associated with the operation of the system.

[0241] In the above embodiments, all or part of the embodiments may be implemented through software, hardware, firmware, or any combination thereof. Furthermore, as will be appreciated by those skilled in the art, the principles herein may be embodied in a computer program product on a computer-readable storage medium pre-installed with computer-readable program code. Any tangible, non-transitory computer-readable storage medium may be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD-ROMs, DVDs, Blu-ray discs, etc.), flash memory, and / or the like. These computer program instructions may be loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing device to form a machine, such that the instructions executed on the computer or other programmable data processing device can generate a device that implements a specified function. These computer program instructions may also be stored in a computer-readable memory, which can instruct the computer or other programmable data processing device to operate in a specific manner, such that the instructions stored in the computer-readable memory can form an article of manufacture, including an implementation device that implements a specified function. The computer program instructions may also be loaded onto a computer or other programmable data processing device, thereby causing the computer or other programmable device to execute a series of operational steps to generate a computer-implemented process, such that the instructions executed on the computer or other programmable device can provide the steps for implementing the specified function.

[0242] Although the principles of this invention have been shown in various embodiments, many modifications of structure, arrangement, proportion, elements, materials and components that are particularly suitable for specific environments and operational requirements can be used without departing from the principles and scope of this invention. The above modifications and other changes or amendments are intended to be included within the scope of this invention.

[0243] The foregoing detailed description has been described with reference to various embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of this disclosure. Therefore, the present disclosure will be considered in an illustrative rather than a restrictive sense, and all such modifications will be included within its scope. Similarly, the advantages, other advantages and solutions to the problems of the various embodiments have been described above. However, the benefits, advantages, solutions to the problems and any elements that can produce these, or make them more specific, should not be interpreted as critical, required or necessary. The term "comprising" and any other variants used in this article are all non-exclusive inclusions, so that a process, method, article or device that includes a list of elements includes not only these elements, but also other elements that are not explicitly listed or do not belong to the process, method, system, article or device. In addition, the term "coupled" and any other variants used in this article refer to physical connections, electrical connections, magnetic connections, optical connections, communication connections, functional connections and / or any other connections.

[0244] Those skilled in the art will recognize that many changes can be made to the details of the above embodiments without departing from the basic principles of the invention. Therefore, the scope of the present invention should be determined solely by the claims.

Claims

1. A display method for an endoscope system, characterized in that: include: Acquiring endoscopic image signals of the part to be observed; generating an endoscopic image based on the endoscopic image signal; acquiring first content related to ultrasound imaging from an ultrasound imaging system communicatively connected to the endoscope system, wherein an interface of an ultrasound display of the ultrasound imaging system has at least a menu area for displaying an interactive menu and an image area for displaying an ultrasound image, and the first content related to ultrasound imaging includes one or more contents selected from the menu area and the image area based on a user selection instruction; The endoscopic image and the first content are displayed on an interface of an endoscope display of the endoscope system.

2. A display method for an endoscope system, characterized in that: include: Acquiring endoscopic image signals of the part to be observed; generating an endoscopic image based on the endoscopic image signal; When the first operating component of the ultrasound probe of the ultrasound imaging system is operated by the user to generate an opening instruction, the endoscopic image and the first content related to the ultrasound imaging are displayed on the interface of the endoscope display of the endoscope system; wherein, the first content related to the ultrasound imaging is obtained from the ultrasound imaging system that is communicatively connected to the endoscope system.

3. The display method according to claim 2, wherein: When the first operating component of the ultrasound probe is operated by the user to generate a closing instruction, the first content displayed on the interface of the endoscope display is closed.

4. The display method according to claim 2, wherein: The interface of the ultrasound display of the ultrasound imaging system has at least a menu area for displaying an interactive menu and an image area for displaying an ultrasound image, and the first content related to ultrasound imaging includes one or more contents selected from the menu area and the image area based on a user selection instruction.

5. The display method according to claim 1 or 2, wherein: The interface of the endoscope display includes a first area and a second area, the endoscope image is displayed in the first area, and the first content is displayed in the second area.

6. The display method according to claim 1 or 2, wherein: The displaying of the endoscopic image and the first content on the interface of the endoscope display of the endoscope system includes: performing position registration on the endoscopic image and the ultrasound image, and performing image fusion on the endoscopic image and the ultrasound image after position registration to generate and display a fused image.

7. The display method according to any one of claims 1 to 6, characterized in that: The first operating component of the ultrasound probe of the ultrasound imaging system is operable by a user to generate a display parameter instruction, wherein the display parameter instruction is used to set a display parameter of the endoscope display; And / or, the first operating component of the ultrasound probe of the ultrasound imaging system can be operated by a user to generate an imaging parameter instruction, wherein the imaging parameter instruction is used to set an imaging parameter of the ultrasound imaging system; and / or, the first operating component of the ultrasound probe of the ultrasound imaging system can be operated by a user to generate an ultrasound operation instruction, wherein the ultrasound operation instruction is used to execute a freeze, save, or unfreeze function on the ultrasound imaging system; and / or, The handheld operating unit of the endoscope system has a second operating component; the second operating component can be operated by a user to generate a display parameter instruction, and the display parameter instruction is used to set the display parameters of the endoscope display; And / or, the second operating component can be operated by a user to generate an imaging parameter instruction, wherein the imaging parameter instruction is used to set imaging parameters of the ultrasound imaging system; And / or, the second operating component can be operated by a user to generate an ultrasound operating instruction, where the ultrasound operating instruction is used to execute a freeze or unfreeze function on the ultrasound imaging system.

8. The display method according to any one of claims 1 to 6, characterized in that: When a first operating component of the ultrasound probe of the ultrasound imaging system is operated by a user, the first content further includes an interactive menu generated when the first operating component is operated; and / or the handheld operating portion of the endoscope system has a second operating component, and when the second operating component is operated by the user, the endoscope display 270 displays the interactive menu generated when the second operating component is operated; The interactive menu is used for the user to set the display parameters of the endoscope display through the first operating component; and / or, the interactive menu is used for the user to set the imaging parameters of the ultrasound imaging system through the first operating component; and / or, the interactive menu is used for the user to perform a freeze, save or unfreeze function on the ultrasound imaging system through the first operating component.

9. The display method according to claim 7 or 8, wherein: The display parameters include one or more of the following: a ratio of the first area to the second area; the size and / or location of the first area; the size and / or location of the second area; The resolution, refresh rate or brightness of the interface of the endoscope display.

10. The display method according to claim 7 or 8, wherein: The imaging parameters include one or more of the following: one or more imaging modalities; Imaging sub-parameters corresponding to each imaging mode; Navigation function parameters.

11. The display method according to claim 10, wherein: The imaging mode includes one or more of a B mode, a C mode, a PW mode, a CW mode, a Power mode, a contrast imaging mode, and an elastic imaging mode; The imaging sub-parameters corresponding to the B-mode include: one or more of gain, frequency, focus, depth and dynamic range; The imaging sub-parameters corresponding to the C-mode include one or more of gain, frequency, pulse repetition frequency, sampling frame size, sampling line angle, inversion color, wall filtering, smoothing, persistence, and color map; The imaging sub-parameters corresponding to the PW mode include one or more of gain, frequency, pulse repetition frequency, sample volume, sampling line angle, wall filter and baseline position; The imaging sub-parameters corresponding to the CW mode include one or more of gain, frequency, sampling line angle, and wall filtering; The imaging sub-parameters corresponding to the Power mode include one or more of gain, frequency, pulse repetition frequency, sampling frame size, sampling line angle, wall filtering, smoothing, persistence, dynamic range and energy spectrum; The imaging sub-parameters corresponding to the contrast imaging mode include one or more of focus, frequency, acoustic output power, and MI mechanical index; The imaging sub-parameters corresponding to the elastic imaging mode include one or more of shear wave elasticity, strain elasticity, transient elasticity and viscoelasticity; The navigation function parameters include one or more of tumor parameters, guide line parameters, and guide region parameters.

12. The display method according to claim 1, wherein: The first operating component of the ultrasound probe of the ultrasound imaging system can be operated by a user to generate a close instruction, wherein the close instruction is used to close the display of the first content on the interface of the endoscope display; and / or the first operating component of the ultrasound probe of the ultrasound imaging system can be operated by a user to generate a start instruction, wherein the start instruction is used to start the display of the first content on the interface of the endoscope display; And / or, a second operating component of the handheld operating unit of the endoscope system can be operated by a user to generate a closing instruction, wherein the closing instruction is used to close the display of the first content on the interface of the endoscope display; And / or, the second operating component of the handheld operating part of the endoscope system can be operated by the user to generate an opening instruction, and the opening instruction is used to start displaying the first content on the interface of the endoscope display.

13. The display method according to any one of claims 1 to 12, characterized in that: The endoscope system and the ultrasound imaging system are communicated via a wired or wireless method; the wireless method includes a Wi-Fi connection, a WHDI connection, or a WiHD connection.

14. An endoscope system, characterized in that: The endoscope comprises an endoscope, a light source, a camera host connected to the endoscope, and an endoscope display for displaying; The light source is used to provide illumination to the patient's part to be observed; The endoscope includes an insertion portion and a handheld operation portion, wherein the insertion portion is used to be inserted into a patient to observe the part to be observed; the handheld operation portion is used for handheld operation by a user; the endoscope is provided with one or more image sensors, and the image sensors are used to collect endoscopic image signals of the part to be observed; The camera host is configured to execute the method according to any one of claims 1 to 13.

15. An ultrasonic imaging system, characterized in that: include: An ultrasonic probe comprises a housing, an array element group, and a first operating component disposed on the housing; the array element group is used to transmit ultrasonic waves and receive echo signals of the ultrasonic waves; the first operating component is used for user operation; an ultrasound host, configured to generate an ultrasound image according to the echo signal; an ultrasound display, configured to display the ultrasound image; in: The first operating component of the ultrasound probe is used for a user to operate to generate an interactive menu, and the interactive menu is used for the user to control ultrasound imaging through the first operating component; the ultrasound display displays the interactive menu.

16. The ultrasound imaging system according to claim 15, wherein: The interactive menu is used for the user to set imaging parameters of the ultrasound image through the first operating component; and / or, the interactive menu is used for the user to freeze, save or unfreeze the ultrasound image through the first operating component.

17. The ultrasound imaging system according to claim 16, wherein: The imaging parameters include one or more of the following: one or more imaging modalities; Imaging sub-parameters corresponding to each imaging mode; Navigation function parameters.

18. The ultrasound imaging system according to claim 17, wherein: The imaging mode includes one or more of a B mode, a C mode, a PW mode, a CW mode, a Power mode, a contrast imaging mode, and an elastic imaging mode; The imaging sub-parameters corresponding to the B-mode include: one or more of gain, frequency, focus, depth and dynamic range; The imaging sub-parameters corresponding to the C-mode include one or more of gain, frequency, pulse repetition frequency, sampling frame size, sampling line angle, inversion color, wall filtering, smoothing, persistence, and color map; The imaging sub-parameters corresponding to the PW mode include one or more of gain, frequency, pulse repetition frequency, sample volume, sampling line angle, wall filter and baseline position; The imaging sub-parameters corresponding to the CW mode include one or more of gain, frequency, sampling line angle, and wall filtering; The imaging sub-parameters corresponding to the Power mode include one or more of gain, frequency, pulse repetition frequency, sampling frame size, sampling line angle, wall filtering, smoothing, persistence, dynamic range and energy spectrum; The imaging sub-parameters corresponding to the contrast imaging mode include one or more of focus, frequency, acoustic output power, and MI mechanical index; The imaging sub-parameters corresponding to the elastic imaging mode include one or more of shear wave elasticity, strain elasticity, transient elasticity and viscoelasticity; The navigation function parameters include one or more of tumor parameters, guide line parameters, and guide region parameters.

19. The ultrasound imaging system according to claim 17 or 18, wherein: The interactive menu includes a first-level menu and a second-level menu, wherein the first-level menu has first-level options for users to select imaging modes and / or navigation functions; after any first-level option of the interactive menu is selected, the interactive menu will expand the second-level menu associated with the selected first-level option; wherein the second-level menu associated with the first-level option of the imaging mode is a setting option for the imaging sub-parameters corresponding to the imaging mode, and the second-level menu associated with the first-level option of the navigation function is a setting option for the navigation function parameters.

20. The ultrasound imaging system according to claim 19, wherein: The first operating component includes a menu button and direction keys arranged on both sides of the menu button. When the menu button is triggered, it is used to generate or close the interactive menu. The direction keys are used to select the first-level options of the first-level menu and the setting options of the second-level menu.

21. The ultrasound imaging system according to claim 15, wherein: It also includes a first communication component, which is used to communicate with the endoscope system; the ultrasound host sends first content related to ultrasound imaging to the endoscope system through the first communication component, so that the endoscope system displays the endoscopic image and the first content on the interface of its endoscope display; wherein the first content includes the ultrasound image and / or the interactive menu.

22. The ultrasound imaging system according to claim 21, wherein: The interface of the ultrasound display of the ultrasound imaging system has at least a menu area for displaying an interactive menu and an image area for displaying an ultrasound image, and the first content related to ultrasound imaging includes one or more contents selected from the menu area and the image area based on a user selection instruction.

23. The ultrasound imaging system according to claim 21, wherein: The ultrasound host sends the first content related to ultrasound imaging to the endoscope system through the first communication component, so that the endoscope system displays the endoscopic image and the first content on the interface of its endoscope display, including: the ultrasound host sends the first content related to ultrasound imaging to the endoscope system through the first communication component, so that the endoscope system displays the endoscopic image in the first area of ​​the interface of its endoscope display, and displays the ultrasound image and / or the interactive menu in the second area.

24. The ultrasound imaging system according to claim 23, wherein: The interactive menu is used for the user to set display parameters of the endoscope display through the first operating component; the display parameters include one or more of the following: a ratio of the first area to the second area; the size and / or location of the first area; the size and / or location of the second area; The resolution, refresh rate or brightness of the interface of the endoscope display.

Images