Image display method of endoscope system, endoscope system

Abstract

An image display method of an endoscope system and an endoscope system, by acquiring endoscope image data and ultrasonic image data, determining an endoscope display partition and an ultrasonic image display partition from a display area of a display according to a same screen display mode, obtaining a first target image corresponding to an endoscope image according to the endoscope image data, obtaining a second target image according to an ultrasonic effective area determined according to the ultrasonic image, so as to display the first target image in the endoscope display partition, and display the second target image in the external device image display partition. In the embodiment of the application, not only can the endoscope image and the external video source image be displayed on the same screen of the display, but also the effective area in the ultrasonic image can be effectively identified, so that the ultrasonic image displayed on the same screen can be displayed for the effective area, which is convenient for the doctor to observe the details of the effective area in the ultrasonic image displayed on the same screen, so that the display effect is better, and the doctor can observe in the clinical application.

Description

Technical Field

[0001] This application relates to the field of endoscopy technology, and in particular to an image display method for an endoscopy system and an endoscopy system. Background Technology

[0002] In recent years, endoscopic imaging systems have been widely used in medical diagnosis and other procedures. An endoscopic imaging system typically includes a camera, an endoscope, an endoscopic camera unit, and a monitor. The camera transmits the captured image data to the endoscopic camera unit via a cable. The endoscopic camera unit processes the video information acquired by the camera and displays the processed video information on the monitor.

[0003] In clinical surgical applications, doctors may need to observe not only endoscopic images but also image data from other imaging devices (such as ultrasound images and 3D reconstruction data). Currently, the common practice is to use two monitors: one connected to the endoscope unit to display the endoscopic images, and the other displaying image data from other imaging devices. This approach involves numerous connected devices, requiring significant time for wiring during surgical setup, and is also costly. Furthermore, using two monitors is inconvenient for observation. While monitors capable of connecting two video sources and simultaneously displaying endoscopic and other images on the left and right sides of the screen are available, this method still suffers from poor display quality and is inconvenient for doctors to observe. Summary of the Invention

[0004] The following is an overview of the subject matter described in detail in this application. This overview is not intended to limit the scope of the claims.

[0005] This application provides an image display method and an endoscope system that can determine the image in the effective area of ​​the ultrasound image and display it on the same screen as the endoscope image in a screen-on-screen display mode, which has a better display effect so that doctors can observe the ultrasound image and the endoscope image.

[0006] On one hand, embodiments of this application provide an image display method for an endoscope system, the endoscope system including an endoscope host and a display, characterized in that the image display method includes:

[0007] Acquire endoscopic image data, which includes multiple frames of endoscopic images;

[0008] At least one ultrasound image data is acquired from at least one ultrasound device that is communicatively connected to the endoscope system, the ultrasound image data comprising multiple frames of ultrasound images;

[0009] According to the screen-sharing mode, an endoscope display partition for displaying the endoscope image on the same screen and at least one ultrasound image display partition for displaying the ultrasound image on the same screen are determined from the display area of ​​the display, wherein the endoscope display partition and the ultrasound image display partition are displayed on the same screen in the display area;

[0010] Multiple frames of the endoscope image are obtained from the endoscope image data, and multiple frames of the ultrasound image are obtained from the ultrasound image data;

[0011] Based on the endoscopic image data, a first target image corresponding to the endoscopic image is obtained;

[0012] Edge detection is performed on the ultrasound image to identify fan-shaped regions in the ultrasound image, resulting in a fan-shaped region image; edge detection is performed on the fan-shaped region image to identify target tissue in the fan-shaped region image; the effective ultrasound region is determined based on the target tissue, resulting in a second target image of the effective ultrasound region;

[0013] The first target image is displayed in the endoscope display section of the display area, and the second target image is displayed in the ultrasound image display section.

[0014] In one embodiment of this application, obtaining a first target image corresponding to the endoscope image based on the endoscope image data includes at least one of the following:

[0015] Image recognition is performed on the endoscopic image to determine the position of the surgical instruments in the endoscopic image, and the effective area of ​​the endoscope is determined based on the position of the surgical instruments to obtain a first target image of the effective area of ​​the endoscope;

[0016] Image recognition is performed on the endoscopic image to determine the surgical field region of the endoscopic image, and the effective region of the endoscope is determined based on the surgical field region to obtain a first target image of the effective region of the endoscope;

[0017] Obtain the endoscope effective area selection information input by the user, determine the endoscope effective area based on the endoscope effective area selection information, and obtain the first target image of the endoscope effective area.

[0018] In one embodiment of this application, determining the effective ultrasound region based on the target tissue and obtaining a second target image of the effective ultrasound region includes:

[0019] Based on the target tissue, the effective area of ​​ultrasound is determined, and a local ultrasound image of the effective area of ​​ultrasound is obtained;

[0020] Text information from the ultrasound image is obtained, and a second target image of the effective ultrasound region is obtained based on the text information and the local ultrasound image.

[0021] In one embodiment of this application, before displaying the first target image and the second target image, display adaptive adjustment is performed on the first target image and the second target image, the display adaptive adjustment including:

[0022] Obtain a first display parameter of the endoscope display partition and a second display parameter of the ultrasound image display partition, wherein the first display parameter includes a first target resolution of the endoscope display partition and the second display parameter includes a second target resolution of the ultrasound image display partition;

[0023] The first target image is scaled and / or cropped according to the first target resolution, wherein the resolution of the first target image matches the first target resolution.

[0024] The second target image is scaled and / or cropped according to the second target resolution, wherein the resolution of the second target image matches the second target resolution.

[0025] In one embodiment of this application, the scaling process specifically includes:

[0026] Acquire the image to be processed and determine the original resolution of the image to be processed, wherein the image to be processed is the first target image, the second target image, the cropped partial first target image, or the cropped partial second target image;

[0027] A target scaling factor is determined based on the target resolution and the original resolution, wherein the target scaling factor is the ratio of the target resolution to the original resolution, and the target resolution is either the first target resolution or the second target resolution;

[0028] The image to be processed is scaled according to the target scaling factor to obtain a scaled target image, wherein the target image is either the first target image or the second target image.

[0029] In one embodiment of this application, the trimming process specifically includes at least one of the following:

[0030] A target resolution and an image to be processed are obtained, wherein the target resolution is either the first target resolution or the second target resolution, and the image to be processed is either the first target image or the second target image; a cropping region is determined in the image to be processed according to the aspect ratio of the target resolution, and a target image is obtained by cropping the image to be processed according to the cropping region, wherein the target image is either the first target image or the second target image;

[0031] Alternatively, a target resolution and an image to be processed are obtained, wherein the target resolution is either the first target resolution or the second target resolution, and the image to be processed is either the first target image or the second target image; a cropping region is determined in the image to be processed based on the aspect ratio of the target resolution, wherein the aspect ratio of the cropping region is the same as the aspect ratio of the target resolution and the cropping region covers the effective region of the image to be processed; a target image is obtained by cropping the image to be processed based on the cropping region, wherein the target image is either the first target image or the second target image, and the effective region is either the effective region of an endoscope or the effective region of an ultrasound.

[0032] In one embodiment of this application, the scaling and / or cropping of the first target image according to the first target resolution includes:

[0033] A first cropping region in the first target image is determined based on the aspect ratio of the first target resolution and / or the effective area of ​​the endoscope, wherein the first cropping region covers the effective area of ​​the endoscope;

[0034] Based on the first cropping region, the first target image is cropped from the first target image to obtain the cropped first target image;

[0035] Obtain the first resolution of the first target image, and determine the first scaling factor based on the ratio of the first target resolution to the first resolution;

[0036] The first target image is scaled according to the first scaling factor;

[0037] The scaling and / or cropping of the second target image according to the second target resolution includes:

[0038] A second cropping region in the second target image is determined based on the aspect ratio of the second target resolution and / or the effective ultrasound region, wherein the second cropping region covers the effective ultrasound region;

[0039] The second target image is cropped from the second target image according to the second cropping region to obtain the cropped second target image;

[0040] Obtain the second resolution of the second target image, and determine the second scaling factor based on the ratio of the second target resolution to the second resolution;

[0041] The second target image is scaled according to the second scaling factor.

[0042] In one embodiment of this application, scaling the first target image according to the first scaling factor includes:

[0043] The first target image is interpolated based on the first resolution and the first scaling factor to scale the first target image.

[0044] In one embodiment of this application, determining a second cropping region in the second target image based on the aspect ratio of the second target resolution and / or the effective ultrasound region, wherein the second cropping region covers the effective ultrasound region, includes:

[0045] Perform a global pixel scan on the ultrasound image to find high-density pixel clusters in the ultrasound image;

[0046] The effective ultrasonic region is determined based on the high-density pixel aggregation region;

[0047] A second cropping region in the second target image is determined based on the aspect ratio of the second target resolution and / or the effective ultrasound region, wherein the second cropping region covers the effective ultrasound region.

[0048] In one embodiment of this application, determining a second cropping region in the second target image based on the aspect ratio of the second target resolution and / or the effective ultrasound region, wherein the second cropping region covers the effective ultrasound region, includes:

[0049] Obtain the selection information of the effective ultrasound region input by the user;

[0050] The effective area of ​​ultrasound is determined based on the selection information;

[0051] A second cropping region in the second target image is determined based on the aspect ratio of the second target resolution and / or the effective ultrasound region, wherein the second cropping region covers the effective ultrasound region.

[0052] In one embodiment of this application, scaling the second target image according to the second scaling factor includes:

[0053] The second target image is interpolated based on the second resolution and the second scaling factor to scale the second target image.

[0054] In one embodiment of this application, the scaling and / or cropping of the first target image according to the first target resolution includes:

[0055] Obtain color adjustment parameters;

[0056] The first target image is scaled and / or cropped according to the first target resolution, and the color of the scaled and / or cropped first target image is adjusted according to the color adjustment parameters.

[0057] In one embodiment of this application, obtaining the color adjustment parameters includes one of the following:

[0058] Obtain preset color adjustment parameters;

[0059] Alternatively, color adjustment parameters can be obtained from the second target image.

[0060] In one embodiment of this application, the scaling and / or cropping of the second target image according to the second target resolution includes:

[0061] Obtain color adjustment parameters;

[0062] The second target image is scaled and / or cropped according to the second target resolution, and the color of the scaled and / or cropped second target image is adjusted according to the color adjustment parameters.

[0063] In one embodiment of this application, obtaining the color adjustment parameters includes one of the following:

[0064] Obtain preset color adjustment parameters;

[0065] Alternatively, color adjustment parameters can be obtained from the first target image.

[0066] In one embodiment of this application, the color adjustment includes:

[0067] Perform Y-domain transformation on the first target image and the second target image to obtain a first target grayscale image and a second target grayscale image;

[0068] Brightness information of the first target image and brightness information of the second target image are obtained from the first target grayscale image and the second target grayscale image, respectively;

[0069] Based on the brightness information of the first target image and the brightness information of the second target image, the brightness of the second target image is adjusted so that the brightness of the second target image is consistent with the brightness of the first target image.

[0070] In one embodiment of this application, the color adjustment further includes:

[0071] The tonal information of the first target image and the tonal information of the second target image are obtained from the first target image and the second target image, respectively.

[0072] Based on the color tone information of the first target image and the color tone information of the second target image, the color tone of the first target image and the color tone of the second target image are adjusted to match the color tone of the first target image and the color tone of the second target image.

[0073] In one embodiment of this application, the color adjustment parameters include at least one of the following:

[0074] brightness;

[0075] tone;

[0076] Contrast;

[0077] Grayscale;

[0078] Saturation.

[0079] In one embodiment of this application, displaying the first target image in the endoscope display partition of the display area and displaying the second target image in the ultrasound image display partition includes:

[0080] The first target image is displayed in the endoscope display section of the display area;

[0081] The second target image is displayed in the external device image display partition of the display area;

[0082] The display area is filled with other areas besides the endoscope display area and the external device image display area.

[0083] In one embodiment of this application, displaying the first target image in the endoscope display partition of the display area and displaying the second target image in the ultrasound image display partition includes:

[0084] Acquire multiple frames of the first target image and multiple frames of the second target image;

[0085] After performing frame synchronization processing on multiple frames of the first target image and multiple frames of the second target image, the first target image is displayed in the endoscope display partition in the display area, and the second target image is displayed in the ultrasound display partition, wherein the frame rate and timing of the display of the first target image in the endoscope display partition and the display of the second target image in the ultrasound image display partition are synchronized.

[0086] In one embodiment of this application, acquiring multiple frames of the first target image and multiple frames of the second target image includes:

[0087] Based on the endoscopic image data, multiple frames of the first target image are cached in the first cache area of ​​the endoscopic system;

[0088] Based on the ultrasound image data, multiple frames of the second target image are cached in the second cache area of ​​the endoscope system.

[0089] In one embodiment of this application, the frame synchronization processing of multiple frames of the first target image and multiple frames of the second target image includes:

[0090] Based on the video output format of the display, the multiple frames of the first target image in the first buffer area and the multiple frames of the second target image in the second buffer area are respectively adjusted in time according to the video output timing sequence so that the frame rate and timing of the first target image and the second target image are synchronized.

[0091] In one embodiment of this application, the step of caching multiple frames of the first target image into a first cache area of ​​the endoscope system based on the endoscopic image data includes:

[0092] Based on the timestamp and frame rate of each endoscope image in the endoscope image data, the first target image corresponding to the multiple frames of the endoscope images is cached in the first cache area.

[0093] The step of caching multiple frames of the second target image into the second buffer area of ​​the endoscope system based on the ultrasound image data includes:

[0094] Based on the timestamp and frame rate of each ultrasound image in the ultrasound image data, multiple frames of the second target image corresponding to multiple frames of ultrasound images are cached in the second cache area.

[0095] In one embodiment of this application, the step of adjusting the time order of multiple frames of the first target image in the first buffer area and multiple frames of the second target image in the second buffer area according to the video output format of the display, so as to synchronize the frame rate and timing of the first target image and the second target image, includes:

[0096] Based on the video output timing and video output format of the display, multiple frames of the first target image are read from the first buffer area, and multiple frames of the second target image are read from the second buffer area, wherein the video output timing includes a video output timestamp, and the video output format includes a video output frame rate;

[0097] Based on the clock signal of the endoscope system, each frame of the first target image and each frame of the second target image are respectively stamped with the video output timestamp and cached in the output storage area of ​​the endoscope system;

[0098] Read multiple frames of the first target image and multiple frames of the second target image with the same time sequence from the output storage area, wherein the video frame rate synthesized from the multiple frames of the first target image and the multiple frames of the second target image is the same as the video output frame rate.

[0099] In one embodiment of this application, the first cache region and the second cache region include multi-level caches.

[0100] In one embodiment of this application, determining, according to the co-screen display mode, an endoscope display partition for co-screening the endoscopic image and at least one ultrasound image display partition for co-screening the ultrasound image from the display area of ​​the display includes:

[0101] Obtain the screen mirroring mode selection information preset by the display and / or input by the user, wherein the screen mirroring mode selection information is used to instruct the display to switch the screen mirroring mode;

[0102] The display area is divided according to the co-screen display mode to obtain the endoscope display partition for co-screen display of the endoscope image and at least one ultrasound image display partition for co-screen display of the ultrasound image.

[0103] In one embodiment of this application, multiple screen-sharing modes are included, and the screen-sharing arrangement of the endoscope display partition and the ultrasound image display partition is different in different screen-sharing modes.

[0104] In one embodiment of this application, the resolution of the endoscope display partition and the ultrasound image display partition may be the same or different.

[0105] In one embodiment of this application, acquiring endoscopic image data includes:

[0106] The endoscope video source image signal is acquired in real time and analyzed to obtain the endoscope image data.

[0107] In one embodiment of this application, acquiring at least one ultrasound image data from at least one ultrasound device communicatively connected to the endoscope system includes:

[0108] The endoscope system establishes a communication connection with at least one of the ultrasound devices via wired and / or wireless means to acquire ultrasound image signals from the ultrasound devices in real time.

[0109] The ultrasound image signal is analyzed to obtain the ultrasound image data.

[0110] On the other hand, embodiments of this application provide an image display method for an endoscope system, the endoscope system including an endoscope host and a display, the image display method including:

[0111] Acquire endoscopic image data, which includes multiple frames of endoscopic images;

[0112] At least one ultrasound image data is acquired from at least one ultrasound device that is communicatively connected to the endoscope system, the ultrasound image data comprising multiple frames of ultrasound images;

[0113] According to the screen-sharing mode, an endoscope display partition for displaying the endoscope image on the same screen and at least one ultrasound image display partition for displaying the ultrasound image on the same screen are determined from the display area of ​​the display, wherein the endoscope display partition and the ultrasound image display partition are displayed on the same screen in the display area;

[0114] Multiple frames of the endoscope image are obtained from the endoscope image data, and multiple frames of the ultrasound image are obtained from the ultrasound image data;

[0115] Based on the endoscopic image data, a first target image corresponding to the endoscopic image is obtained;

[0116] Based on the ultrasound image, the effective ultrasound region is determined, and a second target image of the effective ultrasound region is obtained.

[0117] The first target image is displayed in the endoscope display section of the display area, and the second target image is displayed in the ultrasound image display section.

[0118] In one embodiment of this application, determining the effective ultrasound region based on the ultrasound image and obtaining a second target image of the effective ultrasound region includes:

[0119] Edge detection is performed on the ultrasound image to identify target tissue in the ultrasound image;

[0120] The effective area of ​​ultrasound is determined based on the target tissue, and a second target image of the effective area of ​​ultrasound is obtained.

[0121] In one embodiment of this application, determining the effective ultrasound region based on the target tissue and obtaining a second target image of the effective ultrasound region includes:

[0122] Based on the target tissue, the effective area of ​​ultrasound is determined, and a local ultrasound image of the effective area of ​​ultrasound is obtained;

[0123] Text information from the ultrasound image is obtained, and a second target image of the effective ultrasound region is obtained based on the text information and the local ultrasound image.

[0124] On the other hand, embodiments of this application provide an endoscope system, characterized in that it includes:

[0125] Endoscope;

[0126] A display, the display including a display area for displaying images;

[0127] The endoscope host is communicatively connected to the endoscope and the display, and the endoscope host is also used to execute the image display method of the endoscope system in any of the above embodiments.

[0128] On the other hand, embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the image display method of any of the endoscope systems described in the above embodiments.

[0129] On the other hand, embodiments of this application also provide a computer-readable storage medium storing a computer program, which is executed by a processor to implement the image display method of the endoscope system described in any of the above embodiments.

[0130] On the other hand, embodiments of this application also provide a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform an image display method implementing any of the endoscope systems described in the above embodiments.

[0131] The embodiments of this application include at least the following beneficial effects: By acquiring endoscopic image data and ultrasound image data, wherein the endoscopic image data includes multiple frames of endoscopic images and the external video source image data includes multiple frames of external video source images, and according to the co-screen display mode, an endoscopic display partition and an ultrasound image display partition are determined from the display area of ​​the monitor, a first target image corresponding to the endoscopic image is obtained based on the endoscopic image data, and a second target image is obtained by determining the effective area of ​​the ultrasound based on the ultrasound image, so that the first target image is displayed in the endoscopic display partition and the second target image is displayed in the external device image display partition. The embodiments of this application not only enable the co-screen display of endoscopic images and external video source images on the monitor, but also effectively identify the effective area in the ultrasound image, allowing the co-screen displayed ultrasound image to focus on the effective area, facilitating doctors' observation of the details of the effective area in the co-screen displayed ultrasound image, resulting in better display effects and facilitating observation by doctors in clinical applications.

[0132] In one embodiment, edge detection is performed on the ultrasound image to identify fan-shaped regions in the ultrasound image, resulting in a fan-shaped region image. Then, edge detection is performed again on the fan-shaped region image to identify target tissue in the fan-shaped region image. Based on the target tissue, an effective ultrasound region is determined. The second target image determined by the effective ultrasound region can accurately identify the target tissue to be observed in the ultrasound image, thereby enabling the second target image to display the target tissue in a targeted manner and improving the simultaneous display effect of ultrasound images and endoscopic images. Attached Figure Description

[0133] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0134] Figure 1 This is a schematic diagram of the structure of an endoscopic camera system provided in one embodiment of this application;

[0135] Figure 2 This is a schematic diagram of the connection structure of an endoscope camera system provided in one embodiment of this application;

[0136] Figure 3 A schematic diagram of the connection structure of an endoscope camera system provided in another embodiment of this application;

[0137] Figure 4 This is a flowchart illustrating an image display method provided in one embodiment of this application.

[0138] Figure 5A schematic diagram of the display area is provided for one embodiment of this application;

[0139] Figure 6 A schematic diagram of the display area is provided for one embodiment of this application;

[0140] Figure 7 A schematic diagram of the display area is provided for one embodiment of this application;

[0141] Figure 8 A schematic diagram of the display area is provided for one embodiment of this application;

[0142] Figure 9 A schematic diagram of the display area is provided for one embodiment of this application;

[0143] Figure 10 A schematic diagram of the display area is provided for one embodiment of this application;

[0144] Figure 11 A schematic diagram of the display area is provided for one embodiment of this application;

[0145] Figure 12 A schematic diagram of an ultrasound image provided in one embodiment of this application;

[0146] Figure 13 A schematic diagram of a second target image provided in one embodiment of this application;

[0147] Figure 14 for Figure 4 Detailed flowchart of step 406;

[0148] Figure 15 This is a flowchart illustrating the adaptive adjustment of a display according to one embodiment of the present application.

[0149] Figure 16 This is a flowchart of a scaling process provided in one embodiment of this application;

[0150] Figure 17 This is a flowchart illustrating a trimming process provided in one embodiment of this application.

[0151] Figure 18 This is a flowchart illustrating a trimming process provided in one embodiment of this application.

[0152] Figure 19 This is a flowchart illustrating a trimming process provided in one embodiment of this application.

[0153] Figure 20 This is a flowchart illustrating a trimming process provided in one embodiment of this application.

[0154] Figure 21 for Figure 15 Flowchart of the specific method for step 1502;

[0155] Figure 22 for Figure 15 Flowchart of the specific method for step 1503;

[0156] Figure 23 A schematic diagram illustrating color adjustment processing according to one embodiment of this application;

[0157] Figure 24 A flowchart illustrating a color adjustment method provided in one embodiment of this application;

[0158] Figure 25 for Figure 4 Detailed flowchart of step 407;

[0159] Figure 26 This is a schematic diagram of frame synchronization processing provided in one embodiment of this application;

[0160] Figure 27 This is a flowchart of an image display method provided in one embodiment of this application;

[0161] Figure 28 This is a schematic diagram illustrating the execution of an image display method for an endoscope system according to an embodiment of this application.

[0162] Figure 29 This is a flowchart of an image display method provided in one embodiment of this application. Detailed Implementation

[0163] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0164] It should be noted that in various specific embodiments of this application, when processing data related to the characteristics of the target object, such as target object attribute information or attribute information sets, is required, the permission or consent of the target object will be obtained first. Furthermore, the collection, use, and processing of this data will comply with relevant laws, regulations, and standards. The target object can be a user. In addition, when embodiments of this application need to obtain target object attribute information, separate permission or consent from the target object will be obtained through pop-ups or redirection to a confirmation page. Only after obtaining the target object's separate permission or consent will the necessary target object-related data for the normal operation of the embodiments of this application be obtained.

[0165] To facilitate understanding of the technical solutions provided in the embodiments of this application, some key terms used in the embodiments of this application will be explained below:

[0166] Reference Figure 1 The diagram shown is a structural schematic of an endoscope camera system 1000, which includes, but is not limited to, a light source host 10, a beam guide 20, an endoscope 30, an optical bayonet 40, a camera 50, an endoscope host 60, a display 70, and a communication cable 80.

[0167] One end of the beam guide 20 is connected to the light source interface of the light source host 10, and the other end is connected to the light source interface of the endoscope 30. The beam guide 20 is used to provide a light source for the endoscope to illuminate the part of the object to be observed.

[0168] One end of the camera 50 is connected to the image signal input interface of the endoscope host 60 via a communication cable 80, and the other end is used to snap onto the optical bayonet 40 of the endoscope 30. It is used to acquire the optical signals of the endoscope 30 regarding the observed part 100 of the subject, image the signal, and transmit the acquired image signal to the endoscope host 60 for processing. The camera 50 includes a handle with several buttons, allowing the user to press a button to activate its corresponding function. The observed part 100 can be a human body.

[0169] The display 70 is connected to the endoscope host 60 via a communication cable 80 and is used to display images output by the endoscope host 60. The display 70 may include a liquid crystal display, an LED display, or an OLED display, or it may be a display on an electronic device such as a mobile phone, tablet computer, or personal computer connected to the endoscope host 60.

[0170] The endoscope host 60 is connected to the camera 50 and receives endoscopic image data sent by the camera 50. After processing the endoscopic image data sent by the camera 50, the endoscope host 60 transmits a display signal to the display 70 via the communication cable 80, causing the display 70 to display the endoscopic image from the endoscopic image data. In one embodiment, see... Figure 2 As shown, the endoscope host 60 is also communicatively connected to the ultrasound device 200 to receive ultrasound image data sent by the ultrasound device 200, which is an independent device from the endoscope host 60. The endoscope host 60 processes the endoscope image data and ultrasound image data according to the screen display mode and the first display parameters in the endoscope display zone and the second display parameters in the ultrasound image display zone, respectively. Then, it sends a display signal to the display 70 via the communication cable 80, so that the display area of ​​the display 70 simultaneously displays the endoscope image and the ultrasound image.

[0171] In one embodiment, the endoscope host 60, in addition to connecting to the ultrasound device 200, can also connect to other external imaging devices 210, such as those used in [other applications]. Figure 2As shown, the endoscope host 60 can also be connected to an X-ray device, a 3D reconstruction model display device, or another ultrasound device. The endoscope host 60 can also be connected to another external endoscope host. The functions of this external endoscope host can be... Figure 1 The endoscope host 60 shown has the same function as the endoscope host 60, or it may have a different function. For example, it may be a fluorescence endoscope host, a 3D endoscope host, etc. Of course, in some embodiments, the function of the endoscope host is reflected in the type of endoscope it is connected to. The external device endoscope host may also use the same hardware as the endoscope host 60 but is connected to a different type of endoscope.

[0172] In one embodiment, the endoscope host 60 can communicate with the ultrasound device 200 via wired or wireless means. The communication method can be a point-to-point connection or a network connection via a local area network (LAN) or the Internet. For example, for a wired connection, the endoscope host 60 can be equipped with a communication interface for connecting to the ultrasound device, which is then connected to the endoscope host 60 via a cable. This communication interface can be a dedicated interface or a general-purpose interface, such as a USB interface, HDMI video communication interface, VGA interface, DP interface, or Ethernet port. In another embodiment, the endoscope host 60 can be equipped with a wireless interface, such as an external wireless communication antenna or a built-in board antenna, to wirelessly transmit data with the ultrasound device. For example, it can communicate wirelessly with the ultrasound device via Wi-Fi, UWB, NB-IoT, or Bluetooth.

[0173] In one embodiment, the endoscope host 60 includes a screen displaying a visual interface. Through this visual interface, the user can set the button functions of the camera 50 or obtain the current setting parameters of the camera 50, including a handle button mode area, an image brightness adjustment area, a brightness parameter display area, etc.

[0174] See Figure 3As shown, in one embodiment, the endoscope host 60 may include a camera host 610 and an image processing device 620. The camera host 610 is connected to the camera 50 via a communication cable 80. After receiving the endoscope image signal captured by the camera 50 and processing it, the camera host 610 obtains endoscope image data. The camera host 610 sends the endoscope image data to the image processing device 620. In addition to obtaining the endoscope image data from the camera host 610, the image processing device 620 is also communicatively connected to the ultrasound device 200 and other external imaging devices 210 other than the ultrasound device 200 to obtain the ultrasound image data sent by the ultrasound device 200. In one embodiment, the image processing device 620 is provided with communication interfaces that are connected to the camera host 610 and the image processing device 620 respectively. After processing the endoscope image data and ultrasound image data according to the screen display mode, the first display parameters of the endoscope display partition and the second display parameters of the external device image display partition, the image processing device 620 sends a display signal to the display 70 through the communication cable 80 so that the display area in the display 70 displays the endoscope image and ultrasound image on the same screen.

[0175] Skilled technicians should understand that Figure 1 This is merely an example of an endoscopic camera system 1000 and does not constitute a limitation on the endoscopic camera system 1000. The endoscopic camera system 1000 may include, but is not limited to, endoscopic camera systems 1000. Figure 1 The endoscopic camera system 1000 may include more or fewer components, or combinations of certain components, or different components. It may also include a dilator, a smoke control device, input / output devices, network access devices, etc.

[0176] See Figure 4 As shown in the figure, this application provides an image display method for an endoscope system, which can be performed by... Figures 1 to 3 The endoscope system 1000 shown is performing an image display method including, but not limited to, the following steps 401 to 407:

[0177] Step 401: Acquire endoscopic image data, which includes multiple frames of endoscopic images.

[0178] In this step, the endoscope host 60 in the endoscope system acquires endoscopic image data. In one embodiment, the endoscopic image data is captured by a camera, which is equipped with an optical lens assembly and an image sensor (such as a CCD (Charge Coupled Device) or CMOS (Complementary Meta-Oxide Semiconductor)). Light emitted from the light source host illuminates the patient's body, and the reflected or excitation light is collected by the endoscope and transmitted to the image sensor of the camera. The camera converts the reflected or excitation light into an electrical signal, thereby obtaining an endoscopic image. The camera continuously captures multiple frames of endoscopic images, thus obtaining endoscopic image data. The camera can continuously send (or send in real-time) endoscopic image data to the endoscope host using a video stream or data stream. The endoscopic image data includes multiple frames of endoscopic images.

[0179] In one embodiment, the endoscope host 60 acquires the endoscope video source image signal in real time and analyzes the endoscope video source image signal to obtain the endoscope image data.

[0180] In one embodiment, endoscopic image data can also be referred to as endoscopic video data. The multiple frames of endoscopic images in the endoscopic video data are continuous in time. In another embodiment, the multiple frames of endoscopic images in the endoscopic image data may be discontinuous in time. For example, they may be multiple frames of endoscopic images acquired at different specific times. These endoscopic images are not required to be continuous and cannot form a video or dynamic image. For example, endoscopic images taken every 5 minutes.

[0181] Step 402: Acquire at least one ultrasound image data from at least one ultrasound device communicatively connected to the endoscope system, the ultrasound image data comprising multiple frames of ultrasound images.

[0182] In this step, the endoscope host 60 acquires ultrasound image data from the ultrasound device 200, wherein the ultrasound image data includes multiple frames of ultrasound images. In one embodiment, the ultrasound device 200 is a device independent of the endoscope host 60, and the ultrasound image data includes multiple frames of ultrasound images. In another embodiment, the endoscope host 60 can acquire ultrasound image data from multiple ultrasound devices 200 respectively.

[0183] In one embodiment, the endoscope host 60 establishes a communication connection with at least one of the external imaging devices via wired and / or wireless means, and acquires the ultrasound image signals of the ultrasound device in real time. Then, the ultrasound image signals are analyzed to obtain multiple frames of ultrasound image data.

[0184] Step 403: Based on the screen display mode, determine an endoscope display partition for displaying the endoscope image on the same screen and at least one ultrasound image display partition for displaying the ultrasound image on the same screen from the display area of ​​the display.

[0185] In this step, the endoscope host 60 first determines the screen-sharing mode of the display 70. This screen-sharing mode is the control parameter used by the endoscope host 60 (or the image processing device 620 in the endoscope host 60) to control the display to perform screen-sharing. The screen-sharing mode is used to control the arrangement of different display partitions in the display area of ​​the display 70 when they are displayed on the same screen. For example, see [link to relevant documentation]. Figure 5 The embodiment shown illustrates a simultaneous display mode where the display 70 includes a display area 710, which further includes an endoscope display partition 720 and an ultrasound image display partition 730. Figure 5 In the simultaneous display mode shown (e.g., named the first display mode), the endoscope display area 720 and the ultrasound image display area 730 are displayed side-by-side in the display area 710, and the endoscope display area 720 and the ultrasound image display area 730 have the same resolution. See also Figure 5 As shown, the endoscope display area 720 and the ultrasound image display area 730 are displayed side-by-side in the display area 710 using a common midline. In another embodiment, the endoscope display area 720 and the ultrasound image display area 730 may also be separated by a certain distance instead of sharing a common midline.

[0186] In one embodiment, the resolution of the endoscopic display zone and the ultrasound image display zone may also be different, for example, see [reference needed]. Figure 6 In one embodiment of the simultaneous display mode (e.g., named the second display mode), the endoscope display area 720 and the ultrasound image display area 730 are also displayed simultaneously, left and right respectively, in the display area 710. However, the display area of ​​the endoscope display area 720 is larger than that of the ultrasound image display area 730. Of course, it is also possible to... Figure 7 In the simultaneous display mode shown (e.g., named the third display mode), the positions of the endoscope display area 720 and the ultrasound image display area 730 are swapped, and the display area of ​​the ultrasound image display area 730 is larger than that of the endoscope display area 720. In the simultaneous display mode, it is not necessary for the displayed image to fill the entire display area 710; for example, see [reference needed]. Figure 7As shown, the area between the edge of the endoscope display partition 720 and the display area 710 can be filled with a background color (e.g., displayed as black). In one embodiment, other auxiliary information, such as text information, marking information, etc., can also be displayed in the area next to the display partition (e.g., the endoscope display partition 720 or the external device image display partition 730) where there is no image. Of course, in addition to the same-screen display mode of the above embodiments, the same-screen display can have different arrangement directions. In addition to left-right arrangement, it can also be arranged vertically or in a mixed arrangement (e.g., the display partitions on the left side of the display area are arranged vertically, and the right side area has only one partition or the right side area has two or more display partitions arranged horizontally).

[0187] In one embodiment, in addition to displaying two display partitions on the same screen, more display partitions can be set to display on the same screen. For example, one display partition can be used to display ultrasound images from an ultrasound device, and another display partition can be used to display X-ray images from an X-ray device. See also Figure 8 In another embodiment, the display area 710 may also include multiple endoscope display partitions 720, i.e. Figure 1 The endoscope host 60 shown can receive two endoscopic image signals captured by two endoscopes 30, and in Figure 8 The images are displayed in two different endoscope display zones 720 within the display area 710 shown. For example, one endoscope display zone 720 displays a white light endoscope image, while the other endoscope display zone 720 displays a fluorescence endoscope image.

[0188] In another embodiment, in addition to side-by-side display, the screen-sharing mode can also display images in an overlay format, for example, see [link to relevant documentation]. Figure 9 As shown, the ultrasound image display partition 730 is displayed full-screen in the display area 710, while the endoscope display partition 720 is overlaid in the ultrasound image display partition 730. In another embodiment, see... Figure 10 As shown, the endoscope display partition 720 can also be displayed in full screen in the display area 710, and the ultrasound image display partition 730 can be superimposed on the endoscope display partition 720.

[0189] In one embodiment, the screen display mode of the display 70 can be preset, selected by the user, or automatically set according to conditions. For example, the endoscope host 60 can switch between different screen display modes in response to user input commands. Alternatively, the endoscope host 60 can automatically activate or switch between different screen display modes based on acquired endoscopic image data and / or ultrasound image data. Different screen display modes can be different preset mode schemes for selection, or screen display schemes for on-site setup. In one embodiment, the endoscope host 60 obtains user-inputted manual setting parameters and switches between different screen display modes based on these parameters, or sets the screen display mode, for example, by user-inputted manual setting parameters to adjust the type, arrangement, and size of different display partitions. In another embodiment, user-inputted manual setting parameters can also set the display parameters of the display partitions in the display area 710 (e.g., setting the first display parameter of the endoscope display partition 720 or setting the second display parameter of the external device image display partition 730). Furthermore, the first and second display parameters in different screen display modes can be manually set separately.

[0190] In one embodiment, the endoscope host 60 determines the current screen display mode and determines the display parameters of each display partition in the display area 710 according to the screen display mode, such as determining the first display parameter of the endoscope display partition and the second display parameter of the ultrasound image display partition.

[0191] In one embodiment, the endoscope host 60 can first determine the display parameters of each display partition within the display area 710 according to the current screen-sharing display mode, and then control the display 70 to display based on the display parameters. In another embodiment, the endoscope host 60 can control the display 70 to display based on the current screen-sharing display mode, and then determine the corresponding display parameters from the display status of each display partition in the display 70.

[0192] In one embodiment, when the current screen-sharing display mode is switched or changed, the display state of each display partition in the new screen-sharing display mode will change (e.g., the arrangement or resolution will change), and therefore the corresponding display parameters will also change (also known as refreshing or updating display parameters). For example, after the endoscope host 60 receives a user-input command to switch the screen-sharing display mode, in addition to changing the arrangement and / or resolution of each display partition within the display area 710, it will also refresh (or update) the display parameters of each display partition, such as updating the first or second display parameters summarized in the above embodiments. In one embodiment, the screen-sharing display mode can be switched with a single button on the camera 50 handle. For example, the screen-sharing display mode, the first display parameter, or the second display parameter can be set by inputting the aforementioned manual setting parameters through the buttons on the camera 50 handle.

[0193] In one embodiment, the display parameter of each display partition in the display area 710 is the resolution size. For example, the first display parameter is the first target resolution of the endoscope display partition, and the second display parameter is the second target resolution. See, for example... Figure 11 As shown, the display area 710 includes a first target display partition 1110 (e.g., an endoscope display partition) and a second target display partition 1120 (e.g., an ultrasound image display partition). The first target resolution of the first target display partition 1110 is 2650*2160, and the second target resolution of the second target display partition 1120 is 1280*720. Of course, this embodiment can support various resolutions, such as 1080P / 720P / 540P, etc.

[0194] In one embodiment, the endoscope host 60 divides the display area according to the simultaneous display mode to obtain a first target display area and a second target display area. Then, the endoscope host 60 determines the first target resolution based on the first target display area and the second target resolution based on the second target display area.

[0195] Step 404: Obtain multiple frames of the endoscope image data from the endoscope image data, and obtain multiple frames of the ultrasound image data from the ultrasound image data.

[0196] In this step, the endoscope host 60 acquires multiple frames of the endoscope image data from the endoscope image data and multiple frames of the ultrasound image data from the ultrasound image data. In one embodiment, the endoscope host 60 stores the endoscope image data sent by the camera 50 and the ultrasound image data sent by the ultrasound device 200 in the storage of the endoscope host 60 (e.g., cache, memory, or hard disk). The endoscope host 60 obtains one or more frames of the endoscope image by reading the stored endoscope image data, and obtains one or more frames of the ultrasound image by reading the stored ultrasound image data.

[0197] Step 405: Obtain a first target image corresponding to the endoscope image based on the endoscope image data.

[0198] In this step, the endoscope host 60 selects the endoscope image to be processed from the multiple frames of endoscope images in step 404 above. In one embodiment, the endoscope host 60 can use the selected endoscope image as the first target image. For example, it can select the corresponding endoscope image from the endoscope image data according to local timing requirements, synchronizing it with the timing of the ultrasound image. The endoscope host 60 can subsequently continue to process and display the first target image, for example, it can continue to perform adaptive display processing on the first target image.

[0199] In another embodiment, the processing of the endoscopic image can also be completed in step 405. For example, the endoscope host 60 can adaptively adjust the display of the endoscopic image in the endoscopic image data according to the first display parameters to obtain the first target image. In one embodiment, since the endoscopic image data includes multiple frames of endoscopic images, the endoscope host 60 can sequentially perform adaptive display adjustment on each frame of endoscopic image to obtain multiple frames of the first target image. Here, adaptive display adjustment refers to image processing of the endoscopic image to adapt it to the display effect of the endoscopic display partition (e.g., adapting to the resolution size of the endoscopic display partition). In one embodiment, the first display parameters include the first target resolution of the endoscopic display partition 720. The endoscope host 60 performs scaling and / or cropping processing on the endoscopic image according to the first target resolution to obtain the first target image, wherein the resolution of the first target image matches the first target resolution of the endoscopic display partition 720 (e.g., the same resolution size, the same resolution size in the height direction, the same resolution size in the length direction, or having the same aspect ratio, etc.).

[0200] In one embodiment, this step may involve first cropping the first target image, and then scaling the cropped first target image according to the first target resolution. Alternatively, the first target image may be scaled according to the first target resolution first, and then cropped.

[0201] In one embodiment, the endoscope host 60 can first identify the effective region in the endoscope image, and then determine the first target image highlighted by the endoscope based on the effective region in the endoscope image. For example, in one embodiment, the endoscope host 60 performs image recognition on the endoscope image to determine the position of the surgical instruments in the endoscope image, and determines the effective region of the endoscope based on the position of the surgical instruments, thereby obtaining the first target image of the effective region of the endoscope. In another embodiment, the endoscope host 60 performs image recognition on the endoscope image to determine the surgical field region of the endoscope image, and determines the effective region of the endoscope based on the surgical field region, thereby obtaining the first target image of the effective region of the endoscope. In yet another embodiment, the user can select the effective region in the endoscope image. For example, the endoscope host 60 can obtain the endoscope effective region selection information input by the user, determine the effective region of the endoscope based on the endoscope effective region selection information, and obtain the first target image of the effective region of the endoscope.

[0202] In another embodiment, the endoscope host can also adjust the color of the endoscope image before obtaining the first target image. For example, after the endoscope host 60 obtains the color adjustment parameters, it adjusts the color of the endoscope image according to the color adjustment parameters to obtain the first target image. The color adjustment parameters can be preset parameters or obtained from ultrasound images or other external video source images. For example, by obtaining the color adjustment parameters of the ultrasound image to adjust the color of the endoscope image, the display effect of the endoscope image and the ultrasound image can be made consistent, resulting in a better simultaneous display effect of the endoscope image and the ultrasound image.

[0203] In another embodiment, before obtaining the first target image, the endoscope host 60 can also perform display adaptive adjustment and color adjustment on the endoscope image. For example, the endoscope image can be displayed adaptively adjusted first and then the color adjustment can be performed as described above, or the endoscope image can be color adjusted first and then the endoscope image can be displayed adaptively adjusted.

[0204] Step 406: Perform edge detection on the ultrasound image to identify fan-shaped regions in the ultrasound image and obtain a fan-shaped region image; perform edge detection on the fan-shaped region image to identify target tissue in the fan-shaped region image; determine the effective ultrasound region based on the target tissue and obtain a second target image of the effective ultrasound region.

[0205] In this step, the endoscope host 60 selects the ultrasound image to be processed from the multi-frame ultrasound image data in step 404 above, and performs ultrasound effective region identification processing on the ultrasound image to obtain the second target image. See also Figure 12 The image shown is an ultrasound image (i.e., an ultrasound image) acquired by the endoscope host 60 from an ultrasound device. The ultrasound image includes a fan-shaped region image 1210, representing the signal detection range of the ultrasound device. Within the fan-shaped region image 1210, a target tissue image 1220 is also displayed. The target tissue corresponding to the target tissue image 1220 is the detection target of the ultrasound device, such as follicles, the heart, or blood vessels. The target tissue image 1220 occupies a portion of the fan-shaped region image 1210. Additionally, in the monochrome region (e.g., a black or gray area) outside the fan-shaped region image 1210, there is also ultrasound image-related text information 1630, such as text reflecting detection parameters and results. In this step, the endoscope host 60 acquires... Figure 12 Edge detection is performed on the ultrasound image shown to identify fan-shaped regions within the ultrasound image, resulting in a fan-shaped region image 1210. Then, the endoscope host 60... Figure 16 The fan-shaped region image 1210 shown is subjected to edge detection again to identify the target tissue region in the fan-shaped region image 1210.

[0206] Then, the endoscope host 60 determines the target tissue image 1220 of the target tissue 1620 based on the target tissue region of the fan-shaped region image 1210.

[0207] In one embodiment, the endoscope host 60 may also acquire text information from the ultrasound image (i.e., ultrasound image) (see, for example, [link to relevant documentation]). Figure 16 The text information 1230 shown is used to determine the target tissue image based on the text information and the target tissue region in the fan-shaped region image. For example, see [link to relevant documentation]. Figure 13 As shown, the endoscope host 60 determines the target tissue image 1220 based on the target tissue region in the fan-shaped region image 1210, and overlays text information 1230 onto the target tissue image 1220. In one embodiment, the image at the location of the text information 1230 can be extracted from the ultrasound image and overlaid onto the target tissue image. In another embodiment, the text information 1230 can be recognized in the ultrasound image, and then the recognized text information 1230 can be overlaid onto the target tissue image 1220, which improves the display effect of the target tissue image. This allows medical personnel to observe the target tissue image 1220 more clearly without losing the acquisition of text information during ultrasound detection while the display area 710 is on the same screen.

[0208] In one embodiment, the endoscope host 60 can further process and display the second target image, for example, by performing adaptive display processing on the second target image. In another embodiment, the processing of the ultrasound image can also be completed in step 406. For example, the ultrasound host 60 can adaptively adjust the display of the ultrasound image in the ultrasound image data according to the second display parameters to obtain the second target image. In one embodiment, the endoscope host 60 determines the effective ultrasound region according to the second display parameters and the target tissue identified by edge detection. For example, the size of the effective ultrasound region is consistent with the second target resolution size in the second display parameters, or the size of the cropped area corresponding to the effective ultrasound region is consistent with the second target resolution size.

[0209] In one embodiment, since the ultrasound image data includes multiple frames of ultrasound images, the ultrasound host 60 can sequentially perform adaptive display adjustment on each frame of ultrasound image to obtain multiple frames of second target images. Adaptive display adjustment refers to image processing of the ultrasound images to adapt them to the display effect of the ultrasound display partition (e.g., adapting to the resolution size of the ultrasound display partition). In one embodiment, the second display parameter includes the second target resolution of the ultrasound display partition 730. The ultrasound host 60 performs scaling and / or cropping processing on the effective ultrasound area in the ultrasound image according to the second target resolution to obtain the second target image, wherein the resolution of the second target image matches the second target resolution of the ultrasound display partition 730 (e.g., the same resolution size, the same resolution size in the height direction, the same resolution size in the length direction, or having the same aspect ratio, etc.). The scaling processing can be image magnification or image reduction.

[0210] In one embodiment, this step may involve first cropping the second target image, and then scaling the cropped second target image according to the second target resolution. Alternatively, the second target image may be scaled according to the second target resolution first, and then cropped.

[0211] In another embodiment, the ultrasound host can also perform color adjustment on the ultrasound image before obtaining the second target image. For example, after the ultrasound host 60 obtains the color adjustment parameters, it adjusts the color of the effective ultrasound region of the ultrasound image according to the color adjustment parameters to obtain the second target image. The color adjustment parameters can be preset parameters or obtained from the endoscopic image or other external video source images. For example, by obtaining the color adjustment parameters of the endoscopic image to adjust the color of the ultrasound image, the display effect of the ultrasound image and the endoscopic image can be made consistent, resulting in a better simultaneous display effect of the ultrasound image and the endoscopic image.

[0212] In another embodiment, before obtaining the second target image, the ultrasound host 60 can also perform adaptive display adjustment and color adjustment on the ultrasound image. For example, the ultrasound image can be adaptively adjusted first and then the color adjustment can be performed as described above, or the ultrasound image can be color adjusted first and then the ultrasound image can be adaptively adjusted.

[0213] Step 407: Display the first target image in the endoscope display partition of the display area, and display the second target image in the ultrasound image display partition.

[0214] In this step, after obtaining the first target image and the second target image, the endoscope host 60 controls the display 70 to display the first target image in the endoscope display partition and the second target image in the ultrasound image display partition 730. Referring to the description of step 403 above, the endoscope host 60 controls the first and second target images to be displayed simultaneously in the display area 710 according to the current simultaneous display mode. This can also be understood as follows: since the display area 710 is fixed with the endoscope display partition 720 and the ultrasound image display partition 730 for simultaneous display, the first target image displayed in the endoscope display partition 720 and the second target image displayed in the ultrasound image display partition 730 are naturally also displayed simultaneously in the display area 710. Referring to the description of step 403 in the above embodiment, the arrangement of the first and second target images in the display area 710 can be different under different simultaneous display modes. The resolution of the first and second target images can also be different under different simultaneous display modes.

[0215] In one embodiment, the endoscope host 60 can receive endoscopic image data captured by multiple endoscopes 30, thus enabling the endoscope host 60 to obtain multiple first target images. These multiple first target images can also be displayed simultaneously with second target images in the display area 710. In another embodiment, the endoscope host 60 can receive ultrasound image data transmitted by multiple ultrasound devices 200, thus enabling the endoscope host 60 to obtain multiple second target images. These multiple second target images can also be displayed simultaneously with the first target images in the display area 710. Alternatively, multiple first target images and multiple second target images can be displayed simultaneously in the display area 710.

[0216] In one embodiment, in addition to displaying the first target image in the endoscope display partition and the second target image in the external device image display partition of the display area, the endoscope host 60 also fills other areas of the display area besides the endoscope display partition and the external device image display partition. For example, when the target image (e.g., the first target image or the second target image) after adaptive adjustment by the endoscope host 60 has the same resolution as the display partition in only one direction (e.g., the height or length direction), a predetermined color (e.g., a black border) can be filled around the display partition. See, for example, [link to relevant documentation]. Figure 11 As shown, the left side of the first target display partition 1110 within the display area 710 can be designated as the second target display partition 1120. The resolution of this second target display partition is 1280*720, thus matching the resolution of the second target display partition 1120 in the length direction. However, they do not match in the height direction. Therefore, black can be used to fill the upper and lower areas of the second target display partition 1120, meaning the resolution of both the upper and lower black areas of the second target display partition 1120 is 1280*720. The first target display partition 1110 and the second target display partition 1120 can be endoscope display partitions or external device image display partitions. In another embodiment, other auxiliary information, such as text information or marking information, can also be displayed in the filled areas.

[0217] This application provides an image display method for an endoscope system. By acquiring endoscopic image data and ultrasound image data, wherein the endoscopic image data includes multiple frames of endoscopic images and the external video source image data includes multiple frames of external video source images, and according to a simultaneous display mode, an endoscopic display partition and an ultrasound image display partition are determined from the display area of ​​the monitor. A first target image corresponding to the endoscopic image is obtained based on the endoscopic image data, and a second target image is obtained by determining the effective ultrasound region based on the ultrasound image. This allows the first target image to be displayed in the endoscopic display partition and the second target image to be displayed in the external device image display partition. This application not only enables the simultaneous display of endoscopic images and external video source images on the monitor but also effectively identifies the effective region in the ultrasound image, allowing the simultaneous ultrasound image to be displayed targeting the effective region. This facilitates doctors' observation of details in the effective region of the simultaneously displayed ultrasound image, resulting in better display effects and facilitating clinical observation. Furthermore, by performing edge detection on the ultrasound image to identify fan-shaped regions within it, a fan-shaped region image is obtained. Then, edge detection is performed again on the fan-shaped region image to identify the target tissue within it. Based on the target tissue, an effective ultrasound region is determined. A second target image is obtained through this effective ultrasound region. By performing edge detection twice on the ultrasound image from the ultrasound device, an accurate target tissue image can be obtained in the ultrasound image. Based on the target tissue image, the effective region of the ultrasound image is determined, resulting in an effective region image of the external image's effective region. A second cropping region in the ultrasound image is determined based on the aspect ratio of the second target resolution, thus obtaining the second target image. This makes the target tissue image in the ultrasound image more prominent when the second target image and the first target image are displayed simultaneously in display area 710. This makes it easier for doctors to observe the target tissue image in the simultaneous display in display area 710, improving the simultaneous display effect of ultrasound and endoscopic images.

[0218] See Figure 14 As shown, in one embodiment, step 406 above, determining the effective ultrasound region based on the target tissue and obtaining a second target image of the effective ultrasound region, includes the following steps:

[0219] Step 1401: Determine the effective area of ​​ultrasound based on the target tissue, and obtain a local ultrasound image of the effective area of ​​ultrasound.

[0220] In one embodiment of this step, the endoscope host 60 can determine the effective ultrasound region based on the target tissue identified by edge detection, and determine a local ultrasound image in the ultrasound image based on the effective ultrasound region, for example... Figure 13The target tissue image 1220 is shown. In one embodiment, the endoscope host can determine the size and / or proportion of the effective ultrasound region based on the second display parameters (e.g., the second target resolution) of the ultrasound image display partition 730, so that the effective ultrasound region can cover the target tissue. For example, the center of the target tissue is determined as the center of the effective ultrasound region, and then the aspect ratio of the effective ultrasound region is determined according to the aspect ratio of the second target resolution. Then, the size of the effective ultrasound region is determined under this aspect ratio, so that the effective ultrasound region can cover the identified target tissue.

[0221] Step 1402: Obtain text information from the ultrasound image, and based on the text information and the local ultrasound image, obtain a second target image of the effective ultrasound region.

[0222] In this step, the endoscope host 60 also acquires text information from the ultrasound image (i.e., the ultrasound image) (see, for example, [link to article]). Figure 13 In one embodiment, the image of the location of the text information 1230 (as shown) can be acquired. In another embodiment, the text information 1230 can be recognized by text recognition in the ultrasound image to obtain the character information of the text information 1230. Then, the endoscope host 60 superimposes the text information 1230 onto the local ultrasound image to obtain the second target image. In one embodiment, the image of the location of the text information 1230 can be extracted from the ultrasound image and superimposed on the local ultrasound image. In another embodiment, the character information after text recognition can be superimposed on the local ultrasound image, which can improve the display effect of the target tissue image. This allows medical staff to observe the target tissue in the ultrasound image more clearly without losing the acquisition of text information in the ultrasound detection while displaying the same screen in the display area 710.

[0223] In one embodiment of this application, after obtaining the first target image and the second target image, but before displaying the first target image and the second target image, adaptive adjustment of the display can also be performed on the first target image and the second target image. See also... Figure 15 As shown, the adaptive adjustment of the display includes the following steps:

[0224] Step 1501: Obtain the first display parameters of the endoscope display partition and the second display parameters of the ultrasound image display partition. The first display parameters include the first target resolution of the endoscope display partition, and the second display parameters include the second target resolution of the ultrasound image display partition.

[0225] In one embodiment of this step, the first display parameter includes a first target resolution of the endoscope display partition 720, and the second display parameter includes a second target resolution of the ultrasound image display partition 730. The first and second target resolutions differ in different simultaneous display modes.

[0226] Step 1502: Scaling and / or cropping the first target image according to the first target resolution, wherein the resolution of the first target image matches the first target resolution.

[0227] In this step, the endoscope host 60 performs scaling and / or cropping processing on the first target image according to the first target resolution, so that the resolution of the scaled and / or cropped first target image matches the first target resolution. Then, the endoscope host 60 displays the scaled and / or cropped first target image in the endoscope display partition 720.

[0228] Step 1503: Scaling and / or cropping the second target image according to the second target resolution, wherein the resolution of the second target image matches the second target resolution.

[0229] In this step, the endoscope host 60 performs scaling and / or cropping processing on the second target image according to the second target resolution, so that the resolution of the scaled and / or cropped second target image matches the second target resolution. Then, the endoscope host 60 displays the scaled and / or cropped second target image in the ultrasound image display partition 730.

[0230] See Figure 16 As shown, the scaling process performed on the first target image and / or the second target image in steps 1502 and 1503 above includes the following steps:

[0231] Step 1601: Obtain the image to be processed and determine the original resolution of the image to be processed.

[0232] In one embodiment, the image to be processed is the first target image, the second target image, a cropped partial first target image, or a cropped partial second target image. The endoscope host 60 can analyze the first target image or the second target image to obtain the original resolution of the first target image and the second target image.

[0233] In another embodiment, the image to be processed can also be an image processed by the endoscope host 60. For example, the endoscope host 60 can process the first target image according to the first display parameters and then obtain its original resolution. For example, the endoscope host 60 can crop the endoscope image according to the first display parameters to obtain a partial first target image and determine the original resolution of the partial first target image. Correspondingly, the endoscope host 60 can crop the second target image according to the second display parameters to obtain a partial second target image and determine the original resolution of the partial second target image.

[0234] In one embodiment, besides automatically analyzing the original resolution of the image to be processed, the original resolution of the image to be processed can also be input or selected by the user. Alternatively, the endoscope host 60 can receive a setting command from the user to set the original resolution of the image to be processed. In another embodiment, the original resolution of the image to be processed can also be sent by a host computer (e.g., a server) according to a communication protocol. In yet another embodiment, the original resolution of the image to be processed can also be a default setting. In one embodiment, the original resolution of the ultrasound image can also be sent by the ultrasound device 200 to the endoscope host 60.

[0235] Step 1602: Determine the target scaling factor based on the target resolution and the original resolution.

[0236] In this step, the target scaling factor is the ratio of the target resolution to the original resolution. For example, if the original resolution of the first target image is R1, then the corresponding target resolution is the first target resolution R1' of the endoscopic display partition 720, and the corresponding target scaling factor L1 = R1' / R1. As another example, if the original resolution of the second target image is R2, then the corresponding target resolution is the second target resolution R2' of the ultrasound image display partition 730, and the corresponding target scaling factor L2 = R2' / R2. For example, if the resolution of display area 710 is 3840*2160, the resolution of each display partition can be determined according to the resolution of the image to be processed and the screen display mode. For example, if the resolution of the second target image is 1590*920, then different resolution images can be output under different scaling factors. For example, when L2 = 0.8, the resolution of the second target image output is 1280*710; when L2 = 1.61, the resolution of the second target image output is 2560*1481; and when L2 = 2.35, the resolution of the second target image output is 3732*2160.

[0237] Step 1603: Scale the image to be processed according to the target scaling factor to obtain a scaled target image, wherein the target image is either the first target image or the second target image.

[0238] In this step, the endoscope host 60 scales the image to be processed according to the target scaling factor to obtain the scaled target image. For example, the endoscope host 60 scales the first target image by a factor of L1 according to the target scaling factor L1 corresponding to the first target resolution and displays it in the endoscope display area 720. The endoscope host 60 scales the second target image by a factor of L2 according to the target scaling factor L2 corresponding to the second target resolution and displays it in the ultrasound image display area 730.

[0239] In one embodiment, see Figure 17 As shown, the cropping process performed on the first target image and / or the second target image in steps 1502 and 1503 above includes the following steps:

[0240] Step 1701: Obtain the target resolution and the image to be processed.

[0241] In this step, the image to be processed can be either the first target image or the second target image mentioned above, and the target resolution can be either the first target resolution or the second target resolution mentioned above.

[0242] Step 1702: Determine the cropping region in the image to be processed based on the aspect ratio of the target resolution.

[0243] In this step, the endoscope host 60 first determines the cropping region in the image to be processed based on the aspect ratio of the target resolution. For example, the cropping region is determined in the first target image based on the aspect ratio of a first target resolution, and / or in the second target image based on the aspect ratio of a second target resolution. In one embodiment, the aspect ratio of the cropping region is the same as the aspect ratio of the display partition within the display area 710. In another embodiment, the aspect ratio of the cropping region is preset or set according to the on-screen display mode.

[0244] Step 1703: Crop the target image from the image to be processed according to the cropped area.

[0245] In this step, the endoscope host 60 crops the target image from the image to be processed according to the determined cropping area. For example, it performs cropping processing on the image to be processed according to the cropping area and saves the target image. For example, the endoscope host 60 crops the first target image according to the first cropping area to obtain an image for display in the endoscope display section 720, and crops the second target image according to the second cropping area to obtain an image for display in the ultrasound image display section 730.

[0246] In one embodiment, see Figure 18As shown, the cropping process performed on the first target image and / or the second target image in steps 1502 and 1503 above includes the following steps:

[0247] Step 1801: Obtain the target resolution and the image to be processed.

[0248] In this step, the image to be processed acquired by the endoscope host 60 can be either the first target image or the second target image mentioned above, and the target resolution can be either the first target resolution or the second target resolution mentioned above.

[0249] Step 1802: Determine a cropping region in the image to be processed according to the aspect ratio of the target resolution. The aspect ratio of the cropping region is the same as that of the target resolution and the cropping region covers the effective area of ​​the image to be processed.

[0250] In one embodiment, the endoscope host 60 can determine the cropping region based on the aspect ratio of the target resolution and the effective area of ​​the image to be processed, that is, to ensure that the size of the cropping region can cover the effective area of ​​the image to be processed. For the first target image, in one embodiment, the endoscope host 60 can identify the position of surgical instruments in the first target image and determine the effective area of ​​the endoscope image based on the position of the surgical instruments and the first target resolution. For example, the effective area of ​​the surgical instruments (e.g., scalpels) in the first target image (e.g., the pixel area occupied by the scalpels in the first target image) can be identified by image recognition technology. Then, using this pixel area as a reference (e.g., using the center of the pixel area as a reference) and the aspect ratio of the cropping region determined by the first target resolution, the position and size of the cropping region are determined so that the cropping region can cover the effective area in the first target image (e.g., cover the pixel area of ​​the scalpels).

[0251] In another embodiment, the endoscope host 60 can determine the endoscope image region of the endoscope image in the first target image as the effective region, and determine the aspect ratio and size of the cropped region according to the first target resolution, so that it can cover the endoscope image region in the first target image. In one embodiment, the endoscope image is the field of view circle in the first target image. In the first target image, except for the field of view circle in the central area of ​​the captured image, there is no image in the rest of the image. The endoscope host 60 can determine the endoscope image region (i.e., the field of view circle) in the endoscope image according to the characteristics of the field of view circle, and use the endoscope image region as a reference (e.g., the center of the field of view circle as a reference), and set the size of the cropped region according to the aspect ratio of the first target resolution, so that it can cover the endoscope image region. For example, if the cropped region is rectangular, the field of view circle is inside the rectangle, such as the inscribed circle of the cropped region.

[0252] In another embodiment, the endoscope host 60 can acquire the valid region selection information of the endoscope image input by the user, and determine the valid region of the first target image based on the valid region selection information of the endoscope image. In this step, the user can select the valid region of the endoscope image in the first target image, that is, the user inputs the valid region selection information to the endoscope host 60, and the endoscope host 60 determines the valid region of the first target image based on the valid region selection information of the endoscope image input by the user.

[0253] For the second target image, the target tissue obtained from the ultrasound image can be obtained using the relevant embodiments of step 406 described above as the effective region of the second target image. The aspect ratio and size of the cropped region are determined based on the resolution of the second target image, ensuring that the cropped region covers the effective region (e.g., the target tissue) in the second target image. For example, see... Figure 12 As shown, if the boundary of the target tissue image 1220 is circular or irregular, the endoscope host 60 uses the target tissue image 1220 as a reference (e.g., the center of the target tissue image 1220 as a reference) and sets the size of the cropping area according to the aspect ratio of the second target resolution, so that it can cover the target tissue image 1220. For example, if the cropping area is rectangular, the target tissue image 1220 is located inside the rectangle.

[0254] In another embodiment, the endoscope host 60 can acquire selection information of the effective ultrasound region input by the user, determine the effective ultrasound region based on the selection information, and determine a second cropping region in the second target image based on the aspect ratio of the second target resolution and / or the effective ultrasound region, wherein the second cropping region covers the effective ultrasound region.

[0255] In another embodiment, the endoscope host 60 performs a global pixel scan on the first target image and / or the second target image to find high-density pixel clusters in the images. Then, the endoscope host 60 determines the effective area of ​​the first target image and / or the second target image based on the high-density pixel clusters.

[0256] The above embodiments are merely illustrative examples of how to obtain the effective region of the image to be processed. In practice, other recognition methods can also be used to identify and process the effective region of the image to be processed.

[0257] Step 1803: Crop the target image from the image to be processed according to the cropped area, wherein the target image is the first target image or the second target image, and the effective area is the endoscope effective area or the ultrasound effective area.

[0258] In this step, the endoscope host 60 crops the image to be processed based on the effective area obtained in step 1802 to obtain an image for display in the display area 710. For example, the endoscope host 60 crops the first target image based on the effective area of ​​the endoscope image to obtain an image for display in the endoscope display partition 720, and the endoscope host 60 crops the second target image based on the effective area of ​​the second target image to obtain an image for display in the ultrasound image display partition 730.

[0259] In one embodiment, see Figure 19 As shown, in steps 1502 and 1503 above, scaling and cropping the first target image includes the following steps:

[0260] Step 1901: Determine a first cropping region in the first target image based on the aspect ratio of the first target resolution and / or the effective area of ​​the endoscope, wherein the first cropping region covers the effective area of ​​the endoscope.

[0261] In this step, the endoscope host 60 can determine the first cropping region in the first target image based on the aspect ratio of the first target resolution of the endoscope display partition 720. In another embodiment, the endoscope host 60 can determine the first cropping region in the first target image based on the effective region obtained by image recognition of the first target image. In yet another embodiment, the endoscope host 60 can determine the first cropping region in the endoscope image based on the effective region obtained after image recognition of the first target image, combined with the aspect ratio of the first target resolution. Specific implementation methods for this step can be found in the description of any of the embodiments in steps 1702 and 1802 above, and will not be repeated here.

[0262] Step 1902: Crop the first target image from the first target image according to the first cropping region to obtain the cropped first target image.

[0263] In this step, the endoscope host 60 determines a local endoscopic image in the first target image based on the first interception region.

[0264] Step 1903: Obtain the first resolution of the first target image, and determine the first scaling factor based on the ratio of the first target resolution to the first resolution.

[0265] In this step, the endoscope host 60 determines the first resolution R1 of the local endoscope image and the first target resolution R1' of the endoscope display partition 720, and determines the first scaling factor L1 = R1' / R1.

[0266] Step 1904: Scale the first target image according to the first scaling factor.

[0267] In this step, the endoscope host 60 scales the cropped first target image by a factor of L1 according to the target scaling factor L1 corresponding to the first target resolution, to obtain an image for display in the endoscope display partition 720. In one embodiment, the endoscope host 60 performs interpolation calculations on the cropped first target image based on the resolution of the effective area of ​​the first target image and the first scaling factor, so that the cropped first target image is scaled to obtain an image for display in the endoscope display partition 720.

[0268] In one embodiment, see Figure 20 As shown, in steps 1502 and 1503 above, scaling and cropping the second target image includes the following steps:

[0269] Step 2001: Determine a second cropping region in the second target image based on the aspect ratio of the second target resolution and / or the effective ultrasound region, wherein the second cropping region covers the effective ultrasound region.

[0270] In this step, the endoscope host 60 can determine the second cropping region in the second target image based on the aspect ratio of the second target resolution of the ultrasound image display partition 730. In another embodiment, the endoscope host 60 can determine the second cropping region in the second target image by performing image recognition on the second target image to obtain an effective region. In yet another embodiment, the endoscope host 60 can determine the second cropping region in the endoscopic image based on the effective region obtained after image recognition of the second target image, combined with the aspect ratio of the second target resolution.

[0271] The specific implementation of the identification of the effective region of the ultrasound image in this step can be found in the description of any of the embodiments in steps 1702 and 1802 above, and will not be repeated here. After obtaining the effective region of the ultrasound image, a second cropping region in the ultrasound image can be determined based on the effective region. At this time, the second cropping region covers the effective region of the ultrasound image.

[0272] Step 2002: Crop the second target image from the second target image according to the second cropping region to obtain the cropped second target image.

[0273] In this step, the endoscope host 60 determines the cropped second target image in the endoscope image based on the second cropping region.

[0274] Step 2003: Obtain the second resolution of the second target image, and determine the second scaling factor based on the ratio of the second target resolution to the second resolution.

[0275] In this step, the endoscope host 60 determines the second resolution R2 of the cropped second target image and the second target resolution R2' of the ultrasound image display partition 730, and determines the first scaling factor L2 = R2' / R2.

[0276] Step 2004: Scale the second target image according to the second scaling factor.

[0277] In this step, the endoscope host 60 scales the cropped second target image by a factor of L1 according to the target scaling factor L2 corresponding to the second target resolution to obtain the second target image. In one embodiment, the endoscope host 60 performs interpolation calculations on the cropped second target image according to the resolution of the ultrasound image display partition 730 and the second scaling factor, so that the cropped second target image is scaled to obtain an image for display in the ultrasound image display partition 730.

[0278] In one embodiment, after the endoscope host 60 performs adaptive adjustment on the display of the endoscopic image and the ultrasound image, it also performs color adjustment on the adaptively adjusted endoscopic image and / or ultrasound image. In one embodiment, see... Figure 21 As shown, step 1502 specifically includes:

[0279] Step 2101: Obtain color adjustment parameters.

[0280] In this step, the endoscope host 60 acquires color adjustment parameters. These color adjustment parameters can be preset, such as default settings or user-defined parameters, or they can be calculated based on features of the first target image or the endoscope image. The color adjustment parameters can include, but are not limited to, at least one combination of brightness, hue, contrast, grayscale, or saturation. In another embodiment, the endoscope host 60 can also acquire the color adjustment parameters from the second target image or the ultrasound image. For example, the endoscope host 60 can acquire the second target image and analyze its color parameters, such as collecting the brightness, hue, contrast, grayscale, or saturation of the second target image. In another embodiment, the endoscope host 60 can also calculate the color adjustment parameters by combining the color parameters of the first and second target images; for example, the average brightness, weighted brightness value, or maximum brightness value of the first and second target images can be used as the color adjustment parameters.

[0281] Step 2102: Scaling and / or cropping the first target image according to the first target resolution, and adjusting the color of the scaled and / or cropped first target image according to the color adjustment parameters.

[0282] In this step, the endoscope host 60 performs adaptive adjustment of the endoscope image display according to the first display parameters. The adaptive adjustment can be described in any embodiment of step 1502 above, and will not be repeated here. Then, the endoscope host 60 adjusts the color of the adaptively adjusted endoscope image according to the color adjustment parameters obtained in step 2101 above. Because the image displayed in the endoscope display partition 720 is color-adjusted according to the color adjustment parameters (e.g., adjusting brightness, hue, contrast, grayscale, or saturation), the display effect is better, making it easier for doctors to observe.

[0283] In another embodiment, the endoscope host 60 adjusts the colors of the adaptively adjusted endoscope image based on color adjustment parameters obtained from the second target image, thereby matching the colors of the first target image with those of the second target image. This ensures that the colors of the first and second target images displayed simultaneously in the display area 710 are consistent (e.g., consistent brightness, hue, contrast, grayscale, or saturation), improving the overall display effect. This makes the viewing experience more comfortable for doctors when observing the first and second target images simultaneously, preventing differences in display quality from affecting the efficiency of clinical surgery.

[0284] In another embodiment, see Figure 22 As shown, step 1503 specifically includes:

[0285] Step 2201: Obtain color adjustment parameters.

[0286] In this step, the endoscope host 60 acquires color adjustment parameters. These color adjustment parameters can be preset parameters, such as default settings or user-defined parameters, or they can be calculated based on the characteristics of the ultrasound image. The color adjustment parameters can include, but are not limited to, at least one combination of brightness, hue, contrast, grayscale, or saturation. In another embodiment, the endoscope host 60 can also acquire these color adjustment parameters from the endoscope image or the first target image. For example, the endoscope host 60 can acquire the first target image and analyze its color parameters, such as collecting the brightness, hue, contrast, grayscale, or saturation of the endoscope image.

[0287] Step 2202: Scaling and / or cropping the second target image according to the second target resolution, and adjusting the color of the scaled and / or cropped second target image according to the color adjustment parameters.

[0288] In this step, the endoscope host 60 performs adaptive adjustment of the second target image according to the second display parameters. The adaptive adjustment can be described in any embodiment of step 1503 above, and will not be repeated here. Then, the endoscope host 60 adjusts the color of the adaptively adjusted ultrasound image according to the color adjustment parameters obtained in step 2201 above. Because the second target image is color-adjusted according to the color adjustment parameters (e.g., adjusting brightness, hue, contrast, grayscale, or saturation), the display effect of the second target image in the ultrasound image display partition 730 is better, making it easier for doctors to observe.

[0289] In another embodiment, the endoscope host 60 adjusts the colors of the adaptively adjusted second target image based on color adjustment parameters obtained from the first target image, thereby matching the colors of the second target image with those of the first target image. This ensures that the colors of the first and second target images displayed simultaneously in the display area 710 are consistent (e.g., consistent brightness, hue, contrast, grayscale, or saturation), improving the overall display effect. This makes the viewing experience more comfortable for doctors when observing the first and second target images simultaneously, preventing differences in display quality from affecting the efficiency of clinical surgery.

[0290] In one embodiment of this application, color adjustment can also be performed on the first target image and the second target image before displaying the first target image and the second target image in step 407. See [link to relevant documentation]. Figure 23 and Figure 24 As shown, the color adjustment includes the following steps:

[0291] Step 2401: Perform Y-domain conversion on the first target image and the second target image to obtain the first target grayscale image and the second target grayscale image.

[0292] In one embodiment of this step, the first target image and the second target image are in the RGB domain. The first target image and the second target image are converted to the Y domain, that is, the first target image and the second target image are converted from the RGB domain to the Y domain.

[0293] Step 2402: Obtain the brightness information of the first target image and the brightness information of the second target image from the first target grayscale image and the second target grayscale image, respectively.

[0294] Step 2403: Adjust the brightness of the second target image according to the brightness information of the first target image and the brightness information of the second target image, so that the brightness of the second target image is consistent with the brightness of the first target image.

[0295] In one embodiment, see Figure 23 and Figure 24 As shown, the color adjustment also includes the following steps:

[0296] Step 2404: Obtain the tone information of the first target image and the tone information of the second target image from the first target image and the second target image, respectively;

[0297] Step 2405: Adjust the hue of the first target image and the hue of the second target image according to the hue information of the first target image and the hue information of the second target image, so that the hue of the first target image matches the hue of the second target image.

[0298] In one embodiment, before performing hue adjustment, the first and second target images after brightness adjustment are also converted back from the Y domain to the RGB domain.

[0299] See Figure 25 As shown, in one embodiment of this application, step 407 specifically includes the following steps:

[0300] Step 2501: Acquire multiple frames of the first target image and multiple frames of the second target image.

[0301] In this step, the endoscope host 60 performs adaptive adjustment on the display of multiple frames of the first target image to obtain multiple frames of the first target image. Correspondingly, the endoscope host 60 performs adaptive adjustment on the display of multiple frames of the second target image to obtain multiple frames of the second target image. See also... Figure 26 As shown, in one embodiment, the endoscope host 60 caches multiple frames of the first target image in the first cache area 2410, and the endoscope host 60 caches multiple frames of the second target image in the second cache area 2420.

[0302] In one embodiment, the endoscope host 60 caches multiple frames of the first target image corresponding to multiple frames of the endoscope images in the endoscope image data into the first cache area based on the timestamp and frame rate of each endoscope image in the endoscope image data. Additionally, the endoscope host 60 caches multiple frames of the second target image corresponding to the multiple frames of the ultrasound images in the ultrasound image data into the second cache area based on the timestamp and frame rate of each ultrasound image in the ultrasound image data.

[0303] In one embodiment, the first cache area 2610 and the second cache area 2620 include multi-level cache. By setting up multi-level cache, the video processing capability of the endoscope host 60 and the number of frames of the stored first target image and second target image can be improved.

[0304] Step 2502: After performing frame synchronization processing on multiple frames of the first target image and multiple frames of the second target image, the first target image is displayed in the endoscope display partition in the display area, and the second target image is displayed in the ultrasound display partition, wherein the frame rate and timing of the display of the first target image in the endoscope display partition and the display of the second target image in the ultrasound image display partition are synchronized.

[0305] In this step, the endoscope host 60 performs frame synchronization processing on multiple frames of the first target image and multiple frames of the second target image, that is, it configures the first target image and the second target image with corresponding time sequences together for output, so that the frame rate and time sequence of the first target image displayed in the endoscope display partition and the second target image displayed in the ultrasound image display partition are synchronized. In one embodiment, the endoscope image and ultrasound image acquired by the endoscope host 60 both contain time sequence information, such as the image capture time. In this way, the endoscope host 60 can read the first target image and the second target image with the same or similar time sequence (e.g., the same capture time or the capture time interval is less than a preset value) from the first buffer area 2410 and the second buffer area 2420 respectively for simultaneous display. In one embodiment, if the frame rates of the endoscopic image data and the ultrasound image data are inconsistent (e.g., the endoscopic image data is 120 frames per second, and the ultrasound image data is 60 frames per second), the endoscope host 60 can perform frame interpolation processing on the endoscopic image data and / or the ultrasound image data, that is, copying images with similar frame rates for interpolation or fusing images with similar frame rates for interpolation. Alternatively, the endoscope host 60 can perform frame extraction processing on the endoscopic image data and / or the ultrasound image data, for example, intermittently extracting frames from the endoscopic image data with the higher frame rate to obtain an image of 60 frames per second. In this embodiment, the display 70 can support multiple frame rates, such as 120Hz / 60Hz / 50Hz / 30Hz, which can be set or selected as needed or in a screen-sharing mode.

[0306] Step 2503: Based on the video output format of the display, adjust the time order of multiple frames of the first target image in the first buffer area and multiple frames of the second target image in the second buffer area according to the video output timing sequence, so as to synchronize the frame rate and timing of the first target image and the second target image.

[0307] In one embodiment of this step, the endoscope host 60, based on the video output format of the display, adjusts the time order of multiple frames of the first target image in the first buffer area and multiple frames of the second target image in the second buffer area according to the video output timing sequence, so that the frame rate and timing of the first target image and the second target image are synchronized, and finally synthesized into a video stream, which is sent to the display 70 through the communication cable 80 for simultaneous display of the first target image and the second target image. See also... Figure 27 As shown, the specific steps include:

[0308] Step 2701: Based on the video output timing and video output format of the display, read multiple frames of the first target image from the first buffer area and read multiple frames of the second target image from the second buffer area.

[0309] In this step, the video output timing includes the video output timestamp, and the video output format includes the video output frame rate.

[0310] Step 2702: Based on the clock signal of the endoscope system, each frame of the first target image and each frame of the second target image read are stamped with the video output timestamp and cached in the output storage area of ​​the endoscope system.

[0311] In this step, the endoscope host 60 uses the same local clock signal to stamp the first target image and each frame of the second target image with the video output timestamp and caches them in the output storage area.

[0312] Step 2703: Read multiple frames of the first target image and multiple frames of the second target image with the same time sequence from the output storage area, wherein the video frame rate synthesized from the multiple frames of the first target image and the multiple frames of the second target image is the same as the video output frame rate.

[0313] In this step, the endoscope host 60 reads multiple frames of the first target image and multiple frames of the second target image that are read in the same time sequence from the output storage area. The video frame rate synthesized from the multiple frames of the first target image and the multiple frames of the second target image is the same as the video output frame rate. This allows the images displayed in the endoscope display area 720 and the ultrasound image display area 730 to be synchronized, resulting in a better simultaneous display effect.

[0314] See Figure 28 The diagram shown illustrates the execution of an image display method for an endoscope system according to an embodiment of this application, including the following steps:

[0315] The endoscope host 60 obtains endoscope image data from the camera 50 of the endoscope 30, wherein the endoscope image data includes multiple frames of endoscope images, and the endoscope host 60 stores the obtained multiple frames of endoscope images as the first target image in the first cache area 2410.

[0316] The endoscope host 60 acquires ultrasound image data from the ultrasound device 200, wherein the ultrasound image data includes multiple frames of ultrasound images. The endoscope host can determine the frame rate and original resolution of the ultrasound images (i.e., the second resolution in the above embodiment) based on the ultrasound image data. Then, the endoscope host 60 stores the acquired multiple frames of ultrasound images in the second cache area 2420.

[0317] The endoscope host 60 performs edge detection on the ultrasound image in the second buffer area 2420 to identify the fan-shaped region in the ultrasound image and obtain the fan-shaped region image. Then, it performs edge detection on the fan-shaped region image again to identify the target tissue in the fan-shaped region image. Based on the target tissue, it determines the effective ultrasound area and obtains the second target image of the effective ultrasound area.

[0318] The endoscope host 60 performs image processing on a first target image according to a first target resolution of the endoscope display partition 720, and performs image processing on a second target image according to a second target resolution of the ultrasound image display partition 730, based on the same-screen display mode. The image processing may be the cropping and / or scaling processing described in any embodiment of this application.

[0319] The endoscope host 60 acquires the brightness parameters of the endoscope image and adjusts the brightness of the cropped first target image according to the brightness parameters so that it matches the brightness of the endoscope image.

[0320] The endoscope host 60 acquires the hue parameters of the endoscope image and the second target image, and adjusts the hue of the first target image and the second target image after brightness matching according to the hue parameters to obtain the adjusted first target image and the adjusted second target image.

[0321] After performing frame synchronization processing on multiple frames of first target images and multiple frames of second target images, the endoscope host 60 stitches the first target images and second target images with the same time sequence in the display area 710 and displays them on the same screen through the monitor 70.

[0322] See Figure 29 As shown in the figure, this application provides an image display method for an endoscope system, which can be performed by... Figures 1 to 3 The endoscope system 1000 shown executes an image display method including, but not limited to, the following steps:

[0323] Step 2901: Acquire endoscopic image data, which includes multiple frames of endoscopic images.

[0324] For details of this step, please refer to any embodiment of step 401 above, which will not be repeated here.

[0325] Step 2902: Acquire at least one ultrasound image data from at least one ultrasound device communicatively connected to the endoscope system, the ultrasound image data comprising multiple frames of ultrasound images.

[0326] For details of this step, please refer to any embodiment of step 402 above, which will not be repeated here.

[0327] Step 2903: According to the co-screen display mode, determine an endoscope display partition for co-screening the endoscope image and at least one ultrasound image display partition for co-screening the ultrasound image from the display area of ​​the display, wherein the endoscope display partition and the ultrasound image display partition are co-screened in the display area.

[0328] For details in this step, please refer to any embodiment of step 403 above, which will not be repeated here.

[0329] Step 2904: Obtain multiple frames of the endoscope image data from the endoscope image data, and obtain multiple frames of the ultrasound image data from the ultrasound image data.

[0330] For details in this step, please refer to any embodiment of step 404 above, which will not be repeated here.

[0331] Step 2905: Obtain a first target image corresponding to the endoscope image based on the endoscope image data.

[0332] For details in this step, please refer to any embodiment of step 405 above, which will not be repeated here.

[0333] Step 2906: Determine the effective area of ​​ultrasound based on the ultrasound image to obtain a second target image of the effective area of ​​ultrasound.

[0334] In one embodiment of this step, the endoscope host 60 acquires effective region selection information of the ultrasound image input by the user, the effective region selection information being used to indicate the effective region of the ultrasound image. Then, the endoscope host 60 determines a second cropping region in the ultrasound image based on the aspect ratio of the second target resolution, wherein the second cropping region covers the effective region of the ultrasound image.

[0335] In another embodiment, the endoscope host 60 can determine the effective ultrasound region of the ultrasound image through image recognition. For example, the endoscope host 60 can perform a global pixel scan on the ultrasound image to find high-density pixel clusters in the external image. Then, the endoscope host 60 determines the effective region of the ultrasound image based on the high-density pixel clusters. For example, image recognition processing can also be performed based on the characteristics of the ultrasound image. For instance, if the edge of the ultrasound image is a fan-shaped region, edge detection can be performed on the ultrasound image, and the fan-shaped region in the ultrasound image can be used as the effective region to identify the target tissue in the ultrasound image. Based on the target tissue, the effective ultrasound region is determined, resulting in a second target image of the effective ultrasound region. Alternatively, further recognition processing can be performed on the fan-shaped region in the ultrasound image to determine the effective region image. For example, edge detection can be performed twice on the ultrasound image to determine the effective region image. Or, after first performing edge detection on the ultrasound image, image recognition (e.g., finding high-density regions as described above) can be performed on the image within the edge to determine the target tissue and its effective ultrasound region, and then a second target image of the effective ultrasound region is obtained.

[0336] In one embodiment of this step, see any embodiment of step 406 above, which will not be repeated here.

[0337] Step 2907: Display the first target image in the endoscope display partition of the display area, and display the second target image in the ultrasound image display partition.

[0338] For details of this step, please refer to any embodiment of step 406 above, which will not be repeated here.

[0339] In one embodiment, step 2906 above includes the following steps:

[0340] The effective area of ​​ultrasound is determined based on the target tissue, and a local ultrasound image of the effective area is obtained.

[0341] Text information from the ultrasound image is obtained, and a second target image of the effective ultrasound region is obtained based on the text information and the local ultrasound image.

[0342] For details of the above steps, please refer to any embodiment of step 1402 above, which will not be repeated here.

[0343] This application provides an endoscope system, including:

[0344] Endoscope;

[0345] A display, the display including a display area for displaying images;

[0346] The endoscope host is communicatively connected to both the endoscope and the display. The endoscope host is also used to execute the image display method of the endoscope system in any of the above embodiments.

[0347] On the other hand, embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the image display method of the endoscope system of any of the above embodiments.

[0348] On the other hand, embodiments of this application also provide a computer-readable storage medium storing a computer program, which is executed by a processor to implement the image display method of the endoscope system of any of the above embodiments.

[0349] On the other hand, embodiments of this application also provide a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform an image display method implementing the endoscope system of any of the above embodiments.

[0350] It can be used in various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and driver assistance.

[0351] It is understood that although the steps in the flowcharts of this application are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this embodiment, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0352] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate to describe embodiments of this application, for example, those that can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0353] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0354] It should be understood that in the description of the embodiments of this application, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0355] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0356] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0357] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0358] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0359] It should also be understood that the various implementation methods provided in this application can be combined arbitrarily to achieve different technical effects.

[0360] The above provides a detailed description of the preferred embodiments of this application. However, this application is not limited to the above-described embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. An image display method for an endoscope system, the endoscope system comprising an endoscope host and a display, characterized in that, The image display method includes: Acquire endoscopic image data, which includes multiple frames of endoscopic images; At least one ultrasound image data is acquired from at least one ultrasound device that is communicatively connected to the endoscope system, the ultrasound image data comprising multiple frames of ultrasound images; According to the screen-sharing mode, an endoscope display partition for displaying the endoscope image on the same screen and at least one ultrasound image display partition for displaying the ultrasound image on the same screen are determined from the display area of ​​the display, wherein the endoscope display partition and the ultrasound image display partition are displayed on the same screen in the display area; Multiple frames of the endoscope image are obtained from the endoscope image data, and multiple frames of the ultrasound image are obtained from the ultrasound image data; Based on the endoscopic image data, a first target image corresponding to the endoscopic image is obtained; Edge detection is performed on the ultrasound image to identify fan-shaped regions in the ultrasound image, resulting in a fan-shaped region image; edge detection is performed on the fan-shaped region image to identify target tissue in the fan-shaped region image; the effective ultrasound region is determined based on the target tissue, resulting in a second target image of the effective ultrasound region; The first target image is adaptively adjusted for display, wherein a first display parameter of the endoscope display partition is obtained, the first display parameter includes a first target resolution of the endoscope display partition, and the first target image is scaled and / or cropped according to the first target resolution, wherein the resolution of the first target image matches the first target resolution; The second target image is adaptively adjusted for display, wherein a second display parameter of the ultrasound image display partition is obtained, the second display parameter includes a second target resolution of the ultrasound image display partition, and the second target image is scaled and / or cropped according to the second target resolution, wherein the resolution of the second target image matches the second target resolution; The first target image after the display adaptive adjustment is displayed in the endoscope display partition of the display area, and the second target image after the display adaptive adjustment is displayed in the ultrasound image display partition.

2. The image display method according to claim 1, characterized in that, The step of obtaining a first target image corresponding to the endoscopic image based on the endoscopic image data includes at least one of the following: Image recognition is performed on the endoscopic image to determine the position of the surgical instruments in the endoscopic image, and the effective area of ​​the endoscope is determined based on the position of the surgical instruments to obtain a first target image of the effective area of ​​the endoscope; Image recognition is performed on the endoscopic image to determine the surgical field region of the endoscopic image, and the effective region of the endoscope is determined based on the surgical field region to obtain a first target image of the effective region of the endoscope; Obtain the endoscope effective area selection information input by the user, determine the endoscope effective area based on the endoscope effective area selection information, and obtain the first target image of the endoscope effective area.

3. The image display method according to claim 1, characterized in that, The scaling process specifically includes: Acquire the image to be processed and determine the original resolution of the image to be processed, wherein the image to be processed is the first target image, the second target image, the cropped partial first target image, or the cropped partial second target image; A target scaling factor is determined based on the target resolution and the original resolution, wherein the target scaling factor is the ratio of the target resolution to the original resolution, and the target resolution is either the first target resolution or the second target resolution; The image to be processed is scaled according to the target scaling factor to obtain a scaled target image, wherein the target image is either the first target image or the second target image.

4. The image display method according to claim 1 or 3, characterized in that, The trimming process specifically includes at least one of the following: A target resolution and an image to be processed are obtained, wherein the target resolution is either the first target resolution or the second target resolution, and the image to be processed is either the first target image or the second target image; a cropping region is determined in the image to be processed according to the aspect ratio of the target resolution, and a target image is obtained by cropping the image to be processed according to the cropping region, wherein the target image is either the first target image or the second target image; Alternatively, a target resolution and an image to be processed are obtained, wherein the target resolution is either the first target resolution or the second target resolution, and the image to be processed is either the first target image or the second target image; a cropping region is determined in the image to be processed based on the aspect ratio of the target resolution, wherein the aspect ratio of the cropping region is the same as the aspect ratio of the target resolution and the cropping region covers the effective region of the image to be processed; a target image is obtained by cropping the image to be processed based on the cropping region, wherein the target image is either the first target image or the second target image, and the effective region is either the effective region of an endoscope or the effective region of an ultrasound.

5. The image display method according to claim 3, characterized in that, The scaling and / or cropping of the first target image according to the first target resolution includes one of the following: After cropping the first target image, the cropped first target image is scaled according to the first target resolution. Alternatively, the first target image can be scaled according to the first target resolution, and then the scaled first target image can be cropped. The scaling and / or cropping of the second target image according to the second target resolution includes one of the following: After cropping the second target image, the cropped second target image is scaled according to the resolution of the second target. Alternatively, the second target image can be scaled according to the second target resolution, and then the scaled second target image can be cropped.

6. The image display method according to claim 3, characterized in that, The scaling and / or cropping of the first target image according to the first target resolution includes: Obtain color adjustment parameters; The first target image is scaled and / or cropped according to the first target resolution, and the color of the scaled and / or cropped first target image is adjusted according to the color adjustment parameters.

7. The image display method according to claim 6, characterized in that, The acquisition of color adjustment parameters includes one of the following: Obtain preset color adjustment parameters; Alternatively, color adjustment parameters can be obtained from the second target image.

8. The image display method according to claim 3, characterized in that, The scaling and / or cropping of the second target image according to the second target resolution includes: Obtain color adjustment parameters; The second target image is scaled and / or cropped according to the second target resolution, and the color of the scaled and / or cropped second target image is adjusted according to the color adjustment parameters.

9. The image display method according to claim 8, characterized in that, The acquisition of color adjustment parameters includes one of the following: Obtain preset color adjustment parameters; Alternatively, color adjustment parameters can be obtained from the first target image.

10. The image display method according to any one of claims 6 to 9, characterized in that, The color adjustment parameters include at least one of the following: brightness; tone; Contrast; Grayscale; Saturation.

11. The image display method according to claim 1, characterized in that, Displaying the first target image in the endoscopic display zone of the display area and displaying the second target image in the ultrasound image display zone includes: The first target image is displayed in the endoscope display section of the display area; The second target image is displayed in the ultrasound image display zone of the display area; The display area is filled with other areas besides the endoscope display area and the ultrasound image display area.

12. The image display method according to claim 1, characterized in that, The step of determining, according to the co-screen display mode, an endoscope display partition for co-screening the endoscopic image and at least one ultrasound image display partition for co-screening the ultrasound image from the display area of ​​the monitor includes: Obtain the screen mirroring mode selection information preset by the display and / or input by the user, wherein the screen mirroring mode selection information is used to instruct the display to switch the screen mirroring mode; The display area is divided according to the co-screen display mode to obtain the endoscope display partition for co-screen display of the endoscope image and at least one ultrasound image display partition for co-screen display of the ultrasound image.

13. The image display method according to claim 1, characterized in that, It includes multiple screen-sharing display modes, and the arrangement of the endoscope display zone and the ultrasound image display zone on the same screen is different in different screen-sharing display modes.

14. The image display method according to claim 1 or 13, characterized in that, The resolution of the endoscopic display zone and the ultrasound image display zone may be the same or different.

15. The image display method according to claim 1, characterized in that, The step of determining the effective area of ​​ultrasound based on the target tissue and obtaining a second target image of the effective area of ​​ultrasound includes: Based on the target tissue, the effective area of ​​ultrasound is determined, and a local ultrasound image of the effective area of ​​ultrasound is obtained; Text information from the ultrasound image is obtained, and a second target image of the effective ultrasound region is obtained based on the text information and the local ultrasound image.

16. An image display method for an endoscope system, the endoscope system comprising an endoscope host and a display, characterized in that, The image display method includes: Acquire endoscopic image data, which includes multiple frames of endoscopic images; At least one ultrasound image data is acquired from at least one ultrasound device that is communicatively connected to the endoscope system, the ultrasound image data comprising multiple frames of ultrasound images; According to the co-screen display mode, an endoscope display partition for co-screening the endoscope image and at least one ultrasound image display partition for co-screening the ultrasound image are determined from the display area of ​​the display, wherein the endoscope display partition and the ultrasound image display partition are co-screened in the display area; Multiple frames of the endoscope image are obtained from the endoscope image data, and multiple frames of the ultrasound image are obtained from the ultrasound image data; Based on the endoscopic image data, a first target image corresponding to the endoscopic image is obtained; Based on the ultrasound image, the effective ultrasound region is determined, and a second target image of the effective ultrasound region is obtained. The first target image is adaptively adjusted for display, wherein a first display parameter of the endoscope display partition is obtained, the first display parameter includes a first target resolution of the endoscope display partition, and the first target image is scaled and / or cropped according to the first target resolution, wherein the resolution of the first target image matches the first target resolution; The second target image is adaptively adjusted for display, wherein a second display parameter of the ultrasound image display partition is obtained, the second display parameter includes a second target resolution of the ultrasound image display partition, and the second target image is scaled and / or cropped according to the second target resolution, wherein the resolution of the second target image matches the second target resolution; The first target image after the display adaptive adjustment is displayed in the endoscope display partition of the display area, and the second target image after the display adaptive adjustment is displayed in the ultrasound image display partition.

17. The image display method according to claim 16, characterized in that, The step of determining the effective ultrasound region based on the ultrasound image and obtaining a second target image of the effective ultrasound region includes: Edge detection is performed on the ultrasound image to identify target tissue in the ultrasound image; The effective area of ​​ultrasound is determined based on the target tissue, and a second target image of the effective area of ​​ultrasound is obtained.

18. The image display method according to claim 17, characterized in that, The step of determining the effective area of ​​ultrasound based on the target tissue and obtaining a second target image of the effective area of ​​ultrasound includes: Based on the target tissue, the effective area of ​​ultrasound is determined, and a local ultrasound image of the effective area of ​​ultrasound is obtained; Text information from the ultrasound image is obtained, and a second target image of the effective ultrasound region is obtained based on the text information and the local ultrasound image.

19. An endoscope system, characterized in that, include: Endoscope; A display, the display including a display area for displaying images; An endoscope host is communicatively connected to both the endoscope and the display, and the endoscope host is also used to execute the image display method of the endoscope system according to any one of claims 1 to 18.

Images