Image generation method and apparatus, electronic device, endoscope system, and storage medium

Abstract

Embodiments of the present application provide an image generation method and device, electronic equipment, an endoscope system and a storage medium. The method comprises: in response to a video recording instruction, recording a target video, the target video comprising a plurality of endoscope images; during the recording of the target video, in response to an image interception instruction, intercepting a target endoscope image corresponding to the image interception instruction from the target video; and generating a marked image based on the target endoscope image, the marked image comprising a thumbnail image corresponding to the target endoscope image and image identification information, the image identification information comprising a timestamp of the corresponding target endoscope image. When the marked image is used for display, the scheme can not only improve the viewing efficiency of the user, but also make the user interface more tidy and the user experience better.

Description

Technical Field

[0001] This invention relates to the field of endoscopy technology, and more specifically, to an image generation method, an image generation device, an electronic device, an endoscopy system, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the continuous development of endoscopic technology, endoscopic examinations and treatments are becoming increasingly widespread. For example, colonoscopy is one of the important techniques for diagnosing and treating lower gastrointestinal diseases such as colorectal polyps and tumors.

[0003] In endoscopic examinations, patients primarily rely on images captured by the endoscopic lens for diagnosis. Most endoscopes possess image acquisition capabilities. During lengthy examinations, patients may need to mark one or more key time points for later review of images before and after those points. However, currently, there is no suitable method for marking these key time points. Summary of the Invention

[0004] The present invention was proposed in view of the above-mentioned problems. The present invention provides an image generation method, an image generation apparatus, an electronic device, an endoscope system, a computer-readable storage medium, and a computer program product.

[0005] According to a first aspect of the present invention, an image generation method is provided, the method comprising: recording a target video in response to a video recording instruction, the target video including multiple endoscopic images; during the recording of the target video, extracting a target endoscopic image corresponding to the image extraction instruction from the target video in response to an image extraction instruction; generating a marker image based on the target endoscopic image, the marker image including a thumbnail image corresponding to the target endoscopic image and image identification information, the image identification information including a timestamp of the corresponding target endoscopic image.

[0006] For example, the labeled image includes multiple thumbnails corresponding to target endoscope images and image identification information. The multiple thumbnails are arranged and displayed according to preset rules, and each image identification information is displayed at a preset position in the corresponding thumbnail.

[0007] For example, the number of target videos is at least two, and the image identification information also includes the name and / or recording time and / or recording order of the target video to which the corresponding target endoscope image belongs.

[0008] For example, the image capture instruction is generated based on the user's image capture operation, which may include pressing a preset button on the endoscope or a preset button on the input device.

[0009] For example, the method further includes: during the recording of the target video, using a first target recognition network to identify the target object in the target video; when the target object is identified, generating an image cropping instruction, wherein the target endoscope image is an endoscope image containing the target object.

[0010] For example, if the target endoscope image contains a target object, before generating a labeled image based on the target endoscope image, the method further includes: using a second target recognition network to sequentially identify the target object forward and backward in the target video, starting from the target endoscope image, until no target object can be identified; determining the identified endoscope image containing the target object as a labeled video segment corresponding to the target endoscope image; and in response to a thumbnail selection instruction for selecting any thumbnail image in the labeled image, determining and displaying the labeled video segment corresponding to the target endoscope image corresponding to the selected thumbnail image.

[0011] For example, the image identification information also includes the start and end times and / or duration of the video segment.

[0012] For example, after extracting the target endoscope image corresponding to the image extraction command from the target video in response to the image extraction command, the method further includes: determining whether a screenshot image or a target endoscope image is stored in a preset storage directory of a preset storage device, wherein the preset storage device is used to store the screenshot image or the target endoscope image corresponding to the target endoscope image, and the screenshot image is a target endoscope image with added image identification information; when the screenshot image or the target endoscope image is stored in the preset storage directory, outputting a first prompt message, the first prompt message being used to indicate that the target endoscope image extraction was successful; when the screenshot image or the target endoscope image is not stored in the preset storage directory, outputting a second prompt message, the second prompt message being used to indicate that the target endoscope image extraction failed.

[0013] For example, before generating a marker image based on the target endoscope image, the method further includes: storing the captured target endoscope image or screenshot image in a preset storage directory of a preset storage device, the preset storage device being used to store the screenshot image or target endoscope image corresponding to the target endoscope image, the screenshot image being the target endoscope image with added image identification information; after generating a marker image based on the target endoscope image, the method further includes: automatically deleting the target endoscope image from the preset storage device; or, outputting a third prompt message and deleting the target endoscope image from the preset storage device in response to a deletion command input by the user based on the third prompt message.

[0014] According to a second aspect of the present invention, an image generation apparatus is also provided, the apparatus comprising: a recording module, configured to record a target video in response to a video recording command, the target video including a plurality of endoscopic images; a cropping module, configured to, during the recording of the target video, in response to an image cropping command, crop a target endoscopic image corresponding to the image cropping command from the target video; and a first generation module, configured to generate a marker image based on the target endoscopic image, the marker image including a thumbnail image corresponding to the target endoscopic image and image identification information, the image identification information including a timestamp of the corresponding target endoscopic image.

[0015] According to a third aspect of the present invention, an electronic device is also provided, including a processor and a memory, wherein a computer program is stored in the memory, and the computer program instructions are executed by the processor to perform the image generation method described above.

[0016] According to a fourth aspect of the present invention, an endoscope system is also provided, including the electronic device described above.

[0017] According to a fifth aspect of the present invention, a computer-readable storage medium is also provided, storing a computer program / instructions, which, when executed, are used to perform the image generation method described above.

[0018] According to a sixth aspect of the present invention, a computer program product is also provided, comprising computer program instructions, which, when executed, are used to perform the image generation method described above.

[0019] According to embodiments of the present invention, an image generation method, image generation apparatus, electronic device, endoscope system, computer-readable storage medium, and computer program product can, in response to an image cropping instruction, crop a target endoscope image corresponding to the image cropping instruction from a recorded target video. Then, based on the target endoscope image, a marker image can be generated. This scheme, through the generated marker image, can merge thumbnails of each target endoscope image into a single marker image. This not only improves user viewing efficiency when the marker image is used for display but also makes the user interface cleaner and provides a better user experience. Furthermore, if there are multiple target endoscope images, the generated marker image facilitates comparison or unified observation of multiple endoscope images when displayed. In addition, by merging the thumbnails of each target endoscope image into a single marker image, storage space can be effectively saved when the generated marker image is stored, especially when there are a large number of target endoscope images.

[0020] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0021] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0022] Figure 1 A schematic flowchart of an image generation method according to an embodiment of the present invention is shown;

[0023] Figure 2 A schematic diagram of a marked image according to an embodiment of the present invention is shown;

[0024] Figure 3 A schematic diagram of a marker image according to another embodiment of the present invention is shown;

[0025] Figure 4 A schematic flowchart of an image generation apparatus according to an embodiment of the present invention is shown; and

[0026] Figure 5 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments described in the present invention, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0028] To address the aforementioned technical problems, this invention provides an image generation method. Exemplarily, this image generation method can be applied to any electronic device with image processing capabilities, including but not limited to: electronic devices included in an endoscope system, electronic devices communicatively connected to an endoscope system, or electronic devices independently of the endoscope system. When the electronic device is an electronic device included in or communicatively connected to an endoscope system, static images (hereinafter referred to as endoscope images) or dynamic videos (containing multiple frames of endoscope images) acquired by the image acquisition device within the endoscope system can be transmitted to the electronic device in real time for processing, or they can be first stored in a storage device included in the endoscope system and then retrieved and processed by the electronic device from the storage device. Exemplarily, when the electronic device is an electronic device independently of the endoscope system, the images or videos acquired by the image acquisition device of the endoscope system can be stored in any external storage device, and then retrieved and processed by the electronic device from the storage device via data transmission lines such as a bus, local area network, or the Internet. The image generation method can be specifically executed by a processing device (or processor) within the electronic device. Endoscopic systems can include, but are not limited to, gastroscopes, colonoscopes, laparoscopes, and chromoendoscopy. Endoscopic systems can be used to examine lesions such as inflammation, ulcers, and tumors in the digestive tract and tissues of the liver, gallbladder, and pancreas. Figure 1 A schematic flowchart of an image generation method 100 according to an embodiment of the present invention is shown, such as Figure 1 As shown, the image generation method 100 may include, but is not limited to, the following steps S110, S120 and S130.

[0029] Step S110: In response to a video recording command, a target video is recorded, which contains multiple endoscopic images.

[0030] For example, an endoscope system may include an endoscope body, an endoscope light source, etc. An image acquisition device may be mounted on the endoscope body. The image acquisition device may include components such as an imaging objective lens and an image sensor. Images or videos of a target site can be acquired using the image acquisition device. The target site may include any biological tissue suitable for endoscopic examination, such as the stomach, liver, or pancreatic duct. During the endoscopic examination, one or more static endoscopic images or a video segment can be acquired at the target site using the image acquisition device of the endoscope system. In the case of video acquisition, each video frame in the acquired video may correspond to a separate endoscopic image. The target video can be a video segment of any length acquired during the endoscopic examination. The target video may include at least two endoscopic images, i.e., at least two video frames. The method by which an electronic device acquires video acquired by an endoscope system has been described above and will not be repeated here. Optionally, the number of target videos may be one or more. For example, the electronic device may include an input device or be communicatively connected to an input device, and the video recording command may be input by a user through the input device. The input device can be any device with input functionality, including but not limited to push-button switches, touchpads, mice, keyboards, and touchscreens. A touchpad can be referred to as a control panel. For example, video recording instructions can also be automatically generated by an electronic device (specifically, a processor within the electronic device). For instance, when the electronic device is an electronic device included in an endoscope system or an electronic device communicatively connected to the endoscope system, the electronic device can specifically be communicatively connected to the endoscope body. A video recording switch can be provided on the endoscope body; when the user presses or clicks the video recording switch, the endoscope body can generate a video recording instruction and send it to the electronic device. As another example, the electronic device can include an input device such as a touchpad or touchscreen, through which the processor in the electronic device can obtain the user-inputted video recording instruction. When the input device is a touchscreen, a video recording control can be provided on the touchscreen. After the user clicks the video recording control, the touchscreen can output the corresponding video recording instruction to the processor in the electronic device. For example, the program of an electronic device can be pre-set to start recording a target video when preset conditions are met. The processor of the electronic device can automatically generate a video recording command when the preset conditions are met and perform video recording based on the video recording command. The preset conditions could be, for example, waiting for a preset time after the endoscope system is turned on, or the image brightness of the endoscope image acquired by the endoscope system reaching a preset brightness.

[0031] In step S120, during the recording of the target video, in response to the image capture command, the target endoscope image corresponding to the image capture command is captured from the target video.

[0032] For example, the image capture command can be input by the user through the aforementioned input device. Alternatively, the image capture command can be automatically generated by an electronic device (specifically, a processor within the electronic device). For instance, when the electronic device is an electronic device included in an endoscope system or an electronic device communicatively connected to an endoscope system, the electronic device can specifically be communicatively connected to the endoscope body. The endoscope body can be equipped with a screenshot button switch; when the user presses or clicks the screenshot button switch, the endoscope body can generate an image capture command and send it to the electronic device. As another example, the electronic device can include an input device such as a touchpad or touchscreen, through which the processor in the electronic device can obtain the image capture command input by the user. When the input device is a touchscreen, a screenshot control can be provided on the touchscreen. After the user clicks the screenshot control, the touchscreen can correspondingly output the image capture command to the processor in the electronic device. For example, the program of an electronic device can be pre-set to capture the x-th frame of a target video. During or after acquiring the target video, the processor of the electronic device can automatically generate an image capture instruction corresponding to the x-th frame and perform the image capture based on this instruction. x can be any integer value less than or equal to the total number of frames in the current target video. It can be understood that there is a correspondence between the image capture instruction and the endoscopic image. For example, the electronic device can include a display or be communicatively connected to a display. The electronic device can display each endoscopic image in the acquired target video on the display. When the user sees the endoscopic image to be captured, they can input the image capture instruction through the aforementioned input device. When the electronic device receives the image capture instruction, it can use the endoscopic image currently displayed on the display as the endoscopic image corresponding to the image capture instruction. In response to the image capture instruction, the target endoscopic image corresponding to the image capture instruction can be captured from the target video. For example, if the current image cropping command is to crop the 5th frame of the endoscope image from the target video, then in response to this command, the 5th frame of the endoscope image can be cropped from the target video. The cropped 5th frame of the endoscope image can be used as the target endoscope image. Based on different image cropping commands, the target endoscope image corresponding to each image cropping command can be cropped from the target video.

[0033] Step S130: Generate a labeled image based on the target endoscope image. The labeled image includes a thumbnail image corresponding to the target endoscope image and image identification information. The image identification information includes the timestamp of the corresponding target endoscope image.

[0034] For example, a marker image can be generated based on the captured target endoscopic image. The generated marker image can be output to the aforementioned display for display and / or stored in any storage device. The marker image may include a thumbnail image corresponding to the target endoscopic image and image identification information. The image identification information may include a timestamp of the corresponding target endoscopic image. The timestamp can be used to indicate the order of the target endoscopic image in the target video. The timestamp can also be understood as the moment corresponding to the target endoscopic image on the playback timeline of the target video. For example, the timestamp can be represented in the form of "hour:minute:second". For example, the timestamp of the current target endoscopic image can be represented as "00:00:16". Assuming that the time interval between adjacent endoscopic images in the target video is 1 second, then "00:00:16" can indicate that the current target endoscopic image is the 16th frame endoscopic image in the target video. It is understood that the above representation of the timestamp is merely exemplary, and the present invention does not limit it. For example, the timestamp can also be represented in the form of "m minutes n seconds". The values ​​of m and n can range from 0 to 59.

[0035] For example, in the labeled image, the image identification information corresponding to any target endoscope image can be overlaid on the thumbnail image corresponding to the target endoscope image. Alternatively, in the labeled image, the image identification information corresponding to any target endoscope image can also be presented separately from the thumbnail image corresponding to the target endoscope image. For example, the image identification information corresponding to the target endoscope image can be independently labeled in an image region near the thumbnail image corresponding to the target endoscope image. Figure 2 A schematic diagram of a marked image according to an embodiment of the present invention is shown. Figure 2 As shown, thumbnail image 210 and thumbnail image 220 can respectively represent thumbnail images corresponding to two different target endoscopic images. Figure 2 In the illustrated embodiment, the time interval between adjacent endoscopic images in the target video is 1 second. Timestamp 211 indicates that the corresponding target endoscopic image is a screenshot taken from the 6th frame of the endoscopic image in the target video. Timestamp 221 indicates that the corresponding target endoscopic image is a screenshot taken from the 10th frame of the endoscopic image in the target video. As can be understood from the above description, the timestamp can mark any image position on the corresponding target endoscopic image (e.g., ...). Figure 2 The image shown can be marked in the middle area (or at the edge of the corresponding target endoscope image), and the present invention does not limit this.

[0036] According to the above technical solution, in response to an image capture command, a target endoscopic image corresponding to the image capture command can be captured from the recorded target video. Then, based on the target endoscopic image, a labeled image can be generated. This solution uses the generated labeled image to merge the thumbnails of each target endoscopic image into a single labeled image. This not only improves user viewing efficiency but also makes the user interface cleaner and provides a better user experience. Furthermore, if there are multiple target endoscopic images, the generated labeled image facilitates comparison or unified observation of multiple endoscopic images. In addition, by merging the thumbnails of each target endoscopic image into a single labeled image, storage space can be effectively saved when storing the generated labeled image, especially when there are a large number of target endoscopic images.

[0037] For example, the marked image includes multiple thumbnails corresponding to target endoscope images and image identification information. The multiple thumbnails are arranged and displayed according to a preset rule, and each image identification information is displayed at a preset position within its corresponding thumbnail. The preset rule for arranging the thumbnails and the preset position of the image identification information within the thumbnails can be set as needed, and this document does not impose any limitations on them. For instance, a screenshot image can be obtained by adding corresponding image identification information to a target endoscope image, and then the screenshot image can be reduced to a preset image size to obtain a thumbnail image with added image identification information. This thumbnail image with added image identification information can then be added to the target display area of ​​a template image to obtain the marked image. Of course, the above display scheme is merely an example and not a limitation of the present invention. For example, each target endoscope image can also be directly reduced to a preset image size to obtain a thumbnail image of each target endoscope image, which can then be added to a first target display area of ​​a template image. The image identification information corresponding to each target endoscope image can also be added to a second target display area within a preset distance (e.g., above, below, left, or right) near the thumbnail image corresponding to that target endoscope image to obtain the marked image.

[0038] By adopting the above scheme, multiple thumbnails can be arranged and displayed according to preset rules, and the image identification information of each image can be displayed in a preset position of the corresponding thumbnail. In this way, the image identification information can be displayed closely associated with the thumbnail, making it convenient for users to view the thumbnail and image identification information corresponding to each target endoscope image at a glance, which helps to improve the user's viewing efficiency and user experience.

[0039] For example, the number of target videos is at least two. Generating a labeled image based on the target endoscopic images may include: generating a labeled image based on the target endoscopic images extracted from the at least two target videos respectively; and / or, the at least two target videos are at least two video segments contained in the initial video continuously acquired by the endoscopy system, and the timestamps corresponding to each endoscopic image in each target video are set according to the arrangement time of the endoscopic images in their respective target videos.

[0040] For example, the number of target videos can be at least two. Optionally, the at least two target videos can be at least two video segments acquired one-to-one by the endoscopy system in at least two different endoscopic examinations. That is, the at least two target videos are not acquired in the same endoscopic examination process. Alternatively, the at least two target videos can be two video segments acquired by the endoscopy system in a single endoscopic examination, or at least two video segments contained in the initial video acquired continuously by the endoscopy system. That is, the at least two target videos can be different video segments extracted from the same video acquired by the endoscopy system in the same endoscopic examination process. For example, if the overall duration of the initial video exceeds a preset maximum duration or the memory occupied exceeds a preset file size, the initial video can be split into multiple parts to obtain at least two target videos. The initial video can be a complete examination video obtained in one endoscopic examination process, or it can be a video obtained after deleting the complete examination video. For example, the first and last video frames in the complete examination video can be deleted, and the remaining video frames can be used as the initial video.

[0041] For example, generating a labeled image based on target endoscopic images may include: generating a labeled image based on target endoscopic images extracted from at least two target videos. For each target video, one or more target endoscopic images may be extracted from that target video. A labeled image can be generated based on the target endoscopic images extracted from at least two target videos. For example, if there are three target videos, and two target endoscopic images are extracted from target video 001, target video 002, and target video 003 respectively, then a labeled image can be generated based on the six extracted target endoscopic images. The method of generating the labeled image is similar to step S130 and will not be repeated here. It is understood that the above embodiments are merely exemplary, and the number of target endoscopic images extracted from each target video can be arbitrary.

[0042] According to the above technical solution, when there are at least two target videos, a labeled image can be generated based on the target endoscope images extracted from the at least two target videos respectively. This allows the image information contained in the target endoscope images extracted from the at least two target videos to be summarized into the labeled image, further improving the user's viewing efficiency.

[0043] For example, at least two target videos are at least two video segments contained in the initial video continuously acquired by the endoscope system, and the timestamps corresponding to each endoscope image in each target video are set according to the arrangement time of the endoscope images in their respective target videos.

[0044] The timestamps corresponding to each endoscopic image in each target video can be set according to the arrangement time of the endoscopic images in their respective target videos. In a specific embodiment of the present invention, the initial video contains a total of 100 frames of endoscopic images, and the initial video can be further divided into 3 target videos. Target video 001 contains endoscopic images from frame 1 to frame 30. Target video 002 contains endoscopic images from frame 31 to frame 70. Target video 003 contains endoscopic images from frame 71 to frame 100. For target video 001, the timestamps corresponding to the 30 endoscopic images it contains can be set according to the arrangement time of the endoscopic images in the target video. That is, the timestamps corresponding to these 30 endoscopic images can be set sequentially to 00:00:00 to 00:00:29. For target video 002, the timestamps corresponding to the 40 endoscopic images it contains can be set according to the arrangement time of the endoscopic images in the target video. That is, the timestamps corresponding to these 40 endoscopic images can be set sequentially from 00:00:00 to 00:00:39. For target video 003, the timestamps corresponding to the 30 endoscopic images it contains can be set according to the arrangement time of the endoscopic images in the target video. That is, the timestamps corresponding to these 30 endoscopic images can be set sequentially from 00:00:00 to 00:00:29. Of course, the above timestamp setting method is only an example and not a limitation of the present invention. When at least two target videos are at least two video segments contained in the initial video continuously acquired by the endoscope system, the timestamps corresponding to each endoscopic image in each target video can also be set according to the arrangement time of the endoscopic images in the entire initial video. For example, the timestamps corresponding to the 30 endoscopic images in the target video 001 can be set sequentially from 00:00:00 to 00:00:29, the timestamps corresponding to the 40 endoscopic images in the target video 002 can be set sequentially from 00:00:30 to 00:00:69, and the timestamps corresponding to the 30 endoscopic images in the target video 003 can be set sequentially from 00:00:70 to 00:00:99.

[0045] According to the above technical solution, at least two target videos are at least two video segments contained in the initial video continuously acquired by the endoscope system, and the timestamps corresponding to each endoscope image in each target video are set according to the arrangement time of the endoscope image in its respective target video. This setting method can help users quickly locate the position of the thumbnail image in its respective target video, further improving the user experience.

[0046] For example, the number of target videos is at least two, and the image identification information also includes the name and / or recording time and / or recording order of the target video to which the corresponding target endoscopic image belongs. The recording time can be the time when the endoscope system's system clock arrives, the time when the internet world clock arrives, or the recording time manually marked by the user, etc. The recording order can be the order of the current target video among all target videos, such as target video number 1, target video number 2, target video number 3, etc.

[0047] For example, the image identification information may also include the name of the target video to which the corresponding target endoscopic image belongs. Figure 3 A schematic diagram of a marker image according to another embodiment of the present invention is shown. Figure 3 The labeled image shown may include thumbnails corresponding to 10 target endoscopic images, such as thumbnail image 310. The upper left corner of each thumbnail image can be used to display the name of the target video to which the corresponding target endoscopic image belongs. For example, the "001" displayed in the upper left corner of thumbnail image 310 indicates that the target video to which the target endoscopic image corresponding to thumbnail image 310 belongs is target video 001. Similarly, Figure 3 The marked image shown may also include a thumbnail image corresponding to the target endoscope image extracted from the target video 000.

[0048] According to the above technical solution, the image identification information may include the name and / or recording time and / or recording order of the target video to which the corresponding target endoscope image belongs, which can help users accurately identify the target video to which each target endoscope image belongs and avoid confusion.

[0049] For example, the image capture instruction is generated based on the user's image capture operation, which may include pressing a preset button on the endoscope or a preset button on the input device.

[0050] In one embodiment, before extracting a target endoscopic image corresponding to the image extraction instruction from the target video in response to the image extraction instruction, the method may further include: acquiring an image extraction instruction generated in response to a user's image extraction operation.

[0051] As described above, the electronic device may include an input device or be communicatively connected to an input device. An embodiment of an electronic device receiving an image cropping command generated by a user performing an image cropping operation on the input device has already been described above, and will not be repeated here.

[0052] According to the above technical solution, after obtaining the image capture command generated in response to the user's image capture operation, the target endoscope image corresponding to the image capture command can be captured from the target video in response to the image capture command. This can meet the user's personalized needs and make it convenient for the user to capture the target endoscope image they want to obtain.

[0053] For example, the method may further include: during the recording of the target video, using a first target recognition network to identify the target object in the target video; when the target object is identified, generating an image cropping instruction, wherein the target endoscope image is an endoscope image containing the target object.

[0054] For example, a first target recognition network can be used to identify target objects in a target video. Upon detection of a target object, an image cropping instruction can be automatically generated. The target object can be any region inside a living organism, such as a lesion within a target area. For instance, if the target area is the stomach, the target object could be an ulcer or polyp in the stomach. The first target recognition network can be any suitable existing or future neural network capable of target recognition, including but not limited to: Region-based Convolutional Neural Network (RCNN), Faster RCNN, SingleShot MultiBox Detector (SSD), or You Only Look Once (YOLO). Upon detection of a target object, an image cropping instruction can be automatically generated. The target endoscopic image cropped from the target video based on this image cropping instruction is an endoscopic image containing the target object.

[0055] According to the above technical solution, a first target recognition network can be used to identify target objects in the target video and generate image cropping instructions when a target object is identified. This eliminates the need for complex user operations, automatically generating image cropping instructions and ensuring efficient extraction of target endoscopic images from the target video while reducing user workload. Furthermore, endoscopic images containing target objects are typically a key image type; therefore, cropping endoscopic images containing target objects based on their recognition results is beneficial for obtaining more observational and research-oriented endoscopic images. This solution has high practical application value.

[0056] For example, a first target recognition network can be used to identify target objects in a target video, automatically generating an image cropping command upon detection. Simultaneously, users can also generate image cropping commands in real-time using an input device while viewing the target video; this invention does not limit this. This satisfies users' personalized screenshot needs and allows for the cropping of endoscopic images containing the target object based on its recognition for subsequent observation or processing, thereby obtaining endoscopic images with significant observational and research value.

[0057] For example, if the target endoscopic image contains a target object, before generating a labeled image based on the target endoscopic image, the method may further include: using a second target recognition network to sequentially identify the target object forward and backward in the target video, starting from the target endoscopic image, until no target object can be identified; determining the identified endoscopic image containing the target object as a labeled video segment corresponding to the target endoscopic image; and in response to a thumbnail selection instruction for selecting any thumbnail image in the labeled image, determining and displaying the labeled video segment corresponding to the target endoscopic image corresponding to the selected thumbnail image.

[0058] The captured target endoscopic images typically contain important information, such as specific lesion information. Therefore, when capturing target endoscopic images, a video segment containing that target endoscopic image (i.e., a labeled video segment) can be extracted and stored for each target endoscopic image. The target endoscopic image can serve as a representative image for the labeled video segment. Users can get a general understanding of the video segment's information, such as whether it contains a specific lesion, through the thumbnail of the target endoscopic image. When a user discovers that the video segment may be valuable for further research and viewing through the thumbnail of the target endoscopic image, the user can further view the video segment for more detailed information by clicking on the thumbnail. For example, the labeled video segment can be played and displayed within the image area occupied by the thumbnail of the target endoscopic image on the labeled image, or it can be played and displayed in an additional new window on the display interface used to display the labeled image, or the entire display interface can be switched to a new window (i.e., the labeled image is temporarily not displayed) for playback.

[0059] In one embodiment, the target endoscopic image may contain a target object. Starting from the target endoscopic image, a second target recognition network can be used to sequentially identify target objects forward and backward in the target video until no target object can be identified. When performing the operation of "using the second target recognition network to sequentially identify target objects forward and backward in the target video, starting from the target endoscopic image, until no target object can be identified," the target object in the target endoscopic image may or may not be identified. For example, the presence of a target object in the target endoscopic image may be predetermined. For example, if the target endoscopic image is captured based on a user's image cropping operation, in this case, the user can default to designating an endoscopic image containing the target object as the target endoscopic image according to a preset rule. Or, if the target endoscopic image is automatically captured by the system, in this case, the endoscopic image containing the target object already identified by the first target recognition network can be used as the target endoscopic image. If the presence of a target object in the target endoscopic image has already been confirmed, target object identification in the target endoscopic image can be performed without first executing the operation of "using a second target recognition network to sequentially identify target objects forward and backward in the target video, starting from the target endoscopic image, until no target object can be identified." However, if it is necessary to identify the position of a target object in other endoscopic images based on the position of the target object in the target endoscopic image, and the position of the target object in the target endoscopic image has not been previously identified, target object identification in the target endoscopic image can be performed while executing the operation of "using a second target recognition network to sequentially identify target objects forward and backward in the target video, starting from the target endoscopic image, until no target object can be identified." Conversely, if the presence of a target object in the target endoscopic image is not confirmed, target object identification can be performed while executing the operation of "using a second target recognition network to sequentially identify target objects forward and backward in the target video, starting from the target endoscopic image, until no target object can be identified." For example, the target endoscope image is captured based on the user's image cropping operation, and there is no preset rule that the user-specified target endoscope image contains the target object. That is, it is unknown whether the user-specified target endoscope image contains the target object. In this case, the target object in the target endoscope image can be identified when performing the operation of "using the second target recognition network to identify the target object in the target video in a forward and backward manner, starting from the target endoscope image, until the target object can no longer be identified".

[0060] The implementation of the second target recognition network is similar to that of the first target recognition network, and will not be described in detail here. The second target recognition network may be the same as or different from the first target recognition network, and this invention does not impose any restrictions on this. For example, the acquired target video may contain 30 frames of endoscopic images. The target endoscopic image is obtained by cropping the 5th frame of the endoscopic image in the target video. Starting from the 5th frame of the endoscopic image, the target object can be identified sequentially from the previous 4 frames of endoscopic images and then from the next 25 frames of endoscopic images until no target object can be identified. If the target video is a real-time acquired video, then when identifying the target object sequentially from the target video, the target object can be tracked and identified until no target object can be identified. For the 5th frame of the endoscopic image, the second target recognition network can be used to identify the target object in the 5th frame of the endoscopic image, or the target object in the 5th frame of the endoscopic image can be left unidentified. For example, if the target video is a real-time captured video, then when tracking and identifying the target object backward from the target video, the target object in the 5th frame of the endoscopic image can be identified and used as a reference to identify the target object in subsequent endoscopic images.

[0061] After obtaining all endoscopic images containing the target object, these endoscopic images can be identified as a marked video segment corresponding to the target endoscopic image. For example, when identifying the target object from the 5th frame of the endoscopic image forward 4 frames, the target object cannot be identified in the 3rd frame; similarly, when identifying the target object from the 5th frame forward 25 frames, the target object cannot be identified in the 25th frame. The 4th to 24th frames of the endoscopic image can be identified as the marked video segment. In response to a thumbnail selection command for selecting any thumbnail from the marked images, the marked video segment corresponding to the target endoscopic image corresponding to the selected thumbnail can be identified and displayed. For example, the thumbnail selection command can be input by a user through the aforementioned input device. In one specific embodiment of the present invention, a marked image can be displayed on a monitor. A user can click on the image area occupied by each thumbnail image within the marked image using an input device. In response to the user's click, a corresponding thumbnail selection command can be generated. It is understood that when the monitor is a touchscreen, the input device can be arbitrarily the monitor itself; that is, the monitor can be used simultaneously as an input and output device. For example, in... Figure 3In the illustrated embodiment, a user can click on the area containing the thumbnail image 310 to generate a thumbnail image selection instruction. Upon receiving the image selection instruction, the electronic device can determine and display on the display the marked video segment corresponding to the target endoscopic image corresponding to the thumbnail image 310. In another specific embodiment of the invention, a text input box can be presented on the display interface. The user can input the corresponding number of the thumbnail image into the text input box using an input device such as a mouse, keyboard, or monitor to generate a thumbnail image selection instruction. For example, the user can input "001-2" into the text input box, indicating that the selected thumbnail image is the second thumbnail image in the target video 002. The above description of the thumbnail image selection instruction generation method is merely exemplary, and the invention does not limit it.

[0062] According to the above technical solution, using a second target recognition network, starting from the target endoscopic image, target objects are sequentially identified forward and backward in the target video until no target object can be identified. Then, the identified endoscopic images containing the target objects are determined as the labeled video segments corresponding to the target endoscopic image. After obtaining a thumbnail selection instruction for selecting any thumbnail image from the labeled images, the labeled video segment corresponding to the target endoscopic image can be determined and displayed in response to the thumbnail selection instruction. This solution allows users to conveniently view labeled video segments within the time period before and after any selected thumbnail image without needing to re-view the complete target video, resulting in a better user experience and higher efficiency.

[0063] For example, the image identification information also includes the start and end times and / or duration of the marked video segments. The start and end times of the marked video segments can be represented by absolute times corresponding to the start and end of the marked video segments, or by start and end timestamps corresponding to the marked video segments within the target video. The absolute times can be the time when the endoscope system's system clock arrives, the time when the internet world clock arrives, or the time manually marked by the user, etc. By including the start and end times and / or duration of the marked video segments in the image identification information, users can directly understand the length of each marked video segment through the marked image, providing users with more comprehensive information.

[0064] For example, after extracting the target endoscope image corresponding to the image extraction command from the target video in response to the image extraction command, the method may further include: determining whether a screenshot image or a target endoscope image is stored in a preset storage directory of a preset storage device, wherein the preset storage device is used to store a screenshot image corresponding to the target endoscope image, and the screenshot image is a target endoscope image with added image identification information; when a screenshot image or a target endoscope image is stored in the preset storage directory, outputting a first prompt message, the first prompt message being used to indicate that the target endoscope image extraction was successful; when no screenshot image or target endoscope image is stored in the preset storage directory, outputting a second prompt message, the second prompt message being used to indicate that the target endoscope image extraction failed.

[0065] In one embodiment, the electronic device may include a preset storage device (which may be referred to as a first preset storage device) or be communicatively connected to the preset storage device. The preset storage device may be any device with storage function, including but not limited to random access memory (RAM) or read-only memory (ROM). The preset storage device may be used to store a screenshot image or the target endoscope image corresponding to the target endoscope image. The screenshot image is the target endoscope image with added image identification information. After extracting the target endoscope image corresponding to the image extraction command from the target video, the target endoscope image can be directly stored in the preset storage device, or optionally, image identification information can be added to the target endoscope image before storing the corresponding screenshot image in the preset storage device. The method of adding image identification information has been described in detail in the preceding embodiments and will not be repeated here for brevity. The preset storage directory can be any storage location preset by the user for storing screenshot images or target endoscope images; this invention does not limit this. By determining whether a screenshot image is stored in the preset storage directory of the preset storage device, it can be determined whether the target endoscope image has been successfully extracted. For example, the image name of the screenshot can be searched in the preset storage directory of the preset storage device. If the image name is found in the preset storage directory, the screenshot is considered successfully captured and a first prompt message is output; otherwise, the screenshot is considered to have failed and a second prompt message is output. The screenshot can be named in the format of "name of the target video + _ + timestamp of the target endoscopic image". Of course, this naming method is just an example, and users can use any other method to name the screenshot. The preset storage device can also optionally be used to store the target video, and the target video can also optionally be stored in the preset storage directory. The scheme of storing the target video and the screenshot or the target endoscopic image in the same preset storage directory is more convenient for image management and retrieval.

[0066] The electronic device may also include an output device or be communicatively connected to an output device. Exemplarily, the output device may include one or more of a display module, a buzzer, a speaker, a light-emitting module, and a communication module. The communication module may include any wired communication module and / or a wireless communication module. Exemplarily, the first prompt information may include one or more of audio information, text information, image information, video information, and light information. The display module is used to output one or more of text information, image information, and video information; the buzzer and speaker are used to output audio information; and the light-emitting module is used to output light information. The communication module is used to transmit any form of the first prompt information to any associated device. Associated devices may include, but are not limited to, personal computers, servers, and mobile terminals (e.g., a user's smartphone). For example, the output device may be a buzzer, which, upon determining that the screenshot image has been successfully captured, can output an audio signal of a first preset duration to remind the user that the screenshot image has been successfully captured. Alternatively, the output device may also be a display module, which, upon determining that the screenshot image has been successfully captured, can display characters such as "Successfully Captured" to remind the user that the screenshot image has been successfully captured. In another embodiment, the output device can also be a wireless communication module, through which the first prompt message is sent to the user's terminal device, such as a mobile phone or computer, to indicate that the screenshot was successfully captured. The method of outputting the second prompt message is similar to that of the first prompt message, and for simplicity, it will not be described again here. It is understood that the above-described implementation methods for outputting the first and second prompt messages are merely exemplary, and the present invention does not limit them.

[0067] According to the above technical solution, the success of capturing the target endoscope image can be determined by judging whether the screenshot image is stored in the preset storage directory of the preset storage device, and the corresponding prompt information can be output. This can promptly provide feedback to the user on the screenshot status, so that the user can use other methods to mark the target endoscope image corresponding to the image capture command in the event of a screenshot failure, and avoid the user missing important information.

[0068] For example, before generating a labeled image based on a target endoscopic image, the method may further include: adding image identification information to the target endoscopic image to obtain a screenshot image corresponding to the target endoscopic image; generating a labeled image based on the target endoscopic image may include: obtaining a template image, the template image including a target display area for displaying a thumbnail image corresponding to the target endoscopic image; reducing the size of the screenshot image corresponding to the target endoscopic image according to a preset image size to obtain a thumbnail image corresponding to the target endoscopic image with added image identification information; adding the thumbnail image with added image identification information to the target display area in the template image to obtain a labeled image.

[0069] In one embodiment, before generating a labeled image based on the target endoscope image, image identification information can be added to the target endoscope image to obtain a screenshot image corresponding to the target endoscope image. The implementation method of adding image identification information to the target endoscope image has been described in detail in the preceding embodiments, and for the sake of brevity, it will not be repeated here. The following describes in detail the implementation method of generating a labeled image based on the target endoscope image.

[0070] For example, a template image pre-stored in a preset storage device (which may be referred to as a second preset storage device) can be obtained, or the template image can be directly generated by an electronic device according to a preset template image generation format. Similar to the first preset storage device, the second preset storage device can be contained within an electronic device or communicatively connected to it. The second preset storage device and the first preset storage device can be the same storage device or different storage devices. The template image may include a target display area for displaying a thumbnail image corresponding to the target endoscope image. The size of the template image and the background color can be arbitrarily set, and this invention does not limit this. For example, the template image can be an image with a pure black background and an image size of 3840 (pixels) * 2160 (pixels). The obtained screenshot image corresponding to the target endoscope image is reduced according to the preset image size to obtain a thumbnail image corresponding to the target endoscope image with added image identification information. The preset image size can be any image size, such as 800 (pixels) * 450 (pixels), 800 (pixels) * 450 (pixels), etc., and this invention does not limit this. After obtaining thumbnails with added image identification information, these thumbnails can be added to the target display area of ​​the template image to obtain a labeled image. When there are multiple thumbnails, they can be added sequentially to the target display area of ​​the template image according to the order of their respective timestamps. Furthermore, the thumbnails can be added sequentially in a matrix arrangement. For example, they can be added from left to right and from top to bottom. Optionally, the video name of the initial video (which can be used as the initial video when there is only one target video) can also be added to the labeled image. For example, in... Figure 3 In the illustrated embodiment, "2021031511211005" displayed in the upper right corner of the marked image is the name of the initial video. The naming method for the initial video can be arbitrarily set according to requirements, and this invention does not impose any restrictions on it. It is understood that, in addition to the timestamp corresponding to the target endoscopic image, the name of the target video to which the target endoscopic image belongs, and the name of the initial video, other information can be added as needed, such as preset graphics, usernames, etc., and this invention does not impose any restrictions on this.

[0071] According to the above technical solution, image identification information is added to the target endoscopic image to obtain a screenshot image corresponding to the target endoscopic image. Then, according to a preset image size, the screenshot image corresponding to the target endoscopic image is reduced in size to obtain a thumbnail image of the target endoscopic image with added image identification information. The thumbnail image with added image identification information is added to the target display area of ​​the obtained template image to obtain a labeled image. This solution can quickly obtain a labeled image, and the thumbnails contained in the labeled image are arranged in a relatively regular manner, which is convenient for users to view.

[0072] For example, before generating a marker image based on the target endoscope image, the method further includes: storing the captured target endoscope image or screenshot image in a preset storage directory of a preset storage device, the preset storage device being used to store the screenshot image or target endoscope image corresponding to the target endoscope image, the screenshot image being the target endoscope image with added image identification information; after generating a marker image based on the target endoscope image, the method further includes: automatically deleting the target endoscope image from the preset storage device; or, outputting a third prompt message and deleting the target endoscope image from the preset storage device in response to a deletion command input by the user based on the third prompt message.

[0073] The method for storing the target endoscopic image or screenshot has already been described above and will not be repeated here. The method for outputting the third prompt message is similar to the first and second prompt messages, and for simplicity, will not be repeated here. After generating the marked image, the screenshot or target endoscopic image stored under the preset storage target of the aforementioned preset storage device can be deleted to save storage space.

[0074] According to a second aspect of the present invention, an image generation apparatus is also provided. Figure 4 A schematic diagram of an image generation apparatus 400 according to an embodiment of the present invention is shown, such as... Figure 4 As shown, the image generation device 400 may include a recording module 410, a cropping module 420, and a first generation module 430.

[0075] The recording module 410 is used to record a target video containing multiple endoscopic images in response to a video recording command.

[0076] The capture module 420 is used to capture the target endoscope image corresponding to the image capture command from the target video during the recording of the target video.

[0077] The first generation module 430 is used to generate a labeled image based on the target endoscope image. The labeled image includes a thumbnail image corresponding to the target endoscope image and image identification information. The image identification information includes the timestamp of the corresponding target endoscope image.

[0078] For example, the labeled image includes multiple thumbnails corresponding to target endoscope images and image identification information. The multiple thumbnails are arranged and displayed according to preset rules, and each image identification information is displayed at a preset position in the corresponding thumbnail.

[0079] For example, the number of target videos is at least two, and the image identification information also includes the name and / or recording time and / or recording order of the target video to which the corresponding target endoscope image belongs.

[0080] For example, the image capture instruction is generated based on the user's image capture operation, which may include pressing a preset button on the endoscope or a preset button on the input device.

[0081] For example, the image generation device 400 may further include a first recognition module and a second generation module. The first recognition module is used to identify the target object in the target video using a first target recognition network during the recording of the target video; the second generation module is used to generate an image cropping instruction when the target object is identified, wherein the target endoscope image is an endoscope image containing the target object.

[0082] For example, the target endoscopic image contains a target object. The image generation apparatus 400 may further include a second recognition module, a first determination module, and a second determination and display module. The second recognition module is used to identify target objects sequentially forward and backward in the target video, starting from the target endoscopic image, before the first generation module 430 performs the step of generating a marked image based on the target endoscopic image. This process continues until no target object can be identified. The first determination module is used to determine the identified endoscopic image containing the target object as a marked video segment corresponding to the target endoscopic image. The second determination and display module is used to determine and display the marked video segment corresponding to the target endoscopic image corresponding to the selected thumbnail image in response to a thumbnail selection instruction for selecting any thumbnail image in the marked images.

[0083] For example, the image identification information also includes the start and end times and / or duration of the video segment.

[0084] For example, the image generation device 400 may further include a judgment module, a first output module, and a second output module. The judgment module is used to determine whether a screenshot image or a target endoscope image is stored in a preset storage directory of a preset storage device after the capture module 420 performs the step of capturing a target endoscope image corresponding to the image capture command from the target video. The preset storage device is used to store a screenshot image or a target endoscope image corresponding to the target endoscope image, where the screenshot image is a target endoscope image with added image identification information. The first output module is used to output a first prompt message when a screenshot image or a target endoscope image is stored in the preset storage directory, indicating that the target endoscope image capture was successful. The second output module is used to output a second prompt message when no screenshot image or target endoscope image is stored in the preset storage directory, indicating that the target endoscope image capture failed.

[0085] For example, the image generation apparatus 400 may further include a storage module and a deletion module. The storage module is used to store the captured target endoscope image or screenshot image in a preset storage directory of a preset storage device before the first generation module 430 performs the step of generating a marker image based on the target endoscope image. The preset storage device is used to store the screenshot image or target endoscope image corresponding to the target endoscope image, where the screenshot image is the target endoscope image with added image identification information. The deletion module is used to automatically delete the target endoscope image from the preset storage device after the first generation module 430 performs the step of generating a marker image based on the target endoscope image; or, output a third prompt message and delete the target endoscope image from the preset storage device in response to a deletion command input by the user based on the third prompt message.

[0086] Those skilled in the art can understand the specific implementation scheme and beneficial effects of the above-mentioned image generation device by reading the relevant description of the image generation method. For the sake of brevity, it will not be described in detail here.

[0087] According to a third aspect of the present invention, an electronic device is also provided. Figure 5 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. Figure 5 As shown, the electronic device 500 includes a processor 510 and a memory 520. The memory 520 stores a computer program, and the computer program instructions are executed by the processor 510 to perform the above-described image generation method.

[0088] In one embodiment, the computer program instructions executed by the processor 510 are used to perform the following operations: in response to a video recording instruction, recording a target video, the target video containing multiple endoscopic images; during the recording of the target video, in response to an image cropping instruction, cropping a target endoscopic image corresponding to the image cropping instruction from the target video; generating a marker image based on the target endoscopic image, the marker image including a thumbnail image corresponding to the target endoscopic image and image identification information, the image identification information including a timestamp of the corresponding target endoscopic image.

[0089] According to a fourth aspect of the invention, an endoscope system is also provided. This endoscope system may include the aforementioned electronic equipment.

[0090] According to a fifth aspect of the present invention, a storage medium storing a computer program / instructions is also provided. The storage medium may, for example, include a storage component of a tablet computer, a hard disk of a personal computer, an erasable programmable read-only memory (EPROM), a portable read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The storage medium may be any combination of one or more computer-readable storage media. The computer program / instructions are used by a processor during runtime to execute the image generation method described above.

[0091] In one embodiment, the computer program / instructions are executed by the processor to perform the following operations: in response to a video recording instruction, recording a target video, the target video containing multiple endoscopic images; during the recording of the target video, in response to an image cropping instruction, cropping a target endoscopic image corresponding to the image cropping instruction from the target video; generating a marker image based on the target endoscopic image, the marker image including a thumbnail image corresponding to the target endoscopic image and image identification information, the image identification information including a timestamp of the corresponding target endoscopic image.

[0092] According to a sixth aspect of the present invention, a computer program product is also provided, comprising computer program instructions that, when executed, perform the image generation method described above.

[0093] In one embodiment, the computer program instructions are configured to perform the following operations during runtime: in response to a video recording instruction, record a target video, the target video containing multiple endoscopic images; during the recording of the target video, in response to an image cropping instruction, crop the target endoscopic image corresponding to the image cropping instruction from the target video; generate a marker image based on the target endoscopic image, the marker image including a thumbnail image corresponding to the target endoscopic image and image identification information, the image identification information including a timestamp of the corresponding target endoscopic image.

[0094] Those skilled in the art can understand the specific implementation schemes of the above-mentioned electronic devices, endoscopes, computer-readable storage media, and computer program products by reading the relevant descriptions of the image generation methods above, and for the sake of brevity, they will not be described in detail here.

[0095] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0096] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0097] In the several embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0098] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0099] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0100] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0101] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0102] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the image generation apparatus according to embodiments of the present invention. The present invention can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing some or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0103] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0104] The above description is merely a specific embodiment of the present invention or an explanation of that embodiment. The scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An image generation method, characterized in that, The method includes: In response to a video recording command, a target video is recorded, the target video containing multiple endoscopic images; During the recording of the target video, in response to an image cropping command, a target endoscope image corresponding to the image cropping command is cropped from the target video; A labeled image is generated based on the target endoscope image. The labeled image includes a thumbnail image corresponding to the target endoscope image and image identification information. The image identification information includes the timestamp of the corresponding target endoscope image.

2. The method as described in claim 1, characterized in that, The marked image includes multiple thumbnail images corresponding to the target endoscope images and image identification information. The multiple thumbnail images are arranged and displayed according to a preset rule, and each image identification information is displayed at a preset position in the corresponding thumbnail image.

3. The method as described in claim 2, characterized in that, The number of target videos is at least two, and the image identification information also includes the name and / or recording time and / or recording order of the target video to which the corresponding target endoscope image belongs.

4. The method as described in claim 1, characterized in that, The image capture instruction is generated based on the user's image capture operation, which includes pressing a preset button on the endoscope or a preset button on the input device.

5. The method as described in claim 1, characterized in that, The method further includes: During the recording of the target video, a first target recognition network is used to identify the target objects in the target video; When the target object is identified, the image capture instruction is generated, wherein the target endoscope image is an endoscope image containing the target object.

6. The method according to any one of claims 1 to 5, characterized in that, The target endoscopic image contains a target object. Before generating a labeled image based on the target endoscopic image, the method further includes: Using a second target recognition network, starting from the target endoscopic image, the target object is sequentially identified forward and backward in the target video until it can no longer be identified; The identified endoscopic image containing the target object is identified as a labeled video segment corresponding to the target endoscopic image; In response to a thumbnail selection instruction for selecting any thumbnail in the marked images, the marked video segment corresponding to the target endoscopic image corresponding to the selected thumbnail is determined and displayed.

7. The method as described in claim 6, characterized in that, The image identification information also includes the start and end times and / or duration of the marked video segment.

8. The method according to any one of claims 1 to 5, characterized in that, After extracting a target endoscope image corresponding to the image cropping instruction from the target video in response to the image cropping instruction, the method further includes: Determine whether a screenshot image or the target endoscope image is stored in the preset storage directory of the preset storage device. The preset storage device is used to store a screenshot image or the target endoscope image corresponding to the target endoscope image. The screenshot image is the target endoscope image with the image identification information added. When the screenshot image or the target endoscope image is stored in the preset storage directory, a first prompt message is output, which indicates that the target endoscope image was successfully captured. If the screenshot image or the target endoscope image is not stored in the preset storage directory, a second prompt message is output, which indicates that the capture of the target endoscope image has failed.

9. The method according to any one of claims 1 to 5, characterized in that, Before generating the labeled image based on the target endoscopic image, the method further includes: The captured target endoscope image or screenshot image is stored in a preset storage directory of a preset storage device. The preset storage device is used to store screenshot images or target endoscope images corresponding to the target endoscope image. The screenshot image is the target endoscope image with the image identification information added. After generating the labeled image based on the target endoscopic image, the method further includes: The target endoscope image is automatically deleted from the preset storage device; or... Output a third prompt message and, in response to a deletion command input by the user based on the third prompt message, delete the target endoscope image from the preset storage device.

10. An image generation apparatus, characterized in that, The device includes: The recording module is used to record a target video in response to a video recording command, the target video containing multiple endoscopic images; The cropping module is used to, in response to an image cropping command, crop a target endoscope image corresponding to the image cropping command from the target video during the recording of the target video; The first generation module is used to generate a labeled image based on the target endoscope image. The labeled image includes a thumbnail image corresponding to the target endoscope image and image identification information. The image identification information includes a timestamp of the corresponding target endoscope image.

11. An electronic device comprising a processor and a memory, characterized in that, The memory stores a computer program, the computer program instructions of which, when executed by the processor, are used to perform the image generation method as described in any one of claims 1 to 9.

12. An endoscope system, characterized in that, Including the electronic device as described in claim 11.

13. A computer-readable storage medium storing a computer program / instructions, characterized in that, The computer program / instructions are used to perform the image generation method as described in any one of claims 1 to 9 when the program is executed.

14. A computer program product comprising computer program instructions, characterized in that, The computer program instructions, when executed, are used to perform the image generation method as described in any one of claims 1 to 9.

Images