Image information determination system, image information determination method, image processing apparatus, and endoscope system

Abstract

The invention relates to an image information determination system and method, an image processing apparatus, and an endoscope system. The image information determination system comprises an image acquisition part which is used for acquiring a first to-be-detected image for shooting a to-be-detected part in a white light mode for multiple times and acquiring a second to-be-detected image for shooting the to-be-detected part in a detection light mode for at least one time; the information detection part is used for processing the first to-be-detected image to determine first focus information in the first to-be-detected image, and processing the second to-be-detected image to determine second focus information in the second to-be-detected image; and based on the second focus information obtained in the target detection light mode and the first focus information obtained in the target white light mode, determining target focus information of the to-be-detected part in all the shot to-be-detected images. Through the image information determination system, dependence on subjective experience judgment of doctors is reduced, and the focus detection rate is increased.

Description

Technical Field

[0001] The present application relates to the field of medical technology, and in particular, to an image information determination system, method, image processing device, and endoscope system. Background Art

[0002] An endoscope system is a common medical diagnosis and treatment system, generally including an endoscope, an image processing device, a light source device, and a display. A doctor can use the endoscope system to obtain images of the internal cavities of a patient's body, so as to achieve the examination of the internal cavities.

[0003] The endoscope system can collect corresponding types of images in different light source modes for the doctor's different needs. For example, the endoscope system irradiates white light into the internal cavity in the white light mode, and then collects the white light state image of the internal cavity. Another example is that the endoscope system irradiates detection light (also called special light) into the internal cavity in the detection light mode (also called special light mode), and then collects the special light image of the internal cavity. Among them, the detection light is light other than white light, such as special light such as blue light or purple light.

[0004] In practical applications, a doctor needs to rely on his own experience to select the white light mode or the detection light mode to observe the state of the patient's internal cavity and judge whether there are lesions and related lesion information. This method is highly dependent on the doctor's subjective experience judgment, resulting in a relatively low detection rate of lesions in practical applications. Summary of the Invention

[0005] Based on this, in order to solve the above technical problems, it is necessary to provide an image information determination system, method, image processing device, and endoscope system.

[0006] In a first aspect, the present application provides an image information determination system, including:

[0007] An image acquisition unit, configured to acquire first test images of a part to be tested taken in multiple white light modes, and acquire second test images of the part to be tested taken in at least one detection light mode;

[0008] An information detection unit, configured to process the first test images to determine first lesion information in the first test images, and process the second test images to determine second lesion information in the second test images;

[0009] The information detection unit is further configured to determine target lesion information of the part to be tested in all the captured test images based on the second lesion information obtained in the target detection light mode and the first lesion information obtained in the target white light mode; the target detection light mode is any one of the detection light modes; the target white light mode is the white light mode adjacent to the opening time of the target detection light mode.

[0010] In one embodiment, the target lesion information includes the lesion information of at least one lesion in the part to be examined; in the operation of determining the target lesion information by the information detection unit, the operation on a single lesion includes:

[0011] In the case where the first image to be examined in the target white light mode and the second image to be examined in the target detection light mode include the same lesion, determine the second information of the same lesion in the second lesion information obtained in the target detection light mode, and determine the first information of the same lesion in the first lesion information obtained in the target white light mode;

[0012] Based on the determined first information and second information, determine the target lesion information of the same lesion in all the captured images to be examined.

[0013] In one embodiment, the lesion severity is included in the target lesion information, the first information, and the second information; in the operation of determining the target lesion information by the information detection unit, the operation on the same lesion further includes:

[0014] When the lesion severity in the second information is higher than the lesion severity in the first information, update the lesion severity of the same lesion in all the captured images to be examined according to the lesion severity in the second information.

[0015] In one embodiment, in the operation of the information detection unit for determining the target lesion information, the operation on a single lesion includes:

[0016] Determine the mismatched lesions between the first image to be examined in the target white light mode and the second image to be examined in the target detection light mode;

[0017] Determine the lesion information of the mismatched lesions according to the second lesion information obtained in the target detection light mode and the first lesion information obtained in the target white light mode;

[0018] Determine the target lesion information of the mismatched lesions in all the captured images to be examined according to the lesion information of the mismatched lesions.

[0019] In one embodiment, the operation of the image acquisition unit for acquiring the first image to be examined and the second image to be examined includes:

[0020] Acquire the images to be examined of the part to be examined, and identify the light source mode to which the images to be examined belong;

[0021] When the light source mode is the white light mode, use the image to be examined as the first image to be examined;

[0022] When the light source mode is the detection light mode, use the image to be examined as the second image to be examined.

[0023] In one embodiment, the information detection unit is further configured to:

[0024] Obtain reference lesion information of the part to be detected in a reference detection light mode; the reference detection light mode is a detection light mode adjacent to the target detection light mode and with an opening time after the target detection light mode;

[0025] Determine the target lesion information of the part to be detected in all the captured images of the part to be detected according to the reference lesion information in the reference detection light mode and the second lesion information in the target detection light mode.

[0026] In one embodiment, the target lesion information includes the severity of the lesion; the information detection unit is further configured to:

[0027] When the second image to be detected in the reference detection light mode and the second image to be detected in the target detection light mode include the same lesion, compare the severity of the same lesion in the reference lesion information and the second lesion information;

[0028] Determine the severity of the same lesion in all the captured images of the part to be detected according to the comparison result.

[0029] In one embodiment, the information detection unit is further configured to:

[0030] When the comparison result is that the severity of the same lesion in the reference lesion information is higher than or equal to the severity of the same lesion in the second lesion information, determine the severity of the same lesion in all the captured images of the part to be detected according to the severity of the same lesion in the reference lesion information.

[0031] In one embodiment, the information detection unit is further configured to:

[0032] When the comparison result is that the severity of the same lesion in the reference lesion information is lower than the severity of the same lesion in the second lesion information, obtain the severity of the same lesion in the reference lesion information in the subsequent reference detection light mode; the subsequent reference detection light mode is a detection light mode adjacent to the reference detection light mode and with an opening time after the reference detection light mode;

[0033] If the severity of the same lesion in the reference lesion information in at least a preset number of consecutive subsequent reference detection light modes is lower than the severity of the same lesion in the second lesion information, determine the severity of the same lesion in all the captured images of the part to be detected according to the severity of the same lesion in the reference lesion information in the latest subsequent reference detection light mode.

[0034] In a second aspect, the present application further provides an image information determination method, including:

[0035] Obtain a first image to be examined taken of the part to be examined under multiple white light modes, and obtain a second image to be examined taken of the part to be examined under at least one detection light mode;

[0036] Process the first image to be examined to determine the first lesion information of the first image to be examined, and process the second image to be examined to determine the second lesion information of the second image to be examined;

[0037] Based on the second lesion information obtained under the target detection light mode and the first lesion information obtained under the target white light mode, determine the target lesion information of the part to be examined in all the captured images to be examined; the target detection light mode is any one of the detection light modes; the target white light mode is the white light mode adjacent to the opening time of the target detection light mode.

[0038] In a third aspect, the present application also provides an image processing device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of any one of the above image information determination methods are implemented.

[0039] In a fourth aspect, the present application also provides an endoscope system, including an endoscope, a display, a light source device, and the image information determination system of any one of the above.

[0040] For the above image information determination system, method, image processing device, and endoscope system, the image information determination system includes an image acquisition unit and an information detection unit. The image acquisition unit is used to obtain a first image to be examined taken of the part to be examined under multiple white light modes, and obtain a second image to be examined taken of the part to be examined under at least one detection light mode. The information detection unit is used to process the first image to be examined to determine the first lesion information in the first image to be examined, and process the second image to be examined to determine the second lesion information in the second image to be examined; and based on the second lesion information obtained under the target detection light mode and the first lesion information obtained under the target white light mode, determine the target lesion information of the part to be examined in all the captured images to be examined. Among them, the target detection light mode is any one of the detection light modes; the target white light mode is the white light mode adjacent to the opening time of the target detection light mode. The above image information determination system can combine images under two and multiple light source modes for lesion detection, realizing a comprehensive analysis of the states of lesions at different times, improving the comprehensiveness of data for determining lesion information, and the detection light mode has a better detection effect on lesions. Therefore, the detection method combining data comprehensiveness and the detection light mode can achieve more accurate data detection, improve the credibility of the detection result, and at the same time improve the detection rate of lesions in the image. Description of the Drawings

[0041] Figure 1It is a structural block diagram of an image information determination system in an embodiment;

[0042] Figure 2 It is an alternating schematic diagram of a light source mode in an embodiment;

[0043] Figure 3 It is a schematic flow diagram of obtaining target lesion information in an embodiment;

[0044] Figure 4 It is a tracking schematic diagram of a lesion in an image in an embodiment;

[0045] Figure 5 It is a schematic flow diagram of obtaining target lesion information in another embodiment;

[0046] Figure 6 It is a schematic flow diagram of obtaining a first image to be examined and a second image to be examined in an embodiment;

[0047] Figure 7 It is a schematic flow diagram of obtaining target lesion information in another embodiment;

[0048] Figure 8 It is a schematic flow diagram of obtaining a lesion level in another embodiment;

[0049] Figure 9 It is a schematic flow diagram of obtaining a lesion level in another embodiment;

[0050] Figure 10 It is a schematic flow diagram of an image information determination method in an embodiment;

[0051] Figure 11 It is a schematic flow diagram of an image information determination method in another embodiment;

[0052] Figure 12 It is an internal structure diagram of an image processing device in an embodiment;

[0053] Figure 13 It is a structural block diagram of an endoscope system in an embodiment. Detailed implementation manners

[0054] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] The embodiments of the present application provide an image information determination system. As Figure 1As shown in the figure, the image information determination system includes an image acquisition unit 100 and an information detection unit 200. In practical applications, the image information determination system can be integrated into an entity electronic device such as an image processing device, so that these electronic devices can have the relevant capabilities to detect lesion information in each of the following embodiments.

[0056] Among them, the image information determination system is used to determine the lesion information in the acquired image to be examined.

[0057] The image acquisition unit 100 is used to acquire a first image to be examined taken of the part to be examined in multiple white light modes, and acquire a second image to be examined taken of the part to be examined in at least one detection light mode.

[0058] Among them, the white light mode is to irradiate white light (also known as natural light) on the part to be examined, and the detection light mode is to irradiate special light on the part to be examined. Usually, the special light is non-white light. Exemplarily, the special light can be formed based on narrow band imaging technology (NBI), autofluorescence imaging technology (AFI), or blue laser imaging technology (BLI). In practical applications, the special light has a more prominent imaging effect on the lesion, so that more details of the lesion can be presented in the image, which is beneficial to the detection of the lesion.

[0059] The first image to be examined is an image obtained by photographing the part to be examined in the white light mode, and the second image to be examined is an image obtained by photographing the part to be examined in the detection light mode. Among them, the white light mode and the detection light mode can be provided by a light source providing unit (also known as a light source device or a light source host). Optionally, the light source providing unit can be integrated into the image information determination system or can be independent of the image information determination system.

[0060] During the period when the light source providing unit provides different light source modes including the white light mode and the detection light mode for the part to be examined, the endoscope continuously takes images of the part to be examined to achieve image acquisition in the two light source modes. The image information determination system can communicate with the endoscope, and the image acquisition unit 100 can obtain the first image to be examined taken of the part to be examined in multiple white light modes and obtain the second image to be examined taken of the part to be examined in at least one detection light mode from the endoscope.

[0061] Exemplarily, the light source providing unit can periodically switch between the white light mode and the detection light mode for the image acquisition unit 100 to obtain the first image to be examined and the second image to be examined. For example, as Figure 2As shown, the sampling frequency of the image acquisition unit 100 is 60 Hz, that is, 60 frames of images are acquired per second. The switching period t of the light source providing unit is 0.5 s. During the illumination period, the light source providing unit provides the white light mode, and the detection light mode is switched every 0.5 s. 29 first images to be detected and 1 second image to be detected are obtained in each half sampling period (0.5 s). 58 first images to be detected and 2 second images to be detected can be obtained in one sampling period (1 s).

[0062] Optionally, when the light source providing unit provides different light source modes for the part to be detected, the image acquisition unit 100 can acquire the images to be detected obtained by shooting and determine the light source mode at the time of shooting to determine whether the image to be detected is the first image to be detected or the second image to be detected. Exemplarily, the image acquisition unit 100 can communicate with the light source providing unit to determine the light source mode provided by the light source providing unit when shooting the corresponding image to be detected.

[0063] The information detection unit 200 is used to process the first image to be detected to determine the first lesion information of the first image to be detected, and to process the second image to be detected to determine the second lesion information of the second image to be detected.

[0064] Among them, the first lesion information includes the lesion information of the lesions in the first image to be detected, and the second lesion information includes the lesion information of the lesions in the second image to be detected. The information detection unit 200 is used to detect lesions in the image to obtain the lesion information of the lesions in the image. The lesion information is the attribute information of the lesions. Exemplarily, the lesion information may include at least one of the number, position, shape, size, and severity of the lesions (which can be quantitatively characterized in the form of lesion grades or types / categories, etc.). The lesion position can be represented by the minimum circumscribed rectangle of the lesion or the position where the contour line is located.

[0065] Optionally, after preprocessing the first image to be detected, the first image to be detected is input into the first detection model to detect lesions in the first image to be detected to obtain the first lesion information, and after preprocessing the second image to be detected, the second image to be detected is input into the second detection model to detect lesions in the second image to be detected to obtain the second lesion information. The first detection model is an identification model trained with lesion sample images in the white light mode, and the second detection model is an identification model trained with lesion sample images in the detection light mode. Among them, the preprocessing includes processes such as cropping invalid regions and enhancing the image display effect, such as increasing the image contrast.

[0066] Exemplarily, the first detection model and the second detection model can be neural networks trained based on deep learning algorithms. For example, the first detection model and the second detection model are detection networks of the YOLO series or the Faster-RCNN series. The first detection model and the second detection model can also include two network sub-modules to separately detect and classify the lesions. For example, it includes a detection sub-model and a classification sub-model. The detection sub-model can be a YOLO series detection model, and the classification sub-model can be an AlexNet, VGG, or ResNet classification model.

[0067] The information detection unit 200 is further configured to determine the target lesion information of the part to be examined in all the captured images to be examined based on the second lesion information obtained in the target detection light mode and the first lesion information obtained in the target white light mode. The target detection light mode is any detection light mode; the target white light mode is the white light mode adjacent to the opening time of the target detection light mode.

[0068] Among them, the target white light mode can be the n white light modes adjacent to the opening time of the target detection light mode. The specific value of n can be determined according to actual needs. For example, the specific value of n can be a preset fixed value, or a dynamic value adaptively selected by the image information determination system according to its own algorithm requirements. For example, when performing target motion tracking of the lesion, one or more image frames where the required tracked lesion appears can be selected according to the actual requirements of the tracking algorithm. At this time, the number of white light modes corresponding to the acquisition of these image frames is the actual value of n.

[0069] It should be noted that when n = 1, that is, the target white light mode only includes one white light mode, the opening time of this white light mode should be earlier than the opening time of the target detection light mode to provide the required reference historical data for the target detection light mode. When n > 1, that is, the target white light mode includes multiple white light modes, then the opening time of at least one of these white light modes is earlier than the opening time of the target detection light mode to provide the required reference historical data for the target detection light mode. In addition, the opening times of the remaining n - 1 white light modes can be earlier than or later than the opening time of the target detection light mode. For example, among the n - 1 white light modes, the opening times are all earlier than the opening time of the target detection light mode, or all later than the opening time of the target detection light mode, or some opening times are earlier than the opening time of the target detection light mode, and the remaining part is later than the opening time of the target detection light mode.

[0070] Optionally, the information detection unit 200 may process the image to be detected acquired by the image acquisition unit 100 to obtain corresponding lesion information, determine a first image to be detected and a second image to be detected adjacent to the shooting time, and obtain the target lesion information of the part to be detected in all images to be detected captured according to the first lesion information of the first image to be detected (i.e., the lesion information obtained in the target white light mode) and the second lesion information of the second image to be detected (i.e., the lesion information obtained in the target detection light mode).

[0071] Exemplarily, the image acquisition unit 100 may acquire the image to be detected captured in real time and determine the light source mode at the time of shooting; the information detection unit 200 correspondingly processes the image to be detected in real time and determines the target lesion information. The image acquisition unit 100 may also acquire the image to be detected from the set of images to be detected that have been captured and determine the light source mode at the time of shooting; the information detection unit 200 correspondingly processes the acquired image to be detected and determines the target lesion information.

[0072] Optionally, the image information determination system may further include a display unit for displaying the image to be detected and the target lesion information in each image to be detected. When the image acquisition unit 100 acquires the image to be detected captured in real time, the display unit may correspondingly display each image to be detected and the target lesion information in each image to be detected in real time, and the target lesion information in each image to be detected changes continuously as the image to be detected is acquired. In some other alternative embodiments, it is also possible to select to send the image to be detected and the target lesion information in each image to be detected to a display for display.

[0073] In the embodiments of the present application, the provided image information determination system can perform lesion detection by combining images in two and multiple light source modes, realizing a comprehensive analysis of the states of lesions at different times, improving the comprehensiveness of data for determining lesion information, and the detection light mode has a better detection effect on lesions. Therefore, the detection method combining data comprehensiveness and the detection light mode can achieve more accurate data detection, improve the credibility of the detection result, and at the same time improve the detection rate of lesions in the image.

[0074] The target lesion information includes the lesion information of at least one lesion in the part to be detected, such as Figure 3 As shown, in one embodiment, in the operation of the information detection unit 200 to determine the target lesion information, the operation on a single lesion includes: S310-S320;

[0075] S310. When the first image to be detected in the target white light mode and the second image to be detected in the target detection light mode include the same lesion, determine the first information of the same lesion in the second lesion information obtained in the target detection light mode, and determine the second information of the same lesion in the first lesion information obtained in the target white light mode.

[0076] Optionally, the information detection unit 200 may pre-determine whether the same lesion is included in the first image to be detected in the target white light mode and the second image to be detected in the target detection light mode. In the case where both include the same lesion, the lesion information of the same lesion is determined from the second lesion information obtained in the target detection light mode as the second information, and the lesion information of the same lesion is determined from the first lesion information obtained in the target white light mode as the first information.

[0077] Exemplarily, after detecting lesions in the image to be detected, the information detection unit 200 may further track the lesions to record the entire life cycle of the same lesion from appearance, survival to disappearance in consecutive images to be detected at the shooting moment, and then determine whether the same lesion is included in the first image to be detected in the target white light mode and the second image to be detected in the target detection light mode.

[0078] Among them, the process of tracking lesions includes: after detecting the lesions in the image to be detected, the information detection unit 200 may uniquely mark each lesion and perform trajectory tracking. For example, the lesion position in the next moment's image to be detected is predicted based on the lesion position in the image to be detected, and the predicted position of the lesion is obtained. Then, the lesion position detected in the next moment's image to be detected is matched with the predicted position, and the lesions with matching positions are determined to be the same lesion, and the lesion information of the lesion, such as the severity of the lesion, is updated synchronously, so as to realize the tracking of lesions in the white light mode.

[0079] In practical applications, to improve the matching reliability and achieve accurate tracking, in addition to position matching, other information matching can also be performed, such as lesion contour matching and size matching, so as to determine the same lesion when both the position and other information match. Thus, the tracking and monitoring of the birth, survival and disappearance of lesions are realized.

[0080] During the process of lesion tracking, the position of the lesion in the historical frame image to be detected can also be displayed in the current frame image to be detected and connected to the position of the lesion corresponding to the same mark (i.e., the same lesion) in the current frame image to be detected to form the tracking trajectory of the lesion. For example Figure 4 As shown, lesion 1 and lesion 2 are detected in the current frame image to be detected, and the positions of lesion 1 and lesion 2 in the historical frame image to be detected are also displayed in the current frame image to be detected (at the dotted line positions). After connection, the tracking trajectories S1 of lesion 1 and S2 of lesion 2 are respectively formed.

[0081] Optionally, the information detection unit 200 may use tracking algorithms such as the Sort algorithm, DeepSort algorithm, StrongSort algorithm, optical flow method, particle filter method, mean shift method, Siamese network algorithm, etc. to track the lesion. Specifically, it can utilize the context information of the continuous image sequence at the shooting moment to model the motion information or appearance of the lesion, predict the target motion state, and output the stable lesion position to achieve lesion tracking.

[0082] Exemplarily, after the information detection unit 200 detects the lesion in the image to be inspected, it can further compare the first lesion information of the first image to be inspected in the target white light mode with the second lesion information of the second image to be inspected in the target detection light mode, and then determine whether the first image to be inspected in the target white light mode and the second image to be inspected in the target detection light mode include the same lesion. For example, by matching the first lesion information with the second lesion information, when there is a position match between the first lesion information and the second lesion information and the remaining information of the lesions is the same, the information detection unit 200 determines that the same lesion is included. Among them, the position match can be that the position overlap degree is greater than the preset overlap degree, and the position overlap degree can be characterized by the Intersection Over Union (IOU). The position match can also be that the center distance / Euclidean distance between the lesion positions is less than the preset distance threshold, and the Euclidean distance can be calculated based on the histogram of absolute phase consistency gradients (HAPCG) between the lesions.

[0083] It should be noted that when the image information determination system displays the image to be inspected and the target lesion information in each image to be inspected through the display unit, for the first image to be inspected in the white light mode, for the lesions in the first image to be inspected whose lesion severity is greater than the preset degree, while displaying the lesion severity of the lesion, the lesion is highlighted. For example, the area where the lesion is located is outlined with a red frame, and the confidence level of the lesion severity is displayed for the operator's reference to determine the switching timing of the light source mode. Among them, the confidence level of the lesion severity can be based on the detection model to detect the lesion, and output the confidence level for the lesion severity while outputting the lesion severity.

[0084] S320. Based on the determined first information and second information, determine the target lesion information of the same lesion in all the captured images to be inspected.

[0085] Optionally, after the information detection unit 200 determines the first information and second information of the same lesion, it can compare the first information and second information of the same lesion to determine the target lesion information of the same lesion in all the captured images to be inspected according to the comparison result.

[0086] Exemplarily, when the comparison result is the same, the information detection unit 200 may determine that the first information or the second information of the same lesion is the target lesion information of the same lesion in all the captured images to be examined. When the comparison result is different, the information detection unit 200 may combine a part of the first information and a part of the second information of the same lesion as the target lesion information of the same lesion in all the captured images to be examined.

[0087] For example, when the lesion information includes the shape, color, and severity of the lesion, the image to be examined in the white light mode can restore the real environment of the part to be examined. Therefore, the credibility of the shape and color of the lesion in the first lesion information is higher than that in the second lesion information. The image to be examined in the detection light mode can highlight the surface structure of the part to be examined. Therefore, the credibility of the severity of the lesion in the second lesion information is higher than that of the lesion in the second lesion information. Based on this, when the comparison result is different, the information detection unit 200 may combine the shape and color in the first information of the same lesion and the severity of the lesion in the second information of the same lesion as the target lesion information of the same lesion in all the captured images to be examined.

[0088] In the embodiment of the present application, in the provided image information determination system, the information detection unit comprehensively determines the target lesion information of the same lesion based on the first information in the target white light mode and the second information in the target detection light mode of the same lesion, so as to make full use of the characteristics of different light source modes, reduce the misjudgment of the lesion information relying only on a single light source mode, and improve the reliability of the obtained target lesion information.

[0089] The severity of the lesion is included in the target lesion information, the first information, and the second information. Based on this, in one of the embodiments, in the operation of the information detection unit 200 to determine the target lesion information, the operation on the same lesion further includes: when the severity of the lesion in the second information is higher than that in the first information, updating the severity of the lesion of the same lesion in all the captured images to be examined according to the severity of the lesion in the second information.

[0090] Among them, the severity of the lesion may include multiple levels. Exemplarily, it may include benign and malignant, and the severity of malignant is higher than that of benign; it may also include grade one, grade two, and grade three with gradually decreasing severity.

[0091] In practical applications, the detection light mode can highlight the surface structure of the part to be examined, and the surface structure of the part to be examined directly affects the judgment of the severity of the lesion. Therefore, a higher credibility of the severity of the lesion can be obtained based on the image in the detection light mode.

[0092] Based on this, for the same lesion, the information detection unit 200 can respectively obtain the lesion severity level in the first information and the lesion severity level in the second information, and compare them. When the lesion severity in the second information is higher than that in the first information, the lesion severity of the same lesion in all the captured images to be examined is updated according to the lesion severity in the second information, that is, the lesion severity in the second information is used as the lesion severity of the same lesion in all the captured images to be examined.

[0093] In the embodiment of the present application, in the provided image information determination system, when the second lesion level of the same lesion in the second information is higher than the first lesion level in the first information, the information detection unit determines the lesion severity of the same lesion in all the captured images to be examined based on the second lesion level of the same lesion, so that the surface structure of the part to be examined can be highlighted by means of the detection light mode, and the characteristics of the lesion severity with high credibility can be used to update the lesion severity of the same lesion in all the images to be examined, thereby improving the detection rate of lesions with higher lesion severity.

[0094] In practical applications, in order to reduce the probability of missed detection, in one embodiment, as Figure 5 shown, in the operation of the information detection unit 200 to determine the target lesion information, the operations on a single lesion include: S510 - S530;

[0095] S510. Determine the mismatched lesions between the first image to be examined in the target white light mode and the second image to be examined in the target detection light mode.

[0096] Among them, the mismatched lesions include: the lesions that exist in the first image to be examined in the target white light mode but do not exist in the second image to be examined in the target detection light mode, and the lesions that do not exist in the first image to be examined in the target white light mode but exist in the second image to be examined in the target detection light mode. The embodiment of the present application does not limit the way of lesion comparison too much and can be set according to actual needs. For example, during the lesion tracking process, each lesion can be distinguished, such as by number or lesion location, etc. to distinguish different lesions. On this basis, according to the number or lesion location of each lesion, etc., it can be determined whether each lesion is the same lesion, so as to determine the mismatched lesions in the first image to be measured and the second image to be measured.

[0097] After the information detection unit 200 performs lesion detection on the image to be examined, it can further compare the first lesion information of the first image to be examined in the target white light mode and the second lesion information of the second image to be examined in the target detection light mode, and then determine the mismatched lesions between the first image to be examined in the target white light mode and the second image to be examined in the target detection light mode.

[0098] Exemplarily, the first lesion information is matched with the second lesion information. If there is a lesion position in the first lesion information that does not match any of the lesion positions in the second lesion information, or if there is a lesion position in the first lesion information that matches all of the lesion positions in the second lesion information, but at least two other pieces of information do not match, then the lesion corresponding to this lesion position in the first image to be examined is determined to be a mismatched lesion. For example, if there is a lesion position A in the first lesion information that does not match any of the lesion positions in the second lesion information, the information detection unit 200 can determine that the lesion a corresponding to the lesion position A in the first image to be examined is a mismatched lesion. Or, if there is a lesion position B in the first lesion information that matches the lesion position B' in the second lesion information, but the shape and type corresponding to the lesion position B in the first lesion information are different from the shape and type corresponding to the lesion position B' in the second lesion information, the information detection unit 200 can determine that the lesion b corresponding to the lesion position B in the first image to be examined is a mismatched lesion.

[0099] Similarly, if there is a lesion position in the second lesion information that does not match any of the lesion positions in the first lesion information, or if there is a lesion position in the second lesion information that matches the lesion position in the first lesion information, but at least two other pieces of information do not match, then the lesion corresponding to this lesion position in the second image to be examined is determined to be a mismatched lesion.

[0100] Among them, position matching can mean that the absolute position coordinates of the lesion in the image to be measured are the same or similar, or during the lesion tracking process, the theoretical position of the lesion determined based on the movement of the lesion in each image is the same or similar to the actual position coordinates of the lesion in each image. Here, being similar means that the difference degree between two position coordinates is less than a preset deviation threshold, or the position coincidence degree is higher than a preset coincidence threshold. The deviation threshold and the coincidence threshold can be numerical values or proportional values, which can be specifically set according to actual needs and are not limited here.

[0101] S520. Determine the lesion information of the mismatched lesion based on the second lesion information obtained in the target detection light mode and the first lesion information obtained in the target white light mode.

[0102] After determining the above-mentioned mismatched lesion, the information detection unit 200 correspondingly obtains the lesion information of this mismatched lesion from the second lesion information obtained in the target detection light mode and the first lesion information obtained in the target white light mode. Among them, when the mismatched lesion is a lesion in the first image to be examined, the lesion information of this mismatched lesion is the lesion information of this mismatched lesion in the first lesion information; when the mismatched lesion is a lesion in the second image to be examined, the lesion information of this mismatched lesion is the lesion information of this mismatched lesion in the second lesion information.

[0103] S530. Determine the target lesion information of the mismatched lesion in all captured images to be examined based on the lesion information of the mismatched lesion.

[0104] Optionally, after obtaining the lesion information of the mismatched lesion, the information detection unit 200 can directly use the lesion information of the mismatched lesion as the target lesion information of the mismatched lesion in all captured images to be examined. For example, continuing with the above example, if the information detection unit 200 determines that the lesion a in the first image to be examined is a mismatched lesion, then the corresponding lesion information of lesion a in the first lesion information is obtained as the target lesion information of lesion a in all captured images to be examined.

[0105] In the embodiment of the present application, in the provided image information determination system, the information detection unit determines the mismatched lesion based on the images to be examined in the target white light mode and the target detection light mode, and obtains the lesion information of the mismatched lesion as the target lesion information of the mismatched lesion in all captured images to be examined, reducing the missed detection of lesions caused by a single light source mode, improving the comprehensiveness of detection, and correspondingly increasing the detection rate of lesions.

[0106] It should be understood that in the above operation of determining or updating the target lesion information of the part to be examined in all captured images to be examined, for a single image to be examined (hereinafter referred to as the image to be processed): If the image to be processed contains the lesions recorded in the first lesion information or the second lesion information (hereinafter referred to as the lesions to be processed), then the lesion information of the lesions to be processed in the image to be processed is determined or updated. For details, reference can be made to the Figure 3 illustrated embodiments and Figure 5 the illustrated embodiments for the update operation of the lesion information of the same lesion and the mismatched lesion, which will not be elaborated here. Conversely, if the image to be processed does not contain the lesions to be processed, then the specific lesion information update operation may not be performed on the image to be processed. That is, in the operation of determining or updating the target lesion information of all captured images to be examined in the embodiment of the present application, the lesion information of the images to be examined with changed lesion information can be updated, while the images to be examined without changed lesions are not updated.

[0107] Taking an example for illustration, assume that the lesion information of three lesions to be processed, namely lesion A, lesion B, and lesion C, is recorded in the first lesion information and the second lesion information. On this basis, assume that a certain image to be processed also contains lesion A. At this time, according to Figure 3 the illustrated embodiments and Figure 5After determining the lesion information of lesion A in the illustrated embodiment, the information of lesion A in the to-be-processed image is updated based on the determined lesion information of lesion A; wherein, if the determined lesion information of lesion A is consistent with the original lesion information of lesion A in the to-be-processed image, the lesion information update operation at this time can be the verification of the lesion information (in this case, the original lesion information of lesion A in the to-be-processed image is not modified), or the original lesion information of lesion A in the to-be-processed image can be modified or replaced. On the contrary, if the to-be-processed image does not contain any of lesion A, lesion B, and lesion C (that is, none of them), then at this time, according to Figure 3 the illustrated embodiment and Figure 5 the illustrated embodiment, after determining the lesion information of lesion A, lesion B, and lesion C, the lesion information of the to-be-processed image is not updated.

[0108] The first to-be-examined image and the second to-be-examined image are to-be-examined images taken under different light source modes. Therefore, in one embodiment, as Figure 6 shown, in the operation of the image acquisition unit 100 for acquiring the first to-be-examined image and the second to-be-examined image, it includes: S610 - S630;

[0109] S610. Acquire the to-be-examined image taken of the to-be-examined part, and identify the light source mode to which the to-be-examined image belongs.

[0110] Among them, the light source mode to which the to-be-examined image belongs is the light source mode when the to-be-examined image is taken.

[0111] Optionally, the image acquisition unit 100 can acquire the to-be-examined image taken of the to-be-examined part in real time, or can also acquire the to-be-examined image from the set of to-be-examined images that have been taken of the to-be-examined part, and identify the light source mode when the to-be-examined image is taken as the light source mode to which the to-be-examined image belongs.

[0112] Exemplarily, the image acquisition unit 100 can input the acquired to-be-examined image into a pre-trained light source recognition model, so that the light source recognition model can identify the light source mode to which the to-be-examined image belongs. Among them, the light source recognition model can be a Convolutional Neural Networks (CNN) model, such as ResNet, DenseNet, or MobileNet.

[0113] Optionally, when the light source modes provided during the capture of the image to be inspected include a white light mode and a detection light mode, the light source recognition model can be a recognition model trained using a large number of images to be inspected captured in the detection light mode as training samples, and is used to output a recognition result indicating whether the light source mode to which the input image to be inspected belongs is the detection light mode. Among them, if the output recognition result is detection light, it is determined that the light source mode to which the input image to be inspected belongs is the detection light mode; if the output recognition result is non-detection light, it is determined that the light source mode to which the input image to be inspected belongs is the white light mode. The light source recognition model can also be a classification model trained using a large number of images to be inspected captured in the detection light mode and images to be inspected captured in the white light mode as training samples, and is used to directly output a classification result indicating whether the light source mode to which the input image to be inspected belongs is the detection light mode or the white light mode.

[0114] Exemplarily, different light source modes cause the images to be inspected captured in the corresponding light source modes to exhibit different image features. The image acquisition unit 100 can acquire the image features of the image to be inspected to determine the light source mode to which the image to be inspected belongs based on the image features. For example, using the color feature of the image to be inspected as the image feature, the image acquisition unit 100 can acquire the color feature of the image to be inspected and determine the light source mode to which the image to be inspected belongs according to the color feature. For example, the average value of the RGB channels of the image to be inspected is acquired as the color feature, and the light source mode to which the image to be inspected belongs is determined by means of threshold comparison. The image acquisition unit 100 can also use the Histogram of Oriented Gridients (HOG) as a feature extractor, and then use the SVM (Support Vector Machine) as a classifier to acquire the image features of the image to be inspected.

[0115] S620. When the light source mode is the white light mode, the image to be inspected is taken as the first image to be inspected.

[0116] S630. When the light source mode is the detection light mode, the image to be inspected is taken as the second image to be inspected.

[0117] After obtaining the light source mode to which the image to be inspected belongs, the image acquisition unit 100 can determine the type of the image to be inspected based on the light source mode to which the image to be inspected belongs, that is, determine whether the image to be inspected is the first image to be inspected or the second image to be inspected. Among them, when the light source mode of the part to be inspected is the white light mode, the image acquisition unit 100 takes the image to be inspected as the first image to be inspected; when the light source mode of the part to be inspected is the detection light mode, the image acquisition unit 100 takes the image to be inspected as the second image to be inspected.

[0118] In the embodiments of the present application, in the provided image information determination system, the image acquisition unit acquires the light source mode to which the image to be inspected belongs to determine the type of the image to be inspected, so as to accurately acquire the first image to be inspected in the target white light mode and the second image to be inspected in the target detection light mode subsequently, thereby improving the reliability of the obtained target lesion information.

[0119] The above image information determination system can not only determine the target lesion information based on the lesion information of the first image to be inspected and the second image to be inspected adjacent in the shooting time, but also determine the target lesion information based on the lesion information of the second image to be inspected in the detection light mode adjacent in the opening time. Based on this, in one embodiment, as Figure 7 shown, the information detection unit 200 is further configured to execute: S710-S720;

[0120] S710. Acquire the reference lesion information of the part to be inspected in the reference detection light mode; the reference detection light mode is the detection light mode adjacent in the opening time to the target detection light mode and after the target detection light mode.

[0121] Optionally, after obtaining the second detection information in the second image to be inspected in the target detection light mode, the information detection unit 200 may determine the detection light mode adjacent in the opening time to the target detection light mode and after the target detection light mode as the reference detection light mode, and acquire the lesion information of the second image to be inspected in this reference detection light mode as the reference lesion information.

[0122] As an optional embodiment of the present application, the reference detection light mode may include one or more detection light modes. That is, one or more detection light modes with the opening time after the target detection light mode and adjacent to the target detection light mode may be used as the reference detection light mode.

[0123] It should be understood that in the embodiments of the present application, there may be one or more white light modes between the detection light modes adjacent in the opening time, which does not affect the adjacent relationship between the two detection light modes. For example, refer to Figure 2 , Figure 2 as shown, there are 29 images taken in the white light mode between the two detection light modes shown, but the two detection light modes are still adjacent.

[0124] S720. Determine the target lesion information of the part to be inspected in all the images to be inspected captured according to the reference lesion information in the reference detection light mode and the second lesion information in the target detection light mode.

[0125] Optionally, the information detection unit 200 may pre-determine whether the same lesion is included in the second image to be detected in the reference detection light mode and the second image to be detected in the target detection light mode. In the case where the same lesion is included, the lesion information of the same lesion is determined from the reference lesion information obtained in the reference detection light mode, and the lesion information of the same lesion is determined from the second lesion information obtained in the target detection light mode. Then, based on the lesion information of the same lesion in the reference lesion information and the second lesion information, the target lesion information of the same lesion in all the captured images to be detected is determined. Exemplarily, the information detection unit 200 may determine whether the same lesion is included based on lesion tracking, or may also determine whether the same lesion is included based on the position overlap degree (such as the intersection over union). For the specific process, refer to the relevant content in the foregoing embodiments, which will not be elaborated herein.

[0126] In the embodiments of the present application, in the provided image information determination system, the target lesion information is comprehensively determined based on the lesion information in the reference detection light mode and the target detection light mode with adjacent start times, and the forward correction and backward update of the lesion information in the image are synchronously performed, realizing the unified maintenance of the target lesion information on the entire time line of the image.

[0127] The target lesion information includes the severity of the lesion. Based on this, in one embodiment, as Figure 8 shown, the information detection unit 200 is further configured to perform the following steps: S810 - S820;

[0128] S810. When the same lesion is included in the second image to be detected in the reference detection light mode and the second image to be detected in the target detection light mode, compare the lesion severities of the same lesion in the reference lesion information and the second lesion information.

[0129] For the same lesion, the information detection unit 200 may respectively obtain the lesion severity of the same lesion in the reference lesion information and the lesion severity of the same lesion in the second lesion information, and compare the lesion severities of the same lesion in the reference lesion information and the second lesion information to obtain a comparison result of which lesion grade is higher.

[0130] S820. Determine the lesion severity of the same lesion in all the captured images to be detected according to the comparison result.

[0131] The detection of the optical mode is more sensitive to the severity of the lesion. At the same time, in order to avoid missed detection or missing important lesion conditions, in practical applications, the higher the severity of the lesion obtained in the detection optical mode, the higher the acceptance of the severity result of the lesion. Optionally, in the case where the comparison result is that the severity of the same lesion in the reference lesion information is higher than the severity of the same lesion in the second lesion information, the information detection unit 200 determines the severity of the same lesion in all the captured images to be inspected according to the lesion information of the same lesion in the reference lesion information. Exemplarily, the information detection unit 200 may use the severity of the same lesion in the reference lesion information as the severity of the same lesion in all the captured images to be inspected.

[0132] Since the system operation tends to be stable with the accumulation of the running time. Therefore, the credibility of the lesion information obtained in the post-detection optical mode is higher. Optionally, in the case where the comparison result is that the severity of the same lesion in the reference lesion information is equal to the severity of the same lesion in the second lesion information, the information detection unit 200 determines the severity of the same lesion in all the captured images to be inspected according to the severity of the same lesion in the reference lesion information. Exemplarily, the information detection unit 200 may use the severity of the same lesion in the reference lesion information as the severity of the same lesion in all the captured images to be inspected.

[0133] In the embodiment of the present application, in the provided image information determination system, based on the comparison of the lesion information of the same lesion in the detection optical modes with adjacent activation times, the target lesion information is determined, taking into account the detection characteristics of the detection optical mode and the stability of the system operation, and the reliability of the obtained target lesion information is improved.

[0134] In the case where the comparison result is that the severity of the same lesion in the reference lesion information is lower than the severity of the same lesion in the second lesion information, in order to avoid missed detection of the lesion, in one embodiment, as Figure 9 shown, the information detection unit 200 is further configured to perform the following steps: S910-S920;

[0135] S910. In the case where the comparison result is that the severity of the same lesion in the reference lesion information is lower than the severity of the same lesion in the second lesion information, obtain the severity of the same lesion in the reference lesion information in the subsequent reference detection optical mode; the subsequent reference detection optical mode is a detection optical mode that is adjacent to the activation time of the reference detection optical mode and is after the reference detection optical mode.

[0136] Optionally, in the case where the lesion grade of the same lesion in the reference lesion information is lower than the lesion severity of the same lesion in the second lesion information in the comparison result, the information detection unit 200 further determines a subsequent reference detection light mode that is adjacent to the reference detection light mode in time sequence and is located after the target detection light mode, and determines whether the same lesion is included in the second image to be detected in the subsequent reference detection light mode. If it is included, the information detection unit 200 obtains the lesion severity of the same lesion in the reference lesion information of the second image to be detected in the subsequent reference detection light mode.

[0137] For example, the lesion severity of lesion M in the reference lesion information corresponding to the reference detection light mode is benign, while the lesion severity of lesion M in the second lesion information corresponding to the target detection light mode is malignant. The information detection unit 200 further obtains the reference lesion information corresponding to the subsequent reference detection light mode that is adjacent to the reference detection light mode in time sequence, and obtains the lesion severity of lesion M therefrom.

[0138] S920. If the lesion severity of the same lesion in the reference lesion information in at least a preset number of consecutive subsequent reference detection light modes is lower than the lesion severity of the same lesion in the second lesion information, the lesion severity of the same lesion in all the captured images to be detected is determined according to the lesion severity of the same lesion in the reference lesion information in the latest subsequent reference detection light mode.

[0139] Optionally, the information detection unit 200 can obtain the lesion severity of the same lesion in the reference lesion information corresponding to a plurality of consecutive subsequent reference detection light modes whose start times are after the reference detection light mode. If the lesion severity is lower than the lesion severity of the same lesion in the second lesion information in at least a preset number of consecutive times, the information detection unit 200 determines the lesion severity of the same lesion in all the captured images to be detected according to the lesion severity of the same lesion in the reference lesion information in the latest subsequent reference detection light mode.

[0140] For example, continuing the above example, the lesion severity of lesion M in the reference lesion information corresponding to the reference detection light mode is benign, while the lesion severity of lesion M in the second lesion information corresponding to the target detection light mode is malignant, and the information detection unit 200 obtains that the lesion severity of lesion M in 5 consecutive subsequent reference detection light modes (greater than the preset threshold of 3) is all benign. The information detection unit 200 then determines that the lesion severity of lesion M in the reference lesion information in the last subsequent reference detection light mode is the lesion severity of lesion M in all the captured images to be detected.

[0141] In the embodiment of the present application, in the provided image information determination system, when the severity of a lesion in the subsequent reference detection light mode for a continuous preset number of times of the same lesion is lower than that in the previous reference detection light mode, the latter is taken as the standard to exclude false detections caused by unstable systems in the early stage and improve the accuracy of lesion detection.

[0142] Corresponding to the embodiment of the above image information determination system, the embodiment of the present application further provides an image information determination method. This embodiment is illustrated by taking this method as applied to the above image processing device. As Figure 10 shown, the method includes the following steps: S1010 - S1030;

[0143] S1010. Obtain a first image to be detected of the part to be detected under multiple white light modes, and obtain a second image to be detected of the part to be detected under at least one detection light mode;

[0144] S1020. Process the first image to be detected to determine first lesion information of the lesions existing in the part to be detected, and process the second image to be detected to determine second lesion information of the lesions existing in the part to be detected;

[0145] S1030. Based on the second lesion information obtained under the target detection light mode and the first lesion information obtained under the target white light mode, determine the target lesion information of the lesions existing in the part to be detected during the detection light mode and the white light mode; the target detection light mode is any one of the detection light modes; the target white light mode is the white light mode adjacent to the target detection light mode.

[0146] For the specific principle, implementation process details and beneficial effects of the image information determination method provided in the embodiment of the present application, please refer to the relevant description of the embodiment of the above image information determination system, which will not be elaborated here.

[0147] As an embodiment of the present application, on the basis of the Figure 10 shown embodiment, the image processing device can also implement the embodiments of the above various image information determination systems in the form of a method flow. For the specific principle, implementation process details and beneficial effects, please refer to the relevant description of the embodiment of the above image information determination system, which will not be elaborated here.

[0148] For the convenience of those skilled in the art, the following details the image information determination method implemented by the image information determination system provided in the present application. As Figure 11 shown, the method may include:

[0149] S1101. Obtain images to be detected under different light source modes;

[0150] S1102. Determine the light source mode of the image to be detected;

[0151] S1103. Detect and track the lesions in the image to be inspected in the white light mode, and obtain the first lesion information; the first lesion information includes the lesion location and the lesion severity level of the lesions in the image to be inspected.

[0152] S1104. Detect the lesions in the image to be inspected in the detection light mode, and obtain the second lesion information; the second lesion information includes the lesion location and the lesion severity level of the lesions in the image to be inspected.

[0153] S1105. Determine the target white light mode and the target detection light mode with adjacent start times, and obtain the first lesion information in the target white light mode and the second lesion information in the target detection light mode.

[0154] S1106. For the same lesion in the first lesion information and the second lesion information, use the lesion location and the lesion severity level in the second lesion information as the target lesion information of the same lesion, and display it in the image to be inspected.

[0155] S1107. For the non - same lesions in the first lesion information and the second lesion information, synchronously display the lesion location and the lesion severity level of the non - same lesions in the image to be inspected, and use them as the target lesion information of the non - same lesions.

[0156] As Figure 1 shown, the image information determination system includes an image acquisition unit 100 and an information detection unit 200. The image acquisition unit 100 communicates with the endoscope. The endoscope acquires the image to be inspected during the periodic automatic switching of the light source mode. The image acquisition unit 100 can obtain the image to be inspected in different light source modes collected from the endoscope in real time, and the information detection unit 200 performs light source mode detection on the image to be inspected to determine the light source mode of the image to be inspected.

[0157] Among them, the image acquisition unit 100 can input the image to be inspected into a pre - trained light source recognition model to identify the light source mode to which the image to be inspected belongs by the light source recognition model. The image acquisition unit 100 can also obtain the average value of the RGB channels of the image to be inspected, and compare the average value of the RGB channels with a threshold to determine the light source mode of the image to be inspected.

[0158] After determining the light source mode of the image to be inspected, the information detection unit 200 can input the image to be inspected in the white light mode into the first detection model for lesion detection, obtain the lesion location and lesion severity of the lesion in the image to be inspected as the first lesion information, and track the lesion in the image to be inspected in the continuous white light mode, and synchronously update the lesion location and lesion severity of the same lesion in the image to be inspected. The information detection unit 200 can input the image to be inspected in the detection light mode into the second detection model for lesion detection, and obtain the lesion location and lesion severity of the lesion in the image to be inspected as the second lesion information.

[0159] The information detection unit 200 obtains the first lesion information of the image to be inspected in the target white light mode and the second lesion information of the image to be inspected in the target detection light mode with adjacent start times, and compares the first lesion information with the second lesion information to determine whether the same lesion is included in the first lesion information and the second lesion information. In the case of using the position of the minimum bounding rectangle of the lesion to represent the lesion location, the information detection unit 200 can obtain the IOU between the lesion location in the first lesion information and the lesion location in the second lesion information, and determine that the lesion location in the first lesion information corresponds to the same lesion as the lesion location in the second lesion information when the IOU is greater than the preset overlap degree, that is, the same lesion is included in the first lesion information and the second lesion information.

[0160] For the same lesion in the first lesion information and the second lesion information, the information detection unit 200 can use the lesion location and lesion severity level in the second lesion information as the target lesion information of the same lesion and display it in the image to be inspected including the same lesion; for the non - same lesion in the first lesion information and the second lesion information, the information detection unit 200 can synchronously display the lesion location and lesion severity level of the non - same lesion in the image to be inspected including the non - same lesion and use it as the target lesion information of the non - same lesion.

[0161] It should be understood that although the steps in the flowcharts involved in the above - mentioned embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above - mentioned embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0162] In one embodiment, the present application further provides an image processing device, which may be a terminal, and its internal structure diagram may be as shown in Figure 12 shown. The image processing device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the image processing device is used to provide computing and control capabilities. The memory of the image processing device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the image processing device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an image information determination method. The display screen of the image processing device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the image processing device may be a touch layer covered on the display screen, or a button, a trackball, or a touchpad provided on the housing of the image processing device, or an external keyboard, touchpad, or mouse, etc.

[0163] Those skilled in the art can understand that Figure 12 the structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the image processing device to which the solution of the present application is applied. The specific image processing device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0164] In one embodiment, as shown in Figure 13 shown, an endoscope system is further provided, including an endoscope 10, a display 20, a light source device 30, and the image information determination system 40 according to any one of the above.

[0165] Among them, the endoscope 10, the display 20, and the light source device 30 are respectively connected to the image information determination system 40.

[0166] The endoscope 10 is connected to the image acquisition unit 100 in the image information determination system 40 to acquire the to-be-examined image captured by the endoscope 10; the light source device 30 is disposed at the end of the endoscope 10 and is used to provide a light source mode including a white light mode and a detection light mode; the display 20 is used to display the acquired to-be-examined image and the target lesion information of each to-be-examined image.

[0167] For the specific structure and function of the image information determination system 40 in the embodiments of the present application, please refer to the specific description in the foregoing embodiments, and details are not described herein again.

[0168] In one embodiment, an image processing device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the steps of any one of the above image information determination methods are implemented.

[0169] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above image information determination methods are implemented.

[0170] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps of any one of the above image information determination methods are implemented.

[0171] Those of ordinary skill in the art can understand that all or part of the processes in the above embodiment methods can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0172] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0173] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. An image information determination system, characterized in that, the system includes: an image acquisition unit for acquiring a first image to be inspected of the part to be inspected taken under multiple white light modes, and acquiring a second image to be inspected of the part to be inspected taken under at least one detection light mode; an information detection unit for processing the first image to be inspected to determine first lesion information in the first image to be inspected, and processing the second image to be inspected to determine second lesion information in the second image to be inspected; the information detection unit is further configured to determine target lesion information of the part to be inspected in all the captured images to be inspected based on the second lesion information obtained under the target detection light mode and the first lesion information obtained under the target white light mode; the target detection light mode is any one of the detection light modes; the target white light mode is the white light mode adjacent to the opening time of the target detection light mode.

2. The system according to claim 1, characterized in that, the target lesion information includes lesion information of at least one lesion in the part to be inspected; in the operation of determining the target lesion information by the information detection unit, the operation on a single lesion includes: in the case where the first image to be inspected under the target white light mode and the second image to be inspected under the target detection light mode include the same lesion, determining second information of the same lesion in the second lesion information obtained under the target detection light mode, and determining first information of the same lesion in the first lesion information obtained under the target white light mode; based on the determined first information and second information, determining the target lesion information of the same lesion in all the captured images to be inspected.

3. The system according to claim 2, characterized in that, the target lesion information, the first information and the second information all include the severity of the lesion; in the operation of determining the target lesion information by the information detection unit, the operation on the same lesion further includes: when the severity of the lesion in the second information is higher than the severity of the lesion in the first information, updating the severity of the lesion of the same lesion in all the captured images to be inspected according to the severity of the lesion in the second information.

4. The system according to claim 1, characterized in that, in the operation of determining the target lesion information by the information detection unit, the operation on a single lesion includes: determining mismatched lesions between the first image to be inspected under the target white light mode and the second image to be inspected under the target detection light mode; determining the lesion information of the mismatched lesions according to the second lesion information obtained under the target detection light mode and the first lesion information obtained under the target white light mode; determining the target lesion information of the mismatched lesions in all the captured images to be inspected according to the lesion information of the mismatched lesions.

5. The system according to claim 1, characterized in that, The operation of the image acquisition unit for acquiring the first image to be detected and the second image to be detected includes: Acquiring the image to be detected taken of the part to be detected, and identifying the light source mode to which the image to be detected belongs; When the light source mode is the white light mode, the image to be detected is taken as the first image to be detected; When the light source mode is the detection light mode, the image to be detected is taken as the second image to be detected.

6. The system according to any one of claims 1-5, characterized in that the information detection unit is further configured to: Acquire the reference lesion information of the part to be detected in the reference detection light mode; the reference detection light mode is the detection light mode that is adjacent to the opening time of the target detection light mode and is after the target detection light mode; Determine the target lesion information of the part to be detected in all the acquired images to be detected according to the reference lesion information in the reference detection light mode and the second lesion information in the target detection light mode.

7. The system according to claim 6, characterized in that the target lesion information includes the severity of the lesion; the information detection unit is further configured to: In the case that the second image to be detected in the reference detection light mode and the second image to be detected in the target detection light mode include the same lesion, compare the severity of the same lesion in the reference lesion information and the second lesion information; Determine the severity of the same lesion in all the acquired images to be detected according to the comparison result.

8. The system according to claim 7, characterized in that the information detection unit is further configured to: In the case that the comparison result is that the severity of the same lesion in the reference lesion information is higher than or equal to the severity of the same lesion in the second lesion information, determine the severity of the same lesion in all the acquired images to be detected according to the severity of the same lesion in the reference lesion information.

9. The system according to claim 7, characterized in that the information detection unit is further configured to: In the case that the comparison result is that the severity of the same lesion in the reference lesion information is lower than the severity of the same lesion in the second lesion information, acquire the severity of the same lesion in the reference lesion information in the subsequent reference detection light mode; The subsequent reference detection light mode is the detection light mode that is adjacent to the opening time of the reference detection light mode and is after the reference detection light mode; If the severity of the same lesion in the reference lesion information in at least a preset number of consecutive subsequent reference detection light modes is lower than the severity of the same lesion in the second lesion information, determine the severity of the same lesion in all the acquired images to be detected according to the severity of the same lesion in the latest subsequent reference detection light mode.

10. An image information determination method, characterized in that the method includes: Obtain a first image to be inspected of the part to be inspected taken in multiple white light modes, and obtain a second image to be inspected of the part to be inspected taken in at least one detection light mode; Process the first image to be inspected to determine first lesion information of the first image to be inspected, and process the second image to be inspected to determine second lesion information of the second image to be inspected; Based on the second lesion information obtained in the target detection light mode and the first lesion information obtained in the target white light mode, determine the target lesion information of the part to be inspected in all the captured images to be inspected; the target detection light mode is any one of the detection light modes; the target white light mode is the white light mode adjacent to the opening time of the target detection light mode.

11. An image processing device, characterized in that, it includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps of the method according to claim 10 are implemented.

12. An endoscope system, characterized in that, it includes an endoscope, a display, a light source device, and an image information determination system according to any one of claims 1 to 9.

Images