Image processing method, electronic device and endoscope system

Abstract

The invention provides an image processing method, electronic equipment and an endoscope system. The image processing method comprises the following steps: obtaining the average brightness of a specified area in an image according to the image to be processed; a brightness threshold value is obtained, and the brightness threshold value is related to the average brightness; according to the brightness threshold, binarization processing is carried out on the to-be-processed image to obtain a binarized image, the binarized image comprises a first area, and the first area is a set of pixel points of which the brightness of pixels of the to-be-processed image is greater than the brightness threshold; obtaining a center coordinate of the first area according to the position of the first area; selecting a statistical region from the binarized image according to the center coordinates and the imaging parameters; traversing pixel points in the statistical area to obtain effective view edge pixel points of the binarized image; and obtaining an effective area of the to-be-processed image according to the effective view edge pixel points. According to the invention, the effective area in the to-be-processed image is identified, so that the ISP operation speed is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, an electronic device, and an endoscope system. Background Art

[0002] In the medical field, endoscopic diagnosis and treatment systems are increasingly used. Generally speaking, an endoscopic camera system includes a camera, an endoscope, an adapter, an endoscopic camera host, a light source module, and a display. Among them, the light source module provides good lighting conditions, the camera captures images through the endoscope, and transmits the captured image data to the endoscopic camera host through a transmission cable. The endoscopic camera host processes the video information collected by the camera, and displays the video information processed by the camera host through the display, so that the display displays the endoscopic image captured by the camera.

[0003] In the use scenarios of endoscope camera systems, in order to meet the needs of various types of surgeries, the camera needs to be equipped with endoscopes and adapters of different types and focal lengths. Due to the differences in the calibers of different endoscopes and adapters, the actual imaging field of view is also different, so the actual amount of effective data involved in the image processing process is also different.

[0004] The data of the invalid area at the edge of the image affects the exposure brightness, increases unnecessary calculations in the data processing process, affects the performance of the ISP module, and increases the system power consumption; and due to process errors in the endoscope and adapter, the effective area will deviate from the center of the actual captured image, resulting in the effective area not being displayed in the center of the screen.

[0005] Traditional effective area detection methods generally distinguish effective areas from invalid areas by setting thresholds, or identify effective area edges by edge detection operators; however, the threshold method and edge detection method have certain limitations. Although the threshold method is simple, it is very difficult to select the threshold, and it is often difficult to determine the appropriate threshold according to different scenarios; edge detection operators often need to cache multiple rows of data and perform matrix operations, which requires large amounts of calculations and a complex calculation process. In addition, areas with drastic changes in the effective area of ​​the image will affect the detection results and cause edge misjudgments. Summary of the invention

[0006] In order to solve at least one of the above technical problems, the present disclosure provides an image processing method.

[0007] According to one aspect of the present disclosure, there is provided an image processing method, comprising: Obtaining the average brightness of a designated area in the image according to the image to be processed; Obtaining a brightness threshold, wherein the brightness threshold is related to the average brightness; Binarizing the image to be processed according to the brightness threshold to obtain a binary image, wherein the binary image includes a first region, wherein the first region is a set of pixel points of the image to be processed whose brightness is greater than the brightness threshold; Obtaining the center coordinates of the first area according to the position of the first area; Selecting a statistical region from the binary image according to the center coordinates and imaging parameters; Traversing the pixel points in the statistical area to obtain the effective visual field edge pixel points of the binary image; The effective area of ​​the image to be processed is obtained according to the effective visual field edge pixel points.

[0008] According to the image processing method of at least one embodiment of the present disclosure, the image to be processed includes N frame images arranged in chronological order, the designated area of ​​the first frame image is the image to be processed; the designated area of ​​the i-th frame image is the valid area obtained by the i-1-th frame image, wherein i is greater than or equal to 2 and less than or equal to N, and N is a natural number greater than or equal to 2.

[0009] According to the image processing method of at least one embodiment of the present disclosure, the brightness threshold ;in, is the average brightness, is a limiting parameter related to the average brightness, is greater than 0 and less than 1, wherein the limiting parameter increases with the increase of the average brightness.

[0010] According to the image processing method of at least one embodiment of the present disclosure, the limiting parameter ; Among them, width is the brightness data bit width.

[0011] According to the image processing method of at least one embodiment of the present disclosure, obtaining the center coordinates of the first area according to the position of the first area includes: Get the original center coordinate point of the binary image; Obtain, according to the original center coordinate point, a pixel point with a pixel value of 1 that is farthest in a preset angle direction passing through the original center coordinate point; Obtain a first direction offset and a second direction offset according to the coordinates of the farthest pixel point whose pixel value is 1, wherein the first direction is perpendicular to the second direction; The center coordinates of the first area are obtained according to the original center coordinate point, the first direction offset and the second direction offset.

[0012] According to the image processing method of at least one embodiment of the present disclosure, the first direction and the second direction are both directions of the symmetry axis of the image to be processed.

[0013] According to at least one embodiment of the image processing method of the present disclosure, when the effective area is a circle, traversing the pixel points in the statistical area to obtain the effective field of view edge pixel points of the binary image includes: Taking the central coordinate as the center of the circle, calculating the maximum distance between a point with a pixel value of 1 and the center of the circle in directions of multiple set angles; Count the maximum distance values ​​of multiple set angles, and select the maximum distance value with the largest number as the effective area radius; The effective visual field edge pixel points of the binary image are obtained according to the central coordinates and the effective area radius.

[0014] According to at least one embodiment of the image processing method of the present disclosure, when the effective area is a polygon including two mutually perpendicular axes of symmetry, traversing the pixel points in the statistical area to obtain the effective field of view edge pixel points of the binary image includes: Obtaining a horizontal center axis and a vertical center axis passing through the center coordinates; Traverse the pixel points in the statistical area, calculate the vertical distance between the pixel point with a pixel value of 1 and the horizontal center axis, and calculate the horizontal distance between the pixel point with a pixel value of 1 and the vertical center axis; Count the maximum horizontal distance value of the horizontal distances of each pixel point in the selected row, and select the maximum horizontal distance with the largest number as the vertical boundary; count the maximum vertical distance value of the vertical distances of each pixel point in the selected column, and select the maximum vertical distance with the largest number as the horizontal boundary; The effective visual field edge pixel points of the binary image are obtained according to the horizontal boundary and the vertical boundary.

[0015] According to at least one embodiment of the image processing method of the present disclosure, obtaining the effective area of ​​the image to be processed according to the effective visual field edge pixel points includes: Obtaining an effective visual field area according to the effective visual field edge pixel points, wherein the pixel point values ​​of the pixel points in the binary image corresponding to the effective visual field area are set to 1, and the pixel point values ​​of other pixel points in the binary image are set to 0; and The image to be processed is ANDed with the corresponding pixel points in the binary image to obtain the effective area of ​​the image to be processed.

[0016] According to the image processing method of at least one embodiment of the present disclosure, when displaying the image to be processed, the center coordinates are aligned with the center point of the display.

[0017] According to another aspect of the present disclosure, there is provided an electronic device, comprising: a memory storing execution instructions; and A processor executes the execution instruction stored in the memory, so that the processor executes the above-mentioned image processing method.

[0018] According to another aspect of the present disclosure, an endoscope system is provided, which includes the above-mentioned electronic device.

[0019] The beneficial effects of the present disclosure are as follows: the image processing method of the present disclosure can dynamically adjust the metering area by accurately identifying the effective area of ​​the image, improve the performance of algorithm modules related to brightness such as exposure, and solve the problem of high exposure gain caused by inconsistency between the metering area and the actual area, thereby causing excessive image noise; effectively remove the "ghost" of the invalid area on the edge of the image caused by packaging process errors and reflections from instruments such as endoscopes; and, by identifying the effective area, correct the image that deviates from the center of the display screen; perform deviation correction processing on the captured image to solve the problem of the image not being displayed in the center of the screen due to process errors of the endoscope and the adapter, so that the target image can be displayed in the center of the screen for easy observation by the operator. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0021] Figure 1 is a flowchart of an image processing method according to one embodiment of the present disclosure.

[0022] Figure 2 is a flow chart of obtaining center coordinates according to one embodiment of the present disclosure.

[0023] Figure 3 It is a schematic diagram of obtaining center coordinates according to one embodiment of the present disclosure.

[0024] Figure 4 is a schematic diagram of obtaining center coordinates according to another embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] The present disclosure is further described in detail below in conjunction with the accompanying drawings and implementations. It is understood that the specific implementations described herein are only used to explain the relevant content, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0026] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0027] Unless otherwise specified, the exemplary embodiments / embodiments shown will be understood as providing exemplary features of various details of some ways in which the technical concept of the present disclosure can be implemented in practice. Therefore, unless otherwise specified, the features of the various embodiments / embodiments can be combined, separated, interchanged and / or rearranged without departing from the technical concept of the present disclosure.

[0028] The use of cross-hatching and / or shading in the accompanying drawings is generally used to make the boundaries between adjacent components clear. As such, unless otherwise specified, the presence or absence of cross-hatching or shading does not convey or indicate any preference or requirement for the specific materials, material properties, dimensions, proportions, commonalities between the components shown, and / or any other characteristics, attributes, properties, etc. of the components. In addition, in the accompanying drawings, the sizes and relative sizes of the components may be exaggerated for clarity and / or descriptive purposes. When the exemplary embodiments can be implemented differently, the specific process sequence can be performed in a different order than described. For example, two successively described processes can be performed substantially simultaneously or in an order opposite to the described order. In addition, the same figure numbers represent the same components.

[0029] The terms used herein are for the purpose of describing specific embodiments, and are not intended to be restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups. It should also be noted that, as used herein, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or the values ​​provided that will be recognized by those of ordinary skill in the art.

[0030] Figure 1 is a flowchart of an image processing method according to one embodiment of the present disclosure.

[0031] like Figure 1 As shown, the image processing method of the present invention can accurately obtain the effective area in the image when used. Accordingly, when displaying the image, the image in the effective area is displayed normally, and the image outside the effective area may not be displayed or displayed in black. Therefore, the image processing method of the present invention can also be called an effective area recognition method for an image.

[0032] Specifically, the image processing method of the present disclosure may include: S110, obtaining the average brightness of a designated area in an image according to the image to be processed; S120, obtaining a brightness threshold, wherein the brightness threshold is related to the average brightness; S130, binarizing the image to be processed according to the brightness threshold to obtain a binary image, wherein the binary image includes a first area, wherein the first area is a set of pixel points of the image to be processed whose brightness is greater than the brightness threshold; S140, obtaining the center coordinates of the first area according to the position of the first area; S150, selecting a statistical area from the binary image according to the center coordinates and imaging parameters; S160, traversing the pixel points in the statistical area to obtain the effective field of view edge pixel points of the binary image; S170, obtaining the effective area of ​​the image to be processed according to the effective field of view edge pixel points.

[0033] The image processing method disclosed herein will be described in detail below in conjunction with specific implementation forms.

[0034] S110, obtaining the average brightness of a designated area in the image according to the image to be processed.

[0035] In the present disclosure, different methods can be used to obtain the average brightness of a specified area in an image according to different formats of the image to be processed. Specifically, when the format of the image to be processed is RGB format, the brightness value Y of the pixel point of the image to be processed can be calculated by the following formula: Y=0.299R+0.587G+0.114B; wherein RGB are the red, green and blue values ​​of the pixel point of the image to be processed, respectively, and the value range is [0, 255]. When the image to be processed is a grayscale Bayer image, the Y component of the pixel point of the Bayer image is the brightness value of the pixel point of the image to be processed.

[0036] Correspondingly, the average brightness Y avg It can be calculated by the following formula: ;in, is the sum of the brightness values ​​of the pixels in the specified area of ​​the image to be processed; is the number of pixels in the specified area of ​​the image to be processed.

[0037] In some embodiments, the image to be processed may be a frame of image. Of course, the image to be processed may also be an image in video data, in which case the image to be processed includes N frames of image arranged in chronological order, where N is an integer greater than or equal to 2. Accordingly, the designated area of ​​the first frame of image is the image to be processed; the designated area of ​​the i-th frame of image is the effective area obtained by the i-1-th frame of image, where i is greater than or equal to 2 and less than or equal to N. That is to say, when the image processing method of the present disclosure is used, it is possible to make the effective area of ​​the image converge as quickly as possible through iterative operations, and be the same as or similar to the ideal effective area.

[0038] In a specific embodiment, the image to be processed may be an 8-bit RGB data source.

[0039] In addition, before brightness calculation, the image may be preprocessed, which includes eliminating abnormal pixel data such as image noise and sensor bad pixels.

[0040] S120: Obtain a brightness threshold, where the brightness threshold is related to the average brightness.

[0041] Specifically, the brightness threshold ;in, is the average brightness, is a limiting parameter related to the average brightness, where Greater than 0 and less than 1, preferably, wherein the limiting parameter Increases with increasing average brightness.

[0042] Specifically, the limiting parameters It can be a piecewise function, which is divided into four segments according to different average brightness. In the present disclosure, by setting the piecewise function, the influence of the highlight pixels around the effective area on the image processing process can be effectively suppressed, that is, when there are multiple highlight pixels around the effective area, these highlight pixels increase the average brightness of the image, by setting a higher brightness threshold, these pixels can be set to 0, thereby improving the accuracy of the effective area recognition of the present disclosure. Overall, by The setting makes the decision of the binarization threshold more reasonable.

[0043] In a specific embodiment, the parameter It can be calculated by the following formula: .

[0044] Among them, width is the brightness data bit width.

[0045] Taking width as 8 as an example, the maximum brightness value Y is 256: .

[0046] S130. Binarize the image to be processed according to a brightness threshold to obtain a binary image, where the binary image includes a first region, wherein the first region is a set of pixel points of the image to be processed whose brightness is greater than the brightness threshold.

[0047] Specifically, the pixel values ​​of the pixels in the binary image can be 0 or 1 (or 255); thus, a binary image with the same size as the image to be processed can be generated according to the brightness values ​​and brightness threshold of the pixels of the image to be processed, wherein the binary image includes a first area and a second area, the first area is a set of pixels of the image to be processed whose brightness is greater than the brightness threshold; the second area is a set of pixels of the image to be processed whose brightness is less than or equal to the brightness threshold; accordingly, the pixel values ​​of the pixels in the first area are 1 (or 255), and the pixel values ​​of the pixels in the second area are 0.

[0048] S140. Obtain the center coordinates of the first area according to the position of the first area.

[0049] In the present disclosure, it is necessary to determine whether the center of the first area coincides with the center of the image to be processed (or the binary image). Accordingly, if the center of the first area coincides with the center of the image to be processed, no adjustment is required when the processed image to be processed is subsequently displayed. If the center of the first area does not coincide with the center of the image to be processed, when the processed image to be processed is displayed, the center coordinates of the first area can be aligned with the center point of the display, so that the effective area of ​​the image to be processed can be displayed in the central area of ​​the display, thereby improving the display effect of the endoscopic image.

[0050] Figure 2 is a flow chart of obtaining center coordinates according to one embodiment of the present disclosure.

[0051] Specifically, if Figure 2 As shown, the center coordinates of the first area are obtained according to the position of the first area (for example, the position in the image to be processed), including: S141, obtaining the original center coordinate point of the binary image; S142, obtaining the farthest pixel point with a pixel value of 1 in a preset angle direction passing through the original center coordinate point according to the original center coordinate point; S143, obtaining a first direction offset and a second direction offset according to the coordinates of the farthest pixel point with a pixel value of 1, wherein the first direction is perpendicular to the second direction; S144, obtaining the center coordinates of the first area according to the original center coordinate point, the first direction offset and the second direction offset.

[0052] In S141, the original center coordinate point of the binary image can be obtained according to the center coordinate point of the image to be processed; the full image size of the image to be processed is L total *W total , the image center coordinate point is (x0, y0). Where x0=L total / 2,y0=W total / 2. Specifically, taking the full image size of 1920*1080 as an example, (x0, y0) can be (960, 540).

[0053] In S142, starting from the original center coordinate point, with the horizontal right direction as the 0° direction, rotating counterclockwise, select the pixel points with a pixel value of 1 that are farthest in the 0°, 180°, 90°, and 270° directions.

[0054] In S143, the coordinates of the pixel points with a pixel value of 1 that are farthest in the 0°, 180°, 90°, and 270° directions are (x1, y1), (x2, y2), (x3, y3), and (x4, y4) respectively; calculate the distances between the coordinates of these four pixel points and the original center coordinate point respectively, and obtain d1, d2, d3, and d4. Then, based on the distances d1, d2, d3, and d4, the first direction offset and the second direction offset can be obtained, where both the first direction and the second direction are the directions of the symmetry axes of the image to be processed. Taking the image to be processed as a rectangle as an example, as Figure 3 shown, the first direction is the width direction, that is, the horizontal direction, and the second direction is the height direction.

[0055] Correspondingly, the first direction offset is (d1 - d2) / 2; the offset of the second direction is (d3 - d4) / 2.

[0056] In S144, the center coordinates (x0’, y0’) of the first region can be calculated by the following formula: x0’ = x0 + (d1 - d2) / 2; y0’ = y0 + (d3 - d4) / 2.

[0057] Specifically, if d1 = d2 and d3 = d4, the original center coordinate point of the image to be processed coincides with the center coordinates of the first region, and the image to be processed has no deviation.

[0058] If d1 > d2 and d3 = d4, the original center coordinate point of the image to be processed has a horizontal offset from the center coordinates of the first region, and the effective region of the image to be processed is shifted to the right.

[0059] If d1 < d2 and d3 = d4, the original center coordinate point of the image to be processed has a horizontal offset from the center coordinates of the first region, and the effective region of the image to be processed is shifted to the left.

[0060] If d1 = d2 and d3 > d4, the original center coordinate point of the image to be processed has an up - down offset from the center coordinates of the first region, and the effective region of the image to be processed is shifted up.

[0061] If d1 = d2 and d3 < d4, the original center coordinate point of the image to be processed has an up - down offset from the center coordinates of the first region, and the effective region of the image to be processed is shifted down.

[0062] If d1 < d2 and d3 < d4, there are offsets in the up, down, left, and right directions between the original center coordinate point of the image to be processed and the center coordinate of the first region, and the effective region of the image to be processed is shifted to the lower left.

[0063] If d1 > d2 and d3 > d4, there are offsets in the up, down, left, and right directions between the original center coordinate point of the image to be processed and the center coordinate of the first region, and the effective region of the image to be processed is shifted to the upper right.

[0064] If d1 < d2 and d3 > d4, there are offsets in the up, down, left, and right directions between the original center coordinate point of the image to be processed and the center coordinate of the first region, and the effective region of the image to be processed is shifted to the upper left.

[0065] If d1 > d2 and d3 < d4, there are offsets in the up, down, left, and right directions between the original center coordinate point of the image to be processed and the center coordinate of the first region, and the effective region of the image to be processed is shifted to the lower right.

[0066] In this way, when the effective region of the image to be processed is a rectangle, square, regular hexagon, circle or other shapes, the center coordinate of the first region can be accurately obtained. Correspondingly, the center coordinate of the first region is the center coordinate of the effective region. Among them, when the effective region of the image to be processed is a circle, the schematic diagram of obtaining the center coordinate is as Figure 4 shown.

[0067] In another implementation form, when the effective region is a circle, the outer contour of the circle can be fitted by the coordinates of the pixel points with the farthest distance and pixel value of 1 in at least three different directions, and the center of the outer contour of the circle can be obtained according to the outer contour of the circle. Correspondingly, the center of the circle is the center coordinate of the first region.

[0068] S150. Select a statistical region from the binary image according to the center coordinate and the imaging parameters.

[0069] Generally speaking, different endoscope systems have different imaging parameters. Specifically, if the imaging of the endoscope system is circular, the endoscope system has an imaging radius range; the imaging radius range includes a minimum imaging radius and a maximum imaging radius, that is, the two endpoints of the imaging radius range are the minimum imaging radius and the maximum imaging radius respectively. At this time, a circular region can be selected as the statistical region in the binary image with the center coordinate as the center and the minimum imaging radius and the maximum imaging radius as the radii.

[0070] That is to say, the pixels within the circle formed by the minimum imaging radius with the central coordinate as the center of the circle must be the image collected by the endoscope system, so these pixels can be removed. Correspondingly, the pixels outside the circle formed by the maximum imaging radius with the central coordinate as the center of the circle must not be the image collected by the endoscope system, so these pixels can also be removed.

[0071] If the imaging of the endoscope system is square, the imaging parameters include a horizontal imaging range and a vertical imaging range. Specifically, in the horizontal direction, the horizontal imaging range may include a minimum horizontal imaging distance and a maximum horizontal imaging distance, that is, the two endpoints of the horizontal imaging range are the minimum horizontal imaging distance and the maximum horizontal imaging distance, respectively, wherein the minimum horizontal imaging distance and the maximum horizontal imaging distance are the distances between the edge of the image captured by the endoscope system and the horizontal center axis; similarly, in the vertical direction, the vertical imaging range may include a minimum vertical imaging distance and a maximum vertical imaging distance, that is, the two endpoints of the vertical imaging range are the minimum vertical imaging distance and the maximum vertical imaging distance, respectively, wherein the minimum vertical imaging distance and the maximum vertical imaging distance are the distances between the edge of the image captured by the endoscope system and the vertical center axis.

[0072] Thus, an annular area (such as a rectangular ring) can be constructed with the central coordinate as the center, and the parameters of the outer contour of the annular area are the maximum horizontal imaging distance and the maximum vertical imaging distance; the parameters of the inner contour of the annular area are the minimum horizontal imaging distance and the minimum vertical imaging distance.

[0073] In the present disclosure, by selecting a statistical area, the number of pixels that do not require a budget can be effectively excluded, thereby improving the operation speed of the image processing method of the present disclosure.

[0074] S160, traversing the pixel points in the statistical area to obtain effective field of view edge pixel points of the binary image.

[0075] After the statistical area is obtained, the edge pixels of the effective visual field can be obtained according to the characteristics of the pixels in the statistical area, that is, the outer contour of the effective area can be obtained.

[0076] The following describes the effective area of ​​a circle and the effective area of ​​a rectangle respectively.

[0077] Specifically, when the effective area is a circle, the effective field of view edge pixel points can be obtained through the following steps: S161, taking the central coordinates as the center of the circle, calculating the maximum distance value between the point with a pixel value of 1 and the center of the circle in the directions of multiple set angles; S162, counting the maximum distance values ​​of multiple set angles, and selecting the largest number of maximum distance values ​​as the effective area radius; S163, obtaining the effective field of view edge pixel points of the binary image according to the central coordinates and the effective area radius.

[0078] Specifically, in S161, the multiple set angles can be 0°, 45°, 90°, 135°, 180°, 225°, 270°, 315° and the like; technical personnel in this field should know that the more set angles selected, the more pixels involved in the calculation, and the more accurate the result.

[0079] When the effective area is a rectangle (i.e., a polygon including two mutually perpendicular axes of symmetry), the pixel points in the statistical area are traversed to obtain the effective field of view edge pixel points of the binary image, including: S165, obtaining the horizontal center axis and the vertical center axis passing through the center coordinates; S166, traversing the pixel points in the statistical area, calculating the vertical distance between the pixel points with a pixel value of 1 and the horizontal center axis, and calculating the horizontal distance between the pixel points with a pixel value of 1 and the vertical center axis; S167, counting the maximum horizontal distance value of the horizontal distances of the pixel points in the selected row, and selecting the maximum horizontal distance with the largest number as the vertical boundary; counting the maximum vertical distance value of the vertical distances of the pixel points in the selected column, and selecting the maximum vertical distance with the largest number as the horizontal boundary; S168, obtaining the effective field of view edge pixel points of the binary image according to the horizontal boundary and the vertical boundary.

[0080] Accordingly, the image processing method disclosed in the present invention can construct a rectangular area centered on the central coordinates through the horizontal boundary and the vertical boundary.

[0081] Specifically, in S167, the selected rows may be multiple rows; the selected columns may be multiple columns. Those skilled in the art should know that the more rows and columns are selected, the more pixels are involved in the calculation, and the more accurate the result is.

[0082] The effective visual field edge pixel point disclosed in the present invention is a pixel point generated by the image processing method disclosed in the present invention, and the pixel value of the pixel point is set to 1.

[0083] S170: Obtain a valid area of ​​the image to be processed according to the edge pixels of the effective field of view.

[0084] Specifically, S170 may include: S171, obtaining an effective field of view area based on the pixel points at the edge of the effective field of view, wherein the pixel values ​​of the pixel points in the binary image at the corresponding position of the effective field of view area are set to 1, and the pixel values ​​of other pixel points in the binary image are set to 0; S172, performing an AND operation on the image to be processed and the corresponding pixel points in the binary image to obtain the effective area of ​​the image to be processed.

[0085] In other words, the effective field of view edge pixel points disclosed in the present invention are the outer contour of the effective field of view area (corresponding to the effective area of ​​the image to be processed). After obtaining the effective field of view edge pixel points, the pixel values ​​of the pixel points within the outer contour can be filled with all 1; then, by performing an AND operation on the image to be processed and the binary image containing the effective field of view area, the effective area in the image to be processed can be displayed, and the area outside the effective area in the image to be processed can be displayed as black.

[0086] The image processing method disclosed in the present invention can adapt to different types of endoscopes and adapters with different focal lengths and imaging parameters (imaging ranges) commonly found on the market; and according to the imaging parameters of the endoscope, the peripheral occlusion area is eliminated, and the ISP processing algorithm avoids the invalid area, effectively reducing the amount of data flowing through the ISP, avoiding unnecessary and invalid calculations by the ISP, improving ISP performance, and reducing system power consumption. The image processing method disclosed in the present invention can dynamically adjust the metering area, guide the algorithm modules related to global brightness such as the exposure algorithm, and solve the problem of excessive exposure gain caused by the inconsistency between the metering area and the actual, thereby solving the problem of high image noise.

[0087] In addition, the image processing method disclosed in the present invention can effectively remove the "ghost images" of invalid areas on the edge of the image caused by packaging process errors and reflections from instruments such as endoscopes by accurately identifying the effective area of ​​the image; moreover, by identifying the effective area, it can correct images that deviate from the center of the display screen; and perform deviation correction processing on the captured image to solve the problem of the image not being displayed in the center of the screen due to process errors of the endoscope and adapter, so that the target image can be displayed in the center of the screen for easy observation by the operator.

[0088] Generally speaking, the image processing method disclosed in the present invention can calibrate the pixels within the effective field of view of the image to be processed by identifying the effective area in the image to be processed, and input these calibrated pixels into the ISP module for subsequent processing, thereby reducing the amount of data processed by the ISP and improving the ISP operation speed.

[0089] The effective area obtained by the image processing method of the present invention can be used for the next frame of image to be processed, or in other words, the image processing method of the present invention can calculate the average brightness of the effective area of ​​the current frame image based on the effective area range calculated in the previous frame, and then guide the range of the brightness threshold of the binarized image based on the average brightness, and then take a suitable limiting parameter Ypara based on the average brightness range, so that the brightness threshold can be continuously adjusted according to different image brightnesses, making the decision of the image binarization threshold more reasonable.

[0090] The present disclosure also provides an electronic device, including: a memory, the memory storing execution instructions; and a processor or other hardware module, the processor or other hardware module executes the execution instructions stored in the memory, so that the processor or other hardware module executes the above method.

[0091] The present disclosure also provides a readable storage medium, in which execution instructions are stored. When the execution instructions are executed by a processor, they are used to implement the above method.

[0092] The present disclosure also provides an endoscope system, which includes the above-mentioned electronic device.

[0093] Those skilled in the art should understand that the above embodiments are only for the purpose of clearly illustrating the present disclosure, and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications may be made based on the above disclosure, and these changes or modifications are still within the scope of the present disclosure.

Claims

1. An image processing method, characterized in that: include: Obtaining the average brightness of a designated area in the image according to the image to be processed; Obtaining a brightness threshold, wherein the brightness threshold is related to the average brightness; Binarizing the image to be processed according to the brightness threshold to obtain a binary image, wherein the binary image includes a first region, wherein the first region is a set of pixel points of the image to be processed whose brightness is greater than the brightness threshold; Obtaining the center coordinates of the first area according to the position of the first area; Selecting a statistical region from the binary image according to the center coordinates and imaging parameters; Traversing the pixel points in the statistical area to obtain the effective visual field edge pixel points of the binary image; The effective area of ​​the image to be processed is obtained according to the effective visual field edge pixel points.

2. The image processing method according to claim 1, characterized in that: The image to be processed includes N frames of images arranged in chronological order, the designated area of ​​the first frame of image is the image to be processed; the designated area of ​​the i-th frame of image is the effective area obtained by the i-1-th frame of image, wherein i is greater than or equal to 2 and less than or equal to N, and N is a natural number greater than or equal to 2.

3. The image processing method according to claim 1, characterized in that: The brightness threshold ;in, is the average brightness, is a limiting parameter related to the average brightness, is greater than 0 and less than 1, wherein the limiting parameter increases with the increase of the average brightness.

4. The image processing method according to claim 3, characterized in that: The limiting parameters ； Among them, width is the brightness data bit width.

5. The image processing method according to claim 1, characterized in that: Obtaining the center coordinates of the first area according to the position of the first area includes: Get the original center coordinate point of the binary image; Obtain, according to the original center coordinate point, a pixel point with a pixel value of 1 that is farthest in a preset angle direction passing through the original center coordinate point; Obtain a first direction offset and a second direction offset according to the coordinates of the farthest pixel point whose pixel value is 1, wherein the first direction is perpendicular to the second direction; The center coordinates of the first area are obtained according to the original center coordinate point, the first direction offset and the second direction offset.

6. The image processing method according to claim 5, characterized in that: The first direction and the second direction are both directions of the symmetry axis of the image to be processed.

7. The image processing method according to claim 1, characterized in that: When the effective area is a circle, traversing the pixel points in the statistical area to obtain the effective field of view edge pixel points of the binary image includes: Taking the central coordinate as the center of the circle, calculating the maximum distance between a point with a pixel value of 1 and the center of the circle in directions of multiple set angles; Count the maximum distance values ​​of multiple set angles, and select the maximum distance value with the largest number as the effective area radius; The effective visual field edge pixel points of the binary image are obtained according to the central coordinates and the effective area radius.

8. The image processing method according to claim 1, characterized in that: When the effective area is a polygon including two mutually perpendicular axes of symmetry, traversing the pixel points in the statistical area to obtain the effective field of view edge pixel points of the binary image includes: Obtaining a horizontal center axis and a vertical center axis passing through the center coordinates; Traverse the pixel points in the statistical area, calculate the vertical distance between the pixel point with a pixel value of 1 and the horizontal center axis, and calculate the horizontal distance between the pixel point with a pixel value of 1 and the vertical center axis; Count the maximum horizontal distance value of the horizontal distances of each pixel point in the selected row, and select the maximum horizontal distance with the largest number as the vertical boundary; count the maximum vertical distance value of the vertical distances of each pixel point in the selected column, and select the maximum vertical distance with the largest number as the horizontal boundary; Obtain effective visual field edge pixel points of the binary image according to the horizontal boundary and the vertical boundary; Optionally, obtaining a valid area of ​​the image to be processed according to the effective visual field edge pixel points includes: Obtaining an effective visual field area according to the effective visual field edge pixel points, wherein the pixel point values ​​of the pixel points in the binary image corresponding to the effective visual field area are set to 1, and the pixel point values ​​of other pixel points in the binary image are set to 0; and Perform an AND operation on the image to be processed and the corresponding pixel points in the binary image to obtain the effective area of ​​the image to be processed; Optionally, when displaying the image to be processed, the center coordinates are aligned with the center point of the display.

9. An electronic device, characterized in that: include: A memory storing execution instructions; as well as A processor, wherein the processor executes the execution instruction stored in the memory, so that the processor executes the image processing method according to any one of claims 1 to 8.

10. An endoscope system, characterized in that: An electronic device comprising the electronic device described in claim 9.

Images