Portable endoscope, control method thereof and storage medium

Abstract

The invention discloses a portable endoscope, a control method of the portable endoscope and a storage medium. The control method of the portable endoscope comprises the steps that video data of the portable endoscope are obtained in real time; according to the video data, the state of the surgical field is determined, and the state comprises a static state or a moving state; according to the state of the surgical field, the working modes of the portable endoscope are adjusted, and the working modes comprise a low-power-consumption monitoring mode, a full-power mode and an energy-saving mode.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical devices, in particular to a portable endoscope, a control method thereof, and a storage medium. BACKGROUND

[0002] The portable endoscope is a product of the combination of traditional endoscope technology and miniaturization, light weight, and intelligent technology. Its core advantage lies in flexible mobile operation, adaptation to various scenes, and unique value in primary medical care, emergency care, home monitoring, and remote medical care.

[0003] The portable endoscope needs to balance the endurance time, volume and weight, safety, and use scene adaptability. The traditional portable endoscope causes 30-40% invalid energy consumption due to continuous display, which shortens the endurance time of the portable endoscope. In the prior art, the start and stop display of the portable endoscope are controlled by a manual switch, which affects the continuity of the operation.

[0004] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0005] The main purpose of the present application is to provide a portable endoscope, a control method thereof, and a storage medium, which aims to solve the technical problems of the prior art that the traditional portable endoscope causes 30-40% invalid energy consumption due to continuous display, which shortens the endurance time of the portable endoscope, and the start and stop display of the portable endoscope are controlled by a manual switch, which affects the continuity of the operation.

[0006] To achieve the above purpose, the present application provides a control method of a portable endoscope, which comprises:

[0007] real-time acquisition of video data of the portable endoscope;

[0008] determination of the state of the surgical field according to the video data, the state including a static state or a motion state;

[0009] adjustment of the working mode of the portable endoscope according to the state of the surgical field, the working mode including a low-power monitoring mode, a full-power mode, and an energy-saving mode.

[0010] Preferably, in the control method of the portable endoscope, the determination of the state of the surgical field according to the video data comprises:

[0011] generation of an RGB image according to the video data, and conversion of the RGB image into a grayscale image;

[0012] pixel-by-pixel calculation of the absolute difference between the current frame grayscale image and the last frame grayscale image to obtain a difference image;

[0013] Adaptive computing dynamic threshold according to the gray distribution of the difference map;

[0014] Segmenting the difference map according to the computed dynamic threshold to preliminarily determine the motion region;

[0015] For each feature point in the preliminarily determined motion region, computing the motion vector of each feature point based on its constant brightness in the adjacent frame;

[0016] Determining the state of the surgical field according to the motion vector of each feature point.

[0017] Preferably, in the control method of the portable endoscope, the determining the state of the surgical field according to the motion vector of each feature point comprises:

[0018] Computing the amplitude of a single vector, the formula is as follows:

[0019]

[0020] Computing the average motion amplitude, the formula is as follows: avg_g = (g1+g2+…+gk) / k; k

[0021] Computing the amplitude change rate, the formula is as follows: Δ_g = |avg_g s -avg_g s-1 | / avg_g s-1 ;

[0022] Computing the direction of a single vector reflecting the angle of the motion of the feature point, the direction of a single vector is:

[0023] λ i =arctan2(a i ,b i );

[0024] Computing the standard deviation of all λ i is std_λ

[0025] When the consecutive multiple frames all meet avg_g≥T4 and std_λ≤T5 and Δ_g s

[0026] , it is determined as valid motion, otherwise as invalid interference;

[0027] i is 1, 2, …, k, and k is the number of valid vectors;

[0028] (a i , b i ) is the i-th vector; ​​

[0029] T5 is a direction consistency threshold value;

[0030] T4 is a static threshold value;

[0031] T6 is an amplitude mutation threshold value.

[0032] Preferably, in the control method of the portable endoscope, the adjusting the working mode of the portable endoscope according to the state of the surgical field comprises:

[0033] in response to the surgical field being in a static state, controlling the portable endoscope to enter a low-power monitoring mode;

[0034] in response to the surgical field being in a motion state within a continuous preset number of frame images, controlling the portable endoscope to enter a full-power mode;

[0035] in response to the portable endoscope being in a full-power mode and the surgical field having no effective motion within a preset time length, controlling the portable endoscope to enter an energy-saving mode.

[0036] Preferably, in the control method of the portable endoscope, the controlling the portable endoscope to enter an energy-saving mode in response to the portable endoscope being in a full-power mode and the surgical field having no effective motion within a preset time length comprises:

[0037] in response to the portable endoscope being in a full-power mode and the surgical field having no effective motion within a preset time length, reducing the display brightness or turning off the display screen.

[0038] Preferably, in the control method of the portable endoscope, the controlling the portable endoscope to enter a low-power monitoring mode in response to the surgical field being in a static state comprises:

[0039] in response to the surgical field being in a static state, controlling the display screen of the portable endoscope to reduce the refresh rate.

[0040] Preferably, in the control method of the portable endoscope, the control method further comprises:

[0041] when the portable endoscope is in a low-power monitoring mode, using an FPGA to accelerate image processing of the portable endoscope.

[0042] To achieve the above-mentioned purpose, the present application provides a portable endoscope, characterized in that it comprises:

[0043] an image acquisition module for acquiring video data of the portable endoscope in real time;

[0044] a motion analysis module for determining the state of the surgical field according to the video data, the state including a static state or a motion state;

[0045] a dynamic power control module, configured to adjust a working mode of the portable endoscope according to a state of a surgical field, the working mode including a low-power monitoring mode, a full-power mode, and a power-saving mode.

[0046] To achieve the above object, the present application provides a portable endoscope, which comprises:

[0047] at least one processor; and,

[0048] a memory in communication connection with the at least one processor; wherein,

[0049] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the control method of the portable endoscope as described above.

[0050] To achieve the above object, the present application provides a computer readable storage medium storing a computer program, which is executed by a processor to implement the control method of the portable endoscope as described above.

[0051] The present application has at least the following beneficial effects:

[0052] The control method of the portable endoscope provided by the present application acquires video data of the portable endoscope in real time, determines a state of a surgical field according to the video data, the state including a static state or a dynamic state, and adjusts a working mode of the portable endoscope according to the state of the surgical field, the working mode including a low-power monitoring mode, a full-power mode, and a power-saving mode. Thus, the precise power-saving control of the endoscope system is realized under the premise of ensuring the continuity of medical operation.

[0053] Further, the present application solves the technical problem of 30-40% invalid power consumption caused by continuous display of the conventional portable endoscope, shortens the endurance time of the portable endoscope, and controls the start and stop display of the portable endoscope by manual switch, which affects the continuity of surgery by manual intervention.

[0054] Further, based on the above method, the overall power consumption of the portable endoscope is reduced by more than 30%, the invalid screen-on time is reduced by 90%, and the service life of the display screen is prolonged by at least 3 times; based on the motion analysis module, the display is automatically started and stopped, realizing zero manual intervention, the wake-up delay is less than 50ms, and the continuity of surgery is ensured; the endurance of the portable endoscope is improved by 40%, for example, from 4h to 5.6h. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 The control method of the portable endoscope provided by the present application is shown in a schematic diagram of an embodiment.

[0056] Figure 2 A schematic view of an embodiment of the portable endoscope provided by the present application;

[0057] Figure 3 A schematic view of another embodiment of the portable endoscope provided by the present application.

[0058] The purposes, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0059] The technical solutions of the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0060] In the embodiments of the present application, the term "and / or" describes the association relationship of the associated objects, which means that there can be three kinds of relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. The character " / " generally represents an "or" relationship between the associated objects before and after it.

[0061] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily mean a specific order or sequence.

[0062] In the embodiments of the present application, the term "a plurality of" means two or more, and other quantifiers are similar.

[0063] In the present application, the positional words such as "upper", "lower", "top", "bottom" used without the opposite description are generally for the direction shown in the drawings, or for the vertical, perpendicular or gravity direction of the components themselves; similarly, for the convenience of understanding and description, "inner" and "outer" refer to the inner and outer relative to the contour of each component itself, but the above-mentioned positional words are not used to limit the present application.

[0064] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the drawings. However, those skilled in the art can understand that, in the embodiments of the present application, many technical details are proposed in order to make the readers better understand the present application. However, the technical solutions claimed by the present application can be implemented even without these technical details and based on various changes and modifications of the following embodiments. The division of the following embodiments is for the convenience of description, and should not constitute any limitation on the specific implementation modes of the present application. The embodiments can be combined and referenced with each other without contradiction.

[0065] The portable endoscope needs to balance the endurance time, volume and weight, safety and use scene adaptability. The continuous display of the traditional portable endoscope causes 30-40% of invalid energy consumption, which shortens the endurance time of the portable endoscope. In the prior art, the start and stop display of the portable endoscope is controlled by a manual switch, which affects the continuity of the operation.

[0066] In order to solve the above problems, the present embodiment relates to a control method of a portable endoscope, which can be applied to the portable endoscope.

[0067] The implementation details of the control method of the portable endoscope of the present embodiment will be described below. The following content is only provided for the implementation details for the convenience of understanding, and is not necessary for implementing the present solution. Figure 1 The schematic diagram of the control method of the portable endoscope of the present application is shown.

[0068] The video data of the portable endoscope is acquired in real time at step S100. It should be understood that the real-time picture can be captured by the image sensor of the front end of the portable endoscope to obtain the video data.

[0069] At step S200, the state of the surgical field is determined according to the video data, and the state includes a static state or a motion state. It should be understood that the surgical field mentioned in the present application refers to the range of the surgical operation area and the surrounding related tissues that can be clearly observed by the doctor during the operation process through the naked eye, surgical microscope, endoscope and other tools. It directly determines the accuracy, safety and efficiency of the operation, and is the key basis for operation planning, operation execution and postoperative evaluation.

[0070] The state of the surgical field is used to determine whether the surgical field area is undergoing a surgical operation, a dynamic change in tissue, or a critical physiological activity. In some embodiments, the state of the surgical field needs to be comprehensively analyzed in combination with the surgical stage, the operation behavior, and the state of the tissue in the surgical field. Generally, the state of the surgical field needs to be combined with whether there is active surgical operation, whether there is a "critical dynamic change" in the tissue / physiological indicators in the surgical field, and the relevance of the surgical stage to the surgical field. When there is active surgical operation, i.e., the doctor or the instrument is operating in the surgical field, the corresponding state of the surgical field must be a motion state. When there is no active surgical operation, if the tissue or physiological indicators in the surgical field appear to change, such as a change in the state of the tissue (sudden rupture and bleeding of blood vessels in the surgical field), a change in the physiological indicators, this case can also be considered as a motion state of the surgical field.

[0071] In some embodiments, the state of the surgical field can be detected by first detecting whether there is a significant change area in the endoscope image generated according to the video data; if there is no significant change area, it is considered that the state of the surgical field is a static state; if there is a significant change area, the authenticity of the motion in the change area is further verified, for example, combined with sparse optical flow calculation, a set of optical flow vectors are output; finally, it is necessary to judge whether the motion is valid motion, for example, combined with the average amplitude and direction consistency of the optical flow vector to judge whether it belongs to valid motion.

[0072] More specifically, taking the average amplitude and direction consistency of the optical flow vector as an example to judge whether it belongs to valid motion, two decision thresholds (average amplitude threshold T2 of the optical flow vector, direction consistency threshold T3) are preset, by setting T2, it can be judged whether the motion intensity is sufficient, and the slight jitter is excluded; by setting T3, it can be judged whether the motion has a unified trend, and the clutter interference is excluded. When the average amplitude of the output optical flow vector is greater than or equal to T2 and the standard deviation of the direction of all optical flow vectors is less than or equal to T3, it is determined that the motion is valid, otherwise it is invalid interference.

[0073] Wherein, the calculation formula of the amplitude of a single optical flow vector is as follows:

[0074]

[0075] The average amplitude of the optical flow vector is avg_mag=(mag1+mag2+…+mag k / k;

[0076] The direction of a single optical flow vector is θ i =arctan2(v i ,u i );

[0077] The standard deviation of all θ i is std_θ;

[0078] wherein i is 1, 2, …, k, k is the number of valid optical flow vectors, and when k = 0, it is directly determined as invalid interference;

[0079] The set of optical flow vectors is {(u1, v1), (u2, v2), …, (u k , v k )}.

[0080] In some embodiments, step S200 can further include steps S210 to S260.

[0081] At step S210, an RGB image is generated according to the video data, and the RGB image is converted into a grayscale image. It should be understood that the conversion of the RGB image into a grayscale image can be converted into a single-channel grayscale image through a grayscale formula, eliminating color channel redundancy and reducing the subsequent pixel-by-pixel calculation amount. In some embodiments, the grayscale image can also be preprocessed, such as noise suppression and picture standardization. Specifically, a 3x3 or 5x5 Gaussian filter is used to smooth the grayscale image, preliminarily filtering out possible electronic noise and environmental interference noise in the endoscope image, avoiding the influence of noise on the subsequent threshold segmentation. In some embodiments, if there is slight uneven illumination, the grayscale distribution is adjusted through histogram equalization to enhance the image contrast and provide a more stable grayscale reference for inter-frame difference.

[0082] At step S220, the absolute difference between the current frame grayscale image and the previous frame grayscale image is calculated pixel by pixel to obtain a difference image. The absolute value difference of the grayscale is calculated pixel by pixel for the adjacent two frames of grayscale images to obtain the difference image. D(x, y) = |F n (x, y) - F n-1 (x, y)|.

[0083] wherein (x, y) is the pixel coordinate, and D(x, y) is the inter-frame grayscale change amplitude of the pixel. F n and F n-1 are the grayscale values of the adjacent frames which are time-continuous and spatially-aligned.

[0084] At step S230, a dynamic threshold is adaptively calculated according to the grayscale distribution of the difference image. In other embodiments, the maximum inter-class variance method can also be used to automatically calculate the dynamic threshold, by finding the threshold with the maximum variance of the two classes of pixels in the motion state and the stationary state, to ensure that it can adapt to different scenes, for example, when the portable endoscope enters a strong light area, the dynamic threshold is automatically adjusted upwards to avoid the light change being misjudged as motion.

[0085] At step S240, the difference image is segmented according to the calculated dynamic threshold to preliminarily determine the motion region. Specifically, the difference image is binarized according to the calculated dynamic threshold to generate a binary mask, for example, when D(x, y) ≥ T, the binary mask is 255 (at this time, it is marked as a motion candidate pixel); when D(x, y) < T, the binary mask is 0 (at this time, it is marked as a stationary pixel).

[0086] The preliminary motion candidate region can be intuitively displayed by means of the binary mask, but may contain false motion pixels caused by noise. Therefore, further processing of the binary mask is still needed, including traversing the binary mask using a preset structural element (for example, a 3x3 rectangular kernel), retaining only the center pixel whose all pixels in the kernel are 255, or setting the center pixel to 0, to obtain a first processed mask; traversing the first processed mask using the above-mentioned preset structural element, setting the center pixel to 255 if there is at least one 255 pixel in the kernel, to obtain a second processed mask; calculating the areas of all connected regions in the second processed mask, and removing the connected regions with an area less than a preset threshold, to generate an optimized preliminarily determined motion region.

[0087] Through the first processing of the mask, isolated small noise points such as 1-2 pixel false motion regions can be eliminated, and the "burr" on the edge of the motion region can be reduced. Through the second processing of the mask, the motion region reduced by the first processing of the mask can be restored to a size close to the true size, and scattered pixels in the motion region can be connected, to ensure the integrity of the motion region. In this way, noise can be eliminated while the true motion region is restored, and a more accurate preliminarily determined motion region can be output.

[0088] At step S250, for each feature point in the preliminarily determined motion region, a motion vector of each feature point is calculated based on the constant luminance of the feature point in the adjacent frame. According to the constant luminance of the same feature point in the adjacent frame, the feature point set is tracked in the preliminarily determined motion region of the current frame, and the motion vector of each feature point is calculated.

[0089] At step S260, the state of the surgical field is determined according to the motion vector of each feature point. According to the motion vector of each feature point.

[0090] The amplitude of a single vector reflects the motion distance of a single feature point, and the calculation formula of the amplitude of a single vector is as follows:

[0091]

[0092] The average motion amplitude reflects the overall motion intensity, and the average motion amplitude is avg_g = (g1+g2+…+g k ) / k;

[0093] Amplitude variation rate Δ_g = |avg_g s -avg_g s-1 | / avg_g s-1 ,avg_g s is the average motion amplitude of the s-th frame; the amplitude variation rate Δ_g reflects the degree of sudden change of the motion intensity.

[0094] The direction of a single vector reflects the angle of the motion of the feature point, and the direction of a single vector is:

[0095] λ i = arctan2(a i , b i );

[0096] The standard deviation of all λ i is std_λ, and the smaller std_λ is, the more consistent the direction is;

[0097] wherein i is 1, 2, …, k, and k is the number of effective vectors, and when k = 0, it is directly determined as invalid interference;

[0098] The vector set is {(a1, b1), (a2, b2), …, (a k , b k )}.

[0099] When a plurality of consecutive frames all satisfy avg_g ≥ T4 and std_λ ≤ T5 and Δ_g < T6, it is determined as valid motion, otherwise it is invalid interference. For example, it is required that the above conditions are all satisfied for 3 consecutive frames to determine valid motion. T5 is a threshold for direction consistency, such as 30°-45°, which is converted into radian as 0.52-0.79. T4 is a static threshold, which is usually calibrated according to the frame rate of the portable endoscope, such as 1-2 pixels / frame. T6 is an amplitude mutation threshold, such as 50%.

[0100] At step S300, the working mode of the portable endoscope is adjusted according to the state of the surgical field, and the working mode includes a low-power monitoring mode, a full-power mode, and an energy-saving mode. Specifically, step S300 includes steps S310 to S330.

[0101] At step S310, in response to the surgical field being in a static state, the portable endoscope is controlled to enter a low-power monitoring mode. It should be noted that since the portable endoscope is powered by a battery, power consumption has a greater impact on the endurance time of the portable endoscope. Since the power consumption of the main machine of the portable endoscope is mainly on the display screen, in some embodiments, the low-power monitoring mode is mainly for the display screen, for example, reducing the refresh rate of the display screen to reduce the power consumption of the display screen and the use time. Specifically, in response to the surgical field being in a static state, the display screen of the portable endoscope is controlled to reduce the refresh rate.

[0102] For example, the display screen is maintained at a low refresh rate of 5 Hz. In other embodiments, the low-power monitoring mode can also be to power only the key circuits. Of course, when the portable endoscope is in the low-power monitoring mode, the state changes of the surgical field are also monitored in real time (e.g., light-weight calculation at 10 fps), and the portable endoscope is adjusted to the corresponding working mode according to the state changes of the surgical field. By monitoring the state changes of the surgical field in real time, the display screen can be awakened in time when the state of the surgical field changes.

[0103] When the portable endoscope is in the low-power monitoring mode, the FPGA is used to accelerate the image processing of the portable endoscope.

[0104] At step S320, in response to the surgical field being in a motion state in a continuous preset number of frame images, the portable endoscope is controlled to enter a full-power mode. In order to avoid invalid motion of the surgical field affecting the accuracy of mode switching, whether the continuous preset number of frame images are all in a motion state can be detected, i.e., whether significant motion occurs in the continuous preset number of frame images, such as instrument operation or lens movement. When it is detected that significant motion occurs in the continuous preset number of frame images, the full-power mode is controlled to be entered, such as restoring full-brightness display within 50 ms, to ensure no operation interruption.

[0105] At step S330, in response to the portable endoscope being in the full-power mode and there being no valid motion of the surgical field within a preset time period, the portable endoscope is controlled to enter an energy-saving mode. Specifically, in response to the portable endoscope being in the full-power mode and there being no valid motion of the surgical field within the preset time period, the display brightness is reduced or the display screen is turned off.

[0106] The preset time period can be set according to the user's needs, for example, 15 seconds. Reducing the power consumption of the portable endoscope can be by reducing the display brightness of the display screen, for example, from 100% to 30%. In other embodiments, the display screen can also be turned off, and only the sensor is kept standby.

[0107] Specifically, a first preset time period and a second preset time period can be set, the first preset time period is less than the second preset time period, when there is no valid motion within the first preset time period, the display brightness of the portable endoscope can be controlled to be reduced; when there is no operation within the second preset time period, the display screen is turned off, and only the sensor is kept standby.

[0108] In other embodiments, in order to facilitate anti-interference, state switching can be triggered only when motion is detected in a plurality of consecutive frames to suppress transient noise; only the central area of the picture can be monitored, and the edge non-surgical operation area can be ignored; a Markov model can be established based on historical optical flow data to predict motion persistence.

[0109] The application provides a control method of a portable endoscope, real-time acquisition of video data of the portable endoscope; according to the video data, determination of the state of a surgical field, the state including a static state or a motion state; according to the state of the surgical field, adjustment of the working mode of the portable endoscope, the working mode including a low-power monitoring mode, a full-power mode and an energy-saving mode. Thus, under the premise of ensuring the continuity of medical operation, precise energy-saving control of the endoscope system is realized.

[0110] Further, the application solves the technical problem that 30-40% of invalid energy consumption is caused by continuous display of the conventional portable endoscope in the prior art, shortens the endurance time of the portable endoscope, and controls the start-stop display of the portable endoscope through a manual switch, which influences the continuity of operation.

[0111] Further, based on the above method, the overall power consumption of the portable endoscope is reduced by more than 30%, the invalid screen-on time is reduced by 90%, and the service life of the display screen is prolonged by at least 3 times; based on the motion analysis module, the display is automatically started and stopped, zero manual intervention is realized, the wake-up delay is less than 50ms, and the continuity of operation is ensured; the endurance of the portable endoscope is improved by 40%, for example, from 4h to 5.6h.

[0112] The application also provides a portable endoscope, as shown in the figure, which comprises an image acquisition module 410, a motion analysis module 420 and a dynamic power consumption control module 430. Figure 2

[0113] The image acquisition module 410 is used for real-time acquisition of video data of the portable endoscope. It should be understood that the real-time picture can be captured through an image sensor at the front end of the portable endoscope to obtain the video data.

[0114] The motion analysis module 420 is used for determining the state of a surgical field according to the video data, the state including a static state or a motion state. It should be understood that the surgical field (Surgical Field) mentioned in the application refers to the range of the operation area and the surrounding related tissues that can be clearly observed by the doctor through the naked eye, a surgical microscope, an endoscope and other tools during operation. It directly determines the accuracy, safety and efficiency of the operation, and is the key basis for operation planning, operation execution and postoperative evaluation.

[0115] ​The state of the surgical field is used to determine whether the surgical field region is undergoing a surgical operation, a dynamic change of tissue, or a critical physiological activity. In some embodiments, the state of the surgical field needs to be comprehensively analyzed in combination with the surgical stage, the operation behavior, and the state of the tissue in the surgical field. Generally, the state of the surgical field needs to be combined with whether there is active surgical operation, whether there is a "critical dynamic change" of the tissue / physiological indicators in the surgical field, and the relevance of the surgical stage to the surgical field. When there is active surgical operation, i.e., the doctor or the instrument is operating in the surgical field, the corresponding state of the surgical field must be a motion state. When there is no active surgical operation, if the tissue or physiological indicators in the surgical field appear to change, such as a change in the state of the tissue (sudden rupture and bleeding of blood vessels in the surgical field), a change in the physiological indicators, this case can also be considered as a motion state of the surgical field.

[0116] In some embodiments, the state of the surgical field can be detected by first detecting whether there is a significant change area in the endoscope image generated according to the video data; if there is no significant change area, it is considered that the state of the surgical field is a static state; if there is a significant change area, the authenticity of the motion in the change area is further verified, for example, combined with sparse optical flow calculation, a set of optical flow vectors are output; finally, it is necessary to judge whether the motion is valid motion, for example, combined with the average amplitude and direction consistency of the optical flow vector to judge whether it belongs to valid motion.

[0117] More specifically, taking the average amplitude and direction consistency of the optical flow vector as an example to judge whether it belongs to valid motion, two decision thresholds (average amplitude threshold T2 of the optical flow vector, direction consistency threshold T3) are preset, by setting T2, it can be judged whether the motion intensity is sufficient, and the slight jitter is excluded; by setting T3, it can be judged whether the motion has a unified trend, and the clutter interference is excluded. When the average amplitude of the output optical flow vector is greater than or equal to T2 and the standard deviation of the direction of all optical flow vectors is less than or equal to T3, it is determined that the motion is valid, otherwise it is invalid interference.

[0118] The calculation formula of the amplitude of a single optical flow vector is as follows:

[0119]

[0120] The average amplitude of the optical flow vector is avg_mag=(mag1+mag2+…+mag k / k;

[0121] The direction of a single optical flow vector is θ i =arctan2(v i ,u i );

[0122] The standard deviation of all θ i is std_θ;

[0123] Where i is 1, 2, ..., k, and k is the number of effective optical flow vectors. When k = 0, it is directly determined as invalid interference.

[0124] The optical flow vector set is {(u1, v1), (u2, v2), ..., (u... k v k )}.

[0125] The dynamic power consumption control module 430 is used to adjust the working mode of the portable endoscope according to the state of the surgical field. The working modes include low power consumption monitoring mode, full power mode, and energy-saving mode.

[0126] To achieve the above objectives, the present invention also provides a convenient endoscope, such as... Figure 3 As shown, the portable endoscope includes at least one processor 501; and a memory 502 communicatively connected to the at least one processor 501; wherein the memory 502 stores instructions executable by the at least one processor 501, the instructions being executed by the at least one processor 501 to enable the at least one processor 501 to perform the aforementioned control method of the portable endoscope.

[0127] The memory 502 and processor 501 are connected via a bus, which can include any number of interconnecting buses and bridges. The bus connects various circuits of one or more processors 501 and memory 502 together. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 501 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 501.

[0128] Processor 501 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 502 can be used to store data used by processor 501 during operation.

[0129] To achieve the above objectives, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor 501, implements the aforementioned convenient endoscope control method.

[0130] That is, a person skilled in the art can understand that all or part of the steps in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a program stored in a storage medium, including a plurality of instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the various embodiment methods of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0131] Obviously, the above-described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, a person of ordinary skill in the art can make other different forms of changes or modifications without making creative efforts, and all of them should belong to the protection scope of the present application.

Claims

1. A control method of a handy endoscope, characterized by, The method comprises the following steps: acquiring video data of the portable endoscope in real time; determining the state of the surgical field according to the video data, the state comprising a static state or a motion state; adjusting the working mode of the portable endoscope according to the state of the surgical field, the working mode comprising a low-power monitoring mode, a full-power mode, and an energy-saving mode.

2. The control method of a handy endoscope according to claim 1, wherein The step of determining the state of the surgical field according to the video data comprises: generating an RGB image according to the video data, and converting the RGB image into a grayscale image; calculating the absolute difference between the current frame grayscale image and the previous frame grayscale image pixel by pixel to obtain a difference image; adaptively calculating a dynamic threshold value according to the grayscale distribution of the difference image; segmenting the difference image according to the calculated dynamic threshold value to preliminarily determine a motion region; calculating the motion vector of each feature point in the preliminarily determined motion region based on the constant brightness of the feature point in adjacent frames; determining the state of the surgical field according to the motion vector of each feature point.

3. The control method of a handy endoscope according to claim 2, wherein The step of determining the state of the surgical field according to the motion vector of each feature point comprises: calculating the amplitude of a single vector, the calculation formula being as follows: The average motion amplitude is calculated by the formula: avg_g = (g1+g2+...+g k ) / k; The amplitude variation rate is calculated, the calculation formula is Δ_g = |avg_g s -avg_g s-1 | / avg_g s-1 ; calculating the direction of a single vector to reflect the angle of the motion of the feature point, the direction of a single vector being: λ i = arctan2(a i , b i ); Compute all the lambda i standard deviations std_lambda when consecutive multiple frames all satisfy avg_g≥T4, std_λ≤T5, and △_g<T6, it is determined that there is valid motion, otherwise, it is determined that there is invalid interference; avg_g = avg_g + (g - avg_g) / s s is the average motion amplitude of the s-th frame; i is 1, 2, …, k, k being the number of valid vectors; (a i , b i ) is the ith vector; T5 is a threshold value of direction consistency; T4 is a static threshold value; T6 is a threshold value of amplitude mutation.

4. The control method of a handy endoscope according to claim 1, wherein The step of adjusting the working mode of the portable endoscope according to the state of the surgical field comprises: in response to the surgical field being in a static state, controlling the portable endoscope to enter a low-power monitoring mode; in response to the surgical field being in a motion state in consecutive preset number of frames of images, controlling the portable endoscope to enter a full-power mode; in response to the portable endoscope being in a full-power mode and there being no valid motion of the surgical field within a preset time length, controlling the portable endoscope to enter an energy-saving mode.

5. The control method of a handy endoscope according to claim 4, wherein The step of controlling the portable endoscope to enter an energy-saving mode in response to the portable endoscope being in a full-power mode and there being no valid motion of the surgical field within a preset time length comprises: in response to the portable endoscope being in a full-power mode and there being no valid motion of the surgical field within a preset time length, reducing the display brightness or turning off the display screen.

6. The control method of a handy endoscope according to claim 4, wherein The step of controlling the portable endoscope to enter a low-power monitoring mode in response to the surgical field being in a static state comprises: in response to the surgical field being in a static state, controlling the display screen of the portable endoscope to reduce the refresh rate.

7. The control method of a handy endoscope according to claim 4, wherein The method further comprises: when the portable endoscope is in a low-power monitoring mode, using an FPGA to accelerate the image processing of the portable endoscope.

8. A portable endoscope, characterized by comprising: The device comprises: an image acquisition module configured to acquire video data of the portable endoscope in real time; a motion analysis module configured to determine the state of the surgical field according to the video data, the state comprising a static state or a motion state; and a working mode adjustment module configured to adjust the working mode of the portable endoscope according to the state of the surgical field, the working mode comprising a low-power monitoring mode, a full-power mode, and an energy-saving mode. A dynamic power control module is configured to adjust a working mode of the portable endoscope according to a state of a surgical field, the working mode including a low-power monitoring mode, a full-power mode, and a power-saving mode.

9. A portable endoscope, characterized by comprising: The portable endoscope comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the control method of the portable endoscope according to any one of claims 1 to 7.

10. A computer readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the control method of the portable endoscope according to any one of claims 1 to 7.

Images