An endoscope system

Abstract

An endoscope system comprises an endoscope module for entering a patient's body to acquire endoscopic images; an image display module in communication with the endoscope module for displaying the endoscopic images to an operator. The image display module is configured to display the endoscopic images to the operator with a control display area. The control display area is at least partially associated with a function of a control module. The endoscope system further comprises an information acquisition module configured to acquire voice or visual focus position information of the operator and transmit the information to a central processing module, wherein the central processing module is configured to generate an interaction instruction for controlling the control module based on the voice or visual focus position information. The present application provides a non-contact interaction means in addition to the conventional hand or foot control devices of the endoscope, so that the operator can send operation instructions to the endoscope system without needing to free up hands or feet, thereby improving the efficiency of the endoscopic procedure.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of endoscopes, and relates to an endoscope system. BACKGROUND

[0002] The prior art endoscope is complicated to operate. Such an endoscope system includes a scope component, an image display component and an illumination component. When using the endoscope, the scope is required to be inserted into the patient's body, and the scope has multiple channels inside, including an illumination fiber, an image transmission fiber, an air transmission channel, a water transmission channel, an instrument channel, etc. When operating the endoscope, the medical staff needs to control at least the following: control of the display system, control of the light source, control of the scope (angle rotation, insertion, withdrawal, etc.), control of the suction valve, control of the air / water valve, control of the endoscopic diagnosis and treatment equipment (such as forceps for picking up foreign objects), etc.

[0003] CN116115166A provides an endoscope and an endoscope system using the same. The endoscope system includes a handle, a scope tube connected to the front end of the handle body of the handle, a wire tube connected to the rear end of the handle body, a water pipe extending through the wire tube and the handle body to the scope tube, a first water outlet end of the water pipe located at the front end of the scope tube, and a first water inlet end of the water pipe located outside the wire tube or outside the handle; a first camera is arranged at the center of the front end of the scope tube, a water pipe is formed between the wire tube of the first camera and the side wall of the scope tube, a support is arranged between the first camera and the side wall of the scope tube, and the support is located at the front end of the scope tube. The defect is that a water pipe is arranged outside the scope tube of the endoscope, so that the operator needs to control the related operation of the water pipe in addition to controlling the endoscope. Since the operation of the endoscope itself requires the operator to pay attention to the picture on the screen at all times, and the operator needs to hold the operation assembly of the endoscope with both hands, if the water pipe also needs to be operated, the operator will be distracted, which is likely to cause mistakes.

[0004] However, in the prior art, when using the endoscope system, a large number of control operations are performed by the same operator, and often at the same time, the operator needs to control two or more instruments, so that in the case of the operator controlling the instruments with both hands, the other instruments need to be controlled by stepping on the pedal. Since the endoscope travels inside the human body (intestine, stomach, abdominal cavity, etc.), the operator needs to concentrate when operating and constantly watch the image display module to determine the position of each device of the endoscope, which often causes the operator to touch the pedal by blind stepping. For operators who are not familiar with the equipment or lack experience, they often need to move their eyes away from the display before blind stepping to determine the position of the pedal by briefly looking at it. However, since the operator needs to frequently control multiple instruments during operation, the operator needs to constantly adjust the posture during the operation, which on the one hand increases the operation time and consumes the operator's energy, and on the other hand, when the pedal is placed in an unreasonable position, it is easy to cause missteps and medical accidents. At the same time, there are many operable items in the endoscopic operation, including important operation items such as controlling the turning of the endoscope and controlling the micro tool on the endoscope. These important operation items usually need to be manually controlled by the operator, and some other important operation items need to be controlled by the operator using the pedal. However, in addition to these, there are some operation items that cannot be simultaneously controlled by the operator with both hands and feet, such as the zoom ratio of the endoscopic display screen. This requires the operator to temporarily stop the "important operation items" in hand to perform the "operation items other than important operation items", which disrupts the rhythm of endoscopic operation and is prone to errors when resuming operation (for example, the operator makes a mistake when re-establishing the interaction between the hands and the hand control device), which may cause some serious consequences.

[0005] In addition, on the one hand, there are differences in understanding between those skilled in the art, and on the other hand, the applicant has studied a large number of literatures and patents when making the invention, but due to the limited space, all the details and contents are not listed in detail, which does not mean that the invention does not have these characteristics of the prior art. On the contrary, the invention already has all the characteristics of the prior art, and the applicant reserves the right to add relevant prior art in the background art. SUMMARY

[0006] In view of the deficiencies of the prior art, the present application provides an endoscope system, comprising: an endoscope module which enters a patient's body under the operation of an operator to acquire endoscopic images; an image display module which is in communication connection with the endoscope module and is used to display images to the operator; a control module which can control at least one working parameter of the endoscope system, the image display module is further configured to display a control display area and the endoscopic images to the operator, the control display area is at least functionally associated with part of the control module, and the endoscope system further comprises an information acquisition module which is configured to acquire voice or visual focus position information of the operator and transmit the information to a central processing module, wherein the central processing module is configured to generate an interaction instruction for controlling the control module based on the voice or visual focus information.

[0007] The present application further designs two non-contact control modes of voice control and eye movement control in addition to the hand control or foot control logic of the conventional endoscope operation system, and the operator can control additional operation items in a non-contact manner without releasing the hand operation, for example, control the display screen display picture brightness, electronic zoom ratio, etc., thereby significantly improving the flexibility of the operator in performing complex endoscopic operations and significantly reducing the risk of the operator's concentration decreasing due to the need for secondary operation in addition to the hand operation. The above-mentioned scheme changes the original operation mode in the conventional method, i.e., the operator needs to stop the important operation item being operated to operate the additional operation item, to the non-contact operation by using the eye movement or voice information of the operator, thereby significantly improving the operation convenience. Further, for eye movement, the operation implementation process can utilize the original display device in the endoscope system, and by superimposing a control display area which can be determined by eye movement on the endoscopic image displayed on the display device, the operator can control the system to input control instructions by eye movement without moving the line of sight away from the display content, thereby helping to keep the operator's attention on the endoscopic video content and preventing the operator from being distracted.

[0008] Preferably, the control display area and the endoscopic images are displayed in separate areas, and the control display area contains one or more graphical interaction objects in the form of graphical interfaces, wherein the control display area is configured to: input an interaction instruction to a certain graphical interaction object in the control display area, so that the control module corresponding to the graphical interaction object accepts the interaction to control at least one working parameter of the endoscope system, and / or accepts the state data of the working parameter fed back by the control module to display on the corresponding graphical interaction object in the control display area. The interaction here can be regarded as an instruction, and also as the interaction instruction described below. The state data of the working parameter refers to the current value of the working parameter, for example, the screen brightness is the working parameter, and the screen brightness of 70% is the state data.

[0009] The above scheme displays the control display area and the endoscope collected image display area separately. First, both display areas exist on the image display module, so when the operator's line of sight moves in the above two image display areas, it will not exceed the display range of the image display module, so as not to cause the operator to be distracted due to directing the line of sight outside the image display module. Secondly, by displaying the control display area and the endoscope collected image separately, the difficulty of collecting the operator's point of view position information can be reduced. Specifically, when it is detected that the operator's visual focus position is in the control display area, the frequency of eye movement detection is increased to accurately detect the graphical interactive object that the operator expects to operate, and conversely, when the visual focus position is on the endoscope collected image, the eye movement detection frequency is configured to be relatively low to save data processing power.

[0010] Preferably, the information collection module comprises a focus collection component configured to determine the operator's visual focus position on the displayed image of the image display module and transmit the visual focus position to the central processing module.

[0011] Preferably, the central processing module is configured to perform the following steps after receiving the visual focus position:

[0012] When the operator's visual focus position is in the endoscope collected image area, the central processing module does not perform the action of sending an interactive instruction to the "control module related to the graphical interactive object";

[0013] When the operator's visual focus position is in the control display area and above a certain graphical interactive object, the central processing module determines whether the operator expects to send an interactive instruction through the graphical interactive object based on a preset operation determination rule.

[0014] Preferably, the preset operation determination rule is to determine the length of time the operator's visual focus position stays on a specific graphical interactive object, and when the length of time meets a preset threshold, the central processing module sends an interactive instruction to the control module corresponding to the graphical interactive object.

[0015] The advantage is that the present endoscope system can form a verification and confirmation of the operator's eye movement instruction to prevent the situation of misinputting instructions.

[0016] Preferably, the information collection module further comprises a voice collection component that acquires the voice emitted by the operator to form voice information and transmits the voice information to the central processing module for processing and conversion into control instructions for the control module, and the central processing module transmits the control instructions to the corresponding control module.

[0017] The advantage is that the endoscope system provides another non-contact control mode, i.e. the corresponding function module can be controlled to generate actions by using the voice instruction of the operator. Preferably, in the case that the operation expected to be performed by the operator is relatively complex, some more complex instructions can be given by using voice, such as closing the display of the secondary window while enlarging the display of the primary window. The eye movement and voice two operation modes of the present scheme achieve the effect of mutual cooperation when performing simple or complex operations, significantly reduce the operation difficulty of the operator, and significantly expand the simultaneously operable items of the operator.

[0018] Preferably, in the case that the information acquisition module simultaneously acquires voice information and visual focus position, based on the pre-configured quick voice verification mark, it is determined whether the interaction instruction formed based on the visual focus position is transmitted to the corresponding control module.

[0019] Preferably, in the case that the quick voice verification mark is preset, the central processing module performs the following actions:

[0020] It is determined whether the visual focus position of the operator is on a certain graphical interaction object within a preset time length, and if it is kept on a certain graphical interaction object within the continuous time length, the corresponding control module performs interaction,

[0021] If the visual focus position is away from the graphical interaction object within the continuous time length, it is further confirmed whether the quick voice verification mark is collected by the voice collection component within the continuous time length.

[0022] If the quick voice verification mark is collected, it is further confirmed whether the visual focus position at the time point when the quick voice verification mark is collected is located on the first graphical interaction object with a relatively large time length proportion of the visual focus position within the preset time length.

[0023] If it is "located on the first graphical interaction object with a relatively large time length proportion of the visual focus position within the preset time length", the control module corresponding to the first graphical interaction object performs interaction.

[0024] If it is not "located on the first graphical interaction object with a relatively large time length proportion of the visual focus position within the preset time length", the timing is restarted.

[0025] If the visual focus position is kept on the second graphical interaction object after the preset time length, the control module corresponding to the second graphical interaction object performs interaction.

[0026] Preferably, if the visual focus position at the time point when the quick voice check mark is collected is not located on the first graphical interactive object on which the visual focus position is located for a large proportion of time within the preset time length, but is located on the second graphical interactive object, the timing is restarted, but a second preset time length different from the above-mentioned "preset time length" is used for judgment, and the second preset time length is shorter than the "preset time length", which has the advantage that when the operator changes the graphical interactive object and gives the quick voice check mark, the object that the operator desires to change the control of can be determined more quickly.

[0027] Unlike the prior art, the present scheme proposes a scheme of mutual checking based on two non-contact control methods to quickly form accurate instructions. Specifically, the present scheme finds that the operator cannot or is not convenient to use one of the non-contact control methods in some cases, for example, it is difficult to give a voice instruction while the operator is talking to other auxiliary personnel; and although eye movement control can quickly give instructions with less energy consumption, it is easy to produce false positives (because eye movement collection is easy to drift, and the person's eyes are easy to wander when looking at a certain graphical interactive object); voice control, although the accuracy of the given instruction is relatively high (the brain is more involved in organizing language than the process of looking at things with the eyes), but the process is relatively cumbersome and the timeliness is relatively poor. It is not difficult to see that the two instruction methods have their own advantages and disadvantages. In this dilemma, the present scheme proposes a quick voice check mark to quickly check whether the first interactive instruction is activated based on the quick voice check mark given by the operator. In this case, the operator does not need to speak the interactive instruction completely, but only needs to rely on a general and short voice to cooperate with the eye movement interaction to quickly realize the confirmation of the desired interactive instruction, which can effectively prevent the eye movement interaction from producing false recognition, further reduce the difficulty and waiting time of the eye movement interaction, provide a certain fault tolerance, and enable the system to more accurately recognize the eye movement interaction instruction of the operator and quickly execute, reduce the correction time, and make the endoscopic operation more smooth.

[0028] Preferably, the central processing module is configured to be able to pre-acquire the voiceprint information of the operator, so that when the voice information is subsequently acquired, the legality of the voice information can be checked based on the voiceprint information. The voiceprint information is acquired by the voice collection assembly during the process in which the operator reads the case information or the examination checklist displayed on the image display module before performing the endoscopy.

[0029] Preferably, the control module comprises at least a first control module for controlling the photographing of the endoscope, a second control module for controlling the screen image, a third control module for controlling the negative pressure suction valve, and a fourth control module for controlling the air valve, and the first control module, the second control module, the third control module and the fourth control module are respectively connected with the central processing module, and the display area displays the graphical interactive objects corresponding to the first control module, the second control module, the third control module and the fourth control module.

[0030] The present scheme has the following advantages:

[0031] 1) The endoscope system is adjusted by voice control and operator focus control, which avoids frequent switching of the operation posture due to multiple operation steps and multiple control switches, effectively avoids the consumption of the operator's energy caused by long-time manual operation, changes the operation, and further negatively affects the accuracy and safety of the endoscope system during use. Therefore, it is free from the dependence on the experience of the operator, easy to operate, thereby improving the use efficiency of the endoscope.

[0032] 2) The present scheme can set key nodes for the images obtained by the endoscope during the use of the endoscope system, so as to backtrack according to the key nodes when viewing the video later, facilitate the quick determination of the key points and difficulties (key steps) of the endoscope examination operation, or the quick determination of the lesion site of the patient, generate a preview image of the key information, and reduce the data storage amount. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is a schematic diagram of module connection of the present scheme;

[0034] Figure 2 is a schematic diagram of an image display module of the present scheme.

[0035] Reference signs:

[0036] 110: image display module; 111: first display area; 112: second display area; 120: information acquisition module; 121: voice acquisition component; 122: focus acquisition component; 130: central processing module; 140: control module; 141: first control module; 142: second control module; 143: third control module; 144: fourth control module; 145: fifth control module; 146: sixth control module. DETAILED DESCRIPTION

[0037] The present scheme will be described in detail below with reference to the accompanying drawings. Figure 1 and 2

[0038] ​The prior art endoscope is operated more, and from the most important part of the control endoscope operation, these operations have occupied the hands of the operator, and the operator must always pay attention to the hand control endoscope. In addition, there are some operations in the form of foot pedal to establish interaction with the endoscope system, which means that the operator also needs to issue control instructions by blind stepping. However, in addition, there are some operation items that need the operator to operate the control. Preferably, in order to solve the problem that the medical staff operates the endoscope system and causes the hands and feet to control multiple components or buttons at the same time, and easily causes operation errors, such as Figure 1 As shown in the figure, the present scheme provides an endoscope system, which at least includes an endoscope module (not shown), an image display module 110, an information acquisition module 120 and a central processing module 130. The endoscope module can be placed in the patient's body to obtain endoscopic image acquisition, and in the present scheme, the endoscope module can adopt the existing endoscope scheme. The endoscope module and the image display module 110 form a communication connection. The image display module 110 is used to show the image information obtained by the endoscope to the operator. Preferably, the image display module 110 also shows the image information obtained by the endoscope to the operator. The image display module 110 is configured as a display, which can show the image information to the operator. Basically, the image information is the image information obtained by the endoscope of the present endoscope system. Further, the image information also includes the image information of the control display area, that is, the image information obtained by the endoscope and the image information of the control display area are superimposed to form the total image information shown to the operator. For example, the control display area is one or more option button graphics of the graphical interface type, and then the control display area of the graphical interface type is superimposed on the image obtained by the endoscope without blocking, for example, the edge of the picture. The control display area can also be set to be semi-transparent to reduce its own blocking of the image obtained by the endoscope. Preferably, the image information displayed by the image display module 110 at least includes a first display area 111 of the image obtained by the endoscope and a second display area 112 of the display control display area image.

[0039] The control display region is functionally interactively associated with at least part of the control modules 140. The control modules 140 refer to the control modules 140 configured in the endoscope system and capable of controlling at least one working parameter in the endoscope system. The control modules 140 can be a separate circuit component arranged in the control loop of the endoscope system for controlling at least one working parameter of the endoscope system; in other embodiments, the control modules 140 can also be in an amorphous form, i.e., in the form of a corresponding control program or control loop or control interface, etc. One working parameter of the endoscope system can be any working parameter of any sub-device in the endoscope system, for example, the endoscope system can include an image processing display host, a scope connected to the host, an endoscope part on the scope entering the human body, and additionally can include a ventilation device, a water supply device, etc. The above devices are sub-devices of the endoscope system, and any sub-device can be adjusted in its working parameter, for example, the scope part can be adjusted in its working parameters such as angle, telescopic degree, valve switch, etc., and the endoscope part can be adjusted in its working parameters such as objective lens focal length, image sensor parameter, endoscopic diagnosis and treatment instrument action, etc. The part of the control modules 140 in the above endoscope system are functionally interactively connected with other entity control modules, for example, the hand-operated buttons, knobs, joysticks, foot pedals, etc. that have been maturely used in endoscopic examination, and these entity control modules can be functionally interactively associated with certain control modules 140. Preferably, the above "the control display region can be functionally interactively associated with part of the control modules 140" in one embodiment means that, in addition to the control modules 140 corresponding to the existing configured buttons, knobs, joysticks, foot pedals, etc. entity control modules, the remaining control modules 140 can be all or partially functionally interactively associated with the control display region. The functional interactive association means that the control modules 140 corresponding to a certain specific region of the control display region can accept the interaction to perform corresponding control work to control at least one working parameter of the endoscope system and / or accept the state data of the working parameter fed back by the control modules 140 to display in the corresponding specific region of the control display region. For example, the control display region is configured as an image type having one or more graphical interface buttons, each button can be functionally interactively associated with at least one control module 140, for example, a graphical interface button A is functionally interactively associated with a switch control module 140 of an auxiliary LED light of the endoscope, and a graphical interface button B is functionally interactively associated with a valve switch control module 140 of the scope, then when a control instruction is input to the corresponding A or B button, the corresponding control module 140 controls the corresponding component working parameter change (auxiliary LED on / off or valve on / off). The above interactive image in the form of a graphical interface can be referred to as a graphical interactive object.Further, the graphic interactive object can further have a further interactive mode, for example, an adjustment value of a parameter can be input, and a numerical value bar or an input box is formed as a control display area to constitute a part of the graphic interface. Preferably, each control module 140 is communicatively coupled to the central processing module 130, and the central processing module 130 is uniformly coupled to each graphic interactive object in the control display area.

[0040] Preferably, in an embodiment, the control module 140 at least includes a first control module 141 for controlling the endoscope photographing, a second control module 142 for controlling the screen image, a third control module 143 for controlling the negative pressure suction valve, and a fourth control module 144 for controlling the air valve. Preferably, the first control module 141, the second control module 142, the third control module 143, and the fourth control module 144 are signal-connected to the central processing module 130, and the control display area displays the graphic interactive objects corresponding to the first control module 141, the second control module 142, the third control module 143, and the fourth control module 144. Preferably, the graphic interactive object can include various function buttons, for example, a photographing option of the first control module 141, display zoom, storage, marking, and annotation function options of the second control module 142, and valve opening or closing options of the third control module 143 and the fourth control module 144.

[0041] Preferably, after the control display area constructed in the form of a graphic interface is configured, a relatively "virtual" control path different from the physical control module can be established, so that the operator can have an additional interactive path to operate more device working parameters, expand the working parameter objects that the operator can operate and control, and to a certain extent, reduce the number of physical control modules. Because the operator's hands and feet are occupied by the physical control modules when performing the endoscopic work, it is often necessary to use hands and feet to operate the physical control modules, and it is difficult to spare other operation actions to assign to other physical control modules. Furthermore, if the number of physical control modules is further increased, it may cause problems such as difficulty in placing the physical control modules, mutual position interference, and easy to cause misoperation. Therefore, the control display area provided by the present scheme does not have a physical structure and is not limited by the position of the wire, and can be relatively regularly arranged in the display area, which is convenient for the operator to operate and saves operation space.

[0042] Preferably, as can be known from the above, when performing endoscopic work, the hands and feet of the operator are usually occupied, so it is difficult to separate the hands and feet to operate other control modules 140 in the tense operation. Therefore, preferably, the information acquisition module 120 at least includes a focus acquisition component 122 capable of determining the visual focus position of the operator on the display image of the image display module 110 and transmitting the visual focus position to the central processing module 130. Preferably, the central processing module 130 determines whether the visual focus stays in the control display area based on the position of the visual focus position on the display image (the first display area 111 and the second display area 112), and when the visual focus stays in a certain graphical interactive object in the control display area and meets the preset operation determination rule, the central processing module 130 sends an interactive instruction to the operation module corresponding to the graphical interactive object, so that the corresponding working parameter can be adjusted.

[0043] Preferably, as Figure 2 shown, the focus acquisition component 122 is configured on the image display module 110 to obtain the visual focus position of the operator. Further, the focus acquisition component 122 can be an eye tracker or the like eye movement tracking system. The eye movement tracking system usually includes one or more light sources for providing infrared light to the eye of the operator, so that the image acquisition component configured in the eye movement tracking system can obtain the reflection image of the infrared light on the eye of the operator, and the processing component in the eye movement tracking system processes the reflection image, and according to the recognition of the cornea and the pupil, the processing component obtains the reflection angle vector of the light in the cornea and the pupil, and calculates the gaze direction of the eye of the operator based on the reflection angle vector, and obtains the visual focus position of the operator on the display area based on the known size and positional relationship of the display area. Preferably, calibration is required before using the focus acquisition component 122, for example, the four corner points of the display area can be used as standard points, and the operator is prompted to concentrate the gaze point on the corner point position, and then based on the image acquisition and processing of the optical reflection of the eye of the operator, the eye features of the operator are obtained, thereby facilitating subsequent eye movement recognition tracking.

[0044] Preferably, when the visual focus position of the operator is in the first display area 111, the central processing module 130 does not perform the action of sending an interaction instruction to the control module 140 related to the graphical interaction object, because at this time, the operator is observing the endoscope image within the first display area 111. When the visual focus position of the operator is in the second display area 112 and above a certain graphical interaction object, the central processing module 130 determines whether the operator intends to send an interaction instruction through the graphical interaction object based on a preset operation determination rule. In detail, the preset operation determination rule can be to determine the length of time that the visual focus position of the operator stays on a certain graphical interaction object, and when the length of time meets a preset threshold, the central processing module 130 sends an interaction instruction to the control module 140 corresponding to the graphical interaction object; additionally, a confirmation process can also be added, that is, after meeting the above-mentioned time threshold condition, a pop-up window is popped up in the control display area, and at least two options of determination and cancellation are given, and when the operator further moves the visual focus position to the determination or cancellation position, it is determined whether to send an interaction instruction to the corresponding control module 140 based on the selection result of the operator. Further, some graphical interaction objects can have more complex interaction methods, such as inputting numerical values, pulling numerical value bars, etc., and the corresponding visual focus position operation determination rule can be further designed based on the interaction method, such as providing a two-level input panel when inputting numerical values, and pulling the numerical value bar based on the change of the eye movement position and the change rate of the equal adjustment numerical value bar.

[0045] Preferably, the eye tracker and the like can only track the visual line of the operator. When using the endoscope system, the operator faces the display, and in the viewing direction of the eye tracker, the operator is located at the center position of the viewing direction, and other personnel are located at the edge area of the viewing direction of the eye tracker when viewing the display, therefore, the eye tracker can determine the visual line focus of the operator by tracking the visual line focus of the personnel at the center of the viewing direction, so that the central control module 140 can generate accurate eye movement interaction instructions, avoiding the visual line interference of other personnel when viewing the display.

[0046] Preferably, the information collection module 120 further comprises a voice collection component 121, in the case of using the endoscope system, the voice collection component 121 collects the voice of the operator to form voice information, and transmits the voice information to the central processing module 130 for processing and conversion into control instructions of the control modules 140, the central processing module 130 transmits the control instructions to the corresponding control modules 140 to realize voice control of the endoscope system. Preferably, the voice is related to the control of the endoscope system, and more preferably, the voice is within the range of the control modules 140 that can be interacted with the graphical interactive objects displayed on the second display area, i.e. the voice related to the control modules 140 described above can be recognized. In other embodiments, the range of voice control can also not be limited to the control modules 140 corresponding to the graphical interactive objects, but can also include other control modules 140. The voice collection component 121 can be a microphone or the like. In this case, the graphical interactive objects in the control display area should at least have a voice-associated label, such as a name, and the corresponding graphical interactive objects can be associated with the voice by the name, after the operator speaks the name of the graphical interactive object to the voice collection component 121, the central processing module 130 can know the graphical interactive object that the operator expects to operate based on the corresponding relationship between the collected language and the graphical interactive object, and perform the corresponding operation.

[0047] Preferably, the interactive instructions collected and generated by the focus collection component 122 described above are referred to as first interactive instructions, and the interactive instructions collected and generated by the voice collection component 121 are referred to as second interactive instructions.

[0048] Preferably, during the endoscopic operation, the operator often needs to pay more attention to more important aspects, so there is less energy to perform additional operations, and to ensure the smoothness of the endoscopic operation and improve operation efficiency, the operator is required to quickly perform some adjustment operations. Because the operator also needs to communicate with other auxiliary personnel and instruct auxiliary personnel to work during the endoscopic operation, the operator may not be able to issue a voice interaction instruction, i.e. a second interaction instruction, to the system. In addition, the eye movement interaction instruction, i.e. a first interaction instruction, is higher than the second interaction instruction in terms of input instruction efficiency, so in some cases that require fast input or continuous input, the role of the first interaction instruction is obvious. However, due to the error of eye movement detection, the operator's eye movement cannot reflect his thoughts in some cases, and the first interaction instruction often has inaccurate identification, that is, the first interaction instruction is less accurate than the second interaction instruction, because the second interaction instruction is the operator's voice instruction, which can relatively accurately reflect the operator's thoughts. The conventional method may think of using the operator's voice instruction to correct the eye movement instruction, but since the operator already needs to use the voice instruction to correct, it has violated the original intention of using the eye movement instruction, resulting in reduced operation efficiency. Therefore, the present application provides a preferred solution, a quick voice verification mark is set in the system, the verification mark is sound-based information, which can be voice or sound other than voice, and the setting rule of the verification mark is: the sound information as the verification mark at least does not appear significantly in other time periods or occasions that do not deliberately produce the sound information, in other words, the sound information is special, and it is very small probability to be emitted by natural communication of personnel, equipment sound, and environmental sound in the working process, for example, a specific number voice sound, a word group that appears very small probability in a working scene (for example, "drone", "elephant" and other word groups completely unrelated to the scene in the endoscope working environment) or a special tone that is different from the environmental sound (for example, a special tone whistle, continuous and rhythmic nasal sound, etc.). Preferably, the quick voice verification mark is set as short syllable information to facilitate the operator to quickly emit the corresponding sound.Furthermore, in the eye movement command detection link, the preset operation determination rules are updated to the following rules: determine whether the operator's visual focus position is on a certain graphic interaction object within the preset duration; if it remains on a certain graphic interaction object for the duration, the corresponding control module executes the interaction; if the visual focus leaves the graphic interaction object during the duration, further confirm whether the voice collection component 121 collects a quick voice verification mark within the duration; if it is collected, further confirm whether the visual focus position at the time point when the quick voice verification mark is collected is located on the graphic interaction object (here referred to as the first graphic interaction object, and the other one with a smaller proportion is the second graphic interaction object) with a larger proportion of the visual focus position within the preset duration; if so, the control module corresponding to the first graphic interaction object executes the interaction; if not, the timing is reset; if after the preset duration, if the visual focus position still remains on the second graphic interaction object, the control module 140 corresponding to the second graphic interaction object executes the interaction.

[0049] The above scheme can realize the rapid verification of whether the first interaction instruction is activated based on the quick voice verification mark given by the operator. In this case, the operator does not need to fully say the interaction instruction to be activated, but can rely on general and short voice to cooperate with eye movement interaction to quickly realize the confirmation of the expected interaction instruction. While effectively preventing the eye movement interaction from causing misrecognition, it further reduces the difficulty and waiting time of eye movement interaction, and provides a certain fault tolerance rate, which enables the system to more accurately identify the operator's eye movement interaction instructions and execute them quickly, reducing the correction time and making the endoscopic operation smoother.

[0050] Preferably, before the endoscope system is officially used, there must be steps such as preoperative examination and personnel confirmation. Preferably, before the endoscope system is officially used, the endoscope system can display case information or a checklist on the display, and the operator can read the text information of the case information or the checklist synchronously while looking at the case information or the checklist. The central control module 140 can obtain the voice information obtained by the voice acquisition component when the operator's line of sight is located in the case information or the checklist display area. Preferably, the central control module 140 converts the voice information collected by the voice acquisition component into text information through a voice recognition algorithm. When the text information matches the content of the case information or the checklist, the voiceprint information of the voice information corresponding to the text information is recorded, and the voiceprint information is stored as standard voiceprint information.

[0051] In the process of using the endoscope system, the central control module 140 checks the voice information obtained by the voice collection assembly through the verification model before converting the voice information into the first control instruction. Preferably, the verification model matches the voiceprint information of the voice information with the standard voiceprint information. If the voiceprint information of the voice information matches the standard voiceprint information successfully, the voice information is the voice issued by the operator, and the central control module 140 converts it into the first control instruction. If the voiceprint information of the voice information does not match the standard voiceprint information successfully, the voice information is an interference voice, and the central control module 140 no longer processes it.

[0052] Preferably, during the execution of the endoscopic work, the operator needs to analyze the images returned by the endoscope at the same time, and judge the disease conditions or other conditions of the endoscopic tissue parts of the patient in the images according to his own rich medical experience and knowledge. At the same time, in some endoscopic work, it is also necessary to use the endoscopic diagnosis and treatment equipment carried on the endoscope to process some diseased positions of the patient, such as using a miniature scissors to remove the endovascular sarcoma and the like. In other words, the endoscopic work is not only to obtain endoscopic images, but more importantly, to analyze the images and form treatment opinions. Therefore, during the execution of the endoscopic work, the records of the events occurring in time sequence and the corresponding endoscopic images or other information become very important. The endoscopic operation process involves many nodes, some of which are important, and some of which are relatively useless. If the video of the endoscope is analyzed back in the later period, it is difficult to find the key points, and the operator is also easy to overlook some important information that can only be found in the actual operation of the on-site environment, because compared with watching the video after the operation, the person in the actual operation of the on-site environment can pay attention to more detailed information. If this information is not effectively passed to the review and review of the endoscope, it may lead to a decrease in analysis efficiency, a delay in treatment, and even the possibility of missing some important information, which may have serious consequences, such as leaving the disease hidden, and errors in subsequent treatment plan design.

[0053] Preferably, based on the above problems, the present application proposes a preferred scheme, the endoscope system further comprises a storage module for storing the video stream obtained by the endoscope, and the central processing module 130 is signal connected with the storage module, wherein, in the process of storing the video stream obtained by the endoscope in the storage module, the central processing module 130 can obtain the specific instructions of the operator through the information collection module 120 to form the key nodes, so that when viewing the video, the target video section can be jumped to through the key nodes.

[0054] Preferably, the endoscope system comprises a storage module, which is capable of data communication with the endoscope image acquisition assembly at least, so that the storage module is capable of storing the endoscope acquisition image data, wherein after obtaining the special eye movement and / or voice information sent by the operator to the information acquisition module 120, the storage module stores the eye movement and / or voice information at the time when the special information is sent in combination with the corresponding endoscope acquisition image data according to the time node. The above-mentioned "after obtaining the special eye movement and / or voice information sent by the operator to the information acquisition module 120" can be judged by the central control module, and after the judgment, the central control module controls the storage module to perform the subsequent storage operation. The selected eye movement and / or voice information is stored together with the corresponding endoscope acquisition image data after being processed. For eye movement data, the path data of the visual focus position of the operator within a predetermined time before and after the time node of sending the special information is processed and saved in the image displayed on the image display module 110, and the animation of the visual focus position moving path can be viewed by the operator. For voice information, the audio file recorded within a predetermined time before and after the time node of sending the special information and the text file after voice conversion are packaged to form voice information for processing and saving.

[0055] The above-mentioned special information can be pre-determined eye movement or voice information, and the special information should ensure that the same special information does not appear in the endoscope process if the operator does not deliberately release the special information to the information acquisition module 120. For example, the operator performs a continuous and special eye movement in the detection domain of the focus acquisition assembly 122, such as connecting a special eye movement path in a certain order, such as drawing an 8-shaped or z-shaped figure. In the case of using voice, the special information is set with reference to the above-mentioned quick voice check mark. Thus, when the information acquisition module 120 acquires the related special information, the central processing module 130 can judge the special eye movement and / or voice information, and control the storage module to perform the above-mentioned storage operation.

[0056] The above scheme realizes that, in the endoscopy process, based on the active selection of the operator, the device automatically processes and records the eye movement and / or voice information related to the endoscope collected image without the need for the operator to manually input or operate, so that the operator can quickly find the corresponding image video node by searching for text keywords when reviewing the endoscope collected image to analyze the patient's condition, and can quickly obtain the voice content and focus direction at that time, which can help the operator quickly find the nodes that need attention. Compared with the prior art of saving all endoscope collected images or video content and providing them to the operator for review afterwards, the present scheme can record the eye movement and / or voice recording information of the operator during the endoscopy operation based on the process of the operator using the information collection module 120 of the system to control the working parameters of the related system components during the previous endoscopy operation, and can process it into a visual focus position moving path or voice information retained in a voice-to-text manner, so as to save the eye movement or voice recording of the operator during the actual operation based on his on-site judgment according to the comprehensive environment and conditions at that time. These records can significantly enhance the efficiency of the operator in viewing and analyzing key links when reviewing the endoscope collected data, significantly improve the success rate of the operator in obtaining important and easily missed information, and ensure that the patient can receive a higher success rate of treatment based on detailed and comprehensive analysis of the endoscope collected data.

[0057] Preferably, the central processing module 130 can generate a key node through the voice information of the operator, name the key node through the voice information, and set an explanatory label in the video stream stored in the storage module, so as to introduce the video content when the video is viewed later.

[0058] Preferably, the central processing module 130 can also determine whether to generate a key node by detecting whether the visual focus of the operator is located on the video node generation option in the second display area 112.

[0059] Preferably, the endoscope system further comprises a fifth control module 145 for controlling the movement of the mirror body and a sixth control module 146 for controlling the diagnosis and treatment equipment. Preferably, the movement of the mirror body and the diagnosis and treatment equipment can also be manually controlled by the operator.

[0060] It should be noted that the above-mentioned embodiments are examples, and those skilled in the art can think of various solutions under the inspiration of the disclosure of the present application, and these solutions also belong to the disclosed range of the present application and fall within the protection scope of the present application. Those skilled in the art should understand that the specification and drawings of the present application are illustrative and not limiting to the claims. The protection scope of the present application is defined by the claims and their equivalents.

[0061] The specification of the present application contains a plurality of inventive concepts, such as "preferably", "according to a preferred embodiment" or "optionally", each of which indicates that the corresponding paragraph discloses an independent concept, and the applicant reserves the right to file a divisional application according to each inventive concept. Throughout the text, the features introduced by "preferably" are only optional, and should not be understood as necessarily provided, so the applicant reserves the right to abandon or delete the relevant preferred features at any time.

Claims

1. An endoscope system comprising: An endoscope module, which is operated by an operator to enter the patient's body to obtain endoscopic images; an image display module (110), communicatively connected to the endoscope module, for displaying images to the operator; a control module (140) capable of controlling at least one operating parameter of the endoscope system, It is characterized by: The image display module (110) is configured to be able to display a control display area and an endoscope-collected image to the operator, and the control display area is functionally interactively associated with at least part of the control module (140). The endoscope system further comprises an information acquisition module (120) configured to acquire the operator's voice or visual focus position information and transmit the information to a central processing module (130), wherein the information acquisition module (120) further comprises a voice acquisition component (121), wherein the central processing module (130) is configured to generate an interactive instruction for controlling the control module (140) based on the voice or visual focus position information. When a fast voice verification flag is preset, the central processing module (130) performs the following actions: Determine whether the operator's visual focus is on a certain graphic interaction object within a preset time period. If the operator's visual focus remains on a certain graphic interaction object within the continuous time period, the corresponding control module (140) executes the interaction. If the visual focus position leaves the graphic interaction object within the duration, it is further confirmed whether the voice collection component (121) collects a quick voice verification mark within the duration. If it is collected, it is further confirmed whether the visual focus position at the time point when the quick voice verification mark is collected is located on the first graphic interaction object whose visual focus position duration accounts for a larger proportion within the preset duration. If so, the control module (140) corresponding to the first graphic interaction object executes the interaction. If not, the timing is reset. If the visual focus position still remains on the second graphic interaction object after the preset duration, the control module (140) corresponding to the second graphic interaction object executes the interaction.

2. The system according to claim 1, wherein: The control display area and the endoscope-collected image are displayed in separate areas, and the control display area contains one or more graphic interaction objects in the form of a graphical interface, wherein the control display area is configured to: input an interaction instruction to a certain graphic interaction object in the control display area, so that the control module (140) corresponding to the graphic interaction object accepts the interaction and controls at least one working parameter of the endoscope system.

3. The system according to claim 2, characterized in that The information acquisition module (120) includes a focus acquisition component (122), and the focus acquisition component (122) is configured to determine the visual focus position of the operator on the display image of the image display module (110), and transmit the visual focus position to the central processing module (130).

4. The system according to claim 3, characterized in that The central processing module (130) is configured to perform the following steps after receiving the visual focus position: When the operator's visual focus is in the endoscope image acquisition area, the central processing module (130) does not execute the action of sending an interaction instruction to the "control module (140) related to the graphic interaction object"; When the operator's visual focus is in the control display area and is located on a certain graphic interaction object, the central processing module (130) determines whether the operator expects to send an interaction instruction through the graphic interaction object based on a preset operation determination rule.

5. The system according to claim 4, characterized in that The preset operation determination rule is: judging the duration of time that the operator's visual focus position stays on a specific graphic interactive object, and when the duration meets a preset threshold, the central processing module (130) sends an interaction instruction to the control module (140) corresponding to the graphic interactive object.

6. The system according to claim 5, characterized in that The voice collection component (121) acquires the voice uttered by the operator to form voice information, and transmits the voice information to the central processing module (130) for processing and converting into control instructions of the control module (140). The central processing module (130) transmits the control instructions to the corresponding control module (140).

7. The system according to claim 6, characterized in that When the information acquisition module (120) simultaneously obtains voice information and a visual focus position, it is determined based on a pre-configured fast voice verification mark whether an interactive instruction formed based on the visual focus position is transmitted to the corresponding control module (140).

8. The system according to claim 7, characterized in that The central processing module (130) is configured to pre-acquire the operator's voiceprint information so that when the voice information is subsequently acquired, the legitimacy of the voice information can be verified based on the voiceprint information. The voiceprint information is acquired by the voice acquisition component (121) when the operator reads aloud the case information or the checklist displayed on the image display module (110) before performing the endoscopy.

9. The system according to claim 8, characterized in that The control module (140) at least includes a first control module (141) for controlling endoscopic photography, a second control module (142) for controlling screen images, a third control module (143) for controlling a negative pressure suction valve, and a fourth control module (144) for controlling an air valve. The first control module (141), the second control module (142), the third control module (143), and the fourth control module (144) are respectively connected to the central processing module (130) by signal, and the control display area displays graphic interaction objects corresponding to the first control module (141), the second control module (142), the third control module (143), and the fourth control module (144).

Images