Endoscope device control system and image output control method of endoscope device

Abstract

The invention provides an endoscope equipment control system and an image output control method of endoscope equipment. The control system comprises a field programmable gate array and an application processor, and the field programmable gate array comprises a first microkernel; the first microkernel is used for controlling the display of a data stream collected by an endoscope of the endoscope equipment after the control system is started; and the application processor is used for starting and running an application program of the endoscope equipment after the control system is started, and after the application program is started, the application program is utilized to control the display of the data stream instead of the first microkernel. According to the scheme, the electrifying and drawing speed of the endoscope can be increased.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic control, and particularly to an endoscope device control system and a method for controlling image output of an endoscope device. Background Art

[0002] As a special medical device, the power-on image output time of an endoscope device is a key performance indicator. The power-on image output time of endoscope devices with different design schemes is also different. The power-on image output of an endoscope device refers to the process of outputting the data stream collected by the endoscope for display after the endoscope device is started and the insertion of the endoscope is detected.

[0003] Some existing endoscope devices regulate the power-on image output process of the endoscope device through a software operating system such as Windows or Linux. However, since the startup of the software operating system itself takes a long time, it will greatly prolong the power-on image output process of the endoscope device, thereby affecting the process of using the endoscope for treatment and diagnosis, and causing trouble for the use of the endoscope device.

[0004] There are also some endoscope devices that use a single-chip microcomputer as the main controller for control. However, the single-chip microcomputer cannot complete relatively perfect user interaction, on-screen display (OSD) adjustment, and video recording storage and other functions. This method will affect the user experience during the use process after image output, and the endoscope device cannot be adjusted flexibly.

[0005] Therefore, a solution that can improve the power-on image output speed of the endoscope device and provide flexible human-computer interaction for users is needed. Summary of the Invention

[0006] The present invention is proposed in view of the above problems.

[0007] According to one aspect of the present invention, there is provided an endoscope device control system, including:

[0008] A field programmable gate array (FPGA) and an application processor, wherein the FPGA includes a first microkernel; the first microkernel is used to control the display of the data stream collected by the endoscope of the endoscope device after the control system is started; the application processor is used to start and run the application program of the endoscope device after the control system is started, and after the application program is started, take over the first microkernel to control the display of the data stream by using the application program.

[0009] Exemplarily, the first microkernel controls the display of the data stream collected by the endoscope by performing the following operations: monitoring the connection status between the endoscope and the control system, and when it is determined that the endoscope is connected to the control system, reading the body information of the endoscope; establishing and monitoring a data link between the endoscope and the control system; and based on the body information, controlling the display of the data stream using the data link.

[0010] Exemplarily, the first microkernel is further configured to lock the data link after the first microkernel is started, and unlock the data link after the first microkernel controls the start of the display of the data stream; wherein, when the data link is in a locked state, only the first microkernel can control the display of the data stream.

[0011] Exemplarily, the first microkernel locks the data link after the first microkernel is started, including performing the following operations: when the first microkernel monitors that the endoscope is disconnected from the control system, locking the data link.

[0012] Exemplarily, the first microkernel unlocks the data link after the first microkernel controls the start of the display of the data stream, including performing the following operations: after the first microkernel controls the start of the display of the data stream, when it is monitored that the endoscope is connected to the control system and the data link has been established, unlocking the data link.

[0013] Exemplarily, the body information of the endoscope includes the image output parameters of the endoscope and / or the type of the endoscope. The first microkernel is further configured to, before establishing the data link, obtain the read / write permission for the body information of the endoscope, and determine whether the type of the endoscope is correct. Wherein, the first microkernel establishes and monitors the data link only when the read / write permission is obtained and the type of the endoscope is correct.

[0014] Exemplarily, the first microkernel monitors the connection status between the endoscope and the control system and monitors the data link, which is always executed after the first microkernel is started.

[0015] Exemplarily, the application processor is further configured to, after the application program is started, in response to a user's configuration instruction, transmit the configured body information to the first microkernel for the first microkernel to write into the memory of the endoscope as the body information of the endoscope.

[0016] Exemplarily, the application processor is further configured to prohibit the first microkernel from controlling the display of the data stream after the application program is started.

[0017] Exemplarily, the field programmable gate array further includes a second microkernel; the second microkernel is used to initialize the video output interface of the endoscope device and monitor the status of the video output interface; the first microkernel controls the display of the data stream collected by the endoscope of the endoscope device by performing the following operations:

[0018] Controlling the data stream to be output to a display device via the video output interface and displaying an image corresponding to the data stream on the display device.

[0019] Exemplarily, the application processor is further used to start the first microkernel and the second microkernel.

[0020] Exemplarily, the field programmable gate array and the application processor are integrated in a multi-processor system-on-chip.

[0021] According to another aspect of the present invention, there is also provided a method for controlling image output of an endoscope device. The method for controlling image output is applied to an endoscope device control system. The control system includes: a field programmable gate array and an application processor. Wherein, the field programmable gate array includes a first microkernel; the method for controlling image output includes: using the first microkernel to control the display of the data stream collected by the endoscope of the endoscope device after the control system is started; using the application processor to start and run the application program of the endoscope device after the control system is started. After the application program is started, taking over from the first microkernel to control the display of the data stream by using the application program.

[0022] In the above technical solution, after the control system is started, the first microkernel controls the display of the data stream collected by the endoscope of the endoscope device, and after the application program is started, the application processor takes over from the first microkernel to control the display of the data stream by using the application program. Thereby avoiding the problem of slow image output when the endoscope is powered on caused by using the application program to control the above process, and reducing the time for using the endoscope for diagnosis and treatment. And after the application program in the application processor is started, its functions such as human-computer interaction can be utilized to improve the user experience after the endoscope device outputs an image when powered on.

[0023] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the description. And in order to make the above and other objects, features and advantages of the present invention more obvious and understandable, the following specifically illustrates the specific embodiments of the present invention. Brief Description of the Drawings

[0024] The embodiments of the present invention will be described in more detail with reference to the accompanying drawings, and the above and other objects, features, and advantages of the present invention will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings, the same reference numerals generally represent the same components or steps.

[0025] Figure 1 FIG. 4 shows a schematic block diagram of an endoscope device control system according to an embodiment of the present invention;

[0026] Figure 2 FIG. 8 shows a schematic flow chart of an image output control method for an endoscope device according to an embodiment of the present invention;

[0027] Figure 3 FIG. 12 shows a schematic flow chart of an image output control method for an endoscope device according to another embodiment of the present invention; and

[0028] Figure 4 FIG. 16 shows a schematic diagram of an image output control process for an endoscope device according to an embodiment of the present invention;

[0029] Figure 5 FIG. 20 shows a schematic diagram of a power-on image output control process for an endoscope device according to an embodiment of the present invention; and

[0030] Figure 6 FIG. 24 shows a schematic diagram of an image output process for controlling the display of data streams by an application program after the application program is started according to an embodiment of the present invention. Detailed Embodiments

[0031] In order to make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments of the present invention. It should be understood that the present invention is not limited by the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

[0032] In order to at least partially solve the above problems, according to one aspect of the embodiments of the present application, an endoscope device control system is provided. Figure 1 FIG. 34 shows a structural diagram of an endoscope device control system 100 according to an embodiment of the present application. The endoscope device control system 100 can be disposed inside the endoscope device. The endoscope device may further include an endoscope for collecting data streams of an object to be examined. As Figure 1As shown, the endoscope device control system 100 includes an FPGA 110 and an application processor 120. The endoscope device may further include a video output interface, which can be used to connect a display device, the FPGA 110, and the application processor 120 to implement data and display of data streams. The video output interface is, for example, a High-Definition Multimedia Interface (HDMI), a DisplayPort (DP), etc.

[0033] The FPGA 110 includes a first microkernel 111. The first microkernel 111 can be a processor soft core embedded in the FPGA 110. After the first microkernel 111 is started, it can execute corresponding tasks and implement corresponding functions. It can be understood that compared with the application processor 120, the startup of the FPGA is relatively fast. In other words, the startup of the first microkernel 111 is also relatively fast, for example, it can be started within a few seconds, such as 3 to 8 seconds.

[0034] Exemplarily, the first microkernel 111 is used to control the display of the data stream collected by the endoscope of the endoscope device after the control system is started.

[0035] The endoscope of the endoscope device can be an endoscope, a camera, etc. attached to the endoscope device itself, or an external endoscope, a camera, etc. The endoscope can be a flexible endoscope or a rigid endoscope. The endoscope of the endoscope device is used to collect the data stream of the object to be measured. The data stream can include data in the form of video, pictures, etc. When the data stream is displayed, the video, image, etc. corresponding to the data form of the data stream can be displayed.

[0036] The application processor 120 is used to start and run the application program of the endoscope device after the control system is started. After the application program is started, it takes over from the first microkernel 111 to control the display of the data stream using the application program. After the application program is started, it has loaded all the required running resources and can be used to display the content of the data stream. It can be understood that after the application program is started, it replaces the first microkernel 111 that is controlling the image output to perform the control of the image output operation. In addition to controlling the display of the data stream, the application program can also implement functions such as image parameter adjustment, human-computer interaction, and screenshot and video recording. The application program can be a Linux application program, a Windows application program, etc. The application processor 120 can include multiple application processor cores, and the multiple application processor cores can be different to improve the performance of the application processor 120 and optimize the human-computer interaction experience between the user and the endoscope device.

[0037] In the above technical solution, the first microkernel 111 is used to control the display of the data stream after the control system is started; and the application program running in the application processor 120 is used to perform human-computer interaction and take over the control of the display of the data stream from the first microkernel 111 after the application program is started. Thus, the FPGA 110 and the application processor 120 realize the staged control of the data stream display, which significantly improves the speed of the endoscope power-on image output, for example, the image can be quickly output within 10 seconds, reducing the time for diagnosis and treatment using the endoscope. And after the application program in the application processor 120 is started, it can be used to perform functions such as human-computer interaction, so as to improve the user experience after the endoscope device is powered on and the image is output.

[0038] Exemplarily, the first microkernel 111 can be used to monitor the connection status between the endoscope and the control system, and when it is determined that the endoscope is connected to the control system, the body information of the endoscope is read. When the control system receives the connection signal of the endoscope, it indicates that the endoscope of the endoscope device is connected to the control system at this time. The connection signal of the endoscope can be a real connection signal generated by the insertion of the endoscope, such as an electrical signal, an optical signal, etc., or it can be a generated virtual signal, such as a program instruction, a digital signal, etc., which is not limited here. When the endoscope is connected to the control system, the control system can obtain a data stream from the endoscope for output and display; otherwise, the control system will have no data stream input, and thus cannot output the data stream to be displayed. The first microkernel 111 is also used to read the body information of the endoscope when it is determined that the endoscope is connected to the control system. The body information of the endoscope can be stored in the memory of the endoscope. For example, an encryption chip can be provided in the endoscope, in which a memory can be provided. The body information can be stored in the memory in the encryption chip. The scope information may include the image output parameters of the endoscope, the type of the endoscope, the life of the endoscope, etc. The scope information may be used to control the endoscope image output and configure the image output parameters, etc. When it is determined that the endoscope is connected, the first microkernel 111 may access the memory of the endoscope to read the scope information of the endoscope.

[0039] The first microkernel 111 is also used to establish and monitor a data link between the endoscope and the control system. The data link is a data transmission channel for transmitting the data stream collected by the endoscope, and the data link is a virtual data transmission channel, such as a data channel for communication between program interfaces, rather than a real physical transmission channel. The data link can be requested to be established by accessing the program interface. Only after the data link has been successfully established and the link state is maintained can the data stream collected by the endoscope be correctly transmitted; otherwise, the data stream collected by the endoscope cannot be correctly transmitted. As mentioned above, the body information of the endoscope can be used to control the image output of the endoscope. The first microkernel 111 is also used to determine whether the data link has been established after reading the body information of the endoscope and obtaining the image output parameters of the endoscope. If it is determined that the data link has not been established, the operation of establishing the data link is cyclically executed within a preset time period to control the data stream to be output via the video output interface after the data link has been established.

[0040] The first microkernel 111 is also used to control the display of the data stream based on the body information by using the data link. The first microkernel 111 can read the body information of the endoscope by accessing the memory of the endoscope, determine the corresponding image output configuration parameters based on the read body information, and then control the display of the data stream according to the image output configuration parameters. The data stream can be displayed as a dynamic video or a static image, etc. The format of the video frame or the image in the video can be a black-and-white image, a color image, a grayscale image, etc., which is not limited in this application. The first microkernel 111 can be started prior to the application program of the application processor. The start-up time of the former only takes a few seconds, while the latter may take dozens of seconds, such as 30 seconds. Before the application program is started, the first microkernel 111 can control the display of the data stream to achieve the image output of the endoscope. Thus, the user can obtain the image of the endoscope device relatively quickly.

[0041] In the above technical solution, after the control system is started and before the application program is started, only by using the first microkernel 111, the monitoring of the connection state between the endoscope and the control system, the reading of the body information of the endoscope, and the control of the display of the data stream are realized. Thus, the FPGA 110 controls the display of the data stream, which significantly improves the speed of the endoscope power-on image output compared with directly controlling the display of the data stream by the application processor 120 throughout the process.

[0042] Exemplarily, the operation of the first microkernel 111 monitoring the connection state between the endoscope and the control system and monitoring the data link is always executed after the first microkernel 111 is started.

[0043] The connection states between the endoscope and the control system include that the endoscope is not connected to the control system, the endoscope is first connected to the control system after the control system is powered on, and the endoscope lens is in a state from being pulled out to being inserted after the control system is powered on. The operations of the first microkernel 111 for monitoring the connection state between the endoscope and the control system and for monitoring the data link can be repeatedly executed at a preset frequency during the power-on process of the control system.

[0044] In the above technical solution, by always performing the above monitoring operations during the power-on process of the control system, corresponding processing can be performed in a timely manner when the connection state between the endoscope and the control system and the data link change. This can prevent the endoscope device from displaying incorrect images and ensure that correct images or videos are displayed in a timely manner.

[0045] Exemplarily, the FPGA 110 may further include a second microkernel 112. The second microkernel 112 may be a processor soft core embedded in the FPGA 110. After the second microkernel 112 is started, it can execute corresponding tasks and implement corresponding functions. It can be understood that compared with the application processor 120, the startup of the FPGA is relatively fast. In other words, the startup of the second microkernel 112 is also relatively fast, for example, it can be started within a few seconds, such as 3 to 8 seconds.

[0046] The second microkernel 112 is used to initialize the video output interface of the endoscope device and monitor the state of the video output interface. The video output interface is used to connect the endoscope device and the display device and transmit the data stream to the display for display. The second microkernel 112 can initialize the video output interface and monitor its state to facilitate the transmission of the data stream. The second microkernel 112 can work simultaneously with the first microkernel 111. The first microkernel 111 is also used to control the data stream to be output to the display device via the video output interface and display the image corresponding to the data stream on the display device, for example, output the image corresponding to the data stream to the display device for display. The format of the image can be a black-and-white image, a color image, a grayscale image, etc., and the present application does not limit this.

[0047] In the above technical solution, by initializing the video output interface by the second microkernel 112 and monitoring the state of the video output interface, the workload of the first microkernel 111 can be reduced, and the image output speed of the endoscope device after power-on can be further improved.

[0048] Exemplarily, FPGA 110 and application processor 120 can be integrated into a multi-processor system-on-chip (MPSOC) inside the endoscopic device. This multi-processor system-on-chip can be used to connect the video output interface and the input interface of the endoscope to implement functions such as monitoring the connection status of the endoscope, reading the body information of the endoscope, acquiring the data stream collected by the endoscope, and outputting the data stream for display. Both FPGA 110 and application processor 120 are on a multi-processor system-on-chip, and the timing control and communication speed between them are better. At the same time, this solution better integrates the advantages of FPGA 110 and application processor 120. FPGA 110 has strong image processing capabilities and is only used for operations related to algorithm processing of the data stream. The first microkernel 111 and the second microkernel 112 in FPGA 110 start quickly and have a small capacity, and can be flexibly designed for body control and video output interface driving in the fast startup phase to ensure fast image output of the endoscopic device. The application processor 120 has strong performance and can support scenarios that require strong resources to ensure subsequent human-computer interaction and adjustment of display configuration on the screen.

[0049] In the above technical solution, by integrating FPGA 110 and application processor 120 on the multi-processor system-on-chip, the system operating resources can be effectively allocated during the image output of the endoscope and human-computer interaction, thereby improving the operating efficiency, increasing the image output speed of the endoscope, and ensuring the smooth implementation of functions such as human-computer interaction after image output.

[0050] Exemplarily, the application processor 120 is also used to start the first microkernel 111 and the second microkernel 112.

[0051] The application processor 120 can control the operating system kernel using a Universal Boot Loader (UBOOT). Specifically, the application processor 120 can start the Universal Boot Loader stored thereon and start the first microkernel 111 and the second microkernel 112 in FPGA 110 through this Universal Boot Loader.

[0052] In the above technical solution, the application processor 120 starts the first microkernel 111 and the second microkernel 112, enabling the first microkernel 111 and the second microkernel 112 to start quickly and control image output when the endoscopic device is powered on, thereby reducing the time-consuming of the endoscopic device for power-on image output.

[0053] Exemplarily, the first microkernel 111 is further configured to lock the data link after the first microkernel 111 is started, and unlock the data link after the first microkernel 111 starts to display the control data stream. When the data link is in the locked state, only the first microkernel 111 can control the display of the data stream. When the data link is in the locked state, only the first microkernel 111 can control the display of the data stream. Only when the data link is in the unlocked state can the application start a human-machine interaction and control the display of the data stream after startup. In other words, when the data link is in the locked state, the permission of the application to read and write the mirror body is cancelled. At this time, the application manager cannot use the application to read and write the endoscope mirror body information, so it cannot control the display of the data stream. Thus, it is prevented that while the first microkernel 111 controls the display of the data stream, the application manager also controls the display of the data stream using the application. The data link can be locked by enabling the first microkernel 111 to control its internal process. After the first microkernel 111 starts to display the control data stream, the data link can be unlocked through an internal process. After that, the application manager cannot use the application to obtain the permission to read and write the mirror body and can control the display of the data stream.

[0054] In the above technical solution, by locking the data link after the first microkernel 111 is started and unlocking the data link after the data stream starts to be displayed, the application does not have the permission to access the mirror body during the process of the first microkernel 111 controlling the data stream, preventing the application manager from using the application to preempt the data link and affecting the image output effect.

[0055] Exemplarily, the first microkernel 111 locks the data link after the first microkernel 111 is started, including performing the following operations: when the first microkernel 111 monitors that the endoscope is disconnected from the control system, lock the data link. When the endoscope is disconnected from the control system, it means that the insertion of the endoscope mirror body is not detected at this time, and the first microkernel 111 cannot normally obtain the data stream collected by the endoscope. At this time, the data link can be locked. When the endoscope is monitored to be connected to the control system next time, the first microkernel 111 first controls the display of the data stream.

[0056] In the above technical solution, when it is determined that the endoscope is disconnected from the control system, after the first microkernel 111 locks the data link, the application cannot control the display of the data stream, preventing the application from displaying the wrong data stream because it cannot obtain the correct data stream collected by the endoscope.

[0057] Exemplarily, the first microkernel 111 controls the data stream to start displaying and then unlocks the data link, including performing the following operations: after the first microkernel 111 controls the data stream to start displaying, when it is monitored that the endoscope is connected to the control system and the data link has been established, the data link is unlocked. After the data stream starts to be displayed, the hardware connection status between the endoscope and the control system can be monitored at all times. If the endoscope and the control system have been disconnected, they need to be reconnected. Similarly, the status of the data link is also monitored at all times, and if the data link is not established, the data link is reestablished. For the case where the endoscope is connected to the control system and the data link has been established, a data link unlocking operation can be performed. At this time, after the data link is unlocked, the display of the data stream can still be controlled by the first microkernel 111, but the application can access the data link and control the display of the data stream.

[0058] In the above technical solution, the data link unlocking operation is performed after the data stream starts to be displayed, which ensures that the application has the permission to access the data link, thereby ensuring that it can subsequently access the data link to perform human-computer interaction and control the display of the data stream, thereby improving the user experience of the endoscopic device.

[0059] Exemplarily, the endoscope body information includes the endoscope output parameters and / or the type of the endoscope. The first microkernel 111 is also used to obtain read and write permissions for the endoscope body information before establishing a data link, and to determine whether the type of the endoscope is correct. The first microkernel 111 establishes and monitors the data link only when the read and write permissions are obtained and the type of the endoscope is correct.

[0060] The image output parameters of the endoscope can be used by the first microkernel 111 to control the display of the data stream collected by the endoscope. The type of the endoscope can be used by the first microkernel 111 to determine whether the endoscope information is read successfully. The body information such as the image output parameters of the endoscope and / or the type of the endoscope can be pre-stored in the memory of the endoscope body, and can be changed through a write operation. The memory of the endoscope body can be set with a key. If the key can be correctly decrypted when the first microkernel 111 reads and writes the body information of the endoscope, it can be considered that the first microkernel 111 has the read and write permission of the body information. The endoscope device may only be able to output and display a data stream collected by a specific type of endoscope. Only when the first microkernel 111 obtains the read and write permission and the type of the endoscope is correct, can the data link be established and monitored to transmit the data stream.

[0061] In the above technical solution, the first microkernel 111 obtains read and write permissions and determines the type of endoscope, ensuring that the transmission can be controlled and the correct content of the data stream can be displayed.

[0062] Exemplarily, the application processor 120 is also used to transmit the configured scope information to the first microkernel 111 in response to the user's configuration instructions after the application is started, so that the first microkernel 111 writes it into the memory of the endoscope as the scope information of the endoscope.

[0063] After the application is started, the user's configuration instruction can be generated according to the user's operation of configuring the image output parameters, and the configuration instruction is sent to the first microkernel 111. The configuration instruction may include the configured scope information. The scope information may include the image output parameters of the endoscope and other configuration information. When the first microkernel 111 receives the configuration instruction, it writes the configured scope information into the memory of the endoscope to update the scope information stored therein.

[0064] In the above technical solution, the first microkernel 111 receives the configuration instructions generated by the application to write the scope information, thereby supporting the user to change the display mode and content of the data stream collected by the endoscope through the application, thereby improving the human-computer interaction experience between the user and the endoscope device.

[0065] Exemplarily, the application processor 120 is further configured to prohibit the first microkernel 111 from controlling display of the data stream after the application program is started.

[0066] The application processor 120 may enable a control tool to prohibit the first microkernel 111 from controlling the display of the data stream after the application is started. The control tool may be a program instruction of the application. For example, when the application is a Linux application, the first microkernel 111 may be controlled to run empty through the Linux_debug instruction, that is, only the output of the data stream is controlled but the display of the data stream is not controlled. At this time, the display of the data stream may be controlled through the application.

[0067] In the above technical solution, the application prohibits the first microkernel 111 from controlling the display of the data stream by enabling the control tool, which can reduce the operating resources required by the first microkernel 111 after power-on image output on the endoscope device and improve the efficiency of the control system.

[0068] Figure 2 The flowchart of the image output control method 200 of an endoscope device according to an embodiment of the present invention is shown. The image output control method 200 is executed by an endoscope device control system. The endoscope device control system includes: an FPGA and an application processor. The FPGA includes a first microkernel and a second microkernel. Figure 2 As shown, the image output control method 200 includes step S210 and step S220.

[0069] In step S210, the first microkernel is used to control the display of the data stream collected by the endoscope of the endoscope device after the control system is started.

[0070] In step S220, after the control system is started, the application processor starts and runs the application program of the endoscope device. After the application program is started, it takes over the first microkernel to control the display of the data stream by using the application program.

[0071] In the above technical solution, after the control system is started, the first microkernel controls the display of the data stream collected by the endoscope of the endoscope device, and after the application program is started, the application processor takes over the first microkernel to control the display of the data stream by using the application program. Thereby, the problem of slow image output when the endoscope is powered on caused by using the application program to control the above process is avoided, and the time for using the endoscope for diagnosis and treatment is reduced. And after the application program in the application processor is started, its functions such as human-computer interaction can be utilized to improve the user experience after the endoscope device outputs an image when powered on.

[0072] Exemplarily, controlling the display of the data stream collected by the endoscope of the endoscope device after the control system is started includes: monitoring the connection status between the endoscope and the control system, and when it is determined that the endoscope is connected to the control system, reading the body information of the endoscope; establishing and monitoring the data link between the endoscope and the control system; and based on the body information, controlling the display of the data stream by using the data link.

[0073] Exemplarily, the image output control method further includes: using the first microkernel to lock the data link after the first microkernel is started, and unlocking the data link after the first microkernel controls the start of the display of the data stream; wherein, when the data link is in the locked state, only the first microkernel can control the display of the data stream.

[0074] Exemplarily, locking the data link after the first microkernel is started includes: when it is monitored that the endoscope is disconnected from the control system, using the first microkernel to lock the data link.

[0075] Exemplarily, unlocking the data link after controlling the start of the display of the data stream includes: after the first microkernel controls the start of the display of the data stream, when it is monitored that the endoscope is connected to the control system and the data link has been established, unlocking the data link.

[0076] Exemplarily, the body information of the endoscope includes the image output parameters of the endoscope and / or the type of the endoscope. The image output control method further includes: before establishing the data link, using the first microkernel to obtain the read and write permissions of the body information of the endoscope, and determining whether the type of the endoscope is correct. Among them, the first microkernel establishes and monitors the data link only when the read and write permissions are obtained and the type of the endoscope is correct.

[0077] Exemplarily, the steps of using the first microkernel to monitor the connection status of the endoscope device and the endoscope and the control system and monitor the data link are always executed after the first microkernel is started.

[0078] Exemplarily, the image output control method further includes: after the application program is started, using the application processor to transmit the configured endoscope body information to the first microkernel in response to a user's configuration instruction, so that the first microkernel writes it into the memory of the endoscope as the endoscope body information.

[0079] Exemplarily, the image output control method further includes: using the application processor to prohibit the first microkernel from controlling the display of the data stream after the application program is started.

[0080] Exemplarily, the FPGA further includes a second microkernel, and the image output control method further includes: using the second microkernel to initialize the video output interface and monitor the status of the video output interface; controlling the display of the data stream collected by the endoscope of the endoscope device includes performing the following operations: controlling the data stream to be output to the display device via the video output interface and displaying an image corresponding to the data stream on the display device.

[0081] Exemplarily, the image output control method further includes: using the application processor to start the first microkernel and the second microkernel.

[0082] Exemplarily, the image output control method further includes: the FPGA and the application processor are integrated in a multi-processor system-on-chip.

[0083] Figure 3 The flowchart of the image output control method 300 of the endoscope device according to another embodiment of the present invention is shown. As Figure 3 shown, the image output control method includes the following steps.

[0084] First, after the endoscopic device is powered on and started, the multi-processor system-on-chip FPGA and the application processor of the endoscopic device will be started. Among them, the time taken to start the FPGA is approximately 1.5 seconds. After starting the FPGA, the control system will lock the data link, so that only the first microkernel in the FPGA can control the power-on image output process of the endoscopic device. Then it will enter the power-on image output process of the endoscopic device controlled by the first microkernel. It can be understood that during the operation of the computing device, maintaining a certain process can be achieved by loop-executing the process. In this embodiment, the continuous power-on image output process can be achieved by looping the operations required in the power-on image output process of the endoscopic device. After the first microkernel is started, during the power-on image output process of the endoscope controlled by the first microkernel, the first microkernel can determine whether it is running empty by determining whether the application processor enables the control tool Linux_debug. When the application processor does not enable the control tool, the first microkernel determines that it is not running empty. In this case, the first microkernel can read the body information of the endoscope and perform image output display according to the read body information. When the application processor enables the control tool, it is determined to be running empty. In this case, the first microkernel can only read and write the body information of the endoscope and control the output of the data stream collected by the endoscope body through the video output interface, and will not control the display of the data stream according to the read body information. The display of the data stream is controlled by the application program.

[0085] In Figure 3 the embodiment, when the first microkernel determines that it is not running empty, the first microkernel determines whether the endoscope is connected to the control system. When it is determined that the endoscope is not connected to the control system, the data link can be locked to ensure that the power-on image output process of the endoscopic device is only executed by the first microkernel. The data link is used to transmit the data stream collected by the endoscope. Then the first microkernel will determine whether the endoscope is disconnected from the control system. When it is determined that the endoscope is not in the state of being connected to the control system and then disconnected from the control system, the power-on image output process of the endoscopic device will be loop-executed again. When it is determined that the endoscope is in the state of being connected to the control system and then disconnected from the control system, the isolation board will be controlled to power off. Only when the isolation board is powered on can the control system normally obtain the data stream collected by the endoscope. Through the above process, the first microkernel can monitor the connection state of the endoscope body, and this process is continuously executed after the first microkernel is started. Since the first microkernel can monitor whether the endoscope is connected to the control system by loop judgment, the endoscope being connected to the control system and then disconnected from the control system means that in the previous power-on image output operation loop, it was determined that the endoscope was connected to the control system, and in the current power-on image output operation loop, it is determined that the endoscope body is disconnected from the control system.

[0086] In Figure 3In the embodiment, after the first microkernel determines that the endoscope is connected to the control system, it can further determine whether the control system recognizes that the endoscope is connected to the control system for the first time after power-on or whether the endoscope has never been connected to the control system and has been inserted into the control system. The endoscope has never been connected to the control system and has been inserted into the control system, which means that the endoscope is judged to be not connected to the control system in the last power-on and image-outputting operation cycle during the endoscope device to output images, and the endoscope is judged to be connected to the control system in the current power-on and image-outputting operation cycle. When the first microkernel determines that the current power-on and image-outputting operation cycle is the first time that the endoscope is recognized to be connected to the control system after the control system is powered on or the endoscope has never been connected to the control system and has been inserted into the control system, the key (CID) of the connected endoscope is read and decrypted. The key of the endoscope can be the key of the encryption chip where the memory of the endoscope is located. The memory can be used to store the body information of the endoscope and the corresponding output parameters. The body information of the endoscope in the memory can only be read when the CID decryption is successful.

[0087] exist Figure 3 In the embodiment, when the CID is decrypted correctly, it means that the first microkernel has obtained the read and write permission of the endoscope's body information, and can read and write the endoscope's body information. After that, the first microkernel can store the body information in the shared memory of the control system, and control the isolation board to power on according to the body information. The shared memory can be a storage space in the control system for caching the read body information. Then the first microkernel can read the type information of the endoscope in the body information, and determine whether the type of the endoscope is read correctly. When it is determined that the type of the endoscope is read correctly, the first microkernel can determine whether the data link has been established. If the data link has been established at this time, the first microkernel can control the data stream to be output through the video output interface, and control the display of the data stream to output the image. For example, the first microkernel can control the display of the data stream to output the image in the form of an image or video based on the output parameters stored in the shared memory. Among them, the format of the image or video is not limited here. If the data link is not established at this time, the operation of establishing the data link is executed cyclically until it is determined that the data link has been established after a predetermined time. The first microkernel then controls the data stream to be output through the video output interface and controls the display of the data stream.

[0088] exist Figure 3In the embodiment, the first microkernel controls the data stream to be output through the video output interface, and after controlling the display of the data stream, it can unlock the data link and enter the data link monitoring operation flow. Entering the data link monitoring operation flow is used to monitor the status of the data link. Among them, if the application has been started at this time, the subsequent display of the data stream can be controlled by the application. After entering the data link monitoring operation flow, the first microkernel can determine again whether the data link has been established. Because it is in the continuous image output process of the endoscope device at this time, if the data link is in the established state, the data link will be unlocked at this time to ensure that the application can control the display of the data stream during the continuous image output process of the endoscope device. After that, the first microkernel can also determine whether the application has edited the body information of the endoscope, that is, written the body information, such as editing the image output parameters of the endoscope. If it is determined that the application has edited the body information, the first microkernel writes the edited body information of the endoscope to the memory of the endoscope according to the body information in the shared memory, that is, writes the edited body information into the memory of the endoscope. After the above operations are completed, the first microkernel can continuously control the data stream collected by the endoscope to be output through the video output interface of the endoscope device, and control the display of the data stream before the application of the endoscope device running on the application processor is started, so as to realize the continuous image output after the endoscope device powers on and outputs an image.

[0089] In Figure 3 the embodiment, when the first microkernel determines that the endoscope is connected to the control system, and it is not the first time that the control system recognizes the connection of the endoscope to the control system after power-on, nor is it when the endoscope is inserted into the control system from never being connected to the control system, it means that the data link monitoring operation flow is entered at this time. At this time, the data link is kept unlocked and the application is continuously monitored for writing body information. When the first microkernel determines that the application writes body information, it writes the edited body information into the memory of the endoscope.

[0090] In Figure 3 the embodiment, when the first microkernel determines that the CID decryption is incorrect, the type of the endoscope is read incorrectly, or the operation of circularly establishing the data link has timed out but the data link has not been established, it directly enters the data link monitoring operation flow. At this time, the data link is in the unestablished state. After entering the data link monitoring operation flow, it will try to establish the data link again. If the first microkernel determines that the CID decryption is incorrect or the type of the endoscope is read incorrectly, it will determine that the reason why the current data link cannot be established is a CID decryption error. At this time, the user can be prompted to inform the reason for the failure of the current data link, where the prompting method can be an image, text, sound, etc. The prompting method and the prompting content are not limited here.

[0091] In Figure 3In an embodiment, during the above process, the second microkernel can initialize the video output interface and monitor the status of the video output interface to ensure that the video output interface can normally output data streams.

[0092] In addition, during the above process, the application processor can run the application program of the endoscope device. If the data link is not locked after the application program starts, the application program controls the display of the data stream.

[0093] Figure 4 The figure shows a schematic diagram of the image output control process of an endoscope device according to an embodiment of the present invention.

[0094] As Figure 4 shown, after the endoscope device is powered on and started, the application processor can start the general bootloader and the application program. Among them, the general bootloader can be used to start the first microkernel and the second microkernel in the FPGA, and this process takes 5 seconds. At this time, the startup time of both the first microkernel and the second microkernel is 5 seconds, which is much less than the 30-second startup time of the application program. Therefore, the first microkernel controls the power-on image output speed of the endoscope system faster. After the first microkernel starts, it can read the body information and control the display of the data stream to complete the power-on image output of the endoscope. The time taken for the first microkernel to complete the power-on image output from the moment of power-on startup is 9 seconds. After that, the first microkernel can monitor the plugging, power-on, and data link status of the endoscope, that is, monitor the connection status between the endoscope of the endoscope device and the control system and the current data link status. The second microkernel can initialize the video output interface to output the data stream. The initialization of the video output interface is completed 10 seconds after the power-on startup. After that, the second microkernel can monitor the status of the video output interface.

[0095] The application processor controls the application program to start, which takes 30 seconds. After the application starts, parameter adjustment and human-computer interaction can be performed through the application program.

[0096] It can be understood that controlling the power-on image output of the endoscope device by the first microkernel takes less time and is faster than the application program, and through the application program for human-computer interaction, the user experience can be improved.

[0097] Figure 5 The figure shows a schematic diagram of the power-on image output control process of an endoscope device according to an embodiment of the present invention.

[0098] As Figure 5As shown, after power-on startup in step S001, the first microkernel of the FPGA will be started through step S002. In step S003, the data link can be locked by the first microkernel so that only the first microkernel can control the display of the data stream. Then, in step S004, the connection status of the endoscope can be determined by the first microkernel. When it is determined in step S005 that the endoscope is connected, step S006 can be performed, and when it is determined in step S005 that the endoscope is not connected, step S004 will be repeated to re-determine the connection status of the endoscope. In step S006, the first microkernel reads and decrypts the CID of the connected endoscope, and then in step S007, the first microkernel stores the CID data in the shared memory and controls the power-on of the isolation board according to the body information of the endoscope so that the endoscope device can normally obtain the data stream collected by the endoscope. Then, in step S008, the first microkernel reads the type of the endoscope body. When the type of the endoscope body is read, in step S009, the first microkernel can determine whether the data link has been established. When it is determined in step S009 that the data link has not been established, the data link can be established by looping through step S010, and in step S011, the first microkernel controls the display of the data stream to generate an image. When it is determined in step S009 that the data link has been established, step S010 can be skipped and directly enter step S011. Then, in step S012, the first microkernel unlocks the data link so that the application program can perform human-computer interaction and control the display of the data stream. Finally, after the application program is started in step S013, the application program controls the display of the data stream to generate an image. Among them, during the power-on image generation process of the above endoscope device, when the first microkernel determines that the CID decryption is incorrect, the type of the endoscope is read incorrectly, or the operation of looping to establish the data link has timed out but the data link has not been established, the power-on image generation process can be restarted from step S004 above.

[0099] Figure 6 The figure shows a schematic diagram of the process of generating an image by controlling the display of the data stream by the application program after the application program is started according to an embodiment of the present invention. As Figure 6 shown, step S013 may include steps S014 to S017.

[0100] As Figure 6As shown, in step S014, it is determined that the application controls the display of the data stream for image output. Then, in step S015, the application sends the mirror body information configured by the user to the first microkernel through the application processor. In step S014, the first microkernel receives the mirror body information configured by the user and performs read and write operations on the mirror body information to write the configured mirror body information into the memory of the endoscope through the shared memory. Then, in step S017, the first microkernel changes the data stream for display according to the read and written mirror body information configured by the user. Among them, during the process of the application controlling the display of the data stream for image output after the application starts, the first microkernel will monitor the connection status between the endoscope of the endoscope device and the control system, and monitor the data link between the endoscope and the control system. When it is monitored that the endoscope of the endoscope device is disconnected from the control system, the first microkernel will lock the data link and restart from Figure 5 the power-on image output process starts from step S004 as shown.

[0101] Those of ordinary skill in the art can understand the specific steps and their technical effects in the above-described image output control method of the endoscope device by reading the detailed description of the endoscope device control system 100 above. For the sake of brevity, they will not be elaborated here.

[0102] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present invention thereto. Those of ordinary skill in the art can make various changes and modifications therein without departing from the scope and spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as claimed in the appended claims.

[0103] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present invention.

[0104] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0105] In the description provided herein, numerous specific details are set forth. It will be understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

[0106] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, the methods of the present invention should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, the inventive point lies in that the corresponding technical problems can be solved with features less than all the features of a single disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate embodiment of the present invention.

[0107] Those skilled in the art will appreciate that, except where features are mutually exclusive, any combination can be employed of all the features disclosed in this specification (including the accompanying claims, abstract and drawings), as well as of all the processes or units of any method or apparatus so disclosed. Each feature disclosed in this specification (including the accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

[0108] In addition, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not others, combinations of features of different embodiments are meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

[0109] Each component embodiment of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some modules in the endoscopic device control system according to the embodiments of the present invention. The present invention can also be implemented as a device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or in any other form.

[0110] It should be noted that the above embodiments illustrate the present invention rather than limit the present invention, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In the unit claims listing several devices, several of these devices can be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words can be interpreted as names.

[0111] As described above, it is only the specific implementation manner of the present invention or the description of the specific implementation manner. The protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, and all of them should be covered by the protection scope of the present invention. The protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An endoscope device control system, characterized in that, Comprising: A field programmable gate array and an application processor, wherein the field programmable gate array includes a first microkernel; The first microkernel is configured to control the display of the data stream collected by the endoscope of the endoscope device after the control system is started; The application processor is configured to start and run the application program of the endoscope device after the control system is started. After the application program is started, the application processor takes over the first microkernel to control the display of the data stream by using the application program.

2. The control system according to claim 1, characterized in that, The first microkernel controlling the display of the data stream collected by the endoscope includes performing the following operations: Monitoring the connection status between the endoscope and the control system, and when it is determined that the endoscope is connected to the control system, reading the body information of the endoscope; Establishing and monitoring a data link between the endoscope and the control system; Based on the body information, controlling the display of the data stream by using the data link.

3. The control system according to claim 2, wherein The first microkernel is further configured to lock the data link after the first microkernel is started, and unlock the data link after the first microkernel controls the start of the display of the data stream; Wherein, when the data link is in a locked state, only the first microkernel can control the display of the data stream.

4. The control system according to claim 3, wherein, The first microkernel locking the data link after the first microkernel is started includes performing the following operations: When the first microkernel monitors that the endoscope is disconnected from the control system, locking the data link.

5. The control system according to claim 3, wherein, The first microkernel unlocking the data link after the first microkernel controls the start of the display of the data stream includes performing the following operations: After the first microkernel controls the start of the display of the data stream, when it is monitored that the endoscope is connected to the control system and the data link has been established, unlocking the data link.

6. The control system according to claim 2, wherein The body information of the endoscope includes the image output parameters of the endoscope and / or the type of the endoscope, The first microkernel is further configured to obtain the read / write permission of the body information of the endoscope and determine whether the type of the endoscope is correct before establishing the data link, Wherein, the first microkernel establishes and monitors the data link only when the read / write permission is obtained and the type of the endoscope is correct.

7. The control system according to claim 2, wherein The first microkernel monitors the connection status between the endoscope and the control system and monitors the data link, which is always executed after the first microkernel is started.

8. The control system according to claim 1, wherein The application processor is further configured to transmit the configured body information to the first microkernel in response to a user's configuration instruction after the application program is started, so that the first microkernel writes the body information into the memory of the endoscope as the body information of the endoscope.

9. The control system according to claim 1, characterized in that The application processor is further configured to prohibit the first microkernel from controlling the display of the data stream after the application program is started.

10. The control system according to any one of claims 1-9, characterized in that, The field programmable gate array further includes a second microkernel; The second microkernel is configured to initialize the video output interface of the endoscope device and monitor the status of the video output interface; The first microkernel controls the display of the data stream collected by the endoscope device of the endoscope, including performing the following operations: Controlling the data stream to be output to a display device via the video output interface and displaying an image corresponding to the data stream on the display device.

11. The control system according to claim 10, wherein The application processor is further configured to start the first microkernel and the second microkernel.

12. The control system according to any one of claims 1-9, characterized in that, The field programmable gate array and the application processor are integrated in a multi-processor system-on-chip.

13. A method for controlling image output of an endoscope device, characterized in that, The image output control method is applied to an endoscope device control system, the control system including: a field programmable gate array and an application processor, wherein the field programmable gate array includes a first microkernel; The image output control method includes: Using the first microkernel to control the display of the data stream collected by the endoscope device of the endoscope after the control system is started; Using the application processor to start and run the application program of the endoscope device after the control system is started, and after the application program is started, taking over from the first microkernel to control the display of the data stream using the application program.

Images