Business management method and related equipment

Abstract

The embodiment of the invention provides a business management method applied to an image processing device of an endoscope system and related equipment, which can save hardware resources and reduce cost under the condition of meeting different business requirements. The method provided by the embodiment of the invention comprises the following steps: receiving a video source collected by an endoscope, and accessing the video source to a single video pipeline; managing the operation state of the video pipeline according to a received service demand instruction, and executing the service demand instruction in the corresponding operation state to realize a corresponding service function; wherein the operation state comprises an image storage state, a video recording state and an idle state; the video pipeline is configured as an image storage pipeline in the image storage state, is configured as a video recording pipeline in the video recording state, and is configured as pipeline-free operation in the idle state.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of endoscope technology, and in particular, to a business management method and related equipment. Background Art

[0002] An endoscope system generally includes an endoscope and an image processing device. During the use of the endoscope system, the image processing device can record or store images of the video source collected by the endoscope according to the instructions issued by the user.

[0003] Generally, the image processing device can use the multimedia programming framework gstreamer to create a specific gstreamer pipeline to achieve specific business needs. For example, gstreamer can be used to create a video recording pipeline to achieve a video recording function; and gstreamer can be used to create a storage pipeline to achieve an image storage function.

[0004] In the process of endoscopy, it is usually necessary to switch between two different business requirements: recording video and storing images. Therefore, in the existing solution, two sets of gstreamer pipelines for implementing different functions are usually created in advance: the video recording pipeline and the image storage pipeline. In order to configure the corresponding video sources for these two pipelines respectively, see Figure 1 As shown, after the central processor in the image processing device encodes and decodes the video source collected by the endoscope, it will write the encoded and decoded video stream into different data direction registers (DDR) through the bus, and virtualize different device nodes by driving, two of which, such as DDR1 (corresponding to video source 1) and DDR2 (corresponding to video source 2), can be connected to the video recording pipeline and the image storage pipeline respectively, so that when a video recording instruction is received, the video recording pipeline can be used to record the video source 1 connected thereto, wherein, during the video recording, if an image storage instruction is received, the current frame image can also be extracted and saved from the video source 1 (that is, the image storage requirement can also be met during the video recording); but when an image storage instruction is received during the non-video recording period, it is necessary to use the image storage pipeline to extract and save the corresponding image from the video source 2 connected thereto.

[0005] However, the existing solution needs to rely on the allocation of hardware resources to form at least two parallel video sources, which has high requirements on hardware resources such as bus bandwidth and the number of registers, and is costly.

[0006] Therefore, how to save hardware resources and reduce costs while meeting different business requirements is a technical problem that needs to be solved urgently in this field. Summary of the invention

[0007] The embodiments of the present application provide a business management method and related equipment applied to an image processing device, which can save hardware resources and reduce costs while meeting different business requirements.

[0008] A first aspect of an embodiment of the present application provides a service management method, which is applied to an image processing device, wherein the image processing device is connected to an endoscope, and the method includes:

[0009] Receiving a video source acquired by the endoscope and connecting the video source to a single video pipeline;

[0010] managing the operation state of the video pipeline according to the received business demand instruction, and executing the business demand instruction in the corresponding operation state to realize the corresponding business function;

[0011] Among them, the running state includes a picture storage state, a video recording state and an idle state; the video pipeline is configured as a picture storage pipeline in the picture storage state, is configured as a video recording pipeline in the video recording state, and is configured as a pipeline-free operation in the idle state.

[0012] Optionally, the initial running state of the video pipeline is the image storage state.

[0013] Optionally, managing the operation state of the video pipeline according to the received business requirement instruction and executing the business requirement instruction in the corresponding operation state to implement the corresponding business function includes:

[0014] Determine the target operating state of the video pipeline according to the received business demand instruction; wherein, when the business demand instruction is a start recording instruction or a stop recording instruction, the target operating state of the video pipeline is a recording state, and when the business demand instruction is a save image instruction, the target operating state of the video pipeline is a save image state or a recording state;

[0015] Determining whether the video pipeline is currently in the target operating state;

[0016] If yes, directly execute the business requirement instruction to realize the corresponding business function;

[0017] If not, the video pipeline is reconfigured according to the target operating state, and the business requirement instruction is executed after the configuration is completed to realize the corresponding business function.

[0018] Optionally, reconfiguring the video pipeline according to the target operating state includes:

[0019] Setting the running state of the video pipeline to an idle state and releasing the resources currently occupied by the video pipeline;

[0020] The running state of the video pipeline is updated to the target running state, and resources are reconfigured for the video pipeline according to the target running state to recreate a video pipeline matching the target running state.

[0021] Optionally, releasing the resources currently occupied by the video pipeline includes:

[0022] The object that creates the video pipeline destroys the currently running video pipeline and releases the resources occupied by it.

[0023] Optionally, when the video pipe is currently in the recording state, if the service demand instruction is a stop recording instruction, then, after executing the stop recording instruction to stop recording, the method further includes:

[0024] The running state of the video pipeline is set to an idle state, and the resources currently occupied by the video pipeline are released.

[0025] Optionally, the method further comprises:

[0026] continuously listening for a command to start recording in the idle state;

[0027] If the video recording start instruction is not received within the preset time, a video pipeline matching the image storage state is recreated, and the running state of the video pipeline is updated to the image storage state.

[0028] Optionally, if the video pipeline receives the image storage instruction in an idle state, reconfiguring the video pipeline according to the target running state and executing the image storage instruction after completing the configuration to implement the image storage function includes:

[0029] Determining whether a start video recording instruction is received within a preset time period before receiving the image storage instruction;

[0030] If yes, after configuring the video pipeline as the video recording pipeline, executing the image storage instruction to implement the image storage function;

[0031] If not, the video pipeline is reconfigured according to the image storage state, and the image storage instruction is executed after the configuration is completed to implement the image storage function.

[0032] A second aspect of an embodiment of the present application provides a business management system, which is applied to an image processing device, wherein the image processing device is connected to an endoscope, and the system includes:

[0033] A receiving unit, used for receiving a video source acquired by the endoscope and connecting the video source to a single video pipeline;

[0034] An execution unit is used to manage the operating state of the video pipeline according to the received business demand instructions, and execute the business demand instructions in the corresponding operating state to realize the corresponding business function; wherein the operating state includes a picture storage state, a video recording state and an idle state; the video pipeline is configured as a picture storage pipeline in the picture storage state, as a video recording pipeline in the video recording state, and as a pipeline-free operation in the idle state.

[0035] The second aspect of the embodiment of the present application provides a business management method for executing the first aspect.

[0036] A third aspect of the embodiments of the present application provides an image processing device, including:

[0037] CPU, memory, input and output interfaces and power supply;

[0038] The memory is a short-term storage memory or a persistent storage memory;

[0039] The central processing unit is configured to communicate with the memory and execute instruction operations in the memory to perform the service management method described in the first aspect.

[0040] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium includes instructions, and when the instructions are executed on a computer, the computer executes the business management method described in the first aspect.

[0041] It can be seen from the above technical scheme that the embodiments of the present application have the following advantages: a business management method disclosed in the embodiments of the present application serially manages the operating status of a single video pipeline according to the received business demand instructions (specifically, in response to the business demand instructions received at different times, the video pipeline is serially switched to a pipeline that can realize the corresponding business function, for example, a video recording pipeline or a picture storage pipeline), so as to realize different business functions. Therefore, the scheme of the present application only needs to connect to a single video source to meet different business needs, and does not impose additional requirements on hardware resources such as bus bandwidth and number of registers due to the realization of different business needs, which can effectively save hardware resources and reduce the production cost of the equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0043] Figure 1A schematic diagram of an existing system framework for video source management;

[0044] Figure 2 A schematic diagram of an endoscope system disclosed in an embodiment of the present application;

[0045] Figure 3 A thread interaction diagram of an image processing device disclosed in an embodiment of the present application;

[0046] Figure 4 A schematic diagram of serial switching of a video pipeline disclosed in an embodiment of the present application;

[0047] Figure 5 A schematic diagram of the structure of a picture storage pipeline disclosed in an embodiment of the present application;

[0048] Figure 6 A schematic diagram of the structure of a video recording pipeline disclosed in an embodiment of the present application;

[0049] Figure 7 A flowchart of a business management method disclosed in an embodiment of the present application;

[0050] Figure 8 A flowchart of another business management method disclosed in an embodiment of the present application;

[0051] Fig. 9 A flowchart of another business management method disclosed in an embodiment of the present application;

[0052] Fig.10 A flowchart of another business management method disclosed in an embodiment of the present application;

[0053] Fig.11 A schematic diagram of the structure of a business management system disclosed in an embodiment of the present application;

[0054] Fig.12 A schematic diagram of the structure of an image processing device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] See also Figure 2 , Figure 2 Schematic diagram of an endoscope system disclosed in an embodiment of the present application. Figure 2It can be seen that the endoscope system 20 includes an image processing device 201, a light source 202 and an endoscope 203. The image processing device 201, the light source 202 and the endoscope 203 can be connected to each other in communication. The endoscope 203 is used to extend into the body cavity to obtain images in the body cavity; the light source 202 is used to emit illumination light to provide illumination for the imaging environment of the endoscope 203; and the image processor 201 is used to process the image collected by the endoscope 203. The light source 202 can be an independent component, or it can be integrated with the image processor 201, or it can be set in the endoscope 203, and this application does not limit this.

[0057] Based on the above description of the endoscope system 20, it can be known that the endoscope 203 is the underlying hardware for generating images, and images can only be generated when the endoscope 203 is connected. Therefore, when the endoscope 203 is not connected to the image processing device 201, the image processing device 201 does not need to respond to the recording or image storage requirements; therefore, in this embodiment, in order to avoid invalid occupation of resources, after the image processing device 201 detects that it has been successfully connected to the endoscope 203, it uses the multimedia programming framework gstreamer to create a video pipeline to prepare the operating environment for the subsequent recording or image storage functions.

[0058] In the actual operation process, for example, please refer to Figure 3 , Figure 3 A thread interaction diagram of an image processing device disclosed in an embodiment of the present application.

[0059] Depend on Figure 3 It can be seen that the application of the image processing device in the embodiment of the present application is mainly composed of two threads, one thread is a message thread, and the other thread is a working thread. Among them, the message thread is responsible for message interaction with the application of the image processing device, while the working thread is responsible for creating Gstreamer objects, maintaining status and releasing resources.

[0060] The operation state of the video pipeline created by the image processing device can be set to three states: image storage state, video recording state, and idle state. In different operation states, the video pipeline is configured to realize different functions. For example, the video pipeline is configured as an image storage pipeline in the image storage state, as a video recording pipeline in the video recording state, and as a non-pipeline operation in the idle state. Figure 5 As shown in the figure, the image storage pipeline is mainly used to implement the image storage function. Figure 6 As shown, the video recording pipeline is mainly used for video recording, and in order to simultaneously satisfy the image storage function during video recording, in the present application, the video recording pipeline can also perform the image storage function during video recording.

[0061] During the process of endoscopic examination, when the application of the image processing device receives the endoscope connection message sent by the endoscope, it will set the running state of the video pipeline to the image storage state in the message thread (that is, the initial running state of the video pipeline is the image storage state), and the corresponding working thread will open the image storage pipeline. When the video recording information is received, the message thread will set the running state to the video recording state, and the working thread will close the image storage pipeline and release the resources of the image storage pipeline, and then start to create the video recording pipeline and execute the video recording function. When a message to stop recording is received, the message thread will set the running state to the idle state, and the working thread will close the video recording pipeline and release the video recording pipeline resources. Finally, the running state will be automatically switched to the image storage state and the image storage pipeline will be opened.

[0062] The process of serially managing the video pipeline in the worker thread can be found in Figure 4 As shown. It is not difficult to understand that Figure 4 The serial switching process of the video pipeline shown is only an example, and in practical applications, it may not be limited thereto.

[0063] Specifically, since image storage is the most frequently used business requirement during the endoscopic examination process, and the image storage pipeline occupies fewer resources, in order to improve the response speed to the image storage business requirement, in this embodiment, the video pipeline is usually in the image storage state. Therefore, when the video pipeline obtains the video source, the video pipeline will be configured as an image storage pipeline. Thus, when the image storage requirement instruction is received, the instruction can be directly executed in the image storage pipeline to realize the image storage function. Subsequently, if a start recording instruction is received, the image storage state will be switched to an idle state, and the image storage pipeline will be destroyed to release resources (such as hardware resources such as memory and registers). Then, the idle state is switched to the recording state, and resources are reconfigured to form a recording pipeline, and then the recording function is executed in the recording pipeline. When the running state is the recording state, if a image storage instruction is received, the relevant image storage function can also be executed. Subsequently, if a stop recording instruction is received, the running state will be switched to an idle state, and the recording pipeline will be destroyed to release resources. Thus, pipeline-free operation is realized. Finally, if no relevant work instructions are received within the preset time, the running state is switched to the image storage state, and the image storage pipeline is reorganized to realize the functional cycle.

[0064] Further, to facilitate a detailed description of the execution logic of related business functions, please refer to Figure 5 and Figure 6 .in, Figure 5 This is a schematic diagram of the structure of a picture storage pipeline disclosed in an embodiment of the present application. Figure 6 A schematic diagram of the structure of a video recording pipeline disclosed in an embodiment of the present application.

[0065] Specifically, for Figure 5 For the image storage pipeline, when the video source is received and the image storage pipeline is turned on, the data in the video source can be converted into a format through related functions to filter the data, thereby obtaining a data queue. Finally, all the data in the data queue are cached, and the unnecessary video data is released through the data terminal. During this period, the image frames to be stored are copied from the data queue through the image storage callback function and stored in the shared space for reading by the application layer. In this way, the entire image storage function can be completed through the image storage pipeline.

[0066] for Figure 6 For the video recording pipeline, when the video source is received and the video recording pipeline is turned on, the data in the video source can be converted into a format through related functions to filter the data, and then the same data source can be copied and split. Figure 6 It can be clearly seen that the video recording pipeline has two data queues at the same time, data queue 1 is used for recording videos, and data queue 2 is used for storing images. Since the functions of the video recording pipeline are configured, they will be executed in the same form every time. Therefore, although the video recording pipeline can perform both video recording and image storage, the two functions must be performed at the same time. That is, if you need to start image storage, you must also perform video recording at the same time. Continuous video recording will occupy a lot of hardware resources and cause a waste of resources. Therefore, in actual applications, the video recording pipeline will only be created when there is a need for recording.

[0067] Specifically, in the process of data diversion, the filtered data is copied to form two data queues, wherein data queue 1 is used to execute video recording related instructions, and data queue 2 is used to execute image storage instructions.

[0068] Corresponding to data queue 2, the data in data queue 2 is cached and delivered to the data terminal, and based on Figure 5 The image frames in the video source are obtained in a similar manner to the image storage pipeline shown. Here, the entire image storage function can be completed through the video recording pipeline.

[0069] Corresponding to data queue 1, after completing the data cache, the cached data is video encoded, and then the video source is parsed through related functions to split and output the video encoding data. Then, the video encoding data is parsed to generate a video stream in a fixed format. Finally, the MP4 rectification of the entire video source is completed, and the rectified data is saved to the data terminal, so as to obtain a video file based on the image frame. Here, the entire recording function can be completed through this recording pipeline.

[0070] As can be seen from the above, in this application, by setting a parent class pointer, the generation and release of two subclass objects (such as the image storage pipeline and the video recording pipeline) can be uniformly managed. Among them, the parent class (or base class) can be understood as being used to control the opening or closing of the subclass (or inherited class), while the subclass (or inherited class) controls the opening and closing of the image storage or video recording pipeline. In this way, a clear code structure can be constructed for state maintenance, which is very convenient for the management of various business functions.

[0071] It is not difficult to understand that the above is only a brief description of the message thread and work thread, pipeline type, and pipeline switching method of the image processing device in the embodiment of the present application. Figures 7 to 9 The execution logic of the image processing device is described in detail.

[0072] See also Figure 7 , Figure 7 This is a flow chart of a business management method disclosed in an embodiment of the present application, including but not limited to steps 701 to 702.

[0073] 701. Receive a video source collected by an endoscope, and connect the video source to a single video pipeline.

[0074] When the endoscope system is started, and the application of the image processing device is opened normally, and the endoscope has begun to collect the corresponding video source, the image processing device can receive the video source collected by the endoscope and then connect the video source to the video pipeline.

[0075] In one specific embodiment, the user operates on the operation interface of the endoscope system so that the endoscope works to collect the video source. When the endoscope is in communication with the image processing device, the endoscope can collect the corresponding video source to generate video information or image information. Among them, the video information or image information mainly describes the size or format of the collected image, and the content carried by the video information or image information is not specifically limited here. For the convenience of subsequent understanding and description, the description of the video information or image information is described in detail with the video source.

[0076] 702. Manage the operation state of the video pipeline according to the received business demand instruction, and execute the business demand instruction in the corresponding operation state to realize the corresponding business function.

[0077] The message thread of the image processing device continuously monitors the messages sent by the application. When the corresponding business demand instruction is obtained, the running state of the video pipeline can be managed according to the received business demand instruction, and the business demand instruction can be executed in the corresponding running state, thereby realizing the business function corresponding to the business demand instruction. Among them, the running state includes the image storage state, the recording state and the idle state; the video pipeline is configured as an image storage pipeline in the image storage state, as a recording pipeline in the recording state, and as a pipeline-free operation in the idle state. The business functions include the image storage function and the recording function.

[0078] In one specific embodiment, the message thread in the image processing device updates the running state of the video pipeline according to the business demand instruction, and accordingly, the working thread configures the video pipeline according to the running state, and then executes the business demand instruction on the video source in the configured video pipeline to realize the corresponding business function. It is not difficult to understand that the recording function is to record the image frames in the video source as video. The image storage function is to store the image frames in the video source.

[0079] A business management method disclosed in this embodiment first receives the video source collected by the endoscope, and connects the video source to a single video pipeline; then manages the running state of the video pipeline according to the received business demand instruction, and executes the business demand instruction in the corresponding running state to realize the corresponding business function; wherein the running state includes the image storage state, the recording state and the idle state; the video pipeline is configured as the image storage pipeline in the image storage state, the recording pipeline in the recording state, and the pipeline-free operation in the idle state. When the image processing device receives any business demand instruction for the video source collected by the endoscope, it can switch the running state of the video pipeline to the running state corresponding to the business demand instruction, so as to execute the business demand instruction in the running state to realize the corresponding business function. It can be seen that the embodiment of the present application can realize different business functions by serially managing the running state of a single video pipeline according to the received business demand instruction. Therefore, the present application solution only needs to connect a single video source to meet different business needs, and does not put forward additional requirements for hardware resources such as bus bandwidth and number of registers due to the realization of different business needs, which can effectively save hardware resources and reduce the production cost of the equipment.

[0080] To facilitate the above Figure 7 For a detailed description of the business management methods described in Figure 8 , Figure 8 This is a flowchart of another service management method disclosed in an embodiment of the present application, including steps 801 to 804.

[0081] 801. Receive a video source collected by an endoscope, and connect the video source to a single video pipeline.

[0082] In this embodiment, step 801 is the same as the aforementioned Figure 7 The step 701 is similar to that in step 701, and will not be described in detail here. However, it should be noted that based on Figure 4 From the described video pipeline, it can be seen that the video pipeline in different operating states is switched serially.

[0083] 802. Determine the target operating state of the video pipeline according to the received business demand instruction, and determine whether the video pipeline is currently in the target operating state. If the video pipeline is currently in the target operating state, execute step 803; if the video pipeline is not currently in the target operating state, execute step 804.

[0084] When the image processing device receives the business demand instruction, it can determine the target operating state of the current video pipeline according to the business demand instruction, and by comparing it with the current operating state of the video pipeline, it can determine whether the video pipeline is currently in the target operating state. Therefore, step 803 or step 804 is selected for execution according to the judgment result. It is not difficult to understand that the business demand instruction includes a start recording instruction, a stop recording instruction, or a save image instruction, etc., and the content contained in the business demand instruction is not specifically limited here.

[0085] In one specific embodiment, in combination Figure 3 and Figure 4 It can be seen that the video recording pipeline can perform the image storage function and the video recording function, and the image storage pipeline can only perform the image storage function. Therefore, when the business demand instruction is a start recording instruction or a stop recording instruction, the target operating state of the video pipeline is the recording state; when the business demand instruction is a image storage instruction, the target operating state of the video pipeline is the image storage state or the video recording state. Therefore, when a business demand instruction is received, it can be determined whether the operating state of the video pipeline is the target operating state according to the business demand instruction.

[0086] Based on the above embodiment, for example, the business demand instruction is a start recording instruction. At this time, the running state of the video pipeline is the recording state, then it can be determined that the video pipeline is currently in the target running state, and step 803 is executed. If the running state of the video pipeline is the image storage state at this time, it can be determined that the video pipeline is not currently in the target running state, and step 804 is executed. It is not difficult to understand that the above is only one of the possible implementation methods, and no specific limitation is made here.

[0087] 803. Directly execute the business requirement instructions to realize the corresponding business functions.

[0088] When the running state of the video pipeline is the target running state, the business requirement instructions can be directly executed to realize the corresponding business functions.

[0089] In one specific embodiment, if the video pipeline is currently in a storage state (indicating that the video pipeline is currently configured as a storage pipeline), and the business demand instruction is a storage instruction, the storage instruction is executed in the storage pipeline to save the current frame image.

[0090] In another specific embodiment, if the video pipeline is currently in a recording state (indicating that the video pipeline is currently configured as a recording pipeline), and the business demand instruction is a start recording instruction or a stop recording instruction, then the start recording instruction is executed in the recording pipeline to start recording, or the stop recording instruction is executed in the recording pipeline to stop recording, specifically, the image processing device starts recording image frames in the video source or stops recording image frames. If the business demand instruction is a save image instruction at this time, then the save image instruction is executed in the recording pipeline to save the current frame image.

[0091] 804. Reconfigure the video pipeline according to the target operating state, and execute the business requirement instructions after completing the configuration to realize the corresponding business function.

[0092] In this embodiment, if the video pipeline is not currently in the target operating state, it means that it is currently unable to execute the business demand instructions. At this time, the video pipeline needs to be switched to a pipeline that can execute the business demand instructions. Therefore, reconfiguring the video pipeline according to the target operating state mainly includes two switching processes: destroying the current operating pipeline (releasing the resources occupied by the current operating pipeline) and configuring a new pipeline. After the configuration is completed, the corresponding business functions can be implemented according to the business demand instructions. For example, in one of the specific embodiments, the video pipeline can be switched from a storage pipeline to a video recording pipeline according to the start recording instruction, and after completing the configuration of the video recording pipeline, the start recording instruction is executed to implement the start recording function.

[0093] Specifically, in one embodiment, reconfiguring the video pipeline according to the target operating state may include but is not limited to the following steps:

[0094] 8041. Set the running state of the video pipeline to an idle state, and release the resources currently occupied by the video pipeline.

[0095] Based on step 802, when the running state of the video pipeline is not the target running state, the running state of the video pipeline can be first set to an idle state, and then the resources currently occupied by the video pipeline are released.

[0096] Specifically, the object that creates the video pipeline may destroy the currently running video pipeline and release the resources occupied by the video pipeline.

[0097] For example, in a specific embodiment, when it is necessary to destroy the currently running video pipeline, the application layer (management layer) sends a message to the video pipeline; after the video pipeline receives the message and further processes it, it sends a signal to the object that created the video pipeline to instruct it to destroy the current video pipeline; after the object that created the video pipeline listens to the signal, it destroys the video pipeline and releases resources. It is not difficult to understand that in this embodiment, the object that created the video pipeline can perform the pipeline destruction and resource release operations. Compared with the way that the video pipeline releases resources by itself after receiving the message, the object can uniformly manage the creation and destruction of the video pipeline to ensure that there will be no memory problems.

[0098] Specifically, each time a pipeline is destroyed, the pipeline operation and memory destruction are not stopped immediately. Instead, the gstreamer function gst_event_new_eos() is called to send an Ethernet EOS message to the sinkPad pad of the queue element (after calling this function, the g_object_unref function must be called manually to release the EOS message). Then, a callback function is used to listen to the message on the gstreamer bus, and then the mechanism of the relevant application development framework is used to send an end Eos signal, thereby destroying the entire pipeline and releasing the memory.

[0099] 8042. Update the running state of the video pipeline to the target running state, and reconfigure resources for the video pipeline according to the target running state to recreate the video pipeline that matches the target running state.

[0100] After completing the release of resources, the image processing device may update the running state of the video pipeline to the target running state, and reconfigure resources for the video pipeline according to the target running state to recreate the video pipeline matching the target running state.

[0101] In one specific embodiment, when receiving a start recording instruction, the application can first set the recording flag to true (indicating that the recording function needs to be executed currently), and then update the video pipeline state to an idle state, so that the video pipeline will release resources (i.e., execute step 8041); after the resources are released, the application can decide to switch the target running state to the recording state based on the recording flag, and configure corresponding resources for the pipeline based on the state to recreate a recording pipeline that matches the recording state (i.e., execute step 8042).

[0102] Based on the above scheme, it can be seen that in the embodiment of the present application, the object that creates the video pipeline can perform pipeline destruction and resource release operations. Compared with the way that the video pipeline releases resources by itself after receiving a message, the object can uniformly manage the creation and destruction of the video pipeline to ensure that there will be no memory problems. At the same time, by updating the running state first and then configuring the pipeline (for example, first set it to idle state and then release resources; first update to the target running state and then reconfigure resources), the current running state of the pipeline can be clarified, avoiding interference from erroneous instructions during pipeline configuration, and improving the stability of pipeline operation.

[0103] Furthermore, in this embodiment, the process of destroying the pipeline and releasing resources and the process of running without the pipeline are both in an idle state. In order to avoid the image storage pipeline being rebuilt due to receiving the image storage instruction during the period of destroying the image storage pipeline to rebuild the video recording pipeline, thereby interfering with the execution of the video recording function, in another specific embodiment, if the video pipeline receives the image storage instruction in the idle state, there is another achievable embodiment. For details, please refer to Fig. 9 , Fig. 9 This is a flow chart of another business management method disclosed in the embodiment of the present application. It includes steps 901 to 903. It is not difficult to understand that: Fig. 9 Steps 901 to 903 in the embodiment shown in the figure can be understood as Figure 8 After step 802, it is executed when the judgment result is no.

[0104] 901. Determine whether a start video recording instruction is received within a preset time period before receiving a save image instruction. If a start video recording instruction is received, execute step 902; if no start video recording instruction is received, execute step 903.

[0105] Specifically, within a preset time period before the image processing device determines that a storage instruction has been received, the image processing device can determine whether a start recording instruction has been received within the preset time period. If a start recording instruction has been received, it means that a recording function needs to be executed later, and the image storage pipeline is currently switching to a recording pipeline; if a start recording instruction has not been received, it means that there is no function that needs to be executed, and the video pipeline is currently in a non-pipeline operation state (that is, resources are not occupied).

[0106] Based on the above embodiments, it is not difficult to understand that the specific length of the preset time period is not limited in this embodiment, and can be set according to actual usage requirements, which will not be elaborated here.

[0107] 902. After configuring the video pipeline as a video recording pipeline, execute a save image instruction to implement a save image function.

[0108] If a start recording instruction is received within the preset time period (the recording flag is set to true), it means that the video pipeline is currently in the process of switching from the image storage pipeline to the recording pipeline, and the recording pipeline can also be used to execute the image storage instruction. Therefore, in order to avoid affecting the execution of the start recording instruction, the image storage flag can be set to true first. After the configuration of the recording pipeline is completed, the image storage callback function is called to perform the image storage operation; and the recording callback function is called to execute the start recording operation.

[0109] 903. Reconfigure the video pipeline according to the image storage state, and execute the image storage instruction after completing the configuration to implement the image storage function.

[0110] If the start recording command is not received within the preset time period, it means that the recording function is not needed at present, and the image storage pipeline occupies less resources. Therefore, the video pipeline can be configured as an image storage pipeline according to the image storage command. For specific configuration methods, please refer to Figure 5 Then, the image storage operation can be performed in the image storage pipeline according to the image storage instruction.

[0111] Based on the above scheme, it can be seen that when the embodiment of the present application receives a save image instruction in an idle state, it determines whether there is a current recording demand by judging whether a start recording instruction has been received within a preset time period before the save image instruction is received, and then determines whether the video pipeline is configured as a save image pipeline or a recording pipeline. This can avoid interference with the execution of the recording function due to the receipt of the save image instruction during the destruction of the save image pipeline to rebuild the recording pipeline.

[0112] Further, based on Figure 8 In the embodiment described, after step 803, if the business demand instruction is an instruction to stop recording, then after the recording ends, in another specific embodiment, there is another achievable embodiment.

[0113] For details, please refer to Fig.10 , Fig.10 This is a flowchart of another business management method disclosed in an embodiment of the present application, which may specifically include but is not limited to steps 1001 to 1006.

[0114] 1001. Receive a video source acquired by an endoscope, and connect the video source to a single video pipeline; the initial operation state of the video pipeline is a storage state.

[0115] 1002. According to the received start recording instruction, reconfigure the video pipeline as a recording pipeline, and start recording after completing the configuration.

[0116] 1003. When a stop recording instruction is received, the stop recording instruction is executed in the recording pipeline to stop recording.

[0117] It can be understood that the specific implementation process of the above steps 1001 to 1003 can refer to the description in the above embodiments, and will not be described in detail here.

[0118] 1004. Set the running state of the video pipeline to an idle state, and release the resources currently occupied by the video pipeline.

[0119] After the above-mentioned business functions are executed, the operation state of the video pipeline can be set to an idle state, and the resources currently occupied by the video pipeline can be released.

[0120] In one of the specific embodiments, an example is given below for ease of understanding. When the video pipeline is currently in the recording state, if the business demand instruction is a stop recording instruction, then the stop recording instruction is executed, and after the recording is stopped, the operation state of the video pipeline is set to the idle state, and the resources currently occupied by the video pipeline are released. It is not difficult to understand that the reason why the video pipeline must be destroyed after the recording is stopped is that the recording pipeline needs to store video frames all the time, which will occupy a large amount of resources. If it is not destroyed, it will cause memory resource leakage.

[0121] 1005. Continuously monitor the start video recording command in the idle state.

[0122] Taking into account that during the endoscopic examination, the user may continuously trigger the recording command (that is, after triggering the stop recording command, continue to trigger the start recording command), in this embodiment, the image processing device may continue to monitor whether there is a start recording command within a preset time after the self-recording state is switched to the idle state. If the start recording command is not heard within the preset time, it means that the user currently has no need to continue recording; otherwise, it means that the user currently has a need to continue recording, and the present application maintains an idle state within the predetermined time, which can facilitate a quick response when the start recording command is received again (that is, the process of destroying the image storage pipeline is omitted).

[0123] 1006. If the video recording start instruction is not received within the preset time, a video pipeline matching the image storage state is re-created, and the operation state of the video pipeline is updated to the image storage state.

[0124] When the image processing device does not receive the start video recording instruction within the preset time, a video pipeline matching the image storage state is recreated, and the operation state of the video pipeline is updated to the image storage state.

[0125] In one specific embodiment, if the image processing device does not receive a start recording instruction within a preset time, it can be considered that there is no need for continuous recording. At this time, since the image storage pipeline occupies less resources and the image storage function is a commonly used function during endoscopic examination, re-creating the image storage pipeline can quickly respond to the image storage demand.

[0126] In summary, it can be seen that the embodiment of the present application switches the running state of the video pipeline to the idle state after stopping recording and releases the resources occupied by the recording pipeline, which can save hardware resources and avoid memory leakage; maintaining the idle state within a predetermined time can facilitate a quick response when the start recording instruction is received again; if the start recording instruction is not received within the preset time, it can be considered that there is no need for continuous recording at present. At this time, since the image storage pipeline occupies fewer resources and image storage is a commonly used function in the endoscopic examination process, re-creating the image storage pipeline can quickly respond to the image storage demand.

[0127] Of course, it is understandable that in other embodiments, it is also possible to directly switch the running state of the video pipeline to the image storage state after step 1004, and recreate the image storage pipeline; or, keep the idle state until receiving the next business demand instruction, and then reconfigure the video pipeline.

[0128] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0129] See also Fig.11 , Fig.11 This is a schematic diagram of the structure of a business management system disclosed in an embodiment of the present application. It should be noted that the business management system disclosed in this embodiment is applied to an image processing device, the image processing device is connected to an endoscope, and the system includes:

[0130] A receiving unit 120, for receiving a video source collected by an endoscope and connecting the video source to a single video pipeline;

[0131] The execution unit 130 is used to manage the operation state of the video pipeline according to the received business demand instructions, and execute the business demand instructions in the corresponding operation state to realize the corresponding business function; wherein the operation state includes the image storage state, the recording state and the idle state; the video pipeline is configured as an image storage pipeline in the image storage state, is configured as a recording pipeline in the recording state, and is configured as a pipeline-free operation in the idle state.

[0132] Exemplarily, the initial running state of the video pipeline is a picture storage state.

[0133] Exemplarily, the execution unit 130 includes:

[0134] The first determination module 1301 is used to determine the target operating state of the video pipeline according to the received business demand instruction; wherein, when the business demand instruction is a start recording instruction or a stop recording instruction, the target operating state of the video pipeline is the recording state; when the business demand instruction is a save image instruction, the target operating state of the video pipeline is the save image state or the recording state;

[0135] A second determination module 1302 is used to determine whether the video pipeline is currently in a target running state;

[0136] The execution module 1303 is used to directly execute the business requirement instructions to realize the corresponding business functions when the video pipeline is currently in the target operating state; when the video pipeline is not currently in the target operating state, reconfigure the video pipeline according to the target operating state, and execute the business requirement instructions after completing the configuration to realize the corresponding business functions.

[0137] Exemplarily, the execution module 1303 includes:

[0138] The resource release submodule 13031 is used to set the running state of the video pipeline to an idle state and release the resources currently occupied by the video pipeline when the video pipeline is not currently in the target running state;

[0139] The execution submodule 13032 is used to update the running state of the video pipeline to the target running state after the resource release submodule 1231 completes the release of resources, and reconfigure resources for the video pipeline according to the target running state to recreate the video pipeline matching the target running state.

[0140] Exemplarily, the resource release submodule 13031 is specifically configured to cause the object that creates the video pipeline to destroy the currently running video pipeline and release the resources occupied by the video pipeline.

[0141] Exemplarily, the resource release submodule 13031 is also used to set the running state of the video pipeline to the idle state and release the resources currently occupied by the video pipeline if the business demand instruction is a stop recording instruction when the video pipeline is currently in the recording state.

[0142] Exemplarily, the execution submodule 13032 is also used for:

[0143] Continuously listen for the start recording command in idle state;

[0144] When the video recording start instruction is not received within the preset time, a video pipeline matching the image storage state is re-created, and the running state of the video pipeline is updated to the image storage state.

[0145] Exemplarily, the execution module 1303 further includes:

[0146] The judging submodule 13033 is used to determine whether a start video recording instruction is received within a preset time period before receiving the save image instruction when receiving the save image instruction in the idle state;

[0147] The execution submodule 13032 is also used for:

[0148] When the judgment result of the judgment submodule 13033 is yes, after configuring the video pipeline as a video recording pipeline, executing the image storage instruction to implement the image storage function;

[0149] When the judgment result of the judgment submodule 13033 is no, the video pipeline is reconfigured according to the image storage state, and the image storage instruction is executed after the configuration is completed to implement the image storage function.

[0150] A business management system disclosed in this embodiment can realize different business functions by serially managing the operating status of a single video pipeline according to the received business demand instructions (specifically, in response to the business demand instructions received at different times, the video pipeline is serially switched to a pipeline that can realize the corresponding business function, such as a video recording pipeline or a picture storage pipeline). Therefore, the present application solution only needs to connect to a single video source to meet different business needs, and does not impose additional requirements on hardware resources such as bus bandwidth and number of registers due to the realization of different business needs. It can effectively save hardware resources and reduce the production cost of the equipment.

[0151] See below Fig.12 , a schematic diagram of the structure of an image processing device disclosed in an embodiment of the present application includes:

[0152] CPU 1201, memory 1205, input / output interface 1204, wired or wireless network interface 1203 and power supply 1202;

[0153] The memory 1205 is a temporary storage memory or a permanent storage memory;

[0154] The central processor 1201 is configured to communicate with the memory 1205 and execute instructions in the memory 1205 to perform the service management method in the aforementioned embodiment.

[0155] An embodiment of the present application also provides a computer-readable storage medium, which includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected through a line, and the at least one processor is used to run a computer program or instruction to execute the business management method in the aforementioned embodiment.

[0156] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and beneficial effects of the above-described systems, devices and other related equipment can refer to the description of the corresponding parts in the aforementioned method embodiments and will not be repeated here.

[0157] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0158] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0159] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0160] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A business management method, characterized in that: An image processing device applied to an endoscope system, the image processing device being connected to an endoscope, the method comprising: Receiving a video source acquired by the endoscope and connecting the video source to a single video pipeline; managing the operation state of the video pipeline according to the received business demand instruction, and executing the business demand instruction in the corresponding operation state to realize the corresponding business function; Among them, the running state includes a picture storage state, a video recording state and an idle state; the video pipeline is configured as a picture storage pipeline in the picture storage state, is configured as a video recording pipeline in the video recording state, and is configured as a pipeline-free operation in the idle state.

2. The business management method according to claim 1, characterized in that: The initial operation state of the video pipeline is the image storage state.

3. The business management method according to claim 1 or 2, characterized in that: The managing the operation state of the video pipeline according to the received business demand instruction and executing the business demand instruction in the corresponding operation state to realize the corresponding business function includes: Determine the target operating state of the video pipeline according to the received business demand instruction; wherein, when the business demand instruction is a start recording instruction or a stop recording instruction, the target operating state of the video pipeline is a recording state, and when the business demand instruction is a save image instruction, the target operating state of the video pipeline is a save image state or a recording state; Determining whether the video pipeline is currently in the target operating state; If yes, directly execute the business requirement instruction to realize the corresponding business function; If not, the video pipeline is reconfigured according to the target operating state, and the business requirement instruction is executed after the configuration is completed to realize the corresponding business function.

4. The business management method according to claim 3, characterized in that: The reconfiguring the video pipeline according to the target operating state includes: Setting the running state of the video pipeline to an idle state and releasing the resources currently occupied by the video pipeline; The running state of the video pipeline is updated to the target running state, and resources are reconfigured for the video pipeline according to the target running state to recreate a video pipeline matching the target running state.

5. The business management method according to claim 4, characterized in that: The releasing the resources currently occupied by the video pipeline includes: The object that creates the video pipeline destroys the currently running video pipeline and releases the resources occupied by it.

6. The business management method according to claim 3, characterized in that: In the case that the video pipe is currently in the recording state, if the service demand instruction is a stop recording instruction, then, after executing the stop recording instruction to stop recording, the method further includes: Set the running state of the video pipeline to an idle state and release the resources currently occupied by the video pipeline 。 7. The business management method according to claim 6, characterized in that: The method further comprises: continuously listening for a command to start recording in the idle state; If the video recording start instruction is not received within the preset time, a video pipeline matching the image storage state is recreated, and the running state of the video pipeline is updated to the image storage state.

8. The business management method according to claim 3, characterized in that: If the video pipeline receives the image storage instruction in an idle state, then, reconfiguring the video pipeline according to the target running state and executing the image storage instruction after completing the configuration to implement the image storage function includes: Determining whether a start video recording instruction is received within a preset time period before receiving the image storage instruction; If yes, after configuring the video pipeline as the video recording pipeline, executing the image storage instruction to implement the image storage function; If not, the video pipeline is reconfigured according to the image storage state, and the image storage instruction is executed after the configuration is completed to implement the image storage function.

9. A business management system, characterized in that: Applied to an image processing device, the image processing device is connected to an endoscope, and the system comprises: A receiving unit, used for receiving a video source acquired by the endoscope and connecting the video source to a single video pipeline; An execution unit is used to manage the operating state of the video pipeline according to the received business demand instructions, and execute the business demand instructions in the corresponding operating state to realize the corresponding business function; wherein the operating state includes a picture storage state, a video recording state and an idle state; the video pipeline is configured as a picture storage pipeline in the picture storage state, as a video recording pipeline in the video recording state, and as a pipeline-free operation in the idle state.

10. An image processing device, characterized in that: include: CPU, memory, input and output interfaces and power supply; The memory is a short-term storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instruction operations in the memory to perform the service management method described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes instructions, and when the instructions are executed on a computer, the computer executes the service management method according to any one of claims 1 to 8.

Images