Microfluidic-based multiphase flow image processing method, device, equipment and medium

Abstract

This application relates to a method, apparatus, device, medium, and computer program product for multiphase flow image processing based on microfluidics. The method includes: displaying a multiphase flow image processing interface; displaying parameter values ​​of image processing parameters input via parameter input controls on the multiphase flow image processing interface; cyclically acquiring multiphase flow images captured by a high-speed camera of a microfluidic chip exhibiting multiphase flow; recording multiple phase-spaced bubbles generated by the multiphase flow in the microfluidic chip within the multiphase flow images; performing image processing on each acquired multiphase flow image according to the parameter values ​​of the image processing parameters to obtain information on multiple bubbles identified from each multiphase flow image; and, when the multiphase flow in the microfluidic chip is determined to be in a stable state based on the changing state of the bubbles at a preset inlet region of the microfluidic chip in the processed multiphase flow image, starting to store the information on the multiple bubbles identified from the multiphase flow images. This method can improve data acquisition efficiency.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a microfluidic-based multiphase flow image processing method, apparatus, device, medium, and computer program product. Background Technology

[0002] Microfluidics is a technology that uses microchannels to process or manipulate tiny fluids, integrating basic operational units such as sample preparation, reaction, separation, and detection in biological, chemical, and medical analysis processes onto a micrometer-scale chip (microfluidic chip) to complete the analysis. When conducting analysis and research using microfluidic chips, a high-speed camera is typically used to capture video of the microfluidic chip to record the flow of multiphase flow within it. Frame-by-frame images of the multiphase flow are then obtained from the captured video, and image processing software such as Photoshop and ImageJ is used to process each frame of the multiphase flow image, thereby obtaining information such as the flow characteristics of each phase fluid in the multiphase flow.

[0003] However, when using image processing software such as Photoshop and ImageJ to process captured multiphase flow images, the images need to be transferred to the image processing software one by one. Moreover, the image processing relies on manual operation, resulting in low image processing efficiency and low data acquisition efficiency in multiphase flow. Summary of the Invention

[0004] Therefore, it is necessary to provide a microfluidic-based multiphase flow image processing method, apparatus, device, medium, and computer program product that can improve data acquisition efficiency in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a multiphase flow image processing method based on microfluidics. The method includes:

[0006] The multiphase flow image processing interface is displayed; the multiphase flow image processing interface displays parameter input controls corresponding to the image processing parameters;

[0007] The multiphase flow image processing interface displays the parameter values ​​of the image processing parameters input through the parameter input control;

[0008] The system continuously acquires multiphase flow images of a microfluidic chip containing multiphase flow from a high-speed camera; the multiphase flow images record multiple phase-spaced bubbles generated by the multiphase flow in the microfluidic chip.

[0009] Based on the parameter values ​​of the image processing parameters, image processing is performed on each of the acquired multiphase flow images to obtain information on the plurality of bubbles identified from each of the multiphase flow images;

[0010] When the multiphase flow in the microfluidic chip is determined to be in a stable state based on the change state of the bubbles at the preset inlet region of the microfluidic chip in the processed multiphase flow image, the information of the multiple bubbles identified from the multiphase flow image is stored.

[0011] Secondly, this application also provides a microfluidic-based multiphase flow image processing apparatus. The apparatus includes:

[0012] The display module is used to display a multiphase flow image processing interface; the multiphase flow image processing interface displays parameter input controls corresponding to image processing parameters; the multiphase flow image processing interface displays the parameter values ​​of the image processing parameters input through the parameter input controls;

[0013] The image acquisition module is used to cyclically acquire multiphase flow images of a microfluidic chip with multiphase flow captured by a high-speed camera; the multiphase flow images record multiple phase-spaced bubbles generated by the multiphase flow in the microfluidic chip;

[0014] An image processing module is used to perform image processing based on each of the acquired multiphase flow images according to the parameter values ​​of the image processing parameters, to obtain information on the plurality of bubbles identified from each of the multiphase flow images;

[0015] The storage module is used to determine that the multiphase flow in the microfluidic chip is in a stable state based on the change state of the bubbles at the preset inlet region of the microfluidic chip in the processed multiphase flow image, and then start storing the information of the multiple bubbles identified from the multiphase flow image.

[0016] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0017] The multiphase flow image processing interface is displayed; the multiphase flow image processing interface displays parameter input controls corresponding to the image processing parameters;

[0018] The multiphase flow image processing interface displays the parameter values ​​of the image processing parameters input through the parameter input control;

[0019] The system continuously acquires multiphase flow images of a microfluidic chip containing multiphase flow from a high-speed camera; the multiphase flow images record multiple phase-spaced bubbles generated by the multiphase flow in the microfluidic chip.

[0020] Based on the parameter values ​​of the image processing parameters, image processing is performed on each of the acquired multiphase flow images to obtain information on the plurality of bubbles identified from each of the multiphase flow images;

[0021] When the multiphase flow in the microfluidic chip is determined to be in a stable state based on the change state of the bubbles at the preset inlet region of the microfluidic chip in the processed multiphase flow image, the information of the multiple bubbles identified from the multiphase flow image is stored.

[0022] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0023] The multiphase flow image processing interface is displayed; the multiphase flow image processing interface displays parameter input controls corresponding to the image processing parameters;

[0024] The multiphase flow image processing interface displays the parameter values ​​of the image processing parameters input through the parameter input control;

[0025] The system continuously acquires multiphase flow images of a microfluidic chip containing multiphase flow from a high-speed camera; the multiphase flow images record multiple phase-spaced bubbles generated by the multiphase flow in the microfluidic chip.

[0026] Based on the parameter values ​​of the image processing parameters, image processing is performed on each of the acquired multiphase flow images to obtain information on the plurality of bubbles identified from each of the multiphase flow images;

[0027] When the multiphase flow in the microfluidic chip is determined to be in a stable state based on the change state of the bubbles at the preset inlet region of the microfluidic chip in the processed multiphase flow image, the information of the multiple bubbles identified from the multiphase flow image is stored.

[0028] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0029] The multiphase flow image processing interface is displayed; the multiphase flow image processing interface displays parameter input controls corresponding to the image processing parameters;

[0030] The multiphase flow image processing interface displays the parameter values ​​of the image processing parameters input through the parameter input control;

[0031] The system continuously acquires multiphase flow images of a microfluidic chip containing multiphase flow from a high-speed camera; the multiphase flow images record multiple phase-spaced bubbles generated by the multiphase flow in the microfluidic chip.

[0032] Based on the parameter values ​​of the image processing parameters, image processing is performed on each of the acquired multiphase flow images to obtain information on the plurality of bubbles identified from each of the multiphase flow images;

[0033] When the multiphase flow in the microfluidic chip is determined to be in a stable state based on the change state of the bubbles at the preset inlet region of the microfluidic chip in the processed multiphase flow image, the information of the multiple bubbles identified from the multiphase flow image is stored.

[0034] The aforementioned microfluidic-based multiphase flow image processing method, apparatus, device, medium, and computer program product allows for convenient operation by inputting image processing parameters through a multiphase flow image processing interface. Furthermore, after cyclically acquiring multiphase flow images captured by a high-speed camera on a microfluidic chip containing multiphase flow, image processing is performed on each acquired multiphase flow image according to the input image processing parameters. This enables batch processing of multiphase flow images from the high-speed camera, resulting in high image processing efficiency and rapid acquisition of information on multiple bubbles within the multiphase flow images. Moreover, the microfluidic chip has a preset inlet region. When the multiphase flow in the microfluidic chip is determined to be in a stable state based on the changing state of the bubbles at the preset inlet region in the processed multiphase flow image, information on multiple bubbles identified from the multiphase flow image is stored, reducing the storage of invalid data and enabling rapid and efficient acquisition of information on multiple bubbles in the cyclically transmitted multiphase flow images from the high-speed camera, thus improving data acquisition efficiency. Attached Figure Description

[0035] Figure 1 This is an application environment diagram of a microfluidic-based multiphase flow image processing method in one embodiment.

[0036] Figure 2 This is a flowchart illustrating a microfluidic-based multiphase flow image processing method in one embodiment.

[0037] Figure 3 This is a flowchart illustrating the steps of performing image processing based on the parameter values ​​of image processing parameters in one embodiment to obtain information about multiple bubbles identified from each multiphase flow image.

[0038] Figure 4 This is a schematic diagram of a multiphase flow image processing interface in one embodiment;

[0039] Figure 5 This is a multiphase flow image of each stage in a multiphase flow image processing flow in one embodiment;

[0040] Figure 6 This is a structural block diagram of a microfluidic-based multiphase flow image processing device in one embodiment;

[0041] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0043] The multiphase flow image processing method based on microfluidics provided in this application can be applied to, for example... Figure 1 In the application environment shown, the high-speed camera 102 can communicate with the computer device 104. The computer device 104 can display a multiphase flow image processing interface, and display the parameter input controls corresponding to the image processing parameters and the parameter values ​​of the input image processing parameters in the multiphase flow image processing interface. The computer device 104 can cyclically acquire multiphase flow images captured by the high-speed camera 102 on the microfluidic chip with multiphase flow, and perform image processing based on the parameter values ​​of the image processing parameters for each acquired multiphase flow image to obtain information on multiple bubbles identified from each multiphase flow image. When the multiphase flow in the microfluidic chip is determined to be in a stable state based on the change state of the bubbles at the preset inlet area of ​​the microfluidic chip in the processed multiphase flow image, the information on the multiple bubbles identified from the multiphase flow image is stored. The high-speed camera 102 is a digital industrial camera, which has higher image stability, higher transmission capacity, and higher anti-interference capability compared to ordinary cameras. The computer device 104 can be a computer, laptop, smartphone, or tablet computer, or a server with an external display unit and input device. Communication between the high-speed camera 102 and the computer device 104 can be wired, such as via a USB data cable, or wireless, such as via WiFi or Bluetooth.

[0044] In one embodiment, such as Figure 2 As shown, a multiphase flow image processing method based on microfluidics is provided. This embodiment applies this method to... Figure 1 Taking computer device 104 as an example, the method includes the following steps:

[0045] Step 202: Display the multiphase flow image processing interface; the multiphase flow image processing interface displays parameter input controls corresponding to the image processing parameters.

[0046] The interface is the area used for human-computer interaction. The multiphase flow image processing interface is used to process multiphase flow images. Image processing parameters are the parameters used when processing multiphase flow images. Controls are the operable elements in the interface. Parameter input controls are used to input the parameter values ​​for image processing. Parameter input controls can be text input controls, slider controls, or drop-down selection controls, etc.

[0047] In one embodiment, the computer device can respond to a triggering operation on the identification information of the multiphase flow image processing software displayed on the main interface, displaying a multiphase flow image processing interface, and displaying fields of image processing parameters and corresponding parameter input controls on the multiphase flow image processing interface. The identification information of the multiphase flow image processing software can be text or a representative icon. The main interface can be the main interface of an operating system running on the computer device, such as a desktop, or it can be the main interface of a webpage. The multiphase flow image processing software can be a client or an application on a webpage. The triggering operation on the identification information of the multiphase flow image processing software can be a touch operation or a click operation.

[0048] In one embodiment, the computer device can determine the arrangement of image processing parameter fields and corresponding parameter input controls, and then, when displaying the multiphase flow image processing interface, draw the image processing parameter fields and corresponding parameter input controls presented in the multiphase flow image processing interface according to the arrangement. Specifically, the arrangement can be a matrix-aligned arrangement of image processing parameter fields and corresponding parameter input controls; alternatively, the arrangement can be adaptive based on the image processing parameter fields and corresponding parameter input controls, such as adaptively adjusting at least one of size, relative position, or spacing.

[0049] In one embodiment, the parameter input control may be in the form of a slider control. In this embodiment, the computer device can determine the value range of the image processing parameter, and when drawing the parameter input control in the form of a slider control corresponding to the image processing parameter, the value range of the image processing parameter is displayed in the display area of ​​the drawn parameter input control.

[0050] Step 204: Display the parameter values ​​of the image processing parameters entered through the parameter input control on the multiphase flow image processing interface.

[0051] Among them, the parameter values ​​of the image processing parameters are the values ​​that the image processing parameters take.

[0052] In one embodiment, the computer device can obtain the parameter value of the image processing parameter entered through a parameter input control displayed in the multiphase flow image processing interface, and display the entered parameter value in the display area of ​​the corresponding parameter input control.

[0053] Step 206: Continuously acquire multiphase flow images of the microfluidic chip with multiphase flow captured by a high-speed camera; the multiphase flow images record multiple phase-spaced bubbles generated by the multiphase flow in the microfluidic chip.

[0054] In multiphase flow, a phase refers to different states of matter or different physical properties or mechanical states of the same state of matter in nature. A state of matter is a state of matter, such as gas, liquid, or solid. Multiphase flow involves the mixing and flow of two or more phases of matter. In multiphase flow, the phases are immiscible, and there are interfaces between different phases. Multiphase flow can be a two-phase flow; a two-phase flow can be a two-phase flow of matter in different states of matter, specifically a gas-liquid two-phase flow (gas and liquid), a gas-solid two-phase flow (gas and solid), or a liquid-solid two-phase flow (liquid and solid); a two-phase flow can also be a two-phase flow of matter in the same state of matter with different physical properties or mechanical states, specifically an oil-water two-phase flow (oil and water).

[0055] Microfluidic chips are chips with microchannels that allow for the processing or manipulation of tiny fluids. Fluids are flowable substances, including liquids and gases. High-speed cameras are digital industrial cameras with high shooting speeds. High-speed cameras can capture tens to hundreds of images per second, compared to two to three images per second for ordinary cameras. The multiphase flow images output by high-speed cameras can be in RAW (raw data) format.

[0056] Multiphase flow images are images captured by a high-speed camera of the multiphase flow present in a microfluidic chip. Different multiphase flow images are taken at different times. It is understandable that during the operation of the microfluidic chip, the multiphase material can continuously flow within the chip, therefore, the multiphase flow images captured at different times will be different.

[0057] Bubble-like objects are objects shaped like bubbles, which can appear as circles, ellipses, rounded rectangles, or other shapes in images. Bubble-like objects can be formed from single-phase substances due to the immiscibility of different phases in a multiphase flow. Specifically, they can be bubbles formed from gas in a gas-liquid two-phase flow, or droplets (water droplets formed from water or oil droplets formed from oil) formed in an oil-water two-phase flow.

[0058] In one embodiment, a computer device continuously receives multiphase flow images captured by a high-speed camera onto a microfluidic chip exhibiting multiphase flow. The high-speed camera transmits multiphase flow images captured within a preset time interval to the computer device. This preset time interval can be one minute, one second, one millisecond, or other durations. The high-speed camera can capture two hundred frames of multiphase flow images per second.

[0059] In one embodiment, the computer device can acquire a preset number of multiphase flow images of a microfluidic chip with multiphase flow captured by a high-speed camera, and after performing step 208, acquire the next preset number of multiphase flow images.

[0060] Step 208: Perform image processing based on each acquired multiphase flow image according to the parameter values ​​of the image processing parameters to obtain information on multiple bubbles identified from each multiphase flow image.

[0061] The information about the multiple bubbles refers to data describing the characteristics of the multiple bubbles. This information may include the number of bubbles, the size of each bubble, etc.

[0062] In one embodiment, the computer device may perform denoising processing on each acquired multiphase flow image, and perform image processing on each denoised multiphase flow image according to the parameter values ​​of the image processing parameters to obtain information on multiple bubbles identified from each denoised multiphase flow image.

[0063] Step 210: When it is determined that the multiphase flow in the microfluidic chip is in a stable state based on the change state of the bubbles at the preset inlet region of the microfluidic chip in the processed multiphase flow image, the information of multiple bubbles identified from the multiphase flow image is stored.

[0064] The preset inlet region is a pre-defined inlet area for substances to enter the microchannels within the microfluidic chip. The changing state of the bubbles characterizes the degree of change in the bubbles. The multiphase flow being in a stable state indicates that the flow of each phase in the multiphase flow is relatively stable.

[0065] In one embodiment, a computer device can determine the preset inlet region of a microfluidic chip from different processed multiphase flow images, compare the bubbles in their respective preset inlet regions in the different processed multiphase flow images to obtain the change state of the bubbles, and determine that the multiphase flow in the microfluidic chip is in a stable state when the change state of the bubbles indicates that the degree of change of the bubbles is small.

[0066] In the aforementioned microfluidic-based multiphase flow image processing method, the parameter values ​​for image processing can be input through the multiphase flow image processing interface, making operation convenient. Furthermore, after cyclically acquiring multiphase flow images captured by a high-speed camera on a microfluidic chip containing multiphase flow, image processing is performed on each acquired multiphase flow image according to the input image processing parameter values. This allows for batch processing of multiphase flow images from the high-speed camera, resulting in high image processing efficiency and rapid acquisition of information on multiple bubbles in the multiphase flow images. Moreover, the microfluidic chip has a preset inlet region. When the multiphase flow in the microfluidic chip is determined to be in a stable state based on the changing state of the bubbles at the preset inlet region in the processed multiphase flow image, information on multiple bubbles identified from the multiphase flow image is stored, reducing the storage of invalid data and enabling rapid and efficient acquisition of information on multiple bubbles in the cyclically transmitted multiphase flow images from the high-speed camera, thus improving data acquisition efficiency.

[0067] In one embodiment, such as Figure 3 As shown, the image processing parameters include binarization parameters and bubble screening parameters, and step 208 includes the following steps 302 to 306.

[0068] Step 302: Based on the parameter values ​​of the binarization parameters, perform binarization processing on each acquired multiphase flow image to obtain the corresponding binarized image for each multiphase flow image.

[0069] The binarization parameter is used to perform binarization processing on the image. Specifically, the binarization parameter can be a binarization threshold, and its value can be any value of the threshold. Binarization processing replaces the pixel value of each pixel in the image with one of two preset, different values. Specifically, it updates the pixel value of pixels whose values ​​meet the binarization parameter's value to one of the two preset, different values, and updates the pixel value of pixels whose values ​​do not meet the parameter's value to the other of the two preset, different values. The two preset, different values ​​can be 0 and 255.

[0070] In one embodiment, the computer device can perform denoising processing on each acquired multiphase flow image, and perform binarization processing on each denoised multiphase flow image according to the parameter value of the binarization parameter to obtain the corresponding binarized image for each multiphase flow image.

[0071] Step 304: According to the parameter values ​​of the bubble screening parameters, identify the contours of multiple bubbles in the corresponding multiphase flow image from each binarized image.

[0072] The bubble selection parameters are used to identify and filter out the contours of bubbles from a binary image. The individual contours of multiple bubbles are lines that characterize the shape edges formed by each bubble in the image.

[0073] In one embodiment, the bubble screening parameter can be an opening limitation parameter. In this embodiment, the computer device can identify multiple candidate contours from each binary image, and remove candidate contours with openings larger than the parameter value of the opening limitation parameter from the identified candidate contours. The remaining candidate contours are then used as the respective contours of multiple bubbles in the corresponding multiphase flow image of each binary image. Here, the opening limitation parameter is an opening size limitation parameter.

[0074] Step 306: Based on the contours of the multiple bubbles identified in each binarized image, determine the information of the multiple bubbles in the multiphase flow image corresponding to each binarized image.

[0075] In one embodiment, a computer device can determine the size of the contour of each of the multiple bubbles identified in each binarized image, and use the obtained size of the contour of each of the multiple bubbles as information of the multiple bubbles in the multiphase flow image corresponding to each binarized image.

[0076] In this embodiment, binarization is performed on the parameter values ​​of the binarization parameters and each acquired multiphase flow image to obtain a binarized image. This makes it easier to extract the contours of multiple bubbles. Then, according to the parameter values ​​of the bubble selection parameters, the contours of multiple bubbles in the corresponding multiphase flow image are identified from each binarized image, which improves the recognition efficiency of bubble contours in multiphase flow images. Furthermore, information about multiple bubbles can be obtained based on their respective contours, thus improving data acquisition efficiency.

[0077] In one embodiment, step 302 includes: denoising each acquired multiphase flow image, and when the denoised multiphase flow image is tilted, performing image straightening processing on the denoised multiphase flow image to obtain each corrected multiphase flow image; and performing binarization processing on each corrected multiphase flow image according to the parameter values ​​of the binarization parameters to obtain the corresponding binarized image for each multiphase flow image.

[0078] Denoising is the process of removing noise from an image. Tilted image straightening is the process of adjusting an image from a tilted state to a straight state.

[0079] In this embodiment, the multiphase flow image is denoised, and when the denoised multiphase flow image is tilted, it is straightened to reduce interference information in the corrected multiphase flow image. Then, the corrected multiphase flow image is binarized to obtain a binarized image with less interference information, thereby improving the accuracy of bubble contour recognition.

[0080] In one embodiment, a computer device may employ a denoising algorithm to denoise each acquired multiphase flow image, thereby obtaining a denoised multiphase flow image. The denoising algorithm is the algorithm used to denoise the image. The denoising algorithm may be a median filtering algorithm, a mean filtering algorithm, a Gaussian filtering algorithm, or others.

[0081] In one embodiment, the computer device may employ a denoising algorithm to denoise each acquired multiphase flow image. When the denoised multiphase flow image is not in a tilted state, each denoised multiphase flow image is used as a corrected multiphase flow image.

[0082] In one embodiment, a computer device can extract the image border of the denoised multiphase flow image using an edge recognition algorithm, determine the angle between the image border and the preset reference line in the direction of the preset reference line, and if the angle is greater than the preset angle, it is determined that the denoised multiphase flow image is tilted. The denoised multiphase flow image is then rotated using a spatial transformation algorithm and processed using an interpolation algorithm to achieve image straightening, obtaining each corrected multiphase flow image. Here, the preset reference line is a pre-set reference straight line, which can be a horizontal or vertical straight line. The edge recognition algorithm is an algorithm used to identify edges, and can be the Hough transform algorithm. The spatial transformation algorithm is an algorithm that transforms the image in space, mapping the coordinate position in one image to a new coordinate position in another image. The spatial transformation algorithm can be an affine transformation or a projection transformation. The interpolation algorithm is an algorithm that estimates and fills the pixel values ​​of the spatially transformed image. The interpolation algorithm can be a nearest neighbor interpolation algorithm, a bilinear interpolation algorithm, or a bicubic interpolation algorithm.

[0083] In one embodiment, the bubble screening parameters include an opening limitation parameter and a size range parameter. Step 304 includes: identifying multiple contour points from each binarized image; a contour point is a pixel in the binarized image whose pixel value is different from the pixel value of any adjacent pixel; forming multiple candidate contours based on multiple sets of contour points that meet preset aggregation conditions; removing candidate contours with openings whose size is greater than the parameter value of the opening limitation parameter from the multiple candidate contours identified in each binarized image, and removing candidate contours whose contour size is outside the parameter value of the size range parameter; and using the remaining candidate contours after removal as the respective contours of multiple bubbles in the multiphase flow image corresponding to each binarized image.

[0084] In this context, adjacent pixels of a contour point are pixels sharing an edge with that contour point. Preset aggregation conditions are pre-defined conditions required to aggregate contour points into candidate contours. A preset aggregation condition can be that any contour point within a group of contour points has adjacent contour points with a pixel interval not exceeding a preset pixel distance. The preset pixel distance is a pre-defined pixel distance, which can be one pixel. A candidate contour is a contour to be filtered formed by the contour points in a binarized image. An opening is an unclosed open area in a candidate contour. The contour size is the dimension of the candidate contour, which can be the length of the candidate contour along the flow direction of the multiphase flow.

[0085] In this embodiment, multiple contour points are identified from the binarized image to determine preliminary candidate contours. Then, the candidate contours are filtered according to the parameter values ​​of the opening restriction parameter and the size range parameter input through the parameter input control, thereby obtaining the contours of multiple bubbles in the multiphase flow image corresponding to the binarized image, which improves the accuracy of contour recognition.

[0086] In one embodiment, the image processing parameters may further include a recognition range parameter, the value of which may be a percentage value. In this embodiment, the computer device may, for each binarized image, determine a recognition region that occupies a proportion of the binarized image equal to the value of the recognition range parameter, starting from the center of the binarized image, and identify multiple contour points from the recognition region.

[0087] In one embodiment, step 306 includes: taking the size of the minimum bounding rectangle of the contours of the plurality of bubbles identified in each binarized image as the size of the plurality of bubbles; determining the size of the bubbles at the preset inlet region and the preset outlet region of the microfluidic chip in each binarized image; counting the number of contours of the plurality of bubbles identified in each binarized image as the number of the plurality of bubbles; and taking the size of the plurality of bubbles identified in each binarized image, the size of the bubbles at the preset inlet region, the size of the bubbles at the preset outlet region, and the number of the plurality of bubbles as information of the plurality of bubbles in the multiphase flow image corresponding to each binarized image.

[0088] The minimum bounding rectangle is the rectangle with the smallest area that encloses the outline of the bubble. The dimensions of the minimum bounding rectangle can include its length and width. The length can be the side length of the rectangle in the flow direction of the multiphase flow, and the width can be the side length perpendicular to the length of the rectangle. The preset outlet region is a pre-defined outlet region where the material flows out of the microchannels in the microfluidic chip.

[0089] In this embodiment, the size of the bubble can be quickly determined by determining the size of the smallest bounding rectangle of the bubble's outline. The information of multiple bubbles includes the size of each bubble, the size of the bubbles at the preset inlet area and the preset outlet area, and the number of multiple bubbles. Multi-dimensional information about the bubbles is obtained through the outlines of multiple bubbles, resulting in high information richness.

[0090] In one embodiment, the information of the multiple bubbles includes the length of each bubble along the flow direction of the multiphase flow. The above-mentioned multiphase flow image processing method based on microfluidics further includes: obtaining the length of the bubbles at the preset inlet region of the microfluidic chip in the processed multiphase flow image corresponding to the captured multiphase flow image; when the maximum difference between the lengths of the bubbles at the preset inlet regions corresponding to the multiple multiphase flow images captured within a preset time interval is less than a preset threshold, it is determined that the multiphase flow in the microfluidic chip is in a stable state.

[0091] The preset time interval is a pre-set time interval. The preset time interval can be 10 seconds. The preset threshold is a pre-set threshold value. If the maximum difference between the lengths of the bubbles at the preset inlet region corresponding to each of the multiple multiphase flow images captured within the preset time interval is less than the preset threshold, it indicates that the length of the bubbles at the preset inlet region changes little or remains unchanged during the multiphase flow process.

[0092] In this embodiment, by comparing the difference between the lengths of the bubbles at the preset inlet regions corresponding to the multiple multiphase flow images captured within a preset time interval, the state of the multiphase flow in the microfluidic chip can be conveniently determined. Thus, when the multiphase flow processing is determined to be in a stable state, information of multiple bubbles can be stored, thereby improving data processing efficiency.

[0093] In one embodiment, in a specific application scenario, the multiphase flow can be a gas-liquid two-phase flow, and the bubble can be a bubble. The above-mentioned microfluidic-based multiphase flow image processing method specifically includes the following steps.

[0094] The computer device can respond to the triggering operation of the identification information of the multiphase flow image processing software displayed on the main interface, display the multiphase flow image processing interface, and display the fields of image processing parameters and the corresponding parameter input controls in the multiphase flow image processing interface, and display the parameter values ​​of the image processing parameters entered through the parameter input controls.

[0095] The schematic diagram of the multiphase flow image processing interface can be seen as follows: Figure 4 As shown, the multiphase flow image processing interface may include an operation interface (i.e., Figure 4 The area marked with "a" in the middle) and the image display interface (i.e. Figure 4 The region marked with "b" is shown in the image processing parameters. Fields for these parameters may include binarization threshold (binarization parameter), aperture limitation (aperture limitation parameter), aperture removal range (size range parameter), and recognition range (recognition range parameter). Multiphase flow image processing software can be written in computer programming languages ​​such as Python, C, C++, and Java.

[0096] The computer device can connect to a high-speed camera and continuously receive multiphase flow images of the microfluidic chip exhibiting multiphase flow, transmitted from the camera. See also... Figure 5 The multiphase flow images shown are from each stage of the multiphase flow image processing workflow. Figure 5 Image (a) is the unprocessed multiphase flow image. In the multiphase flow image processing software, the "Select Image" option (displayed as shown in the image) is used. Figure 4 When this function (as shown in the multiphase flow image processing interface) is enabled, the computer device can acquire and process selected images from images transmitted from a high-speed camera; when "video recognition" is enabled in the multiphase flow image processing software (displayed as shown in the interface), the computer device can acquire and process selected images from images transmitted from a high-speed camera; Figure 4 In the case of this function (as shown in the multiphase flow image processing interface), the computer device can acquire multiple frames of multiphase flow images from historical videos captured by a microfluidic chip containing multiphase flow, transmitted from a high-speed camera.

[0097] Computer equipment can use a median filtering algorithm to denoise each acquired multiphase flow image, obtaining a denoised multiphase flow image. The denoised multiphase flow image can then be processed using a Hough transform algorithm to identify and extract the image borders. The angle between the image border and a preset reference line is determined. If this angle is greater than a preset angle, the denoised multiphase flow image is considered tilted. The denoised multiphase flow image is then rotated using a spatial transformation algorithm followed by interpolation to achieve image straightening, obtaining a corrected multiphase flow image. See also... Figure 5 The multiphase flow images shown are from each stage of the multiphase flow image processing workflow. Figure 5 (b) in the image is the multiphase flow image after image denoising and straightening, i.e., the corrected multiphase flow image.

[0098] The computer equipment can perform binarization processing on each corrected multiphase flow image according to the parameter values ​​of the binarization parameters, obtaining a corresponding binarized image for each multiphase flow image. A median filtering algorithm is then used to denoise the binarized images, resulting in a denoised binarized image. See also... Figure 5 The multiphase flow images shown are from each stage of the multiphase flow image processing workflow. Figure 5 (c) in the image is the binarized image. Figure 5 In the image, (d) is the binarized image after denoising.

[0099] The computer device can, for each denoised binary image, determine a recognition region starting from the center of the denoised binary image, with a parameter value equal to the proportion of the recognition range parameter to the denoised binary image, and recognize multiple contour points from the recognition region; based on multiple sets of contour points that meet preset aggregation conditions, multiple candidate contours are formed; from the multiple candidate contours recognized in each denoised binary image, candidate contours with openings whose size is greater than the opening limit parameter are removed, and candidate contours whose contour size is outside the size range parameter parameter value are also removed; the remaining candidate contours after removal are used as the contours of multiple bubbles in the multiphase flow image corresponding to each denoised binary image.

[0100] The computer device can use the smallest bounding rectangle of the contours of multiple bubbles identified in each denoised binarized image as the size of each bubble; determine the size of the bubbles at the preset inlet and outlet regions of the microfluidic chip in each denoised binarized image; count the number of contours of multiple bubbles identified in each denoised binarized image as the number of bubbles; and use the size of each bubble identified in each denoised binarized image, the size of the bubbles at the preset inlet region, the size of the bubbles at the preset outlet region, and the number of bubbles as information about multiple bubbles in the multiphase flow image corresponding to each binarized image. The computer device can also acquire the following information about multiple bubbles: multiple liquid segment lengths, multiple unit lengths, gas flow rates, multiple unit volumes, multiple bubble volumes, and multiple liquid segment volumes.

[0101] In this context, a liquid segment is the liquid section between two adjacent bubbles along the flow direction of the gas-liquid two-phase flow. The liquid segment length is the length of the liquid segment along the flow direction. A unit consists of one adjacent bubble and one liquid segment along the flow direction of the gas-liquid two-phase flow. The unit length is the length of the unit along the flow direction of the liquid two-phase flow. Gas flow rate is the amount of gas passing through any cross-section per unit time in the gas-liquid two-phase flow. Unit volume refers to the volume of the unit, which can be the sum of the corresponding bubble volume and liquid segment volume. Bubble volume refers to the volume of a bubble, which can be calculated using the bubble length, bubble width, and bubble thickness. Liquid segment volume refers to the volume of a liquid segment, which can be calculated using the liquid segment length, liquid segment width, and liquid segment thickness. The bubble thickness and liquid segment thickness can be the same, or both can be the thickness of the microchannels in the microfluidic chip.

[0102] like Figure 4 The area marked with "b" allows the computer device to display a multiphase flow image after denoising and alignment processing on the image display interface. The computer device draws the outlines of each identified bubble in the displayed multiphase flow image and the minimum bounding rectangle of each bubble's outline. Each bubble's outline is numbered to obtain a bubble number, which is then displayed. The bubble's length and width are displayed in the bubble size display area, and information about the multiple bubbles is displayed in the information display area of ​​the image display interface. See also... Figure 5 The multiphase flow images shown are from each stage of the multiphase flow image processing workflow. Figure 5 Image (e) shows a multiphase flow image with the bubble outline and minimum bounding rectangle drawn. Figure 5 (f) in the middle is Figure 5 Detailed image of (e) in the image.

[0103] in, Figure 4In the information display area of ​​the image display interface, “frame

[24] ” represents the 24th frame of the processed image, “number of bubbles

[41] ” represents 41 bubbles identified, “inlet,outlet[63.00,47.00]” represents the length of the bubble at the preset inlet area (63) and the length of the bubble at the preset outlet area (47), both in pixels, “waterlen & cell len[33,96]” represents the length of the liquid segment (33) and the length of the unit (96), and “QG, V_cell,Vg, V_L,[52.49,4.80,2.46,2.34,2.34,96,63]” represents the gas flow rate (52.49 ml / h), the unit volume (4.80 mm^3), the bubble volume (2.46 mm^3), the liquid segment volume (2.34 mm^3), the unit length (96), and the length of the bubble at the inlet (63).

[0104] When the maximum difference between the lengths of bubbles at the preset inlet regions corresponding to multiple multiphase flow images captured within a preset time interval is less than a preset threshold, the computer device can determine that the multiphase flow in the microfluidic chip is in a stable state and begin storing information about multiple bubbles identified from the multiphase flow images.

[0105] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0106] Based on the same inventive concept, this application also provides a microfluidic-based multiphase flow image processing apparatus for implementing the microfluidic-based multiphase flow image processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more embodiments of the microfluidic-based multiphase flow image processing apparatus provided below can be found in the limitations of the microfluidic-based multiphase flow image processing method described above, and will not be repeated here.

[0107] In one embodiment, such as Figure 6As shown, a multiphase flow image processing device 600 based on microfluidics is provided, including: a display module 610, an image acquisition module 620, an image processing module 630, and a storage module 640, wherein:

[0108] The display module 610 is used to display the multiphase flow image processing interface; the multiphase flow image processing interface displays parameter input controls corresponding to the image processing parameters; and the multiphase flow image processing interface displays the parameter values ​​of the image processing parameters input through the parameter input controls.

[0109] The image acquisition module 620 is used to cyclically acquire multiphase flow images of a microfluidic chip with multiphase flow captured by a high-speed camera; the multiphase flow images record multiple phase-spaced bubbles generated by the multiphase flow in the microfluidic chip.

[0110] The image processing module 630 is used to perform image processing based on each acquired multiphase flow image according to the parameter values ​​of the image processing parameters, and to obtain information on multiple bubbles identified from each multiphase flow image.

[0111] The storage module 640 is used to start storing information of multiple bubbles identified from the multiphase flow image when the multiphase flow in the microfluidic chip is determined to be in a stable state based on the change state of the bubbles at the preset inlet region of the microfluidic chip in the processed multiphase flow image.

[0112] In one embodiment, the image processing parameters include binarization parameters and bubble screening parameters. The image processing module 630 is further configured to perform binarization processing on each acquired multiphase flow image according to the parameter values ​​of the binarization parameters to obtain a binarized image corresponding to each multiphase flow image; to identify the contours of multiple bubbles in the corresponding multiphase flow image from each binarized image according to the parameter values ​​of the bubble screening parameters; and to determine the information of multiple bubbles in the multiphase flow image corresponding to each binarized image based on the contours of the multiple bubbles identified in each binarized image.

[0113] In one embodiment, the image processing module 630 is further configured to perform denoising processing on each acquired multiphase flow image, and when the denoised multiphase flow image is in a tilted state, perform image straightening processing on the denoised multiphase flow image to obtain each corrected multiphase flow image; and perform binarization processing on each corrected multiphase flow image according to the parameter value of the binarization parameter to obtain the corresponding binarized image of each multiphase flow image.

[0114] In one embodiment, the bubble screening parameters include an opening restriction parameter and a size range parameter. The image processing module 630 is further configured to identify multiple contour points from each binarized image. A contour point is a pixel in the binarized image whose pixel value is different from the pixel value of any adjacent pixel. Based on multiple sets of contour points that meet preset aggregation conditions, multiple candidate contours are formed. From the multiple candidate contours identified in each binarized image, candidate contours with openings whose size is greater than the parameter value of the opening restriction parameter are removed, and candidate contours whose contour size is outside the parameter value of the size range parameter are also removed. The remaining candidate contours after removal are used as the contours of the multiple bubbles in the multiphase flow image corresponding to each binarized image.

[0115] In one embodiment, the image processing module 630 is further configured to: use the size of the minimum bounding rectangle of the contours of the multiple bubbles identified in each binarized image as the size of the multiple bubbles; determine the size of the bubbles at the preset inlet region and the preset outlet region of the microfluidic chip in each binarized image; count the number of contours of the multiple bubbles identified in each binarized image as the number of multiple bubbles; and use the size of the multiple bubbles identified in each binarized image, the size of the bubbles at the preset inlet region, the size of the bubbles at the preset outlet region, and the number of multiple bubbles as information about the multiple bubbles in the multiphase flow image corresponding to each binarized image.

[0116] In one embodiment, the information of the multiple bubbles includes the length of each bubble along the flow direction of the multiphase flow; the microfluidic-based multiphase flow image processing device 600 further includes a state determination module, which is used to obtain the length of the bubbles at the preset inlet region of the microfluidic chip in the processed multiphase flow image corresponding to the captured multiphase flow image; when the maximum difference between the lengths of the bubbles at the preset inlet regions corresponding to the multiple multiphase flow images captured within a preset time interval is less than a preset threshold, it is determined that the multiphase flow in the microfluidic chip is in a stable state.

[0117] Each module in the aforementioned microfluidic-based multiphase flow image processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0118] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a microfluidic-based multiphase flow image processing method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0119] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0120] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0121] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0122] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0123] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0124] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A multiphase flow image processing method based on microfluidics, characterized in that, The method includes: The multiphase flow image processing interface is displayed; the multiphase flow image processing interface displays parameter input controls corresponding to the image processing parameters; The multiphase flow image processing interface displays the parameter values ​​of the image processing parameters input through the parameter input control; the image processing parameters include binarization parameters, aperture limitation parameters, and size range parameters; The system continuously acquires multiphase flow images of a microfluidic chip with multiphase flow captured by a high-speed camera; the multiphase flow images record multiple phase-spaced bubbles generated by the multiphase flow in the microfluidic chip. According to the parameter values ​​of the binarization parameters, binarization processing is performed on each of the acquired multiphase flow images to obtain the binarized image corresponding to each of the multiphase flow images. Multiple contour points are identified from each of the binarized images, and multiple candidate contours are formed based on multiple sets of contour points that meet preset aggregation conditions. From the multiple candidate contours identified in each of the binarized images, candidate contours with openings having a size greater than the parameter value of the opening limit parameter are removed, and candidate contours with contour sizes outside the parameter value of the size range parameter are also removed, thus obtaining the contours of each of the multiple bubble-like structures. The size of the smallest bounding rectangle of the contour of each of the plurality of bubbles identified in each of the binarized images is taken as the size of each of the plurality of bubbles; The size of the bubbles at the preset inlet and preset outlet regions of the microfluidic chip in each binarized image is determined, and the number of contours of each of the multiple bubbles identified in each binarized image is counted as the number of the multiple bubbles. The size of each of the multiple bubbles, the size of the bubbles at the preset inlet region, the size of the bubbles at the preset outlet region, and the number of the multiple bubbles are used as information about the multiple bubbles in the multiphase flow image corresponding to each binarized image. When the multiphase flow in the microfluidic chip is determined to be in a stable state based on the change state of the bubbles at the preset inlet region of the microfluidic chip in the processed multiphase flow image, the information of the multiple bubbles identified from the multiphase flow image is stored.

2. The method according to claim 1, characterized in that, The step of performing binarization processing on each of the acquired multiphase flow images according to the parameter values ​​of the binarization parameters to obtain a binarized image corresponding to each of the multiphase flow images includes: Each of the acquired multiphase flow images is denoised, and when the denoised multiphase flow image is tilted, the denoised multiphase flow image is straightened to obtain each of the corrected multiphase flow images. According to the parameter values ​​of the binarization parameters, each of the corrected multiphase flow images is binarized to obtain a binarized image corresponding to each multiphase flow image.

3. The method according to claim 1, characterized in that, The contour point is a pixel in the binarized image whose pixel value is different from the pixel value of any adjacent pixel.

4. The method according to claim 1, characterized in that, The information about the plurality of bubbles also includes the length of each bubble along the flow direction of the multiphase flow; the method further includes: The length of the bubble at the preset inlet region of the microfluidic chip is obtained in the processed multiphase flow image corresponding to the captured multiphase flow image; If the maximum difference between the lengths of the bubbles at the preset inlet regions of multiple multiphase flow images captured within a preset time interval is less than a preset threshold, the multiphase flow in the microfluidic chip is determined to be in a stable state.

5. A multiphase flow image processing device based on microfluidics, characterized in that, The device includes: The display module is used to display a multiphase flow image processing interface; the multiphase flow image processing interface displays parameter input controls corresponding to image processing parameters; the multiphase flow image processing interface displays the parameter values ​​of the image processing parameters input through the parameter input controls; the image processing parameters include binarization parameters, aperture limitation parameters, and size range parameters; The image acquisition module is used to cyclically acquire multiphase flow images of a microfluidic chip with multiphase flow captured by a high-speed camera; the multiphase flow images record multiple phase-spaced bubbles generated by the multiphase flow in the microfluidic chip; The image processing module is configured to perform binarization processing on each acquired multiphase flow image according to the parameter values ​​of the binarization parameters, to obtain a binarized image corresponding to each multiphase flow image; identify multiple contour points from each binarized image, and form multiple candidate contours based on multiple sets of contour points that meet preset aggregation conditions; from the multiple candidate contours identified in each binarized image, remove candidate contours with openings whose size is greater than the parameter value of the opening limitation parameter, and remove candidate contours whose contour size is outside the parameter value of the size range parameter, to obtain the contours of each of the multiple bubbles; and process each of the multiple bubbles into a single image. The dimensions of the smallest bounding rectangle of the contours of the plurality of bubbles identified in the binarized image are taken as the dimensions of the plurality of bubbles; the dimensions of the bubbles at the preset inlet region and preset outlet region of the microfluidic chip in each binarized image are determined, and the number of contours of the plurality of bubbles identified in each binarized image is counted as the number of the plurality of bubbles; the dimensions of the plurality of bubbles, the dimensions of the bubbles at the preset inlet region, the dimensions of the bubbles at the preset outlet region, and the number of the plurality of bubbles are taken as the information of the plurality of bubbles in the multiphase flow image corresponding to each binarized image; The storage module is used to determine that the multiphase flow in the microfluidic chip is in a stable state based on the change state of the bubbles at the preset inlet region of the microfluidic chip in the processed multiphase flow image, and then start storing the information of the multiple bubbles identified from the multiphase flow image.

6. The apparatus according to claim 5, characterized in that, The image processing module is used to perform denoising processing on each of the acquired multiphase flow images, and to perform image straightening processing on the denoised multiphase flow images when the denoised multiphase flow images are in a tilted state, so as to obtain each of the corrected multiphase flow images. According to the parameter values ​​of the binarization parameters, each of the corrected multiphase flow images is binarized to obtain a binarized image corresponding to each multiphase flow image.

7. The apparatus according to claim 5, characterized in that, The contour point is a pixel in the binarized image whose pixel value is different from the pixel value of any adjacent pixel.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images