Layered recognition method and device of electrophoresis image, electronic equipment and storage medium

Abstract

The invention provides a layered recognition method and device for an electrophoresis image, electronic equipment and a storage medium. The method comprises the steps that computer equipment carries out image segmentation on a target electrophoresis image to obtain a plurality of lane sub-images; identifying a strip region contained in each lane sub-image to obtain one or more initial strip regions of one or more lane sub-images; performing density evaluation on each initial strip region to obtain a density score of each initial strip region; screening the initial strip region with the density fraction greater than a density threshold value as a first strip region; determining an overlapping degree between each first strip region in each other lane sub-image and each first strip region in the contrast lane sub-image, and determining a second strip region; and determining the positive degree of the target electrophoresis image according to the density score of each second strip region. According to the method and the device, the accuracy of identifying the strip region in the electrophoresis image by the computer equipment can be improved, and the accuracy of layered identification of the electrophoresis image is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and particularly to a method, apparatus, electronic device, and storage medium for hierarchical recognition of electrophoresis images. Background Art

[0002] Immunofixation Electrophoresis (IFE) is a qualitative analysis technique for detecting abnormal proteins. After adding specific immunoglobulin heavy chains (such as anti-IgG, IgA, IgM) and light chains (anti-K (κ, Kappa), anti-L (λ, Lambda)) antibodies to a sample in the IFE technique, the immunoglobulin of the antibody in the sample binds to the antibody to form bands.

[0003] Currently, corresponding analysis results can be obtained by analyzing the positioning of bands between the heavy chains (G, A, and M) and light chain lanes (K and L) in the IFE map. However, since the content and distribution of immunoglobulins affect the resolution of the bands formed in the IFE map after the binding of immunoglobulins to the corresponding antibodies, it affects the accuracy of identifying the bands in the IFE map, and further leads to deviations in the analysis of the positive degree reflected by the IFE map. Summary of the Invention

[0004] Embodiments of the present application disclose a method, apparatus, electronic device, and storage medium for hierarchical recognition of electrophoresis images, which can improve the accuracy of a computer device in identifying band regions in an electrophoresis image and improve the accuracy of hierarchical recognition of electrophoresis images.

[0005] A first aspect of embodiments of the present application discloses a method for hierarchical recognition of electrophoresis images, including:

[0006] A computer device performs image segmentation on a target electrophoresis image to obtain multiple sub-images of electrophoresis lanes, and the multiple sub-images of electrophoresis lanes include a control sub-image of an electrophoresis lane and multiple other sub-images of electrophoresis lanes;

[0007] The computer device identifies band regions included in each of the sub-images of electrophoresis lanes to obtain one or more initial band regions in one or more of the sub-images of electrophoresis lanes;

[0008] The computer device performs density evaluation on each of the initial band regions to obtain a density score corresponding to each of the initial band regions;

[0009] The computer device screens out initial band regions with density scores greater than a density threshold as first band regions;

[0010] The computer device determines the degree of overlap between each first strip region in each of the other lane sub-images and each first strip region in the control lane sub-image;

[0011] The computer device determines, according to the degree of overlap corresponding to each first strip region in each of the other lane sub-images, the first strip regions that exist simultaneously in the other lane sub-images and the control lane sub-image as second strip regions;

[0012] The computer device determines the positive degree corresponding to the target electrophoresis image according to the density fraction corresponding to each of the second strip regions.

[0013] In some possible embodiments, the computer device identifies the strip regions included in each of the lane sub-images to obtain one or more initial strip regions in each of the lane sub-images, including:

[0014] The computer device searches for each strip region included in the first lane sub-image to obtain a first strip coordinate set corresponding to the first lane sub-image, where the first lane sub-image is any one of the lane sub-images, and the first strip coordinate set includes the image vertical coordinates of each strip region in the first lane sub-image;

[0015] The computer device identifies the foreground region and the background region of the first lane sub-image and fills the background region with a target pixel value to obtain a second lane sub-image;

[0016] The computer device searches for each strip region included in the second lane sub-image to obtain a second strip coordinate set corresponding to the second lane sub-image, where the second strip coordinate set includes the image vertical coordinates of each strip region in the second lane sub-image;

[0017] The computer device determines each initial strip region included in the first lane sub-image according to the first strip coordinate set and the second strip coordinate set.

[0018] In some possible embodiments, the computer device searches for each strip region included in the first lane sub-image to obtain a first strip coordinate set corresponding to the first lane sub-image, including:

[0019] The computer device converts the first lane sub-image into a grayscale image;

[0020] The computer device accumulates the grayscale values of each pixel of the grayscale image in the horizontal direction and calculates the average value to generate an index pixel array. The index pixel array includes N elements, and the N elements correspond one by one to the N rows of pixels of the grayscale image. The value of the Pth element is the average value corresponding to the Pth row of pixels of the grayscale image; N and P are positive integers, and P is less than or equal to N;

[0021] The computer device generates a gradient change curve according to the index pixel array;

[0022] The computer device determines a target element where a gradient mutation occurs according to the gradient change curve and obtains the image ordinate corresponding to the target element;

[0023] The computer device calculates the line segment distance corresponding to every two adjacent target elements according to the image ordinates corresponding to each target element;

[0024] The computer device filters out the target elements whose corresponding line segment distances are less than the distance threshold to obtain the first strip coordinate set of the first sub-image of the lane.

[0025] In some possible embodiments, the computer device determines each initial strip region included in the first sub-image of the lane according to the first strip coordinate set and the second strip coordinate set, including:

[0026] The computer device obtains the first image ordinate and the second image ordinate corresponding to each strip region in the first sub-image of the lane according to the first strip coordinate set and the second strip coordinate set. The first image ordinate is the image ordinate of the top edge of the strip region, and the second image ordinate is the image ordinate of the bottom edge of the strip region;

[0027] The computer device expands the positions of each strip region in the first sub-image of the lane according to the expansion coefficient, the first image ordinate and the second image ordinate corresponding to each strip region in the first sub-image of the lane to obtain each initial strip region included in the first lane image.

[0028] In some possible embodiments, the computer device performs a density evaluation on each initial strip region to obtain the density score corresponding to each initial strip region, including:

[0029] The computer device calculates the color histogram corresponding to the first initial strip region, and the first initial strip region is any one of the initial strip regions;

[0030] The computer device calculates the kurtosis, color entropy and brightness uniformity corresponding to the color histogram according to the color histogram;

[0031] The computer device calculates the weighted sum of the kurtosis, color entropy, and brightness uniformity corresponding to the color histogram to obtain the density fraction corresponding to the first initial strip region.

[0032] In some possible embodiments, the computer device calculates the kurtosis, color entropy, and brightness uniformity corresponding to the color histogram according to the color histogram, including:

[0033] The computer device calculates the initial kurtosis, initial color entropy, and brightness uniformity corresponding to the color histogram according to the color histogram;

[0034] The computer device performs normalization processing on the initial kurtosis and the initial color entropy respectively to obtain the target kurtosis and the target color entropy;

[0035] The computer device calculates the weighted sum of the kurtosis, color entropy, and brightness uniformity corresponding to the color histogram to obtain the density fraction corresponding to the first initial strip region, including:

[0036] The computer device calculates the weighted sum of the target kurtosis, target color entropy, and brightness uniformity corresponding to the color histogram to obtain the density fraction corresponding to the first initial strip region.

[0037] In some possible embodiments, before the computer device performs image segmentation on the target electrophoresis image to obtain multiple lane sub-images, the method further includes:

[0038] The computer device converts the target electrophoresis image from the RGB color space to the HSV color space to obtain an HSV image;

[0039] The computer device identifies the blue pixel regions included in the HSV image and determines whether there is an intersection of region contours in the multiple blue pixel regions;

[0040] The computer device performs image segmentation on the target electrophoresis image to obtain multiple lane sub-images, including:

[0041] The computer device performs segmentation on the target electrophoresis image to obtain the multiple lane sub-images when there is no intersection of region contours in the multiple blue pixel regions.

[0042] A second aspect of the embodiments of the present application discloses a hierarchical recognition device for electrophoresis images, and the device includes:

[0043] A lane segmentation module for segmenting a target electrophoresis image to obtain multiple lane sub-images, where the multiple lane sub-images include a control lane sub-image and multiple other lane sub-images;

[0044] A band recognition module for recognizing the band regions included in each of the lane sub-images to obtain one or more initial band regions in one or more of the lane sub-images;

[0045] A calculation module for evaluating the density of each initial band region to obtain the density score corresponding to each initial band region;

[0046] A band screening module for screening out the initial band regions with density scores greater than the density threshold as the first band regions;

[0047] The calculation module is further configured to determine the overlapping degree between each first band region in each of the other lane sub-images and each first band region in the control lane sub-image;

[0048] A band determination module for determining, according to the overlapping degree corresponding to each first band region in each of the other lane sub-images, the first band regions that exist in both the other lane sub-images and the control lane sub-image as the second band regions;

[0049] A stratification recognition module for determining the positive degree corresponding to the target electrophoresis image according to the density score corresponding to each of the second band regions.

[0050] A third aspect of the embodiments of the present application discloses an electronic device, including a memory and a processor, where a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is enabled to implement the electrophoresis image stratification recognition method described in any of the above embodiments.

[0051] A fourth aspect of the embodiments of the present application discloses a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the processor is enabled to implement the electrophoresis image stratification recognition method described in any of the above embodiments.

[0052] A method, device, electronic device, and storage medium for hierarchical recognition of electrophoresis images provided by the present application. The computer device performs image segmentation on the target electrophoresis image to obtain multiple sub-images of electrophoresis lanes. The computer device recognizes the band regions included in each sub-image of electrophoresis lanes to obtain one or more initial band regions in one or more sub-images of electrophoresis lanes. The computer device evaluates the density of each initial band region to obtain the density score corresponding to each initial band region. The computer device filters out the initial band regions with density scores greater than the density threshold as the first band regions. The computer device determines the overlapping degree between each first band region in each other sub-image of electrophoresis lanes and each first band region in the control sub-image of electrophoresis lanes. The computer device determines the first band regions that exist simultaneously in the other sub-images of electrophoresis lanes and the control sub-image of electrophoresis lanes according to the overlapping degrees corresponding to each first band region in each other sub-image of electrophoresis lanes.

[0053] In the embodiment of the present application, the computer device can first recognize the band regions included in each sub-image of electrophoresis lanes to obtain one or more initial band regions in one or more sub-images of electrophoresis lanes, and then calculate the density scores corresponding to each initial band region. The first band regions with the depth of band color meeting the requirements are filtered out by using the density threshold. According to the overlapping degree between each first band region in each other sub-image of electrophoresis lanes and each first band region in the control sub-image of electrophoresis lanes, the band regions with higher density that exist simultaneously in the other sub-images of electrophoresis lanes and the control sub-image of electrophoresis lanes are accurately recognized, which improves the accuracy of the computer device in recognizing the band regions in the electrophoresis image and also improves the accuracy of hierarchical recognition of the electrophoresis image. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0055] Figure 1 A schematic diagram of an electrophoresis image provided for an embodiment of the present application;

[0056] Figure 2 A flowchart of a method for hierarchical recognition of an electrophoresis image provided for an embodiment of the present application;

[0057] Figure 3 A schematic diagram of image segmentation of a target electrophoresis image provided for an embodiment of the present application;

[0058] Figure 4Schematic diagram for identifying band regions included in each sub-image of a swimming lane provided by an embodiment of the present application;

[0059] Figure 5 Schematic diagram for screening the first band region according to the density score provided by an embodiment of the present application;

[0060] Figure 6 Schematic diagram for scanning multiple first band regions in a control swimming lane sub-image provided by an embodiment of the present application;

[0061] Figure 7 Flowchart for determining each initial band region included in the first swimming lane sub-image provided by an embodiment of the present application;

[0062] Figure 8 Flowchart for determining the set of first band coordinates of the first swimming lane sub-image provided by an embodiment of the present application;

[0063] Figure 9 Flowchart for determining each initial band region included in the first swimming lane image provided by an embodiment of the present application;

[0064] Figure 10 Flowchart for determining the density score corresponding to the initial band region provided by an embodiment of the present application;

[0065] Figure 11 Flowchart for determining the density score corresponding to the initial band region after normalization processing provided by an embodiment of the present application;

[0066] Figure 12 Flowchart for performing quality control detection on a target electrophoresis image provided by an embodiment of the present application;

[0067] Figure 13 Structural block diagram of a hierarchical recognition device for an electrophoresis image provided by an embodiment of the present application;

[0068] Figure 14 Structural block diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0069] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0070] It should be noted that the terms "include" and "have" and any variations thereof in the embodiments of the present application and the accompanying drawings are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0071] In addition, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one (item)" or a similar expression thereof refers to any combination of these items, including any combination of a single item or plural items. For example, at least one (item) of a, b, and c can represent: a, or b, or c, or a and b, or a and c, or b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0072] The method for hierarchical recognition of electrophoresis images provided by the embodiments of the present application can be applied to a computer device, which may include but is not limited to a personal computer, a tablet computer, a laptop computer, a mobile phone, etc.

[0073] Exemplarily, Figure 1 is a schematic diagram of an electrophoresis image provided by the embodiments of the present application. As Figure 1 shown, the computer device can obtain an electrophoresis image generated by the IFE technology. The electrophoresis image may include characteristic band regions formed after the immunoglobulins in the sample bind to antibodies such as anti-IgG, IgA, IgM, K, and L. The electrophoresis image includes a control lane ELP (Electrophoresis, abbreviated as ELP), lane G, lane A, lane M, lane K, and lane L. The control lane ELP includes five band regions, two of which have darker colors and clearer boundaries, another two have lighter colors but clearer boundaries, and there is still one band region with a very light color and being too narrow. Lane G includes a band region with a darker color and clearer boundaries. Lanes A, M, and K do not have band regions. Lane L includes a band region with a darker color and clearer boundaries and a band region with a lighter color but clearer boundaries.

[0074] In some embodiments, the sample may include one or more of a serum sample, a urine sample, or other biological samples containing immunoglobulins.

[0075] The lanes of an electrophoresis image can refer to independent regions arranged in parallel on the electrophoresis medium. Each lane corresponds to a specific antibody or control treatment, with the aim of separating and immobilizing the corresponding immunoglobulins through the antibody, so that the distribution of immunoglobulins can be shown by staining. The distribution region of immunoglobulins is the band region of the lane.

[0076] In some embodiments, the lanes of an electrophoresis image can include a control lane ELP, a heavy chain lane, and a light chain lane. The heavy chain lane can include lane G, lane A, and lane M, and the light chain lane can include lane K and lane L. Among them, the control lane ELP generally does not use any antibody, but is directly stained to show the distribution of all proteins in the sample; lane G binds and immobilizes IgG-type immunoglobulins by adding anti-IgG antibody to show the distribution of IgG-type immunoglobulins. The same applies to lane A and lane M, which will not be elaborated here; lane K immobilizes all immunoglobulins carrying K light chains by adding anti-K antibody, and then shows the distribution of all immunoglobulins carrying K light chains. The same applies to lane L, which will not be elaborated here.

[0077] Under the action of an electric field, various immunoglobulins migrate according to their charges and molecular weights, and thus are separated into different band regions in the lane. Moreover, due to the different contents and concentrations of various immunoglobulins, the resolution degrees of the band regions in each lane of the electrophoresis image are also inconsistent.

[0078] The resolution degree of the band region can include the blurring degree of the band region boundary and the depth of the band region color. If the immunoglobulin content in the sample is higher and the immunoglobulin distribution is more concentrated, the boundary of the band region is more obvious, the color of the band region is deeper, the band region is denser and narrower, that is, the resolution degree of the band region is higher. For example Figure 1 the last band region in the control lane ELP shown in the order from top to bottom. If the immunoglobulin content in the sample is lower and the immunoglobulin distribution is more dispersed, the boundary of the band region is more blurred, the color of the band region is lighter, and the band region is more diffuse, that is, the resolution degree of the band region is lower. For example Figure 1 the first band region corresponding to lane L shown. If there is no immunoglobulin corresponding to the antibody in the sample, there will be no band region in the corresponding lane. For example, there is no IgM-type immunoglobulin in the sample, so the anti-IgM antibody cannot bind to the IgM-type immunoglobulin. Therefore, as Figure 1 shown, there is no band region in lane M.

[0079] In some embodiments, the darkness of the color of the band region is positively correlated with the positivity degree presented by the band region. The darker the color of the band region, the higher the distribution content of the immunoglobulin that can bind to the corresponding antibody in the band region. Therefore, the higher the positivity degree of the band region. On the contrary, if the color of the band region is lighter, the lower the distribution content of the immunoglobulin that can bind to the corresponding antibody in the band region, and the lower the positivity degree of the band region. Therefore, when it is determined that there is the same band region between the control lane ELP and other lanes, that is, there is a second band region that exists in both the control lane ELP and other lanes, the positivity degree of the second band region in each other lane can be judged by the darkness of the color of the band region, so as to determine the positivity degree of the electrophoresis image according to the positivity degree of the second band region in each other lane.

[0080] Exemplarily, as Figure 1 shown, both lane G and lane L have band regions with relatively clear boundaries, and the band region of lane G and the second band region of lane L respectively correspond to the last band region in the control lane ELP in the order from top to bottom, indicating that there is IgG-L type monoclonal immunoglobulin in this sample, which conforms to one of the characteristics of IgG-L type multiple myeloma. And because the colors of the band region of lane G and the second band region of lane L are darker, it indicates that there is a large amount of IgG-L type monoclonal immunoglobulin in this sample. Therefore, it can be determined that the positivity degrees of lane G and lane L are higher, that is, the positivity degree of the target electrophoresis image is higher.

[0081] In the process of layer-by-layer recognition of each band region in the electrophoresis image, since the resolution degrees of the band regions shown in each lane in the electrophoresis image are inconsistent, if the band regions are manually recognized, not only does the detector need to have professional recognition ability, but there are also certain deviations between the recognition results of different detectors, thus affecting the accuracy of the recognition of the band regions. In addition, since the detector usually relies on the human eye to recognize the darkness of the color of the band regions in each lane, rather than quantifying the darkness of the color of the band regions. Therefore, for lanes with lighter colors due to the low concentration of immunoglobulin content and the uneven distribution of immunoglobulin, or lanes with decolorization phenomena caused by protein staining during electrophoresis, due to the influence of the darker band regions in other lanes (especially the darker band regions in the control lane), it is easy for the detector to ignore those band regions with lighter colors in the lane but also having positive characteristics during the recognition process, thus affecting the accuracy of the layer-by-layer recognition of the band regions.

[0082] The embodiment of the present application provides a method for hierarchical recognition of electrophoresis images. The computer device can first identify the band regions included in each sub-image of the lane, obtain one or more initial band regions in one or more sub-images of the lane, and then calculate the density scores corresponding to each initial band region. The first band region with the depth of the band color meeting the requirements is screened out by using the density threshold. According to the overlapping degree between each first band region in each other lane sub-image and each first band region in the control lane sub-image, the band region with a higher density that exists in both the other lane sub-image and the control lane sub-image is accurately identified, improving the accuracy of the computer device in identifying the band region in the electrophoresis image and the accuracy of hierarchical recognition of the electrophoresis image.

[0083] As Figure 2 shown, in one embodiment, a method for hierarchical recognition of electrophoresis images is provided. The method may include the following steps:

[0084] Step 202, the computer device performs image segmentation on the target electrophoresis image to obtain multiple sub-images of the lane.

[0085] The target electrophoresis image may include the electrophoresis image obtained after the target sample is processed by IFE. The target sample may include one or more of a serum sample, a urine sample, or other biological samples containing immunoglobulins. The multiple sub-images of the lane may include a control lane sub-image and multiple other lane sub-images. Optionally, the other lane sub-images may include a heavy chain lane sub-image and a light chain lane sub-image. The heavy chain lane sub-image may include a lane G sub-image, a lane A sub-image, and a lane M sub-image. The light chain lane sub-image may include a lane K sub-image and a lane L sub-image.

[0086] The computer device can perform image segmentation on the target electrophoresis image to obtain multiple sub-images of the lane corresponding one-to-one to multiple lanes, and the dimensions and sizes of the multiple sub-images of the lane are the same.

[0087] In some embodiments, the computer device can perform image segmentation on the target electrophoresis image based on the position information of each lane included in the target electrophoresis image. The position information may include the coordinate information and size information of the lane.

[0088] Exemplarily, as Figure 3 shown, the computer device performs image segmentation on the Figure 1 target electrophoresis image shown to obtain a control lane sub-image, a lane G sub-image, a lane A sub-image, a lane M sub-image, a lane K sub-image, and a lane L sub-image. The size of each sub-image of the lane is the same, and each sub-image of the lane corresponds to a lane respectively. For example, the lane G sub-image only corresponds to and displays lane G and does not display other lanes.

[0089] The computer device can obtain the position information of each lane in the target electrophoresis image based on an image recognition algorithm, and then perform image segmentation on the target electrophoresis image according to the image segmentation algorithm and the position information of each lane to obtain multiple sub-images of the lanes corresponding one by one to the multiple lanes. Alternatively, the computer device can also directly perform image segmentation on the target electrophoresis image based on the established position information of each lane, but this is not limited thereto.

[0090] In some embodiments, each lane in the target electrophoresis image is vertically arranged at a predetermined position, and the width and height of each lane are the same. Therefore, the computer device can obtain the position information of each lane in the electrophoresis image, and then segment the target electrophoresis image into multiple sub-images of the lanes according to the position information of each lane, so that each sub-image of the lane corresponds to one lane, and the dimensions and sizes of each sub-image of the lane are the same.

[0091] By performing image segmentation on the target electrophoresis image, the computer device makes a sub-image of a lane display only one lane, so that when the computer device processes the sub-image of the lane, it only processes one lane, which can not only improve the processing efficiency of the computer device, but also avoid the possible interference of adjacent lanes on the currently processed lane, and improve the accuracy of the computer device in analyzing and processing the target electrophoresis image.

[0092] Step 204, the computer device identifies the band regions included in each sub-image of the lane to obtain one or more initial band regions in one or more sub-images of the lane.

[0093] The band region may include the region in the sub-image of the lane that is different from the background color of the lane. The band regions included in each sub-image of the lane may include all regions in each sub-image of the lane that are different from the background color of the lane. The identification of the band regions included in each sub-image of the lane may include the identification of the boundaries of the band regions included in each sub-image of the lane. The initial band region may include the band region with boundaries in each sub-image of the lane.

[0094] It can be understood that during the electrophoresis process, the immunoglobulin is stained to clearly show its specific position in the lane. Since under the action of the electric field, the same type of immunoglobulin will aggregate at a specific position in the lane, thus forming a region different from the background color of the lane, the computer device can use these regions as the band regions. However, since the content and distribution of the immunoglobulin will affect the degree of boundary blurring of the band regions, and then make the boundaries corresponding to each band region unclear, the computer device can identify the boundaries corresponding to each band region, clarify the boundaries corresponding to each band region, so as to determine the initial band region with boundaries, facilitate the subsequent steps, and improve the accuracy of band region identification.

[0095] The computer device can extract all regions in each lane sub-image that are different from the lane background color as band regions, and identify the band regions included in each lane sub-image to determine the boundaries of each band region, so as to obtain one or more initial band regions with boundaries in each lane sub-image. It should be noted that when the computer device fails to extract the band region of the lane sub-image, it can directly determine that the lane sub-image does not have an initial band region. For example Figure 1 lane M shown.

[0096] In some embodiments, a lane sub-image may include one or more band regions, or may not have a band region.

[0097] In some embodiments, the computer device can identify the band regions included in each lane sub-image, obtain the image coordinates of each band region in each lane sub-image, so as to obtain one or more initial band regions in one or more lane sub-images.

[0098] The image coordinates may include the image abscissa and image ordinate in the plane rectangular coordinate system constructed with the lower left vertex of the lane sub-image as the origin. Optionally, the image ordinate may include the image ordinates corresponding to the top and bottom edges of the band region respectively, and the image abscissa may include the image abscissas corresponding to the two side edges of the band region respectively.

[0099] In some embodiments, the computer device can construct a rectangular coordinate system with the lower left vertex of the band region as the origin, so as to determine the image ordinates corresponding to the top and bottom edges of the band region respectively, and the image abscissas corresponding to the two side edges respectively.

[0100] In other embodiments, the identification of the band region may further include identifying the boundary of the band region through an image processing algorithm. The image processing algorithm may include but is not limited to the Canny edge detection algorithm and the Hough transform, etc. The computer device identifies the boundary of the band region through the image processing algorithm, and can accurately locate the band region in the lane sub-image, so as to determine the initial band region.

[0101] Exemplarily, as Figure 4 shown, the band regions within the square areas in each lane sub-image are the band regions identified by the computer device, and the square areas are only used to indicate the approximate positions of the band regions, rather than representing the true sizes of the band regions. The control lane sub-image contains initial band region 401, initial band region 402, initial band region 403, initial band region 404 and initial band region 405. The lane G sub-image has initial band region 406. The lane L sub-image contains initial band region 407 and initial band region 408. The lane A sub-image, the lane M sub-image and the lane K sub-image do not have initial band regions.

[0102] Step 206: The computer device performs a density evaluation on each initial strip region to obtain a density score corresponding to each initial strip region.

[0103] The density evaluation may refer to the process of quantifying the darkness of the color of the initial strip region, and the density score may refer to a quantitative index used to evaluate the darkness of the color of the initial strip region. The higher the density score of the initial strip region, the darker the color of the initial strip region, the denser the initial strip region, and the easier it is to distinguish the initial strip region; on the contrary, the lower the density score, the lighter the color of the initial strip region, and the more difficult it is to distinguish the initial strip region from the background region of the lane sub-image.

[0104] The computer device may perform a density evaluation on each initial strip region in each lane sub-image according to the pixel values corresponding to each initial strip region in each lane sub-image, so as to quantify the darkness of the color of each initial strip region into a density score.

[0105] In some embodiments, the computer device may convert the lane sub-image into a grayscale image and perform a density evaluation according to the grayscale values corresponding to each initial strip region in each lane sub-image to obtain a density score for quantifying the darkness of the color corresponding to each initial strip region.

[0106] In some embodiments, the density score may be obtained by quantifying the characteristics of the darkness of the color. The characteristics of the darkness of the color may include the concentration of pixel value distribution, the complexity of pixel value distribution, and the consistency of pixel value brightness. Among them, the concentration of pixel value distribution can be used to reflect the aggregation degree of the main pixel values of the initial strip region, the complexity of pixel value distribution can be used to reflect the richness of details and the amount of information in the initial strip region, and the consistency of pixel value brightness can be used to reflect the fluctuation range of pixel values in the initial strip region.

[0107] For example, the quantification of the concentration of pixel value distribution can be represented by kurtosis, the quantification of the complexity of pixel value distribution can be represented by color entropy, and the consistency of pixel value brightness can be represented by brightness uniformity. The computer device may determine the density score corresponding to each initial strip region according to the kurtosis, color entropy, and brightness uniformity corresponding to each initial strip region.

[0108] Step 208: The computer device filters out the initial strip regions with density scores greater than the density threshold as the first strip regions.

[0109] Since the characteristic of the shade of color is quantified as the density fraction, the first strip region may include an initial strip region where the shade of color is greater than a specific degree, that is, a strip region where the shade of color is greater than a specific degree and has a certain boundary. Among them, the density threshold can be set as needed according to the actual situation.

[0110] The computer device can compare the density fractions corresponding to each initial strip region with a preset density threshold, and use the initial strip regions with density fractions greater than the density threshold as the first strip regions. Optionally, the computer device can also use the initial strip regions with density fractions equal to the density threshold as the first strip regions.

[0111] It can be understood that by setting the density threshold and screening out the initial strip regions with density fractions greater than or equal to the density threshold, the initial strip regions with lighter and more diffuse colors can be excluded, so as to ensure that the first strip regions selected by the computer device more conform to the "significant" characteristics of the strip regions in the swim lane sub-images, thereby improving the accuracy and reliability of the analysis of the first strip regions in the subsequent steps.

[0112] Exemplarily, let the density threshold be 0.7. As Figure 5 shown, the computer device can determine that the density fractions of the five initial strip regions in the control swim lane sub-image from top to bottom are 0.9, 0.7, 0.7, 0.6, 0.9, the density fraction of the initial strip region in the swim lane G sub-image is 0.85, and the density fractions of the two initial strip regions in the swim lane L sub-image from top to bottom are 0.65 and 0.85. Therefore, the computer device can screen out the initial strip regions 401, initial strip region 402, initial strip region 403, initial strip region 405, initial strip region 406, initial strip region 407, and initial strip region 408 with density fractions greater than or equal to the density threshold 0.7 as the first strip regions, while the initial strip region 404 cannot be used as the first strip region.

[0113] Step 210, the computer device determines the overlapping degree between each first strip region in each other swim lane sub-image and each first strip region in the control swim lane sub-image.

[0114] The overlapping degree may refer to the degree of coincidence of the positions of the first strip regions in other swim lane sub-images and each first strip region in the control swim lane sub-image in the image. When the overlapping degree corresponding to the first strip region is greater than the preset overlapping threshold, it indicates that the first strip region corresponds to a first strip region in the control swim lane sub-image.

[0115] The computer device can determine the image area and image position of the first strip area based on the image coordinates of the first strip area in the corresponding lane sub-image, and then determine the overlap degree between each first strip area in each other lane sub-image and each first strip area in the reference lane sub-image according to the image area and image position corresponding to each first strip area.

[0116] In some embodiments, the overlap degree may include the ratio of the intersection area and the union area between each first strip area in each other lane sub-image and each first strip area in the reference lane sub-image, that is, the Intersection over Union (IoU for short). The IoU value ranges from 0 to 1. The closer the IoU value is to 1, the higher the overlap degree between each first strip area in each other lane sub-image and each first strip area in the reference lane sub-image.

[0117] The intersection area may refer to the area of the region where the positions of the first strip area in the other lane sub-image and the first strip area in the reference lane sub-image overlap in the image. The union area may refer to the total area covered by the positions of the first strip area in the other lane sub-image and the first strip area in the reference lane sub-image in the image.

[0118] In some embodiments, the computer device can calculate the intersection area and the union area between each first strip area in each other lane sub-image and each first strip area in the reference lane sub-image according to the image coordinates of each first strip area in each other lane sub-image and the image coordinates of each first strip area in the reference lane sub-image, and calculate the IoU according to the intersection area and the union area corresponding to each first strip area in each other lane sub-image.

[0119] Exemplarily, taking the strip area O of the first other lane sub-image as an example, the first other lane sub-image is any other lane sub-image, and the strip area O is any first strip area in the first other lane sub-image. The computer device can determine that the intersection area between the strip area O and the strip area V is B1 and the union area between the strip area O and the strip area V is B2 according to the image coordinates of the strip area O of the first other lane sub-image and the image coordinates of the strip area V of the reference lane sub-image, where the strip area V is any first strip area in the reference lane sub-image. Therefore, the IoU between the strip area O of the first other lane sub-image and the strip area V in the reference lane sub-image is:

[0120]

[0121] where C 1,gThe IoU of the band region O of the first other lane sub-image and the g-th band region V in the control lane sub-image, B 1,g The intersection area of the band region O and the g-th band region V in the control lane sub-image, B 2,g The union area of the band region O and the g-th band region V in the control lane sub-image, and G is the total number of the first band regions in the control lane sub-image.

[0122] By calculating the IoU, the computer device represents the overlapping degree between each first band region in each other lane sub-image and each first band region in the control lane sub-image. It can not only achieve an accurate quantitative evaluation of the overlapping degree of the first band region, facilitate subsequent further comparison and analysis, but also reduce the error of manual subjective judgment, and contribute to improving the accuracy and reliability of the analysis.

[0123] In some embodiments, the computer device can scan multiple first band regions in the control lane sub-image. The computer device can determine whether a first band region in the control lane sub-image overlaps with each first band region in each other lane sub-image within its height range. If not, it selects the next first band region in the control lane sub-image in the order from top to bottom and repeats the above steps. If there is an overlap, it calculates the overlapping degree between this first band region and each first band region in each other lane sub-image.

[0124] It can be understood that the computer device sequentially determines and calculates the overlapping degree between a first band region in the control lane sub-image and each first band region in each other lane sub-image in the order from top to bottom. By comparing one by one, the possibility of omission or incorrect matching can be reduced, thereby improving the accuracy of the analysis.

[0125] Exemplarily, such as Figure 6As shown, the computer device can sequentially scan the first strip regions 601, 602, 603, and 604 in the lane sub-image from top to bottom. The dashed lines indicate whether the computer device determines that a first strip region in the comparison lane sub-image overlaps with each first strip region in each other lane sub-image. After the computer device determines that there is no overlapping region between the first strip region 601 and the first strip region 602, it can then scan the first strip region 603, determine that the first strip region 603 overlaps with the first strip region 606 in the lane L sub-image, and calculate the degree of overlap between the first strip region 603 and the first strip region 606. Then it scans the first strip region 604, determines that the first strip region 604 overlaps not only with the first strip region 605 in the lane G sub-image but also with the first strip region 607 in the lane L sub-image. Therefore, it calculates the degree of overlap between the first strip region 604 and the first strip region 605, and the degree of overlap between the first strip region 604 and the first strip region 607.

[0126] In some other embodiments, during the process of scanning multiple first strip regions in the comparison lane sub-image, when the computer device detects that the comparison lane sub-image has at least five first strip regions, the fifth first strip region is located in a specific region, and the computer device does not identify a first strip region in the lane sub-image corresponding to the light chain lane, the computer device can, according to the preset interpretation rules, mark the lane sub-image corresponding to the light chain lane, obtain the corresponding interpretation result, and send a positive warning instruction corresponding to the interpretation result.

[0127] It can be understood that when the comparison lane sub-image has at least five first strip regions and the fifth first strip region is located in a specific region, it indicates that after the various immunoglobulins in the comparison lane migrate according to charge and molecular weight, they conform to one of the multiple characteristics shown in the comparison lane sub-image for some special diseases. And the computer device does not identify a first strip region in the lane sub-image corresponding to the light chain lane, indicating that there may be problems such as insufficient light chain expression or light chain deletion in the target sample. Therefore, the computer device can, according to the preset interpretation rules, determine that the interpretation result of the lane sub-image corresponding to the light chain lane is in the state of "control valid but target missing", and then trigger the corresponding positive warning instruction to remind the detection personnel that the target sample may conform to the early characteristics of some special diseases, and it is necessary to re-do the target sample or use other detection methods to further confirm, so as to improve the accuracy of hierarchical recognition of the target electrophoresis image.

[0128] Step 212: The computer device determines, according to the overlapping degree corresponding to each first band region in each other lane sub-image, the first band regions that exist in both the other lane sub-images and the control lane sub-image as the second band regions.

[0129] The computer device may determine whether the overlapping degree corresponding to each first band region in each other lane sub-image is greater than a preset overlapping threshold. If the overlapping degree corresponding to a certain first band region in an other lane sub-image is greater than the preset overlapping threshold, it may be determined that this first band region exists in both the other lane sub-image and the control lane sub-image, that is, the first band region in the other lane sub-image corresponds to a first band region in the control lane sub-image. Therefore, this first band region may be used as the second band region.

[0130] In some embodiments, the overlapping degree may include IoU. The computer device may determine whether the IoU corresponding to each first band region in each other lane sub-image is greater than a preset IoU threshold, and when the IoU corresponding to each first band region in the other lane sub-images is greater than the preset IoU threshold, determine the first band regions that exist in both the other lane sub-images and the control lane sub-image as the second band regions.

[0131] In some embodiments, after the computer device determines the second band regions that exist in both the other lane sub-images and the control lane sub-image, it may mark the other lane sub-images to indicate that there is a first band region in the other lane sub-image corresponding to the first band region in the control lane sub-image, that is, the other lane sub-image has a second band region.

[0132] Exemplarily, as Figure 1 shown, when the computer device determines that the first band region 603 of the control lane sub-image overlaps with the first band region 606 in the lane L sub-image, the first band region 604 of the control lane sub-image overlaps with the first band region 605 in the lane G sub-image, and also overlaps with the first band region 607 in the lane L sub-image, it may mark the lane G sub-image and the lane L sub-image. For example, the lane G sub-image and the lane L sub-image may be marked as positive.

[0133] In some embodiments, the computer device may complete the hierarchical recognition of the target electrophoresis image according to the marks of each other lane sub-image and based on a preset interpretation rule. For example, the computer device determines that both the lane G sub-image and the lane L sub-image have second band regions. Therefore, the computer device may mark the lane G sub-image and the lane L sub-image as positive, and according to the preset interpretation rule, output the interpretation result corresponding to both the lane G sub-image and the lane L sub-image having positive marks.

[0134] After the computer device recognizes the second band regions of each other lane sub-image, it automatically marks the other lane sub-images, and outputs corresponding interpretation results based on the marks of each other lane sub-image according to the preset interpretation rules. This can not only assist the detection personnel in interpreting the target electrophoresis image, improve the accuracy and efficiency of interpreting the target electrophoresis image, reduce the time and effort required for manual interpretation, but also improve the objectivity and consistency of the interpretation results, and avoid misinterpretation and interpretation errors of different detection personnel.

[0135] Step 214, the computer device determines the positive degree corresponding to the target electrophoresis image according to the density scores corresponding to each second band region.

[0136] It can be understood that since the depth of color of the second band region is positively correlated with the positive degree of the second band region, the density score can also be used to reflect the positive degree of the second band region. That is to say, the deeper the depth of color of the second band region, the higher the density score of the second band region, and the higher the positive degree of the second band region.

[0137] In some embodiments, the computer device can divide the density scores into multiple density intervals by setting multiple degree thresholds, so that the multiple density intervals correspond to different positive degrees.

[0138] The positive degree can include strong positive, positive, and weak positive. The density intervals can include a first density interval, a second density interval, and a third density interval, where the first density interval corresponds to strong positive, the second density interval corresponds to positive, and the third density interval corresponds to weak positive. The density thresholds can be set as needed according to the actual situation.

[0139] The computer device can compare the density scores corresponding to each second band region with the preset multiple density intervals, so as to determine the target density interval corresponding to each second band region, and determine the positive degree corresponding to each second band region according to the target density interval corresponding to each second band region.

[0140] In some embodiments, the computer device can determine the positive degree corresponding to the corresponding other lane sub-image according to the positive degree corresponding to each second band region. For example, if the positive degree of the second band region is weak positive, then the positive degree of the other lane sub-image corresponding to the second band region is weak positive.

[0141] Another lane sub-image may include one or more second band regions. In the case where another lane sub-image includes multiple second band regions, the computer device may select the second band region with the largest density fraction among the multiple second band regions as the third band region, and determine the positivity degree of the other lane sub-image corresponding to the third band region according to the positivity degree corresponding to the third band region.

[0142] Exemplarily, as Figure 6 shown, the computer device may determine that the first band region 603 in the control lane sub-image overlaps with the first band region 606 in the lane L sub-image, and the first band region 604 in the control lane sub-image overlaps with the first band region 605 in the lane G sub-image and also overlaps with the first band region 607 in the lane L sub-image. Therefore, the computer device may determine that the first band region 605 in the lane G sub-image is the second band region 605, and the first band regions 606 and 607 in the lane L sub-image are the second band regions 606 and 607 respectively. And since the density fraction of the second band region 606 in the lane L sub-image is 0.65 and the density fraction of the second band region 607 is 0.85, and the density fraction 0.85 of the second band region 607 is greater than the density fraction 0.65 of the second band region 606, the computer device may determine that the positivity degree corresponding to the lane L sub-image is strongly positive according to the density fraction 0.85 of the second band region 607.

[0143] In some embodiments, the computer device may determine the positivity degree of the target electrophoresis image according to the positivity degrees corresponding to each other lane sub-image, and send a positivity warning instruction corresponding to the positivity degree of the target electrophoresis image.

[0144] The positivity degree of the target electrophoresis image may be determined according to the positivity degrees corresponding to multiple other lane sub-images. For example, in the case where the positivity degrees corresponding to multiple other lane sub-images are weakly positive, the positivity degree of the target electrophoresis image is weakly positive.

[0145] The positivity warning instruction includes a warning instruction corresponding to strongly positive, a warning instruction corresponding to positive, and a warning instruction corresponding to weakly positive. For example, in the case where the positivity degree of the target electrophoresis image is weakly positive, the computer device may send a warning instruction corresponding to weakly positive to implement a warning for the weakly positive positivity degree of the target electrophoresis image.

[0146] It should be noted that when the positive degree of the target electrophoresis image is weakly positive, it indicates that the concentration degree of the immunoglobulin content in each second band area of the target electrophoresis image is not high and the distribution of the immunoglobulin is uneven. In this case, the computer device needs to issue a weakly positive warning instruction to achieve weakly positive warning, so that the detection personnel can review the target sample in time, avoid misjudgment or missed detection, and improve the hierarchical recognition efficiency of the target electrophoresis image corresponding to the target sample.

[0147] Exemplarily, as Figure 1 shown, when the computer device determines that the positive degrees of the lane G sub-image and the lane L sub-image are strongly positive, the computer device can determine that the positive degree of the target electrophoresis image is strongly positive and send a positive warning instruction corresponding to the strongly positive degree.

[0148] In the embodiment of the present application, the computer device identifies the band areas included in each lane sub-image to obtain each initial band area of each lane sub-image, and then performs a density evaluation on each initial band area of each lane sub-image to determine the first band area with a density score higher than the density threshold, and accurately identifies the band area with a higher density that exists in other lane sub-images and the control lane sub-image according to the overlap degree between each first band area in other lane sub-images and each first band area in the control lane sub-image, improving the accuracy of the computer device in identifying the band areas in the electrophoresis image and the accuracy of hierarchical recognition of the electrophoresis image.

[0149] Figure 7 The flowchart for determining each initial band area included in the first lane sub-image provided by the embodiment of the present application. As Figure 7 shown, the step that the computer device identifies the band areas included in each lane sub-image to obtain one or more initial band areas in each lane sub-image may include the following steps:

[0150] Step 701, the computer device searches each band area included in the first lane sub-image to obtain the first band coordinate set corresponding to the first lane sub-image.

[0151] The first lane sub-image is any lane sub-image, and the first band coordinate set includes the image vertical coordinates of each band area in the first lane sub-image. The image vertical coordinates of each band area may include the image vertical coordinates corresponding to the top edge and the bottom edge of each band area respectively.

[0152] The computer device can search and determine the initial band area in each band area included in the first lane sub-image, and obtain the image vertical coordinates corresponding to each initial band area included in the first lane sub-image, forming the first band coordinate set corresponding to the first lane sub-image.

[0153] In some embodiments, the computer device can identify each band region included in the first lane sub-image based on the gradient change of the pixel values corresponding to each pixel point in the first lane sub-image, and determine the initial band regions included in the first lane sub-image according to the lengths of the respective band regions in the vertical direction, thereby obtaining the image ordinates corresponding to the respective initial band regions.

[0154] Figure 8 This is a flowchart for determining the first band coordinate set of the first lane sub-image provided by the embodiments of the present application. As Figure 8 shown, step 701 may include the following steps:

[0155] Step 802, the computer device converts the first lane sub-image into a grayscale image.

[0156] The grayscale image may include the pixel values corresponding to each pixel point in the first lane sub-image, and each pixel value is between 0 and 255. By converting the first lane sub-image into a grayscale image, the pixel value data of the first lane sub-image can be simplified, thereby enhancing the image features of the first lane sub-image, and further improving the recognition ability of the computer device for each band region in the first lane sub-image, and improving the recognition efficiency and accuracy of the computer device for the band region.

[0157] The pixel value may include a red channel pixel value, a blue channel pixel value, and a green channel pixel value.

[0158] The computer device may perform weighted averaging on the red channel pixel value, the blue channel pixel value, and the green channel pixel value corresponding to each pixel point in the first lane sub-image according to a certain weight to obtain the grayscale value of the corresponding pixel point, thereby converting the first lane sub-image into a grayscale image.

[0159] Step 804, the computer device accumulates the grayscale values of each pixel point in the grayscale image along the horizontal direction and calculates the average value to generate an index pixel array.

[0160] The index pixel array may refer to a one-dimensional array used to represent the average value of the accumulated grayscale values of each row in the horizontal direction of the grayscale image. The index pixel array may include N elements, and the N elements correspond one-to-one to the N rows of pixels in the grayscale image. The value of the Pth element is the average value corresponding to the Pth row of pixels in the grayscale image. N and P are positive integers, and P is less than or equal to N.

[0161] From Figure 1As can be seen, the pixel values of each strip region in each lane sub-image are relatively stable in the horizontal direction without obvious fluctuations, while there are significant differences and variations in the vertical direction. Therefore, during the process of generating the index pixel array, by statistically calculating the arithmetic mean of the pixel values of each row in the horizontal direction, the differences and variations in the arithmetic means between rows in the vertical direction become more obvious, so as to better reflect the pixel value differences in the strip region, which is conducive to quickly determining the boundary of the strip region.

[0162] In some embodiments, the computer device may statistically calculate the arithmetic mean of the grayscale values of each pixel point in the P-th row of the grayscale image in the horizontal direction, and construct an index pixel array based on the arithmetic means corresponding to the grayscale values of N rows.

[0163] Exemplarily, the computer device may convert the first lane sub-image L1 into a grayscale image L1 gray ∈R 1*N*M , where R represents the set of real numbers, N is the number of rows of the grayscale image L1 gray and M is the number of columns of the grayscale image L1 gray . The grayscale value of the pixel point at the n-th row and the m-th column in the grayscale image L1 gray can be represented by L1 gray (n, m), where n = 1, 2, 3..., N and m = 1, 2, 3..., M. The computer device may calculate the arithmetic mean of the grayscale values of each pixel point in the grayscale image in the horizontal direction to generate an index pixel array:

[0164]

[0165] where A(n) is the arithmetic mean of the grayscale values of M pixel points in the n-th row of the grayscale image L1 gray , that is, the n-th element in the index pixel array A, and the index pixel array A = {A(1), A(2),..., A(N)}.

[0166] Step 806, the computer device generates a gradient change curve according to the index pixel array.

[0167] The gradient change curve can be used to reflect the change trend of the arithmetic mean of the grayscale values of each row in the grayscale image in the vertical direction, and the gradient change curve can be generated by each element in the gradient change array. The gradient change array may refer to a one-dimensional array composed of the differences between every two adjacent elements in the index pixel array, and the gradient change array can be used to represent the change rate and direction of the arithmetic mean of the grayscale values of each row in the grayscale image in the vertical direction.

[0168] In some embodiments, the computer device may use the absolute value of the difference between every two adjacent elements among the N elements in the index pixel array as the gradient value, and construct a gradient change array based on the N - 1 gradient values, so as to generate a gradient change curve from the N - 1 gradient values in the gradient change array.

[0169] Exemplarily, based on the index pixel array A = {A(1), A(2),..., A(N)}, the computer device calculates the absolute value of the difference between every two adjacent elements in the index pixel array as the gradient value:

[0170] G(x) = |A(x + 1) - A(x)| Equation (3);

[0171] where G(x) is the absolute value of the difference between the (x + 1)-th element A(x + 1) and the x-th element A(x) in the index pixel array A, x = 1, 2, 3..., N - 1, and the N - 1 G(x) form the gradient change array G = {G(1), G(2),..., G(N - 1)}. The x-th gradient value G(x) in the gradient change array G can represent the gradient value of the x-th element A(x) in the index pixel array A.

[0172] In some embodiments, the computer device may use the indices of the N - 1 gradient values in the gradient change array G as the abscissa, and the corresponding numerical values of the gradient values in the gradient change array G as the ordinate to form N - 1 coordinate points, and generate a curve that can smoothly connect the above N - 1 coordinate points through methods such as interpolation or fitting to form a gradient change curve.

[0173] By forming the gradient change curve, not only can the trend of the gradient value changing with the number of rows of the first swim lane sub - image be intuitively displayed, but also the data characteristics and abnormal points of the index pixel array corresponding to the first swim lane sub - image can be visualized, thereby improving the recognition efficiency and accuracy.

[0174] Step 808, the computer device determines the target element where the gradient mutation occurs according to the gradient change curve, and obtains the image ordinate corresponding to the target element.

[0175] The gradient mutation may refer to the case where the curve value of the gradient change curve is greater than the set gradient threshold, that is, the absolute value of the difference between two adjacent elements in the index pixel array is greater than the set gradient threshold. The target element may include the elements in the index pixel array whose gradient values are greater than the set gradient threshold.

[0176] Since the gradient mutation can reflect an obvious difference between the arithmetic means of the gray values of two adjacent rows in the first lane sub-image, and in the first lane sub-image, the gray value usually changes significantly only at the boundary of the band area, the target elements with gradient mutations in the index pixel array can be determined through the gradient change curve, and the image ordinate corresponding to the target element can be obtained, so as to determine the boundary of the band area corresponding to the target element.

[0177] The computer device can compare each gradient value in the gradient change array with a set gradient threshold, and take the elements with gradient values greater than the set gradient threshold as target elements, and then determine the image ordinate corresponding to the target element according to the position of the target element in the index pixel array.

[0178] Exemplarily, the computer device can set a gradient threshold T1, and screen the elements with gradient values greater than the gradient threshold T1 as target elements according to the index pixel array A and the gradient change array G, so as to construct a set of image ordinates of gradient mutations:

[0179] F = {x|G(x)>T1, x = 1, 2, 3..., N - 1} Equation (4);

[0180] Where F is the set of image ordinates corresponding to the target elements with gradient mutations.

[0181] Since the x-th gradient value G(x) in the gradient change array G can represent the gradient value of the x-th element A(x) in the index pixel array A, that is, the x-th row of the grayscale image L1 gray Therefore, when the gradient value G(x) is greater than the gradient threshold T1, the computer device can determine that a gradient mutation occurs at the x-th row of the grayscale image L1 gray and construct the set of image ordinates F according to x corresponding to G(x).

[0182] In addition, the set of image ordinates F also includes elements with image ordinates of 0 and image ordinates of N to indicate that gradient mutations also occur in the first row elements and the last row elements of the grayscale image L1 gray It can be understood that if the set of image ordinates F does not include elements with an image ordinate of 0, in the case where the gradient mutation occurs for the first time, such as G(5), the computer device will not be able to consider the elements before the 5th row, and the same is true for elements with an image ordinate of N.

[0183] Step 810, the computer device calculates the line segment distance corresponding to every two adjacent target elements according to the image ordinates corresponding to each target element.

[0184] The line segment distance can refer to the Euclidean distance between the image ordinates corresponding to two adjacent target elements in the set of image ordinates.

[0185] It can be understood that, through the image vertical coordinates corresponding to each target element in the set of image vertical coordinates, the lanes in the first lane sub-image can be divided into multiple intervals, and the multiple intervals include the intervals corresponding to the band regions and the intervals corresponding to the empty regions between two adjacent band regions.

[0186] The computer device can construct line segments based on two adjacent target elements in the set of image vertical coordinates to form a set of line segments, and calculate the line segment distances corresponding to each two adjacent target elements according to the image vertical coordinates corresponding to each two adjacent target elements, so as to obtain a set of line segment distances.

[0187] Exemplarily, the computer device can construct line segments based on each two adjacent target elements in the set of image vertical coordinates F to form a set of line segments:

[0188] I={(F(i),F(i + 1)), i=1, 2, 3,..., size(F)-1} Equation (5);

[0189] wherein, F(i) represents the i-th image vertical coordinate in the set of image vertical coordinates F, F(i + 1) represents the (i + 1)-th image vertical coordinate in the set of image vertical coordinates F, F(i)=0, 1,..., N - 1, size(F) represents the total number of image vertical coordinates in the set of image vertical coordinates F, I represents the set of line segments constructed by the computer device according to the set of image vertical coordinates F, and the set of line segments I can include size(F)-1 line segments;

[0190] The computer device can determine the line segment distance by calculating the Euclidean distance of the line segment (F(i), F(i + 1)) in the set of line segments I:

[0191] d i =F(i + 1)-F(i), i=1, 2, 3,..., size(F)-1 Equation (6);

[0192] wherein, d i represents the Euclidean distance of the line segment (F(i), F(i + 1)). The computer device can construct a set of line segment distances D={d1, d2,.., d size(F)-1} according to the line segment distances corresponding to each line segment.

[0193] Step 812, the computer device filters out the target elements whose corresponding line segment distances are less than the distance threshold to obtain the first strip coordinate set of the first lane sub-image.

[0194] The distance threshold can include the length in the vertical direction corresponding to the "narrow" characteristic that the strip region should have.

[0195] It can be understood that the strip region to be recognized usually has the characteristic of "narrowness", that is, the length of the strip region in the vertical direction is small. Therefore, when the line segment distance between two adjacent target elements is less than the distance threshold, the interval formed by the image vertical coordinates corresponding to the two adjacent target elements conforms to the "narrow" characteristic of the strip region, so this interval can be considered as the strip region to be recognized, that is, the initial strip region.

[0196] The computer device can compare the line segment distances corresponding to each line segment in the line segment distance set with the distance threshold, and regard the line segments with line segment distances less than the distance threshold as target line segments. Then, by obtaining the image vertical coordinates of two adjacent target elements corresponding to each target line segment, the first strip coordinate set of the first sub-image of the lane is obtained.

[0197] Exemplarily, the computer device can screen the line segment distance set D according to the distance threshold T2, so as to determine the image vertical coordinates of two target elements of the line segments with line segment distances less than the distance threshold, and form the first strip coordinate set:

[0198] Z1 = {(F(i), F(i + 1))|D(i) < T2, i = 1, 2, 3,..., size(F) - 1} Equation (7);

[0199] Where, D(i) is the i-th line segment distance in the line segment distance set D, F(i) is the i-th image vertical coordinate in the image vertical coordinate set F, F(i + 1) is the (i + 1)-th image vertical coordinate in the image vertical coordinate set F, and Z1 is the first strip coordinate set.

[0200] The computer device accurately identifies the boundaries of each strip region by converting the first sub-image of the lane into a grayscale image and according to the gradient mutation of the grayscale value gradient of the grayscale image, improves the recognition ability of the boundaries of the strip regions, realizes the precise positioning of the boundaries of the strip regions. At the same time, screening the line segments that conform to the "narrow" characteristic according to the line segment distances between multiple boundary lines can avoid misidentifying non-strip regions as strip regions, and improves the accuracy and reliability of the computer device in identifying strip regions.

[0201] Step 703, the computer device identifies the foreground region and the background region of the first sub-image of the lane, and fills the background region with the target pixel value to obtain the second sub-image of the lane.

[0202] The foreground region may refer to the main object or region of interest in the first lane sub-image, and the foreground region may include the band region in the first lane sub-image. The background region may refer to other irrelevant regions in the first lane sub-image except the main object or region of interest, and the background region may include the regions in the first lane sub-image that are irrelevant to the band region. The target pixel value may include the pixel value corresponding to white, the pixel value corresponding to black, or the pixel value corresponding to the main color of the background region.

[0203] It can be understood that in the process of determining the first set of band coordinates of the first lane sub-image according to the steps described in steps 802 to 812, when the blurring degree of the boundary of the band region is relatively high, due to the slow and unobvious gradient change, that is, the pixel values of the background region will affect the recognition of the gradient mutation of the foreground region, it is difficult to define the boundary of the band region with a high blurring degree through the gradient mutation, thereby affecting the accuracy of the first set of band coordinates. Therefore, it is necessary to fill the background region with a unified target pixel value to enhance the contrast between the foreground region and the background region, make the boundary of the band region clearer, and thus improve the accuracy of the initial band region recognition of the first lane sub-image.

[0204] The computer device can identify the foreground region and the background region of the first lane sub-image according to the foreground-background image segmentation algorithm, obtain the pixel points corresponding to the foreground region and the pixel points corresponding to the background region, and correct the pixel values of the pixel points corresponding to the background region to the target pixel value, and retain the pixel values of the pixel points corresponding to the foreground region, so as to obtain the second lane sub-image.

[0205] The foreground-background image segmentation algorithm may include methods such as deep learning algorithms, watershed algorithms, or region growing algorithms. Since the way the computer device can identify the foreground region and the background region of the first lane sub-image according to the foreground-background image segmentation algorithm is similar to the prior art, it will not be elaborated here.

[0206] In some embodiments, after the computer device identifies the foreground region and the background region of the first lane sub-image, it can fill the background region with the target pixel value and strengthen the features of the foreground region, thereby forming the second lane sub-image. For example, the pixel values corresponding to the foreground region can be strengthened to make the difference between the target pixel value corresponding to the background region and the strengthened pixel values corresponding to the foreground region more obvious.

[0207] Step 705, the computer device searches for each band region included in the second lane sub-image to obtain the second set of band coordinates corresponding to the second lane sub-image.

[0208] The second set of band coordinates includes the image vertical coordinates of each band region in the second lane sub-image.

[0209] The method for determining the set of coordinates of the second band corresponding to the second lane sub-image is similar to the method for determining the set of coordinates of the first band corresponding to the first lane sub-image described in steps 802 to 812 above, and will not be elaborated here.

[0210] Optionally, in steps 802 to 812 above, the preset parameters such as the gradient threshold and the distance threshold used to determine the set of coordinates of the second band corresponding to the second lane sub-image may be the same as or different from the preset parameters such as the gradient threshold and the distance threshold used to determine the set of coordinates of the first band corresponding to the first lane sub-image, and no specific limitation is made here.

[0211] Step 707, the computer device determines each initial band region included in the first lane sub-image according to the set of coordinates of the first band and the set of coordinates of the second band.

[0212] It can be understood that combining the set of coordinates of the first band and the set of coordinates of the second band can comprehensively consider the advantages of both, so as to ensure the accuracy of the band region determined according to the set of coordinates of the first band and the set of coordinates of the second band.

[0213] The computer device can determine each band region on the first lane sub-image according to the set of coordinates of the first band, determine each band region on the first lane sub-image according to the set of coordinates of the second band, and use each band region on the first lane sub-image and each band region on the second lane sub-image as each initial band region included in the first lane sub-image.

[0214] In some embodiments, since there are image vertical coordinates with similar values in the set of coordinates of the first band and the set of coordinates of the second band, the computer device can fuse two image vertical coordinates with similar values in the set of coordinates of the first band and the set of coordinates of the second band into one image vertical coordinate, or remove the image vertical coordinates with similar values in one of the set of coordinates of the first band and the set of coordinates of the second band to avoid redundant calculations or data interference caused by subsequent processing flows of duplicate or similar image vertical coordinates, thereby improving the processing efficiency of the computer device. For example, the average value of two image vertical coordinates with similar values can be used as the new image vertical coordinate.

[0215] In some embodiments, considering that the boundaries of most strip regions in the lane sub-image are usually relatively blurred, although the strip regions of the first strip coordinate set and the second strip coordinate set determined through steps 701 to 705 conform to the "narrow" characteristic, they do not cover the blurred parts of the boundaries of the strip regions. Therefore, it is necessary to appropriately expand the strip regions of the first strip coordinate set and the second strip coordinate set to further improve the accuracy of recognizing each initial strip region included in the first lane sub-image.

[0216] The computer device can expand the image vertical coordinates of the first strip coordinate set and the second strip coordinate set according to a preset expansion coefficient, so as to determine each initial strip region included in the first lane sub-image.

[0217] In some embodiments, the first strip coordinate set and the second strip coordinate set include multiple strip regions. The computer device can expand the image vertical coordinates corresponding to the multiple strip regions according to the expansion coefficient, so as to form the corresponding initial strip regions. As Figure 9 shown, step 707 may include the following steps:

[0218] Step 901, the computer device obtains the first image vertical coordinate and the second image vertical coordinate corresponding to each strip region in the first lane sub-image according to the first strip coordinate set and the second strip coordinate set.

[0219] Each strip region in the first lane sub-image may include the strip region corresponding to the first strip coordinate set and the strip region corresponding to the second strip coordinate set. The first image vertical coordinate may include the image vertical coordinate corresponding to the top edge of each strip region in the first lane sub-image, and the second image vertical coordinate may include the image vertical coordinate corresponding to the bottom edge of each strip region in the first lane sub-image.

[0220] The computer device can identify each strip region on the first lane sub-image according to the first strip coordinate set and the second strip coordinate set, and obtain the first image vertical coordinate corresponding to the top edge and the second image vertical coordinate corresponding to the bottom edge in each strip region according to the image vertical coordinates corresponding to each strip region on the first lane sub-image.

[0221] Step 903, the computer device expands the positions of each strip region in the first lane sub-image according to the expansion coefficient, the first image vertical coordinate and the second image vertical coordinate corresponding to each strip region in the first lane sub-image, and obtains each initial strip region included in the first lane image.

[0222] The position expansion may include expanding the regions of each strip region in the first lane sub-image in the vertical direction. The expansion coefficient may refer to a parameter used to control the expansion degree of the strip region, and the expansion coefficient can be set as needed according to the actual situation.

[0223] The computer device can determine the first target ordinate corresponding to the top edge of each strip region according to the sum value between the first image ordinate corresponding to each strip region in the first lane sub-image and the expansion coefficient; the computer device can determine the second target ordinate corresponding to the bottom edge of each strip region according to the difference value between the second image ordinate corresponding to each strip region in the first lane sub-image and the expansion coefficient.

[0224] The computer device can obtain each initial strip region included in the first lane sub-image according to the first target ordinate corresponding to each strip region in the first lane sub-image, the preset first target abscissa, the second target ordinate corresponding to each strip region, and the preset second target abscissa.

[0225] It can be understood that since the pixel values of each strip region in the first lane sub-image have relatively consistent characteristics in the horizontal direction, the computer device can directly determine the first target abscissa and the second target abscissa corresponding to both sides of each strip region through the position information and width information of the lane in the first lane sub-image.

[0226] In some embodiments, the computer device can construct a coordinate frame corresponding to each strip region according to the first target ordinate, the first target abscissa, the second target ordinate, and the second target abscissa corresponding to each strip region in the first lane sub-image, and represent each initial strip region included in the first lane sub-image through the coordinate frame.

[0227] Exemplarily, the computer device can determine the upper left vertex coordinates of the coordinate frame corresponding to the strip region according to the expansion coefficient ∈, the first target ordinate, and the preset first target abscissa, and determine the lower right vertex coordinates of the coordinate frame corresponding to the strip region according to the expansion coefficient ∈, the second target ordinate, and the preset second target abscissa. The coordinate frame corresponding to the strip region is expressed as:

[0228] B y =[(a,q y2 +∈),(c,q y1 -∈)] Formula (8);

[0229] Wherein, ∈ represents the expansion coefficient, q y1 represents the first target ordinate of the y-th strip region in the first lane sub-image, q y2 represents the second target ordinate of the y-th strip region in the first lane sub-image, B yIt represents the coordinate box corresponding to the y-th band area of the first lane sub-image, where y = 1, 2, 3,..., Y, and Y represents the total number of initial band areas in the first lane sub-image, and Y is less than or equal to the sum of the number of band areas corresponding to the first band coordinate set and the number of band areas corresponding to the second band coordinate set. a represents the preset first target abscissa, and c represents the preset second target abscissa.

[0230] In some embodiments, a is usually preset to 0, and c is usually preset to M.

[0231] The computer device enlarges the image ordinates of the first band coordinate set and the second band coordinate set by a preset enlargement factor, so as to ensure the integrity of the band area through appropriate enlargement, which helps to cover more pixels that may belong to the band area, and further reduces the loss of boundary information caused by the division of the band area, and helps to ensure the accurate recognition of each initial band area contained in the first lane image.

[0232] In the embodiments of the present application, the computer device extracts the first band coordinate set of the first lane sub-image, then extracts the second band coordinate set of the second lane sub-image formed after background filling, and determines each initial band area contained in the first lane sub-image according to the first band coordinate set and the second band coordinate set, so as to improve the accuracy of determining the boundaries of each band area according to the first band coordinate set and the second band coordinate set, and further improve the accuracy of recognizing each initial band area contained in the first lane sub-image.

[0233] In some embodiments, after the computer device determines the boundary of the band area, it also needs to evaluate the density of the band area, so as to quantify the depth of the color of the band area into a density score, in order to determine the first band area from each initial band area contained in the first lane sub-image. Figure 10 This is the flowchart for determining the density score corresponding to the initial band area provided by the embodiments of the present application. As Figure 10 shown, in step, the computer device performs density evaluation on each initial band area to obtain the density score corresponding to each initial band area, which may include the following steps:

[0234] Step 1002, the computer device calculates the color histogram corresponding to the first initial band area.

[0235] The color histogram can be used to describe the distribution of the overall pixel values of the first initial band area, and the first initial band area can be any initial band area.

[0236] The computer device can calculate the color histogram corresponding to the first initial band area according to the pixel values and the occurrence frequencies of the pixel values of each pixel point included in the first initial band area.

[0237] Exemplarily, the computer device can determine that the width of the first initial strip area is W and the height is H according to the image ordinate and image abscissa corresponding to the first initial strip area. Therefore, the first initial strip area includes W×H pixel points. The computer device can calculate the occurrence frequency of the pixel value according to the pixel values of each pixel point included in the first initial strip area and the total number of occurrences of the pixel value as follows:

[0238]

[0239] where T(w, h) represents the pixel value at the w-th row and h-th column of the first initial strip area, w = 1, 2, 3,..., W, h = 1, 2, 3,..., H, count(T(w, h)) represents the total number of occurrences of the pixel value at the w-th row and h-th column of the first initial strip area, and F hist (T(w, h)) represents the frequency of the pixel value at the w-th row and h-th column of the first initial strip area;

[0240] The computer device can construct the abscissa set T = {T(1, 1), …, T(w, h), …, T(W, H)} and the ordinate set F of the color histogram according to each pixel value and the corresponding occurrence frequency in the first initial strip area hist = {F hist (T(1, 1)), …, F hist (T(w, h)), …, F hist (T(W, H))}, where the number of elements in the abscissa set T and the ordinate set F of the color histogram is the same, and both the abscissa set T and the ordinate set F hist are ordered sets with non-repeating elements. hist

[0241] Step 1004, the computer device calculates the kurtosis, color entropy, and luminance uniformity corresponding to the color histogram according to the color histogram.

[0242] The kurtosis of the color histogram can refer to the sharpness of the pixel value distribution in the color histogram, which is used to reflect the concentration degree of the pixel values in the color histogram. The larger the kurtosis, the more concentrated the pixel values in the color histogram are in a certain range; the smaller the kurtosis, the more uniform the pixel value distribution in the color histogram is, without obvious concentration or extreme values.

[0243] ​The color entropy of a color histogram can refer to the complexity of pixel values in the color histogram, which is used to reflect the complexity of the color histogram and the diversity of pixel values. The higher the color entropy, the more types of pixel values in the color histogram, the higher the complexity of the color histogram, and the more information contained in the first initial strip region; the lower the color entropy, the more concentrated the distribution of pixel values in the color histogram, the lower the complexity of the color histogram, and the less information contained in the first initial strip region.

[0244] The brightness uniformity of a color histogram can refer to the uniformity of the light and dark distribution in the color histogram. The higher the brightness uniformity, the smaller the difference between the light and dark regions of the color histogram; the lower the brightness uniformity, the greater the difference between the light and dark regions of the color histogram.

[0245] The computer device can calculate the mean and variance of the color histogram based on the pixel values and the occurrence frequencies of the pixel values in the color histogram. Then, the computer device can calculate the kurtosis, color entropy, and brightness uniformity corresponding to the color histogram based on the mean and variance of the color histogram.

[0246] Exemplarily, the computer device can calculate the mean and variance of the color histogram based on the pixel values and the occurrence frequencies of the pixel values in the color histogram as follows:

[0247]

[0248] where, T(s) represents the s-th pixel value in the set T of the abscissas of the color histogram, F hist (s) represents the occurrence frequency of the s-th pixel value in the set F hist of the ordinates of the color histogram, s = 1, 2, 3,..., S, and S represents the total number of pixel values in the color histogram without repeated pixel values, H mean represents the mean of the color histogram, and H var represents the variance of the color histogram;

[0249] The computer device can calculate the kurtosis, color entropy, and brightness uniformity corresponding to the color histogram based on the mean and variance of the color histogram:

[0250]

[0251] where, T avg represents the average value of each pixel value in the set T of the abscissas of the color histogram, T max represents the maximum value of each pixel value in the set T of the abscissas of the color histogram, K represents the kurtosis of the color histogram, E represents the color entropy of the color histogram, and U represents the brightness uniformity of the color histogram.

[0252] Step 1006: The computer device calculates the weighted sum of the kurtosis, color entropy, and brightness uniformity corresponding to the color histogram to obtain the density score corresponding to the first initial strip region.

[0253] The computer device can determine the weights corresponding to the kurtosis, color entropy, and brightness uniformity of the color histogram respectively, and then perform a weighted sum calculation based on the kurtosis, color entropy, and brightness uniformity of the color histogram and their weights to obtain the density score corresponding to the first initial strip region.

[0254] Exemplarily, the computer device performs a weighted sum calculation on the kurtosis K, color entropy E, and brightness uniformity U corresponding to the color histogram to obtain the density score Score1 corresponding to the first initial strip region:

[0255] Score1 = α1E + α2K + α3U Equation (15);

[0256] Where, α1 represents the weight corresponding to the kurtosis of the color histogram, α2 represents the weight corresponding to the color entropy of the color histogram, α3 represents the weight corresponding to the brightness uniformity of the color histogram, and α1, α2, and α3 satisfy the relationship α1 + α2 + α3 = 1, and Score1 represents the density score corresponding to the first initial strip region.

[0257] In some embodiments, α1, α2, and α3 can be set as needed according to the actual situation.

[0258] In some embodiments, before the computer device performs a weighted sum calculation on the kurtosis, color entropy, and brightness uniformity corresponding to the color histogram, it can perform normalization processing on the kurtosis and color entropy so that the kurtosis and color entropy are in the same dimension. As Figure 11 shown, step 1006 may include the following steps:

[0259] Step 1101: The computer device calculates the initial kurtosis, initial color entropy, and brightness uniformity corresponding to the color histogram according to the color histogram.

[0260] The method for calculating the initial kurtosis, initial color entropy, and brightness uniformity corresponding to the color histogram according to the color histogram is similar to the method for calculating the kurtosis, color entropy, and brightness uniformity corresponding to the color histogram described in step 1004 above, and will not be elaborated here.

[0261] Step 1103: The computer device performs normalization processing on the initial kurtosis and initial color entropy respectively to obtain the target kurtosis and target color entropy.

[0262] It can be understood that the numerical ranges of the initial kurtosis and the initial color entropy are greatly affected by the pixel value distribution corresponding to the first initial strip region. If the initial kurtosis and the initial color entropy are directly used to calculate the density fraction, the numerical scale differences among the initial kurtosis, the initial color entropy, and the brightness uniformity usually lead to the density fraction being dominated by the initial kurtosis and / or the initial color entropy during the weighted sum calculation, resulting in the distortion of the density fraction. Therefore, before calculating the density fraction, it is necessary to perform a normalization process on the initial kurtosis and the initial color entropy to ensure that the numerical values of the kurtosis and the color entropy are within the comparable scale range of the brightness uniformity, thereby ensuring the objectivity and reliability of the density fraction and achieving an accurate assessment of the positive degree of the strip region.

[0263] The computer device can perform a normalization process on the initial kurtosis and the initial color entropy respectively according to the value ranges of the kurtosis and the color entropy of the color histogram to obtain the target kurtosis and the target color entropy.

[0264] Exemplarily, the computer device can perform a normalization process on the initial kurtosis and the initial color entropy respectively to obtain the target kurtosis and the target color entropy:

[0265]

[0266] Among them, K max represents the maximum kurtosis value of the color histogram, K min represents the minimum kurtosis value of the color histogram, K ori represents the initial kurtosis of the color histogram, K norm represents the initial kurtosis K ori represents the target kurtosis obtained after the normalization process, E max represents the maximum color entropy value of the color histogram, E min represents the minimum color entropy value of the color histogram, E ori represents the initial color entropy of the color histogram, E norm represents the initial color entropy E ori represents the target color entropy obtained after the normalization process.

[0267] Step 1105, the computer device performs a weighted sum calculation on the target kurtosis, the target color entropy, and the brightness uniformity corresponding to the color histogram to obtain the density fraction corresponding to the first strip region.

[0268] The computer device can determine the weights corresponding to the target kurtosis, the target color entropy, and the brightness uniformity respectively, and then perform a weighted sum calculation according to the target kurtosis, the target color entropy, the brightness uniformity, and their weights to obtain the density fraction corresponding to the first strip region.

[0269] Exemplarily, the computer device can perform a weighted sum calculation on the target kurtosis, target color entropy, and brightness uniformity corresponding to the color histogram to obtain the density score corresponding to the first initial strip region:

[0270] Score2 = α4E norm + α5K norm + α6U Equation (18);

[0271] Among them, α4 represents the weight corresponding to the target kurtosis of the color histogram, α5 represents the weight corresponding to the target color entropy of the color histogram, α6 represents the weight corresponding to the brightness uniformity of the color histogram, and α4 + α5 + α6 = 1. Score2 represents the density score corresponding to the first initial strip region after normalization processing.

[0272] The computer device performs normalization processing on the initial kurtosis and initial color entropy to ensure that the values of the target kurtosis, target color entropy, and brightness uniformity are within a comparable scale range, so that the density score calculated by weighted sum according to the target kurtosis, target color entropy, and brightness uniformity is objective and reliable, and can truly reflect the depth of color in the first initial strip region.

[0273] In the embodiments of the present application, the computer device calculates the color histogram corresponding to the first initial strip region, and performs a weighted sum calculation according to the kurtosis, color entropy, and brightness uniformity of the color histogram to obtain the density score corresponding to the first initial strip region. By using kurtosis, color entropy, and brightness uniformity to evaluate density, it is possible to comprehensively consider the concentration of pixel value distribution, the complexity of pixel value distribution, and the consistency of pixel value brightness in the initial strip region, and can more accurately reflect the actual color characteristics of the strip region, so as to more accurately quantify the depth of color in the strip region, and further more accurately reflect the positive degree of the strip region, thereby improving the accuracy and recognition efficiency of the computer device for identifying the strip region in the electrophoresis image.

[0274] In some embodiments, before the computer device performs image segmentation on the target electrophoresis image, it is necessary to perform a preliminary identification on the target electrophoresis image to ensure that each sub-image of the electrophoresis lane obtained by segmenting the target electrophoresis image can completely display the corresponding lane. Figure 12 This is the flowchart for performing quality control detection on the target electrophoresis image provided by the embodiments of the present application. As Figure 12 shown, before the step of the computer device performing image segmentation on the target electrophoresis image to obtain multiple sub-images of the electrophoresis lane, the following steps may be included:

[0275] Step 1202, the computer device converts the target electrophoresis image from the RGB color space to the HSV color space to obtain an HSV image.

[0276] The RGB color space may refer to an additive color model based on the three primary colors of red, green, and blue, representing various colors through combinations of different intensities of red, green, and blue light. The HSV color space may refer to a color model based on hue, saturation, and value, where hue is used to distinguish different color types, saturation represents the purity of the color, and value represents the brightness of the color. By converting the target electrophoresis image from the RGB color space to the HSV color space, the HSV image corresponding to the target electrophoresis image can better reflect the color characteristics, thereby enabling more accurate identification and differentiation of the band region and the non-band region.

[0277] The computer device can determine the hue, saturation, and value of each pixel point in the target electrophoresis image based on the red-channel pixel value, blue-channel pixel value, and green-channel pixel value corresponding to each pixel point, thereby converting the target electrophoresis image into an HSV image.

[0278] Since the above method of converting the target electrophoresis image from the RGB color space to the HSV color space to obtain the HSV image is similar to the prior art, it will not be elaborated here.

[0279] Step 1204, the computer device identifies the blue pixel regions included in the HSV image.

[0280] The blue pixel regions may include the band regions shown by the color development after the binding of the electrophoresis image with a blue staining agent and immunoglobulin. The blue staining agent may include, but is not limited to, Coomassie Brilliant Blue staining solution. Since immunoglobulin itself is colorless or very light in color, it is almost impossible to directly observe the distribution of immunoglobulin in the electrophoresis image. Therefore, a staining agent needs to be added to label the immunoglobulin.

[0281] The computer device can identify the blue pixel regions included in the HSV image based on the hue, saturation, and value of each pixel point in the HSV image, thereby determining the positions and ranges of the respective blue pixel regions in the HSV image.

[0282] Step 1206, determine whether there is an intersection of the region contours of multiple blue pixel regions. If not, execute step 1208. If so, execute step 1210.

[0283] The computer device can perform image quality control based on whether there is an intersection of the region contours of multiple blue pixel regions. Image quality control may refer to the quality control of the target electrophoresis image to ensure that the respective band regions included in the target electrophoresis image are independent and non-overlapping, thereby ensuring the accuracy and reliability of the subsequent analysis of the band regions.

[0284] In some embodiments, the computer device can determine whether there is an intersection of regional contours among multiple blue pixel regions according to the positions and ranges of the blue pixel regions in the HSV image. If there is an intersection of regional contours, it indicates that there is an intersection phenomenon among multiple band regions of the target electrophoresis image, that is, it indicates that the lanes in the HSV image are contaminated and there are problems with the image quality of the target electrophoresis image, and it is not recommended to perform subsequent steps; if there is no intersection of regional contours, it indicates that the target electrophoresis image is normal and can be segmented to obtain multiple sub-images of the lanes.

[0285] Step 1208, the computer device segments the target electrophoresis image to obtain multiple sub-images of the lanes.

[0286] The computer device can segment the target electrophoresis image to obtain multiple sub-images of the lanes when there is no intersection of regional contours among multiple blue pixel regions, and execute steps 202 - 212 to complete the recognition and comparison interpretation of each band region in the multiple sub-images of the lanes in the target electrophoresis image.

[0287] Step 1210, the computer device stops performing subsequent steps on the target electrophoresis image.

[0288] It can be understood that if the lanes in the HSV image are contaminated, when the computer device recognizes the band regions of the contaminated target electrophoresis image, it will not be able to recognize or mis-recognize the specific ranges and positions of the band regions. Therefore, it is necessary to perform image quality control on the band regions included in the target electrophoresis image before recognizing the band regions.

[0289] In some embodiments, when the computer device determines that there is an intersection of regional contours among multiple blue pixel regions, it can send a warning instruction for the intersection of regional contours to remind the detection personnel that they need to re-operate the target sample, that is, re-acquire the target sample, or re-acquire the electrophoresis image generated by the IFE technology, so as to improve the hierarchical recognition efficiency of the target electrophoresis image corresponding to the target sample.

[0290] In the embodiments of the present application, the computer device can detect the image quality of the target electrophoresis image based on the contour shapes of each band region in the lane of the HSV image corresponding to the target electrophoresis image, so as to realize low-latency and highly sensitive image quality control of the target electrophoresis image and ensure the accuracy and reliability of subsequent analysis.

[0291] Based on the electrophoresis image hierarchical recognition method provided in the above embodiments, Figure 13 This is a structural block diagram of a device for hierarchical recognition of an electrophoresis image provided in an embodiment of the present application. As Figure 13As shown, in one embodiment, a hierarchical recognition device 1300 for an electrophoresis image is provided. The hierarchical recognition device 1300 for an electrophoresis image can be applied to an electronic device. The hierarchical recognition device 1300 for an electrophoresis image includes a lane segmentation module 1301, a band recognition module 1302, a calculation module 1303, a band screening module 1304, a band determination module 1305, and a hierarchical recognition module 1306.

[0292] The lane segmentation module 1301 is configured to perform image segmentation on a target electrophoresis image to obtain multiple lane sub-images, where the multiple lane sub-images include a control lane sub-image and multiple other lane sub-images.

[0293] The band recognition module 1302 is configured to recognize band regions included in each lane sub-image to obtain one or more initial band regions in one or more lane sub-images.

[0294] The calculation module 1303 is configured to perform density evaluation on each initial band region to obtain a density score corresponding to each initial band region.

[0295] The band screening module 1304 is configured to screen out initial band regions with density scores greater than a density threshold as first band regions.

[0296] The calculation module 1303 is further configured to determine the degree of overlap between each first band region in each other lane sub-image and each first band region in the control lane sub-image.

[0297] The band determination module 1305 is configured to determine, according to the degree of overlap corresponding to each first band region in each other lane sub-image, the first band regions that exist in both the other lane sub-images and the control lane sub-image as second band regions.

[0298] The hierarchical recognition module 1306 is configured to determine the positive degree corresponding to the target electrophoresis image according to the density scores corresponding to each second band region.

[0299] In some embodiments, the band recognition module 1302 may include a band search sub-module, an image generation sub-module, and a band set sub-module.

[0300] The band search sub-module is configured to search for each band region included in a first lane sub-image to obtain a first band coordinate set corresponding to the first lane sub-image, where the first lane sub-image is any one of the lane sub-images, and the first band coordinate set includes the image vertical coordinates of each band region in the first lane sub-image.

[0301] An image generation sub-module, configured to identify the foreground region and the background region of the first lane sub-image, and fill the background region with a target pixel value to obtain a second lane sub-image.

[0302] A stripe search sub-module, further configured to enable the computer device to search for each stripe region included in the second lane sub-image, and obtain a second stripe coordinate set corresponding to the second lane sub-image, where the second stripe coordinate set includes the image vertical coordinates of each stripe region in the second lane sub-image.

[0303] A stripe set sub-module, configured to determine each initial stripe region included in the first lane sub-image according to the first stripe coordinate set and the second stripe coordinate set.

[0304] In some embodiments, the stripe set sub-module further includes an image conversion unit, a curve generation unit, a data acquisition unit, and a set determination unit.

[0305] The image conversion unit is configured to convert the first lane sub-image into a grayscale image.

[0306] Calculation module 1303 is further configured to enable the computer device to accumulate the grayscale values of each pixel point of the grayscale image in the horizontal direction and calculate the average value to generate an index pixel array. The index pixel array includes N elements, and the N elements correspond one by one to the N rows of pixels of the grayscale image. The value of the Pth element is the average value corresponding to the Pth row of pixels of the grayscale image; N and P are positive integers, and P is less than or equal to N.

[0307] The curve generation unit is configured to generate a gradient change curve according to the index pixel array.

[0308] The data acquisition unit is configured to determine a target element where a gradient mutation occurs according to the gradient change curve, and obtain the image vertical coordinate corresponding to the target element.

[0309] Calculation module 1303 is further configured to calculate the line segment distance corresponding to every two adjacent target elements according to the image vertical coordinates corresponding to each target element.

[0310] The set determination unit is configured to filter out the target elements whose corresponding line segment distances are less than the distance threshold to obtain the first stripe coordinate set of the first lane sub-image.

[0311] In some embodiments, the stripe set sub-module further includes a stripe region expansion unit.

[0312] The data acquisition unit is further configured to obtain the first image vertical coordinate and the second image vertical coordinate corresponding to each stripe region in the first lane sub-image according to the first stripe coordinate set and the second stripe coordinate set. The first image vertical coordinate is the image vertical coordinate of the top edge of the stripe region, and the second image vertical coordinate is the image vertical coordinate of the bottom edge of the stripe region.

[0313] A strip area expansion unit is configured to expand the positions of the strip areas in the first sub-image of the lane according to an expansion coefficient, the first image vertical coordinates and the second image vertical coordinates corresponding to the strip areas in the first sub-image of the lane, so as to obtain the initial strip areas included in the first lane image.

[0314] In some embodiments, the calculation module 1303 may include a first calculation sub-module, a second calculation sub-module, and a third calculation sub-module.

[0315] The first calculation sub-module is configured to calculate the color histogram corresponding to the first initial strip area, where the first initial strip area is any one of the initial strip areas.

[0316] The second calculation sub-module is configured to calculate the kurtosis, color entropy, and brightness uniformity corresponding to the color histogram according to the color histogram.

[0317] The third calculation sub-module is configured to perform a weighted sum calculation on the kurtosis, color entropy, and brightness uniformity corresponding to the color histogram to obtain the density score corresponding to the first initial strip area.

[0318] In some embodiments, the second calculation sub-module is further configured to calculate the initial kurtosis, initial color entropy, and brightness uniformity corresponding to the color histogram according to the color histogram;

[0319] The second calculation sub-module is further configured to calculate the computer device to perform normalization processing on the initial kurtosis and the initial color entropy respectively to obtain the target kurtosis and the target color entropy.

[0320] The third calculation sub-module is further configured to perform a weighted sum calculation on the target kurtosis, target color entropy, and brightness uniformity corresponding to the color histogram to obtain the density score corresponding to the first initial strip area.

[0321] In some embodiments, the electrophoresis image layer recognition device 1300 further includes a contour detection module.

[0322] The image conversion unit is further configured to convert the target electrophoresis image from the RGB color space to the HSV color space to obtain an HSV image.

[0323] The contour detection module is configured to identify the blue pixel regions included in the HSV image and determine whether there is an intersection of the region contours of the multiple blue pixel regions.

[0324] The lane segmentation module 1301 is further configured to segment the target electrophoresis image to obtain multiple sub-images of the lane in the case that there is no intersection of the region contours of the multiple blue pixel regions.

[0325] Figure 14The block diagram of an electronic device provided by an embodiment of the present application. As Figure 14 shown, the electronic device 1400 may include a memory 1402 and a processor 1401. A computer program is stored in the memory 1402. When the computer program is executed by the processor 1401, the electronic device 1400 implements the hierarchical recognition method of the electrophoresis image described in the above embodiments.

[0326] The processor 1401 may include one or more processing cores. The processor 1401 connects various parts within the entire electronic device through various interfaces and lines. By running or executing instructions, programs, code sets, or instruction sets stored in the memory, and by calling data stored in the memory, the processor 1401 executes various functions of the electronic device and processes data. Optionally, the processor 1401 may be implemented in at least one hardware form of digital signal processing, field programmable gate array, or programmable logic array. The processor 1401 may integrate one or a combination of several of a central processing unit (CPU for short), a graphics processing unit (GPU for short), and a modem. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the display content; the modem is used to process wireless communication. It can be understood that the above modem may not be integrated into the processor 1401 and may be implemented separately through a communication chip.

[0327] The memory 1402 may include a random access memory and may also include a read-only memory. The memory can be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for implementing at least one function, instructions for implementing the above method embodiments, etc. The data storage area may also store data created during the use of the electronic device.

[0328] An embodiment of the present application discloses a computer-readable storage medium that stores a computer program. When the computer program is executed by a processor, the processor implements the hierarchical recognition method of the electrophoresis image described in the above embodiments.

[0329] An embodiment of the present application discloses a computer program product that includes a computer program. When the computer program is executable by a processor, the processor implements the hierarchical recognition method of the electrophoresis image described in the above embodiments.

[0330] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above various methods. Among them, the storage medium can be a magnetic disk, an optical disk, a ROM, etc.

[0331] The above are only specific examples of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for hierarchical recognition of electrophoresis images, characterized in that, Including: The computer device performs image segmentation on the target electrophoresis image to obtain multiple lane sub-images, where the multiple lane sub-images include a control lane sub-image and multiple other lane sub-images; The computer device identifies the band regions included in each of the lane sub-images to obtain one or more initial band regions in one or more of the lane sub-images; The computer device performs density evaluation on each of the initial band regions to obtain the density scores corresponding to the respective initial band regions; The computer device filters out the initial band regions with density scores greater than the density threshold as the first band regions; The computer device determines the degree of overlap between each of the first band regions in each of the other lane sub-images and each of the first band regions in the control lane sub-image; The computer device determines, based on the degree of overlap corresponding to each of the first band regions in each of the other lane sub-images, the first band regions that exist in both the other lane sub-images and the control lane sub-image as the second band regions; The computer device determines the positive degree corresponding to the target electrophoresis image based on the density scores corresponding to each of the second band regions.

2. The method according to claim 1, wherein The computer device identifies the band regions included in each of the lane sub-images to obtain one or more initial band regions in each of the lane sub-images, including: The computer device searches for each of the band regions included in the first lane sub-image to obtain the first band coordinate set corresponding to the first lane sub-image, where the first lane sub-image is any one of the lane sub-images, and the first band coordinate set includes the image vertical coordinates of each of the band regions in the first lane sub-image; The computer device identifies the foreground region and the background region of the first lane sub-image and fills the background region with a target pixel value to obtain a second lane sub-image; The computer device searches for each of the band regions included in the second lane sub-image to obtain the second band coordinate set corresponding to the second lane sub-image, where the second band coordinate set includes the image vertical coordinates of each of the band regions in the second lane sub-image; The computer device determines each of the initial band regions included in the first lane sub-image based on the first band coordinate set and the second band coordinate set.

3. The method according to claim 2, wherein The computer device searches for each of the band regions included in the first lane sub-image to obtain the first band coordinate set, including: The computer device converts the first lane sub-image into a grayscale image; The computer device accumulates the grayscale values of each pixel point of the grayscale image in the horizontal direction and calculates the average value to generate an index pixel array, where the index pixel array includes N elements, the N elements correspond one-to-one to the N rows of pixels of the grayscale image, and the value of the Pth element is the average value corresponding to the Pth row of pixels of the grayscale image; N and P are positive integers, and P is less than or equal to N; The computer device generates a gradient change curve based on the index pixel array; The computer device determines a target element with a gradient mutation according to the gradient change curve, and obtains the image ordinate corresponding to the target element; The computer device calculates the line segment distance corresponding to every two adjacent target elements according to the image ordinates corresponding to each of the target elements; The computer device filters out the target elements with a corresponding line segment distance less than the distance threshold to obtain the first strip coordinate set of the first sub-image of the lane; 4. The method according to claim 2, wherein The computer device determines each initial strip region included in the first sub-image of the lane according to the first strip coordinate set and the second strip coordinate set, including: The computer device obtains the first image ordinate and the second image ordinate corresponding to each strip region in the first sub-image of the lane according to the first strip coordinate set and the second strip coordinate set, where the first image ordinate is the image ordinate of the top edge of the strip region, and the second image ordinate is the image ordinate of the bottom edge of the strip region; The computer device performs position expansion on each strip region in the first sub-image of the lane according to the expansion coefficient, the first image ordinate and the second image ordinate corresponding to each strip region in the first sub-image of the lane to obtain each initial strip region included in the first lane image; 5. The method according to claim 1, wherein The computer device performs density evaluation on each initial strip region to obtain the density score corresponding to each initial strip region, including: The computer device calculates the color histogram corresponding to the first initial strip region, where the first initial strip region is any one of the initial strip regions; The computer device calculates the kurtosis, color entropy and brightness uniformity corresponding to the color histogram according to the color histogram; The computer device performs a weighted sum calculation on the kurtosis, color entropy and brightness uniformity corresponding to the color histogram to obtain the density score corresponding to the first initial strip region; 6. The method according to claim 5, wherein The computer device calculates the kurtosis, color entropy and brightness uniformity corresponding to the color histogram according to the color histogram, including: The computer device calculates the initial kurtosis, initial color entropy and brightness uniformity corresponding to the color histogram according to the color histogram; The computer device respectively performs normalization processing on the initial kurtosis and the initial color entropy to obtain the target kurtosis and the target color entropy; The computer device performs a weighted sum calculation on the kurtosis, color entropy and brightness uniformity corresponding to the color histogram to obtain the density score corresponding to the first initial strip region, including: The computer device performs a weighted sum calculation on the target kurtosis, target color entropy and brightness uniformity corresponding to the color histogram to obtain the density score corresponding to the first initial strip region; 7. The method according to any one of claims 1 to 6, characterized in that Before the computer device performs image segmentation on the target electrophoresis image to obtain multiple sub-images of the lane, the method further includes: The computer device converts the target electrophoresis image from the RGB color space to the HSV color space to obtain an HSV image; The computer device identifies the blue pixel regions included in the HSV image and determines whether there is an intersection of the regional contours of the multiple blue pixel regions; The computer device performs image segmentation on the target electrophoresis image to obtain multiple lane sub-images, including: When there is no intersection of the regional contours of the multiple blue pixel regions, the computer device segments the target electrophoresis image to obtain the multiple lane sub-images.

8. A layered recognition device for an electrophoresis image, characterized in that The device includes: A lane segmentation module, configured to perform image segmentation on a target electrophoresis image to obtain multiple lane sub-images, where the multiple lane sub-images include a control lane sub-image and multiple other lane sub-images; A band recognition module, configured to recognize the band regions included in each of the lane sub-images to obtain one or more initial band regions in one or more of the lane sub-images; A calculation module, configured to evaluate the density of each initial band region to obtain the density score corresponding to each initial band region; A band screening module, configured to screen out the initial band regions with density scores greater than the density threshold as the first band regions; The calculation module is further configured to determine the degree of overlap between each first band region in each of the other lane sub-images and each first band region in the control lane sub-image; A band determination module, configured to determine the first band regions that exist in both the other lane sub-images and the control lane sub-image as the second band regions according to the degree of overlap corresponding to each first band region in each of the other lane sub-images; A stratification recognition module, configured to determine the positive degree corresponding to the target electrophoresis image according to the density score corresponding to each of the second band regions.

9. An electronic device, characterized in that, It includes a memory and a processor. A computer program is stored in the memory. When the computer program is executed by the processor, the processor implements the electrophoresis image stratification recognition method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the processor implements the electrophoresis image stratification recognition method according to any one of claims 1-7.

Images