Color conversion method and system for reagent detection images based on attention mechanism

Abstract

The present invention discloses a reagent detection picture color conversion method based on an attention mechanism, a reagent detection picture color conversion method based on an attention mechanism, the reagent detection picture is used to form a to-be-predicted image, and the to-be-predicted image is the prediction basis of the concentration value of a vital sign parameter, including: S1, establishing an attention mechanism model, the attention mechanism model is a self-attention mechanism model; S2, obtaining corresponding multiple matrices based on multiple sample pictures and gradients of known concentrations, and obtaining a three-dimensional array of RGB eigenvalues ​​of the to-be-predicted image and a three-dimensional array of RGB eigenvalues ​​of the calibrated image based on multiple matrices; S3, obtaining a similarity matrix based on the to-be-predicted image and multiple matrices; S4, obtaining weights based on the similarity matrix and the self-attention mechanism model; S5, obtaining the corresponding vital sign parameter concentration value of the reagent detection picture based on the weights and the concentration value corresponding to the calibrated image. A system, an electronic device, and a computer-readable storage medium are also disclosed.

Description

Technical Field

[0001] The present invention relates to the field of reagent image decolorization processing and deep learning technology, and in particular to a reagent detection image color conversion method and system based on an attention mechanism. Background Art

[0002] Currently, the conversion of three-dimensional color images to grayscale images uses a dimensionality reduction image decolorization technology, which is a process of reducing three-dimensional images to two-dimensional images. A common method is to linearly map the brightness value of the pixel to the grayscale value, as shown in the following formulas (1) and (2):

[0003] like (1)

[0004] or (2)

[0005] However, the above formula is still not very user-friendly for distinguishing certain colors. Even if the human eye can see obvious differences, they still cannot be distinguished according to the above formula. That is, even if the image has obvious color contrast, it is difficult to distinguish or the distinction is poor after being converted to grayscale.

[0006] Therefore, algorithms have emerged to address the problem of insufficient differentiation after converting the same image to grayscale, and to solve the problem of poor differentiation by enhancing the contrast within the image. However, in the actual application requirements of image recognition after reagent testing, multiple images are converted to one dimension (i.e., grayscale value) using the same algorithm for comparison. If the grayscale values ​​after dimensionality reduction are very similar, this will cause the concentration prediction values ​​of images of different concentrations to be very similar when identifying the reagent concentration, resulting in misjudgment or failure to make a reasonable judgment.

[0007] According to the color development of the reagent at concentrations of 60 (mmol / L) and 100 (mmol / L), there is a significant difference in visual observation. It is expected that after conversion to grayscale, the grayscale value will be smaller when the concentration is darker. However, as shown in Table 1, when converted to grayscale using equations (1) and (2), the grayscale values ​​are very close and contrary to expectations, so it is difficult to distinguish the predicted concentrations.

[0008] Table 1

[0009]

[0010] In addition, the attention mechanism method is usually used in deep network models, which requires a large amount of data to train and obtain its weight parameters for predictive use. In the case of in vivo component metabolism testing, this mechanism will reduce the efficiency of the test and affect the tester's experience. Summary of the invention

[0011] The purpose of the present invention is to provide a method and system for color conversion of reagent detection images based on an attention mechanism. In view of the situation where there is still a problem of discrimination after multiple images are converted into one-dimensional grayscale images, a method and system without converting them is proposed. By adopting an optimized self-attention mechanism, the grayscale value is directly calculated after operating the image with a set of images of existing concentrations to obtain the concentration result required for reagent detection.

[0012] The first aspect of the present invention is to provide a reagent detection picture color conversion method based on an attention mechanism, wherein the reagent detection picture is used to form an image to be predicted, and the image to be predicted is a basis for predicting a concentration value of a vital sign parameter, comprising:

[0013] S1, establishing an attention mechanism model, wherein the attention mechanism model is a self-attention mechanism model;

[0014] S2, obtaining a corresponding plurality of matrices based on a plurality of sample images of known concentrations and gradients, and obtaining a three-dimensional array of RGB eigenvalues ​​of the image to be predicted and a three-dimensional array of RGB eigenvalues ​​of the calibrated image based on the plurality of matrices;

[0015] S3, obtaining a similarity matrix based on the image to be predicted and the multiple matrices;

[0016] S4, obtaining a weight based on the similarity matrix and the self-attention mechanism model;

[0017] S5, obtaining the corresponding vital sign parameter concentration value of the reagent detection image based on the weight and the concentration value corresponding to the calibrated image.

[0018] Preferably, the self-attention mechanism model is a Query-Key-Value (QKV) mode, as shown in formula (3):

[0019] (3);

[0020] in, represents the RGB feature value of the image to be predicted, ; Represents the RGB feature value of the calibrated image, ; Indicates the concentration value corresponding to the calibrated image. ; The number of gradients representing sample images of known concentrations; Represents the RGB feature value dimension; Represents the dimension of color channels; express Dimension.

[0021] Preferably, S2 includes:

[0022] S21, selecting M sample images of known concentration, setting the number of gradients to N, and grouping the M sample images according to the number of gradients N to obtain N matrices corresponding to the gradients;

[0023] S22, averaging the RGB eigenvalues ​​of M / N sample images corresponding to each gradient in the N matrices to obtain two three-dimensional arrays, wherein the two three-dimensional arrays correspond to the three-dimensional array of RGB eigenvalues ​​of the image to be predicted and the three-dimensional array of RGB eigenvalues ​​of the calibrated image, respectively; wherein m=M / N.

[0024] Preferably, grouping the M sample images according to the number of gradients N to obtain N matrices corresponding to the gradients is performing equal clustering and grouping of the M sample images according to the number of gradients N to obtain N matrices corresponding to the gradients.

[0025] Preferably, S3 includes:

[0026] S31, calculating similarity values ​​based on the three-dimensional array of RGB feature values ​​of the image to be predicted and the N matrices respectively;

[0027] S32, forming a corresponding similarity matrix from the N similarity values ​​obtained by calculation.

[0028] Preferably, S4 includes:

[0029] The similarity matrix is ​​input into the self-attention mechanism model to calculate and obtain an attention value, where the attention value represents the N weights corresponding to the N similarity values.

[0030] Preferably, S5 includes:

[0031] The N weights are weighted and summed with the concentration value corresponding to the calibrated image to obtain the corresponding vital sign parameter concentration value of the reagent detection image.

[0032] The second aspect of the present invention is to provide a reagent detection picture color conversion system based on an attention mechanism, wherein the reagent detection picture is used to form an image to be predicted, and the image to be predicted is a basis for predicting a concentration value of a vital sign parameter, including:

[0033] An attention mechanism model building module (101) is used to build an attention mechanism model, wherein the attention mechanism model is a self-attention mechanism model;

[0034] A matrix and feature array determination module (102) is used to obtain a corresponding plurality of matrices based on a plurality of sample images of known concentrations and gradients, and to obtain a three-dimensional array of RGB feature values ​​of the image to be predicted and a three-dimensional array of RGB feature values ​​of the calibrated image based on the plurality of matrices;

[0035] A similarity matrix determination module (103), used for obtaining a similarity matrix based on the image to be predicted and the multiple matrices;

[0036] A weight determination module (104), configured to obtain a weight based on the similarity matrix and the self-attention mechanism model;

[0037] A vital sign parameter concentration determination module (105) is used to obtain a vital sign parameter concentration value corresponding to the reagent detection image based on the weight and the concentration value corresponding to the calibrated image.

[0038] A third aspect of the present invention is to provide an electronic device, comprising a processor and a memory, wherein the memory stores a plurality of instructions, and the processor is used to read the instructions and execute the method described in the first aspect.

[0039] A fourth aspect of the present invention is to provide a computer-readable storage medium, wherein the computer-readable storage medium stores a plurality of instructions, and the plurality of instructions can be read by a processor to execute the method described in the first aspect.

[0040] Beneficial effects of the method and system of the present invention:

[0041] For the same batch of reagent images, which have a relatively simple image format and highly consistent shadow reflection points and device problems, the concentration of reagents can be predicted based on the attention mechanism method using only a limited number of images without conducting a large amount of deep network training, and avoiding the problems caused by different extraction rules and weights. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A flow chart of a method for color conversion of reagent detection images based on an attention mechanism according to an embodiment of the present invention;

[0044] Figure 2 A flowchart of step S2 of a reagent detection image color conversion method based on an attention mechanism according to an embodiment of the present invention;

[0045] Figure 3 A flowchart of step S3 of a reagent detection image color conversion method based on an attention mechanism according to an embodiment of the present invention;

[0046] Figure 4 This is an architecture diagram of a reagent detection image color conversion system based on an attention mechanism according to an embodiment of the present invention;

[0047] Figure 5 A schematic diagram of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0050] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. Embodiment 1

[0051] like Figure 1 As shown, this embodiment provides a reagent detection picture color conversion method based on an attention mechanism, wherein the reagent detection picture is used to form an image to be predicted, and the image to be predicted is a basis for predicting a concentration value of a vital sign parameter, including:

[0052] S1, establishing an attention mechanism model, wherein the attention mechanism model is a self-attention mechanism model;

[0053] As a preferred implementation, the self-attention mechanism model is a Query-Key-Value (QKV) model, as shown in formula (3):

[0054] (3);

[0055] in, represents the RGB feature value of the image to be predicted, ; Represents the RGB feature value of the calibrated image, ; Indicates the concentration value corresponding to the calibrated image. ; The number of gradients representing sample images of known concentrations; Represents the RGB feature value dimension; Represents the dimension of color channels; express Dimension.

[0056] S2, obtaining a corresponding plurality of matrices based on a plurality of sample images of known concentrations and gradients, and obtaining a three-dimensional array of RGB eigenvalues ​​of the image to be predicted and a three-dimensional array of RGB eigenvalues ​​of the calibrated image based on the plurality of matrices;

[0057] like Figure 2 As shown, as a preferred embodiment, the S2 includes:

[0058] S21, selecting M sample images of known concentration, setting the number of gradients to N, and grouping the M sample images according to the number of gradients N to obtain N matrices corresponding to the gradients;

[0059] S22, averaging the RGB eigenvalues ​​of M / N sample images corresponding to each gradient in the N matrices to obtain two three-dimensional arrays, wherein the two three-dimensional arrays correspond to the three-dimensional array of RGB eigenvalues ​​of the image to be predicted and the three-dimensional array of RGB eigenvalues ​​of the calibrated image, respectively; wherein m=M / N.

[0060] As a preferred implementation, the M sample images are grouped according to the number of gradients N to obtain N matrices corresponding to the gradients, which is to obtain N matrices corresponding to the gradients after clustering and grouping the M sample images in equal amounts according to the number of gradients N.

[0061] S3, obtaining a similarity matrix based on the image to be predicted and the multiple matrices;

[0062] like Figure 3 As shown, as a preferred implementation, S3 includes:

[0063] S31, calculating similarity values ​​based on the three-dimensional array of RGB feature values ​​of the image to be predicted and the N matrices respectively;

[0064] S32, forming a corresponding similarity matrix from the N similarity values ​​obtained by calculation.

[0065] S4, obtaining a weight based on the similarity matrix and the self-attention mechanism model;

[0066] As a preferred implementation, the S4 includes:

[0067] The similarity matrix is ​​input into the self-attention mechanism model to calculate and obtain an attention value, where the attention value represents the N weights corresponding to the N similarity values.

[0068] S5, obtaining the corresponding vital sign parameter concentration value of the reagent detection image based on the weight and the concentration value corresponding to the calibrated image.

[0069] As a preferred implementation, S5 includes:

[0070] The N weights are weighted and summed with the concentration value corresponding to the calibrated image to obtain the corresponding vital sign parameter concentration value of the reagent detection image.

[0071] Application Examples

[0072] For example, given a picture whose size is 112*112*3, determine its concentration value.

[0073] (1) Q-query is the image to be queried, K-key is a total of 30 images with known concentrations, 6 gradients and 5 parallel samples for each gradient. V-value is the corresponding concentration value. First, average the samples of each gradient of the sample image with known concentration to obtain 6 112*112*3 matrices, calculate the RGB mean of the image, and Q-query and K-key can obtain a 3D array;

[0074] like:

[0075] query = torch.Tensor([[[116, 130, 129]]])

[0076] keys = torch.Tensor([[[123, 132, 130], [110, 129, 129], [80, 116,119], [77, 119, 124], [63, 111, 118], [54, 106, 114]]])

[0077] values ​​= torch.Tensor([[[0],

[20] ,

[60] ,

[100] ,

[200] ,

[250] ]])

[0078] In this example , ,

[0079] (2) Calculate the similarity between the image to be queried and the six known matrices to obtain the similarity matrix, and then perform Softmax to calculate the value of its attention-score, i.e., the weight;

[0080] The attention-score value after softmax calculation is as follows:

[0081] tensor([0.49, 0.61, 0, 0, 0, 0])

[0082] (3) The concentration value can be obtained by weighted summing the attention-score and the known concentration.

[0083] After the weight and value are calculated, the predicted concentration is obtained: tensor([

[12] ]). Embodiment 2

[0084] like Figure 4 As shown, this embodiment provides a reagent detection picture color conversion system based on an attention mechanism, wherein the reagent detection picture is used to form an image to be predicted, and the image to be predicted is a basis for predicting a concentration value of a vital sign parameter, including:

[0085] An attention mechanism model building module 101 is used to build an attention mechanism model, wherein the attention mechanism model is a self-attention mechanism model;

[0086] A matrix and feature array determination module 102 is used to obtain a plurality of corresponding matrices based on a plurality of sample images and gradients of known concentrations, and to obtain a three-dimensional array of RGB feature values ​​of the image to be predicted and a three-dimensional array of RGB feature values ​​of the calibrated image based on the plurality of matrices;

[0087] A similarity matrix determination module 103, configured to obtain a similarity matrix based on the image to be predicted and the multiple matrices;

[0088] A weight determination module 104, configured to obtain a weight based on the similarity matrix and the self-attention mechanism model;

[0089] The vital sign parameter concentration determination module 105 is used to obtain the vital sign parameter concentration value corresponding to the reagent detection image based on the weight and the concentration value corresponding to the calibrated image.

[0090] The present invention also provides a memory storing a plurality of instructions, wherein the instructions are used to implement the method as in the first embodiment.

[0091] like Figure 5As shown, the present invention also provides an electronic device, including a processor 301 and a memory 302 connected to the processor 301, the memory 302 stores a plurality of instructions, and the instructions can be loaded and executed by the processor so that the processor can execute the method as in the first embodiment.

[0092] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiments can be implemented by software, or by means of software plus necessary general hardware platforms. Based on such understanding, the technical solutions of the above embodiments can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A reagent detection image color conversion method based on attention mechanism, characterized in that: A reagent detection image color conversion method based on an attention mechanism, wherein the reagent detection image is used to form an image to be predicted, and the image to be predicted is a basis for predicting a concentration value of a vital sign parameter, including: S1, establishing an attention mechanism model, wherein the attention mechanism model is a self-attention mechanism model; S2, obtaining a corresponding plurality of matrices based on a plurality of sample images of known concentrations and gradients, and obtaining a three-dimensional array of RGB eigenvalues ​​of the image to be predicted and a three-dimensional array of RGB eigenvalues ​​of the calibrated image based on the plurality of matrices; S3, obtaining a similarity matrix based on the image to be predicted and the multiple matrices; S4, obtaining weights based on the similarity matrix and the self-attention mechanism model; S5, obtaining the corresponding vital sign parameter concentration value of the reagent detection image based on the weight and the concentration value corresponding to the calibrated image; The self-attention mechanism model is a Query-Key-Value (QKV) model, as shown in formula (3): （3）； in, represents the RGB feature value of the image to be predicted, ; Represents the RGB feature value of the calibrated image, ; Indicates the concentration value corresponding to the calibrated image. ; The number of gradients representing sample images of known concentrations; Represents the RGB feature value dimension; Represents the dimension of color channels; express Dimension.

2. According to claim 1, a reagent detection image color conversion method based on attention mechanism is characterized in that: The S2 includes: S21, selecting M sample images of known concentration, setting the number of gradients to N, and grouping the M sample images according to the number of gradients N to obtain N matrices corresponding to the gradients; S22, averaging the RGB eigenvalues ​​of M / N sample images corresponding to each gradient in the N matrices to obtain two three-dimensional arrays, wherein the two three-dimensional arrays correspond to the three-dimensional array of RGB eigenvalues ​​of the image to be predicted and the three-dimensional array of RGB eigenvalues ​​of the calibrated image, respectively; wherein m=M / N.

3. According to claim 2, a reagent detection image color conversion method based on attention mechanism is characterized in that: After grouping the M sample images according to the number of gradients N, N matrices corresponding to the gradients are obtained by equally clustering and grouping the M sample images according to the number of gradients N to obtain N matrices corresponding to the gradients.

4. According to claim 3, a reagent detection image color conversion method based on attention mechanism is characterized in that: The S3 includes: S31, calculating similarity values ​​based on the three-dimensional array of RGB feature values ​​of the image to be predicted and the N matrices respectively; S32, forming a corresponding similarity matrix from the N similarity values ​​obtained by calculation.

5. According to claim 4, a reagent detection image color conversion method based on attention mechanism is characterized in that: The S4 includes: The similarity matrix is ​​input into the self-attention mechanism model to calculate and obtain an attention value, where the attention value represents the N weights corresponding to the N similarity values.

6. According to the method of claim 5, the color conversion method of reagent detection image based on attention mechanism is characterized in that: The S5 includes: The N weights are weighted and summed with the concentration value corresponding to the calibrated image to obtain the corresponding vital sign parameter concentration value of the reagent detection image.

7. A reagent detection image color conversion system based on attention mechanism, characterized in that: Used to implement the method described in any one of claims 1 to 6, wherein the reagent detection picture is used to form an image to be predicted, and the image to be predicted is the basis for predicting the concentration value of the vital sign parameter, comprising: An attention mechanism model building module (101) is used to build an attention mechanism model, wherein the attention mechanism model is a self-attention mechanism model; A matrix and feature array determination module (102) is used to obtain a corresponding plurality of matrices based on a plurality of sample images of known concentrations and gradients, and to obtain a three-dimensional array of RGB feature values ​​of the image to be predicted and a three-dimensional array of RGB feature values ​​of the calibrated image based on the plurality of matrices; A similarity matrix determination module (103), used for obtaining a similarity matrix based on the image to be predicted and the multiple matrices; A weight determination module (104), configured to obtain a weight based on the similarity matrix and the self-attention mechanism model; A vital sign parameter concentration determination module (105) is used to obtain a vital sign parameter concentration value corresponding to the reagent detection image based on the weight and the concentration value corresponding to the calibrated image.

8. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a plurality of instructions, and the processor is used to read the instructions and execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the plurality of instructions can be read by a processor to execute the method according to any one of claims 1 to 6.

Images