Visual monitoring service system and method based on product detection

Abstract

The invention discloses a visual monitoring service system and method based on product detection, and relates to the technical field of abnormal loss monitoring. Historical data of image records used by reagents in a laboratory are collected, and the types of the reagents in the image records and the actions of using the reagents by operators are marked; calculating the duration time of each action type in the action sequence of the reference time period and the occurrence frequency of two adjacent action combinations to obtain the unit time threshold value of the action type and the frequency threshold value of the action combination, and respectively obtaining the difference between the unit time threshold value and the frequency threshold value; and calculating an influence coefficient of the action duration time difference and the occurrence frequency difference on the abnormal consumption amount of the reagent, identifying the duration time of the action and the occurrence frequency of the action combination in a unit time period, and predicting the abnormal consumption amount of the reagent in the unit time period in combination with the influence coefficient.

Description

Technical Field

[0001] The present invention relates to the technical field of abnormal loss monitoring, and specifically to a visualization monitoring service system and method based on product detection. Background Art

[0002] The efficient operation of a product testing laboratory depends on the effective management of various consumables. Consumables in a product testing laboratory are characterized by a wide variety of types, diverse specifications, large differences in usage frequency and quantity, which makes consumable management a highly challenging task. Traditional laboratory consumable management mainly relies on manual records and empirical judgments.

[0003] During the use of consumables, it is difficult for manual management to monitor the usage of consumables in real time and accurately trace them. When situations such as consumable waste, abuse, or even loss occur, it is difficult to quickly locate the responsible person and the specific reasons, and it is impossible to effectively control costs and ensure the quality of experiments. In addition, under the traditional management mode, the management efficiency of laboratory consumables is low, and the abnormal usage of consumables cannot be timely feedback, which easily causes repeated purchases and waste of resources. Summary of the Invention

[0004] The purpose of the present invention is to provide one to solve the problems raised in the prior art.

[0005] To achieve the above purpose, the present invention provides the following technical solution: A visualization monitoring service method based on product detection, the method includes:

[0006] Step S100: Collect historical data of image records of reagent usage in the laboratory, and mark the types of reagents and the actions of operators using reagents in the image records;

[0007] Step S200: Collect a reference time period without abnormal loss, obtain the image records within the reference period, and collect the actions of the staff using reagents in the image records to obtain the action sequence of the reference time period;

[0008] Step S300: Calculate the duration of various action types and the occurrence frequency of adjacent two-action combinations in the action sequence of the reference time period to obtain the unit time threshold of the action type and the frequency threshold of the action combination;

[0009] Step S400: Obtain the duration of the action and the occurrence frequency of the action combination in a unit detection cycle, respectively obtain the differences of the unit time threshold and the frequency threshold, and calculate the influence coefficients of the action duration difference and the occurrence frequency difference on the abnormal loss amount of the reagent;

[0010] Step S500: Collect the current image record of the laboratory, identify the duration of actions and the occurrence frequency of action combinations in a unit time period, predict the abnormal consumption of reagents in a unit time period in combination with the influence coefficient, and send an information reminder to the relevant management personnel of the laboratory when the predicted value of abnormal consumption is greater than the threshold.

[0011] In a laboratory scenario, the irregular operation and management of equipment by staff will result in abnormal consumption of laboratory consumables.

[0012] Situations of irregular operation, such as operational errors like failure to accurately weigh reagents, for example, not replacing the pipette tip when transferring different reagents with the same pipette, or the pipette tip not being thoroughly cleaned and having residues of the previous batch of reagents, may all lead to reagent cross - contamination and loss.

[0013] Lack of effective management will increase the abnormal loss of laboratory consumables. For example, the received consumables are not classified in a timely manner and are placed randomly, and when in use, they cannot be found in time, resulting in repeated receipt of consumables and additional consumption of consumables.

[0014] In addition, if the experimenter is not familiar with the experimental content and the operation sequence or steps are incorrect during the experiment, it may lead to reagent contamination and accelerate the consumption of consumables.

[0015] Furthermore, step S100 includes:

[0016] Step S101: Set a camera at the position of the laboratory operation table to obtain the image of the reagent container on the laboratory operation table and the action image of the laboratory staff operating the reagent container.

[0017] Step S102: Collect the image record captured by the camera, mark the reagent container and the action image in the image record, obtain the corresponding relationship between the reagent container and the reagent, and get the reagent identification database and the action identification database. The image record includes picture images and video images.

[0018] Furthermore, step S200 includes:

[0019] Step S201: Collect the consumption of a certain reagent in a time period of duration T. When there is no abnormal loss in the consumption of a certain reagent in the time period, take the time period of duration T as the reference time period.

[0020] Step S202: Obtain the image record captured by the camera in the reference time period, identify the reagent container corresponding to a certain reagent, take the reagent container as the target container, and obtain all the video segments in the reference time period where the target container appears.

[0021] Step S203: Identify the actions of laboratory staff operating the target container in the video clip, and gather the actions in the order of their appearance in time to obtain an action sequence.

[0022] Structurally process the surface features of the operator's actions obtained from the video images, and transform the features into pattern codes so that the operation processing system can process the performance features.

[0023] Further, step S300 includes:

[0024] Step S301: Gather the action records in the action sequence separately according to the types of actions, and accumulate the duration of each type of action in a time period of length T to obtain the reference action duration.

[0025] Step S302: Obtain the reference action duration tj of the j-th type of action, and calculate the unit time threshold τj of the j-th type of action, τj = tj / T.

[0026] Step S303: Denote the i-th action in the action sequence as ai, and the (i + 1)-th action as ai+1, and form a continuous action group with ai and ai+1.

[0027] Step S304: Gather all continuous action groups, classify the continuous action groups with the same previous action into one category, divide the types of the subsequent actions in the same category of continuous action groups, calculate the occurrence frequency of the subsequent action types, and use the occurrence frequency as the frequency threshold of the subsequent action types in the same category of continuous action groups.

[0028] Capture the behavior habits of the staff by recording the duration of the operation actions and the combination of operation actions during the period of abnormal loss, and capture and analyze the attributes of the abnormal loss through the external manifestations and internal associations, that is, the duration of the abnormal actions and the combination of actions.

[0029] Further, step S400 includes:

[0030] Step S401: Set a unit time period of length T2, where T2 < T, obtain the usage records of a certain reagent for several unit time periods, and denote the unit time periods with abnormal loss of a certain reagent as target time periods.

[0031] Among them, the setting that T2 is less than T helps to obtain more objective threshold data.

[0032] Step S402: Obtain the image records of the target time periods, identify the actions of operating the target container, and form a target action sequence for each unit time period in the image records respectively.

[0033] Step S403: Cumulatively calculate the duration of the actions in the target action sequence to obtain the cumulative duration kj of the j-th type of action in a certain target action sequence, and calculate the duration deviation value rtj, where rtj = |kj - τj × T2|;

[0034] Step S404: Extract all consecutive action groups of a certain target action sequence, classify the consecutive action groups with the same previous action in the consecutive action groups into one category, calculate the occurrence frequency of the subsequent action in each category of consecutive action groups, and compare it with the frequency threshold to obtain the frequency difference between the occurrence frequency and the frequency threshold;

[0035] Step S405: Obtain the loss amount of abnormal loss of a certain reagent in all target time periods, set the time influence coefficient of the duration deviation value and the frequency influence coefficient of the frequency difference, and establish a linear equation of the duration deviation value and the frequency difference with the loss amount;

[0036] Step S406: Collect the linear equations corresponding to each target time period to form a system of linear equations, solve the time influence coefficient and the frequency influence coefficient, and obtain the time influence coefficient corresponding to the time difference of the duration of all types of actions and the frequency influence coefficient corresponding to the frequency difference of the current frequency.

[0037] By calculating the degree of deviation from the reference threshold, instead of directly analyzing the error, unitize the generated error to avoid the problem of inconsistent data structures caused by different reagent types or different operation procedures during the detection process.

[0038] Further, step S500 includes:

[0039] Step S501: Collect the current image record of the laboratory operation table, obtain the image record of using a certain reagent in a unit time period, identify the actions in the image record, and collect the actions in the image record to form the current action sequence;

[0040] Step S502: Classify the actions in the current action sequence, obtain the consecutive action groups of the current action sequence, obtain the duration of various actions in the current action sequence, and the occurrence frequency of the subsequent action in the consecutive action groups;

[0041] Step S503: Compare the duration of the action with the time threshold to obtain the time difference of various actions, compare the time difference with the frequency threshold to obtain the frequency difference, multiply the time difference by the time influence coefficient to obtain the first estimated value h1, and multiply the frequency difference by the frequency sound coefficient to obtain the second estimated value h2;

[0042] Step S504: Set the abnormal loss management threshold G. When G < h1 + h2, remind the management staff of the laboratory to check the usage amount of a certain reagent.

[0043] By constructing a flexible alarm mechanism, circularly detect the possible abnormal loss situations in a unit time, which can timely discover the occurrence situation and occurrence time of abnormal losses, remind the relevant management staff to timely detect the remaining amount of relevant reagents, timely replenish and check the reagent usage situation, improve the work efficiency of the testing laboratory, and reduce the occurrence of waste situations.

[0044] To better implement the above method, a visualization monitoring service system based on product detection is also proposed. The system includes: a feature management module, an action sequence management module, a threshold management module, an influence coefficient management module, and a real-time monitoring module. Among them, the feature management module is used to manage the reagent container features and the action features of relevant workers and operating reagent containers. The action sequence management module is used to manage the action sequences within a time range. The threshold management module is used to manage the unit time threshold of action types and the frequency threshold of action combinations. The influence coefficient management module is used to manage the differences between the unit time threshold and the frequency threshold, calculate the influence coefficients of the action duration difference and the occurrence frequency difference on the abnormal loss amount of reagents. The real-time monitoring module is used to manage the current image records, predict the abnormal loss value, and send an alarm message when the alarm condition is met.

[0045] Furthermore, the threshold management module includes: an action type management unit, a time threshold management unit, an action group management unit, and a frequency threshold management unit. Among them, the action type management unit is used to obtain the types of actions. The time threshold management unit is used to calculate the unit time threshold of actions. The action group management unit is used to manage continuous action groups. The frequency threshold management unit is used to calculate the frequency threshold.

[0046] Furthermore, the influence coefficient management module includes: a duration deviation value management unit, a frequency difference management unit, a loss amount management unit, and a coefficient management unit. Among them, the duration deviation value management unit is used to manage the duration deviation value. The frequency difference management unit is used to manage the frequency difference. The loss amount management unit is used to manage the loss amount in the target time period. The coefficient management unit is used to manage the time influence coefficient and the frequency influence coefficient.

[0047] Further, the real-time monitoring module includes: a current sequence management unit, a sequence feature management unit, a consumption prediction unit, and an information reminder unit; among them, the current sequence management unit is used to manage the action sequence in the current image, the sequence feature management unit is used to manage the action duration and the occurrence frequency of action combinations in the current action sequence, the consumption prediction unit is used to manage the predicted value of reagent consumption, and the information reminder unit is used to give an alarm prompt to relevant management personnel when the alarm condition is met.

[0048] Compared with the prior art, the beneficial effects of the present invention are: by extracting the surface features and inherent hidden features of the abnormal consumption of consumables, modeling the generation events of abnormal consumption. Further, feature analysis is carried out on the generation events of abnormal consumption to give an early warning of the abnormal consumption of consumables in the detection laboratory. The present invention combines a visualization platform to dynamically monitor the abnormal loss situation in the laboratory. Compared with the manual registration method, the monitoring cycle is shortened and the monitoring frequency is increased, so the abnormal consumption of reagent consumables can be found in time. It helps to reduce waste in the process of laboratory work and also helps to replenish missing consumables in time, so that the laboratory work can proceed in an orderly manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a schematic structural diagram of a visualization monitoring service system based on product detection according to the present invention;

[0050] Figure 2 is a schematic flow diagram of a visualization monitoring service method based on product detection according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0052] Embodiment: As Figure 1 - Figure 2 shown, the present invention provides a technical solution, a visualization monitoring service system and method based on product detection, and the method includes:

[0053] Step S100: Collect historical data of image records of reagent use in the laboratory, and mark the reagent types and the actions of operators using reagents in the image records;

[0054] Among them, step S100 includes:

[0055] Step S101: Set up a camera at the position of the laboratory operation table to obtain images of reagent containers on the laboratory operation table and images of the actions of laboratory staff operating the reagent containers;

[0056] Step S102: Collect the image records taken by the camera, mark the reagent containers and action images in the image records, obtain the corresponding relationship between the reagent containers and the reagents, and obtain a reagent recognition database and an action recognition database. The image records include picture images and video images.

[0057] In the embodiment, recognizable actions include, for example, pipetting, pipette aspiration, pipette pressing, drainage, pouring, and container opening and closing, etc.

[0058] Step S200: Collect a reference time period without abnormal loss, obtain the image records within the reference time period, and gather the actions of the staff using the reagents in the image records to obtain an action sequence for the reference time period;

[0059] Among them, Step S200 includes:

[0060] Step S201: Collect the consumption of a certain reagent in a time period of duration T. When the consumption of a certain reagent in the time period does not show abnormal loss, take the time period of duration T as the reference time period;

[0061] Step S202: Obtain the image records taken by the camera in the reference time period, identify the reagent container corresponding to a certain reagent, take the reagent container as the target container, and obtain all video segments in the reference time period where the target container appears;

[0062] Step S203: Identify the actions of the laboratory staff operating the target container in the video segment, and gather the actions in the order of the time of appearance to obtain an action sequence.

[0063] Step S300: Calculate the duration of each action type and the occurrence frequency of adjacent two-action combinations in the action sequence of the reference time period to obtain the unit time threshold of the action type and the frequency threshold of the action combination;

[0064] Among them, Step S300 includes:

[0065] Step S301: According to the types of actions, separately gather the action records in the action sequence, and in a time period of duration T, accumulate the duration of each action type to obtain the reference action duration;

[0066] Step S302: Obtain the reference action duration tj of the jth action type, and calculate the unit time threshold τj of the jth action type, τj = tj / T;

[0067] Step S303: Denote the i-th action in the action sequence as ai and the (i + 1)-th action as ai+1, and form a continuous action group with ai and ai+1;

[0068] Step S304: Gather all the continuous action groups, classify the continuous action groups with the same previous action into one category. In the continuous action groups of the same category, divide the types of the subsequent actions, calculate the occurrence frequency of the types of the subsequent actions, and use the occurrence frequency as the frequency threshold of the types of the subsequent actions in the continuous action groups of the same category.

[0069] In the embodiment, different actions are encoded. In the action sequence of a reference time period, the continuous action groups with A1 as the previous action are obtained and denoted as (A1, B1), (A1, B2), (A1, B1), (A1, B3), (A1, B2), (A1, B1);

[0070] The occurrence frequency of B1 after A1 is 3 / 6 = 0.5, the occurrence frequency of B2 after A1 is 2 / 6 = 0.33, and the occurrence frequency of B3 after A1 is 1 / 6 = 0.17;

[0071] Set the thresholds of B1, B2, and B3 actions after the A1 action to 0.5, 0.33, and 0.17 respectively.

[0072] Step S400: Obtain the duration of the action and the occurrence frequency of the action combination in the unit detection period, respectively obtain the differences between the unit time threshold and the frequency threshold, and calculate the influence coefficients of the action duration difference and the occurrence frequency difference on the abnormal reagent loss amount;

[0073] Among them, Step S400 includes;

[0074] Step S401: Set a unit time period with a duration of T2, where T2 < T, obtain the usage records of a certain reagent in several unit time periods, and denote the unit time periods with abnormal loss of a certain reagent as target time periods;

[0075] Step S402: Obtain the image records of the target time periods, identify the actions of operating the target container, and form a target action sequence for each unit time period in the image records respectively;

[0076] Step S403: Accumulate the durations of the actions in the target action sequence, obtain the cumulative duration kj of the j-th action in a certain target action sequence, and calculate the duration deviation value rtj, rtj = |kj - τj × T2|;

[0077] Step S404: Extract all consecutive action groups of a certain target action sequence. Classify the consecutive action groups with the same previous action in the consecutive action groups into one category. Calculate the occurrence frequencies of the subsequent actions in each category of consecutive action groups, and compare them with the frequency threshold to obtain the frequency difference between the occurrence frequency and the frequency threshold.

[0078] Step S405: Obtain the loss amounts of abnormal losses of a certain reagent in all target time periods. Set the time influence coefficient of the duration deviation value and the frequency influence coefficient of the frequency difference. Establish a linear equation of the duration deviation value and the frequency difference with the loss amount.

[0079] Step S406: Collect the linear equations corresponding to each target time period to form a linear equation system. Solve the time influence coefficient and the frequency influence coefficient to obtain the time influence coefficient corresponding to the time difference of the duration of all types of actions and the frequency influence coefficient corresponding to the frequency difference of the current frequency.

[0080] In the embodiment, obtain 4 types of actions A1, B1, B2, and B3 in the target time period. Collect the time of each occurrence of the action, and respectively collect the time lengths of the occurrences of the actions according to the action types. Among them, the duration of the A1 action in the target time period is kA1, the duration of the B1 action in the target time period is kB1, the duration of the B2 action in the target time period is kB2, and the duration of the B3 action in the target time period is kB3.

[0081] Respectively obtain the unit time thresholds of the 4 types of actions, denoted as τA1, τB1, τB2, and τB3,

[0082] Respectively calculate the time differences rA1 = |kA1 - τA1 × T2|, rB1 = |kB1 - τB1 × T2|, rB2 = |kB2 - τB2 × T2|, rB3 = |kB3 - τB3 × T2|;

[0083] Respectively calculate the consecutive action groups composed of A1, B1, B2, and B3. Taking A1 as the previous action as an example, in the target period, obtain the occurrence frequencies of the subsequent action types, denoted as fB1, fB2, and fB3,

[0084] Obtain the frequency thresholds, denoted as φB1, φB2, and φB3, respectively. Calculate the frequency differences pB1 = |fB1 - φB1|, pB2 = |fB2 - φB2|, pB3 = |fB3 - φB3|;

[0085] Obtain the loss amount d of abnormal loss in a unit time period, and set the time influence coefficients α1, α2, α3, and α4, where the dimensions of α1, α2, α3, and α4 are all unit loss amount / unit time difference. Set the frequency influence coefficients β1, β2, and β3, where the dimensions of β1, β2, and β3 are all unit loss amount / unit frequency difference;

[0086] Construct a linear equation: α1×rA1 + α2×rB1 + α3×rB2 + α4×rB3 + β1×pB1 + β2×pB2 + β3×pB3 = d;

[0087] Collect several linear equations, and solve for the time influence coefficients α1, α2, α3, and α4 and the frequency influence coefficients β1, β2, and β3;

[0088] Preferably, if multiple solutions are obtained for the same coefficient through different equation systems, interpolation method, fitting method, or clustering analysis method can be used to obtain the substitution value of the coefficient to replace the time influence coefficient or frequency influence coefficient.

[0089] Step S500: Collect the current image record of the laboratory, identify the duration of the actions and the occurrence frequency of the action combinations in a unit time period, and predict the abnormal consumption of the reagent in a unit time period in combination with the influence coefficients. When the predicted value of the abnormal consumption is greater than the threshold, information reminder is sent to the relevant management personnel of the laboratory;

[0090] Among them, step S500 includes:

[0091] Step S501: Collect the current image record of the laboratory operation table, obtain the image record of using a certain reagent in a unit time period, identify the actions in the image record, and collect the actions in the image record to form the current action sequence;

[0092] Step S502: Classify the actions in the current action sequence, obtain the continuous action groups of the current action sequence, obtain the duration of various actions in the current action sequence, and the occurrence frequency of the subsequent action in the continuous action group;

[0093] Step S503: Compare the duration of the action with the time threshold to obtain the time difference of various actions, compare the time difference with the frequency threshold to obtain the frequency difference, multiply the time difference by the time influence coefficient to obtain the first predicted value h1, and multiply the frequency difference by the frequency influence coefficient to obtain the second predicted value h2;

[0094] Step S504: Set the abnormal loss management threshold G. When G < h1 + h2, remind the management personnel of the laboratory to check the usage amount of a certain reagent.

[0095] Obtain the current action sequence, count the time of each action in the current action sequence, and a total of m actions are counted. Among them, the duration of the nth action is c n , calculate h1,

[0096] Among them, γ n represents the time influence coefficient of the nth action, τ n represents the unit time threshold of the nth action, τ n ×T2 represents the time threshold;

[0097] Obtain y consecutive action groups, and obtain the occurrence frequency of the e-th action type in the l-th consecutive action group, denoted as f le , where the l-th consecutive action group includes x action types, calculate h2, Among them, η e represents the frequency influence coefficient of the e-th action type, φ le represents the frequency threshold of the e-th action type in the l-th consecutive action group.

[0098] The system includes: a feature management module, an action sequence management module, a threshold management module, an influence coefficient management module, and a real-time monitoring module;

[0099] Among them, the feature management module is used to manage the characteristics of reagent containers and the action characteristics of relevant workers and the operation of reagent containers;

[0100] Among them, the action sequence management module is used to manage the action sequence within a time range;

[0101] Among them, the threshold management module is used to manage the unit time threshold of action types and the frequency threshold of action combinations. Among them, the threshold management module includes: an action type management unit, a time threshold management unit, an action group management unit, and a frequency threshold management unit. The action type management unit is used to obtain the types of actions, the time threshold management unit is used to calculate the unit time threshold of actions, the action group management unit is used to manage consecutive action groups, and the frequency threshold management unit is used to calculate frequency thresholds;

[0102] Among them, the influence coefficient management module is used to manage the differences between the unit time threshold and the frequency threshold, and calculate the influence coefficients of the differences in action duration and occurrence frequency on the abnormal loss amount of reagents. Among them, the influence coefficient management module includes: a duration deviation value management unit, a frequency difference management unit, a loss amount management unit, and a coefficient management unit. The duration deviation value management unit is used to manage the duration deviation value, the frequency difference management unit is used to manage the frequency difference, the loss amount management unit is used to manage the loss amount in the target time period, and the coefficient management unit is used to manage the time influence coefficient and the frequency influence coefficient;

[0103] Among them, the real-time monitoring module is used to manage the current image record, predict the abnormal loss value, and send an alarm message when the alarm condition is met. Among them, the real-time monitoring module includes: a current sequence management unit, a sequence feature management unit, a loss prediction unit, and an information reminder unit. The current sequence management unit is used to manage the action sequence in the current image. The sequence feature management unit is used to manage the action duration and the occurrence frequency of the action combination in the current action sequence. The loss prediction unit is used to manage the predicted value of the reagent loss. The information reminder unit is used to give an alarm reminder to relevant management personnel when the alarm condition is met.

[0104] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above-described exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention. Any reference signs in the claims should not be construed as limiting the claims involved.

Claims

1. A visualization monitoring service method based on product detection, characterized in that: The method includes the steps of: Step S100: Collect the historical data of the image records of reagent use in the laboratory, and mark the reagent types and the actions of the operators using the reagents in the image records; Step S200: Collect the reference time period without abnormal loss, obtain the image records within the reference time period, and pool the actions of the staff using the reagents in the image records to obtain the action sequence of the reference time period; Step S300: Calculate the duration of each action type and the occurrence frequency of adjacent two-action combinations in the action sequence of the reference time period to obtain the unit time threshold of the action type and the frequency threshold of the action combination; Step S400: Obtain the duration of the action and the occurrence frequency of the action combination in the unit detection cycle, respectively obtain the differences between the unit time threshold and the frequency threshold, and calculate the influence coefficients of the action duration difference and the occurrence frequency difference on the abnormal loss amount of the reagent; Step S500: Collect the current image records of the laboratory, identify the duration of the actions and the occurrence frequency of the action combinations in the unit time cycle, combine the influence coefficients to predict the abnormal consumption amount of the reagent in the unit time cycle, and send an information reminder to the relevant management personnel of the laboratory when the predicted value of the abnormal consumption is greater than the threshold.

2. The visual monitoring service method based on product detection according to claim 1, wherein: Step S100 includes: Step S101: Set a camera at the position of the laboratory operation table to obtain the image of the reagent container on the laboratory operation table and the image of the laboratory staff operating the reagent container; Step S102: Collect the image records taken by the camera, mark the reagent containers and the action images in the image records, obtain the corresponding relationship between the reagent containers and the reagents, and obtain the reagent identification database and the action identification database. The image records include picture images and video images.

3. The visualization monitoring service method based on product detection according to claim 2, wherein: Step S200 includes: Step S201: Collect the consumption amount of a certain reagent in a time period of length T. When the consumption amount of a certain reagent in the time period does not show abnormal loss, use the time period of length T as the reference time period; Step S202: Obtain the image records taken by the camera in the reference time period, identify the reagent container corresponding to the certain reagent, use the reagent container as the target container, and obtain all the video segments in which the target container appears in the reference time period; Step S203: Identify the actions of the laboratory staff operating the target container in the video segment, and pool the actions in the order of the time when the actions appear to obtain the action sequence.

4. A visualization monitoring service method based on product detection according to claim 3, characterized in that: Step S300 includes: Step S301: Pool the action records in the action sequence according to the types of actions, and accumulate the duration of each action type in the time period of length T to obtain the reference action duration; Step S302: Obtain the reference action duration tj of the j-th action type, and calculate the unit time threshold τj of the j-th action, τj = tj / T; Step S303: Denote the i-th action in the action sequence as ai, and the i + 1-th action as ai+1, and form a continuous action group with ai and ai+1; Step S304: Aggregate all continuous action groups, classify the continuous action groups with the same previous action into one category. In the same category of continuous action groups, classify the types of the subsequent actions, calculate the occurrence frequency of the types of the subsequent actions, and use the occurrence frequency as the frequency threshold of the types of the subsequent actions in the same category of continuous action groups.

5. A visualization monitoring service method based on product detection according to claim 4, characterized in that: Step S400 includes: Step S401: Set a unit time period with a duration of T2, where T2 < T. Obtain the usage records of a certain reagent for several unit time periods, and record the unit time periods with abnormal loss of a certain reagent as target time periods. Step S402: Obtain the image records of the target time periods, identify the actions of operating the target container, and respectively form a target action sequence for the actions of each unit time period in the image records. Step S403: Accumulate the durations of the actions in the target action sequence, obtain the cumulative duration kj of the j-th type of action in a certain target action sequence, and calculate the duration deviation value rtj, where rtj = |kj - τj × T2|. Step S404: Extract all continuous action groups of a certain target action sequence, classify the continuous action groups with the same previous action in the continuous action groups into one category, calculate the occurrence frequency of the subsequent actions in each category of continuous action groups, and compare it with the frequency threshold to obtain the frequency difference between the occurrence frequency and the frequency threshold. Step S405: Obtain the loss amount of abnormal loss of a certain reagent in all target time periods, set the time influence coefficient of the duration deviation value and the frequency influence coefficient of the frequency difference, and establish a linear equation between the duration deviation value, the frequency difference and the loss amount. Step S406: Aggregate the linear equations corresponding to each target time period to form a system of linear equations, and solve the time influence coefficient and the frequency influence coefficient to obtain the time influence coefficient corresponding to the time difference of the durations of all types of actions and the frequency influence coefficient corresponding to the frequency difference of the current frequencies.

6. The visual monitoring service method based on product detection according to claim 5, characterized in that: Step S500 includes: Step S501: Collect the current image records of the laboratory operation table, obtain the image records of using a certain reagent in a unit time period, identify the actions in the image records, and aggregate the actions in the image records to form the current action sequence. Step S502: Classify the actions in the current action sequence, obtain the continuous action groups of the current action sequence, obtain the durations of various actions in the current action sequence, and the occurrence frequency of the subsequent actions in the continuous action groups. Step S503: Compare the duration of the action with the time threshold to obtain the time difference of various actions, compare the time difference with the frequency threshold to obtain the frequency difference, multiply the time difference by the time influence coefficient to obtain the first estimated value h1, and multiply the frequency difference by the frequency influence coefficient to obtain the second estimated value h2. Step S504: Set an abnormal loss management threshold G. When G < h1 + h2, remind the management personnel of the laboratory to check the usage amount of a certain reagent.

7. A visualization monitoring service system based on product detection, which is used to execute a visualization monitoring service method based on product detection according to any one of claims 1-6, characterized in that: The system includes: A feature management module, an action sequence management module, a threshold management module, an influence coefficient management module, and a real-time monitoring module. Among them, the feature management module is used to manage the features of reagent containers and the action features of relevant workers and the actions of operating reagent containers. The action sequence management module is used to manage the action sequences within a time range. The threshold management module is used to manage the unit time threshold of action types and the frequency threshold of action combinations. The influence coefficient management module is used to manage the differences between the unit time threshold and the frequency threshold, calculate the influence coefficients of the action duration difference and the occurrence frequency difference on the abnormal loss amount of reagents. The real-time monitoring module is used to manage the current image records, predict the abnormal loss value, and send an alarm message when the alarm condition is met.

8. The visual monitoring service system based on product detection according to claim 7, characterized in that: The threshold management module includes: an action type management unit, a time threshold management unit, an action group management unit, and a frequency threshold management unit; Among them, the action type management unit is used to obtain the types of actions. The time threshold management unit is used to calculate the unit time threshold of actions. The action group management unit is used to manage continuous action groups. The frequency threshold management unit is used to calculate the frequency threshold.

9. The visual monitoring service system based on product detection according to claim 7, characterized in that: The influence coefficient management module includes: a duration deviation value management unit, a frequency difference management unit, a loss amount management unit, and a coefficient management unit; Among them, the duration deviation value management unit is used to manage the duration deviation value. The frequency difference management unit is used to manage the frequency difference. The loss amount management unit is used to manage the loss amount in the target time period. The coefficient management unit is used to manage the time influence coefficient and the frequency influence coefficient.

10. A visual monitoring service system based on product detection according to claim 7, characterized in that: The real-time monitoring module includes: a current sequence management unit, a sequence feature management unit, a loss amount prediction unit, and an information reminder unit; Among them, the current sequence management unit is used to manage the action sequences in the current image. The sequence feature management unit is used to manage the action duration and the occurrence frequency of action combinations in the current action sequence. The loss amount prediction unit is used to manage the predicted value of the reagent loss amount. The information reminder unit is used to give an alarm prompt to relevant management personnel when the alarm condition is met.

Images