Early warning control method of intelligent experiment table and intelligent experiment table

Abstract

The application provides a pre-warning control method of an intelligent experiment table and the intelligent experiment table. The method comprises the following steps: acquiring a first image of a region of the intelligent experiment table, obtaining first operation data according to the first image; performing matching calculation on the first operation data and standard operation data, and if the matching calculation result does not satisfy a preset condition, executing a pre-warning scheme. The scheme of the application can identify non-standard or even incorrect experimental operation behaviors of an experimenter in time, and effectively reduces experimental risks through execution of the pre-warning scheme.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent experiment equipment, in particular to a pre-warning control method of an intelligent experiment table, an intelligent experiment table, an electronic device and a storage medium. BACKGROUND

[0002] An experiment table is a table used for experimental detection and instrument storage in hospitals, schools, chemical plants, research institutes and other enterprises and institutions. The existing experiment table can only provide installation or line interface of related experimental equipment, and cannot recognize and analyze the operation behavior of experimenters, especially for students who have not fully mastered the experimental norms. The non-standard experimental operation cannot be pre-warned, which leads to high experimental risk. SUMMARY

[0003] In order to solve the technical problems in the background art, the present application provides a pre-warning control method of an intelligent experiment table, an intelligent experiment table, an electronic device and a storage medium.

[0004] The first aspect of the present application provides a pre-warning control method of an intelligent experiment table, which comprises:

[0005] Obtaining a first image of the intelligent experiment table area, and obtaining first operation data according to the first image;

[0006] Matching and calculating the first operation data with standard operation data, and if the matching and calculating result does not satisfy the first condition, executing a pre-warning scheme.

[0007] Optionally, the first operation data is obtained according to the first image, which comprises:

[0008] Identifying and continuously tracking the hand region and the experimental equipment region in the first image to obtain palm operation data and palm position sequence data, experimental equipment operation data and experimental equipment position sequence data;

[0009] Position matching and calculating the palm position sequence data and the experimental equipment position sequence data to obtain a plurality of operation data pairs, and taking the plurality of operation data pairs as the first operation data.

[0010] Optionally, the standard operation data is obtained by the following way:

[0011] Receiving experimental attribute data issued by a master control experiment table, and determining the standard operation data according to the experimental attribute data;

[0012] Or,

[0013] According to the first image, the experimental equipment attribute is extracted, the candidate experimental attribute data is determined according to the experimental equipment attribute, the candidate experimental attribute data is output to the experimenter, the target experimental attribute data is determined based on the feedback operation of the experimenter, and the standard operation data is determined according to the target experimental attribute data.

[0014] Optionally, the standard operation data includes a plurality of operation data, and the operation data includes an operation serial number.

[0015] The matching calculation of the first operation data and the standard operation data includes:

[0016] According to the operation serial number, the first similarity of each first operation data and the corresponding operation data in the standard operation data is calculated, and if the first similarity is greater than or equal to a first threshold, a mark is recorded, otherwise the mark is empty.

[0017] According to the operation serial number and the mark record, a mark sequence is established, and it is judged whether the mark sequence satisfies a second condition, if yes, the first operation data is set to satisfy a first condition, otherwise the first operation data is set to not satisfy the first condition.

[0018] Optionally, the operation data further includes an operation attribute.

[0019] The judgment of whether the mark sequence satisfies the second condition includes:

[0020] Each breakpoint in the mark sequence is determined, the operation attribute is determined according to the operation serial number corresponding to each breakpoint, if the operation attribute is a key operation, it is determined that the mark sequence does not satisfy the second condition, otherwise, it is determined that the mark sequence satisfies the second condition.

[0021] Optionally, the operation data includes action data and position data, the action data includes palm operation data and experimental equipment operation data, and the position data includes palm position sequence data and experimental equipment position sequence data.

[0022] The calculation of the first similarity of each first operation data and the corresponding operation data in the standard operation data includes:

[0023]

[0024] In the formula, sim represents the first similarity of the first operation data and the corresponding operation data in the standard operation data; Y i represents the i th data of the action data in the operation data, Y j represents the j th data of the position data in the operation data. representing the mean of each data in the action data in the first operation data, representing the mean of each data in the position data in the first operation data; m is the data amount in the action data, and n is the data amount in the position data.

[0025] Optionally, after the matching calculation result does not satisfy the first condition, the method further comprises:

[0026] acquiring a second image of a main control experiment table region, and acquiring second operation data according to the second image;

[0027] calculating a second similarity between the second operation data and the first operation data, if the second similarity is greater than or equal to a second threshold, not executing the pre-warning scheme, otherwise executing the pre-warning scheme.

[0028] A second aspect of the present application provides an intelligent experiment table, comprising a processing module, a storage module and an acquisition module, the processing module is connected with the storage module and the acquisition module respectively; wherein,

[0029] the storage module stores a computer program;

[0030] the acquisition module is used for acquiring image data of an experiment table region and transmitting the image data to the processing module;

[0031] the processing module is used for calling the computer program to realize the method according to any one of the above.

[0032] A third aspect of the present application provides a computer storage medium, the storage medium stores a computer program, and the computer program is run by a processor to execute the method according to any one of the above.

[0033] A fourth aspect of the present application provides an electronic device, which comprises a processor and a memory, and the memory stores a computer program, and the computer program is run by the processor to execute the method according to any one of the above.

[0034] The present application has the advantages that: the present application acquires the image of the experiment table region, extracts the first operation data of the experimenter through image recognition technology, and then matches and calculates the first operation data with the standard operation data corresponding to the current experiment, if the matching calculation result is unqualified, the pre-warning scheme is executed in time. Therefore, the scheme of the present application can timely identify the non-standard or even wrong experimental operation behavior of the experimenter, and effectively reduce the experimental risk through the execution of the pre-warning scheme. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be considered as limiting the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0036] Figure 1 is a flow diagram of a warning control method of an intelligent experiment table disclosed by the embodiments of the present application;

[0037] Figure 2 is a structural diagram of an intelligent experiment table disclosed by the embodiments of the present application;

[0038] Figure 3 is a structural diagram of an electronic device disclosed by the embodiments of the present application. DETAILED DESCRIPTION

[0039] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations.

[0040] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0041] It should be noted that: similar numbers and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0042] In the description of the present application, it should be noted that if the terms "upper", "lower", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship when the product of the present application is usually placed, and is only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0043] In addition, if the terms "first", "second" and the like appear, they are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0044] It should be noted that the features in the embodiments of the present application can be combined with each other without conflict.

[0045] Embodiment one

[0046] Please refer to Figure 1 , Figure 1 is a flowchart of a warning control method of an intelligent experiment table disclosed by an embodiment of the present application. As shown in Figure 1 , the warning control method of the intelligent experiment table comprises the following steps:

[0047] acquiring a first image of the region of the intelligent experiment table, and obtaining first operation data according to the first image;

[0048] matching and calculating the first operation data with standard operation data, and executing a warning scheme if the matching and calculating result does not satisfy a first condition.

[0049] In the embodiment of the present application, as described in the background, the experiment table in the prior art can only provide experiment-related equipment and line interfaces, and cannot recognize and analyze the experiment operation behavior of beginners, and cannot effectively reduce the risk caused by non-standard experiment operation behavior. Therefore, the present application acquires the image of the experiment table region, extracts the first operation data of the experimenter through image recognition technology, and then matches and calculates the first operation data with the standard operation data corresponding to the current experiment. If the matching and calculating result is unqualified, a warning scheme is executed in time. Thus, the scheme of the present application can timely recognize the non-standard or even incorrect experiment operation behavior of the experimenter, and effectively reduce the experiment risk through the execution of the warning scheme. The warning scheme can be freely set, for example, alarm output through sound and / or light.

[0050] It should be noted that the first image in the present application can be obtained by a camera arranged at each intelligent experiment table, or can be obtained by at least one camera arranged in the laboratory. The camera can be a gun camera or a ball camera, and the specific type is not limited.

[0051] In addition, the solution of the present invention can be implemented in a variety of processing entities, such as field processing equipment and servers. Among them, the field processing equipment can be a processing device deployed in a single laboratory, such as a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a discrete gate or transistor logic device, a discrete hardware component, etc.; and the server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The server can monitor multiple laboratories.

[0052] Optionally, extracting and obtaining first operation data according to the first image includes:

[0053] Identifying a hand region and an experimental device region in the first image and continuously tracking them to obtain palm operation data and palm position sequence data, experimental device operation data and experimental device position sequence data;

[0054] A position matching calculation is performed on the palm position sequence data and the experimental equipment position sequence data to obtain a plurality of operation data pairs, and the plurality of operation data pairs are used as the first operation data.

[0055] In an embodiment of the present invention, the experimenter may perform many actions unrelated to the experiment itself during the experiment, such as talking to others or organizing equipment from the previous experiment. These unrelated actions may lead to misjudgment. Therefore, the present invention simultaneously tracks and identifies palm operation data and palm position sequence data, as well as experimental equipment operation data and experimental equipment position sequence data from the first image, and performs matching calculations on the above two types of data to extract the actual experimental operation, thereby reducing the probability of misjudgment. The matching calculation can be to calculate the position overlap of the above two types of data, that is, when the palm and the experimental equipment positions overlap, it can be determined that the experimenter is operating the experimental equipment.

[0056] The palm operation data can include various gesture data, such as grabbing, pressing, and pinching, and the experimental device operation data can include various posture data, such as tilt angle, horizontal movement, and vertical movement. The palm operation data and experimental device operation data can describe the experimenter's various operation actions, while the palm position sequence data and experimental device position sequence data are the operation positions corresponding to each operation action.

[0057] Optionally, the standard operation data is obtained by the following way:

[0058] receiving experiment attribute data issued by a master control experiment table, and determining the standard operation data according to the experiment attribute data;

[0059] Alternatively,

[0060] extracting experiment equipment attributes according to the first image, determining candidate experiment attribute data according to the experiment equipment attributes, outputting the candidate experiment attribute data to an experimenter, determining target experiment attribute data based on feedback operation of the experimenter, and determining the standard operation data according to the target experiment attribute data.

[0061] In the embodiment of the application, the standard operation data can be determined in two ways, i.e., an experiment leader (for example, a teacher) can set the experiment types of each experiment table on the master control experiment table, and then the standard operation data can be determined based on the experiment attribute data issued by the master control experiment table; or, the experiment equipment attributes can be extracted by image recognition technology, and then a plurality of candidate experiment attribute data related to the experiment equipment can be screened out, which are output to the experimenter and determined based on the selection operation of the experimenter.

[0062] Optionally, the standard operation data includes a plurality of operation data, and the operation data includes an operation serial number.

[0063] The matching calculation of the first operation data and the standard operation data includes:

[0064] calculating a first similarity of each first operation data and corresponding operation data in the standard operation data according to the operation serial number, and if the first similarity is greater than or equal to a first threshold value, marking and recording, otherwise, marking as empty;

[0065] forming a marking sequence according to the operation serial number and the marking record, and judging whether the marking sequence satisfies a second condition, if yes, setting the first operation data to satisfy a first condition, otherwise, setting the first operation data to not satisfy the first condition.

[0066] In the embodiment of the application, the standard operation data in the application includes a plurality of operation data arranged in sequence, after each first operation data is identified, a first similarity calculation is performed on the first operation data and corresponding operation data in the standard operation data, if the similarity is greater than or equal to a first threshold value, it means that the first operation data is matched, and the matching condition is marked and recorded; along with the first similarity calculation, a comparison sequence data can be obtained according to the operation serial number and the marking record, and when the comparison sequence data satisfies a second condition, it means that a series of experiment operations of the experimenter in advance is in conformity with the specification.

[0067] Optionally, the operation data further comprises an operation attribute.

[0068] The judging whether the mark sequence meets the second condition comprises:

[0069] Determining the operation attribute of each breakpoint in the mark sequence according to the operation number corresponding to each breakpoint; if the operation attribute is a key operation, it is determined that the mark sequence does not meet the second condition, otherwise, it is determined that the mark sequence meets the second condition.

[0070] In the actual experimental situation, not all experimental steps are critical and dangerous, and appropriate adjustment of the order of some experimental steps will not have too much influence. In view of the actual situation, the present application further marks (i.e. operation attribute) each operation data in the standard operation data, and correspondingly, for each breakpoint in the mark sequence, if it is a key operation, it means that the operation of the critical and dangerous experimental step is not standardized, at which time a warning should be given; otherwise, it means that the non-standard operation only involves unimportant experimental steps, and no warning is needed.

[0071] Optionally, the operation data in the pair comprises action data and position data, the action data comprises palm operation data and experimental equipment operation data, and the position data comprises palm position sequence data and experimental equipment position sequence data.

[0072] The calculating the first similarity of each first operation data and the corresponding operation data in the standard operation data comprises:

[0073]

[0074] In the formula, sim represents the first similarity of the first operation data and the corresponding operation data in the standard operation data; Y i represents the i th data of the action data in the operation data, Y j represents the j th data of the position data in the operation data; represents the mean value of each data in the action data in the first operation data, represents the mean value of each data in the position data in the first operation data; m is the data amount of the action data, and n is the data amount of the position data.

[0075] In the embodiment of the present application, the palm operation data and the experimental equipment operation data are integrated as action data, and the palm position sequence data and the experimental equipment position sequence data are integrated as position data, so that the operation data pair constitutes first operation data, and then the first similarity of the corresponding operation data in the first operation data and the standard operation data is calculated, so that whether the current first operation matches the corresponding standard operation in the standard operation data can be analyzed.

[0076] It should be noted that the action data and the position data involved in the similarity calculation formula should be data after normalization processing.

[0077] Optionally, after the matching calculation result does not satisfy the first condition, the method further includes:

[0078] obtaining a second image of the main control experimental bench area, and extracting second operation data from the second image;

[0079] calculating a second similarity of the second operation data and the first operation data, and if the second similarity is greater than or equal to a second threshold value, the pre-warning scheme is not executed, otherwise the pre-warning scheme is executed.

[0080] In the embodiment of the present application, the standard operation data is generally instructive but not mandatory, so that the case of actual first operation data different from the standard operation data may also be reasonable and does not need to be pre-warned. In view of this actual situation, the second image of the main control experimental bench area is further obtained, and the second operation data of the instructor is extracted therefrom, and if the second similarity of the first operation data of the experimenter and the second operation data of the instructor is greater than or equal to a second threshold value, it is indicated that the experimenter is conducting the experiment according to the requirements of the instructor, although it does not completely conform to the standard operation data, but also belongs to the category of standard operation, and at this time, the pre-warning operation is not performed. The second similarity can be calculated in the same way as the first similarity, of course, other similarity calculation methods can also be used, and the present application does not make any limitation.

[0081] In addition, for this embodiment, the present application further provides the following improved scheme:

[0082] The operation attribute includes a key level.

[0083] The second image of the main control experimental bench area is obtained, including:

[0084] The second image of the main control experimental bench area in a preset time period is obtained, and the length of the preset time period is negatively correlated with the key level.

[0085] In this improved embodiment, critical operations also have a certain level of hierarchy. The higher the critical level, the higher the risk of the critical operation. Accordingly, the instructor will delay the experimental steps to give the experimenter more time to explain and understand. To address this situation, the present invention further determines the length of a preset period based on the critical level and extracts the instructor's second operation data from the second image corresponding to the preset period. Therefore, the higher the critical level of the critical operation, the longer the preset period of second operation data is obtained, thereby obtaining more comprehensive and accurate second operation data. Conversely, the shorter the preset period of second operation data is obtained to avoid obtaining second operation data from adjacent steps, thereby improving the accuracy of the second operation data.

[0086] The preset time period may be a time period obtained by tracing back from the time / time period corresponding to the first operation data as a starting point.

[0087] Example 2

[0088] See also Figure 2 , Figure 2 This is a schematic diagram of the structure of an intelligent experimental platform disclosed in an embodiment of the present invention. Figure 2 As shown, an intelligent experimental platform according to an embodiment of the present invention comprises a processing module (101), a storage module (102) and an acquisition module (103), wherein the processing module (101) is connected to the storage module (102) and the acquisition module (103) respectively; wherein,

[0089] The storage module (102) stores a computer program;

[0090] The acquisition module (103) is used to acquire image data of the experimental platform area and transmit the image data to the processing module (101);

[0091] The processing module (101) is used to call the computer program to implement the method described in the first embodiment.

[0092] The specific functions of an intelligent experimental platform in this embodiment refer to the above-mentioned embodiment 1. Since the intelligent experimental platform in this embodiment adopts all the technical solutions of the above-mentioned embodiments, it has at least all the beneficial effects brought by the technical solutions of the above-mentioned embodiments, which will not be described one by one here.

[0093] Example 3

[0094] See also Figure 3 , Figure 3 An electronic device disclosed in an embodiment of the present invention includes: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the method described in embodiment 1.

[0095] Embodiment Four

[0096] The embodiment of the present application further discloses a computer storage medium, which stores a computer program, and the computer program is executed by a processor to perform the method in the embodiment one.

[0097] The present application is described in reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device that realizes the functions specified in the flowcharts and / or block diagrams. Figure 1 one flow or multiple flows and / or blocks Figure 1 an apparatus that realizes the functions specified in one block or multiple blocks.

[0098] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing devices to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which realizes the functions specified in the flowcharts and / or block diagrams. Figure 1 one flow or multiple flows and / or blocks Figure 1 an apparatus that realizes the functions specified in one block or multiple blocks.

[0099] These computer program instructions can also be loaded to the computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable devices to produce a computer-implemented process, so that the instructions executed on the computer or other programmable devices provide a process for realizing the functions specified in the flowcharts and / or block diagrams. Figure 1 one flow or multiple flows and / or blocks Figure 1 an apparatus that realizes the functions specified in one block or multiple blocks.

[0100] The above description is only a specific embodiment of the present application, but the protection scope of the present application is not limited to this. Any changes or replacements within the technical range disclosed by the present application can be easily thought by those skilled in the art, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A pre-warning control method of an intelligent experiment bench, characterized in that, The method comprises: acquiring a first image of a smart experiment table region, and obtaining first operation data from the first image; performing matching calculation on the first operation data and standard operation data, and executing an early warning scheme if the matching calculation result does not satisfy a first condition; the standard operation data comprises a plurality of operation data, and the operation data comprises an operation serial number; the matching calculation on the first operation data and the standard operation data comprises: calculating a first similarity of each of the first operation data and the corresponding operation data in the standard operation data according to the operation serial number, and recording a mark if the first similarity is greater than or equal to a first threshold value, or otherwise recording a null mark; grouping a mark sequence according to the operation serial number and the mark record, and determining whether the mark sequence satisfies a second condition, and setting the first operation data as satisfying the first condition if yes, or otherwise setting the first operation data as not satisfying the first condition; the operation data further comprises an operation attribute; the determination of whether the mark sequence satisfies the second condition comprises: determining each breakpoint in the mark sequence, determining the operation attribute according to the operation serial number corresponding to each breakpoint, determining that the mark sequence does not satisfy the second condition if the operation attribute is a key operation, or otherwise determining that the mark sequence satisfies the second condition; after the matching calculation result does not satisfy the first condition, the method further comprises: acquiring a second image of a master experiment table region, and obtaining second operation data from the second image; calculating a second similarity of the second operation data and the first operation data, and not executing the early warning scheme if the second similarity is greater than or equal to a second threshold value, or otherwise executing the early warning scheme; the operation attribute comprises a key level; the acquisition of the second image of the master experiment table region comprises: acquiring the second image in a preset time period of the master experiment table region, wherein the length of the preset time period is negatively correlated with the key level. 2.The early warning control method of the intelligent experiment table according to claim 1, characterized in that: the obtaining of the first operation data from the first image comprises: recognizing a hand region and an experimental equipment region in the first image and performing continuous tracking to obtain palm operation data and palm position sequence data, experimental equipment operation data and experimental equipment position sequence data; performing position matching calculation on the palm position sequence data and the experimental equipment position sequence data to obtain a plurality of operation data pairs, and taking the plurality of operation data pairs as the first operation data. 3.The early warning control method of the intelligent experiment table according to claim 1 or 2, characterized in that: the standard operation data is obtained by: receiving experimental attribute data issued by a master experiment table, and determining the standard operation data according to the experimental attribute data; or extracting experimental equipment attributes from the first image, determining candidate experimental attribute data according to the experimental equipment attributes, outputting the candidate experimental attribute data to an experimenter, determining target experimental attribute data based on feedback operation of the experimenter, and determining the standard operation data according to the target experimental attribute data. 4.The early warning control method of the intelligent experiment table according to claim 1, characterized in that: The operation data pairs include action data and position data, the action data includes palm operation data and experimental equipment operation data, and the position data includes palm position sequence data and experimental equipment position sequence data; The first similarity between each of the first operation data and the corresponding operation data in the standard operation data is calculated, including: wherein sim represents a first similarity of the first operation data to corresponding operation data in the standard operation data; Y i represents an i-th data of action data in the operation data, j represents an i-th data of action data in the operation data, represents a mean of each data of action data in the first operation data, represents a mean of each data of position data in the first operation data; m is the data amount in the action data, and n is the data amount in the position data.

5. An intelligent experiment table, comprising a processing module, a storage module and an acquisition module, the processing module being connected with the storage module and the acquisition module respectively; wherein, The storage module stores a computer program; the acquisition module is configured to acquire image data of the experimental bench area and transmit the image data to the processing module; and the processing module is configured to call the computer program to implement the method according to any one of claims 1-4.

6. A computer storage medium having stored thereon a computer program, characterized in that: The computer program is run by the processor to implement the method according to any one of claims 1-4.

7. An electronic device comprising a processor and a memory, said memory having stored thereon a computer program, characterized in that: The computer program is run by the processor to implement the method according to any one of claims 1-4.

Images