Large instrument management method based on machine vision and Internet of Things technology

Abstract

The invention provides a large-scale instrument management method based on machine vision and the Internet of Things technology. An instrument reservation platform, acquisition equipment and large-scale instruments in a test room are included. The method further comprises the following steps: S1, acquiring instrument reservation range data; s2, acquiring indicator light data and current data of the instrument by using acquisition equipment; s3, judging the working state and the working mode of the instrument; s4, collecting operation video data; s5, judging and analyzing whether the operation video data exceed a reservation range or not; according to the method, the whole process of the experiment is analyzed, and finally all judgment results are generated into a test record and are stored. Management personnel can conveniently master the condition of the instrument at any time, traceability is improved, management loopholes are avoided, whether an operator carries out violation operation or not in the experiment process is conveniently tracked, the working state of the instrument is monitored, the whole experiment process is analyzed and recorded, the state of the instrument is mastered, people in charge can be conveniently found, and use and management safety of large instruments in a laboratory is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of instrument management, and in particular to a large-scale instrument management method based on machine vision and Internet of Things technologies. Background Art

[0002] At present, there are a large number of large instruments in laboratories. These instruments are usually expensive and complex equipment with multiple modules and components, which can be used to conduct a variety of experiments. However, due to frequent personnel changes in laboratories, especially university laboratories, many people are unable to use large instruments correctly, resulting in damage to many instrument components. Due to the complexity and high value of large instruments, improper use and maintenance may cause instrument failure and damage, which not only wastes resources, but also affects the progress of laboratory work. In this case, laboratory instrument management is seriously lagging behind, and there are the following problems: 1. Unable to understand the actual status of the instrument: With the traditional management method, it is impossible to automatically obtain instrument data, and the real-time status of the instrument cannot be effectively managed. Often, faulty instruments can only be discovered after daily inspections by personnel, and it is difficult to find the person who caused the damage.

[0003] 2. The usage process is not comprehensive: Traditional management methods can often only solve the management of instrument usage registration. It is unable to manage whether the experimenters conduct experiments according to the contents of the registered experiments, whether they conduct other experiments, whether there are any illegal operations during the experiment, etc., and there are obvious management loopholes. Summary of the invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a large-scale instrument management method based on machine vision and Internet of Things technology.

[0005] To achieve the above object, the present invention adopts the following technical solutions: A large-scale instrument management method based on machine vision and Internet of Things technology, including an instrument reservation platform, acquisition equipment, and various large-scale instruments in the laboratory; The following steps are also included: S1: Collect instrument reservation range data; According to the user's reservation or registration on the reservation platform, the name of the person making the reservation, the scheduled experiment, the scheduled instrument, and the reservation time period are obtained; S2: Use the acquisition device to collect the indicator light data and current data of the instrument; Install acquisition equipment at the mainboard and power supply of large instruments to collect the operating data of the instruments themselves and obtain indicator light data and current data; S3: Determine the working status and working mode of the instrument; According to the current data and indicator light data collected at a fixed frequency, the working status and working mode of the instrument can be confirmed; S4: Collecting operation video data; S5: judging and analyzing whether the operation video data exceeds the reservation range; By inputting the collected actions of the personnel and the equipment into the trained recognition model, the algorithm recognition judgment is performed to confirm whether the user has used the instrument correctly according to the appointment or registration; S51: Collect data to train recognition model; Collect videos of correct operation of large instruments and videos of common incorrect operation, decompose these correct and incorrect operation videos into frames, label them as samples, and input the samples as training data into the machine learning algorithm to train the corresponding recognition model; S52: Multidimensional data full process analysis; The experimental process is judged based on the combination of current data, indicator light data and operation video data recognition, and a test record is generated.

[0006] Furthermore, the instrument reservation platform includes an information database, which stores personnel information, instrument information, and instrument detection content information; The personnel information includes name, contact information, and image information; The instrument information includes the instrument name, instrument model and instrument code. The instrument code is the unique identification of the instrument. The instrument detection content information covers all working modes of each instrument and the current size information of each working mode.

[0007] Further, step S2 includes: S21: Collect instrument indicator light signals: Install a photoelectric sensor at the position of the indicator light on the mainboard of the large instrument to collect the color signal of the indicator light; S22: Collect instrument operation signals: Install smart sockets or Hall sensors at the front end of the power supply of large instruments to collect current information of the instrument operation; S23: Data processing: The signals collected by S21 and S22 are centrally transmitted to the central controller unit, which processes the signals and converts them into digital signals to obtain indicator light data and current data.

[0008] Further, in step S3, the indicator light data includes the color of the indicator light, including green, yellow, and red; wherein green indicates a normal working state, yellow indicates a fault state, and red indicates a dangerous state; the working state of the instrument is determined according to the indicator light data, and the working state is stored as a working state information library according to the order of acquisition time; the working state information library includes the instrument code, working state, and time; Obtain the instrument detection content information of the corresponding instrument in the information database, compare the obtained current data with the current data of the instrument in each working mode, compare the current size, obtain the working mode of the currently used instrument, and finally store the working mode as a working mode information library according to the order of collection time; the working mode information library includes instrument code, working mode and time.

[0009] Furthermore, cameras are installed at places or processes involving operation steps, handling of laboratory objects, and use of different components and accessories of instruments to capture and collect video data including facial information of personnel, movements of personnel, and movement operation of equipment; Assign a unique device identifier to each camera, set the unique device identifier in the camera configuration, encode the device identifier in the collected operation video data, and establish a corresponding code table between the device identifier and the collected instrument code.

[0010] Further, step S52 includes: S521: Obtaining instrument reservation range data; Before the experiment, the name of the person making the reservation, the experiment to be scheduled, the instrument to be scheduled, and the reservation time period are obtained from the instrument reservation platform; the scheduled experiment includes the experimental content and the instrument detection content required; the instrument detection content is the working mode of the instrument; S522: Identify whether the experimenter and the reservation person are the same person; Collect the facial information of the people in the reservation time period experiment, use visual face level technology to compare the collected facial information of the people with the image information in the personnel information, identify the image information of the operator, obtain the name of the operator according to the image information, and compare the name of the operator with the name of the reservation person to determine whether they are the same person; S523: Determine whether the operated instrument is consistent with the reserved instrument; First, according to the scheduled time period, the working status and working mode of all instruments in the corresponding time period in the working status information database and the working mode information database are obtained respectively, and the codes of all instruments running in the scheduled time period are obtained; Next, according to the established corresponding code table, the device identifier of the corresponding camera is obtained according to the obtained running instrument code, the operation video data containing the device identifier is obtained, the operation video data is cropped according to time, and the operation video data within the scheduled time period is obtained; Then, according to the image information of the operator identified in S522, the visual face recognition technology is used to split the video into single-frame images and then perform face recognition. Face detection is performed in each frame to identify the person in the video, and the video containing the operator is screened out. The time interval containing the operator is marked in the video, and the time interval is cropped to obtain valid video data; Next, the valid video data is input into the trained recognition model to determine the instrument actually operated by the appointment person; Finally, the actual operated instrument is compared with the reserved instrument, and the instrument codes are compared to determine whether the operated instrument is consistent with the reserved instrument. If not, the instrument beyond the reservation range is marked; according to the actual operated instrument, the working mode in the working mode information library is obtained, and compared with the instrument detection contents required in the reserved experiment to determine whether they are the same. If they are the same, proceed to the next step. If they are different, the instrument detection contents beyond the reservation range are marked; S524: Identify and judge whether the operation of the experimenter during the experiment is correct; Input the effective video data into the trained recognition model to obtain the recognition model output result, and determine whether the operation process is correct based on the recognition result; if it is an incorrect operation, identify the instrument with the incorrect operation; S525: After the experiment, determine whether to turn off the power of the instrument; Obtain the actual operating current data and determine whether to turn off the instrument power supply according to the current size; S526: Generate experimental records; Built-in experiment record template, call the experiment record template, write the judgment result into the experiment record template, generate the experiment record and store it.

[0011] Furthermore, the experiment record template includes whether it is a reservation personnel area, whether it is a reservation instrument area, whether it is a reservation experiment area, whether it is a correct operation area, and whether the power is turned off area; According to the judgment result of S522, write yes or no in the area of ​​whether it is a reserved person, and according to the judgment result of S523, write yes or no in the area of ​​whether it is a reserved instrument. If it is no, write the instrument beyond the reservation range into the area of ​​whether it is a reserved instrument, and record the instrument code of the instrument beyond the reservation range. According to the result of the determination of whether the working mode exceeds the instrument detection content within the reservation range, write yes or no in the experimental area of ​​whether it is a reservation, and if it is no, write the instrument detection content that exceeds the reservation range that is marked into the experimental area of ​​whether it is a reservation; According to the determination result of S524, yes or no is written in the whether the operation is correct area, and the instrument marked as an operation error is written in the whether the operation is correct area; According to the determination result of S525, yes or no is written in the whether to turn off the power area.

[0012] Further, the video shows common operating errors including not wearing personal protective equipment, not preheating and calibrating the instrument, not properly disposing of waste, and using an electronic balance without closing the draft shield and touching the sample pan with bare hands.

[0013] Furthermore, the machine learning algorithm selects a recurrent neural network.

[0014] Compared with the prior art, the present invention has the following beneficial effects: (1) The method of the present invention combines the management of large-scale instruments with machine vision technology and Internet of Things technology, realizes the management of the data of the large-scale instrument itself through the Internet of Things technology, and realizes the management of the process of the large-scale instrument operation through machine vision technology, thereby improving the use and management safety and management efficiency of large-scale instruments in the laboratory. (2) The present invention analyzes the whole process of the experiment, and finally generates and stores the test records of all the judgment results. It is convenient for the management personnel to grasp the situation of the instrument at any time, improve traceability, avoid management loopholes, and conveniently track whether the operator has violated the rules during the experiment, monitor the working status of the instrument, analyze and record the whole process of the experiment, grasp the status of the instrument, and facilitate the search for the person responsible. (3) During the whole process analysis, the present invention processes and filters out valid data from the operation video data collected within the appointment time period, reduces the processing volume of subsequent model recognition, improves the speed of processing and judgment, meets the market demand of public laboratories, and can improve the accuracy of recognition and avoid misrecognition of recognition models. Since the recognition model can only recognize operating actions but cannot distinguish the operations of a specific operator and cannot determine whether it exceeds the reservation range, the present invention screens valid video data, accurately determines whether it exceeds the reservation range, effectively obtains information, simplifies the requirements of the tracking algorithm through visual face recognition technology, and greatly reduces the demand for model training samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The present invention is a flowchart of the steps of a large-scale instrument management method based on machine vision and Internet of Things technology. DETAILED DESCRIPTION

[0016] In order to provide a further understanding of the purpose, structure, features, and functions of the present invention, the following detailed description is given in conjunction with the embodiments.

[0017] A large-scale instrument management method based on machine vision and Internet of Things technology, including instrument reservation platform, acquisition equipment, and various large-scale instruments in the laboratory; The instrument reservation platform may be an instrument reservation level platform or a project management platform.

[0018] The instrument reservation platform includes an information database, which stores personnel information, instrument information, instrument detection content information, etc. The instrument detection content information covers all working modes of each instrument, the current size information of each working mode, etc.

[0019] Personnel information includes name, contact information, image information, etc.

[0020] The instrument information includes the instrument name, instrument model, instrument code, etc. The instrument code is the unique identification of the instrument.

[0021] Collection equipment: photoelectric sensors, current sensors and other IoT devices.

[0022] The following steps are also included: S1: Collect instrument reservation range data; Users can make reservations or register for the large instruments they need to use on the instrument reservation level platform or project management platform. When making reservations or registering, they can fill in the information including the user, the user object, the usage content, the required auxiliary materials, the experimental content to be conducted, and the instrument detection content to be used. According to the reservation or registration, the name of the person making the reservation, the scheduled experiment, the scheduled instrument, the reservation time period and other information can be obtained; S2: Use the acquisition device to collect the indicator light data and current data of the instrument; Technicians install IoT devices such as photoelectric sensors and current sensors on the mainboards and power supplies of large instruments to collect the operating data of the instruments themselves.

[0023] S21: Collect instrument indicator light signals: By installing photoelectric sensors at the indicator lights and other locations on the mainboard of large instruments, the color signals of the indicator lights can be collected; S22: Collect instrument operation signals: By installing smart sockets, Hall sensors and other devices at the front end of the power supply of large instruments, the current information of the instrument operation can be collected; S23: Data processing: The signals collected by S21 and S22 are centrally transmitted to the central controller unit, which processes the signals and converts them into digital signals to obtain indicator light data and current data.

[0024] S3: Determine the working status and working mode of the instrument; According to the current data and indicator light data collected at a fixed frequency, the working status and working mode of the instrument can be confirmed. The fixed frequency can be collected once every 30 seconds.

[0025] The indicator light data includes the color of the indicator light, including green, yellow, and red. Green indicates normal working state, yellow indicates fault state, and red indicates dangerous state. The working state of the instrument is determined based on the indicator light data. Finally, the working state is stored as a working state information library according to the order of acquisition time. The working state information library includes instrument code, working state, and time.

[0026] The current data is the magnitude of the current. The instrument detection content information of the corresponding instrument in the information database is obtained, and the obtained current data is compared with the current data of the instrument in each working mode. The current magnitudes of the two are compared and analyzed to obtain the working mode of the currently used instrument. Finally, the working modes are stored as a working mode information library according to the order of acquisition time. The working mode information library includes instrument code, working mode, and time.

[0027] The present invention automatically acquires instrument data through the Internet of Things acquisition equipment, effectively manages the instrument status and working mode, facilitates the monitoring of the operation of the instrument, and has strong time traceability, convenient accountability, and easy to find the person who damaged the instrument.

[0028] Taking the fume hood as an example, the current size is analyzed. If it is less than 100 mA, it is off. If it is greater than 100 and less than 500 mA, the lighting function is enabled. If it is greater than 500 mA, the fan is enabled. The green signal light indicates normal working state, yellow indicates fault state, and red indicates dangerous state. The temperature or smoke in the fume hood exceeds the specification, and there is a risk of fire or smoke leakage.

[0029] S4: Collecting operation video data; The operation video data includes operator operation data or instrument motion data; Install cameras at places or processes involving operation steps, handling of laboratory objects, and use of different components and accessories of instruments to capture and collect operational video data including facial information of personnel, movements of personnel, movements of equipment, etc.

[0030] Assign a unique device identifier to each camera, set the unique device identifier in the camera configuration, encode the device identifier in the collected operation video data, and establish a corresponding code table between the device identifier and the collected instrument code.

[0031] S5: judging and analyzing whether the operation video data exceeds the reservation range; By collecting the actions of people and equipment, the trained recognition model is input to perform algorithm recognition and judgment to confirm whether the user has used the instrument correctly according to the appointment or registration.

[0032] S51: Collect data to train recognition model; Collect videos of correct operation of large instruments and videos of common incorrect operation, such as: not wearing personal protective equipment, not preheating and calibrating the instrument, not properly handling waste, using an electronic balance without closing the windshield and touching the sample tray directly with hands, etc. Decompose these videos of correct and incorrect operation into frames of pictures and label them accordingly as samples. Input the samples as training data into the machine learning algorithm to train the corresponding recognition model. The recognition model training optimization and its algorithm are not the midpoint of the present invention. The present invention only uses the machine learning algorithm in the prior art and does not innovate the machine learning algorithm. Therefore, the recognition model training optimization and its algorithm are not elaborated.

[0033] The machine learning algorithm can choose recurrent neural network. This algorithm is conducive to processing and analyzing time series data, can retain and process the information of the previous time step, has memory characteristics, can be used to analyze the time series relationship between video frames, can effectively model complex time series dynamics, and more accurately identify behaviors; it can be applied in real-time or online learning scenarios for immediate response and judgment.

[0034] S52: Multidimensional data full process analysis; The current data, indicator light data and operation video data are combined through the Internet of Things technology to judge the experimental process and generate experimental records. The present invention combines video data, appointment data, Internet of Things data, multi-dimensional data, and multi-modal judgment.

[0035] S521: Obtaining instrument reservation range data; Before the experiment, obtain the name of the person making the reservation, the scheduled experiment, the scheduled instrument, and the reservation time period from the instrument reservation platform.

[0036] The scheduled experiment includes the experimental content and the instruments needed to test the content.

[0037] S522: Identify whether the experimenter and the reservation person are the same person; Collect facial information of people in the experiment during the appointment time period, use visual face level technology to compare the collected facial information with the image information in the personnel information, identify the image information of the operator, obtain the name of the operator based on the image information, and compare the operator's name with the name of the obtained appointment person to determine whether they are the same person.

[0038] S523: Determine whether the operated instrument is consistent with the reserved instrument; First, the working status and working mode of all instruments in the corresponding time period in the working status information library and the working mode information library are obtained according to the scheduled time period, and the codes of all running instruments in the scheduled time period are obtained.

[0039] Next, according to the established corresponding code table, the device identifier of the corresponding camera is obtained according to the obtained running instrument code, the operation video data containing the device identifier is obtained, the operation video data is cropped according to time, and the operation video data within the scheduled time period is obtained. According to the actual situation, the instrument running in this period is searched, and the video data of the running instrument is obtained. First, the instrument range is screened to reduce the waste of unnecessary video data, and at the same time, the influence of complex scenes of personnel actions in public laboratories on the judgment results is reduced, which is convenient for analyzing the whole process of experimental use.

[0040] Then, according to the image information of the operator identified in S522, the visual face recognition technology is used to split the video into single-frame images, and then the existing deep learning model (including Haar Cascades of OpenCV, Dlib or more advanced FaceNet and DeepFace, etc.) is used for face recognition, and face detection is run in each frame to identify the person in the video, screen out the video containing the operator, and mark the time interval containing the operator in the video, and cut the time interval to obtain valid video data. In some public laboratories, there are many people. By first screening the effective video data in the collected video, the processing amount of subsequent model recognition is reduced, the speed of processing and judgment is improved, the market demand of public laboratories is met, and the accuracy of recognition can be improved to avoid misidentification of the recognition model. Since the recognition model can only recognize operating actions, it cannot distinguish the operations of a specific operator and cannot determine whether it exceeds the reservation range. The present invention screens the effective video data, accurately determines whether it exceeds the reservation range, and effectively obtains information. The present invention simplifies the requirements of the tracking algorithm and greatly reduces the demand for model training samples.

[0041] Next, the valid video data is input into the trained recognition model to determine the instrument actually operated by the appointment person.

[0042] Finally, compare the actual instrument with the scheduled instrument, and compare the instrument codes of the two to determine whether the instrument is consistent with the scheduled instrument. If not, identify the instrument that is out of the scheduled range. According to the actual instrument, obtain the working mode in the working mode information library, and compare it with the instrument detection content required in the scheduled experiment to determine whether they are the same. If they are the same, proceed to the next step. If they are different, identify the instrument detection content (i.e., working mode) that is out of the scheduled range and proceed to the next step.

[0043] S524: Identify and determine whether the experimenter's operation during the experiment is correct.

[0044] Input the effective video data into the trained recognition model, obtain the recognition model output result, and determine whether the operation process is correct based on the recognition result. If it is an incorrect operation, identify the instrument with the incorrect operation.

[0045] S525: After the experiment, determine whether to turn off the power of the instrument.

[0046] The current data of the actual operating instrument smart socket is obtained, and whether to turn off the instrument power supply is determined according to the current size.

[0047] S526: Generate experimental records; Built-in experiment record template, the experiment record template includes whether it is an appointment personnel area, whether it is an appointment instrument area, whether it is an appointment experiment area, whether the operation is correct area and whether the power is turned off area.

[0048] Call the experiment record template, write yes or no in the area of ​​whether it is a reservation personnel according to the judgment result of S522, write yes or no in the area of ​​whether it is a reservation instrument according to the judgment result of S523, if it is no, write the instrument that is beyond the reservation range into the area of ​​whether it is a reservation instrument, and record the instrument code of the instrument beyond the reservation range. According to the judgment result of whether the working mode exceeds the reservation range of the instrument detection content, write yes or no in the experimental area of ​​whether it is a reservation, if it is no, write the instrument detection content that is beyond the reservation range into the experimental area of ​​whether it is a reservation. According to the judgment result of S524, write yes or no in the area of ​​whether the operation is correct, and write the instrument marked as an operation error into the area of ​​whether the operation is correct. According to the judgment result of S525, write yes or no in the area of ​​whether to turn off the power. Finally, all the judgment results are generated into test records and stored. It is convenient for managers to grasp the situation of the instrument at any time, improve traceability, avoid management loopholes, and conveniently track whether the operator has violated the rules during the experiment, monitor the working status of the instrument, analyze and record the whole process of the experiment, grasp the status of the instrument, and find the person responsible.

[0049] The method of the present invention combines the management of large-scale instruments with machine vision technology and Internet of Things technology, manages the data of the large-scale instrument's own operation through the Internet of Things technology, and manages the process of the large-scale instrument's operation through machine vision technology, thereby improving the use and management safety and management efficiency of large-scale instruments in the laboratory.

[0050] The present invention discloses an application of machine vision and Internet of Things technology in the operation and management of large-scale instruments, and its excellent application in the use safety, use process, personnel authentication, etc. of large-scale instruments. It not only embodies the perfect combination of technology and creativity, but also is a major breakthrough in the management of large-scale instruments. By constructing an algorithm model of the use process of large-scale instruments and cleverly combining machine vision technology, the use process is managed, and at the same time, the operation data of large-scale instruments is obtained through Internet of Things technology, and the management efficiency and safety of large-scale instruments are improved through machine vision and Internet of Things technology.

[0051] The present invention has been described by the above-mentioned relevant embodiments, however, the above-mentioned embodiments are only examples for implementing the present invention. It must be pointed out that the disclosed embodiments do not limit the scope of the present invention. On the contrary, changes and modifications made without departing from the spirit and scope of the present invention are all within the scope of patent protection of the present invention.

Claims

1. A large-scale instrument management method based on machine vision and Internet of Things technology, characterized by: Including instrument reservation platform, collection equipment, and various large instruments in the laboratory; The following steps are also included: S1: Collect instrument reservation range data; According to the user's reservation or registration on the reservation platform, the name of the person making the reservation, the scheduled experiment, the scheduled instrument, and the reservation time period are obtained; S2: Use the acquisition device to collect the indicator light data and current data of the instrument; Install acquisition equipment at the mainboard and power supply of large instruments to collect the operating data of the instruments themselves and obtain indicator light data and current data; S3: Determine the working status and working mode of the instrument; According to the current data and indicator light data collected at a fixed frequency, the working status and working mode of the instrument can be confirmed; S4: Collecting operation video data; S5: judging and analyzing whether the operation video data exceeds the reservation range; By inputting the collected actions of the personnel and the equipment into the trained recognition model, the algorithm recognition judgment is performed to confirm whether the user has used the instrument correctly according to the appointment or registration; S51: Collect data to train recognition model; Collect videos of correct operation of large instruments and videos of common incorrect operation, decompose these correct and incorrect operation videos into frames, label them as samples, and input the samples as training data into the machine learning algorithm to train the corresponding recognition model; S52: Multidimensional data full process analysis; The experimental process is judged based on the combination of current data, indicator light data and operation video data recognition, and a test record is generated.

2. A large-scale instrument management method based on machine vision and Internet of Things technology as claimed in claim 1, characterized in that: The instrument reservation platform includes an information database, which stores personnel information, instrument information, and instrument detection content information; The personnel information includes name, contact information, and image information; The instrument information includes the instrument name, instrument model and instrument code. The instrument code is the unique identification of the instrument. The instrument detection content information covers all working modes of each instrument and the current size information of each working mode.

3. A large-scale instrument management method based on machine vision and Internet of Things technology as claimed in claim 1, characterized in that: Step S2 includes: S21: Collect instrument indicator light signals: Install a photoelectric sensor at the position of the indicator light on the mainboard of the large instrument to collect the color signal of the indicator light; S22: Collect instrument operation signals: Install smart sockets or Hall sensors at the front end of the power supply of large instruments to collect current information of the instrument operation; S23: Data processing: The signals collected by S21 and S22 are centrally transmitted to the central controller unit, which processes the signals and converts them into digital signals to obtain indicator light data and current data.

4. A large-scale instrument management method based on machine vision and Internet of Things technology as claimed in claim 2, characterized in that: In step S3, the indicator light data includes the color of the indicator light, including green, yellow, and red; green indicates a normal working state, yellow indicates a fault state, and red indicates a dangerous state; the working state of the instrument is determined according to the indicator light data, and the working state is stored as a working state information library according to the order of acquisition time; the working state information library includes the instrument code, working state, and time; Obtain the instrument detection content information of the corresponding instrument in the information database, compare the obtained current data with the current data of the instrument in each working mode, compare the current size, obtain the working mode of the currently used instrument, and finally store the working mode as a working mode information library according to the order of collection time; the working mode information library includes instrument code, working mode and time.

5. A large-scale instrument management method based on machine vision and Internet of Things technology as claimed in claim 4, characterized in that: Install cameras at places or processes involving operation steps, handling of laboratory objects, and use of different components and accessories of instruments to capture and collect video data including facial information of personnel, movements of personnel, and movement operation of equipment; Assign a unique device identifier to each camera, set the unique device identifier in the camera configuration, encode the device identifier in the collected operation video data, and establish a corresponding code table between the device identifier and the collected instrument code.

6. A large-scale instrument management method based on machine vision and Internet of Things technology as claimed in claim 5, characterized in that: Step S52 includes: S521: Obtaining instrument reservation range data; Before the experiment, the name of the person making the reservation, the experiment to be scheduled, the instrument to be scheduled, and the reservation time period are obtained from the instrument reservation platform; the scheduled experiment includes the experimental content and the instrument detection content required; the instrument detection content is the working mode of the instrument; S522: Identify whether the experimenter and the reservation person are the same person; Collect the facial information of the people in the reservation time period experiment, use visual face level technology to compare the collected facial information of the people with the image information in the personnel information, identify the image information of the operator, obtain the name of the operator according to the image information, and compare the name of the operator with the name of the reservation person to determine whether they are the same person; S523: Determine whether the operated instrument is consistent with the reserved instrument; First, according to the scheduled time period, the working status and working mode of all instruments in the corresponding time period in the working status information database and the working mode information database are obtained respectively, and the codes of all instruments running in the scheduled time period are obtained; Next, according to the established corresponding code table, the device identifier of the corresponding camera is obtained according to the obtained running instrument code, the operation video data containing the device identifier is obtained, the operation video data is cropped according to time, and the operation video data within the scheduled time period is obtained; Then, according to the image information of the operator identified in S522, the visual face recognition technology is used to split the video into single-frame images and then perform face recognition. Face detection is performed in each frame to identify the person in the video, and the video containing the operator is screened out. The time interval containing the operator is marked in the video, and the time interval is cropped to obtain valid video data; Next, the valid video data is input into the trained recognition model to determine the instrument actually operated by the appointment person; Finally, the actual operated instrument is compared with the reserved instrument, and the instrument codes are compared to determine whether the operated instrument is consistent with the reserved instrument. If not, the instrument beyond the reservation range is marked; according to the actual operated instrument, the working mode in the working mode information library is obtained, and compared with the instrument detection contents required in the reserved experiment to determine whether they are the same. If they are the same, proceed to the next step. If they are different, the instrument detection contents beyond the reservation range are marked; S524: Identify and judge whether the operation of the experimenter during the experiment is correct; Input the effective video data into the trained recognition model to obtain the recognition model output result, and determine whether the operation process is correct based on the recognition result; if it is an incorrect operation, identify the instrument with the incorrect operation; S525: After the experiment, determine whether to turn off the power of the instrument; Obtain the actual operating current data and determine whether to turn off the instrument power supply according to the current size; S526: Generate experimental records; Built-in experiment record template, call the experiment record template, write the judgment result into the experiment record template, generate the experiment record and store it.

7. A large-scale instrument management method based on machine vision and Internet of Things technology as claimed in claim 6, characterized in that: The experiment record template includes whether it is the appointment personnel area, whether it is the appointment instrument area, whether it is the appointment experiment area, whether the operation is correct area, and whether the power is turned off area; According to the judgment result of S522, write yes or no in the area of ​​whether it is a reserved person, and according to the judgment result of S523, write yes or no in the area of ​​whether it is a reserved instrument. If it is no, write the instrument beyond the reservation range into the area of ​​whether it is a reserved instrument, and record the instrument code of the instrument beyond the reservation range. According to the result of the determination of whether the working mode exceeds the instrument detection content within the reservation range, write yes or no in the experimental area of ​​whether it is a reservation, and if it is no, write the instrument detection content that exceeds the reservation range that is marked into the experimental area of ​​whether it is a reservation; According to the determination result of S524, yes or no is written in the whether the operation is correct area, and the instrument marked as an operation error is written in the whether the operation is correct area; According to the determination result of S525, yes or no is written in the whether to turn off the power area.

8. A large-scale instrument management method based on machine vision and Internet of Things technology as claimed in claim 1, characterized in that: The video shows common operating errors including not wearing personal protective equipment, not warming up and calibrating the instrument, not properly disposing of waste, and using an electronic balance without closing the draft shield and touching the sample pan with bare hands.

9. A large-scale instrument management method based on machine vision and Internet of Things technology as claimed in claim 1, characterized in that: The machine learning algorithm selected is a recurrent neural network.

Images