Experiment compliance detection method and device based on artificial intelligence

Abstract

The embodiment of the invention discloses an artificial intelligence-based experiment compliance detection method and device, and the method comprises the steps: selecting a target detection object in response to a detection instruction, obtaining a target detection region corresponding to the target detection object, and collecting the image data of the target detection region; preprocessing the image data, extracting a feature region from the preprocessed image data, identifying the feature region, and comparing key features of the feature region with preset reference features to obtain a comparison result; evaluating an experiment compliance index of the target detection object based on the comparison result, and generating alarm information when the experiment compliance index is lower than a preset standard; according to the embodiment of the invention, compliance detection of the experiment process is realized in an automatic and intelligent manner, the accuracy and efficiency of the system can be further improved, the compliance of the experiment is ensured, the safety risk is reduced, and the experiment quality is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an artificial intelligence-based experimental compliance detection method and device. Background Art

[0002] Currently, in scenarios where students take lab exams, ensuring the safety of all participants is a basic and crucial task. To achieve this goal, students must correctly wear appropriate personal protective equipment, including lab coats, gloves, goggles, etc. However, traditional safety inspection methods mainly rely on manual supervision, which is not only inefficient but also easily affected by subjective factors, which may lead to safety hazards.

[0003] At the same time, in the teaching and examination of physical, chemical and biological experiments, many experimental operation steps often carry certain risks and the operation needs to ensure compliance to ensure experimental safety and personnel safety. However, in the actual operation process, it is often difficult for teachers to monitor the experimental operations of all students at the same time, resulting in some dangerous operations or non-compliant behaviors that are difficult to detect and deal with in time. This not only increases the risk of the experiment, but may also affect the accuracy of the experimental results and the improvement of students' experimental skills. At present, artificial intelligence methods based on posture estimation can determine whether some actions and behaviors meet the requirements. This method can accurately identify and track key parts of the human body, such as the movements of hands and arms, monitor students' experimental operations in real time, quickly identify potential non-compliant behaviors, and provide immediate feedback. Although posture estimation technology has made great progress, it still has limitations in the recognition of some complex actions, especially in laboratory environments. If fine hand movements or special experimental equipment are involved, the recognition accuracy may be affected. Summary of the invention

[0004] In response to the above-mentioned defects, an embodiment of the present invention discloses an artificial intelligence-based experimental compliance detection method and device, which have high recognition accuracy.

[0005] The first aspect of the embodiment of the present invention discloses an artificial intelligence-based experimental compliance detection method, comprising:

[0006] In response to a detection instruction, a target detection object is selected, a target detection area corresponding to the target detection object is acquired, and image data of the target detection area is collected; the detection instruction includes image detection and behavior detection;

[0007] Preprocessing the image data, extracting a feature region from the preprocessed image data, identifying the feature region, and comparing key features of the feature region with preset reference features to obtain a comparison result;

[0008] An experimental compliance index of the target detection object is evaluated based on the comparison result, and an alarm message is generated when the experimental compliance index is lower than a preset standard.

[0009] As an optional implementation manner, in the first aspect of the embodiment of the present invention, preprocessing the image data includes:

[0010] Denoising the image data based on bilateral filtering;

[0011] Perform contrast adjustment on the denoised image data;

[0012] An interference area in the image data is identified and the interference area is cropped.

[0013] As an optional implementation manner, in the first aspect of the embodiment of the present invention, extracting a feature region from the preprocessed image data and identifying the feature region includes:

[0014] Convert the preprocessed image data from the RGB color space to the HSV color space or the LAB color space, and calculate the color histogram or color mean of the image data;

[0015] Acquire the color distribution of the image data based on the color histogram or the color mean, and select a target color distribution area consistent with a preset color from the image data according to the color distribution;

[0016] Calculate image texture features in the target color distribution area.

[0017] As an optional implementation manner, in the first aspect of the embodiment of the present invention, comparing the key features of the feature area with the preset reference features includes:

[0018] The image texture feature is compared with a preset image texture feature.

[0019] As an optional implementation manner, in the first aspect of the embodiment of the present invention, extracting a feature region from the preprocessed image data and identifying the feature region includes:

[0020] Input the preprocessed image data into the YOLO target detection model to extract feature areas;

[0021] Generate a segmentation mask image of the feature area based on the image segmentation technology to obtain the position information of the feature area, wherein the position information includes the center coordinates, the boundary and the area;

[0022] The preprocessed image data is input into a preset deep convolutional learning model to extract the feature areas in the image data and identify the target actions in the feature areas.

[0023] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the key feature of the feature area is compared with the preset reference feature, including:

[0024] The position information is compared with a preset position feature, and / or the target action is compared with a reference action; the preset reference feature includes a preset position feature and a reference action.

[0025] As an optional implementation, in the first aspect of the embodiment of the present invention, evaluating the experimental compliance index of the target detection object based on the comparison result includes:

[0026] Generate preset evaluation indexes and evaluation weights for different comparison items; the comparison items include location information, target actions and key features;

[0027] Based on the comparison results, the evaluation index corresponding to each comparison item is obtained respectively;

[0028] Obtain the compliance index corresponding to each comparison item according to the formula F1*A1+F2*A2+F3*A3=S, where F1 is the evaluation index of the location information, F2 is the evaluation index of the target action, F3 is the evaluation index of the key feature, A1 is the evaluation weight corresponding to the location information, A2 is the evaluation index corresponding to the target action, and A3 is the evaluation index corresponding to the key feature;

[0029] Set the maximum compliance index and the minimum compliance index according to the formula (SL min ) / (L max -L min ) The compliance index is standardized to obtain the experimental compliance index of the target detection object, and the L max is the maximum compliance index, L min is the minimum compliance index.

[0030] A second aspect of an embodiment of the present invention discloses an artificial intelligence-based experimental compliance detection device, comprising:

[0031] Image acquisition module: used to select a target detection object in response to a detection instruction, obtain a target detection area corresponding to the target detection object, and collect image data of the target detection area; the detection instruction includes image detection and behavior detection;

[0032] Feature comparison module: used for preprocessing the image data, extracting feature areas from the preprocessed image data, identifying the feature areas, comparing key features of the feature areas with preset reference features, and obtaining comparison results;

[0033] Compliance calculation module: used to evaluate the experimental compliance index of the target detection object based on the comparison result, and generate an alarm message when the experimental compliance index is lower than a preset standard.

[0034] As an optional implementation manner, in a second aspect of the embodiment of the present invention, preprocessing the image data includes:

[0035] Denoising the image data based on bilateral filtering;

[0036] Perform contrast adjustment on the denoised image data;

[0037] An interference area in the image data is identified and the interference area is cropped.

[0038] As an optional implementation manner, in the second aspect of the embodiment of the present invention, extracting a feature region from the preprocessed image data and identifying the feature region includes:

[0039] Convert the preprocessed image data from the RGB color space to the HSV color space or the LAB color space, and calculate the color histogram or color mean of the image data;

[0040] Acquire the color distribution of the image data based on the color histogram or the color mean, and select a target color distribution area consistent with a preset color from the image data according to the color distribution;

[0041] Calculate image texture features in the target color distribution area.

[0042] As an optional implementation manner, in the second aspect of the embodiment of the present invention, comparing the key features of the feature area with the preset reference features includes:

[0043] The image texture feature is compared with a preset image texture feature.

[0044] As an optional implementation manner, in the second aspect of the embodiment of the present invention, extracting a feature region from the preprocessed image data and identifying the feature region includes:

[0045] Input the preprocessed image data into the YOLO target detection model to extract feature areas;

[0046] Generate a segmentation mask image of the feature area based on the image segmentation technology to obtain the position information of the feature area, wherein the position information includes the center coordinates, the boundary and the area;

[0047] The preprocessed image data is input into a preset deep convolutional learning model to extract the feature areas in the image data and identify the target actions in the feature areas.

[0048] As an optional implementation manner, in the second aspect of the embodiment of the present invention, the key feature of the feature area is compared with the preset reference feature, including:

[0049] The position information is compared with a preset position feature, and / or the target action is compared with a reference action; the preset reference feature includes a preset position feature and a reference action.

[0050] As an optional implementation, in the second aspect of the embodiment of the present invention, evaluating the experimental compliance index of the target detection object based on the comparison result includes:

[0051] Generate preset evaluation indexes and evaluation weights for different comparison items; the comparison items include location information, target actions and key features;

[0052] Based on the comparison results, the evaluation index corresponding to each comparison item is obtained respectively;

[0053] Obtain the compliance index corresponding to each comparison item according to the formula F1*A1+F2*A2+F3*A3=S, where F1 is the evaluation index of the location information, F2 is the evaluation index of the target action, F3 is the evaluation index of the key feature, A1 is the evaluation weight corresponding to the location information, A2 is the evaluation index corresponding to the target action, and A3 is the evaluation index corresponding to the key feature;

[0054] Set the maximum compliance index and the minimum compliance index according to the formula (SL min ) / (L max -L min ) The compliance index is standardized to obtain the experimental compliance index of the target detection object, and the L max is the maximum compliance index, L min is the minimum compliance index.

[0055] The third aspect of an embodiment of the present invention discloses an electronic device, comprising: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the artificial intelligence-based experimental compliance detection method disclosed in the first aspect of the embodiment of the present invention.

[0056] A fourth aspect of an embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the artificial intelligence-based experimental compliance detection method disclosed in the first aspect of an embodiment of the present invention.

[0057] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0058] In the embodiment of the present invention, when a detection instruction is received, the type of detection (image detection or behavior detection) is first determined, and the corresponding target detection object is selected according to the detection type. After the target detection object is determined, its corresponding detection area is obtained, and the collected image data is preprocessed, including denoising, contrast enhancement, color correction, etc. The extracted key features are compared with the preset benchmark features. According to the comparison results, the experimental compliance index of the target detection object is evaluated. The embodiment realizes the compliance detection of the experimental process in an automated and intelligent manner, and the accuracy and efficiency of the system can be further improved by optimizing the steps of instruction parsing, image acquisition, preprocessing, feature extraction, comparison evaluation and alarm information generation. This will help ensure the compliance of the experiment, reduce safety risks, and improve the quality of the experiment. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0060] Figure 1 It is a flowchart of an artificial intelligence-based experimental compliance detection method disclosed in an embodiment of the present invention;

[0061] Figure 2 It is a structural schematic diagram of an artificial intelligence-based experimental compliance detection device provided in an embodiment of the present invention;

[0062] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] It should be noted that the terms "first", "second", "third", "fourth", etc. in the specification and claims of the present invention are used to distinguish different objects rather than to describe a specific order. The terms "including" and "having" in the embodiments of the present invention and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0065] The embodiments of the present invention disclose an experimental compliance detection method, device, electronic device and storage medium based on artificial intelligence. In the embodiments, when a detection instruction is received, the type of detection (image detection or behavior detection) is first determined, and the corresponding target detection object is selected according to the detection type. After the target detection object is determined, its corresponding detection area is obtained, and the collected image data is preprocessed, including denoising, contrast enhancement, color correction, etc. The extracted key features are compared with the preset benchmark features. According to the comparison results, the experimental compliance index of the target detection object is evaluated. The embodiments realize the compliance detection of the experimental process in an automated and intelligent manner. By optimizing the steps of instruction parsing, image acquisition, preprocessing, feature extraction, comparison evaluation and alarm information generation, the accuracy and efficiency of the system can be further improved. This will help ensure the compliance of the experiment, reduce safety risks, and improve the quality of the experiment.

[0066] Embodiment 1

[0067] See also Figure 1 , Figure 1 It is a flow chart of the XX method disclosed in the embodiment of the present invention. Among them, the execution subject of the method described in the embodiment of the present invention is an execution subject composed of software and / or hardware, and the execution subject can receive relevant information by wired or / and wireless means, and can send certain instructions. Of course, it can also have certain processing functions and storage functions. The execution subject can control multiple devices, such as a remote physical server or cloud server and related software, or it can be a local host or server and related software that performs related operations on a device placed somewhere. In some scenarios, multiple storage devices can also be controlled, and the storage devices can be placed in the same place or different places as the devices. For example Figure 1 As shown, the artificial intelligence-based experimental compliance detection method includes the following steps:

[0068] 101. In response to a detection instruction, a target detection object is selected, a target detection area corresponding to the target detection object is acquired, and image data of the target detection area is collected; the detection instruction includes image detection and behavior detection.

[0069] When the system receives a detection instruction, it first determines the type of detection (image detection or behavior detection). Among them, image detection mainly includes the clothing of student users, including goggles, protective clothing, gloves, etc., which can be set according to the actual experimental scene, safety requirements, etc.; behavior detection mainly focuses on whether the experimental tools are used correctly, whether the position is placed in compliance, whether the lids of bottles and jars are closed after use, whether the objects are inverted, and whether the students' experimental movements are standard.

[0070] In the above, the detection command is started by the corresponding operation user. After the detection is started, the system automatically generates the detection command and then starts the detection process. The target detection object is the object to be detected this time. It can be one or more. The target detection object can be selected at the same time when the input detection is started.

[0071] For this detection instruction, whether it is image detection or behavior detection, or both, different target detection areas are generated. For example, for image detection, the user's head, body clothing, hands, etc. are mainly detected. A target detection object library is established to quickly select and locate the target detection object. The image data of the target detection area is mainly collected. The image data can be picture information or video information. Use a high-definition camera or other image acquisition equipment to collect image data of the target detection area.

[0072] In the embodiments, computer vision technology is used for image acquisition and preprocessing to ensure that the image quality meets the requirements of subsequent analysis; deep learning models are trained and optimized, and the performance of the models directly affects the recognition accuracy of clothing features; the judgment of detection results is based on preset safety clothing rules to accurately evaluate the compliance of clothing; the formulation and implementation of safety clothing rules, formulate reasonable safety clothing rules, and ensure that they are effectively implemented.

[0073] The embodiment can be applied in school laboratories and used in laboratories such as chemistry, biology, and physics to ensure that students correctly wear personal protective equipment such as lab coats, gloves, and goggles before conducting experimental operations. By capturing the video images and item location information of students' experimental operations in real time, and combining the deep learning model for behavior recognition and object state perception, the present invention can timely discover and warn potential safety hazards, such as improper use of chemicals, equipment operation errors, etc. This instant safety monitoring and early warning mechanism effectively reduces the risk of safety accidents during the experiment, ensures the personal safety of teachers and students, and provides a safer environment for experimental teaching. The system's built-in experimental operation specifications and standard processes enable each step of the student's operation to be evaluated in real time for compliance. When it is found that the student's operation does not meet the specifications, the system will immediately display compliance prompts on the display screen to guide students to correct errors, thereby ensuring the standardization and accuracy of the experimental operation. This not only helps to cultivate students' good experimental habits, but also improves the reliability and effectiveness of experimental results. It can be applied in distance education and online experimental courses to monitor whether students wear appropriate protective equipment as required, especially when conducting home experiments. It can also be used in industry, in production workshops, to monitor whether workers are wearing prescribed personal protective equipment, such as helmets, protective glasses, protective shoes, etc. Ensure that employees wear appropriate protective equipment when handling hazardous chemicals to reduce the occurrence of occupational injury accidents. In an intelligent manufacturing environment, behavior recognition and object state perception technology can monitor workers' operating behaviors in real time to ensure that they perform operations in accordance with standard procedures, reduce errors and repetitive work, and thus improve overall production efficiency. It can identify irregular or dangerous actions in workers' operations and issue early warnings in time to effectively prevent the occurrence of industrial accidents. At the same time, it can also monitor the operating status of equipment, prevent equipment failures, and ensure the stable operation of the production line. By analyzing the operating behavior of workers, the system can evaluate work efficiency and quality, identify bottlenecks in the production process, and provide companies with suggestions for optimizing production processes. It can also be used in the field of healthcare, in operating rooms or other sensitive areas, to ensure that medical staff wear the correct personal protective equipment to reduce the risk of cross infection. In biomedical research laboratories, monitor whether scientific researchers comply with safety dress rules. It can be applied to daily monitoring and care of patients. By identifying the patient's behavior patterns and object status (such as drug use, rehabilitation equipment status, etc.), the system can evaluate the patient's health status in real time and provide personalized care plans. Combined with IoT technology, behavior recognition and object state perception technology can realize remote medical and rehabilitation services. Doctors can remotely guide patients to conduct rehabilitation training or adjust treatment plans by analyzing patients' behavior data and object status information. It can also provide auxiliary decision support for doctors. By analyzing patients' behavior patterns and changes in physiological parameters, the system can predict the development trend of the disease and provide a basis for doctors to formulate more accurate treatment plans.It can also be used in the field of security to monitor abnormal behaviors in surveillance videos in real time, such as intrusion, theft, etc. By identifying behavioral patterns and changes in the state of objects, the system can automatically issue an alarm and notify relevant personnel to handle it. Combined with technologies such as face recognition and voiceprint recognition, the behavior recognition and object state perception system can achieve intelligent early warning and response to specific personnel. When the system identifies suspicious persons or abnormal situations, it can automatically trigger the alarm mechanism and take corresponding safety measures. By processing surveillance video data in an automated and intelligent way, manual intervention and false alarm rates are reduced, and security costs are reduced. It is used in sports training. In sports training venues, it monitors whether athletes are wearing appropriate protective equipment, such as elbow pads and knee pads, to prevent injuries.

[0074] 102. Preprocess the image data, extract a feature region from the preprocessed image data, identify the feature region, compare key features of the feature region with preset reference features, and obtain a comparison result.

[0075] The collected image data is preprocessed, including denoising, contrast enhancement, color correction, etc. The image recognition algorithm is used to extract feature areas from the preprocessed image data. Feature areas can be facial features, gesture features, or specific marks on the device, etc. The extracted feature areas are identified to extract key features.

[0076] Specifically, the image data is preprocessed, including: denoising the image data based on bilateral filtering; adjusting the contrast of the denoised image data; identifying interference areas in the image data, and cropping the interference areas.

[0077] Bilateral filtering is a nonlinear, edge-preserving smoothing filtering method. It considers the weights of both the spatial domain (i.e., the physical distance between pixels) and the pixel value domain (i.e., the difference between pixel values). This method can effectively remove noise while retaining the edge information of the image. During operation, set the parameters of bilateral filtering, including the Gaussian standard deviation (σ_s) in the spatial domain and the Gaussian standard deviation (σ_r) in the pixel value domain, apply the bilateral filtering algorithm to the image data, and obtain the denoised image. Contrast adjustment is a common step in image processing, which aims to enhance the brightness difference between different regions in the image to make the image clearer and easier to distinguish. During operation, calculate the global or local contrast of the image, and adjust the brightness value of the image according to the contrast calculation result to increase or decrease the contrast. Contrast adjustment can be achieved by using methods such as histogram equalization and adaptive contrast enhancement. In image data, interference areas may include irrelevant backgrounds, occluders, noise, etc., which may affect the results of subsequent image analysis. Therefore, it is necessary to identify and crop these interference areas. Use methods such as image segmentation, edge detection, and color space analysis to identify interference areas. According to the recognition results, the location and range of the interference area are determined. The image is cropped to remove the interference area and retain the target area.

[0078] Furthermore, the denoising method in the embodiment can also achieve higher quality denoising by learning more complex noise patterns through a large amount of data training through convolutional networks or adversarial networks. A denoising network is constructed, and noisy images and clean images are used as training data to train the denoising network. After training, the denoising network is used to denoise the image data. For contrast enhancement and color correction processing, multiple preset areas in the image can be identified based on image segmentation technology, and different contrast enhancement and color correction strategies can be applied to each area.

[0079] Based on the above preprocessing steps, feature areas are extracted from the preprocessed image data and the feature areas are identified, including: converting the preprocessed image data from the RGB color space to the HSV color space or the LAB color space, and calculating the color histogram or color mean of the image data; obtaining the color distribution of the image data based on the color histogram or the color mean, and selecting a target color distribution area consistent with a preset color from the image data according to the color distribution; and calculating image texture features in the target color distribution area.

[0080] The preprocessed image data is converted from the RGB color space to a color space that is more suitable for color feature extraction, such as HSV (hue, saturation, brightness) or Lab (brightness, two chromaticity channels a and b) color space. HSV space is more sensitive to hue recognition, while Lab space is closer to human visual perception. In the converted color space, the color histogram or color mean of the image data is calculated. The color histogram can reflect the distribution of various colors in the image, while the color mean provides a general description of the overall color of the image. Automatically select the most suitable color space for conversion according to the image content. For example, for colorful natural scene images, Lab space can be selected; for images with large hue changes, HSV space can be selected. Based on the calculated color histogram or color mean, the color distribution of the image data is obtained. According to the preset color range or threshold, the target color distribution area consistent with the preset color is selected from the image data, which can be achieved by color threshold segmentation, cluster analysis and other methods. In the selected target color distribution area, the texture features of the image are calculated. Texture features can include gray level co-occurrence matrix (GLCM), local binary pattern (LBP), wavelet transform, etc. According to the needs of subsequent processing or analysis, the most representative texture features are selected and optimized (such as dimensionality reduction, feature fusion, etc.). Texture features are extracted at different scales to capture the details and overall structure of the image. Deep learning models (such as convolutional neural networks) can be used to automatically learn the texture features of the image.

[0081] After the above-mentioned feature region extraction and identification, the key features of the feature region are compared with preset reference features, including: comparing the image texture features with preset image texture features.

[0082] First, extract image texture features from the feature region. This can be achieved through traditional texture feature extraction methods (such as gray level co-occurrence matrix, local binary pattern, etc.) or deep learning-based methods (such as feature maps extracted by convolutional neural networks). Convert the extracted features into a representation suitable for comparison. This may include normalization of feature vectors, dimensionality reduction (such as PCA, LDA, etc.) or feature encoding (such as bag of words model, Fisher vector, etc.). Extract benchmark texture features from a preset image or image set. These benchmark features should have the same representation as the target features for direct comparison. The benchmark features are the comparison benchmarks, and comparison with the benchmark features means comparison with features that meet the standards. Use appropriate similarity metrics (such as Euclidean distance, cosine similarity, Manhattan distance, etc.) to calculate the similarity between the feature region features and the benchmark features. Based on the similarity calculation results, determine whether the feature region matches the preset benchmark features. This can be achieved by setting a threshold. If the similarity is higher than the threshold, the match is considered successful. The accuracy of the matching results can be further verified by methods such as manual inspection or cross-validation. In addition to image texture features, other types of features (such as color features, shape features, etc.) can also be combined for multi-feature fusion comparison to improve the accuracy and robustness of the comparison.

[0083] In another example, feature regions are extracted from preprocessed image data and the feature regions are identified, including: inputting the preprocessed image data into a YOLO target detection model to extract feature regions; generating a segmentation mask image of the feature region based on image segmentation technology to obtain position information of the feature region, wherein the position information includes center coordinates, boundaries, and areas; inputting the preprocessed image data into a preset deep convolutional learning model to extract feature regions in the image data and identify target actions in the feature regions.

[0084] If the image is in color, it may be necessary to convert it to a grayscale image to reduce the data dimension while retaining enough information for subsequent processing. Adjust the image's brightness, contrast and other parameters to make it fit the optimal range for model processing. Remove noise from the image through methods such as filters to improve image quality. After performing the above conventional preprocessing measures, extract feature regions based on the YOLO object detection model. YOLO (You Only Look Once) is a real-time object detection algorithm that can simultaneously predict bounding boxes and categories of multiple objects in a single forward pass. Input the preprocessed image into the YOLO model, and the model will output a series of bounding boxes, each of which corresponds to a detected object with a corresponding confidence level. According to the confidence threshold, the bounding box that is most likely to contain the feature area is screened out. Image segmentation technology can divide an image into multiple regions, each of which corresponds to a specific object or category. Use a pre-trained image segmentation model (such as UNet, Mask R-CNN, etc.) to process the image. The model will output a segmentation mask of the same size as the input image, where each pixel value represents the probability that the pixel belongs to a specific category. Through the segmentation mask, the feature region of a specific category can be extracted and its location information (such as center coordinates, boundaries, and area) can be obtained. The deep convolutional neural network (CNN) inputs the image containing the feature region (or only the feature region) into the preset deep convolutional learning model. The model will extract high-level features in the image. These features can characterize the shape, texture, and motion pattern of the target. Based on the extracted features, the model will predict the action category of the target in the feature region.

[0085] Based on this processing step, the key features of the feature area are compared with preset reference features, including: comparing the position information with preset position features, and / or comparing the target action with the reference action; the preset reference features include preset position features and reference actions.

[0086] In the field of image processing and analysis, comparing the key features of the feature region with the preset reference features is a core step. First, obtain the position information from the feature region, which usually includes parameters such as the center coordinates, boundaries (i.e., the coordinates of the four vertices), and area. The preset reference position feature may be a specific coordinate point, a rectangular area, or a more complex shape. The reference position feature represents the expected position of the target object. Compare the extracted position information with the preset position feature. The comparison method may include calculating the distance between the center coordinates, calculating the overlap between the bounding boxes (such as IOU, intersection over union), or comparing the area size. The preset reference action may be a specific action category (such as raising a hand, letting go, etc.), or it may be an action sequence or pattern. Compare the identified target action with the reference action. The comparison method may include comparing the output results of the classification algorithm, calculating the similarity of the action sequence, or using a distance metric based on the feature vector. Before comparison, it is necessary to carefully select the features for comparison and ensure that they are sufficiently representative. The choice of threshold has a great impact on the comparison results. If the threshold is set too low, it may result in too many false positives; if it is set too high, it may result in missed negatives. By comparing the key features of the feature area with the preset reference features, it is possible to effectively verify whether the target object or event in the image is as expected.

[0087] 103. Evaluate the experimental compliance index of the target detection object based on the comparison result, and generate an alarm message when the experimental compliance index is lower than a preset standard.

[0088] Once an illegal operation or potential danger is found in the embodiment, a warning message is immediately generated and sent to the teacher's mobile device or a specific terminal in the laboratory through the server. At the same time, compliance prompts and danger warning messages are displayed on the display screen in the laboratory so that students can view and adjust experimental operations in time. In this step, a specific implementation method is to generate preset evaluation indexes and evaluation weights for different comparison items; the comparison items include position information, target actions and key features; based on the comparison results, the evaluation index corresponding to each comparison item is obtained respectively; the compliance index corresponding to each comparison item is obtained according to the formula F1*A1+F2*A2+F3*A3=S, where F1 is the evaluation index of the position information, F2 is the evaluation index of the target action, F3 is the evaluation index of the key feature, A1 is the evaluation weight corresponding to the position information, A2 is the evaluation index corresponding to the target action, and A3 is the evaluation index corresponding to the key feature; the maximum compliance index and the minimum compliance index are set according to the formula (SL min ) / (L max -L min ) The compliance index is standardized to obtain the experimental compliance index of the target detection object, and the L maxis the maximum compliance index, L min is the minimum compliance index.

[0089] In the above, the evaluation index is the accuracy or the degree of match. The accuracy score is usually (number of correctly detected targets / total number of targets) × 100%. The evaluation index can be obtained accordingly based on the comparison results. For example, the comparison of position information can be the difference between the detected target position and the preset position information, such as the offset of the center coordinates, the overlap of the bounding box (IOU), etc. A score range is set according to the size of the difference. The smaller the difference, the higher the score. For action recognition, the output of the action recognition model is compared with the preset benchmark action. If the recognized action is consistent with the benchmark action, full marks are awarded; otherwise, points are deducted based on the similarity or error of the action.

[0090] Embodiment 2

[0091] See also Figure 2 , Figure 2 Schematic diagram of the structure of an artificial intelligence-based experimental compliance detection device disclosed in an embodiment of the present invention. Figure 2 As shown, the artificial intelligence-based experimental compliance detection device may include: an image acquisition module 201, a feature comparison module 202 and a compliance calculation module 203, the image acquisition module 201: used to select a target detection object in response to a detection instruction, obtain a target detection area corresponding to the target detection object, and collect image data of the target detection area; the detection instruction includes image detection and behavior detection; the feature comparison module 202: used to preprocess the image data, extract feature areas from the preprocessed image data, and identify the feature areas, compare the key features of the feature areas with preset reference features, and obtain comparison results; the compliance calculation module 203: used to evaluate the experimental compliance index of the target detection object based on the comparison result, and generate an alarm message when the experimental compliance index is lower than the preset standard.

[0092] In the feature comparison module 202, the image data is preprocessed, including: denoising the image data based on bilateral filtering; adjusting the contrast of the denoised image data; identifying interference areas in the image data, and cropping the interference areas.

[0093] Further, as an example, extracting a feature region from preprocessed image data and identifying the feature region includes: converting the preprocessed image data from RGB color space to HSV color space or LAB color space, and calculating the color histogram or color mean of the image data; obtaining the color distribution of the image data based on the color histogram or color mean, selecting a target color distribution region consistent with a preset color from the image data according to the color distribution; and calculating the image texture feature in the target color distribution region. Accordingly, comparing the key features of the feature region with the preset reference features includes: comparing the image texture feature with the preset image texture feature.

[0094] In another example, extracting feature regions from preprocessed image data and identifying the feature regions include: inputting the preprocessed image data into a YOLO target detection model to extract feature regions; generating a segmentation mask image of the feature region based on image segmentation technology to obtain location information of the feature region, wherein the location information includes center coordinates, boundaries, and areas; inputting the preprocessed image data into a preset deep convolutional learning model to extract feature regions in the image data and identify target actions in the feature regions. Accordingly, the key features of the feature region are compared with preset reference features, including: comparing the location information with preset location features, and / or comparing the target action with the reference action; the preset reference features include preset location features and reference actions.

[0095] In the compliance calculation module 203, the experimental compliance index of the target detection object is evaluated based on the comparison result, including: generating preset evaluation indexes and evaluation weights for different comparison items; the comparison items include position information, target action and key features; obtaining the evaluation index corresponding to each comparison item based on the comparison result; obtaining the compliance index corresponding to each comparison item according to the formula F1*A1+F2*A2+F3*A3=S, where F1 is the evaluation index of the position information, F2 is the evaluation index of the target action, F3 is the evaluation index of the key feature, A1 is the evaluation weight corresponding to the position information, A2 is the evaluation index corresponding to the target action, and A3 is the evaluation index corresponding to the key feature; setting the maximum compliance index and the minimum compliance index, according to the formula (SL min ) / (L max -L min ) The compliance index is standardized to obtain the experimental compliance index of the target detection object, and the L max is the maximum compliance index, L min is the minimum compliance index.

[0096] Embodiment 3

[0097] See also Figure 3, Figure 3 Schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention. The electronic device may be a computer, a server, etc. Of course, in certain circumstances, it may also be a smart device such as a mobile phone, a tablet computer, a monitoring terminal, and an image acquisition device with processing functions. Figure 3 As shown, the electronic device may include:

[0098] A memory 301 storing executable program codes;

[0099] a processor 302 coupled to the memory 301;

[0100] The processor 302 calls the executable program code stored in the memory 301 to execute part or all of the steps in the artificial intelligence-based experimental compliance detection method in Example 1.

[0101] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute part or all of the steps in the artificial intelligence-based experimental compliance detection method in Embodiment 1.

[0102] An embodiment of the present invention further discloses a computer program product, wherein when the computer program product runs on a computer, the computer executes part or all of the steps in the artificial intelligence-based experimental compliance detection method in Embodiment 1.

[0103] An embodiment of the present invention also discloses an application publishing platform, wherein the application publishing platform is used to publish a computer program product, wherein when the computer program product runs on a computer, the computer executes part or all of the steps in the artificial intelligence-based experimental compliance detection method in embodiment one.

[0104] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0105] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed over multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0106] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0107] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the method described in each embodiment of the present invention.

[0108] In the embodiments provided by the present invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0109] A person of ordinary skill in the art can understand that some or all of the steps in the various methods of the embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0110] The above is a detailed introduction to the artificial intelligence-based experimental compliance detection method, device, electronic device and storage medium disclosed in the embodiments of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An artificial intelligence-based experimental compliance detection method, characterized in that: include: In response to a detection instruction, a target detection object is selected, a target detection area corresponding to the target detection object is acquired, and image data of the target detection area is collected; the detection instruction includes image detection and behavior detection; Preprocessing the image data, extracting a feature region from the preprocessed image data, identifying the feature region, and comparing key features of the feature region with preset reference features to obtain a comparison result; An experimental compliance index of the target detection object is evaluated based on the comparison result, and an alarm message is generated when the experimental compliance index is lower than a preset standard.

2. The experimental compliance detection method according to claim 1, characterized in that: Preprocessing the image data includes: Denoising the image data based on bilateral filtering; Perform contrast adjustment on the denoised image data; An interference area in the image data is identified and the interference area is cropped.

3. The experimental compliance detection method according to claim 1, characterized in that: Extracting feature regions from the preprocessed image data and identifying the feature regions include: Convert the preprocessed image data from the RGB color space to the HSV color space or the LAB color space, and calculate the color histogram or color mean of the image data; Acquire the color distribution of the image data based on the color histogram or the color mean, and select a target color distribution area consistent with a preset color from the image data according to the color distribution; Calculate image texture features in the target color distribution area.

4. The experimental compliance detection method according to claim 3, characterized in that: Comparing the key features of the feature area with the preset reference features includes: The image texture feature is compared with a preset image texture feature.

5. The experimental compliance detection method according to claim 1, characterized in that: Extracting feature regions from the preprocessed image data and identifying the feature regions include: Input the preprocessed image data into the YOLO target detection model to extract feature areas; Generate a segmentation mask image of the feature area based on the image segmentation technology to obtain the position information of the feature area, wherein the position information includes the center coordinates, the boundary and the area; The preprocessed image data is input into a preset deep convolutional learning model to extract the feature areas in the image data and identify the target actions in the feature areas.

6. The experimental compliance detection method according to claim 5, characterized in that: The key features of the feature area are compared with the preset reference features, including: The position information is compared with a preset position feature, and / or the target action is compared with a reference action; the preset reference feature includes a preset position feature and a reference action.

7. The experimental compliance detection method according to any one of claims 1 to 6, characterized in that: The experimental compliance index of the target detection object is evaluated based on the comparison result, including: Generate preset evaluation indexes and evaluation weights for different comparison items; the comparison items include location information, target actions and key features; Based on the comparison results, the evaluation index corresponding to each comparison item is obtained respectively; Obtain the compliance index corresponding to each comparison item according to the formula F1*A1+F2*A2+F3*A3=S, where F1 is the evaluation index of the location information, F2 is the evaluation index of the target action, F3 is the evaluation index of the key feature, A1 is the evaluation weight corresponding to the location information, A2 is the evaluation index corresponding to the target action, and A3 is the evaluation index corresponding to the key feature; Set the maximum compliance index and the minimum compliance index according to the formula (SL min ) / (L max -L min ) The compliance index is standardized to obtain the experimental compliance index of the target detection object, and the L max is the maximum compliance index, L min is the minimum compliance index.

8. An experimental compliance detection device based on artificial intelligence, characterized in that: include: Image acquisition module: used to select a target detection object in response to a detection instruction, obtain a target detection area corresponding to the target detection object, and collect image data of the target detection area; the detection instruction includes image detection and behavior detection; Feature comparison module: used for preprocessing the image data, extracting feature areas from the preprocessed image data, identifying the feature areas, comparing key features of the feature areas with preset reference features, and obtaining comparison results; Compliance calculation module: used to evaluate the experimental compliance index of the target detection object based on the comparison result, and generate an alarm message when the experimental compliance index is lower than a preset standard.

9. An electronic device, characterized in that: include: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the artificial intelligence-based experimental compliance detection method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program enables a computer to execute the artificial intelligence-based experimental compliance detection method according to any one of claims 1 to 7.

Images