Experimental compliance detection method and device based on artificial intelligence

Abstract

An embodiment of the present invention discloses an artificial intelligence-based experimental compliance detection method and device, which includes selecting a target detection object in response to a detection instruction, obtaining a target detection area corresponding to the target detection object, and collecting image data of the target detection area; preprocessing the image data, extracting a feature area from the preprocessed image data, identifying the feature area, comparing the key features of the feature area with preset reference features, and obtaining a comparison result; evaluating the experimental compliance index of the target detection object based on the comparison result, and generating an alarm message when the experimental compliance index is lower than the preset standard; the embodiment realizes compliance detection of the experimental process in an automated and intelligent manner, which can further improve the accuracy and efficiency of the system, help ensure the compliance of the experiment, reduce safety risks, and improve the quality of the experiment.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an artificial intelligence-based experimental compliance detection method and device. Background Art

[0002] Ensuring the safety of all participants during student lab exams is a fundamental and crucial task. To achieve this, students must correctly wear appropriate personal protective equipment, including lab coats, gloves, and goggles. However, traditional safety inspection methods rely primarily on manual supervision, which is not only inefficient but also susceptible to subjective factors, potentially leading to safety hazards.

[0003] At the same time, in physics, chemistry, and biology lab teaching and exams, many experimental procedures often carry certain risks and require compliance to ensure both experimental and personal safety. However, in practice, it is often difficult for teachers to monitor all students' experimental operations simultaneously, making it difficult to promptly detect and address dangerous or non-compliant behaviors. This not only increases experimental risk but also may affect the accuracy of experimental results and the development of students' experimental skills. Currently, artificial intelligence methods based on pose estimation can determine whether certain movements and behaviors meet requirements. This method can accurately identify and track the movements of key human parts, such as hands and arms, monitor students' experimental operations in real time, quickly identify potential non-compliant behaviors, and provide immediate feedback. Although pose estimation technology has made great progress, it still has limitations in recognizing certain complex movements, especially in laboratory environments where delicate hand movements or specialized experimental equipment are involved, and recognition accuracy may be affected. Summary of the Invention

[0004] In response to the above-mentioned defects, an embodiment of the present invention discloses an artificial intelligence-based experimental compliance detection method and device, which has a high recognition accuracy.

[0005] A first aspect of an embodiment of the present invention discloses an artificial intelligence-based experimental compliance detection method, comprising:

[0006] In response to a detection instruction, a target detection object is selected, a target detection area corresponding to the target detection object is acquired, and image data of the target detection area is collected; the detection instruction includes image detection and behavior detection;

[0007] Preprocessing the image data, extracting a feature region from the preprocessed image data, identifying the feature region, and comparing key features of the feature region with preset reference features to obtain a comparison result;

[0008] An experimental compliance index of the target detection object is evaluated based on the comparison result, and an alarm message is generated when the experimental compliance index is lower than a preset standard.

[0009] As an optional implementation manner, in the first aspect of the embodiment of the present invention, preprocessing the image data includes:

[0010] Denoising the image data based on bilateral filtering;

[0011] Perform contrast adjustment on the denoised image data;

[0012] Identify interference areas in the image data and crop the interference areas.

[0013] As an optional implementation manner, in the first aspect of the embodiment of the present invention, extracting feature regions from the preprocessed image data and identifying the feature regions includes:

[0014] Convert the preprocessed image data from the RGB color space to the HSV color space or the LAB color space, and calculate the color histogram or color mean of the image data;

[0015] Acquire a color distribution of the image data based on the color histogram or the color mean, and select a target color distribution area consistent with a preset color from the image data according to the color distribution;

[0016] Calculate image texture features in the target color distribution area.

[0017] As an optional implementation manner, in the first aspect of the embodiment of the present invention, comparing the key features of the feature area with preset reference features includes:

[0018] The image texture features are compared with preset image texture features.

[0019] As an optional implementation manner, in the first aspect of the embodiment of the present invention, extracting feature regions from the preprocessed image data and identifying the feature regions includes:

[0020] Input the preprocessed image data into the YOLO target detection model to extract feature areas;

[0021] Generate a segmentation mask image of the feature area based on image segmentation technology to obtain position information of the feature area, wherein the position information includes center coordinates, boundaries and area;

[0022] The preprocessed image data is input into a preset deep convolutional learning model to extract the feature areas in the image data and identify the target actions in the feature areas.

[0023] As an optional implementation manner, in the first aspect of the embodiment of the present invention, comparing the key features of the feature area with the preset reference features includes:

[0024] The position information is compared with a preset position feature, and / or the target action is compared with a reference action; the preset reference feature includes a preset position feature and a reference action.

[0025] As an optional implementation manner, in the first aspect of the embodiment of the present invention, evaluating the experimental compliance index of the target detection object based on the comparison result includes:

[0026] Generate preset evaluation indexes and evaluation weights for different comparison items; the comparison items include location information, target actions, and key features;

[0027] Based on the comparison results, the evaluation index corresponding to each comparison item is obtained respectively;

[0028] Obtain the compliance index corresponding to each comparison item according to the formula F1*A1+F2*A2+F3*A3=S, where F1 is the evaluation index of the location information, F2 is the evaluation index of the target action, F3 is the evaluation index of the key feature, A1 is the evaluation weight corresponding to the location information, A2 is the evaluation index corresponding to the target action, and A3 is the evaluation index corresponding to the key feature;

[0029] Set the maximum compliance index and minimum compliance index according to the formula (SL min ) / (L max -L min ) The compliance index is standardized to obtain the experimental compliance index of the target detection object, the L max is the maximum compliance index, L min is the minimum compliance index.

[0030] A second aspect of an embodiment of the present invention discloses an artificial intelligence-based experimental compliance detection device, comprising:

[0031] Image acquisition module: used to select a target detection object in response to a detection instruction, obtain a target detection area corresponding to the target detection object, and collect image data of the target detection area; the detection instruction includes image detection and behavior detection;

[0032] Feature comparison module: used to preprocess the image data, extract feature areas from the preprocessed image data, identify the feature areas, compare key features of the feature areas with preset reference features, and obtain comparison results;

[0033] Compliance calculation module: used to evaluate the experimental compliance index of the target detection object based on the comparison results, and generate an alarm message when the experimental compliance index is lower than a preset standard.

[0034] As an optional implementation manner, in the second aspect of the embodiment of the present invention, preprocessing the image data includes:

[0035] Denoising the image data based on bilateral filtering;

[0036] Perform contrast adjustment on the denoised image data;

[0037] Identify interference areas in the image data and crop the interference areas.

[0038] As an optional implementation manner, in the second aspect of the embodiment of the present invention, extracting feature regions from the preprocessed image data and identifying the feature regions includes:

[0039] Convert the preprocessed image data from the RGB color space to the HSV color space or the LAB color space, and calculate the color histogram or color mean of the image data;

[0040] Acquire a color distribution of the image data based on the color histogram or the color mean, and select a target color distribution area consistent with a preset color from the image data according to the color distribution;

[0041] Calculate image texture features in the target color distribution area.

[0042] As an optional implementation manner, in the second aspect of the embodiment of the present invention, comparing the key features of the feature area with preset reference features includes:

[0043] The image texture features are compared with preset image texture features.

[0044] As an optional implementation manner, in the second aspect of the embodiment of the present invention, extracting feature regions from the preprocessed image data and identifying the feature regions includes:

[0045] Input the preprocessed image data into the YOLO target detection model to extract feature areas;

[0046] Generate a segmentation mask image of the feature area based on image segmentation technology to obtain position information of the feature area, wherein the position information includes center coordinates, boundaries and area;

[0047] The preprocessed image data is input into a preset deep convolutional learning model to extract the feature areas in the image data and identify the target actions in the feature areas.

[0048] As an optional implementation manner, in the second aspect of the embodiment of the present invention, comparing the key features of the feature area with the preset reference features includes:

[0049] The position information is compared with a preset position feature, and / or the target action is compared with a reference action; the preset reference feature includes a preset position feature and a reference action.

[0050] As an optional implementation manner, in the second aspect of the embodiment of the present invention, evaluating the experimental compliance index of the target detection object based on the comparison result includes:

[0051] Generate preset evaluation indexes and evaluation weights for different comparison items; the comparison items include location information, target actions, and key features;

[0052] Based on the comparison results, the evaluation index corresponding to each comparison item is obtained respectively;

[0053] Obtain the compliance index corresponding to each comparison item according to the formula F1*A1+F2*A2+F3*A3=S, where F1 is the evaluation index of the location information, F2 is the evaluation index of the target action, F3 is the evaluation index of the key feature, A1 is the evaluation weight corresponding to the location information, A2 is the evaluation index corresponding to the target action, and A3 is the evaluation index corresponding to the key feature;

[0054] Set the maximum compliance index and minimum compliance index according to the formula (SL min ) / (L max -L min ) The compliance index is standardized to obtain the experimental compliance index of the target detection object, the L max is the maximum compliance index, L min is the minimum compliance index.

[0055] The third aspect of an embodiment of the present invention discloses an electronic device, comprising: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the artificial intelligence-based experimental compliance detection method disclosed in the first aspect of the embodiment of the present invention.

[0056] A fourth aspect of an embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the artificial intelligence-based experimental compliance detection method disclosed in the first aspect of an embodiment of the present invention.

[0057] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0058] In the embodiment of the present invention, when a detection instruction is received, the type of detection (image detection or behavior detection) is first determined, and the corresponding target detection object is selected according to the detection type. After the target detection object is determined, its corresponding detection area is obtained, and the collected image data is preprocessed, including denoising, contrast enhancement, color correction, etc. The extracted key features are compared with the preset benchmark features. Based on the comparison results, the experimental compliance index of the target detection object is evaluated. The embodiment realizes the compliance detection of the experimental process in an automated and intelligent manner. By optimizing the steps of instruction parsing, image acquisition, preprocessing, feature extraction, comparison evaluation and alarm information generation, the accuracy and efficiency of the system can be further improved. This will help to ensure the compliance of the experiment, reduce safety risks and improve the quality of the experiment. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0060] Figure 1 This is a flow chart of an artificial intelligence-based experimental compliance detection method disclosed in an embodiment of the present invention;

[0061] Figure 2 1 is a schematic structural diagram of an artificial intelligence-based experimental compliance detection device provided by an embodiment of the present invention;

[0062] Figure 3 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0064] It should be noted that the terms "first," "second," "third," "fourth," etc. in the description and claims of the present invention are used to distinguish different objects rather than to describe a specific order. The terms "including" and "having," as well as any variations thereof, in the embodiments of the present invention, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus.

[0065] The embodiments of the present invention disclose an artificial intelligence-based experimental compliance detection method, device, electronic device and storage medium. In the embodiments, when a detection instruction is received, the type of detection (image detection or behavior detection) is first determined, and the corresponding target detection object is selected according to the detection type. After the target detection object is determined, its corresponding detection area is obtained, and the collected image data is preprocessed, including denoising, contrast enhancement, color correction, etc. The extracted key features are compared with the preset benchmark features. Based on the comparison results, the experimental compliance index of the target detection object is evaluated. The embodiments realize compliance detection of the experimental process in an automated and intelligent manner. By optimizing the steps of instruction parsing, image acquisition, preprocessing, feature extraction, comparison evaluation and alarm information generation, the accuracy and efficiency of the system can be further improved. This will help ensure the compliance of the experiment, reduce safety risks and improve the quality of the experiment.

[0066] Example 1

[0067] See also Figure 1 , Figure 1 It is a flow chart of the XX method disclosed in the embodiment of the present invention. Among them, the execution subject of the method described in the embodiment of the present invention is an execution subject composed of software and / or hardware, which can receive relevant information by wired or / and wireless means, and can send certain instructions. Of course, it can also have certain processing functions and storage functions. The execution subject can control multiple devices, such as a remote physical server or cloud server and related software, or it can be a local host or server and related software that performs related operations on a device placed somewhere. In some scenarios, multiple storage devices can also be controlled, and the storage devices can be placed in the same place or different places as the devices. For example Figure 1 As shown, the artificial intelligence-based experimental compliance detection method includes the following steps:

[0068] 101. In response to a detection instruction, a target detection object is selected, a target detection area corresponding to the target detection object is acquired, and image data of the target detection area is collected; the detection instruction includes image detection and behavior detection.

[0069] When the system receives a detection instruction, it first determines the type of detection (image detection or behavior detection). Among them, image detection mainly includes the student user's clothing, including goggles, protective clothing, gloves, etc., which can be specifically set according to the actual experimental scenario and safety requirements. Behavior detection mainly focuses on whether the experimental tools are used correctly, whether the positioning is legal, whether the lids of bottles and jars are closed after use, whether the objects are upside down, and whether the student's experimental movements are standard.

[0070] In the above, the detection command is initiated by the corresponding operating user. After the detection is started, the system automatically generates the detection command and starts the detection process. The target detection object is also the object to be detected this time. It can be one or more. The target detection object can be selected at the same time when the detection start input is entered.

[0071] Depending on whether the detection command is image detection, behavior detection, or both, different target detection areas are generated. For example, for image detection, the focus is on the user's head, clothing, hands, and other areas. A target detection object library is established to quickly select and locate the target detection objects. Image data of the target detection area is primarily collected. This image data can be either images or videos. A high-definition camera or other image acquisition device is used to capture image data of the target detection area.

[0072] In the embodiment, computer vision technology is used for image acquisition and preprocessing to ensure that the image quality meets the requirements of subsequent analysis; deep learning models are trained and optimized, and the performance of the model directly affects the recognition accuracy of clothing features; the judgment of detection results is based on preset safety clothing rules to accurately evaluate clothing compliance; the formulation and implementation of safety clothing rules, formulate reasonable safety clothing rules, and ensure that they are effectively implemented.

[0073] The embodiments can be applied in school laboratories, such as chemistry, biology, and physics laboratories, to ensure that students properly wear personal protective equipment (PPE) such as lab coats, gloves, and goggles before conducting experiments. By capturing real-time video footage of students' experimental operations and object location information, and combining it with deep learning models for behavior recognition and object state perception, the present invention can promptly detect and warn of potential safety hazards, such as improper use of chemicals and equipment operation errors. This real-time safety monitoring and warning mechanism effectively reduces the risk of safety accidents during experiments, ensures the personal safety of teachers and students, and provides a safer environment for experimental teaching. The system's built-in experimental operation specifications and standard procedures ensure that every step of a student's operation is evaluated for compliance in real time. If a student's operation is found to be non-compliant, the system will immediately display a compliance prompt on the display screen to guide the student to correct the error, thereby ensuring the standardization and accuracy of the experimental operation. This not only helps cultivate good experimental habits in students, but also improves the reliability and validity of experimental results. It can be used in distance education and online experimental courses to monitor whether students are wearing appropriate protective equipment as required, especially when conducting home experiments. It can also be used in industry, monitoring whether workers are wearing required personal protective equipment (PPE) such as helmets, safety glasses, and protective shoes in production workshops. This ensures that employees wearing appropriate PPE when handling hazardous chemicals reduces the risk of occupational injuries. In smart manufacturing environments, behavior recognition and object state perception technologies can monitor workers' operating behavior in real time, ensuring they follow standard procedures, reducing errors and duplication of work, and thus improving overall production efficiency. They can identify irregular or dangerous actions and issue timely warnings, effectively preventing workplace accidents. Furthermore, they can monitor equipment operating status, prevent equipment failures, and ensure stable production line operation. By analyzing worker behavior, the system can assess work efficiency and quality, identify bottlenecks in the production process, and provide companies with recommendations for optimizing production processes. It can also be used in healthcare, ensuring that medical staff wear the correct PPE in operating rooms and other sensitive areas, reducing the risk of cross-infection. In biomedical research laboratories, it can monitor whether researchers adhere to safety attire regulations. It can also be used in daily patient monitoring and care. By identifying patients' behavioral patterns and the status of objects (such as medication use and rehabilitation equipment status), the system can assess patients' health status in real time and provide personalized care plans. Combined with IoT technology, behavior recognition and object state perception technologies can enable remote medical and rehabilitation services. Doctors can analyze patients' behavioral data and object state information to remotely guide patients in rehabilitation training or adjust treatment plans. It can also provide decision support for doctors. By analyzing patients' behavioral patterns and changes in physiological parameters, the system can predict disease progression, providing a basis for doctors to develop more precise treatment plans.It can also be used in the security field to monitor abnormal behavior in surveillance video in real time, such as intrusion and theft. By identifying behavioral patterns and changes in object status, the system can automatically issue an alarm and notify relevant personnel to handle the situation. Combined with technologies such as facial recognition and voiceprint recognition, the behavior recognition and object state perception system can achieve intelligent early warning and response to specific individuals. When the system identifies suspicious individuals or abnormal situations, it can automatically trigger the alarm mechanism and take appropriate security measures. By processing surveillance video data in an automated and intelligent manner, manual intervention and false alarm rates are reduced, reducing security costs. It is also used in sports training to monitor whether athletes are wearing appropriate protective equipment, such as elbow pads and knee pads, to prevent injuries.

[0074] 102. Preprocess the image data, extract a feature region from the preprocessed image data, identify the feature region, compare key features of the feature region with preset reference features, and obtain a comparison result.

[0075] The collected image data is preprocessed, including noise reduction, contrast enhancement, and color correction. Image recognition algorithms are used to extract feature regions from the preprocessed image data. Feature regions can include facial features, gesture features, or specific markings on the device. Key features are then identified within the extracted feature regions.

[0076] Specifically, the image data is preprocessed, including: denoising the image data based on bilateral filtering; adjusting the contrast of the denoised image data; identifying interference areas in the image data, and cropping the interference areas.

[0077] Bilateral filtering is a nonlinear, edge-preserving smoothing filtering method. It considers both the spatial domain (i.e., the physical distance between pixels) and the pixel value domain (i.e., the difference between pixel values) in weight. This method effectively removes noise while preserving edge information in the image. During operation, the bilateral filtering parameters, including the Gaussian standard deviation (σ_s) in the spatial domain and the Gaussian standard deviation (σ_r) in the pixel value domain, are set. The bilateral filtering algorithm is then applied to the image data to produce a denoised image. Contrast adjustment is a common step in image processing, aiming to enhance brightness differences between different regions in an image for better clarity and legibility. During operation, the global or local contrast of the image is calculated, and based on the contrast calculation results, the image brightness is adjusted to increase or decrease contrast. Contrast adjustment can be achieved using methods such as histogram equalization and adaptive contrast enhancement. In image data, interfering regions may include irrelevant background, obstructions, and noise, which can affect the results of subsequent image analysis. Therefore, identifying and cropping these interfering regions is essential. Methods such as image segmentation, edge detection, and color space analysis are used to identify interfering regions. Based on the recognition results, the location and range of the interference area are determined. The image is cropped to remove the interference area and retain the target area.

[0078] Furthermore, the denoising method in the embodiment can also achieve higher-quality denoising by learning more complex noise patterns through large-scale data training, such as convolutional neural networks or adversarial networks. A denoising network is constructed, and noisy images and clean images are used as training data to train the denoising network. After training, the denoising network is used to denoise the image data. For contrast enhancement and color correction processing, multiple preset areas in the image can be identified based on image segmentation technology, and different contrast enhancement and color correction strategies can be applied to each area.

[0079] Based on the above-mentioned preprocessing steps, feature areas are extracted from the preprocessed image data, and the feature areas are identified, including: converting the preprocessed image data from the RGB color space to the HSV color space or the LAB color space, and calculating the color histogram or color mean of the image data; obtaining the color distribution of the image data based on the color histogram or color mean, and selecting a target color distribution area consistent with a preset color from the image data according to the color distribution; and calculating the image texture features in the target color distribution area.

[0080] The preprocessed image data is converted from the RGB color space to a color space more suitable for color feature extraction, such as the HSV (hue, saturation, value) or Lab (brightness, two chromaticity channels, a and b). The HSV space is more sensitive to hue, while the Lab space is closer to human visual perception. In the converted color space, the color histogram or color mean of the image data is calculated. The color histogram reflects the distribution of various colors in the image, while the color mean provides a general description of the overall image color. The most suitable color space for conversion is automatically selected based on the image content. For example, for colorful natural scene images, the Lab space can be selected, while for images with large hue variations, the HSV space can be selected. Based on the calculated color histogram or color mean, the color distribution of the image data is obtained. Based on a preset color range or threshold, a target color distribution area consistent with the preset color is selected from the image data. This can be achieved through methods such as color threshold segmentation and cluster analysis. Within the selected target color distribution area, the image texture features are calculated. Texture features can include gray-level co-occurrence matrix (GLCM), local binary pattern (LBP), wavelet transform, etc. Based on the needs of subsequent processing or analysis, the most representative texture features are selected and optimized (such as dimensionality reduction, feature fusion, etc.). Texture features are extracted at different scales to capture the details and overall structure of the image. Deep learning models (such as convolutional neural networks) can be used to automatically learn the texture features of the image.

[0081] After the feature region is extracted and identified as described above, the key features of the feature region are compared with preset reference features, including: comparing the image texture features with preset image texture features.

[0082] First, image texture features are extracted from the feature region. This can be achieved using traditional texture feature extraction methods (such as gray-level co-occurrence matrix and local binary pattern extraction) or deep learning-based methods (such as feature maps extracted by convolutional neural networks). The extracted features are converted into a representation suitable for comparison. This may include feature vector normalization, dimensionality reduction (such as PCA and LDA), or feature encoding (such as bag-of-words models and Fisher vectors). Reference texture features are extracted from a pre-set image or image set. These reference features should have the same representation as the target features to facilitate direct comparison. The reference features serve as a benchmark for comparison, and comparison with the reference features means comparing against features that meet the criteria. An appropriate similarity metric (such as Euclidean distance, cosine similarity, or Manhattan distance) is used to calculate the similarity between the features of the feature region and the reference features. Based on the similarity calculation results, a match is determined between the feature region and the pre-set reference features. This can be achieved by setting a threshold; if the similarity exceeds the threshold, the match is considered successful. The accuracy of the matching results can be further verified through manual inspection or cross-validation. In addition to image texture features, other types of features (such as color features, shape features, etc.) can also be combined for multi-feature fusion comparison to improve the accuracy and robustness of the comparison.

[0083] In another example, feature regions are extracted from preprocessed image data and the feature regions are identified, including: inputting the preprocessed image data into a YOLO target detection model to extract feature regions; generating a segmentation mask image of the feature region based on image segmentation technology to obtain position information of the feature region, wherein the position information includes center coordinates, boundaries, and areas; inputting the preprocessed image data into a preset deep convolutional learning model to extract feature regions in the image data and identify target actions in the feature regions.

[0084] If the image is in color, it may be necessary to convert it to grayscale to reduce the data dimension while retaining sufficient information for subsequent processing. Adjust image parameters such as brightness and contrast to ensure they are within the optimal range for model processing. Remove noise from the image through methods such as filters to improve image quality. After performing the above conventional preprocessing measures, feature regions are extracted based on the YOLO object detection model. YOLO (You Only Look Once) is a real-time object detection algorithm that can simultaneously predict the bounding boxes and categories of multiple objects in a single forward pass. The preprocessed image is input into the YOLO model, which will output a series of bounding boxes, each corresponding to a detected object and with a corresponding confidence level. Based on the confidence threshold, the bounding box most likely to contain the feature region is selected. Image segmentation technology can divide an image into multiple regions, each corresponding to a specific object or category. Using a pre-trained image segmentation model (such as UNet, Mask R-CNN, etc.) to process the image, the model will output a segmentation mask of the same size as the input image, where each pixel value represents the probability that the pixel belongs to a specific category. Through the segmentation mask, the feature region of a specific category can be extracted and its location information (such as center coordinates, boundaries, and area) can be obtained. The deep convolutional neural network (CNN) inputs the image containing the feature region (or only the feature region) into the preset deep convolutional learning model. The model will extract high-level features in the image. These features can represent the shape, texture, and motion pattern of the target. Based on the extracted features, the model will predict the action category of the target in the feature region.

[0085] Based on this processing step, the key features of the feature area are compared with preset reference features, including: comparing the position information with preset position features, and / or comparing the target action with the reference action; the preset reference features include preset position features and reference actions.

[0086] In the field of image processing and analysis, comparing the key features of a feature region with pre-set benchmark features is a core step. First, position information is obtained from the feature region, typically including parameters such as center coordinates, bounding box (i.e., coordinates of four vertices), and area. The pre-set benchmark position feature can be a specific coordinate point, a rectangular area, or a more complex shape. The benchmark position feature represents the expected location of the target object. The extracted position information is compared with the pre-set position features. Comparison methods may include calculating the distance between center coordinates, calculating the overlap between bounding boxes (such as IOU, or intersection over union), or comparing area sizes. The pre-set benchmark action may be a specific action category (such as raising a hand or releasing a hand), or a sequence or pattern of actions. The identified target action is compared with the benchmark action. Comparison methods may include comparing the output of a classification algorithm, calculating the similarity of action sequences, or using a distance metric based on feature vectors. Before comparison, the features to be compared must be carefully selected and ensured to be sufficiently representative. The choice of threshold has a significant impact on the comparison results. If the threshold is set too low, it may result in excessive false positives; if it is set too high, it may result in missed negatives. By comparing the key features of the feature area with the preset reference features, it is possible to effectively verify whether the target object or event in the image is consistent with the expected one.

[0087] 103. Evaluate the experimental compliance index of the target detection object based on the comparison result, and generate an alarm message when the experimental compliance index is lower than a preset standard.

[0088] In the embodiment, once non-compliant operations or potential dangers are discovered, a warning message is immediately generated and sent to the teacher's mobile device or a specific terminal in the laboratory through the server. At the same time, compliance prompts and danger warning messages are displayed on the display screen in the laboratory so that students can view and adjust experimental operations in a timely manner. In this step, a specific implementation method is to generate preset evaluation indexes and evaluation weights for different comparison items; the comparison items include position information, target actions and key features; based on the comparison results, the evaluation index corresponding to each comparison item is obtained; the compliance index corresponding to each comparison item is obtained according to the formula F1*A1+F2*A2+F3*A3=S, where F1 is the evaluation index of the position information, F2 is the evaluation index of the target action, F3 is the evaluation index of the key feature, A1 is the evaluation weight corresponding to the position information, A2 is the evaluation index corresponding to the target action, and A3 is the evaluation index corresponding to the key feature; the maximum compliance index and the minimum compliance index are set according to the formula (SL min ) / (L max -L min ) The compliance index is standardized to obtain the experimental compliance index of the target detection object, the L maxis the maximum compliance index, L min is the minimum compliance index.

[0089] In the above, the evaluation index is the accuracy or the degree of matching. The accuracy score is usually (number of correctly detected targets / total number of targets) × 100%. The evaluation index can be obtained according to the comparison results. For example, the comparison of position information can be the difference between the detected target position and the preset position information, such as the offset of the center coordinates, the overlap of the bounding box (IOU), etc. A score range is set according to the size of the difference. The smaller the difference, the higher the score. For action recognition, the output of the action recognition model is compared with the preset benchmark action. If the recognized action is consistent with the benchmark action, full marks are awarded; otherwise, points are deducted based on the similarity or error of the action.

[0090] Example 2

[0091] See also Figure 2 , Figure 2 This is a schematic diagram of the structure of an artificial intelligence-based experimental compliance detection device disclosed in an embodiment of the present invention. Figure 2 As shown, the artificial intelligence-based experimental compliance detection device may include: an image acquisition module 201, a feature comparison module 202 and a compliance calculation module 203, the image acquisition module 201: for selecting a target detection object in response to a detection instruction, obtaining a target detection area corresponding to the target detection object, and collecting image data of the target detection area; the detection instruction includes image detection and behavior detection; the feature comparison module 202: for preprocessing the image data, extracting feature areas from the preprocessed image data, and identifying the feature areas, comparing the key features of the feature areas with preset reference features, and obtaining comparison results; the compliance calculation module 203: for evaluating the experimental compliance index of the target detection object based on the comparison result, and generating an alarm message when the experimental compliance index is lower than the preset standard.

[0092] In the feature comparison module 202 , the image data is preprocessed, including: denoising the image data based on bilateral filtering; adjusting the contrast of the denoised image data; identifying interference areas in the image data, and cropping the interference areas.

[0093] Furthermore, as an example, extracting and identifying feature regions from preprocessed image data includes: converting the preprocessed image data from an RGB color space to an HSV color space or a LAB color space, and calculating a color histogram or color mean of the image data; obtaining a color distribution of the image data based on the color histogram or color mean, selecting a target color distribution region consistent with a preset color from the image data based on the color distribution; and calculating image texture features within the target color distribution region. Accordingly, comparing key features of the feature region with preset reference features includes: comparing the image texture features with preset image texture features.

[0094] In another example, extracting feature regions from preprocessed image data and identifying the feature regions include: inputting the preprocessed image data into a YOLO target detection model to extract feature regions; generating a segmentation mask image of the feature region based on image segmentation technology to obtain position information of the feature region, wherein the position information includes center coordinates, boundaries, and area; inputting the preprocessed image data into a preset deep convolutional learning model to extract feature regions from the image data and identify target actions in the feature regions. Accordingly, the key features of the feature regions are compared with preset reference features, including: comparing the position information with preset position features, and / or comparing the target action with a reference action; the preset reference features include preset position features and reference actions.

[0095] In the compliance calculation module 203, the experimental compliance index of the target detection object is evaluated based on the comparison results, including: generating preset evaluation indexes and evaluation weights for different comparison items; the comparison items include position information, target action and key features; obtaining the evaluation index corresponding to each comparison item based on the comparison results; obtaining the compliance index corresponding to each comparison item according to the formula F1*A1+F2*A2+F3*A3=S, where F1 is the evaluation index of the position information, F2 is the evaluation index of the target action, F3 is the evaluation index of the key feature, A1 is the evaluation weight corresponding to the position information, A2 is the evaluation index corresponding to the target action, and A3 is the evaluation index corresponding to the key feature; setting the maximum compliance index and the minimum compliance index, according to the formula (SL min ) / (L max -L min ) The compliance index is standardized to obtain the experimental compliance index of the target detection object, the L max is the maximum compliance index, L min is the minimum compliance index.

[0096] Example 3

[0097] See also Figure 3, Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention. The electronic device can be a computer, a server, etc. Of course, in certain circumstances, it can also be a smart device such as a mobile phone, a tablet computer, a monitoring terminal, and an image acquisition device with processing functions. Figure 3 As shown, the electronic device may include:

[0098] A memory 301 storing executable program code;

[0099] a processor 302 coupled to the memory 301;

[0100] The processor 302 calls the executable program code stored in the memory 301 to execute part or all of the steps in the artificial intelligence-based experimental compliance detection method in Example 1.

[0101] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute some or all of the steps in the artificial intelligence-based experimental compliance detection method in Example 1.

[0102] An embodiment of the present invention further discloses a computer program product, wherein when the computer program product runs on a computer, the computer is enabled to execute some or all of the steps in the artificial intelligence-based experimental compliance detection method in embodiment one.

[0103] An embodiment of the present invention also discloses an application publishing platform, wherein the application publishing platform is used to publish a computer program product, wherein when the computer program product runs on a computer, the computer executes some or all of the steps in the artificial intelligence-based experimental compliance detection method in Example 1.

[0104] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the processes does not necessarily mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0105] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of these units may be selected based on actual needs to achieve the objectives of this embodiment.

[0106] In addition, the functional units in the embodiments of the present invention may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The integrated unit may be implemented in the form of hardware or software functional units.

[0107] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a memory and includes several requests for causing a computer device (which can be a personal computer, server, or network device, specifically a processor in the computer device) to execute some or all of the steps of the method described in each embodiment of the present invention.

[0108] In the embodiments provided herein, it should be understood that "B corresponding to A" means that B is associated with A and B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B based solely on A; B can also be determined based on A and / or other information.

[0109] Those skilled in the art will appreciate that some or all of the steps in the various methods of the embodiments may be performed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, such as a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0110] The above is a detailed introduction to the artificial intelligence-based experimental compliance detection method, device, electronic device and storage medium disclosed in the embodiments of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An artificial intelligence-based experimental compliance detection method, characterized in that: include: In response to a detection instruction, a target detection object is selected, a target detection area corresponding to the target detection object is acquired, and image data of the target detection area is collected; the detection instruction includes image detection and behavior detection; Preprocessing the image data, extracting a feature region from the preprocessed image data, identifying the feature region, and comparing key features of the feature region with preset reference features to obtain a comparison result; The extracting feature regions from the pre-processed image data and identifying the feature regions includes: Convert the preprocessed image data from the RGB color space to the HSV color space or the LAB color space, and calculate the color histogram or color mean of the image data; Acquire a color distribution of the image data based on the color histogram or the color mean, and select a target color distribution area consistent with a preset color from the image data according to the color distribution; Calculate image texture features in the target color distribution area; and Input the preprocessed image data into the YOLO target detection model to extract feature areas; Generate a segmentation mask image of the feature area based on image segmentation technology to obtain position information of the feature area, wherein the position information includes center coordinates, boundaries and area; Input the preprocessed image data into a preset deep convolutional learning model to extract the feature areas in the image data and identify the target actions in the feature areas; The experimental compliance index of the target detection object is evaluated based on the comparison result, and an alarm message is generated when the experimental compliance index is lower than a preset standard; and a compliance prompt and a danger warning message are displayed on a display screen in the laboratory. The experimental compliance index of the target detection object is evaluated based on the comparison result, including: Generate preset evaluation indexes and evaluation weights for different comparison items; the comparison items include location information, target actions, and key features; Based on the comparison results, the evaluation index corresponding to each comparison item is obtained respectively; Obtain the compliance index corresponding to each comparison item according to the formula F1*A1+F2*A2+F3*A3=S, where F1 is the evaluation index of the location information, F2 is the evaluation index of the target action, F3 is the evaluation index of the key feature, A1 is the evaluation weight corresponding to the location information, A2 is the evaluation index corresponding to the target action, and A3 is the evaluation index corresponding to the key feature; Set the maximum compliance index and minimum compliance index according to the formula (SL min ) / (L max -L min ) The compliance index is standardized to obtain the experimental compliance index of the target detection object, the L max is the maximum compliance index, L min is the minimum compliance index.

2. The experimental compliance detection method according to claim 1, characterized in that: Preprocessing the image data includes: Denoising the image data based on bilateral filtering; Perform contrast adjustment on the denoised image data; Identify interference areas in the image data and crop the interference areas.

3. The experimental compliance detection method according to claim 1, characterized in that: Comparing the key features of the feature area with the preset reference features includes: The image texture features are compared with preset image texture features.

4. The experimental compliance detection method according to claim 1, characterized in that: The key features of the feature area are compared with the preset reference features, including: The position information is compared with a preset position feature, and / or the target action is compared with a reference action; the preset reference feature includes a preset position feature and a reference action.

5. An artificial intelligence-based experimental compliance detection device, characterized in that: include: Image acquisition module: used to select a target detection object in response to a detection instruction, obtain a target detection area corresponding to the target detection object, and collect image data of the target detection area; the detection instruction includes image detection and behavior detection; Feature comparison module: used to preprocess the image data, extract feature areas from the preprocessed image data, identify the feature areas, compare key features of the feature areas with preset reference features, and obtain comparison results; The extracting feature regions from the pre-processed image data and identifying the feature regions includes: Convert the preprocessed image data from the RGB color space to the HSV color space or the LAB color space, and calculate the color histogram or color mean of the image data; Acquire a color distribution of the image data based on the color histogram or the color mean, and select a target color distribution area consistent with a preset color from the image data according to the color distribution; Calculate image texture features in the target color distribution area; and Input the preprocessed image data into the YOLO target detection model to extract feature areas; Generate a segmentation mask image of the feature area based on image segmentation technology to obtain position information of the feature area, wherein the position information includes center coordinates, boundaries and area; Input the preprocessed image data into a preset deep convolutional learning model to extract the feature areas in the image data and identify the target actions in the feature areas; Compliance calculation module: used to evaluate the experimental compliance index of the target detection object based on the comparison results, generate an alarm message when the experimental compliance index is lower than the preset standard; display compliance prompts and danger warning information on the display screen in the laboratory; the experimental compliance index of the target detection object evaluated based on the comparison results includes: Generate preset evaluation indexes and evaluation weights for different comparison items; the comparison items include location information, target actions, and key features; Based on the comparison results, the evaluation index corresponding to each comparison item is obtained respectively; Obtain the compliance index corresponding to each comparison item according to the formula F1*A1+F2*A2+F3*A3=S, where F1 is the evaluation index of the location information, F2 is the evaluation index of the target action, F3 is the evaluation index of the key feature, A1 is the evaluation weight corresponding to the location information, A2 is the evaluation index corresponding to the target action, and A3 is the evaluation index corresponding to the key feature; Set the maximum compliance index and minimum compliance index according to the formula (SL min ) / (L max -L min ) The compliance index is standardized to obtain the experimental compliance index of the target detection object, the L max is the maximum compliance index, L min is the minimum compliance index.

6. An electronic device, characterized in that: include: a memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the artificial intelligence-based experimental compliance detection method described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program enables a computer to execute the artificial intelligence-based experimental compliance detection method according to any one of claims 1 to 5.

Images