An AI-assisted homework guidance method

Abstract

This invention discloses an AI-assisted work guidance method, comprising the following specific steps: collecting video images of workers' standardized work processes and their voice explanations during work; extracting video frames and corresponding voice explanations for multiple key work periods from the video images; collecting video images of workers' standardized work processes, voice explanations of the work processes, and video images of workers' non-standard work processes; processing the collected data to establish a model database for judging workers' work standards; and monitoring the workers' work processes and recognizing their work actions through monitoring equipment. This method can intelligently identify and judge workers' work processes, which is beneficial for timely detection of workers' non-standard or non-standard work actions and improves the effectiveness of work guidance assistance.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to an AI-assisted task guidance method. Background Technology

[0002] Work instructions are instructions given to standard operators on standard operating procedures. Before factory workers can perform their work proficiently, they need to be guided by relevant staff.

[0003] There are two existing methods for providing work guidance to operators. One is manual work guidance, which requires face-to-face instruction from the instructor, affecting the instructor's normal work. The other is to provide auxiliary work guidance through monitoring equipment and supplementary teaching materials. Existing patent literature: Work Guidance Effect Display Device and Work Guidance Effect Display Method (Application No.: 201180004290.1) discloses a display device that can display work guidance knowledge and rules to guide the operator. However, this work guidance method can only passively guide the operator by displaying guidance knowledge and rules through electronic devices. When the operator is unaware that their work is not standardized or violates regulations, it cannot play an auxiliary guidance role, resulting in poor work guidance effect. Summary of the Invention

[0004] To address the problems existing in the above-mentioned work guidance methods, the purpose of this invention is to provide an AI-assisted work guidance method that intelligently identifies and judges the work process of workers, which helps to promptly detect non-standard or illegal work actions of workers and improve the effectiveness of work guidance assistance.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] An AI-assisted homework guidance method includes the following specific steps:

[0007] S1. Collect video images of the standardized operation process of the operators and the audio explanation information of the operators during the operation;

[0008] S2. Extract the video footage and corresponding audio narration information for multiple key operations from the video images;

[0009] S3. Extract one or more sets of key operation frames from the same video frame, and extract the corresponding audio explanation information.

[0010] S4. Collect images of workers performing key tasks in multiple sets of standardized operations, compare the collected images with the key task images of one or more sets of continuous multi-frame standardized operations captured in S3, and establish an interval standardized operation model database and a key standardized operation model database.

[0011] S5. Operators simulate illegal work processes, and video images of the illegal work processes are collected.

[0012] S6. Extract the key operation segments from the video images of the illegal operation process, and capture one or more sets of consecutive multi-frame key operation segments from the video segments.

[0013] S7. Collect images of multiple groups of workers performing key operations in violation of regulations, compare the collected images with one or more groups of consecutive frames of key operations in violation of regulations captured in S6, and establish an interval violation operation model database and a key violation operation database.

[0014] S8. Embed video footage of standardized operating procedures, a database of key standardized operating models, video images of non-compliant operating procedures, and a database of key non-compliant operating procedures into AI smart devices;

[0015] S9. Use AI intelligent devices to monitor the work process of operators, extract video footage of the work process of operators during key operations, and identify and judge whether the work process of operators is standardized or violates regulations based on the model database embedded in the AI ​​intelligent devices.

[0016] S10. When the operator's work process is irregular or non-standard, retrieve the video footage of the standardized work process and the corresponding voice explanation information from the AI ​​intelligent device to provide auxiliary guidance to the operator's work process.

[0017] Furthermore, the specific steps for establishing the interval standardized operation model database and the key standardized operation model database in S4 are as follows:

[0018] S401. Compare the similarity of the key operation images of multiple sets of standardized operations with the key operation images of the corresponding set of continuous multi-frame standardized operations in the interval video images captured in S3.

[0019] S402. Obtain the key operation screen with the highest average similarity among consecutive key operation screens in the corresponding group and the lowest similarity value when compared with the key operation screen with the highest similarity.

[0020] S403. Obtain multiple sets of key operation frames with the highest average similarity, the lowest similarity value, and the corresponding voice explanation information from the same interval video frames in the standardized operation video frame, and combine them to form an interval standardized operation model database.

[0021] S404. Combine the standardized operation model databases of each section to form a key standardized operation model database.

[0022] Furthermore, the specific steps in S7 for establishing the interval violation operation model database and the key violation operation database include:

[0023] S701. Compare the similarity of the key operation images of multiple groups of illegal operations with the corresponding group of multiple consecutive frames of key operation images of illegal operations in the interval video images captured in S6.

[0024] S702. Obtain the key operation screen with the highest average similarity among multiple consecutive frames of key operation screens that violate regulations in the corresponding group, and the lowest similarity value when compared with the key operation screen with the highest similarity.

[0025] S703. Obtain multiple sets of key operation frames with the highest average similarity and the lowest similarity value in the same interval of video frames in the video images of illegal operations, and combine them to form an interval illegal operation model database.

[0026] S704. Combine the databases of illegal operations in each section to form a database of key illegal operations.

[0027] Furthermore, the method for similarity comparison in S4 and S7 is as follows:

[0028] Based on the structural similarity algorithm SSIM, the similarity between two images is calculated:

[0029] Brightness contrast function:

[0030] Contrast function:

[0031] Structure comparison function:

[0032] Where x and y are image samples, c1, c2, and c3 are constants, and μ x μ y The mean of x and y are respectively, and σ is the mean of x and y. 2 x σ 2 y The variances of x and y are σ and y, respectively. xy Let SSIM(x,y) be the covariance of x and y. The larger the value of SSIM(x,y), the higher the similarity between the two images.

[0033] Furthermore, the specific steps in S402 for comparing the key operation frame with the highest average similarity among consecutive key operation frames with the lowest similarity value are as follows:

[0034] S4021. Collect images of key operations from multiple sets of standardized operations. Using the SSIM algorithm, calculate the average similarity value between the images of key operations from multiple standardized operations and one image from a series of key operation frames in S402.

[0035]

[0036] S4022. Calculate the average similarity values ​​S1, S2, ..., S3 between the key operation frames in S402 and the key operation frames of multiple sets of standard operations. m ;

[0037] S4023. Based on the average similarity value of the key operation images obtained from multiple sets of standardized operations, obtain the key operation image with the highest average similarity among multiple consecutive frames of key operation images and the lowest similarity value S when compared with the key operation image with the highest similarity. min .

[0038] Furthermore, the specific steps in S702 for comparing the key operation frame with the highest average similarity among consecutive frames of key operation frames that violate regulations, and the lowest similarity value when comparing it with the key operation frame with the highest similarity are as follows:

[0039] S7021. Collect images of key operations from multiple sets of non-compliant operations, and use the SSIM algorithm to calculate the average similarity value between the images of key operations from multiple non-compliant operations and one operation image from multiple consecutive frames of key operation images in S402:

[0040]

[0041] S7022. Calculate the average similarity values ​​A1, A2, ..., A3 between the key operation images of multiple consecutive frames in S402 and the key operation images of multiple groups of violations. m ;

[0042] S7023. Based on the average similarity value of the key operation images of multiple sets of violations, obtain the key operation image with the highest average similarity among multiple consecutive frames of key operation images and the lowest similarity value A when compared with the key operation image with the highest similarity. min .

[0043] Furthermore, the specific steps in S9 for identifying and determining whether the operator's work process is standardized are as follows:

[0044] S901. Use AI intelligent equipment to monitor the work process of operators and extract video footage of the work process of operators during key operations.

[0045] S902, Divide the interval video frame into a continuous multi-frame image frame;

[0046] S903. Retrieve the key operation screen with the highest average similarity among all standardized operations within the interval standardized operation model database of the key standardized operation model database;

[0047] S904. Compare the key operation screen with the highest average similarity of the retrieved standard operation with the differentiated continuous multi-frame image screen in S902.

[0048] S905. Based on the lowest similarity value in the interval standardized operation model database, determine whether the operator's operation process is standardized.

[0049] Furthermore, the specific steps in S9 for identifying and determining whether the operator's work process violates regulations are as follows:

[0050] S906. Retrieve the key operation screen with the highest average similarity among all non-compliant operations within the interval non-compliant operation model database of the key non-compliant operation database;

[0051] S907. Compare the key operation images with the highest average similarity of the retrieved violation operation images with the differentiated continuous multi-frame images in S902.

[0052] S908. Based on the lowest similarity value in the interval violation operation model database, determine whether the operator's operation process is in violation of regulations.

[0053] Furthermore, the specific method for judging whether the operator's work process is standardized is as follows:

[0054] When the similarity value of each key operation image with the highest average similarity among the retrieved standardized operation images exceeds the lowest similarity value S in the interval standardized operation model database, the similarity value of each image with the highest average similarity among the retrieved images exceeds the lowest similarity value S. min When determining the standardization of the operator's work process, if the similarity value of one or more key operation images with the highest average similarity to the differentiated consecutive frames in S902 does not exceed the lowest similarity value S in the interval standard operation model database, then the operator's work process is deemed to be in compliance with the standardization criteria. min At that time, it was determined that the operator's work process was not standardized.

[0055] Furthermore, the specific method for determining whether the operator's work process violated regulations is as follows:

[0056] When the similarity value of each key operation image with the highest average similarity among the retrieved images of the violation operation is compared with the similarity value of the differentiated multi-frame images in S902, and none of the similarity values ​​exceed the lowest similarity value A in the interval violation operation model database, the following similarity values ​​are obtained. min When it is determined that the operator's work process is not in violation of regulations, if the similarity value of one or more key operation images with the highest average similarity to the differentiated consecutive frames of images in S902 exceeds the lowest similarity value A in the interval violation operation model database. min At that time, it was determined that the operator's work process was in violation of regulations.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] This AI-powered job guidance and assistance method collects video images of workers performing standardized work processes, audio explanations of the work processes, and video images of workers performing non-standard work processes. It processes the collected data to build a model database for judging workers' work standards. Monitoring equipment monitors the workers' work processes and identifies their actions. The method can intelligently identify and judge workers' work processes. When a worker's actions are determined to be non-standard or non-compliant, it can proactively issue a warning and provide work assistance guidance using standardized video images and audio explanations. This helps to promptly identify non-standard or non-compliant work actions and improves the effectiveness of job guidance. Attached Figure Description

[0059] Figure 1 This is a flowchart of the AI-assisted homework guidance method of the present invention;

[0060] Figure 2 The flowchart for establishing the interval standardized operation model database and the key standardized operation model database in this invention is shown below;

[0061] Figure 3 The flowchart shows the process of establishing the interval violation operation model database and the key violation operation database in this invention. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] An example of this AI-assisted homework guidance method is as follows:

[0064] Please see Figures 1-3 An AI-assisted homework guidance method includes the following specific steps:

[0065] S1. Collect video images of the standardized operation process of the operators and the audio explanation information of the operators during the operation;

[0066] S2. Extract the video footage and corresponding audio narration information for multiple key operations from the video images;

[0067] S3. Extract one or more sets of key operation frames from the same video frame, and extract the corresponding audio explanation information.

[0068] S4. Collect images of workers performing key tasks in multiple sets of standardized operations. Compare the collected images with the key task images of one or more consecutive frames of standardized operations captured in S3 to establish an interval standardized operation model database and a key standardized operation model database.

[0069] S401. Compare the similarity of the key operation images of multiple sets of standardized operations with the key operation images of the corresponding set of continuous multi-frame standardized operations in the interval video images captured in S3.

[0070] S402. Obtain the key operation frame with the highest average similarity among consecutive key operation frames in the corresponding group, and the lowest similarity value when comparing it with the key operation frame with the highest similarity:

[0071] S4021. Collect images of key operations from multiple sets of standardized operations. Using the SSIM algorithm, calculate the average similarity value between the images of key operations from multiple standardized operations and one image from a series of key operation frames in S402.

[0072]

[0073] S4022. Calculate the average similarity values ​​S1, S2, ..., S3 between the key operation frames in S402 and the key operation frames of multiple sets of standard operations. m ;

[0074] S4023. Based on the average similarity value of the key operation images obtained from multiple sets of standardized operations, obtain the key operation image with the highest average similarity among multiple consecutive frames of key operation images and the lowest similarity value S when compared with the key operation image with the highest similarity. min ;

[0075] S403. Obtain multiple sets of key operation frames with the highest average similarity, the lowest similarity value, and the corresponding voice explanation information from the same interval video frames in the standardized operation video frame, and combine them to form an interval standardized operation model database.

[0076] S404. Combine the standardized operation model databases of each section to form a key standardized operation model database.

[0077] S5. Operators simulate illegal work processes, and video images of the illegal work processes are collected.

[0078] S6. Extract the key operation segments from the video images of the illegal operation process, and capture one or more sets of consecutive multi-frame key operation segments from the video segments.

[0079] S7. Collect images of workers performing key operations during multiple sets of violations, and compare the collected images with one or more sets of consecutive frames of key operation images captured in S6 to establish an interval violation operation model database and a key violation operation database.

[0080] S701. Compare the similarity of the key operation images of multiple groups of illegal operations with the corresponding group of multiple consecutive frames of key operation images of illegal operations in the interval video images captured in S6.

[0081] S702. Obtain the key operation frame with the highest average similarity among multiple consecutive frames of key operation frames that violate regulations in the corresponding group, and the lowest similarity value when comparing it with the key operation frame with the highest similarity:

[0082] S7021. Collect images of key operations from multiple sets of non-compliant operations, and use the SSIM algorithm to calculate the average similarity value between the images of key operations from multiple non-compliant operations and one operation image from multiple consecutive frames of key operation images in S402:

[0083]

[0084] S7022. Calculate the average similarity values ​​A1, A2, ..., A3 between the key operation images of multiple consecutive frames in S402 and the key operation images of multiple groups of violations. m ;

[0085] S7023. Based on the average similarity value of the key operation images obtained from multiple sets of violations, obtain the key operation image with the highest average similarity among multiple consecutive frames of key operation images and the lowest similarity value A when compared with the key operation image with the highest similarity. min ;

[0086] S703. Obtain multiple sets of key operation frames with the highest average similarity and the lowest similarity value in the same interval of video frames in the video images of illegal operations, and combine them to form an interval illegal operation model database.

[0087] S704. Combine the databases of illegal operations in each section to form a database of key illegal operations.

[0088] Based on the structural similarity algorithm SSIM, the similarity between two images is calculated:

[0089] Brightness contrast function:

[0090] Contrast function:

[0091] Structure comparison function:

[0092] Where x and y are image samples, c1, c2, and c3 are constants, and μ x μ y The mean of x and y are respectively, and σ is the mean of x and y. 2 x σ 2 y The variances of x and y are σ and y, respectively. xy Let SSIM(x,y) be the covariance of x and y. The larger the value of SSIM(x,y), the higher the similarity between the two images.

[0093] S8. Embed video footage of standardized operating procedures, a database of key standardized operating models, video images of non-compliant operating procedures, and a database of key non-compliant operating procedures into AI smart devices;

[0094] S9. Utilize AI-powered intelligent devices to monitor the work processes of operators, extract video footage of key work periods, and identify and determine whether the operators' work processes are standardized or violate regulations based on the model database embedded in the AI-powered intelligent devices.

[0095] S901. Use AI intelligent equipment to monitor the work process of operators and extract video footage of the work process of operators during key operations.

[0096] S902, Divide the interval video frame into a continuous multi-frame image frame;

[0097] S903. Retrieve the key operation screen with the highest average similarity among all standardized operations within the interval standardized operation model database of the key standardized operation model database;

[0098] S904. Compare the key operation screen with the highest average similarity of the retrieved standard operation with the differentiated continuous multi-frame image screen in S902.

[0099] S905. Based on the lowest similarity value in the interval standardized operation model database, determine whether the operator's operation process is standardized.

[0100] S906. Retrieve the key operation screen with the highest average similarity among all non-compliant operations within the interval non-compliant operation model database of the key non-compliant operation database;

[0101] S907. Compare the key operation images with the highest average similarity of the retrieved violation operation images with the differentiated continuous multi-frame images in S902.

[0102] S908. Based on the lowest similarity value in the interval violation operation model database, determine whether the operator's operation process is in violation of regulations.

[0103] S10. When workers violate regulations or work procedures, the system retrieves video footage and corresponding audio explanations of standardized work procedures from the AI-powered smart device to provide auxiliary guidance to the workers.

[0104] When the similarity value of each key operation image with the highest average similarity among the retrieved standardized operation images exceeds the lowest similarity value S in the interval standardized operation model database, the similarity value of each image with the highest average similarity among the retrieved images exceeds the lowest similarity value S. min When determining the standardization of the operator's work process, if the similarity value of one or more key operation images with the highest average similarity to the differentiated consecutive frames in S902 does not exceed the lowest similarity value S in the interval standard operation model database, then the operator's work process is deemed to be in compliance with the standardization criteria. min At that time, it was determined that the operator's work process was not standardized.

[0105] When the similarity value of each key operation image with the highest average similarity among the retrieved images of the violation operation is compared with the similarity value of the differentiated multi-frame images in S902, and none of the similarity values ​​exceed the lowest similarity value A in the interval violation operation model database, the following similarity values ​​are obtained. min When it is determined that the operator's work process is not in violation of regulations, if the similarity value of one or more key operation images with the highest average similarity to the differentiated consecutive frames of images in S902 exceeds the lowest similarity value A in the interval violation operation model database. min At that time, it was determined that the operator's work process was in violation of regulations.

[0106] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An AI-assisted homework guidance method, characterized in that: Includes the following steps: S1. Collect video images of the standardized operation process of the operators and the audio explanation information of the operators during the operation; S2. Extract the video footage and corresponding audio narration information for multiple key operations from the video images; S3. Extract one or more sets of key operation frames from the same video frame, and extract the corresponding audio explanation information. S4. Collect images of workers performing key tasks in multiple sets of standardized operations, compare the collected images with the key task images of one or more sets of continuous multi-frame standardized operations captured in S3, and establish an interval standardized operation model database and a key standardized operation model database. S5. Operators simulate illegal work processes, and video images of the illegal work processes are collected. S6. Extract the key operation segments from the video images of the illegal operation process, and capture one or more sets of consecutive multi-frame key operation segments from the video segments. S7. Collect images of multiple groups of workers performing key operations in violation of regulations, compare the collected images with one or more groups of consecutive frames of key operations in violation of regulations captured in S6, and establish an interval violation operation model database and a key violation operation database. S8. Embed video footage of standardized operating procedures, a database of key standardized operating models, video images of non-compliant operating procedures, and a database of key non-compliant operating procedures into AI smart devices; S9. Use AI intelligent devices to monitor the work process of operators, extract video footage of the work process of operators during key operations, and identify and judge whether the work process of operators is standardized or violates regulations based on the model database embedded in the AI ​​intelligent devices. S10. When the operator's work process is irregular or non-standard, retrieve the video footage of the standardized work process and the corresponding voice explanation information from the AI ​​intelligent device to provide auxiliary guidance to the operator's work process.

2. The AI-assisted homework guidance method according to claim 1, characterized in that: The specific steps for establishing the interval standardized operation model database and the key standardized operation model database in S4 are as follows: S401. Compare the similarity of the key operation images of multiple sets of standardized operations with the key operation images of the corresponding set of continuous multi-frame standardized operations in the interval video images captured in S3. S402. Obtain the key operation screen with the highest average similarity among consecutive key operation screens in the corresponding group and the lowest similarity value when compared with the key operation screen with the highest similarity. S403. Obtain multiple sets of key operation frames with the highest average similarity, the lowest similarity value, and the corresponding voice explanation information from the same interval video frames in the standardized operation video frame, and combine them to form an interval standardized operation model database. S404. Combine the standardized operation model databases of each section to form a key standardized operation model database.

3. The AI-assisted homework guidance method according to claim 2, characterized in that: The specific steps in S402 for obtaining the key operation frame with the highest average similarity among consecutive key operation frames and the lowest similarity value when comparing it with the key operation frame with the highest similarity are as follows: S4021. Collect images of key operations from multiple sets of standardized operations. Using the SSIM algorithm, calculate the average similarity value between the images of key operations from multiple standardized operations and one image from a series of key operation frames in S402. ， S4022. Calculate the average similarity value between the key operation images of multiple consecutive frames in S402 and the key operation images of multiple sets of standard operations. ; S4023. Based on the average similarity value of the key operation images obtained from multiple sets of standardized operations, obtain the key operation image with the highest average similarity among multiple consecutive key operation images and the lowest similarity value when compared with the key operation image with the highest similarity. .

4. The AI-assisted homework guidance method according to claim 1, characterized in that: The specific steps for establishing the interval violation operation model database and the key violation operation database in S7 include: S701. Compare the similarity of the key operation images of multiple groups of illegal operations with the corresponding group of multiple consecutive frames of key operation images of illegal operations in the interval video images captured in S6. S702. Obtain the key operation screen with the highest average similarity among multiple consecutive frames of key operation screens that violate regulations in the corresponding group, and the lowest similarity value when compared with the key operation screen with the highest similarity. S703. Obtain multiple sets of key operation frames with the highest average similarity and the lowest similarity value in the same interval of video frames in the video images of illegal operations, and combine them to form an interval illegal operation model database. S704. Combine the databases of illegal operations in each section to form a database of key illegal operations.

5. The AI-assisted homework guidance method according to claim 4, characterized in that: The specific steps in S702 for obtaining the key operation frame with the highest average similarity among consecutive frames of key operation frames with violations and comparing it with the lowest similarity value is as follows: S7021. Collect images of key operations from multiple sets of non-compliant operations. Using the SSIM algorithm, calculate the average similarity value between the images of key operations from multiple non-compliant operations and one operation image from a series of consecutive frames of key non-compliant operations in S702. ， S7022. Calculate the average similarity value between the key operation images of multiple consecutive frames of violations in S702 and the key operation images of multiple sets of violations. ; S7023. Based on the average similarity value of the key operation images of multiple sets of violations, obtain the key operation image with the highest average similarity among multiple consecutive frames of key operation images and the lowest similarity value when compared with the key operation image with the highest similarity. .

6. The AI-assisted homework guidance method according to claim 1, characterized in that: The similarity comparison methods in S4 and S7 are as follows: Based on the structural similarity algorithm SSIM, the similarity between two images is calculated: Brightness contrast function: , Contrast function: , Structure comparison function: , ， Where x and y are image samples, It is a constant. Let x and y be the means, respectively. Let x and y be the variances, respectively. Let x and y be the covariances. The higher the value, the higher the similarity between the two images.

7. The AI-assisted homework guidance method according to claim 1, characterized in that: The specific steps in S9 to identify and determine whether the operator's work process is standardized are as follows: S901. Use AI intelligent equipment to monitor the work process of operators and extract video footage of the work process of operators during key operations. S902, Divide the interval video frame into a continuous multi-frame image frame; S903. Retrieve the key operation screen with the highest average similarity among all standardized operations within the corresponding interval of the key standardized operation model database. S904. Compare the key operation screen with the highest average similarity of the retrieved standard operation with the differentiated continuous multi-frame image screen in S902. S905. Based on the lowest similarity value in the interval standardized operation model database, determine whether the operator's operation process is standardized. The specific steps in S9 to identify and determine whether the operator's work process violates regulations are as follows: S906. Retrieve the key operation screen with the highest average similarity among all non-compliant operations within the corresponding interval of the key non-compliant operation database; S907. Compare the key operation images with the highest average similarity of the retrieved violation operation images with the differentiated continuous multi-frame images in S902. S908. Based on the lowest similarity value in the interval violation operation model database, determine whether the operator's operation process is in violation of regulations.

8. The AI-assisted homework guidance method according to claim 7, characterized in that: The specific method for judging whether the operator's work process is standardized is as follows: When the similarity value of each key operation image with the highest average similarity among the retrieved standardized operation images exceeds the lowest similarity value in the interval standardized operation model database, the comparison between the image and the differentiated multi-frame images in S902 shows that the similarity value exceeds the lowest similarity value. When determining the standardization of the operator's work process, if the similarity value of one or more key operation images with the highest average similarity to the differentiated consecutive frames in S902 does not exceed the lowest similarity value in the interval standard operation model database. At that time, it was determined that the operator's work process was not standardized; The specific method for determining whether the operator's work process violated regulations is as follows: When the similarity value of each key operation image with the highest average similarity among the retrieved images of the violation operation is compared with the similarity value of the differentiated multi-frame images in S902, none of them exceed the lowest similarity value in the interval violation operation model database. When it is determined that the operator's work process is not in violation of regulations, if the similarity value of one or more key operation images with the highest average similarity to the differentiated consecutive frames of images in S902 exceeds the lowest similarity value in the interval violation operation model database. At that time, the operator was deemed to have violated regulations during the work process.

Images