An operation recognition method and system based on artificial intelligence
By designing an operation recognition system based on artificial intelligence, using universal adjustment bracket, 360° rotating hose and deep learning algorithm, the problem of traditional operation supervision relying on manual and insufficient flexibility is solved, and efficient and accurate operation step judgment and auxiliary learning functions are achieved.
Patent Information
- Application Number
- CN202411935520.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-26
- Publication Date
- 2025-04-01
AI Technical Summary
Traditional operation supervision relies on labor, consumes a lot of labor costs and is prone to subjective judgment errors. The existing auxiliary tools are insufficient in flexibility and functionality, making it difficult to meet the needs of complex and changeable operating environments.
An operation recognition system based on artificial intelligence is designed, including a universal adjustment bracket module, a 360° rotating hose module, a terminal device module, an operation recognition module and a blackboard functional module, and real-time judgment and feedback of operation steps are achieved through deep learning and support vector machine algorithms.
The system can achieve all-round and flexible adjustments in three-dimensional space, judge the correctness of operation steps in real time and accurately, reduce labor costs, improve work and training efficiency, and provide convenient blackboard writing and scaling functions to assist in operation practice and skill learning.
Smart Images
Figure CN119741758A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an operation recognition method and system based on artificial intelligence. Background Art
[0002] In many operation scenarios, such as industrial production, medical surgery, education and training, etc., the accuracy of the operator's operating steps is required to be extremely high.
[0003] Traditional operation supervision often relies on manual work, which not only consumes a lot of manpower costs, but is also prone to subjective judgment errors. With the development of technology, the demand for intelligent operation recognition is becoming increasingly urgent. Some existing auxiliary tools are insufficient in flexibility and functionality, and it is difficult to meet the needs of complex and changing operating environments. For example, it is impossible to flexibly adjust the viewing angle in all directions, and intelligently identify the correctness of operations. In this regard, we propose an operation recognition method and system based on artificial intelligence to solve the above problems. Summary of the invention
[0004] In view of the deficiencies in the prior art, the present invention provides an operation recognition method and system based on artificial intelligence, which solves the problems raised in the background technology.
[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: an operation identification system based on artificial intelligence, including a universal adjustment bracket module, a 360° rotating hose module, a terminal device module, an operation identification module, and a blackboard writing function module, the output end of the universal adjustment bracket module is electrically connected to the input end of the 360° rotating hose module, the output end of the 360° rotating hose module is electrically connected to the input end of the terminal device module, the output end of the terminal device module is electrically connected to the input end of the operation identification module, and the output end of the operation identification module is electrically connected to the input end of the blackboard writing function module.
[0006] Preferably, the universal adjustment bracket module includes a moving unit, an angle adjustment unit, and an angle adjustment unit. The moving unit adopts high-precision rollers and guide rails to enable the bracket to move smoothly and accurately on a plane, and the rollers are equipped with damping devices and can be fixed at any position. The angle adjustment unit uses a multi-joint mechanical arm structure, and each joint is equipped with a high-precision servo to achieve universal adjustment of the bracket in three-dimensional space with an adjustment accuracy of 0.1 degrees.
[0007] Preferably, the 360° rotating hose module includes a rotation drive unit and an angle monitoring unit. The rotation drive unit has a built-in micro motor connected to the outer wall of the hose through a gear transmission system to drive the hose to rotate 360°. The motor has an overload protection function, and the rotation speed can be accurately adjusted from 0 to 10 rpm. The angle monitoring unit uses Hall sensors evenly distributed along the circumference of the hose to monitor the rotation angle of the hose in real time, with a monitoring accuracy of ±0.5 degrees.
[0008] Preferably, the terminal device module includes a built-in battery unit, an automatic focus and resolution adjustment unit, and a fill light unit. The built-in battery unit is a high-energy-density lithium battery with a capacity of 5000mAh and an intelligent charge and discharge management chip. It automatically warns when the power is lower than 20%, supports fast charging, and can be fully charged to 80% within 1 hour. The automatic focus and resolution adjustment unit uses a voice coil motor to drive the lens to focus, combined with an image sensor, and uses a fuzzy control algorithm to complete automatic focus within 0.1 seconds. The focus distance is 0.1 meters to infinity, and the image resolution can be adjusted between 720p-4K as needed through a digital image processing chip. The fill light unit is composed of a ring-shaped light array of multiple high-color rendering LED lights arranged around the lens. Through PWM dimming technology, the brightness of 0-1000 lumens and the color temperature of 2700K-6500K can be switched, and automatic fill light can be provided according to ambient light.
[0009] Preferably, the operation recognition module includes an image acquisition unit, an image preprocessing unit, a feature extraction unit, and an operation judgment unit. The image acquisition unit controls the camera of the terminal device to periodically acquire images of the operation area. The acquisition frame rate can be set between 25fps and 60fps, and the image sequence is transmitted to the image preprocessing unit. The image preprocessing unit performs denoising, grayscale, and normalization on the acquired image, uses median filtering to remove salt and pepper noise, and uses histogram equalization to enhance image contrast, and then sends the preprocessed image to the feature extraction unit. The feature extraction unit automatically extracts key features such as operation objects, tools, and operator hand movements based on a convolutional neural network model of deep learning, and transmits the extracted feature vector to the operation judgment unit. The operation judgment unit compares the received feature vector with a preset standard operation feature library, and uses a support vector machine classification algorithm to determine the correctness of the operation steps. If an error occurs, an error prompt message is generated and sent to the display control module.
[0010] Preferably, the blackboard writing function module includes an image annotation unit and a zoom control unit. The image annotation unit receives the image transmitted by the operation recognition module. The operator can circle, draw lines, and add text annotations on the image through a touch screen or an external mouse, handwriting board, etc. The annotation information is superimposed on the original image in the form of a layer. The zoom control unit adopts a bilinear interpolation algorithm to enlarge or reduce the annotated image according to user instructions. The zoom ratio is continuously adjustable between 0.5-5 times, and the zoomed image is updated and displayed in real time.
[0011] The present invention discloses an operation recognition method based on artificial intelligence, which specifically comprises the following steps: Step 1: System initialization phase: The universal adjustment bracket module moves to the designated location according to the preset position information and is adjusted to the initial shooting angle through the angle adjustment unit; the 360° rotating hose module returns to zero position, the terminal device module starts, completes the self-check of the camera, battery, fill light and other components, and the software system loads the pre-trained deep learning model and standard operation feature library; Step 2, operation monitoring stage: the image acquisition unit continuously acquires images of the operation area at a set frame rate, and through a series of processes including image preprocessing, feature extraction, and operation judgment, the correctness of the operation is judged in real time; at the same time, if the operator turns on the blackboard writing function, the image annotation unit responds to the operation at any time and records the annotation information; Step 3, feedback adjustment stage: When the operation judgment unit finds an operation error, a striking error prompt box will pop up on the terminal screen immediately through the display control module, displaying the error type and correction suggestions; the operator adjusts the operation according to the prompt, and the system continues to monitor until the operation is completed correctly; during the whole process, the zoom control unit can be used as needed to view details to assist operation and learning.
[0012] Beneficial Effects The present invention provides an artificial intelligence-based operation recognition method and system. Compared with the prior art, it has the following beneficial effects: (1) The universal adjustment bracket is combined with a 360° rotating hose to achieve full and flexible adjustment in three-dimensional space. Whether it is a complex industrial production workshop or a medical operating environment that requires precision, it can ensure that there are no blind spots in the shooting and accurately capture every detail of the operating area to meet the layout requirements of different scenes.
[0013] (2) The built-in advanced high-intelligence AI system, based on deep learning convolutional neural network and support vector machine algorithm, can accurately judge in real time whether each step of the operator is correct. Compared with traditional manual supervision, it greatly improves the accuracy of judgment, while significantly reducing labor cost investment and improving work and training efficiency.
[0014] (3) The terminal equipment has high-performance imaging and fill-in-light capabilities, and the blackboard and zoom functions at the software level are practical and convenient. Operators can operate under conditions of sufficient light and clear images, and can also mark and zoom in on key image information at any time. This provides strong support for learning complex operating procedures and mastering fine operating points, and effectively assists in operating practice and skill learning. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 is a system block diagram of the present invention; Figure 2 It is a schematic diagram of a universal adjustment bracket module of the present invention; Figure 3 This is a schematic diagram of a 360° rotating hose module of the present invention; Figure 4 It is a schematic diagram of a terminal device module of the present invention; Figure 5 This is a schematic diagram of an operation identification module of the present invention; Figure 6 It is a schematic diagram of the blackboard writing function module of the present invention.
[0016] In the figure: 01, universal adjustment bracket module; 02, 360° rotating hose module; 03, terminal equipment module; 04, operation identification module; 05, blackboard writing function module; 011, moving unit; 012, angle adjustment unit; 021, rotation drive unit; 022, angle monitoring unit; 031, built-in battery unit; 032, automatic focus and resolution adjustment unit; 033, fill light unit; 041, image acquisition unit; 042, image preprocessing unit; 043, feature extraction unit; 044, operation judgment unit; 051, image annotation unit; 052, zoom control unit. DETAILED DESCRIPTION
[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0018] The present invention provides three technical solutions, including the following embodiments: Example 1 See also Figure 1-Figure 3An operation recognition system based on artificial intelligence includes a universal adjustment bracket module 01, a 360° rotating hose module 02, a terminal device module 03, an operation recognition module 04, and a blackboard writing function module 05. The output end of the universal adjustment bracket module 01 is electrically connected to the input end of the 360° rotating hose module 02, the output end of the 360° rotating hose module 02 is electrically connected to the input end of the terminal device module 03, the output end of the terminal device module 03 is electrically connected to the input end of the operation recognition module 04, and the output end of the operation recognition module 04 is electrically connected to the input end of the blackboard writing function module 05.
[0019] The universal adjustment bracket module 01 includes a moving unit 011, an angle adjustment unit 012, and an angle adjustment unit 012. The moving unit 011 adopts high-precision rollers and guide rails to enable the bracket to move smoothly and accurately on a plane, and the rollers are equipped with damping devices and can be fixed at any position. The angle adjustment unit 012 uses a multi-joint mechanical arm structure, and each joint is equipped with a high-precision servo to achieve universal adjustment of the bracket in three-dimensional space with an adjustment accuracy of 0.1 degrees.
[0020] The 360° rotating hose module 02 includes a rotating drive unit 021 and an angle monitoring unit 022. The rotating drive unit 021 has a built-in micro motor connected to the outer wall of the hose through a gear transmission system to drive the hose to rotate 360°. The motor has an overload protection function and the rotation speed can be accurately adjusted from 0 to 10 rpm. The angle monitoring unit 022 uses Hall sensors evenly distributed along the circumference of the hose to monitor the hose rotation angle in real time with a monitoring accuracy of ±0.5 degrees.
[0021] Example 2 Based on Example 1, see Figure 4-Figure 6 As shown, the terminal device module 03 includes a built-in battery unit 031, an automatic focus and resolution adjustment unit 032, and a fill light unit 033. The built-in battery unit 031 uses a high-energy density lithium battery with a capacity of 5000mAh and an intelligent charge and discharge management chip. It automatically warns when the power is lower than 20%, supports fast charging, and can be fully charged to 80% within 1 hour. The automatic focus and resolution adjustment unit 032 uses a voice coil motor to drive the lens focus. Combined with the image sensor, the fuzzy control algorithm is used to complete automatic focus within 0.1 seconds. The focus distance is 0.1 meters to infinity, and the image resolution can be adjusted between 720p-4K as needed through the digital image processing chip. The fill light unit 033 is composed of a ring array of multiple high-color rendering LED lights arranged around the lens. Through PWM dimming technology, the brightness is 0-1000 lumens and the color temperature is 2700K-6500K switchable, and automatic fill light is provided according to the ambient light.
[0022] The operation recognition module 04 includes an image acquisition unit 041, an image preprocessing unit 042, a feature extraction unit 043, and an operation judgment unit 044. The image acquisition unit 041 controls the camera of the terminal device to regularly acquire images of the operation area. The acquisition frame rate can be set between 25fps and 60fps, and transmits the image sequence to the image preprocessing unit. The image preprocessing unit 042 performs denoising, grayscale, and normalization on the acquired image, uses median filtering to remove salt and pepper noise, and uses histogram equalization to enhance image contrast, and then sends the preprocessed image to the feature extraction unit. The feature extraction unit 043 automatically extracts key features such as operation objects, tools, and operator hand movements based on a deep learning convolutional neural network model, and transmits the extracted feature vector to the operation judgment unit. The operation judgment unit 044 compares the received feature vector with a preset standard operation feature library, and uses a support vector machine classification algorithm to determine the correctness of the operation steps. If an error occurs, an error prompt message is generated and sent to the display control module.
[0023] The blackboard writing function module 05 includes an image annotation unit 051 and a zoom control unit 052. The image annotation unit 051 receives the image transmitted by the operation recognition module. The operator can circle, draw lines, and add text annotations on the image through a touch screen or an external mouse, handwriting board, etc. The annotation information is superimposed on the original image in the form of a layer. The zoom control unit 052 adopts a bilinear interpolation algorithm to enlarge or reduce the annotated image according to user instructions. The zoom ratio is continuously adjustable between 0.5-5 times, and the image is updated and displayed in real time after zooming.
[0024] Example 3 On the basis of Example 2, the present invention further discloses an operation recognition method based on artificial intelligence, which specifically comprises the following steps: Step 1, system initialization stage: the universal adjustment bracket module 01 moves to the designated location according to the preset position information, and adjusts to the initial shooting angle through the angle adjustment unit 012; the 360° rotating hose module 02 returns to the zero position, the terminal device module 03 starts, completes the self-check of the camera, battery, fill light and other components, and the software system loads the pre-trained deep learning model and standard operation feature library; Step 2, operation monitoring stage: the image acquisition unit 041 continuously acquires images of the operation area at a set frame rate, and judges the correctness of the operation in real time through a series of processes including image preprocessing, feature extraction, and operation judgment; at the same time, if the operator turns on the blackboard writing function, the image annotation unit 051 responds to the operation at any time and records the annotation information; Step 3, feedback adjustment stage: When the operation judgment unit 044 finds an operation error, a striking error prompt box will pop up on the terminal screen immediately through the display control module, displaying the error type and correction suggestions; the operator adjusts the operation according to the prompt, and the system continues to monitor until the operation is completed correctly; during the whole process, the zoom control unit 052 can be used as needed to view details to assist operation and learning.
[0025] Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.
[0026] The above embodiments of the invention are described in detail, but the contents are only preferred embodiments of the invention and cannot be considered to limit the scope of the invention. All equivalent changes and improvements made within the scope of the invention should still fall within the scope of the invention.
Claims
1. An artificial intelligence-based operation recognition system, characterized in that: The invention comprises a universal adjustment bracket module (01), a 360° rotating hose module (02), a terminal device module (03), an operation identification module (04), and a blackboard writing function module (05), wherein the output end of the universal adjustment bracket module (01) is electrically connected to the input end of the 360° rotating hose module (02), the output end of the 360° rotating hose module (02) is electrically connected to the input end of the terminal device module (03), the output end of the terminal device module (03) is electrically connected to the input end of the operation identification module (04), and the output end of the operation identification module (04) is electrically connected to the input end of the blackboard writing function module (05).
2. The artificial intelligence-based operation recognition system according to claim 1, characterized in that: The universal adjustment bracket module (01) comprises a moving unit (011), an angle adjustment unit (012), and an angle adjustment unit (012). The moving unit (011) uses a high-precision roller in conjunction with a guide rail so that the bracket can move smoothly and accurately on a plane, and the roller is provided with a damping device so that it can be fixed at any position. The angle adjustment unit (012) uses a multi-joint mechanical arm structure, and each joint is equipped with a high-precision steering gear to achieve universal adjustment of the bracket in three-dimensional space, with an adjustment accuracy of 0.1 degrees.
3. The artificial intelligence-based operation recognition system according to claim 1, characterized in that: The 360° rotating hose module (02) comprises a rotating drive unit (021) and an angle monitoring unit (022). The rotating drive unit (021) has a built-in micro motor connected to the outer wall of the hose via a gear transmission system to drive the hose to rotate 360°. The motor has an overload protection function and the rotation speed can be accurately adjusted at 0-10 rpm. The angle monitoring unit (022) uses Hall sensors evenly distributed along the circumference of the hose to monitor the hose rotation angle in real time, with a monitoring accuracy of ±0.5 degrees.
4. The artificial intelligence-based operation recognition system according to claim 1, characterized in that: The terminal device module (03) comprises a built-in battery unit (031), an automatic focus and resolution adjustment unit (032), and a fill light unit (033). The built-in battery unit (031) is a high-energy-density lithium battery with a capacity of 5000mAh, an intelligent charge and discharge management chip, an automatic warning when the power is less than 20%, and supports fast charging, which can fully charge 80% of the power within 1 hour. The automatic focus and resolution adjustment unit (032) uses a voice coil motor to drive the lens focus, combined with an image sensor, and uses a fuzzy control algorithm to complete automatic focus within 0.1 seconds. The focus distance is 0.1 meters to infinity, and the image resolution can be adjusted between 720p-4K as needed through a digital image processing chip. The fill light unit (033) is composed of a ring light array composed of multiple high-color rendering LED lights arranged around the lens. Through PWM dimming technology, the brightness is switchable from 0 to 1000 lumens and the color temperature is 2700K to 6500K, and the fill light is automatically adjusted according to the ambient light.
5. The artificial intelligence-based operation recognition system according to claim 1, characterized in that: The operation recognition module (04) comprises an image acquisition unit (041), an image preprocessing unit (042), a feature extraction unit (043), and an operation judgment unit (044). The image acquisition unit (041) controls the camera of the terminal device to regularly acquire images of the operation area. The acquisition frame rate can be set between 25fps and 60fps, and transmits the image sequence to the image preprocessing unit. The image preprocessing unit (042) performs denoising, grayscale, and normalization processing on the acquired image, uses median filtering to remove salt and pepper noise, and uses histogram equalization to enhance image contrast. The preprocessed image is then sent to the feature extraction unit. The feature extraction unit (043) automatically extracts key features such as the operation object, tool, and operator's hand movements based on a deep learning convolutional neural network model, and transmits the extracted feature vector to the operation judgment unit. The operation judgment unit (044) compares the received feature vector with a preset standard operation feature library, uses a support vector machine classification algorithm to determine the correctness of the operation step, and generates an error prompt message if an error occurs and sends it to the display control module.
6. The artificial intelligence-based operation recognition system according to claim 1, characterized in that: The blackboard writing function module (05) comprises an image annotation unit (051) and a zoom control unit (052). The image annotation unit (051) receives the image transmitted by the operation recognition module. The operator can circle, draw lines, and add text annotations on the image by touching the screen or an external mouse, handwriting board, etc. The annotation information is superimposed on the original image in the form of a layer. The zoom control unit (052) adopts a bilinear interpolation algorithm to enlarge or reduce the annotated image according to user instructions. The zoom ratio is continuously adjustable between 0.5 and 5 times. The zoomed image is updated and displayed in real time.
7. An operation recognition method based on artificial intelligence, characterized in that: The specific steps include: Step 1, system initialization stage: the universal adjustment bracket module (01) moves to the designated location according to the preset position information, and is adjusted to the initial shooting angle through the angle adjustment unit (012); the 360° rotating hose module (02) returns to the zero position, the terminal device module (03) is started, and the camera, battery, fill light and other components are self-checked, and the software system loads the pre-trained deep learning model and standard operation feature library; Step 2, operation monitoring stage: the image acquisition unit (041) continuously acquires images of the operation area at a set frame rate, and judges the correctness of the operation in real time through a series of processes including image preprocessing, feature extraction, and operation judgment; at the same time, if the operator turns on the blackboard writing function, the image annotation unit (051) responds to the operation at any time and records the annotation information; Step 3, feedback adjustment stage: When the operation judgment unit (044) finds an operation error, a striking error prompt box will pop up on the terminal screen through the display control module, displaying the error type and correction suggestions; the operator adjusts the operation according to the prompt, and the system continues to monitor until the operation is completed correctly; during the whole process, the zoom control unit (052) can be used as needed to view details to assist operation and learning.