A lightweight real-time image recognition system and method for intelligent terminals

Abstract

The application discloses a kind of lightweight real-time image recognition systems and methods for intelligent terminal, comprising: the hardware state of power dynamic perception module collection terminal generates power level data;Module linkage control center generates scheduling instruction according to power level data;Lightweight attention module generates attention weight matrix according to scheduling instruction;Hierarchical feature extraction module triggers the feature extraction of corresponding semantic level according to scheduling instruction, and attention weight matrix is fused to convolution calculation process, and output feature map;Recognition result output module obtains recognition result and feedback confidence data by processing feature map;Small sample self-learning module generates new scene feature parameters by metric learning;Module linkage control center drives attention module parameter iteration according to new scene feature parameters and carries out incremental update to fine-grained semantic feature layer.The application realizes terminal side power self-adaptation, scene customization recognition and lightweight self-learning.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a lightweight real-time image recognition system and method for smart terminals. Background Technology

[0002] With the continuous development of intelligent manufacturing, industrial automation, and edge intelligence technologies, intelligent terminals such as industrial tablets, embedded acquisition terminals, and inspection terminals have been widely used in industrial image recognition scenarios such as component inspection, material identification, product traceability, and equipment inspection. These applications typically require intelligent terminals to be able to directly complete image acquisition, processing, and recognition tasks in the field environment, thus placing high demands on the real-time performance, lightweight design, local processing capabilities, and recognition accuracy of image recognition systems.

[0003] However, due to limitations in hardware resources, power consumption, and operating environment, smart terminals often need to simultaneously perform multiple tasks such as data acquisition, communication transmission, and control interaction in practical applications, leading to fluctuations in the available processing resources. Under these circumstances, image recognition tasks struggle to consistently balance processing efficiency and recognition performance under different operating conditions, easily resulting in a trade-off between response speed and recognition accuracy, thus affecting the system's effectiveness in real-world industrial scenarios.

[0004] Furthermore, image data from industrial sites typically features complex backgrounds, numerous interfering factors, and subtle target features. It is also susceptible to changes in lighting, reflections, occlusion, vibration, and contamination, leading to significant fluctuations in image quality and further increasing the difficulty of recognition processing. In such complex application environments, existing image recognition solutions often struggle to simultaneously balance computational overhead and recognition accuracy, particularly in fine-grained target recognition or complex scene recognition tasks, where system stability and adaptability still require improvement.

[0005] Furthermore, in smart manufacturing scenarios, image recognition tasks often face strong demands for adaptability to changing scenarios due to product switching, changes in operating conditions, or the continuous emergence of new types of targets. Especially when the number of samples in new scenarios is limited, existing solutions often suffer from long adaptation cycles and low adjustment efficiency, making it difficult to meet the application requirements for rapid on-site deployment and continuous updates.

[0006] Furthermore, existing solutions typically lack effective overall coordination in terms of terminal resource adaptation, recognition performance maintenance, and response to scene changes, resulting in insufficient overall robustness, flexibility, and continuous adaptability of the system in complex and dynamic industrial application environments.

[0007] Therefore, there is an urgent need for a lightweight real-time image recognition technology solution suitable for deployment on smart terminals, so as to balance recognition real-time performance and recognition accuracy under limited resource conditions, and improve the system's adaptability to complex industrial scenarios and changes in new scenarios, thereby better meeting the actual needs of smart manufacturing field applications. Summary of the Invention

[0008] To address the shortcomings of existing technologies, the present invention aims to provide a lightweight real-time image recognition system and method for smart terminals, which enables real-time, efficient, accurate, stable, and adaptive image recognition under conditions of dynamic changes in terminal computing power, complex industrial scenarios, and small sample sizes in new scenarios.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a lightweight real-time image recognition system for smart terminals, comprising:

[0010] The module linkage control center, and the computing power dynamic perception module, lightweight attention module, hierarchical feature extraction module, few-shot self-learning module and recognition result output module respectively connected to the module linkage control center;

[0011] The computing power dynamic sensing module is used to collect hardware status parameters of the smart terminal and process them to obtain terminal computing power level data, which is then sent to the module linkage control center.

[0012] The module linkage control center is used to generate and issue scheduling instructions to the lightweight attention module and the hierarchical feature extraction module based on the terminal computing power level data and a preset linkage threshold table; and to drive the lightweight attention module to perform parameter iteration based on the new scene feature parameters sent by the few-sample self-learning module, and control the hierarchical feature extraction module to incrementally update the fine-grained semantic feature layer.

[0013] The lightweight attention module is used to dynamically adjust the attention calculation range according to the scheduling instruction, generate an attention weight matrix, and send it to the hierarchical feature extraction module.

[0014] The hierarchical feature extraction module is used to trigger feature extraction at the corresponding semantic level according to the scheduling instruction, and to fuse the attention weight matrix into the convolution calculation process of feature extraction, and output the feature map to the recognition result output module.

[0015] The recognition result output module is used to process the feature map to obtain the recognition result, and to feed back the confidence data of the recognition result to the module linkage control center.

[0016] The few-sample self-learning module is used to process the collected few-sample image data of the new scene through metric learning, obtain the feature parameters of the new scene, and send them to the module linkage control center.

[0017] Furthermore, the module linkage control center includes:

[0018] A threshold storage unit is used to store a preset linkage threshold table and a preset industrial scenario basic feature parameter library; wherein, the linkage threshold table contains the mapping relationship between computing power level and attention calculation range and feature extraction level, and the industrial scenario basic feature parameter library contains target feature parameters of multiple typical intelligent manufacturing scenarios.

[0019] The scheduling decision unit is connected to the threshold storage unit and is used to receive the terminal computing power level data sent by the computing power dynamic perception module, query the linkage threshold table according to the terminal computing power level data, determine the attention calculation range and feature extraction level that match the current computing power level, and generate a scheduling instruction based on the determination result, and issue the scheduling instruction to the lightweight attention module and the hierarchical feature extraction module.

[0020] The parameter evolution unit, connected to the threshold storage unit, is used to receive the new scene feature parameters sent by the few-sample self-learning module, store the new scene feature parameters in the industrial scene basic feature parameter library, update the mapping relationship related to the new scene feature parameters in the linkage threshold table according to the new scene feature parameters, drive the lightweight attention module to perform parameter iteration, and control the hierarchical feature extraction module to incrementally update the fine-grained semantic feature layer.

[0021] Furthermore, the module linkage control center also includes:

[0022] The computing power monitoring unit is used to receive the terminal computing power level data sent by the computing power dynamic sensing module, and to monitor the changing trend of the terminal computing power level data in real time.

[0023] An emergency adaptation unit, connected to the computing power monitoring unit and the scheduling decision unit, is used to trigger an emergency computing power adaptation strategy when the computing power monitoring unit detects that the change in the terminal computing power level data exceeds a preset mutation threshold. This strategy forces the current computing power level to be downgraded by one level, and instructs the scheduling decision unit to regenerate scheduling instructions based on the downgraded computing power level. These instructions are then sent to the lightweight attention module and the hierarchical feature extraction module to freeze non-core attention calculations and switch to a lower level of feature extraction.

[0024] The recovery determination unit is connected to the computing power monitoring unit and the emergency adaptation unit. After the emergency adaptation unit triggers the emergency adaptation strategy, it continuously monitors the terminal computing power level data. When the terminal computing power level data recovers to the normal range and stabilizes for more than a preset stable time threshold, it cancels the emergency adaptation strategy and restores the normal scheduling mode of the scheduling decision unit.

[0025] Furthermore, the computing power dynamic sensing module includes:

[0026] The parameter acquisition unit is used to acquire the hardware status parameters of the smart terminal in real time based on the system application programming interface of the smart terminal, and adjust the acquisition frequency according to the preset acquisition frequency threshold.

[0027] The parameter processing unit, connected to the parameter acquisition unit, is used to normalize the hardware status parameters and divide the processed hardware status parameters into multiple computing power levels according to a preset computing power level threshold, thereby generating the terminal computing power level data.

[0028] The preset computing power level threshold is based on the hardware configuration of the smart terminal and the preset industrial scene recognition requirements, and supports manual fine-tuning on the terminal side.

[0029] Furthermore, the lightweight attention module includes:

[0030] The channel attention unit is used to receive the scheduling instructions issued by the module linkage control center, dynamically determine the target high-frequency feature channel range according to the scheduling instructions, and allocate attention weights to the feature channels within the target high-frequency feature channel range to generate channel attention weights.

[0031] The spatial attention unit is used to receive the scheduling instructions issued by the module linkage control center, dynamically determine the range of typical spatial target areas according to the scheduling instructions, and perform attention mask calculation on the spatial positions within the range of typical spatial target areas to generate spatial attention masks.

[0032] The weight fusion unit is connected to the channel attention unit and the spatial attention unit respectively, and is used to fuse the channel attention weights and the spatial attention mask to generate the attention weight matrix and send it to the hierarchical feature extraction module;

[0033] The target high-frequency feature channel range and the typical spatial target region range are defined based on a preset industrial scenario prior knowledge base, and the calculation range of the channel attention unit and the spatial attention unit is dynamically adjusted according to the scheduling instructions.

[0034] Furthermore, the hierarchical feature extraction module includes:

[0035] The hierarchical control unit is used to receive the scheduling instructions issued by the module linkage control center, determine the semantic level of the current feature extraction according to the scheduling instructions, and generate a hierarchical trigger signal.

[0036] The feature extraction unit, connected to the hierarchical control unit, is used to receive the hierarchical trigger signal and the attention weight matrix sent by the lightweight attention module, trigger the feature extraction network of the corresponding semantic level according to the hierarchical trigger signal, and fuse the attention weight matrix into the convolution calculation process of the feature extraction network to output the feature map;

[0037] The data interaction unit is connected to the feature extraction unit and is used to extract intermediate data generated by the feature extraction unit during the feature extraction process, and send the intermediate data to the module linkage control center. The intermediate data is used by the few-sample self-learning module for parameter iteration.

[0038] Furthermore, the feature extraction unit includes:

[0039] The low-level feature extraction subunit is used to extract the low-level contour features of industrial images, corresponding to the preset low computing power level requirements.

[0040] The mid-layer feature extraction subunit is connected to the bottom-layer feature extraction subunit and is used to extract mid-layer texture features based on the bottom-layer contour features, corresponding to the preset mid-level computing power requirements.

[0041] A high-level feature extraction subunit, connected to the mid-level feature extraction subunit, is used to extract fine-grained semantic features based on the mid-level texture features, corresponding to a preset high computing power level requirement;

[0042] The hierarchical control unit triggers one or more of the bottom-level feature extraction subunit, the middle-level feature extraction subunit, and the high-level feature extraction subunit to perform feature extraction according to the scheduling instruction, and integrates the attention weight matrix into the convolution calculation process of the triggered subunit.

[0043] Furthermore, the recognition result output module includes:

[0044] The classification and detection unit is used to receive the feature map output by the hierarchical feature extraction module, process the feature map through a preset lightweight classification and detection head, and generate a recognition result; wherein, the recognition result includes target category, defect location, and confidence data;

[0045] The confidence determination unit, connected to the classification detection unit, is used to acquire the confidence data in the recognition result, compare the confidence data with a preset confidence threshold, and when the confidence data is lower than the confidence threshold, generate an iteration trigger signal and send it to the module linkage control center to trigger the small sample self-learning module to perform another iteration optimization.

[0046] The result output unit, connected to the classification and detection unit, is used to output the identification results to the local storage medium and the industrial local area network transmission interface, adapting to the industrial control and data traceability requirements of intelligent manufacturing production lines.

[0047] Furthermore, the few-shot self-learning module includes:

[0048] An image preprocessing unit is used to receive new scene small sample image data collected by the terminal, and preprocess the new scene small sample image data to generate preprocessed small sample image data; wherein, the number of new scene small sample image data is less than a preset small sample number threshold.

[0049] The feature mining unit, connected to the image preprocessing unit, is used to perform feature clustering on the preprocessed small sample image data based on metric learning, mine the target high-frequency feature channels and typical spatial target regions corresponding to the small sample image data of the new scene, generate the feature parameters of the new scene, and send the feature parameters of the new scene to the module linkage control center.

[0050] The parameter iteration unit, connected to the module linkage control center, is used to incrementally update the parameter table of the lightweight attention module and incrementally update the convolution kernel of the fine-grained semantic feature layer of the hierarchical feature extraction module according to the drive of the module linkage control center, while freezing the parameters of the bottom contour feature layer and the middle texture feature layer of the hierarchical feature extraction module.

[0051] When the parameter iteration unit performs incremental updates, the proportion of frozen model parameters to the total number of model parameters is not less than a preset parameter freezing ratio threshold, and the time consumed in a single iteration is less than a preset fine-tuning time threshold.

[0052] A lightweight real-time image recognition method for smart terminals, applied to the lightweight real-time image recognition system for smart terminals as described above, includes:

[0053] Step S1: The computing power dynamic sensing module collects the hardware status parameters of the smart terminal and processes them to obtain the terminal computing power level data, which is then sent to the module linkage control center.

[0054] In step S2, the module linkage control center generates and sends scheduling instructions to the lightweight attention module and the hierarchical feature extraction module based on the terminal computing power level data and a preset linkage threshold table; and drives the lightweight attention module to perform parameter iteration based on the new scene feature parameters sent by the few-sample self-learning module, and controls the hierarchical feature extraction module to incrementally update the fine-grained semantic feature layer.

[0055] Step S3: The lightweight attention module dynamically adjusts the attention calculation range according to the scheduling instruction, generates an attention weight matrix, and sends it to the hierarchical feature extraction module.

[0056] Step S4: The hierarchical feature extraction module triggers feature extraction at the corresponding semantic level according to the scheduling instruction, and integrates the attention weight matrix into the convolution calculation process of feature extraction, and outputs the feature map to the recognition result output module.

[0057] Step S5: The recognition result output module processes the feature map to obtain the recognition result, and feeds back the confidence data of the recognition result to the module linkage control center.

[0058] In step S6, the few-shot self-learning module processes the collected few-shot image data of the new scene through metric learning to obtain the feature parameters of the new scene and sends them to the module linkage control center.

[0059] The beneficial effects of this invention are:

[0060] 1. Enhance terminal-side resource adaptability: Through the collaborative control of the computing power dynamic sensing module and the module linkage control center, this invention can dynamically schedule the relevant identification and processing processes according to the changes in the computing power of the smart terminal, thereby taking into account both the real-time performance and the identification effect, and improving the system's adaptability to terminal resource fluctuations.

[0061] 2. Improve recognition performance in complex scenarios: By combining a lightweight attention module with a hierarchical feature extraction module, this invention can enhance the ability to extract key image information under limited terminal resources, thereby improving the accuracy and stability of image recognition in complex industrial scenarios.

[0062] 3. Enhanced rapid adaptability to new scenarios: This invention sets up a small-sample self-learning module, which can process small-sample image data of new scenarios and drive relevant modules to perform parameter iteration and incremental updates, thereby improving the system's adaptability to new materials, new defect types or new working conditions.

[0063] 4. Enhance overall system synergy: This invention achieves coordinated cooperation among multiple functional modules through a module linkage control center, changing the situation where each optimization link is relatively independent in the prior art, which is conducive to improving the overall system operating efficiency, robustness and scenario adaptability.

[0064] 5. More suitable for local deployment on smart terminals: This invention is designed to build a lightweight real-time image recognition system for smart terminals, which can better meet the application needs of industrial sites for localization, real-time performance and continuous adaptability, and has good practical value. Attached Figure Description

[0065] Figure 1 This is a schematic diagram of the lightweight real-time image recognition system for smart terminals in this invention.

[0066] Figure 2 This is a schematic diagram of the feature extraction unit in this invention;

[0067] Figure 3 This is a flowchart of the steps of the lightweight real-time image recognition method for smart terminals in this invention.

[0068] Figure labeling: 1. Module linkage control center; 11. Threshold storage unit; 12. Scheduling decision unit; 13. Parameter evolution unit; 14. Computing power monitoring unit; 15. Emergency adaptation unit; 16. Recovery judgment unit; 2. Computing power dynamic perception module; 21. Parameter acquisition unit; 22. Parameter processing unit; 3. Lightweight attention module; 31. Channel attention unit; 32. Spatial attention unit; 33. Weight fusion unit; 4. Hierarchical feature extraction module; 41. Hierarchical control unit; 42. Feature extraction unit; 421. Low-level feature extraction subunit; 422. Mid-level feature extraction subunit; 423. High-level feature extraction subunit; 43. Data interaction unit; 5. Recognition result output module; 51. Classification detection unit; 52. Confidence judgment unit; 53. Result output unit; 6. Small sample self-learning module; 61. Image preprocessing unit; 62. Feature mining unit; 63. Parameter iteration unit. Detailed Implementation

[0069] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Identical components are denoted by the same reference numerals. It should be noted that the terms "front," "rear," "left," "right," "upper," and "lower" used in the following description refer to directions in the accompanying drawings, and the terms "bottom surface," "top surface," "inner," and "outer" refer to directions toward or away from the geometric center of a specific component, respectively.

[0070] like Figure 1As shown, this embodiment deploys the lightweight real-time image recognition system for smart terminals of the present invention in the scenario of bolt defect detection in a smart manufacturing production line.

[0071] The inspection targets bolts in automotive parts production lines, detecting defects including thread damage, head cracks, and missing bolts, with further adaptation to include bolt shank bending defects. The system is deployed on an industrial tablet with a quad-core CPU, 4GB of RAM, 64GB of storage, and runs on Android. The industrial camera captures images at a resolution of 640×480, with a production line transmission speed of 10 pieces / second. System performance requirements include a single-frame recognition time of less than 20ms and a recognition accuracy of no less than 95%.

[0072] The image recognition system in this embodiment includes a module linkage control center 1, and a computing power dynamic perception module 2, a lightweight attention module 3, a hierarchical feature extraction module 4, a few-shot self-learning module 6, and a recognition result output module 5, all connected to the module linkage control center 1.

[0073] The computing power dynamic sensing module 2 is used to collect hardware status parameters of the smart terminal and process them to obtain terminal computing power level data, which is then sent to the module linkage control center 1. Hardware status parameters include CPU utilization, available memory, current task count, and device temperature. CPU utilization serves as the primary basis for classifying the terminal's computing power level, while the other parameters are used to assist in verifying the terminal's operating status. The collection frequency is set to 10ms / time.

[0074] The module linkage control center 1 generates and issues scheduling instructions to the lightweight attention module 3 and the hierarchical feature extraction module 4 based on the terminal computing power level data and a preset linkage threshold table. It also drives the lightweight attention module 3 to iterate parameters based on the new scene feature parameters sent by the few-sample self-learning module 6, and controls the hierarchical feature extraction module 4 to incrementally update the fine-grained semantic feature layer. The module linkage control center 1 also receives confidence data from the recognition result output module 5 and determines whether to trigger new scene sample accumulation and subsequent self-learning processes based on the confidence threshold.

[0075] The lightweight attention module 3 is used to dynamically adjust the attention calculation range according to the scheduling instructions, generate an attention weight matrix, and send it to the hierarchical feature extraction module 4. The attention calculation range includes two parts: spatial range and channel range. The spatial range switches between the entire target region, the main region, and the local region according to the computing power level, while the channel range switches between all key channels and core feature channels according to the computing power level.

[0076] The hierarchical feature extraction module 4 is used to trigger feature extraction at the corresponding semantic level according to the scheduling instructions, and integrates the attention weight matrix into the convolution calculation process of feature extraction, outputting the feature map to the recognition result output module 5. The hierarchical feature extraction module 4 is built based on the MobileNetV3 lightweight basic model, with a total parameter size of 4.2M. The feature layers include a bottom contour feature layer, a middle texture feature layer, and a fine-grained semantic feature layer.

[0077] The identification result output module 5 processes the feature map to obtain the identification result and feeds back the confidence level data of the identification result to the module linkage control center 1. The identification result includes the defect category and the corresponding confidence level, with the confidence level threshold set to 95%.

[0078] The few-shot self-learning module 6 is used to process the acquired few-shot image data of new scenes through metric learning, obtain the feature parameters of the new scenes, and send them to the module linkage control center 1. The threshold for the number of few samples is set to 5 images. When the cumulative number of valid labeled samples of the same new scene reaches 5, the few-shot self-learning process is triggered.

[0079] Working principle of Example 1:

[0080] 1. System deployment and initialization;

[0081] The industrial camera is mounted above the conveyor belt, with the distance from the lens center to the inspection surface set to 180mm, the exposure time set to 2ms, and the acquisition frequency set to 10 frames / second. A ring-shaped light source is used to ensure stable imaging quality for the bolt surface contour, thread texture, and crack areas. The industrial tablet connects to the industrial camera via USB 3.0 or Ethernet and communicates with the production line control system via the local network.

[0082] During system initialization, the basic feature parameter library for the industrial scene is loaded first. This library includes typical target regions and high-frequency feature channels for bolts. The typical target region is set to a 300×300 pixel area at the image center, used to cover the bolt body and major defect areas. The high-frequency feature channels include convolutional channels related to thread texture and edge contours, used to prioritize and enhance key defect information during attention calculation and feature extraction.

[0083] Subsequently, a threshold table for the linkage between computing power, attention, and feature layer was calibrated, as follows:

[0084] When the CPU utilization rate is less than 40%, the terminal's computing power level is determined to be high. The module linkage control center 1 issues fine-grained semantic feature extraction instructions, and simultaneously controls the lightweight attention module 3 to perform attention calculations on the entire target region. The entire target region is the 300×300 pixel area in the center of the image.

[0085] When the CPU utilization rate is between 40% and 70%, the terminal's computing power level is determined to be medium. The module linkage control center 1 issues a mid-level texture feature extraction command, and simultaneously controls the lightweight attention module 3 to perform attention calculation on the main bolt area. The main area is set as a 240×240 pixel area at the center of the image.

[0086] When the CPU utilization rate exceeds 70%, the terminal's computing power level is determined to be low. The module linkage control center 1 issues a low-level contour feature extraction command, while simultaneously controlling the lightweight attention module 3 to perform attention calculations only on the core feature channels. The core feature channels are the thread texture channel and the edge contour channel pre-set in the basic feature parameter library.

[0087] To improve system stability under sudden changes in computing power, an emergency adaptation rule is further implemented: when the CPU utilization rate is greater than 70% for three consecutive sampling periods, a sudden change in computing power is determined, and the system immediately enters a low computing power emergency mode; when the CPU utilization rate recovers to less than 40% for five consecutive sampling periods, the system exits the emergency mode and resumes high computing power operation. Since the sampling frequency is 10ms / time, the emergency trigger time is 30ms, and the recovery confirmation time is 50ms.

[0088] 2. Real-time recognition stage;

[0089] During normal production line operation, the industrial camera captures bolt images at 10 frames per second and transmits them to the industrial tablet. After preprocessing, the images enter the recognition process. The computing power dynamic sensing module 2 reads the operating status of the industrial tablet in real time. Under normal operating conditions, the CPU utilization rate is stable at 30% to 35%, the terminal computing power level is determined to be high computing power level, and the data is sent to the module linkage control center 1.

[0090] The module linkage control center 1 generates scheduling instructions based on the linkage threshold table: on the one hand, it controls the lightweight attention module 3 to perform attention calculations within a 300×300 pixel area in the center of the image, focusing on enhancing the thread texture channel and edge contour channel; on the other hand, it controls the hierarchical feature extraction module 4 to enable the fine-grained semantic feature layer to participate in feature extraction. The attention weight matrix generated by the lightweight attention module 3 is fused into the convolution calculation process, making the convolution response more concentrated on key areas such as thread damage, head cracks, and missing boundaries. After the hierarchical feature extraction module 4 outputs the feature map, the recognition result output module 5 completes the defect category discrimination and confidence calculation, and feeds back the confidence data to the module linkage control center 1.

[0091] During this stage, the system's single-frame recognition time is 12ms to 15ms, with a recognition accuracy of 96.8%, meeting the production line's requirements for real-time performance and accuracy. This demonstrates that when the terminal has sufficient computing power, the system can automatically utilize higher-level feature extraction capabilities and a more complete attention calculation range, improving the accuracy of complex defect recognition while ensuring real-time performance.

[0092] 3. Small sample iteration stage;

[0093] After the production line had been running for a period of time, a new type of bolt with a special specification was added, and a new defect type of bolt shank bending appeared. Since the basic feature parameter library did not yet contain the specific features for this new defect type, the system's initial recognition results for this defect type had a confidence level of less than 95%. The module-linked control center 1, based on the confidence level data fed back by the recognition result output module 5, marked the corresponding images as samples to be reviewed. After manual confirmation and labeling, a total of 5 small sample images of bolt shank bending defects were obtained, reaching the small sample quantity threshold. The system then triggered the small sample self-learning module 6 to start the iterative process.

[0094] The few-shot self-learning module 6 processes five labeled images based on metric learning, extracts sample embedding features, and obtains new scene feature parameters through clustering of similar samples and inter-class difference analysis. The new scene feature parameters include: newly added high-frequency feature channels corresponding to the deformation of the pole contour, a newly added spatial attention mask for the pole region, and a set of parameters for incremental updates of the fine-grained semantic feature layer.

[0095] After receiving new scene feature parameters, the module linkage control center 1 drives the lightweight attention module 3 to iterate the parameters, increasing the spatial attention range of the rod region based on the original attention parameters and including the rod contour deformation channel in the priority calculation channel; at the same time, it controls the hierarchical feature extraction module 4 to incrementally update the fine-grained semantic feature layer to improve the system's ability to distinguish bolt rod bending defects. After the update is completed, the linkage threshold table is adapted synchronously so that the attention region in high computing power mode and medium computing power mode can cover the newly added rod defect region.

[0096] The total iteration time for this phase was 28 seconds. After the update, the system achieved a recognition accuracy of 95.5% in the new defect scenario, with a single frame recognition time of 13ms to 16ms. This demonstrates that the system can complete new scenario adaptation on the terminal side using a small number of samples, and simultaneously update attention calculation and fine-grained feature extraction capabilities, shortening the new scenario deployment cycle and improving continuous adaptability in the field.

[0097] 4. Emergency computing power adaptation phase;

[0098] In actual operation, industrial tablets may perform image recognition tasks as well as production line data transmission and equipment linkage tasks. When concurrent tasks increase, CPU utilization may rise rapidly in a short period of time. During testing, when CPU utilization instantly increased to 75% and remained above 70% for three consecutive sampling periods, the computing power dynamic sensing module 2 determined that a computing power mutation had occurred and sent a mutation signal to the module linkage control center 1.

[0099] Upon receiving the sudden change signal, the module linkage control center 1 immediately switches the terminal's computing power level to a low level and issues emergency dispatch instructions to the lightweight attention module 3 and the hierarchical feature extraction module 4. The lightweight attention module 3 stops performing large-scale spatial attention calculations, retaining only attention calculations for core feature channels; the hierarchical feature extraction module 4 shuts down the high-load fine-grained semantic feature layer and mid-level texture feature layer, retaining only the bottom contour feature layer for rapid feature extraction. This prioritizes the retention of the most critical edge, gap, and contour information for defect identification when computing power is insufficient. The identification result output module 5 continues to output the defect category and confidence level and feeds back the identification results to the module linkage control center 1.

[0100] In this emergency mode, the system's single-frame recognition time was reduced to 10ms-12ms, while the recognition accuracy remained at 92.3%, and no stuttering or crashes occurred during operation. After the concurrent tasks ended, the CPU utilization rate recovered to 32% and remained below 40% for five consecutive sampling cycles. The system then automatically exited the emergency mode and returned to high-computing-power real-time recognition. This process demonstrates that the present invention can quickly reduce load and maintain basic recognition capabilities when terminal computing power changes abruptly, thereby improving the system's stability and robustness in complex industrial environments.

[0101] 5. Test conditions and results;

[0102] To verify system performance, continuous testing was conducted under the following conditions: ambient temperature 25±2℃, relative humidity 45%~60%, production line speed 10 pieces / second, industrial camera resolution 640×480, exposure time 2ms, continuous operation of the industrial flat panel for 8 hours, and continuous online operation of the system. Test objects included normal bolts, thread damage defects, head crack defects, missing bolts, and newly added bolt shank bending defects.

[0103] Test results show that under normal real-time recognition conditions, the CPU utilization rate is 30%–35%, the system's single-frame recognition time is 12ms–15ms, and the recognition accuracy is 96.8%. Under small-sample self-learning conditions, using 5 labeled samples to complete the update takes 28s, and the recognition accuracy of the new scene after the update is 95.5%, with a single-frame recognition time of 13ms–16ms. Under sudden changes in computing power, the system automatically switches to a low-computing-power emergency mode after the CPU utilization rate rises to 75%, with a single-frame recognition time of 10ms–12ms and a recognition accuracy of 92.3%, and there is no lag or crash throughout the process.

[0104] Technical effects of Example 1:

[0105] In summary, this embodiment demonstrates that the present invention, through the coordinated operation of the modular linkage control center 1, the computing power dynamic perception module 2, the lightweight attention module 3, the hierarchical feature extraction module 4, the few-sample self-learning module 6, and the recognition result output module 5, can achieve adaptive scheduling of the image recognition process under dynamic changes in the computing power of the intelligent terminal, and complete rapid iterative updates on the terminal side when new scenarios appear. This balances real-time performance, recognition accuracy, scene adaptability, and system stability, making it suitable for deployment and application in intelligent terminals of intelligent manufacturing production lines.

[0106] Example 2 is the second embodiment of the present invention. Based on Example 1, this embodiment further describes the specific implementation of the module linkage control center 1.

[0107] The module linkage control center 1 includes a threshold storage unit 11, a scheduling decision unit 12, a parameter evolution unit 13, a computing power monitoring unit 14, an emergency adaptation unit 15, and a recovery judgment unit 16. As the core scheduling module of the system, the module linkage control center 1 receives in real time terminal computing power level data sent by the computing power dynamic perception module 2, new scene feature parameters sent by the few-shot self-learning module 6, and intermediate feature extraction data output by the hierarchical feature extraction module 4. Based on the preset linkage threshold table and the industrial scene basic feature parameter library, it issues scheduling instructions to the lightweight attention module 3 and the hierarchical feature extraction module 4. Simultaneously, it drives the few-shot self-learning module 6 to complete parameter iteration for the lightweight attention module 3 and the hierarchical feature extraction module 4, and updates the linkage threshold table and the industrial scene basic feature parameter library, realizing the dynamic evolution of the entire system. When a sudden change in terminal computing power is detected, the module linkage control center 1 immediately triggers an emergency computing power adaptation strategy, quickly adjusting the feature extraction level and attention calculation range to ensure that the recognition system does not crash and that real-time performance is not interrupted.

[0108] 1. Application scenarios and deployment methods;

[0109] In this embodiment, the modular linkage control center 1 is deployed in an embedded acquisition terminal within a multi-workstation hybrid intelligent manufacturing scenario. This terminal simultaneously serves three typical intelligent manufacturing scenarios: part defect detection, material classification, and equipment inspection. The embedded acquisition terminal employs an octa-core ARM processor, 8GB of RAM, and 64GB of eMMC storage. It runs on Ubuntu 20.04 and features a built-in gigabit Ethernet port and USB 3.0 interface for receiving image acquisition data from multiple workstations and communicating with the production line control system.

[0110] The module linkage control center 1 operates in a resident scheduling service mode. The threshold storage unit 11 is implemented by combining a local SQLite database with a memory cache, with the database space allocated as 128MB and the memory cache space allocated as 32MB. The scheduling decision unit 12, parameter evolution unit 13, computing power monitoring unit 14, emergency adaptation unit 15, and recovery judgment unit 16 all run on independent threads, with thread cycles set to 20ms, 50ms, 10ms, 10ms, and 20ms, respectively.

[0111] In this embodiment, the industrial scenario basic feature parameter library pre-stores target feature parameters for three typical smart manufacturing scenarios. The target feature parameters for the part defect detection scenario include:

[0112] The target area size ranges from 300×300 pixels to 352×352 pixels at the image center, the edge density ranges from 0.20 to 0.50, the high-frequency texture energy ranges from 0.30 to 0.70, and the defect area ratio ranges from 0.01 to 0.08.

[0113] The target feature parameters for the material classification scenario include: the main area size ranges from 288×288 pixels to 416×416 pixels, the outline compactness ranges from 0.55 to 0.92, the area fill rate ranges from 0.45 to 0.88, and the grayscale centroid offset ranges from 0.05 to 0.18.

[0114] The target characteristic parameters for equipment inspection scenarios include: the size of the instrument panel or display window area ranges from 224×224 pixels to 320×320 pixels, the roundness of the circular dial ranges from 0.72 to 0.97, the pointer length ratio ranges from 0.28 to 0.42, and the character window area ratio ranges from 0.03 to 0.15.

[0115] All of the above parameters can be obtained through statistical analysis of the collected sample images, and can serve as the basis for subsequent scheduling decisions and parameter evolution.

[0116] 2. Internal implementation of module linkage control center 1;

[0117] The threshold storage unit 11 is used to store a preset linkage threshold table and a basic feature parameter library for industrial scenarios. The linkage threshold table contains the mapping relationship between computing power level and attention calculation range, feature extraction level, and establishes mapping sub-tables according to scenarios. To ensure feasibility, the specific threshold for computing power level division in this embodiment is set as follows:

[0118] When CPU utilization is less than 35%, memory pressure is less than 0.55, and chip temperature is less than 65℃, it is judged as a high computing power level;

[0119] When the CPU utilization rate is between 35% and 65%, or the memory pressure is between 0.55 and 0.75, it is judged as medium computing power level;

[0120] When the CPU utilization rate is greater than 65%, or the memory pressure is greater than 0.75, or the chip temperature is greater than 75°C, it is judged as a low computing power level.

[0121] For the component defect detection scenario, high computing power corresponds to attention calculation of the entire target area and fine-grained semantic feature extraction, medium computing power corresponds to attention calculation of the main body area and mid-level texture feature extraction, and low computing power corresponds to attention calculation of the core feature channel and low-level contour feature extraction.

[0122] For material classification scenarios, high computing power corresponds to attention calculation of the entire pallet area and extraction of mixed features in the middle and upper layers; medium computing power corresponds to attention calculation of the main material area and extraction of mid-layer texture features; and low computing power corresponds to attention calculation of the contour channel and extraction of the bottom contour features.

[0123] For equipment inspection scenarios, high computing power corresponds to joint attention calculation and fine-grained semantic feature extraction of the dashboard and character window, medium computing power corresponds to local attention calculation and mid-level texture feature extraction of the pointer area or character area, and low computing power corresponds to attention calculation of the core edge channel and low-level contour feature extraction.

[0124] The scheduling decision unit 12 is connected to the threshold storage unit 11. It is used to receive the terminal computing power level data sent by the computing power dynamic perception module 2, and combine it with the intermediate feature extraction data transmitted by the hierarchical feature extraction module 4. It queries the linkage threshold table to determine the attention calculation range and feature extraction level that match the current computing power level, and then generates a scheduling instruction to send to the lightweight attention module 3 and the hierarchical feature extraction module 4.

[0125] The scheduling instruction must include at least the current scene identifier, target area coordinates, enabled channel set, target feature level, and instruction version number. To avoid scheduling jitter caused by frequent switching, the scheduling decision unit 12 sets a level switching hysteresis threshold:

[0126] When switching from a high computing power level to a medium computing power level, the CPU utilization rate must be no less than 35% for two consecutive monitoring periods; when switching from a medium computing power level to a low computing power level, the CPU utilization rate must be no less than 65% for three consecutive monitoring periods; when recovering from a low computing power level to a medium computing power level, the CPU utilization rate must be less than 60% for five consecutive monitoring periods; and when recovering from a medium computing power level to a high computing power level, the CPU utilization rate must be less than 32% for eight consecutive monitoring periods.

[0127] The parameter evolution unit 13 is connected to the threshold storage unit 11. It is used to receive new scene feature parameters sent by the few-sample self-learning module 6, write the new scene feature parameters into the industrial scene basic feature parameter library, update the mapping relationship related to the new scene feature parameters in the linkage threshold table, drive the lightweight attention module 3 to perform parameter iteration, and control the hierarchical feature extraction module 4 to perform incremental updates on the fine-grained semantic feature layer.

[0128] To ensure version consistency, parameter evolution unit 13 employs a dual-version switching mechanism. A new version number is generated with each update, and the old version is replaced only after the new version has completed verification. The verification conditions are specifically set as follows: the average confidence score of the new version on the validation set is not lower than 0.93, the average inference latency increment per frame is not higher than 2ms, and the false positive rate increment is not higher than 2%. Only when all of these conditions are met simultaneously is the new version written to threshold storage unit 11 and participates in subsequent scheduling.

[0129] The computing power monitoring unit 14 receives terminal computing power level data and monitors the changing trend of the terminal computing power level data in real time. The changing trend is calculated using a sliding window statistical method, with a window length set to 10 sampling points, corresponding to a time length of 100ms. In addition to recording the current level, the computing power monitoring unit 14 also records the change range of CPU utilization, memory pressure, and temperature change slope, which are used to provide the emergency adaptation unit 15 with a basis for judging computing power sudden changes.

[0130] The emergency adaptation unit 15 connects the computing power monitoring unit 14 and the scheduling decision unit 12. When the computing power monitoring unit 14 detects that the change in the terminal computing power level data exceeds the preset mutation threshold, it triggers the computing power emergency adaptation strategy, forcibly lowers the current computing power level by one level, and instructs the scheduling decision unit 12 to regenerate the scheduling instruction according to the lowered computing power level, so as to freeze non-core attention computing and switch to a lower level of feature extraction.

[0131] In this embodiment, the mutation threshold is set as follows: within a 50ms time window, if the instantaneous increase in CPU utilization reaches or exceeds 50 percentage points, or the increase in memory pressure reaches or exceeds 0.20, or the rate of temperature increase reaches or exceeds 8°C per second, it is determined to be a computing power mutation. When the current level is high computing power level, the emergency adaptation unit 15 forcibly downgrades the level to medium computing power level; when the current level is medium computing power level, it forcibly downgrades it to low computing power level; when the current level is already low computing power level, it maintains the low computing power level, while further freezing non-core attention calculations, retaining only the attention calculations of core feature channels, and switching to low-level contour feature extraction.

[0132] The recovery determination unit 16 connects the computing power monitoring unit 14 and the emergency adaptation unit 15. It continuously monitors the terminal's computing power level data after the emergency adaptation strategy is triggered. When the terminal's computing power level data recovers to the normal range and stabilizes above a preset stabilization time threshold, the emergency adaptation strategy is canceled, and the system reverts to the normal scheduling mode of the scheduling decision unit 12. In this embodiment, the stabilization time threshold is set to 200ms, meaning that the CPU utilization, memory pressure, and temperature indicators must continuously meet the normal range of the current target level for 200ms before exiting the emergency adaptation state is allowed. To prevent frequent fluctuations, a 100ms protection window is set after exiting emergency adaptation, during which upgrade scheduling is not allowed to be triggered again.

[0133] 3. Scheduling decision rules and formulas;

[0134] To ensure that the scheduling decision unit 12 can stably generate scheduling instructions under concurrent conditions of multiple scenarios, multiple loads, and new scenarios, this embodiment introduces a linked scheduling evolution index, which comprehensively characterizes terminal resource pressure, identification uncertainty, scenario drift degree, and computing power mutation intensity. The linked scheduling evolution index is calculated using the following formula:

[0135] ;

[0136] in, For coordinated scheduling evolution index, a) is the computing power utilization ratio normalized from 0 to 1, b) is the current memory pressure ratio (valued as 1 minus the ratio of available memory to total memory), c) is the task congestion ratio formed by the ratio of concurrent tasks to the preset maximum number of tasks, d) is the chip temperature pressure ratio (valued as the current temperature minus the ambient baseline temperature, then divided by the difference between the maximum allowable temperature and the ambient baseline temperature, truncated to the range of 0 to 0.95), e) is the average recognition confidence decay ratio within the sliding window (valued as 1 minus the average recognition confidence), f) is the entropy offset ratio of intermediate data for feature extraction (valued as the absolute value of the difference between the current intermediate feature map information entropy and the corresponding scene template entropy, normalized from 0 to 1), g) is the scene drift ratio (valued as the cosine distance between the new scene feature parameter vector and the nearest neighbor template vector in the industrial scene basic feature parameter library, normalized from 0 to 1), and h) is the computing power mutation ratio (valued as the cutoff value of the difference between the current CPU utilization and the CPU utilization 50ms ago, within the range of 0 to 1). It is the arctangent function. It is an inverse hyperbolic sine function. It is the natural logarithm function. It is a natural exponential function.

[0137] All quantities in the above formula are dimensionless, so the linkage scheduling evolution index is also dimensionless, and the dimensions match. Since the values ​​of a, b, c, d, e, f, g, and h are all limited to the interval between 0 and 1, the actual value range of the linkage scheduling evolution index in this embodiment is [0, 2.24].

[0138] when When 0.82 < 0.82, it indicates that terminal resources are sufficient, the current scenario is stable, and the intermediate feature offset is small, and the scheduling decision unit executes high-computing-power-level mapping; when 0.82 ≤ When 1.32 < 1.32, it indicates that the terminal load and scene fluctuations are under control, and the scheduling decision unit is executing the computing power level mapping; when 1.32 ≤ When the value is less than 1.82, it indicates resource scarcity or increased scene drift, and the scheduling decision unit executes a low-computing-power-level mapping; when... When the value is ≥1.82, it indicates that the terminal has entered a high-risk operating zone. If the computing power mutation ratio h≥0.50 is also met, the emergency adaptation unit will immediately start the computing power emergency adaptation strategy.

[0139] In this calculation formula, the first term is used to characterize the resource load intensity after CPU usage, memory pressure, task congestion, and thermal pressure coupling. The second term is used to characterize the recognition instability after confidence decay, feature entropy shift, and scene drift coupling. The exponential term in the denominator is used to enhance the sensitivity of the result to sudden computing power events. Therefore, this calculation formula can reflect both regular scheduling needs and respond faster to sudden load increases, thereby improving the foresight and stability of the scheduling decision unit.

[0140] 4. Working principle of Example 2;

[0141] Under normal operating conditions, the computing power dynamic perception module 2 continuously sends terminal computing power level data to the computing power monitoring unit 14, while the hierarchical feature extraction module 4 simultaneously provides intermediate feature extraction data to the scheduling decision unit 12. The threshold storage unit 11 determines the current scenario type based on the current workstation task identifier and the industrial scenario basic feature parameter library. The scheduling decision unit 12, combining the linkage scheduling evolution index and the linkage threshold table, selects the attention calculation range and feature extraction level that match the current scenario and current computing power level, and generates scheduling instructions to be sent to the lightweight attention module 3 and the hierarchical feature extraction module 4. Because the target feature parameters of the three scenarios—part defect detection, material classification, and equipment inspection—differ significantly, the threshold storage unit 11 stores independent scenario mapping sub-tables locally. Therefore, the scheduling decision unit 12 can quickly adapt between different workstations without reloading the entire recognition process, simply by switching the target area, feature channel, and feature level. This reduces module switching overhead and improves the real-time performance of multi-scenario shared terminals.

[0142] When the few-sample self-learning module 6 outputs new scene feature parameters, the parameter evolution unit 13 first performs pre-entry verification on the new scene feature parameters. The verification includes target region size, edge density, texture energy, entropy shift, and average confidence improvement. After verification, the parameter evolution unit 13 writes the new scene feature parameters into the industrial scene basic feature parameter library and updates the mapping relationship related to the scene in the linkage threshold table. Subsequently, the parameter evolution unit 13 drives the lightweight attention module 3 to perform parameter iteration, so that the newly added target region and the newly added key channel can be included in the attention calculation set; at the same time, it controls the hierarchical feature extraction module 4 to incrementally update the fine-grained semantic feature layer, so that the new scene features are absorbed into the feature expression system without destroying the original typical scene feature distribution. Since the parameter evolution unit 13 adopts a dual-version switching mechanism, the old version parameters are always retained until the new version verification is completed, thus avoiding recognition interruption caused by parameter instability during the update process. Through this process, the module linkage control center 1 realizes a closed-loop dynamic evolution from "threshold matching" to "parameter evolution" and then to "threshold reconfiguration".

[0143] When the terminal experiences sudden concurrent tasks, batch data transmission, or centralized issuance of control commands, the computing power monitoring unit 14 will detect an abnormally rapid increase in CPU utilization, memory pressure, or temperature. If the instantaneous increase in CPU utilization reaches or exceeds 50 percentage points within a 50ms time window, or if the linkage scheduling evolution index enters a high-risk range accompanied by a computing power mutation ratio exceeding 0.50, the emergency adaptation unit 15 immediately triggers the computing power emergency adaptation strategy, downgrading the current computing power level by one level and instructing the scheduling decision unit 12 to regenerate the scheduling instructions. The regenerated scheduling instructions will freeze non-core attention calculations, narrow the scope of attention calculations, and switch the hierarchical feature extraction module 4 to a lower-level feature extraction, thereby quickly reducing computing power consumption. If the current computing power level is already low, non-core attention calculations will be further frozen, retaining only core feature channels and low-level contour feature extraction to ensure uninterrupted system operation. This allows for priority retention of the basic features that contribute most to the recognition results during computing power mutations, preventing system lag or crashes.

[0144] After the sudden load disappears, the recovery determination unit 16 continuously monitors CPU utilization, memory pressure, and temperature changes. If the normal range of the target level is met for 200ms consecutively, the emergency adaptation strategy is canceled, and scheduling control is returned to the scheduling decision unit 12, which restores the normal scheduling mode according to the linkage threshold table. Because the recovery determination unit 16 sets a stabilization time threshold and a 100ms protection window after exiting, it can effectively avoid repeated jittering near the load boundary and ensure smooth switching of scheduling states. Thus, the module linkage control center 1 forms a complete closed loop between normal scheduling, dynamic evolution, emergency load reduction, and stable recovery, enabling the system to balance recognition accuracy, real-time performance, and stability in a multi-scenario mixed operation environment.

[0145] 5. Specific test conditions and results;

[0146] This embodiment was validated under a three-station mixed task condition. The three stations were a part defect detection station, a material sorting station, and an equipment inspection station. The total input frequency of the image stream was 12 frames per second, with the frame ratios for the three stations being 5:4:3. The test environment temperature was 24±2℃, the relative humidity was 40%~60%, and the continuous operating time of the terminal was 10 hours. The scheduling cycle of the scheduling decision unit 12 was 20ms, the monitoring cycle of the computing power monitoring unit 14 was 10ms, and the version switching cycle of the parameter evolution unit 13 was 50ms.

[0147] Under normal mixed operation, the CPU utilization rate is 29%–47%, the memory pressure is 0.41–0.58, and the linkage scheduling evolution index is stable between 0.66 and 1.18. The scheduling decision unit 12 automatically selects high or medium computing power level mapping according to different scenarios. The average single-frame processing latency of the three workstations is 14.2ms, 15.1ms, and 16.4ms, respectively, and the average recognition accuracy is 96.4%, 95.9%, and 95.6%, respectively. When a new type of pressure gauge image is introduced to the equipment inspection workstation and 5 sets of new scene feature parameters are output by the small sample self-learning module 6, the parameter evolution unit 13 completes parameter writing, version verification, and mapping table update. The total update time is 31s. After the update, the average recognition accuracy of the new type of pressure gauge scene increases from 91.8% to 95.2%, and the average single-frame processing latency increases by 1.3ms, meeting the data entry verification conditions. This result shows that the parameter evolution unit 13 can achieve rapid inclusion of new scenes and threshold reconfiguration without significantly increasing latency, improving the system's scene expansion capability.

[0148] In a burst load test, by simultaneously initiating batch data backhaul and device control command broadcasting, the CPU utilization rate increased from 31% to 83% within 50ms, a sudden increase of 52 percentage points. The computing power monitoring unit 14 completed the anomaly detection within the first 50ms window, and the emergency adaptation unit 15 immediately triggered the computing power emergency adaptation strategy. The scheduling decision unit 12 downgraded the scheduling mode of the three workstations to medium or low computing power levels, froze non-core attention calculations, and switched to a lower-level feature extraction. After emergency adaptation, the average single-frame processing latency of the three workstations was controlled at 12.8ms, 13.6ms, and 14.9ms, respectively, and the system did not experience any stuttering, queue backlog, or abnormal process exits throughout the process. After the batch backhaul ended, the CPU utilization rate dropped to 44% at 160ms and remained within the normal range for the subsequent 200ms. The recovery decision unit 16 then lifted the emergency adaptation, and the system re-entered the normal scheduling mode. The results show that the module linkage control center 1 has strong computing power surge response and state recovery capabilities, and can remain continuously available on shared smart terminals in multiple scenarios.

[0149] Technical effects of Example 2:

[0150] In summary, this embodiment demonstrates that the module linkage control center 1, through the collaborative work of the threshold storage unit 11, scheduling decision unit 12, parameter evolution unit 13, computing power monitoring unit 14, emergency adaptation unit 15, and recovery judgment unit 16, can not only accurately map the computing power level with the attention calculation range and feature extraction level based on the linkage threshold table, but also dynamically update the industrial scenario basic feature parameter library and linkage threshold table based on new scenario feature parameters. Furthermore, it can quickly downgrade the scheduling level, freeze non-core attention calculations, and switch to a lower level of feature extraction when computing power changes abruptly, thereby improving the system's real-time performance, stability, and continuous evolution capability in multi-scenario intelligent manufacturing environments.

[0151] Example 3 is the third embodiment of the present invention. Based on Examples 1 and 2, this embodiment further elaborates on the internal structure, linkage mode and corresponding method flow of the computing power dynamic perception module 2, the industrial scene-specific lightweight attention module 3, the hierarchical feature extraction module 4, the recognition result output module 5 and the small sample self-learning module 6.

[0152] In this embodiment, the confidence threshold for the recognition result is set to 90%, the small sample size threshold is set to 8 images, the parameter freeze ratio threshold is set to 90%, and the fine-tuning time threshold is set to 30 seconds.

[0153] 1. Dynamic computing power sensing module 2;

[0154] The computing power dynamic sensing module 2 includes a parameter acquisition unit 21 and a parameter processing unit 22. The parameter acquisition unit 21 is implemented based on the system application programming interface of the smart terminal. In this embodiment, it reads / proc / stat, / proc / meminfo, / proc / loadavg, / sys / class / thermal, and the GPU driver status node through the Linux system interface to collect CPU utilization, GPU utilization, remaining memory, inference thread count, chip temperature, and I / O wait ratio in real time. If the terminal does not enable the independent GPU, the GPU utilization is treated as 0. The parameter acquisition unit 21 dynamically adjusts the acquisition frequency according to a preset acquisition frequency threshold: when the production line cycle time is not less than 6 pieces / second, the acquisition frequency is set to 10ms / time; when the production line cycle time is between 1 piece / second and 6 pieces / second, the acquisition frequency is set to 50ms / time; when the terminal is in a static review scenario, the acquisition frequency is set to 500ms / time.

[0155] The parameter processing unit 22 is connected to the parameter acquisition unit 21 and is used to normalize the hardware status parameters and divide the processed hardware status parameters into three computing power levels (high, medium, and low) according to the preset computing power level threshold, thereby generating terminal computing power level data.

[0156] For the 2-core CPU + 2GB memory terminal used in this embodiment, the preset thresholds are specifically defined as follows: when the CPU utilization is less than 45%, the GPU utilization is less than 35%, the remaining memory is not less than 900MB, the number of inference threads is not greater than 2, and the chip temperature is less than 65℃, it is judged as a high computing power level; when the CPU utilization is between 45% and 75%, or the GPU utilization is between 35% and 60%, or the remaining memory is between 450MB and 900MB, or the number of inference threads is between 3 and 4, or the chip temperature is between 65℃ and 75℃, it is judged as a medium computing power level; when the CPU utilization is greater than 75%, or the GPU utilization is greater than 60%, or the remaining memory is less than 450MB, or the number of inference threads is greater than 4, or the chip temperature is higher than 75℃, it is judged as a low computing power level. To adapt to different terminals, the parameter processing unit 22 supports manual fine-tuning on the terminal side, with the manual fine-tuning range set at ±5% for each threshold.

[0157] To avoid misjudgment based on a single parameter, parameter processing unit 22 constructs a computing power pressure index in this embodiment, as follows:

[0158] ;

[0159] in, As a computing power pressure indicator, This is the normalized value of CPU utilization. This is the normalized value of GPU utilization. The ratio of the number of inference threads to the preset maximum number of threads. It is the memory deficit ratio, which is equal to 1 minus the ratio of the current remaining memory to the total memory. The chip temperature-pressure ratio is equal to the current temperature minus the ambient reference temperature, then divided by the difference between the maximum allowable temperature and the ambient reference temperature. The scene cycle time pressure ratio is equal to the ratio of the current workstation cycle time to the rated workstation cycle time. This is the normalized value of the I / O wait ratio. This is a normalized result that represents a short-term load jump variable and is equal to the difference in CPU utilization between two adjacent sampling windows.

[0160] In this formula, all input quantities are dimensionless normalized quantities, therefore P is a dimensionless quantity with dimension matching. Based on the terminal configuration in this embodiment, the actual value range of P stably falls within the interval (0, 3.10). When P is less than 0.85, it is determined to be a high computing power level; when P is greater than or equal to 0.85 and less than 1.55, it is determined to be a medium computing power level; when P is greater than or equal to 1.55, it is determined to be a low computing power level. This formula adopts a structure of "principal fraction plus hyperbolic tangent compensation term". The first half reflects the coupled load of processor, graphics computing, threads, and memory, while the second half reflects the additional impact of I / O wait and short-term transitions on inference continuity. Therefore, it can improve the stability and foresight of computing power level classification.

[0161] The working principle of the computing power dynamic sensing module 2 is as follows:

[0162] The parameter acquisition unit 21 continuously collects terminal hardware status parameters. The parameter processing unit 22 first performs normalization, and then compares the computing power pressure index P with the preset computing power level threshold to generate terminal computing power level data and send it to the module linkage control center 1 in real time. This allows the module linkage control center 1 to comprehensively judge the available computing power of the current terminal based on multi-dimensional hardware status, thereby improving scheduling accuracy and reducing false switching.

[0163] 2. Lightweight attention module specifically designed for industrial applications;

[0164] The lightweight attention module 3, specifically designed for industrial applications, includes a channel attention unit 31, a spatial attention unit 32, and a weighted fusion unit 33. In this embodiment, this module is referred to as IS-LAM. It is specifically designed for lightweight attention calculations on pin edges, snap notches, housing contours, and cavity textures in electronic connector images, and does not employ the full-channel, full-space calculation method of general attention mechanisms.

[0165] The prior knowledge base for industrial scenarios is stored locally, containing coordinate templates for typical connector pin array areas, shell symmetry boundaries, snap-fit ​​areas, and cavity opening areas. Based on a 320×320 input image, the typical spatial target area is set to the central region coordinates (32,32) to (288,288). In high-performance computing mode, this is expanded to a 256×256 spatial range; in medium-performance computing mode, it is compressed to a 224×224 spatial range; and in low-performance computing mode, only a 160×160 core area is retained. The target high-frequency feature channel range is predefined according to the backbone network hierarchy: the bottom layer contour-related channel set consists of channels 4 to 11, the middle layer texture-related channel set consists of channels 17 to 28, and the high layer semantic-related channel set consists of channels 63 to 78. High-performance computing mode enables three sets of channels, medium-performance computing mode enables the first two sets of channels, and low-performance computing mode enables only the bottom layer contour-related channels.

[0166] The channel attention unit 31 receives scheduling instructions from the module linkage control center 1, dynamically determines the target high-frequency feature channel range according to the scheduling instructions, and assigns attention weights to the feature channels within the target high-frequency feature channel range to generate channel attention weights. Specifically, the channel attention unit 31 first performs generalized mean pooling and max pooling on the selected channels, then concatenates the two pooling results and inputs them into a two-layer fully connected bottleneck network with a compression ratio of 4, outputting the channel attention weights. The spatial attention unit 32 receives scheduling instructions from the module linkage control center 1, dynamically determines the typical spatial target region range according to the scheduling instructions, and calculates the spatial position attention mask within this region. Specifically, the spatial attention unit 32 first truncates the target region, then performs 3×3 depthwise separable convolution, edge enhancement, and threshold mask generation to form a spatial attention mask. The weight fusion unit 33 connects the channel attention unit 31 and the spatial attention unit 32 respectively, and is used to fuse the channel attention weights and the spatial attention mask to generate an attention weight matrix, which is then sent to the hierarchical feature extraction module 4.

[0167] To ensure a unified expression for both channel and spatial attention at different computing power levels, IS-LAM constructs a fused weighted formula in this embodiment, as follows:

[0168] ;

[0169] in, This represents the fusion weight value for a single channel-region combination location in the attention weight matrix. Let be the normalized energy after generalized mean pooling of the target high-frequency feature channel, and k be the maximum pooling response ratio of the target high-frequency feature channel. The activation contrast between the target channel and the non-target channel. The non-target channel leakage ratio. This represents the edge density ratio within a typical spatial target region. The background clutter ratio outside the target area, As a mask inflation penalty factor, This is the offset ratio between the current target region and the prior knowledge base template of the industrial scenario. is the deload factor corresponding to the current computing power level, exp is the natural exponential function, erf is the error function, and cosh is the hyperbolic cosine function.

[0170] The theoretical range is (0,1), and the actual operating range is (0.05,0.95); when A value greater than or equal to 0.70 indicates that the current channel and region combination contributes significantly to recognition and should be prioritized for preservation in convolution calculations; when... A value greater than or equal to 0.40 and less than 0.70 indicates that the current combination is auxiliary information; when... A value less than 0.40 indicates that the current combination contribution is low and can be weakened or frozen in de-load mode.

[0171] The formula adopts a structure of "multi-factor multiplication plus exponential suppression". The first term highlights the energy and contrast of the target channel, the middle term strengthens the difference between the edge density of the target region and the background, and the last term imposes attenuation constraints on the template offset and load reduction coefficient. Therefore, it can reduce the redundant overhead caused by full-space and full-channel calculation while ensuring the relevance of industrial scenarios.

[0172] The working principle of the lightweight attention module 3 specifically designed for industrial scenarios is as follows:

[0173] Based on the terminal's computing power level data and current scene feature parameters, the module linkage control center 1 issues channel range and spatial range instructions to the IS-LAM. The channel attention unit 31 and spatial attention unit 32 calculate the attention weights and spatial masks respectively within the defined range. The weight fusion unit 33 then generates the attention weight matrix and sends it to the hierarchical feature extraction module 4 in real time. Since this module only operates in the target high-frequency feature channels and typical spatial target areas, the computational load is significantly less than that of the general CBAM mode. In the comparative test of this embodiment, based on the same backbone network, IS-LAM reduces the attention calculation FLOPs by 82.1% compared to CBAM, while maintaining sensitivity to pin defects and abnormal shell boundaries. Therefore, it can balance real-time performance with industrial scenario specificity.

[0174] 3. Hierarchical feature extraction module 4;

[0175] The hierarchical feature extraction module 4 includes a hierarchical control unit 41, a feature extraction unit 42, and a data interaction unit 43. (Refer to...) Figure 2 The feature extraction unit 42 further includes a low-level feature extraction subunit 421, a mid-level feature extraction subunit 422, and a high-level feature extraction subunit 423. This module is based on the modified... Construct the image, with a 320×320×3 image as input.

[0176] The network structure is as follows: the input first undergoes 3×3 convolution and BN activation to form 16-channel shallow features; the bottom feature extraction subunit 421 consists of 3 inverted residual blocks with an output size of 80×80×24, mainly extracting pin edges, shell contours and hole boundaries; the middle feature extraction subunit 422 consists of 4 inverted residual blocks with an output size of 40×40×48, mainly extracting shell texture, pin spacing and local notch texture; the high-level feature extraction subunit 423 consists of 3 inverted residual blocks and 1 1×1 convolution block with an output size of 20×20×96, mainly extracting connector category differences, fine-grained defect morphology and multi-part combination semantics.

[0177] The hierarchical control unit 41 receives scheduling instructions from the module linkage control center 1, determines the semantic level of the current feature extraction based on the scheduling instructions, and generates a hierarchical trigger signal. At high computing power levels, the bottom-level feature extraction subunit 421, the middle-level feature extraction subunit 422, and the high-level feature extraction subunit 423 are triggered simultaneously; at medium computing power levels, the bottom-level feature extraction subunit 421 and the middle-level feature extraction subunit 422 are triggered; at low computing power levels, only the bottom-level feature extraction subunit 421 is triggered. The feature extraction unit 42 receives the hierarchical trigger signal and the attention weight matrix sent by IS-LAM, applies the channel attention weights before the depthwise convolution input, applies the spatial attention mask after the convolution output, and then proceeds to the next layer of convolution calculation, thereby achieving dual feature extraction that is both computing power adaptive and scene-customized. The data interaction unit 43 connects to the feature extraction unit 42 and is used to extract intermediate data generated during the feature extraction process and send it to the module linkage control center 1. In this embodiment, the intermediate data includes the 96-dimensional channel activation peak vector, the Shannon entropy of the feature map, the target region occupancy ratio, and the gradient density ratio. These data are used by the few-sample self-learning module 6 for parameter iteration.

[0178] The working principle of the hierarchical feature extraction module 4 is as follows: the hierarchical control unit 41 determines the semantic level to be triggered according to the scheduling instructions; the feature extraction unit 42 performs convolution calculations for the corresponding level in the triggered sub-unit, and fuses the attention weight matrix during the convolution process; the data interaction unit 43 synchronously extracts intermediate data and feeds it back to the module linkage control center 1. In this way, the system can retain fine-grained semantics when the computing power is high, and high-level features are not weakened; when the computing power is low, the system only retains high-contribution, low-cost features such as edges and contours, thereby reducing unnecessary high-level semantic calculations.

[0179] 4. Recognition result output module 5;

[0180] The recognition result output module 5 includes a classification detection unit 51, a confidence determination unit 52, and a result output unit 53. The classification detection unit 51 receives the feature map output by the hierarchical feature extraction module 4 and processes the feature map using a preset lightweight classification detection head to generate the recognition result. For the electronic connector scenario in this embodiment, the lightweight classification detection head adopts a dual-branch structure: one branch is used for category judgment, and the other branch is used for defect location regression. If the input comes from the high-level feature extraction subunit 423, the 20×20×96 feature map is directly reduced to 20×20×64 through 1×1 convolution, then global average pooling is used to obtain a 64-dimensional vector, which is then passed through two fully connected layers to obtain 6 material category outputs and 4-dimensional defect location outputs. If the input comes from the middle or low-level feature extraction subunit 421, it is first unified to 20×20 through adaptive pooling, and then the same classification detection head is executed. The category branch uses Softmax to output the category probability, and the location branch outputs the normalized rectangular box center coordinates and width and height to represent the defect location.

[0181] The confidence level determination unit 52 is connected to the classification detection unit 51 and is used to acquire the confidence level data in the recognition result and compare the confidence level data with the preset confidence level threshold. When the confidence level is not lower than 90%, the result takes effect directly; when the confidence level is lower than 90%, an iteration trigger signal is generated and sent to the module linkage control center 1 to trigger the small sample self-learning module 6 to perform another iteration optimization. In order to avoid occasional false triggering in a single frame, this embodiment further sets a confidence level buffer rule: only when the confidence level of the same category is lower than 90% in at least 2 out of 3 consecutive frames is it allowed to be reported as a valid iteration trigger event. The result output unit 53 is connected to the classification detection unit 51 and is used to output the recognition result to the local storage medium and the industrial LAN transmission interface. In this embodiment, the local storage medium is the SQLite database in the eMMC, and the industrial LAN transmission interface adopts two industrial protocols, OPC UA and Modbus TCP. The output content includes target category, defect location, confidence level, timestamp and workstation number to adapt to the needs of industrial control and data traceability.

[0182] The working principle of the recognition result output module 5 is as follows: After the classification and detection unit 51 completes category recognition and location regression, it sends the results to the confidence determination unit 52. The confidence determination unit 52 determines whether to generate an iteration trigger signal based on a threshold. The result output unit 53 then synchronously writes the recognition results to the local machine and the industrial LAN. Through this structure, the system can not only complete real-time recognition on the terminal side, but also automatically convert low-confidence results into data entry points for subsequent self-learning, thereby improving the ability to discover new scenes.

[0183] 5. Small sample self-learning module 6;

[0184] The few-shot self-learning module 6 includes an image preprocessing unit 61, a feature mining unit 62, and a parameter iteration unit 63. The image preprocessing unit 61 receives new scene few-shot image data collected by the terminal and preprocesses the few-shot images to generate preprocessed few-shot image data. In this embodiment, the few-shot number threshold is set to 8 images, with an actual trigger range of 5 to 10 images; when the cumulative number of new scene images reaches 8, the self-learning process begins. The image preprocessing process includes bilateral filtering for noise reduction, CLAHE brightness enhancement, 320×320 uniform scaling, and manual bounding box writing. The bilateral filtering parameters are a neighborhood diameter of 5, a color domain variance of 25, and a spatial domain variance of 25; the CLAHE cropping threshold is 2.0.

[0185] The feature mining unit 62 is connected to the image preprocessing unit 61. It performs feature clustering on the preprocessed small sample image data based on metric learning, mining the target high-frequency feature channels and typical spatial target regions corresponding to the small sample image data of the new scene, generating new scene feature parameters, and sending these parameters to the module linkage control center 1. In this embodiment, the metric learning model adopts a prototype embedding structure: after the high-level feature extraction subunit 423 outputs a 20×20×96 feature map, it first undergoes global average pooling, then a 128-dimensional fully connected embedding layer to obtain a 128-dimensional feature vector, and finally L2 normalization. Hierarchical clustering is performed on the 128-dimensional feature vector formed from 8 small sample images. When the average distance within a cluster is less than 0.18 and the cosine distance between the cluster and the nearest old category prototype is greater than 0.24, it is determined to be a valid new scene cluster. Subsequently, the feature mining unit 62 sorts the channels according to the average activation difference and selects the top 8 channels from the activation peaks of the 96-dimensional channels as the new target high-frequency feature channels; it extracts typical spatial target regions according to the activation heatmap threshold of 0.62 and the 3×3 closed operation results to form new scene feature parameters.

[0186] The parameter iteration unit 63 is connected to the module linkage control center 1. Driven by the module linkage control center 1, it incrementally updates the IS-LAM parameter table and the fine-grained semantic feature layer convolutional kernels of the hierarchical feature extraction module 4. Simultaneously, it freezes the parameters of the bottom contour feature layer and the middle texture feature layer of the hierarchical feature extraction module 4. During parameter iteration, the freezing ratio is set to 91.7%, meaning only the 18 convolutional kernels in the last two layers of the high-level feature extraction subunit 423 and the IS-LAM parameter table are updated. The updated convolutional kernels account for 8.3% of the total number of convolutional kernels in the entire model, ensuring that the proportion of frozen model parameters is not less than the preset parameter freezing ratio threshold. The optimizer uses AdamW, and the learning rate is set to... The batch size is 4, the maximum number of iterations is 160, and two termination conditions are set: first, stop immediately when the cumulative time reaches 30 seconds, and second, stop early when the confidence of the validation set increases by less than 0.3% for 20 consecutive steps.

[0187] To determine whether new scene feature parameters are suitable for inclusion in the industrial scene basic feature parameter library, this embodiment constructs a new scene evolution score, as follows:

[0188] ;

[0189] Where Y is the evolution score of the new scene, s is the normalized value of the separation between the prototype of the new scene cluster and the most recent old scene prototype, t is the activation concentration of the first 8 newly added high-frequency feature channels, u is the mask stability of the typical spatial target region on multiple enhanced samples, v is the actual update parameter ratio, w is the threshold of the allowed update parameter ratio which is fixed at 0.10 in this embodiment, x is the trial run verification error rate, y is the manual annotation noise ratio, ln is the natural logarithm function, arctan is the arctangent function, exp is the natural exponential function, and arsinh is the inverse hyperbolic sine function.

[0190] In this embodiment, the actual range of Y is When Y is greater than or equal to 0.78, it indicates that the new scene is clearly distinguishable from the old scene, the attention target is stable, and the update cost is controllable. The parameter evolution unit allows the new scene feature parameters to be written into the industrial scene basic feature parameter library and the computing power-attention-feature layer linkage threshold table to be updated. When Y is less than 0.78, it indicates that the new scene samples are still unstable or the update cost is too high. At this time, only the samples are cached and no formal iteration is performed. This formula adopts the structure of "square root gain term plus exponential release term". The first term emphasizes the class separation degree, channel concentration degree, and spatial stability, while the second term emphasizes the inhibitory effect of parameter update cost, verification error, and annotation noise on formal evolution. Therefore, it can make "whether it can evolve" an objective and calculable quantitative criterion.

[0191] The working principle of the small sample self-learning module 6 is as follows:

[0192] The image preprocessing unit 61 first performs denoising, enhancement, and annotation on small sample images of the new scene. The feature mining unit 62 then mines the high-frequency feature channels and spatial target regions corresponding to the new scene through metric learning and generates feature parameters for the new scene. Subsequently, the parameter iteration unit 63, driven by the module linkage control center 1, only updates the IS-LAM parameter table and the high-level feature extraction-related convolutional kernels, without changing the low-level and mid-level parameters. This allows the model to be adapted on the terminal side with fewer samples and in a shorter time, while avoiding the destruction of basic features.

[0193] 6. Working principle of Example 3;

[0194] The method corresponding to this embodiment includes a system initialization stage, a real-time identification stage, a small sample iteration stage, and an emergency computing power adaptation stage. The four stages are seamlessly connected on the terminal side.

[0195] During system initialization, the terminal first loads the basic feature parameter library for the industrial scenario corresponding to the electronic connector scenario and the initial computing power-attention-feature layer linkage threshold table. Then, it loads the lightweight basic model integrating IS-LAM and hierarchical feature extraction module 4, and sets the confidence threshold to 90%, the small sample size threshold to 8 images, the parameter freeze ratio threshold to 90%, the fine-tuning time threshold to 30 seconds, and the acquisition frequency thresholds to 10ms / 50ms / 500ms. After initialization, the computing power dynamic sensing module 2 enters real-time acquisition mode, while the remaining modules enter standby mode.

[0196] During the real-time recognition phase, the image acquisition unit links the connector image input module to the control center 1, and the computing power dynamic perception module 2 synchronously sends terminal computing power level data. The module linkage control center 1 determines the current attention calculation range and feature extraction level to be activated based on the linkage threshold table, and issues scheduling instructions to the IS-LAM and hierarchical feature extraction module 4 respectively. The IS-LAM generates attention weight matrices for the target high-frequency feature channels corresponding to pin edges, shell contours, and cavity textures, as well as the typical spatial target region in the center, according to the instructions. The hierarchical feature extraction module 4 triggers the bottom, middle, or high-level semantic layers according to the hierarchical control signals, and integrates the attention weight matrices into the convolution calculation process to generate feature maps. The recognition result output module 5 classifies and regresses the feature maps, outputting the target category, defect location, and confidence level. When the confidence level is not lower than 90%, the result is directly output to the local network and industrial LAN; when the confidence level is lower than 90%, the confidence level determination unit 52 generates an iteration trigger signal and feeds it back to the module linkage control center 1.

[0197] During the small-sample iteration phase, the terminal collects 5 to 10 small-sample images of the new scene; in this embodiment, 8 images are used as the formal trigger quantity. After the target region and category are manually labeled, the image preprocessing unit 61 performs denoising, enhancement, and size normalization. The feature mining unit 62 forms a new scene cluster through metric learning and extracts new channels and new spatial target regions from high-level features. After receiving the new scene feature parameters, the module linkage control center 1 drives the IS-LAM to update the parameter table, adding new channel weight allocation rules and spatial attention mask range; at the same time, it freezes the bottom-level feature extraction subunit 421 and the middle-level feature extraction subunit 422, and only performs incremental updates on a small number of convolutional kernels of the high-level feature extraction subunit 423. After the parameter iteration is completed, if the new scene evolution score Y is not lower than 0.78, the new scene feature parameters are written into the industrial scene basic feature parameter library, and the linkage threshold table is updated synchronously. The system automatically switches to the new scene real-time recognition mode.

[0198] During the emergency computing power adaptation phase, the computing power dynamic sensing module 2 continuously monitors the terminal hardware status. When it detects a sudden increase of at least 50 percentage points in CPU utilization within 100ms, a sudden decrease of at least 30% in memory availability within 100ms, or a rise of at least 8°C in chip temperature within 1 second, it immediately sends a computing power surge signal to the module linkage control center 1. Upon receiving the surge signal, the module linkage control center 1 immediately triggers the emergency computing power adaptation strategy, forcibly downgrading the terminal's computing power level by one level; if the current computing power level is high, it switches to medium; if the current computing power level is medium, it switches to low. Subsequent emergency scheduling instructions freeze some non-core attention calculations, narrow the typical spatial target area, and switch to a lower-level feature extraction. Once the CPU utilization, memory availability, and temperature have continuously and stably returned to normal ranges for at least 5 seconds, the module linkage control center 1 cancels the emergency adaptation strategy and reverts to the regular real-time recognition mode. This ensures that the terminal maintains basic recognition capabilities during load surges, preventing lag and crashes.

[0199] 7. Technical effects of Example 3;

[0200] Under the test conditions of this embodiment, the terminal runs continuously for 8 hours, the input cycle time of the dynamic sorting station is 8 pieces / second, the input cycle time of the static verification station is 1 piece / 2 seconds, and the ambient temperature is 25±2℃. Test results show that: at the dynamic sorting station, the system can smoothly switch between high and medium computing power levels according to the computing power status, with an average single-frame recognition time of 17.4ms and a combined accuracy rate of 96.2% for connector category recognition and defect detection; at the static verification station, the parameter acquisition unit 21 automatically adjusts the acquisition frequency to 500ms / time, reducing the CPU usage of the computing power perception thread by 6.8%, indicating that the dynamic frequency adjustment mechanism can reduce static scene overhead; in the newly added "pin bending" defect scenario, the system completes terminal-side iteration with 8 small samples, with an actual fine-tuning time of 25.6s and a freeze ratio of 91.7%, and the accuracy rate of new defect recognition after iteration increases from 89.1% to 95.4%; in the burst load test, the CPU usage rate increases by 53 percentage points within 100ms and the remaining memory decreases by 31%, and the system enters the computing power emergency adaptation state within 30ms, with the single-frame recognition time remaining within 15.2ms, and no stuttering or abnormal process exits occur during operation.

[0201] In summary, this embodiment achieves real-time perception of multi-source hardware status and refined classification of computing power levels through parameter acquisition unit 21 and parameter processing unit 22, realizes customized lightweight attention calculation for high-frequency feature channels and typical spatial target regions of industrial images through IS-LAM, realizes hierarchical semantic extraction triggered by computing power level through hierarchical feature extraction module 4, realizes closed-loop feedback of low confidence results through recognition result output module 5, and realizes rapid evolution of new scenarios on the terminal side through few-sample self-learning module 6. Therefore, it can simultaneously take into account real-time performance, recognition accuracy, scene adaptability and operational stability on resource-constrained smart terminals.

[0202] A lightweight real-time image recognition method for smart terminals is proposed, applied to the aforementioned lightweight real-time image recognition system for smart terminals, with reference to... Figure 3 ,include:

[0203] Step S1: The computing power dynamic sensing module 2 collects the hardware status parameters of the smart terminal and processes them to obtain the terminal computing power level data, which is then sent to the module linkage control center 1.

[0204] In step S2, the module linkage control center 1 generates and sends scheduling instructions to the lightweight attention module 3 and the hierarchical feature extraction module 4 based on the terminal computing power level data and the preset linkage threshold table; and drives the lightweight attention module 3 to perform parameter iteration based on the new scene feature parameters sent by the few-sample self-learning module 6, and controls the hierarchical feature extraction module 4 to incrementally update the fine-grained semantic feature layer.

[0205] Step S3: The lightweight attention module 3 dynamically adjusts the attention calculation range according to the scheduling instructions, generates an attention weight matrix, and sends it to the hierarchical feature extraction module 4.

[0206] In step S4, the hierarchical feature extraction module 4 triggers feature extraction at the corresponding semantic level according to the scheduling instruction, and integrates the attention weight matrix into the convolution calculation process of feature extraction, and outputs the feature map to the recognition result output module 5.

[0207] Step S5: The recognition result output module 5 processes the feature map to obtain the recognition result and feeds back the confidence data of the recognition result to the module linkage control center 1.

[0208] In step S6, the few-shot self-learning module 6 processes the collected few-shot image data of the new scene through metric learning, obtains the feature parameters of the new scene, and sends them to the module linkage control center 1.

[0209] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A lightweight real-time image recognition system for smart terminals, characterized in that, include: The module linkage control center, and the computing power dynamic perception module, lightweight attention module, hierarchical feature extraction module, few-shot self-learning module and recognition result output module respectively connected to the module linkage control center; The computing power dynamic sensing module is used to collect hardware status parameters of the smart terminal and process them to obtain terminal computing power level data, which is then sent to the module linkage control center. The module linkage control center is used to query a preset linkage threshold table based on the terminal computing power level data to generate scheduling instructions, and then send them to the lightweight attention module and the hierarchical feature extraction module. And based on the new scene feature parameters sent by the few-sample self-learning module, the lightweight attention module is driven to perform parameter iteration, and the hierarchical feature extraction module is controlled to incrementally update the fine-grained semantic feature layer; wherein, the linkage threshold table contains the mapping relationship between computing power level and attention calculation range and feature extraction level, high computing power level corresponds to full target region attention calculation and fine-grained semantic feature extraction, medium computing power level corresponds to main body region attention calculation and mid-layer texture feature extraction, and low computing power level corresponds to core feature channel attention calculation and bottom contour feature extraction; The lightweight attention module is used to dynamically adjust the attention calculation range according to the scheduling instruction, generate an attention weight matrix, and send it to the hierarchical feature extraction module. The hierarchical feature extraction module is used to trigger feature extraction at the corresponding semantic level according to the scheduling instruction, and to fuse the attention weight matrix into the convolution calculation process of feature extraction, and output the feature map to the recognition result output module. The recognition result output module is used to process the feature map to obtain the recognition result, and to feed back the confidence data of the recognition result to the module linkage control center. The few-sample self-learning module is used to mine high-frequency feature channels and spatial target regions corresponding to new scenes through metric learning, generate new scene feature parameters, and send them to the module linkage control center.

2. The lightweight real-time image recognition system for smart terminals according to claim 1, characterized in that, The module linkage control center includes: A threshold storage unit is used to store a preset linkage threshold table and a preset industrial scenario basic feature parameter library; the industrial scenario basic feature parameter library contains target feature parameters of multiple typical intelligent manufacturing scenarios. The scheduling decision unit is connected to the threshold storage unit and is used to receive the terminal computing power level data sent by the computing power dynamic perception module, query the linkage threshold table according to the terminal computing power level data, determine the attention calculation range and feature extraction level that match the current computing power level, and generate a scheduling instruction based on the determination result, and issue the scheduling instruction to the lightweight attention module and the hierarchical feature extraction module. The parameter evolution unit, connected to the threshold storage unit, is used to receive the new scene feature parameters sent by the few-sample self-learning module, store the new scene feature parameters in the industrial scene basic feature parameter library, update the mapping relationship related to the new scene feature parameters in the linkage threshold table according to the new scene feature parameters, drive the lightweight attention module to perform parameter iteration, and control the hierarchical feature extraction module to incrementally update the fine-grained semantic feature layer.

3. The lightweight real-time image recognition system for smart terminals according to claim 2, characterized in that, The module linkage control center also includes: The computing power monitoring unit is used to receive the terminal computing power level data sent by the computing power dynamic sensing module, and to monitor the changing trend of the terminal computing power level data in real time. An emergency adaptation unit, connected to the computing power monitoring unit and the scheduling decision unit, is used to trigger an emergency computing power adaptation strategy when the computing power monitoring unit detects that the change in the terminal computing power level data exceeds a preset mutation threshold. This strategy forces the current computing power level to be downgraded by one level, and instructs the scheduling decision unit to regenerate scheduling instructions based on the downgraded computing power level. These instructions are then sent to the lightweight attention module and the hierarchical feature extraction module to freeze non-core attention calculations and switch to a lower level of feature extraction. The recovery determination unit is connected to the computing power monitoring unit and the emergency adaptation unit. After the emergency adaptation unit triggers the emergency adaptation strategy, it continuously monitors the terminal computing power level data. When the terminal computing power level data recovers to the normal range and stabilizes for more than a preset stable time threshold, it cancels the emergency adaptation strategy and restores the normal scheduling mode of the scheduling decision unit.

4. The lightweight real-time image recognition system for smart terminals according to claim 1, characterized in that, The computing power dynamic sensing module includes: The parameter acquisition unit is used to acquire the hardware status parameters of the smart terminal in real time based on the system application programming interface of the smart terminal, and adjust the acquisition frequency according to the preset acquisition frequency threshold. The parameter processing unit, connected to the parameter acquisition unit, is used to normalize the hardware status parameters and divide the processed hardware status parameters into multiple computing power levels according to a preset computing power level threshold, thereby generating the terminal computing power level data. The preset computing power level threshold is based on the hardware configuration of the smart terminal and the preset industrial scene recognition requirements, and supports manual fine-tuning on the terminal side.

5. The lightweight real-time image recognition system for smart terminals according to claim 1, characterized in that, The lightweight attention module includes: The channel attention unit is used to receive the scheduling instructions issued by the module linkage control center, dynamically determine the target high-frequency feature channel range according to the scheduling instructions, and allocate attention weights to the feature channels within the target high-frequency feature channel range to generate channel attention weights. The spatial attention unit is used to receive the scheduling instructions issued by the module linkage control center, dynamically determine the range of typical spatial target areas according to the scheduling instructions, and perform attention mask calculation on the spatial positions within the range of typical spatial target areas to generate spatial attention masks. The weight fusion unit is connected to the channel attention unit and the spatial attention unit respectively, and is used to fuse the channel attention weights and the spatial attention mask to generate the attention weight matrix and send it to the hierarchical feature extraction module; The target high-frequency feature channel range and the typical spatial target region range are defined based on a preset industrial scenario prior knowledge base, and the calculation range of the channel attention unit and the spatial attention unit is dynamically adjusted according to the scheduling instructions.

6. The lightweight real-time image recognition system for smart terminals according to claim 1, characterized in that, The hierarchical feature extraction module includes: The hierarchical control unit is used to receive the scheduling instructions issued by the module linkage control center, determine the semantic level of the current feature extraction according to the scheduling instructions, and generate a hierarchical trigger signal. The feature extraction unit, connected to the hierarchical control unit, is used to receive the hierarchical trigger signal and the attention weight matrix sent by the lightweight attention module, trigger the feature extraction network of the corresponding semantic level according to the hierarchical trigger signal, and fuse the attention weight matrix into the convolution calculation process of the feature extraction network to output the feature map; The data interaction unit is connected to the feature extraction unit and is used to extract intermediate data generated by the feature extraction unit during the feature extraction process, and send the intermediate data to the module linkage control center. The intermediate data is used by the few-sample self-learning module for parameter iteration.

7. The lightweight real-time image recognition system for smart terminals according to claim 6, characterized in that, The feature extraction unit includes: The low-level feature extraction subunit is used to extract the low-level contour features of industrial images, corresponding to the preset low computing power level requirements. The mid-layer feature extraction subunit is connected to the bottom-layer feature extraction subunit and is used to extract mid-layer texture features based on the bottom-layer contour features, corresponding to the preset mid-level computing power requirements. A high-level feature extraction subunit, connected to the mid-level feature extraction subunit, is used to extract fine-grained semantic features based on the mid-level texture features, corresponding to a preset high computing power level requirement; The hierarchical control unit triggers one or more of the bottom-level feature extraction subunit, the middle-level feature extraction subunit, and the high-level feature extraction subunit to perform feature extraction according to the scheduling instruction, and integrates the attention weight matrix into the convolution calculation process of the triggered subunit.

8. The lightweight real-time image recognition system for smart terminals according to claim 1, characterized in that, The recognition result output module includes: The classification and detection unit is used to receive the feature map output by the hierarchical feature extraction module, process the feature map through a preset lightweight classification and detection head, and generate a recognition result; wherein, the recognition result includes target category, defect location, and confidence data; The confidence determination unit, connected to the classification detection unit, is used to acquire the confidence data in the recognition result, compare the confidence data with a preset confidence threshold, and when the confidence data is lower than the confidence threshold, generate an iteration trigger signal and send it to the module linkage control center to trigger the small sample self-learning module to perform another iteration optimization. The result output unit, connected to the classification and detection unit, is used to output the identification results to the local storage medium and the industrial local area network transmission interface, adapting to the industrial control and data traceability requirements of intelligent manufacturing production lines.

9. The lightweight real-time image recognition system for smart terminals according to claim 1, characterized in that, The few-sample self-learning module includes: An image preprocessing unit is used to receive new scene small sample image data collected by the terminal, and preprocess the new scene small sample image data to generate preprocessed small sample image data; wherein, the number of new scene small sample image data is less than a preset small sample number threshold. The feature mining unit, connected to the image preprocessing unit, is used to perform feature clustering on the preprocessed small sample image data based on metric learning, mine the target high-frequency feature channels and typical spatial target regions corresponding to the small sample image data of the new scene, generate the feature parameters of the new scene, and send the feature parameters of the new scene to the module linkage control center. The parameter iteration unit, connected to the module linkage control center, is used to incrementally update the parameter table of the lightweight attention module and incrementally update the convolution kernel of the fine-grained semantic feature layer of the hierarchical feature extraction module according to the drive of the module linkage control center, while freezing the parameters of the bottom contour feature layer and the middle texture feature layer of the hierarchical feature extraction module. When the parameter iteration unit performs incremental updates, the proportion of frozen model parameters to the total number of model parameters is not less than a preset parameter freezing ratio threshold, and the time consumed in a single iteration is less than a preset fine-tuning time threshold.

10. A lightweight real-time image recognition method for smart terminals, applied to the lightweight real-time image recognition system for smart terminals as described in any one of claims 1-9, characterized in that, include: Step S1: The computing power dynamic sensing module collects the hardware status parameters of the smart terminal and processes them to obtain the terminal computing power level data, which is then sent to the module linkage control center. Step S2: The module linkage control center queries the preset linkage threshold table based on the terminal computing power level data to generate a scheduling instruction, and sends it to the lightweight attention module and the hierarchical feature extraction module. And based on the new scene feature parameters sent by the few-sample self-learning module, the lightweight attention module is driven to perform parameter iteration, and the hierarchical feature extraction module is controlled to incrementally update the fine-grained semantic feature layer; wherein, the linkage threshold table contains the mapping relationship between computing power level and attention calculation range and feature extraction level, high computing power level corresponds to full target region attention calculation and fine-grained semantic feature extraction, medium computing power level corresponds to main body region attention calculation and mid-layer texture feature extraction, and low computing power level corresponds to core feature channel attention calculation and bottom contour feature extraction; Step S3: The lightweight attention module dynamically adjusts the attention calculation range according to the scheduling instruction, generates an attention weight matrix, and sends it to the hierarchical feature extraction module. Step S4: The hierarchical feature extraction module triggers feature extraction at the corresponding semantic level according to the scheduling instruction, and integrates the attention weight matrix into the convolution calculation process of feature extraction, and outputs the feature map to the recognition result output module. Step S5: The recognition result output module processes the feature map to obtain the recognition result, and feeds back the confidence data of the recognition result to the module linkage control center. In step S6, the few-sample self-learning module mines the high-frequency feature channels and spatial target regions corresponding to the new scene through metric learning, generates new scene feature parameters, and sends them to the module linkage control center.

Images