Attention electric power visual model optimization method and system, medium and server

Abstract

The invention discloses an attention electric power visual model optimization method and system, a medium and a server. The method comprises the following steps: identifying an original module in an existing visual model; converting the obtained original module into an attention module, and then constructing a visual model search space for attention optimization; determining a search strategy, and searching the constructed visual model search space to obtain an attention power visual model; selecting a performance evaluation strategy to evaluate the obtained attention power vision model; and repeatedly searching and evaluating until a termination condition is reached, and obtaining an optimized attention power vision model. The problem that an existing model is insufficient in performance can be solved, and the performance of a current electric power vision model is improved.

Description

Background Art

[0002] In cognitive science, due to the bottleneck of information processing, we selectively focus on a part of all information while ignoring other visible information. By using the attention mechanism, the model can more automatically focus on the important parts of the input data related to the task, and can flexibly adjust the focus according to the needs of the task. However, current convolutional neural networks often do not have attention modules. By adding attention modules, their expression ability can be enhanced. However, blindly adding attention modules will cause overfitting problems and even reduce the performance of the model.

[0003] As a powerful tool for processing image data, convolutional neural networks (CNNs) have achieved great success in many visual tasks. However, standard CNN architectures usually lack explicit attention modules, which means that they may not optimally simulate the ability of the human visual system to focus attention and suppress irrelevant information. Therefore, integrating attention mechanisms has become a research direction to improve the performance of CNNs. Due to the bottleneck of information processing, current convolutional neural networks often do not have attention modules. By adding attention modules, their expressive power can be enhanced. However, blindly adding attention modules will cause overfitting problems and even reduce the performance of the model.

[0004] The attention mechanism is a function that simulates human visual attention selection. Introducing the attention mechanism in a deep learning model can significantly improve the model's ability to capture key information, thereby improving model performance. Integrating the attention module in a convolutional neural network (CNN) can indeed enhance the model's focus on important features, but if used improperly, it will bring the risk of overfitting or have a negative impact on model performance. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide an attention power visual model optimization method, system, medium and server for the above-mentioned deficiencies in the prior art, so as to solve the technical problem that the performance of the original model is insufficient and needs to be improved. However, blindly using the attention mechanism optimization will lead to overfitting, and the performance may even decrease. The existing network architecture search (NAS) algorithm cannot be fully and directly applied to the search for the attention mechanism.

[0006] The present invention adopts the following technical solutions: A method for optimizing an attention power visual model comprises the following steps: Identify and analyze primitive modules in existing vision models; By converting some of the original modules into attention modules, we construct an attention-optimized visual model search space. Determine the search strategy, search the constructed visual model search space, and obtain the attention power visual model; Select a performance evaluation strategy to evaluate the obtained attention power visual model; Repeat the above steps until the termination condition is reached to obtain the optimized attention power visual model.

[0007] Preferably, the original module analysis identifies modules including convolutional modules and residual modules that can be converted into attention mechanisms.

[0008] Preferably, attention optimization includes but is not limited to spatial attention mechanism, channel attention mechanism, residual attention mechanism, multi-scale attention mechanism, and a combination of these attention mechanisms.

[0009] Preferably, the search strategy includes but is not limited to a random search strategy, a reinforcement learning-based strategy, a genetic algorithm-based strategy, a Bayesian optimization, and a gradient-based strategy.

[0010] Preferably, the gradient-based strategy adopts an improved DARTS algorithm, specifically: The original modules that can be converted into attention modules are divided into cells, and each cell shares parameters in multiple places in the network.

[0011] Preferably, the performance evaluation strategy includes but is not limited to multi-objective optimization, single-objective optimization, and direct model evaluation, proxy model, approximate evaluation, shared parameters, and parameter-level migration.

[0012] Preferably, the termination condition includes but is not limited to reaching a maximum number of iterations.

[0013] In a second aspect, an embodiment of the present invention provides an attention power visual model optimization system, comprising: Recognition module, identifying and analyzing the original modules in the existing visual model; The conversion module converts part of the original module into an attention module, and then constructs the attention-optimized visual model search space; A search module determines the search strategy, searches the constructed visual model search space, and obtains an optimized attention power visual model; An evaluation module selects a performance evaluation strategy to evaluate the optimized attention power visual model; The optimization module is repeated until the termination condition is reached to obtain the attention power visual model after attention optimization.

[0014] In a third aspect, a server comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned attention power visual model optimization method when executing the computer program.

[0015] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, comprising a computer program, which, when executed by a processor, implements the steps of the above-mentioned attention power visual model optimization method.

[0016] In a fifth aspect, a chip comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned attention power visual model optimization method when executing the computer program.

[0017] In a sixth aspect, an embodiment of the present invention provides an electronic device, comprising a computer program, which, when executed by the electronic device, implements the steps of the above-mentioned attention power visual model optimization method.

[0018] Compared with the prior art, the present invention has at least the following beneficial effects: A method for optimizing power vision models with attention. The attention mechanism can help the model focus on important features, while NAS can ensure that these attention modules are accurately placed in the most useful positions in the model. NAS can avoid unnecessary complexity by searching for the optimal structure while enhancing the model, thereby reducing the risk of overfitting. It is performed directly on existing datasets without the need for additional samples or annotations, thus saving cost and time. NAS automatically searches for the best attention module configuration, reducing the process of manual trial and error and improving the efficiency of model optimization. It can fully utilize the existing model network structure and optimize the existing model network.

[0019] Furthermore, we make full use of existing models and reasonably set up the search space to facilitate structural search; select the most appropriate strategy based on specific task requirements, available computing resources, and the characteristics of the search space; achieve faster and more accurate network structure search, significantly improve the efficiency of attention power visual model optimization, and provide strong support for solving real-world visual tasks.

[0020] Furthermore, by setting up the attention mechanism, the module effect and ultimately the model effect can be improved, which not only improves the performance of the model on specific tasks, but also increases the model's ability to handle complex and richly changing data.

[0021] Furthermore, the existing search strategy is used to save search time, which greatly reduces search time and computing resource consumption while ensuring model performance improvement, thereby improving the efficiency of the entire model development process.

[0022] Furthermore, strategies are set according to the purpose to obtain the desired model.

[0023] Furthermore, the termination strategy can be set as needed to obtain the desired model.

[0024] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0025] In summary, the present invention can improve the performance of current power vision models.

[0026] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 It is a flow chart of the present invention; Figure 2 Schematic diagram of CBAM combining channel attention and spatial attention; Figure 3 : is a network search strategy diagram in an embodiment of the present invention, wherein (a) the search algorithm needs to select candidate operations from the search space, (b) there are many pre-designed candidate operations in the search space, and the search strategy combines different candidate operations, (c) the evaluation strategy performs performance evaluation on the combined network architecture, and (d) retains the candidate operations that contribute most to performance improvement, and finally obtains the target architecture; Figure 4 A schematic diagram of a computer device provided by an embodiment of the present invention; Figure 5 The block diagram of a chip provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0031] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0032] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects are in an "or" relationship.

[0033] It should be understood that, although the terms first, second, third, etc. may be used to describe preset ranges, etc. in the embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0034] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0035] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0036] The present invention provides an attention power visual model optimization method, which enhances the performance of the existing model through the attention mechanism, and uses the network structure search (Neural Architecture Search, NAS) neural structure method to obtain the best attention-optimized visual model. It can optimize the existing visual model and avoid the overfitting problem caused by blindly adding attention modules. Compared with the traditional method of improving the quality of sample sets or increasing the size of sample sets, it does not require additional samples and annotations, and can be performed directly on the original visual model.

[0037] Example 1 See also Figure 1 The present invention provides an attention power visual model optimization method, comprising the following steps: S1. Analyze the network structure of the visual model and identify the original modules in the visual model that can be converted into attention modules. The original modules include convolution modules and residual modules. The complete network structure information of the visual model includes: Convolutional layer: responsible for extracting features of input data.

[0038] Activation function: adds nonlinear factors to help the network learn complex patterns.

[0039] Pooling layer: reduces the dimension of the feature map while retaining important feature information.

[0040] Residual connections: Allow information flow across layers, helping to alleviate the vanishing gradient problem in deep networks.

[0041] Normalization layer (such as Batch Normalization): speeds up network training and improves stability.

[0042] And other visual model network modules.

[0043] S2, convert the original module obtained in step S1 into an attention module, and then construct an attention-optimized visual model search space; S201. Constructing the search space Define variable parameters: During the transformation, make the variable parameters part of the search space. For example, the number of heads in the attention module, the dimension of the hidden layer.

[0044] Formulate a search range: Define a search range for each variable parameter that covers enough possibilities to explore the best configuration of the attention module.

[0045] Select the type of attention mechanism: According to the analysis results of step S1, select a suitable type of attention mechanism. The optional attention mechanisms include but are not limited to spatial attention mechanism, channel attention mechanism, residual attention mechanism, multi-scale attention mechanism, and a combination of these attention mechanisms.

[0046] According to the specific requirements of the visual task and the characteristics of the original model, the existing attention mechanisms are adjusted or combined to create an attention module that is more suitable for the current scenario.

[0047] S202, convert the original module Replacement or enhancement: For the convolutional modules and residual modules identified in step S1, they are replaced or enhanced with the designed attention modules, including redesigning the network layers or adding attention layers on top of the original network layers.

[0048] Maintain interface compatibility: Ensure seamless connection between old and new modules during the conversion process to maintain the functional integrity of the entire network structure.

[0049] S203, Preliminary Verification and Screening Rapid evaluation mechanism: A rapid evaluation mechanism is established to perform preliminary performance evaluation on the generated model structures at an early stage. This helps to quickly screen out structures with obvious poor performance, narrow the search scope, and improve the efficiency of subsequent detailed searches.

[0050] Dynamically adjust the search strategy: Based on the results of the initial evaluation, the search strategy is adjusted to focus on searching the space near the model structure that shows initial potential.

[0051] S3, determining a search strategy, searching the visual model search space constructed in step S2, and obtaining an attention power visual model; Find the optimal network structure in the visual model search space containing the attention module constructed in step S2, as follows: S301. Select search strategy Optional search strategies include but are not limited to random search strategies, strategies based on reinforcement learning, strategies based on genetic algorithms, Bayesian optimization, and gradient-based strategies. Preferably, the gradient-based strategy adopts the DARTS algorithm improved for the attention module.

[0052] The improved DARTS algorithm is as follows: It is no longer divided into Normal-Cell and Reduction-Cell, but is divided into Cells according to the original modules that can be converted into attention modules. For example, Cells contain optimized modules with only channel attention mechanism, optimized modules with only spatial attention mechanism, optimized modules with spatial channel attention combined mechanism, and modules without attention, etc. The number of cells included is determined according to the scale and computing power of the network. A Cell can share parameters in multiple places in the network. For example, ResNet can share parameters on its BasicBlock or Bottleneck, and VGGNet shares parameters on the VGG block, thereby reducing the search space and improving search efficiency.

[0053] The random search strategy explores by randomly sampling candidate structures in the search space. Although this method is simple, it can quickly discover potential excellent structures in the search space at an early stage. The advantages of random search are its ease of implementation and its lack of assumptions about the shape of the search space, which makes it effective even when faced with highly non-convex or unknown search spaces. By parallelizing the execution of random search, a large number of different network structures can be evaluated in a short period of time, thus saving search time.

[0054] Reinforcement learning (RL) strategies view the search process as a sequential decision problem, where the agent takes actions in the search space to build a network structure and is rewarded based on the network's performance. RL-based strategies improve search strategies by continuously learning how to maximize future rewards. This approach enables effective exploration and exploitation over long time scales, gradually focusing on promising areas in the search space, thereby avoiding wasting time in less promising areas. In this way, RL-based strategies can effectively reduce search time while finding high-performance network structures.

[0055] Genetic Algorithms (GAs) draw on the ideas of natural selection and genetic mechanisms to evolve excellent network structures by implementing selection, crossover, and mutation operations. A significant advantage of GA is that it can evaluate multiple candidate structures in parallel, thereby accelerating the search process. In addition, GA can combine the advantages of different individuals through crossover operations to promote the spread of excellent characteristics, while maintaining the diversity of the population through mutations to prevent premature convergence. Therefore, GA can discover high-quality network structures in relatively few iterations, effectively saving search time.

[0056] Bayesian optimization is an efficient global optimization method that is particularly suitable for evaluating costly objective functions. In NAS, Bayesian optimization approximates the true objective function (such as the accuracy on the validation set) by building a proxy model, and then uses this proxy model to predict which network structures may have better performance. In this way, Bayesian optimization can guide the search in a more promising direction, reduce the evaluation of inefficient structures, and thus significantly reduce the search time.

[0057] Gradient-based strategies directly search for the optimal structure in a continuous search space by parameterizing the network structure and using the gradient descent algorithm to optimize these parameters. This method transforms the discrete structure search problem into a continuous optimization problem, making it possible to use the efficient gradient descent algorithm. Since gradient descent can provide a clear search direction, gradient-based strategies can quickly approach the optimal structure, significantly reducing the amount of computation and time required for the search.

[0058] S302: Implementing search strategy Initialization: Perform necessary initialization work according to the selected search strategy.

[0059] Model architecture evaluation: For each model architecture proposed by the search strategy, it is evaluated using the performance evaluation strategy defined in step S4. This typically involves training the model on a specific dataset and recording key performance metrics.

[0060] Iterative search and optimization: Based on the results of the model structure evaluation, the state of the search strategy is updated (such as the agent model in Bayesian optimization, the policy network in reinforcement learning, and the population in evolutionary algorithms), and then a new model structure is proposed for evaluation. This process is repeated, gradually moving towards a model structure with better performance.

[0061] S303. Monitoring and adjustment of the search process Real-time monitoring: During the search process, real-time monitoring of the performance of different model structures and the learning progress or optimization trend of the search strategy helps to identify problems in a timely manner and make corresponding adjustments.

[0062] Dynamically adjust the search strategy: Based on the feedback information during the search process, dynamically adjust the parameters or direction of the search strategy to improve the search efficiency and quality. For example, the search range can be gradually narrowed to more intensively explore structures near known high-performance areas.

[0063] S304. Termination conditions and result analysis Set termination conditions: Set the termination conditions of the search process in advance, such as reaching a certain number of iterations, the improvement in model performance being less than a certain threshold, etc.

[0064] Result analysis and selection: After the search process is completed, the performance data of all evaluated model structures are analyzed, and the model structure with the best performance is selected as the final result.

[0065] S4, selecting a performance evaluation strategy to evaluate the optimized attention power visual model obtained in step S3; Optional performance evaluation strategies include, but are not limited to, multi-objective optimization (considering factors such as model accuracy, computational cost, and model size simultaneously), single-objective optimization (considering only one factor, such as accuracy), direct model evaluation, surrogate models, lower fidelity evaluation (shortening training time, training with a subset of the dataset, or using low-pixel data, etc.), weight sharing (also known as one-short learning), and parameter-level migration (using previously trained model weight parameters to assign values ​​to the target problem, and starting the optimization from a high starting point will greatly improve efficiency).

[0066] S401. Define evaluation indicators First, we need to define appropriate evaluation metrics to measure the performance of the visual model. These metrics may include: Accuracy: The proportion of correct predictions made by the model.

[0067] Loss function value (Loss): The loss value of the model on the training set or validation set, which is used to measure the difference between the model's predicted value and the actual value.

[0068] Recall, Precision, F1 score: For classification problems with imbalanced datasets, these metrics can provide a more detailed performance perspective than accuracy.

[0069] Computational efficiency: This includes model size, running time, FLOPS (floating point operations), etc. This is especially important when deploying models on resource-constrained devices.

[0070] S402. Data preprocessing and division To ensure fairness and accuracy of the evaluation, the data used for evaluation needs to be properly preprocessed and divided into training, validation, and test sets (if not already divided). Preprocessing may include standardization, normalization, data augmentation, and other operations, while the division of the data set facilitates model tuning during training and objective evaluation of the final model performance.

[0071] S403, Model training and verification Use the model structure generated in step S3 to train the model on the training set. During the training process, the validation set can be used to adjust hyperparameters and prevent overfitting. The key purpose of this step is to enable the model to learn on the given training data and perform a preliminary performance evaluation on the validation set.

[0072] S404, Performance evaluation and recording After the model training is completed, the test set is used to evaluate the final performance of the model. During the evaluation, the performance of the model is calculated according to the evaluation indicators defined in step S401, and the values ​​of these key indicators are recorded. These evaluation results will be used to compare the advantages and disadvantages of different model structures during the search process.

[0073] S405. Result analysis and feedback Analyze the performance evaluation results obtained on the test set to determine the advantages and disadvantages of the current model structure. This information is crucial for subsequent model structure search and optimization, and it can guide the search algorithm to explore in a more promising direction.

[0074] S406. Adjust the evaluation strategy Based on the evaluation results and the actual performance of the model, it may be necessary to return to the previous steps (such as S1, S2, or S3) to make corresponding adjustments. This may include changing the search space, adjusting the search strategy, modifying the hyperparameters of model training, etc., in order to obtain better model performance in the next iteration.

[0075] Through this cyclic iterative process, the visual model is gradually optimized until the predetermined termination conditions are met, such as reaching the maximum number of iterations, the model performance is no longer significantly improved, etc. Finally, the model with the best performance is selected as the visual model after attention optimization.

[0076] S5. Repeat steps S3 and S4 until the termination condition is reached (for example, the maximum number of iterations is reached, the performance requirements are met, etc.), and obtain the attention-optimized visual model, which is applied to the search attention mechanism.

[0077] S501, iterative optimization Based on the evaluation results, according to the search strategy, the search space is used to select the corresponding model structure for evaluation.

[0078] S502, termination condition setting The iterative process needs to set clear termination conditions, which can be: Maximum number of iterations: Pre-set a number of iterations, and stop searching and optimizing after reaching this number.

[0079] Performance improvement threshold: When the model performance improvement is lower than a certain threshold after several consecutive iterations, it is considered that there is limited room for further optimization, and the iteration is terminated.

[0080] Resource constraints: include constraints on computing resources and time resources. When these constraints are reached, the iteration will stop even if other termination conditions are not met.

[0081] S503. Evaluation termination condition After each iteration, check whether any termination conditions are met. This requires reviewing the performance indicators of the current model and comparing them with the pre-set termination conditions. If the termination conditions are met, proceed to the next step; otherwise, return to step S3 to continue the search and optimization process.

[0082] S504. Selection and evaluation of the final model Once the termination condition is met, the best performing model is selected from all the models generated in the iterations as the final attention-optimized visual model. This is usually determined based on the performance on the validation set or test set. In addition, a comprehensive evaluation of the final model is required to ensure that it has good generalization ability and performance under various conditions.

[0083] S505, Post-processing and application Perform necessary post-processing on the selected best model, such as pruning and quantization, to facilitate deployment in actual application scenarios. In addition, corresponding documents and instructions should be prepared to help understand and use this model.

[0084] Through step S5, the entire attention power visual model optimization method is completed, realizing the output from the setting of the initial model to the final optimized model, providing an efficient and optimized solution for solving specific visual tasks.

[0085] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0086] Example 2 The present invention provides an attention power visual model optimization system, which can be used to implement the above-mentioned attention power visual model optimization method. Specifically, the attention power visual model optimization system includes a recognition module, a conversion module, a search module, an evaluation module and an optimization module.

[0087] Among them, the recognition module recognizes and analyzes the original modules in the existing visual model; The conversion module converts part of the original module into an attention module, and then constructs the attention-optimized visual model search space; A search module determines the search strategy, searches the constructed visual model search space, and obtains an optimized attention power visual model; An evaluation module selects a performance evaluation strategy to evaluate the optimized attention power visual model; The optimization module is repeated until the termination condition is reached to obtain the attention power visual model after attention optimization.

[0088] Example 3 The present invention provides a terminal device, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, graphics processors (GPU), tensor processors (TPU), digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the attention power visual model optimization method, including: Identify the original modules in the existing visual model; convert the obtained original modules into attention modules, and then construct an attention-optimized visual model search space; determine the search strategy, search the constructed visual model search space, and obtain an attention power visual model; select a performance evaluation strategy to evaluate the obtained attention power visual model; repeat the evaluation until the termination condition is reached, and obtain the optimized attention power visual model.

[0089] See also Figure 4, the terminal device is a computer device. The computer device 60 of this embodiment includes: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When the computer program 63 is executed by the processor 61, the attention power visual model optimization method in the embodiment is implemented. To avoid repetition, it is not described one by one here. Alternatively, when the computer program 63 is executed by the processor 61, the functions of each model / unit in the attention power visual model optimization system in the embodiment are implemented. To avoid repetition, it is not described one by one here.

[0090] The computer device 60 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will appreciate that Figure 4 This is only an example of the computer device 60 and does not constitute a limitation of the computer device 60. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, etc.

[0091] The processor 61 may be a central processing unit (CPU), or other general-purpose processors, graphics processing units (GPU), tensor processing units (TPU), digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0092] The memory 62 may be an internal storage unit of the computer device 60, such as a hard disk or memory of the computer device 60. The memory 62 may also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the computer device 60.

[0093] Furthermore, the memory 62 may include both an internal storage unit of the computer device 60 and an external storage device. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 may also be used to temporarily store data that has been output or is to be output.

[0094] See also Figure 5 The terminal device is an electronic device 600, which is in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0095] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above method section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 Follow the steps shown in .

[0096] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0097] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0098] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0099] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0100] Example 4 The present invention also provides a storage medium, specifically a computer-readable storage medium, which is a memory device in a terminal device for storing programs and data. It is understandable that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs. It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory (Non-Volatile Memory), such as at least one disk memory.

[0101] The processor may load and execute one or more instructions stored in a computer-readable storage medium to implement the corresponding steps of the method for optimizing the attention power visual model in the above embodiment; the processor may load and execute the following steps: Identify the original modules in the existing visual model; convert the obtained original modules into attention modules, and then construct an attention-optimized visual model search space; determine the search strategy, search the constructed visual model search space, and obtain an attention power visual model; select a performance evaluation strategy to evaluate the obtained attention power visual model; repeat the evaluation until the termination condition is reached, and obtain the optimized attention power visual model.

[0102] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can usually be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0103] Example 5 See also Figure 2 , using CBAM (Convolutional Block Attention Module) which combines channel attention and spatial attention, and further enhancing the representation ability of convolutional neural networks by applying channel attention and spatial attention successively.

[0104] One implementation of CBAM is as follows: class CBAM(nn.Module): def __init__(self, in_channels, ratio=16, kernel_size=3): super(CBAM, self).__init__() self.channelattention = ChannelAttention(in_channels, ratio=ratio) self.spatialattention = SpatialAttention(kernel_size=kernel_size) def forward(self, x): x = self.channelattention(x) x = self.spatialattention(x) return x After being processed by two consecutive attention modules, the output feature map x is optimized not only in the channel dimension but also in the spatial dimension, allowing the model to focus more on the features that are most useful for the task.

[0105] In this way, the CBAM module enables the network to focus on important channels and spatial regions at the same time, thereby improving the network's perception of key information. When integrated into a convolutional neural network, the CBAM module acts as an effective attention mechanism to enhance the network's ability to recognize useful features, thereby improving the model's performance on a variety of visual tasks. This modular design allows the CBAM module to be easily inserted into existing network structures as a plug-and-play improvement component.

[0106] The design idea of ​​CBAM is to consider both the channel and spatial information of the feature map and enhance the network's perception in this way. In practical applications, CBAM can be embedded into existing convolutional neural networks as a plug-and-play module to improve the performance of the model on various visual tasks, such as image classification, object detection, and semantic segmentation. In this way, the network can obtain better feature expression capabilities while maintaining a low additional computational cost.

[0107] One implementation of ChannelAttention is as follows: class ChannelAttention(nn.Module): def __init__(self, in_channels, ratio=16): super(ChannelAttention, self).__init__() self.avg_pool = nn.AdaptiveAvgPool2d(1) self.max_pool = nn.AdaptiveMaxPool2d(1) self.fc = nn.Sequential( nn.Conv2d(in_channels, in_channels / / ratio, 1, bias=False), nn.ReLU(inplace=True), nn.Conv2d(in_channels / / ratio, in_channels, 1, bias=False) ) self.sigmoid = nn.Sigmoid() def forward(self, x): avg_out = self.fc(self.avg_pool(x)) max_out = self.fc(self.max_pool(x)) out = avg_out + max_out out = self.sigmoid(out) return out * x The ChannelAttention module enables the network to focus more on channel features that are useful for the current task, while weakening those less important features. When integrated into a convolutional neural network, this module can help improve the performance of the model on various visual tasks, such as image classification, object detection, and semantic segmentation.

[0108] Define the PyTorch module of SpatialAttention to implement the spatial attention mechanism.

[0109] One implementation of SpatialAttention is as follows: class SpatialAttention(nn.Module): def __init__(self, kernel_size=7): super(SpatialAttention, self).__init__() assert kernel_size in (3, 7), 'kernel size must be 3 or 7' padding = 3 if kernel_size == 7 else 1 self.conv1 = nn.Conv2d(2, 1, kernel_size, padding=padding, bias=False) self.sigmoid = nn.Sigmoid() def forward(self, x): avg_out = torch.mean(x, dim=1, keepdim=True) max_out, _ = torch.max(x, dim=1, keepdim=True) out = torch.cat([avg_out, max_out], dim=1) out = self.sigmoid(self.conv1(out)) return out * x The ChannelAttention module helps the model focus on the channel features that are most useful for the current task, thereby improving the performance of the model. This module is usually embedded in a convolutional neural network as a plug-and-play enhancement component to improve the network's performance on tasks such as image classification and object detection. In this way, the network can better understand and utilize the relationship and information between different channels to improve the final prediction accuracy.

[0110] Example 6 See also Figure 3 , using a network construction technique based on an improved relaxed continuous network architecture search algorithm suitable for attention. The gradient descent method is used to simultaneously optimize the network architecture parameter α and the weight parameter ω. The optimization of α can improve the accuracy of the final training, and the optimization of ω can accelerate the improvement of the training accuracy; Figure 3 (a) indicates that the search algorithm needs to select candidate operations from the search space. Figure 3 In (b), there are many pre-designed candidate operations in the search space, and different candidate operations are combined by the search strategy. Figure 3 (c) indicates the performance evaluation of the combined network architecture by the evaluation strategy. Figure 3 (d) indicates retaining candidate operations that contribute most to performance improvement, and finally obtaining the target architecture.

[0111] The specific algorithm is as follows:

[0112] However, the search unit of DARTS is Cell, which is Normal-Cell and Reduction-Cell. The same type of Cell structure in the same network is shared, so the network only needs to predict the structure of two Cells. This method is not completely suitable for searching the attention mechanism module. This application improves the algorithm. The specific method is as follows: It is no longer divided into Normal-Cell and Reduction-Cell, but is divided into Cells according to the original modules that can be converted into attention modules. For example, Cells contain optimized modules with only channel attention mechanism, optimized modules with only spatial attention mechanism, optimized modules with spatial channel attention combined mechanism, and modules without attention, etc. The number of cells included is determined according to the scale and computing power of the network. A Cell can share parameters in multiple places in the network. For example, ResNet can share parameters on its BasicBlock or Bottleneck, and VGGNet shares parameters on the VGG block, thereby reducing the search space and improving search efficiency.

[0113] In summary, the present invention provides an attention power visual model optimization method, system, medium and server. By using the attention mechanism, the model can more automatically focus on the important parts of the input data related to the task, and can flexibly adjust the focus according to the needs of the task. However, current convolutional neural networks often do not have attention modules. By adding attention modules, their expression capabilities can be enhanced. However, blindly adding attention modules will cause overfitting problems and even reduce the performance of the model.

[0114] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0115] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0116] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0117] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0118] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0119] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0120] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0121] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0122] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0123] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0124] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A method for optimizing an attention power visual model, characterized in that: The following steps are involved: Identify and analyze primitive modules in existing vision models; By converting some of the original modules into attention modules, we construct an attention-optimized visual model search space. Determine the search strategy, search the constructed visual model search space, and obtain the attention power visual model; Select a performance evaluation strategy to evaluate the obtained attention power visual model; Repeat the above steps until the termination condition is reached to obtain the optimized attention power visual model.

2. The method for optimizing the attention power visual model according to claim 1, characterized in that: The original modules include convolutional modules and / or residual modules and other modules that can be converted to attention mechanisms.

3. The method for optimizing the attention power visual model according to claim 1, characterized in that: Attention optimization includes but is not limited to spatial attention mechanism, channel attention mechanism, residual attention mechanism, multi-scale attention mechanism, and the combination of these attention mechanisms.

4. The method for optimizing the attention power visual model according to claim 1, characterized in that: Search strategies include, but are not limited to, random search strategies, reinforcement learning-based strategies, genetic algorithm-based strategies, Bayesian optimization, and gradient-based strategies.

5. The method for optimizing the attention power visual model according to claim 4, characterized in that: The gradient-based strategy uses the improved DARTS algorithm, specifically: The original modules that can be converted into attention modules are divided into cells, and each cell shares parameters in multiple places in the network.

6. The method for optimizing the attention power visual model according to claim 1, characterized in that: Performance evaluation strategies include but are not limited to multi-objective optimization, single-objective optimization, direct model evaluation, proxy models, approximate evaluation, shared parameters, and parameter-level migration.

7. The method for optimizing the attention power visual model according to claim 1, characterized in that: The termination condition includes but is not limited to reaching the maximum number of iterations.

8. An attention power visual model optimization system, characterized in that: include: Recognition module, identifying and analyzing the original modules in the existing visual model; The conversion module converts part of the original module into an attention module, and then constructs the attention-optimized visual model search space; A search module determines the search strategy, searches the constructed visual model search space, and obtains an optimized attention power visual model; An evaluation module selects a performance evaluation strategy to evaluate the optimized attention power visual model; The optimization module is repeated until the termination condition is reached to obtain the attention power visual model after attention optimization.

9. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform the method of any one of claims 1-7.

10. A server, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include steps for executing the method according to any one of claims 1 to 7.

Images