Neural network architecture searching method based on hierarchical structure information of image component

Abstract

The invention discloses a neural network architecture searching method based on hierarchical structure information of an image component. The method comprises the following steps: step 1, inputting image features to an image component hierarchical structure, and extracting each component feature; step 2, taking the component features as input of a hierarchical information reasoning rule module for establishing reasoning logic of a hierarchical structure, and dynamically weighting the component features of different hierarchies for transmitting hierarchical information between the components; step 3, carrying out weight distribution and optimization on the dynamically weighted component features for modeling an interaction relationship between the components and dynamically adjusting an influence weight; 4, combining a hierarchical information inference rule module and a loss function module, and performing optimization search of the network architecture through an evolutionary algorithm; and 5, accelerating the convergence process of the network. According to the method, the search efficiency of a network architecture, the effectiveness of information transmission and the performance of a training process are improved, and finally better understanding and expression of a complex image scene are realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of neural network architecture search, and in particular relates to a neural network architecture search method based on image component hierarchical structure information. Background Art

[0002] Neural Architecture Search (NAS) mainly includes three steps: 1) defining the search space, 2) executing the search algorithm, and 3) evaluating the candidate network. In recent years, a large number of research results have emerged in the definition of search space and search algorithms, especially the introduction of different methods such as reinforcement learning and evolutionary algorithms, which have significantly improved the efficiency and effectiveness of network architecture search. However, most of the existing NAS methods only focus on learning the overall features of the dataset, while ignoring the hierarchical relationship between the various components in the image.

[0003] Specifically, the existing technical solutions have the following main defects:

[0004] ① Ignoring the hierarchical relationship between image components: Existing NAS methods are mainly based on learning the overall features of the dataset, and perform architecture search in a data-driven manner, usually without considering the hierarchical structure and correlation between the components in the image. For example, for complex images containing multiple components (such as cabins, radars, decks, etc.), the hierarchical relationship and interdependence between these components are often ignored in the expression of the network structure. This will cause the network to be unable to fully capture the contextual associations of these components when processing complex image scenes, thereby affecting the network's recognition effect and reasoning ability.

[0005] ② Inefficiency in the retraining phase: Existing NAS methods have problems with slow convergence and low efficiency in the network retraining phase. Due to the structural differences between different architectures, traditional methods find it difficult to effectively use pre-trained models for initialization, resulting in slow convergence of the network during retraining. In addition, although OhemCELoss in traditional methods can help the model focus on learning samples that are difficult to classify, it has weak processing capabilities for class imbalance problems and is prone to ignoring minority class samples, thereby affecting the model's ability to learn special components in complex image scenes, and thus affecting the performance of the overall network.

[0006] ③ Lack of effective component information transmission mechanism: Traditional NAS methods usually lack the optimization of the information transmission mechanism between components, which makes the information transmission between parent nodes and child nodes insufficient during hierarchical learning of the network, and it is difficult to fully utilize the hierarchical relationship between components to improve the overall expression ability. For example, the contextual information between important components in the image (such as the deck and the shell) and their related components cannot be effectively transmitted, which will limit the performance of the network when processing multi-component and multi-level information.

[0007] ④ Insufficient handling of the problem of class imbalance: In the existing technology, the OhemCELoss loss function is usually used to give priority to samples that are difficult to classify, but OhemCELoss performs poorly in dealing with the problem of class imbalance and is difficult to effectively enhance the learning effect of minority classes. This makes it easy for the model to ignore low-frequency categories in image tasks with obvious differences in the number of categories, affecting the overall classification performance

[0008] Therefore, although existing NAS methods have made some progress in searching network architectures, they still have problems such as inability to effectively utilize component hierarchical relationships, insufficient information transfer, inefficient retraining, and insufficient handling of class imbalance when processing complex image scenes. These shortcomings limit the efficiency of network architecture search and the final performance of the model. Summary of the invention

[0009] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a neural network architecture search method based on image component hierarchical information, aiming to improve the search efficiency of the network architecture, the effectiveness of information transmission and the performance of the training process, and ultimately achieve a better understanding and expression of complex image scenes.

[0010] In order to achieve the above object, the technical solution adopted by the present invention is:

[0011] A neural network architecture search system based on image component hierarchical information, comprising an image component hierarchical structure building module, a hierarchical information inference rule module, a loss function module between component information, an architecture search module and a network retraining module;

[0012] The image component hierarchical structure building module is used to extract and analyze the input image information and build the image component hierarchical structure;

[0013] The image component hierarchical structure is constructed by analyzing each component of the object in the image to construct a tree structure based on the components of the object;

[0014] The hierarchical information reasoning rule module defines and transmits the reasoning relationship information between each component based on the hierarchical structure of the image components; each component refers to the different components contained in the object in the data set;

[0015] The loss function module between component information is used to optimize the architecture search module and the retraining module according to the defined rules during the architecture search and retraining phases;

[0016] The architecture search module and the network retraining module jointly use the rules defined by the hierarchical information inference rule module to gradually find the optimal network architecture and improve the model performance through multiple iterations.

[0017] A neural network architecture search method based on image component hierarchical structure information includes the following steps:

[0018] Step 1: constructing an image component hierarchical structure using the image component hierarchical structure construction module, and extracting features of each image component;

[0019] Step 2: for the hierarchical information reasoning rule module, the component features are used as input of the hierarchical information reasoning rule module, and hierarchical reasoning is implemented in the reasoning rule module through classification scores and feature sharing to dynamically weight component features at different levels, so as to transmit hierarchical information between components;

[0020] Step 3: Use the loss function module between component information to assign and optimize the weights of the dynamically weighted component features to model the interaction between components and dynamically adjust the influence weights;

[0021] Step 4: In the architecture search phase, the hierarchical information inference rule module and the loss function module are combined to perform an optimization search for the network architecture through an evolutionary algorithm;

[0022] Step 5: In the architecture retraining phase, the network convergence process is further accelerated by integrating the improved loss function module and information exchange module.

[0023] In step 1, the image component hierarchy is a tree structure that organizes different components in the image according to importance levels. The image component hierarchy construction module defines the root node and child nodes of the tree structure. The root node represents the most important component or information in the image, and the child node represents the relatively less important component or local feature.

[0024] The specific steps of step 1 are:

[0025] Step (1): Preprocess the complex image, including denoising, normalization and scale adjustment, to extract more detailed and multi-level features;

[0026] Step (2): extract global and local features of the preprocessed image through convolutional neural network (CNN) or other deep learning models (such as ResNet, VGG, etc.);

[0027] Step (3): Use semantic segmentation technology to divide the global and local features of the image into different regions, separate the objects and parts in the image, and assign semantic labels to each region;

[0028] Step (4): Based on the semantic labels, objects and components are organized into a tree structure according to their importance. The root node represents the most core object or scene, and the child nodes represent less important objects or local features.

[0029] Step (5): Dynamically optimize the weights and information flows between different components through an adaptive adjustment mechanism.

[0030] In step 2, the logic rules of the hierarchical information reasoning rule module include: mutually exclusive rules, combination rules and decomposition rules, which are used to describe the relationship between parent and child nodes, between sibling nodes, and between child nodes in the hierarchical structure; the hierarchical information reasoning rule module uses the component information extracted in step 1 for reasoning;

[0031] Among them, the combination rule refers to the reasoning process of the less important components to the important components, and the more complex object or scene structure is derived from the less important components through the relationship with the important components;

[0032] Decomposition rules refer to the process of decomposing important components into less important components, breaking down important objects or areas into smaller, less important parts or details in order to further understand their internal structure or relationships.

[0033] The mutually exclusive rules are:

[0034] At a pixel point, if two or more categories have scores higher than the threshold, the exclusive constraint is strengthened between these high-scoring categories; the modified logic is expressed as follows:

[0035]

[0036] Where: T is the confidence threshold, which is used to determine which categories are considered high confidence; is an indicator function that indicates whether the category score exceeds the threshold;

[0037] Indicates that only pixels containing two or more high-confidence categories are exclusively optimized;

[0038] Loss function of mutually exclusive rule: When there are two or more high confidence categories in a node (pixel position), the mutually exclusive rule is defined as follows:

[0039]

[0040] In this formula:

[0041] s k [v] represents the score of category v on the kth pixel;

[0042] T is the confidence threshold, which is used to determine whether the score is high enough;

[0043] and is the indicator function, ensuring that only those categories with confidence exceeding T are calculated into the exclusive loss;

[0044] The calculation of the mutually exclusive rule loss function is as follows:

[0045]

[0046] Multi-category constraints: The formula introduces similarity constraints between multiple high-confidence categories, and the exclusive loss suppresses the similarities between these high-confidence categories to enhance the distinction between categories;

[0047] High confidence screening: The indicator function ensures that only categories with confidence higher than the threshold T are affected;

[0048] Loss calculation: By minimizing L E value, making the similarities between multiple high-confidence categories smaller for better distinction.

[0049] The combination rules are specifically:

[0050]

[0051] In the inference formula, 1-s k [v] indicates that the lower the confidence value of node v for the kth sample, the more it depends on the influence of its parent node pv. k [pv] represents the confidence of the parent node pv for the kth sample.

[0052] The calculation formula of the loss function of the combination rule is defined as follows:

[0053]

[0054] s k [v] and s k [pv] also represents the confidence of node v and parent node pv for the kth sample; q is the adjustment index used to control the influence between each node;

[0055] The final loss function is written as:

[0056]

[0057] Where |V| is the number of all nodes and |VL| is the number of leaf nodes.

[0058] The decomposition rules are specifically:

[0059] The relationship between the parent node and the child node is combined by introducing (1-MCMB)*MCMA.max(...), where MCMB represents the prediction result of the parent node and MCMA represents the prediction results of multiple child nodes;

[0060] For the combination of multiple nodes, a similar mechanism is used to merge the prediction results of all child nodes into the parent node; for each non-leaf node v∈V\V, the new D-rule loss is:

[0061]

[0062] where GD(v) is given by:

[0063]

[0064] In the formula, the value of node v is adjusted to s_k[v] and is connected to the maximum value of its child node max(s k [cn v ]) q Combine and take the maximum value among the child nodes to modify the parent node;

[0065]

[0066] For k [v] and the maximum value of the child nodes are adjusted differently, introducing a new power operation q, max({s k [cn v ]}) q It is a power operation of the maximum value of the child node, which helps to strengthen the influence of the child node on the parent node or make other nonlinear adjustments;

[0067] s k [v] represents the predicted value of the parent node v, max({s k [cn v ]}) q is the maximum value among the predicted values ​​of the child nodes, and this maximum value is adjusted by the power q. The formula represents the transfer relationship between the parent node and the child node, which is specifically calculated by the value s of the parent node. k The reasoning is performed by combining [v] with the maximum value of the child nodes.

[0068] The step 3 is specifically as follows:

[0069] The loss function is designed based on mutually exclusive rules, combination rules and decomposition rules. The loss functions constructed by these three rules are integrated into the optimization process of framework search, and the framework parameters β and α are optimized in each round of training to dynamically adjust the structure and performance of the network.

[0070] In the framework search process, β and α represent two key optimization parameters, β: controls the selection strategy of cells (i.e., neural network units) in the framework, affecting the network structure during the search process;

[0071] α: controls the importance of selected operations (such as convolution, pooling, etc.) in the search process and determines the selection weight of each operation type in the network architecture. Based on the dynamic optimization of these two parameters, the framework search adjusts the selection of the network architecture through each round of training. At the same time, the loss function provides guidance for weight adjustment and inference optimization in the optimization process.

[0072] The specific steps are:

[0073] (1) Initialize the framework parameters β and α. β controls the strategy of selecting different cells. During the framework search process, β affects which cells are selected and form the basic modules of the network architecture. α controls the weights of selecting different operation types, which plays a decisive role in selecting the appropriate operation structure and adjusts the selection and combination strategy of the operation.

[0074] (2) Calculation and adjustment of the loss function: Based on the loss function in step 3, in each round of training, the loss function will calculate the error of each component based on the current network architecture;

[0075] (3) Calculate the loss function and adjust β and α: Through back propagation, the network calculates the error of each component according to the current loss function and gradually adjusts β and α so that the network architecture can better adapt to the relationship between components in the image;

[0076] Optimize β: β controls the strategy for selecting cells in the frame search. If the currently selected cell performs poorly when processing the image, you need to adjust β to select a more suitable cell.

[0077] Optimize α: α controls the weight of selecting different operations in the framework search. The current operation type may have a greater impact on some inference rules. If an operation performs poorly, a more suitable operation can be selected by adjusting α;

[0078] (4) Update network parameters and architecture: At the end of each round of training, the gradient of the loss function is calculated through back propagation, and the network parameters (such as weights and biases) are updated. At the same time, the structure and operation type of the network architecture are readjusted according to the optimized β and α. Through multiple rounds of training, the model gradually converges and the loss function gradually decreases. In each round of training, the framework search optimizes β and α based on the performance of the current network.

[0079] (5) Convergence and final model architecture: As training progresses, framework search and dynamic adjustment of the loss function will cause the model to eventually converge to an optimized network architecture.

[0080] The step 4 is specifically as follows:

[0081] (1) Initialize the initial population for architecture search; each individual in the population represents a potential network architecture;

[0082] (2) Define the loss function module and loss calculation; design the loss function based on three inference rules;

[0083] Mutually exclusive rule loss (L e ): Ensure the discrimination between mutually exclusive categories and avoid the model from mistakenly predicting multiple high-confidence categories at the same time;

[0084] Combination rule loss (L C ): handles the inference relationship between parent and child nodes;

[0085] Decomposition rule loss (L d ): describes how to decompose a complex object into multiple simple parts and optimize the reasoning of the parent node based on the information of the child node. The final loss function combination is:

[0086] L total =L e +L c +L d

[0087] These loss function modules will be dynamically optimized during the training process to guide the selection and optimization of the network architecture.

[0088] (3) Dynamically optimize β and α: In each round of training, by calculating the loss function L total , after back-propagation and updating the network weights, the β and α parameters are dynamically adjusted;

[0089] The update formula is:

[0090]

[0091] Where η is the learning rate, and is the gradient of the loss function with respect to β and α;

[0092] (4) In the conventional neural network training process, back propagation calculates the gradient of each parameter and updates it;

[0093] (5) Selection, crossover, and mutation selection of evolutionary algorithms: Individuals (i.e., network architectures) with higher fitness in the current population, which perform better in optimizing loss functions and in the inference process, will be selected to enter the next generation and form the parent generation. Through the crossover operation, some features of the selected parent generation architectures are combined into a new architecture.

[0094] (6) Iteration and convergence: Through multiple generations of evolution, the network architecture will gradually converge to an optimal solution.

[0095] The step (4) is specifically as follows:

[0096] After the back-propagation is completed, the feature vector of each component is introduced into the hierarchical information inference rule module for calculation;

[0097] Feature vector extraction: The feature vectors generated by each component will be passed to the hierarchical information inference rule module. These feature vectors represent the output of each component in the network during the forward propagation process.

[0098] Inference rule module: The hierarchical information inference rule module is responsible for further calculation of the feature vectors of each component based on three inference rules;

[0099] Mutual Exclusion Rule: Ensure that at each pixel location, there is enough difference between multiple high-confidence categories to avoid misclassification;

[0100] Combination rules: According to the relationship between parent nodes and child nodes, adjust the feature vector and reasoning process to ensure that the child node can accurately reflect the characteristics of the parent node;

[0101] Decomposition rules: Optimize the decomposition and reasoning process of each object by adjusting the maximum score of the child nodes and the characteristics of the parent nodes;

[0102] Calculate new feature vectors and segmentation scores: Update the feature vectors and corresponding segmentation scores of each component based on the calculation of the hierarchical information inference rule module. The new feature vectors will help the model better understand the dependencies between components, further adjust the weights in the loss function, and optimize the network architecture.

[0103] The step 5 is specifically as follows:

[0104] The AsymmetricLossOptimized loss function will be introduced into the network retraining phase;

[0105] Loss function formula:

[0106] L asym (p,y)=-α(y)·(1-p) γ ·log(p)

[0107] in:

[0108] α(y) is the weight associated with category y, which specifically provides higher weights for low-frequency categories and reduces the influence of high-frequency categories;

[0109] (1-p) γ It is a regulation factor used to increase the loss weight of low-confidence positive samples, so that the network pays more attention to samples that are difficult to classify;

[0110] γ is a hyperparameter for adjusting the difficulty of samples. A larger value makes the network pay more attention to low-confidence samples.

[0111] During the retraining process, the AsymmetricLossOptimized loss function adjusts the weights to ensure that the network can balance the processing of high-frequency and low-frequency categories, thereby improving the learning effect of low-frequency categories.

[0112] Beneficial effects of the present invention:

[0113] The present invention clarifies the tree relationship between object components by constructing a hierarchical structure of image components, setting important components as root nodes and other components as child nodes or leaf nodes. This approach allows the correlation between components to be effectively captured by the network structure, especially when processing complex image tasks, it can better understand and express the complex relationship between components. Compared with the prior art method of performing architecture search based only on the overall characteristics of the data set, the hierarchical modeling proposed in the present invention enables the network to have better adaptability to the contextual information between components, thereby improving the expressiveness of the model in complex scenarios.

[0114] The present invention designs mutually exclusive rules, combination rules and decomposition rules, and considers the different relationships between components in more detail in the reasoning logic. Through these rules, the information between parent-child nodes and brother nodes can be dynamically adjusted and effectively transmitted, which plays a key role in the hierarchical modeling of complex components.

[0115] Compared with existing methods, traditional NAS usually lacks dynamic modeling of component hierarchical relationships, while the reasoning logic rules of the present invention can better capture these relationships, optimize the information transmission path of the model in architecture search and reasoning, and significantly enhance the performance of the model.

[0116] The present invention introduces the AsymmetricLossOptimized loss function in the network retraining stage. The AsymmetricLossOptimized loss function will be introduced in the network retraining stage to replace the traditional OhemCELoss, specifically to deal with the problem of class imbalance. By asymmetric adjustment of positive and negative samples of different categories, the loss function effectively reduces the neglect of low-frequency categories and enhances the influence of low-frequency categories on network training.

[0117] During the retraining process, the AsymmetricLossOptimized loss function adjusts the weights to ensure that the network can balance the processing of high-frequency and low-frequency categories, thereby improving the learning effect of low-frequency categories.

[0118] In the retraining stage, the present invention significantly improves the performance in dealing with the class imbalance problem, greatly enhancing the network's learning ability for minority classes, and ultimately effectively improving the overall classification performance. In addition, in the retraining process of the present invention, the loss function decreases faster and the network convergence process is more efficient.

[0119] In the architecture search stage, the present invention integrates an improved loss function module and a hierarchical information reasoning module, so that the search of the network architecture is not only based on the overall characteristics of the data set, but also can effectively capture and utilize the hierarchical information and mutual influence relationship between each component. In this way, the adaptability of the network architecture is greatly improved, and the search efficiency is also significantly improved.

[0120] During the retraining phase, by further integrating mutually exclusive rules, combination rules, and decomposition rules, the network convergence speed is significantly accelerated and the training efficiency is improved, especially in the absence of a pre-trained model, which significantly reduces the time and computing resources required for training.

[0121] The multiple-level information transfer rules and improved loss functions proposed in this invention enable the network to show significant advantages when processing complex image scenes (such as images containing multiple object parts). Through these improvements, the network's ability to capture complex dependencies and interactions between components is significantly improved, and it shows higher accuracy and generalization ability in image recognition and reasoning.

[0122] In summary, the present invention significantly solves the shortcomings of existing NAS methods in capturing complex image component relationships, handling category imbalance problems, insufficient information transfer, and low efficiency in the retraining phase by modeling the hierarchical structure of image components, introducing new reasoning logic rules, and improving the loss function module. Ultimately, the technical solution of the present invention can greatly improve the efficiency of neural network architecture search and retraining, reduce the time and computing cost of network training, and achieve better recognition and reasoning effects in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0123] Figure 1 It is the overall framework diagram of the present invention.

[0124] Figure 2 A schematic diagram of the hierarchical structure of image components of the present invention.

[0125] Figure 3 It is a schematic diagram of the hierarchical information inference rule module of the present invention.

[0126] Figure 4 Schematic diagram of the loss function module between component information of the present invention. DETAILED DESCRIPTION

[0127] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0128] like Figure 1As shown, a neural network architecture search system based on image component hierarchical information is divided into five modules: image component hierarchical structure construction module, hierarchical information inference rule module, loss function module between component information, architecture search module and network retraining module. Each module complements each other and constitutes the overall framework of the neural network architecture search method proposed in the present invention.

[0129] Image component hierarchy building module: The task of this module is to extract the information of each component and build a hierarchical component structure based on the features of the input image.

[0130] In the attached figure, it can be seen that the construction of the image component hierarchy includes defining root nodes and child nodes. For example, important components such as decks and hulls are used as root nodes, and other components such as masts and cabins are used as child nodes or leaf nodes to form a tree-like hierarchical relationship.

[0131] In the image component hierarchy construction stage, in addition to using a tree hierarchy, a method based on a graph neural network (GNN) can also be used to model components. In this alternative, the components of the image can be represented as nodes of a graph, and the relationships between components can be represented by edges in the graph. Using GNN to model hierarchical relationships can more flexibly describe complex dependencies between components while retaining hierarchical information.

[0132] The arrows in the figure mark the process from image to structured representation, showing how the component hierarchy of an object is built layer by layer.

[0133] The hierarchical structure of image components refers to the construction of a tree structure based on the components of an object by analyzing the various components of the object in the image. Specifically, this structure groups the components of an object hierarchically according to their importance in representing the object. The important components serve as the root nodes of the tree structure, and the less important components serve as child nodes, forming a hierarchical structure. This hierarchical structure reflects the pattern of how the components of an object are composed and related to each other.

[0134] Build steps:

[0135] 1. Analyze the component features of the object: By extracting and analyzing the components of each object in the data set, the characteristics of each component are identified.

[0136] 2. Distinguish between important parts and less important parts: Classify each part according to its importance in the object structure. Important parts are generally the core parts of the object, while less important parts are supporting or additional parts.

[0137] 3. Build a tree structure: Take important components as root nodes and less important components as child nodes, and form a tree structure according to the hierarchical relationship to represent the component hierarchy of the object.

[0138] Specifically, the degree of importance can be determined through the following aspects:

[0139] General perception: For example, for an elephant, people generally believe that the trunk (trunk) and ears of an elephant are key components to determine whether an elephant exists. Through these salient features, people can easily identify an elephant. Therefore, in the hierarchical structure of image components, the trunk and ears may be classified as important components and used as the root nodes of the tree structure.

[0140] Expert knowledge: In addition to general knowledge, the professional understanding of an object by experts (such as biologists, zoologists, etc.) will also affect the importance assessment of parts. For example, for an elephant, the trunk and ears are considered to be key parts closely related to its identity characteristics. Expert knowledge may also include structural relationships or functional analysis of other parts, such as the legs and body of an elephant, which may occupy a lower level in the tree structure.

[0141] Further explanation:

[0142] Priority of trunk and ears: The trunk and ears of elephants are not only very significant in appearance, but they also play a very important role in the ecology and behavior of elephants. The trunk is used for breathing, smelling and grasping objects, while the ears play a vital role in regulating body temperature and communication. Therefore, their features may occupy a higher level in the structure and become the root node in the tree structure.

[0143] The composition of the hierarchical structure: the root node is the most representative and recognizable part of the object, while the child nodes below it represent the parts that are less important but still help to fully identify the object. For example, with the elephant's head determined, the next parts may be its eyes, mouth, etc., forming further hierarchical branches.

[0144] This knowledge-driven hierarchical structure reflects how humans analyze images through cognition and experience to identify the structure of objects and the relationship between their various parts. This approach has great potential for achieving more accurate image understanding, especially in the fields of computer vision and automated recognition.

[0145] Hierarchical information reasoning rule module: This module mainly involves the reasoning logic for establishing hierarchical relationships, which is used to transmit hierarchical information between components.

[0146] The hierarchical information reasoning rule module defines and transmits the reasoning relationship information between the components of the object based on the hierarchical structure of the image components. The components here refer to the different parts contained in the object in the data set, and these parts are organized according to the construction of the hierarchical structure graph. In the reasoning rule module, hierarchical reasoning is achieved through the information exchange of classification scores between components, that is, the accuracy and relevance of component classification are improved through the information exchange between segmentation features. Component definition: refers to the various parts of the object, which are derived from the component features after the object is segmented in the data set. Information exchange: In the reasoning rule module, different components exchange scores and share features through hierarchical graphs, which strengthens the relevance between segmentation features.

[0147] Purpose: To improve the accuracy and consistency of component classification and identification in hierarchical structures through information interaction and reasoning.

[0148] In the attached figure, the design of the module is based on three main logic rules: mutual exclusion rule, combination rule and decomposition rule. These rules are used to describe the relationship between parent and child nodes, between sibling nodes, and between child nodes in the hierarchy. Through the reasoning logic module, it is ensured that during the architecture search phase, the network can fully understand the correlation between components and appropriately adjust the model structure.

[0149] Mutually exclusive rules, combination rules, and decomposition rules can be replaced by other types of logical rules. For example, a relational modeling module based on a self-attention mechanism can be introduced, which can describe the mutual relationship between components through attention scores, thereby adjusting the weight and information transfer of each component in reasoning. Unlike existing logical rules, the scheme based on the self-attention mechanism can adaptively capture the interaction relationship between components and achieve more fine-grained adjustments.

[0150] The mutual exclusion rule can also be replaced by an entropy-based relationship adjustment method, which uses entropy to measure the confidence distribution of different nodes, ensure the differences between high-confidence categories, and reduce the impact on low-confidence nodes.

[0151] Loss function module between component information: This module designs a loss function suitable for the component hierarchy, with the aim of modeling the interaction relationship between components and dynamically adjusting the influence weights.

[0152] The accompanying figure shows how the module models the interactions between different components and adjusts the weights between components according to corresponding rules, so that the model can better understand these complex relationships during training.

[0153] Another alternative is to adopt a sample re-weighting strategy, that is, to assign different loss weights to each type of sample, so that the minority class samples account for a larger proportion of the overall loss, thereby improving the network's learning effect on the minority class. This sample weighting method can be used in conjunction with any standard loss function to achieve similar purposes. Architecture Search Module: This module is responsible for guiding the optimization search of the network architecture during the architecture search phase by combining the hierarchical information inference rule module with the loss function module.

[0154] The arrows in the figure indicate the process of architecture search, explaining how to drive network architecture search by integrating the information of the above modules and ultimately find a network structure suitable for complex image scenes.

[0155] In the architecture search stage, the guidance of the improved loss function module and the hierarchical information exchange module ensures that the search process not only focuses on the overall characteristics of the dataset, but also effectively captures and utilizes the hierarchical relationships and mutual influences of various components in the image.

[0156] In the network architecture search phase, a search method based on evolutionary algorithms can be used to replace the original data-driven and logical rule-based approach. Evolutionary algorithms gradually optimize the network structure by simulating the process of natural selection. In particular, after defining the hierarchical relationship between components, evolutionary algorithms can use this information as part of fitness evaluation to select a suitable architecture.

[0157] It is also possible to use a policy gradient method based on reinforcement learning to optimize the architecture search by defining a reward function that takes into account the component hierarchy information, thereby better preserving the hierarchical relationship during the optimization process.

[0158] Network retraining module: By further training the network generated in the architecture search phase, an improved loss function module and an information exchange module are further integrated in this phase to accelerate network convergence.

[0159] The process in the attached figure also illustrates how the retraining module uses mutually exclusive rules, decomposition rules, and combination rules to optimize the training performance of the model.

[0160] The retraining process also significantly improved the performance and generalization ability of the model by introducing mutually exclusive rules (allowing multiple categories to have high confidence), decomposition rules (controlling the dependencies between parent and child nodes through power operations), and combination rules (through flexible modeling of parent and child nodes).

[0161] In the network retraining phase, the initialization of the model can be replaced with the weights obtained by transfer learning, thereby accelerating the convergence process of the model. Specifically, a model pre-trained on a similar task can be used for weight transfer, and the hierarchical relationship information of the present invention can be combined to further fine-tune the model.

[0162] Another alternative is to introduce a learning rate adaptive adjustment mechanism in the retraining phase, such as based on the LAMB optimizer or the AdaBound optimizer. These optimizers can dynamically adjust the learning rate, combined with the improved loss function and the hierarchical information exchange module, making the retraining process more stable and efficient, reducing the number of iterations and improving the overall training efficiency.

[0163] In this application, "important components" and "other components" are terms for classifying different components (such as objects, regions, backgrounds, etc.) in the image based on their importance, influence, and reasoning priority in the image in the image understanding task. The distinction between these components helps the model prioritize and optimize the most helpful parts for the task in complex image scenes, while suppressing or reducing attention to less important parts, thereby improving the overall reasoning effect and efficiency. Important components refer to components that have a key impact on the target task during image understanding and reasoning. According to the contextual information, semantic relationships, and physical structures in the image, important components are usually those: ① with high semantic importance: the main objects that need to be identified in the target task, which are directly related to the task objectives; ② components that need to be reasoned first: in multi-component images, some components may dominate the reasoning process or have a greater impact on the final classification, recognition, segmentation, and other tasks. For example, "wheels" may be the most critical component when identifying "cars", while "windows" may have less impact. ③ Core components in a hierarchical structure: In the hierarchical structure of an image, important components are often root nodes or top-level nodes, representing the main objects, objects, or key areas of the image. Other components refer to relatively less important components, usually those that have little impact on the final output of the task in the image, or are relatively minor elements in the background or environment. These components have a lower reasoning priority, but can still provide background information or context for overall image understanding. ① With lower semantic importance: For example, in complex scenes, the background or certain minor objects (such as the road surface, skyline, etc.) are usually considered to be minor components because they do not directly affect the recognition of core objects. ② Components that have little impact on task reasoning: For example, some background areas or details, although important to the overall image, have a minor impact in specific tasks (such as object detection or semantic segmentation). ③ Sub-nodes in a hierarchical structure: In a hierarchical representation, other components are usually sub-nodes, representing more local or detailed parts of the image. Their role is more auxiliary. Although they can provide information under certain conditions, they are not the core of reasoning.

[0164] A neural network architecture search method based on image component hierarchical structure information includes the following steps:

[0165] Step 1: Input the image features into the image component hierarchy, and the image component hierarchy extracts the features of each component;

[0166] Step 2: construct a hierarchical information reasoning rule module, use the component features as input of the hierarchical information reasoning rule module, establish hierarchical reasoning logic, dynamically weight component features at different levels, and transfer hierarchical information between components;

[0167] Step 3: Construct a loss function module between component information, assign and optimize weights for dynamically weighted component features, and use it to model the interaction between components and dynamically adjust the influence weights;

[0168] Step 4: In the architecture search phase, the hierarchical information inference rule module and the loss function module are combined to perform an optimization search for the network architecture through an evolutionary algorithm;

[0169] Step 5: During the architecture training phase, the network convergence process is further accelerated by integrating the improved loss function module and information exchange module.

[0170] In step 1, when the input image is a complex image, it is first necessary to preprocess the input complex image to extract more detailed and multi-level features;

[0171] By introducing a multi-scale convolutional neural network (CNN) to capture global and local information in the image, it is used to process the diverse information contained in complex image scenes (such as background, relationships between objects, and image features of different scales); through this method, the network can effectively process multiple scale information in complex scenes, thereby enhancing the understanding and expression of complex images.

[0172] Step 1 provides refined component features for the subsequent inference rule module (step 2), which will help the model understand the hierarchical relationship between different components during the inference process.

[0173] In step 2, the hierarchical information reasoning rule module defines and transmits the reasoning relationship information between components based on the hierarchical structure of image components. Components refer to the different components contained in the objects in the data set, and these components are organized based on the hierarchical structure. The function is to use component features as input and realize hierarchical reasoning through classification scores and feature sharing in the reasoning rule module. Information exchange is the dynamic weighting and information transmission of segmentation features between components to strengthen feature relevance. The purpose is to improve the accuracy and consistency of component classification through reasoning and information interaction.

[0174] When constructing the hierarchical information inference rule module, considering the component relationships and hierarchical complexity in complex images, a reasoning rule is designed by introducing domain knowledge to describe the hierarchical relationship between different components in the image (such as objects, scenes, backgrounds, etc.). The reasoning rule module not only has dynamic reasoning capabilities, but also can adaptively adjust the reasoning path according to the image component features. The design combines domain knowledge. The reasoning rule module combines the features of the components in the image and incorporates knowledge in specific fields (such as the combination relationship between object components and their importance levels) to establish a hierarchical reasoning mechanism. This domain knowledge can be input through external resources such as knowledge graphs to identify which components are more important in specific scenes. The component features extracted in step 1 provide input to the reasoning rule module, enabling it to effectively model the hierarchical relationship between components in the image. This module will dynamically weight components at different levels in the image, thereby more accurately processing scene information in complex images.

[0175] In step 3, in order to solve the complex interaction and dynamic influence between components in complex image scenes, a loss function module with adaptability and hierarchical structure is designed. This module will adaptively adjust the weights in the loss function according to the relative importance of each component in the image (such as foreground objects, backgrounds, and relationships between objects), thereby improving the performance of the model when processing complex images. The loss function not only includes traditional pixel-level losses, but also contextual information constraints and structured losses can be added to model the relationship between objects. By dynamically optimizing the loss function, the model can flexibly adjust the influence of different components during the training process according to the interaction and importance levels between components. The inference rules designed in step 2 provide a dynamic weight mechanism for the loss function module, so that the loss function can be effectively weighted and optimized according to the priority and importance of different components. In this way, the network can focus on optimizing important components during training.

[0176] In step 4, in the architecture search phase, the hierarchical information reasoning rule module and the loss function module are combined to optimize the network architecture through the evolutionary algorithm, with special attention paid to the scene information and structural characteristics of complex images. The evolutionary algorithm automatically searches for the optimal network architecture by simulating the process of natural selection (such as selection, crossover, mutation, etc.), especially in complex scenes with multi-level component interactions, by dynamically adjusting the components and information flow paths in the architecture to adapt to the needs of complex images. The network architecture design is not only based on traditional performance evaluation, but also specifically considers how to better process the component information at different levels in the image, so that key components (such as core objects, important scene elements) can obtain higher weights in the reasoning process, while the influence of background or secondary objects is controlled. Through the architecture search process, the network architecture dynamically adjusts the architecture and the interaction path between components according to the loss function module and reasoning rules defined in step 3. This hierarchical optimization can ensure that the network architecture can cope with the hierarchical relationship and diversified scene expression in complex images.

[0177] In the step 5, during the architecture training phase, the convergence process of the network is further accelerated by integrating the improved loss function module and the information exchange module, especially when processing complex image scenes, the network needs to quickly adapt to the complex relationship between different components. The design of the information exchange module at this stage is particularly important. It can ensure that the model can more effectively integrate information at different levels by dynamically adjusting the information flow between different components, thereby accelerating convergence and improving the ability to express complex images. The information exchange module enables the network to share knowledge of different tasks while processing multiple tasks through a multi-task learning mechanism, promoting the rapid convergence of the network. In addition, by strengthening the dynamic exchange mechanism of different components in the image, the network's understanding and expression capabilities of complex image scenes are further enhanced. The optimized network architecture designed in step 4 provides an effective framework for subsequent training, and the information exchange module ensures that the network can quickly adapt to the hierarchical features in complex images during training, further accelerating the convergence of the model.

[0178] In step 1, the image component hierarchy is a tree structure that organizes different components in the image according to the level of importance. The image component hierarchy construction module includes defining a root node and child nodes. The root node represents the most important component or information in the image, while the child node represents a relatively less important component or local feature. Through this tree structure, the network can understand the relationship between the various components in the image and help reason and optimize the task of processing complex images. The root node is the node in the image that best reflects the core features of the object or the component of the object, representing the key component of the object or the key features adjacent to the core component of the object; it is used to describe the main characteristics of the object or the information of other components that are most closely related to the object;

[0179] Subnodes are secondary component information, representing important secondary features of the object or other components adjacent to the primary component represented by the root node;

[0180] Sub-nodes provide the network with secondary features of the object or detailed information associated with the core components. This hierarchical structure can help the network better understand the dependencies and importance between the parts within the object, thereby optimizing the reasoning and feature extraction process.

[0181] The specific steps of step 1 are:

[0182] First, the complex image is preprocessed, including denoising, normalization and scale adjustment, so as to better process the details in the image and extract more detailed and multi-level features;

[0183] Next, the global and local features of the image are extracted through a convolutional neural network (CNN) or other deep learning models (such as ResNet, VGG, etc.);

[0184] On this basis, semantic segmentation technology is used to divide the image into different regions, accurately separate the objects and parts in the image, and assign semantic labels to each region;

[0185] Then, according to the semantic and spatial relationships in the image, the components in the image are organized into a tree structure according to their importance. The root node usually represents the most core object or scene, while the child nodes represent less important objects or local features.

[0186] Finally, the weights and information flows between different components are dynamically optimized through an adaptive adjustment mechanism, ensuring that important components receive more attention during the reasoning process. This hierarchical structure helps the network understand the overall and local information of the image, thereby improving the accuracy of reasoning and processing in subsequent tasks.

[0187] In step 2, the hierarchical information reasoning rule module is based on three main logical rules: mutual exclusion rule, combination rule and decomposition rule; in the hierarchical structure, it is used to describe the relationship between parent and child nodes, between sibling nodes, and between child nodes;

[0188] In step 2, the hierarchical information reasoning rule module is based on three main logical rules: mutual exclusion rules, combination rules, and decomposition rules, which help the model understand the relationship between different parts in the image. The image features and hierarchical structure extracted in step 1 provide the basis for the application of these rules. In step 1, the image is organized into a hierarchical structure through semantic segmentation and deep feature extraction, where the root node represents the most important component and the child nodes represent the less important components. Through this hierarchical structure, each component and region in the image is clearly identified and organized to form a clear knowledge graph.

[0189] The inference rule module uses the component information extracted in step 1 to perform inference;

[0190] Combination rules refer to the reasoning process of less important components to important components, inferring more complex object or scene structures from less important components through their relationship with important components; for example, some less important local features or details (such as "wheels") may help to infer the overall structure of an object (such as "car").

[0191] Decomposition rules refer to the process of decomposing important components into less important components, decomposing important objects or regions into smaller, less important parts or details in order to further understand their internal structure or relationships. For example, a core object (such as "car") can be decomposed into multiple smaller components (such as "windows" or "wheels") for more detailed reasoning and analysis. Therefore, a close connection is formed between step 1 and step 2. The former provides a clear feature and relationship framework for the latter through a hierarchical component structure, and the latter uses this information to further enhance the understanding and analysis of the image through reasoning rules.

[0192] (1) The relationship between parent and child nodes reflects the hierarchical structure of the components in the image. The parent node represents the more important component or global information, while the child node represents the less important component or detail information related to the parent node. In step 1, through semantic segmentation and feature extraction, the image is decomposed into multiple components and hierarchical, forming a parent-child node relationship. The combination rules and decomposition rules in step 2 depend on the hierarchical structure between these parent-child nodes.

[0193] (2) Sibling nodes refer to different components at the same level, which usually describe different objects or regions in the image. In step 1, different objects or regions are organized into sibling nodes based on semantic labels and spatial features. For example, “pedestrian” and “vehicle” can be sibling nodes at the same level, representing different object categories.

[0194] (3) The relationship between sub-nodes describes the connection between the various components or details of the object in the image. For example, in a complex object (such as a "car"), the wheels, windows, body, etc. are different sub-nodes of the object. In step 1, these sub-nodes are separated from the image through semantic segmentation and feature extraction to form a multi-level feature representation.

[0195] The inference rules in step 2 (mutually exclusive rules, combination rules, and decomposition rules) help the model understand the spatial and semantic relationships between the various components in the image through the relationships between parent-child nodes, sibling nodes, and child nodes. These rules are closely related to the image features extracted in step 1, which hierarchically organizes the components and assemblies of the image by importance and provides the feature information required for reasoning. The inference rule module uses these hierarchical component features to help the model deduce the relationship between the components, thereby optimizing the image understanding and reasoning process.

[0196] The reasoning logic module ensures that during the architecture search phase, the network can fully understand the correlation between components and appropriately adjust the model structure. The mutual exclusion rules are:

[0197] At a pixel point, if two or more categories have scores higher than the threshold, the exclusive constraint is strengthened between these high-scoring categories; the modified logic is expressed as follows:

[0198]

[0199] Where: T is the confidence threshold (such as e.6), which is used to determine which categories are considered to have high confidence; is an indicator function that indicates whether the category score exceeds the threshold;

[0200] Indicates that only pixels containing two or more high-confidence categories are exclusively optimized;

[0201] Loss function of mutually exclusive rule: When there are two or more high confidence categories in a node (pixel position), the mutually exclusive rule is defined as follows:

[0202]

[0203] In this formula:

[0204] s k [v] represents the score of category v at the kth pixel.

[0205] T is the confidence threshold, which is used to judge whether the score is high enough.

[0206] and is an indicator function that ensures that only those categories with confidence exceeding T are calculated into the exclusive loss.

[0207] The calculation of the entire mutually exclusive rule loss function is as follows:

[0208]

[0209] Multi-category constraints: The formula introduces similarity constraints between multiple high-confidence categories, and the exclusive loss suppresses the similarities between these high-confidence categories to enhance the distinction between categories;

[0210] High confidence screening: The indicator function ensures that only categories with confidence higher than the threshold T are affected;

[0211] Loss calculation: By minimizing L E value, making multiple high-confidence categories less similar to each other in order to better distinguish them.

[0212] The combination rules are specifically:

[0213]

[0214] In the inference formula, 1-s k [v] indicates that the lower the confidence value of node v for the kth sample, the more it depends on the influence of its parent node pv. k [pv] represents the confidence of the parent node pv for the kth sample;

[0215] In this way, when the confidence of node v is low, the information of parent node pv can play a greater role in the reasoning process.

[0216] The calculation formula of the loss function of the combination rule is defined as follows:

[0217]

[0218] s k [v] and s k [pv] also represents the confidence of node v and parent node pv for the kth sample.

[0219] q is the adjustment index used to control the degree of influence between each node.

[0220] Combined with the above calculation method, the final loss function is written as:

[0221]

[0222] Where |V| is the number of all nodes and |VL| is the number of leaf nodes.

[0223] The decomposition rules are specifically:

[0224] The relationship between the parent node and the child node is combined by introducing (1-MCMB)*MCMA.max(...), where MCMB represents the prediction result of the parent node and MCMA represents the prediction results of multiple child nodes;

[0225] For the combination of multiple nodes (such as the hull, thruster, and bridge), a similar mechanism is used to merge the prediction results of all child nodes into the parent node, further strengthening the confidence information fusion of multiple paths;

[0226] For each non-leaf node v∈V\V, the new D-rule loss is:

[0227]

[0228] where GD(v) is given by:

[0229]

[0230] In the formula, the value of node v is adjusted to s_k[v] and is connected to the maximum value of its child node max(s k [cn v ]) q Combine and take the maximum value among the child nodes to modify the parent node.

[0231]

[0232] In this formula, for s k [v] and the maximum value of the child nodes are adjusted differently, introducing a new power operation q, max({s k [cn v ]}) q It is a power operation of the maximum value of the child node, which helps to strengthen the influence of the child node on the parent node or make other nonlinear adjustments;

[0233] s k [v] represents the predicted value of the parent node v, max({s k [cn v ]}) q is the maximum value among the predicted values ​​of the child nodes, and this maximum value is adjusted by the power q. The formula represents the transfer relationship between the parent node and the child node, which is specifically calculated by the value s of the parent node. k The reasoning is performed by combining [v] with the maximum value of the child nodes.

[0234] The step 3 is specifically as follows:

[0235] The loss function is designed based on mutually exclusive rules, combination rules and decomposition rules. The loss functions constructed by these three rules are integrated into the optimization process of framework search, and the framework parameters β and α are optimized in each round of training to dynamically adjust the structure and performance of the network.

[0236] In the framework search process, β and α represent two key optimization parameters, β: controls the selection strategy of cells (i.e., neural network units) in the framework, affecting the network structure during the search process;

[0237] α: controls the importance of selected operations (such as convolution, pooling, etc.) in the search process and determines the selection weight of each operation type in the network architecture. Based on the dynamic optimization of these two parameters, the framework search adjusts the selection of the network architecture through each round of training. At the same time, the loss function (based on three inference rules: mutually exclusive rules, combination rules, and decomposition rules) provides guidance for weight adjustment and inference optimization in the optimization process.

[0238] The specific steps are:

[0239] (1) Initialize framework parameters β and α: At the beginning of training, the initial values ​​of β and α can be set by experience, random initialization, or based on previous training experience. These parameters determine how to select appropriate network units (cells) and operation types during the framework search process: β: controls the strategy for selecting different cells, which can be convolution blocks, pooling layers, fully connected layers, etc. During the framework search process, β affects which cells are selected and form the basic modules of the network architecture. α: controls the weights for selecting different operation types (such as convolution operations, pooling operations, etc.), which plays a decisive role in selecting the appropriate operation structure and adjusts the selection and combination strategy of operations.

[0240] (2) Calculation and adjustment of the loss function: According to the loss function design in step 3 (based on three inference rules: mutual exclusion rule, combination rule and decomposition rule), in each round of training, the loss function will calculate the error of each component according to the current network architecture.

[0241] (3) Calculate the loss function and adjust β and α: Through back propagation, the network calculates the error of each component according to the current loss function, and gradually adjusts β and α so that the network architecture can better adapt to the component relationship in the image. Optimize β: β controls the strategy of selecting cells in the framework search. If the currently selected cell performs poorly when processing the image (for example, a cell causes a large loss of the mutual exclusion rule, or the influence of the child node is not fully inferred), it is necessary to adjust β to select a more suitable cell. In this way, the model can dynamically select a better cell structure in each round of training. In the loss function, a higher β value will make the framework search more inclined to select a specific type of cell (such as a convolutional layer), thereby optimizing the inference result and reducing the loss. Optimize α: α controls the weight of selecting different operations (such as convolution, pooling, etc.) in the framework search. The current operation type may have a greater impact on certain inference rules (such as combination rules or decomposition rules). If a certain operation performs poorly (for example, the influence of parent and child nodes is not effectively fused during the inference process), a more suitable operation is selected by adjusting α. In the loss function, a higher value of α will increase the impact of specific operations, for example, increasing the influence of parent nodes on child nodes (such as convolution operations). By optimizing the selection of operations, the model can adjust the predictions of each component more accurately and reduce the decomposition rule loss.

[0242] (4) Update network parameters and architecture: At the end of each round of training, the gradient of the loss function is calculated through back propagation, and the network parameters (such as weights and biases) are updated. At the same time, the structure and operation type of the network architecture are readjusted according to the optimized β and α. Through multiple rounds of training, the model gradually converges and the loss function gradually decreases. In each round of training, the framework search optimizes β and α based on the performance of the current network, making the model more flexible and efficient in dealing with the relationship between different components.

[0243] (5) Convergence and final model architecture: As training progresses, framework search and dynamic adjustment of the loss function will cause the model to eventually converge to an optimized network architecture. This architecture can not only take into account the independent characteristics of each component when processing images, but also further improve the overall performance through reasoning of mutually exclusive rules, combination rules, and decomposition rules.

[0244] In each round of training, the selection strategy and operation type of the network architecture can be dynamically adjusted by optimizing the framework parameters β and α. The loss function is designed based on the rules of mutual exclusion, combination and decomposition, which provides a basis for framework search and enables the model to adapt to the complex relationships between components in the image. By gradually adjusting these parameters, the network can optimize its architecture, thereby improving its understanding and reasoning capabilities of complex image scenes.

[0245] The step 4 is specifically as follows:

[0246] The loss function module based on three inference rules (mutually exclusive rule, combination rule and decomposition rule) is introduced into the architecture search process, and the network architecture is optimized through the evolutionary algorithm. The evolutionary algorithm automatically searches for the optimal network architecture by simulating the process of natural selection (such as selection, crossover, mutation, etc.), and combines these loss function modules to optimize the architecture. During the training process, not only the network architecture is optimized according to the loss function, but also the feature representation of the architecture and components is further adjusted by introducing the hierarchical information inference rule module.

[0247] The specific steps are:

[0248] (1) Initializing the initial population for architecture search: The first step in architecture search is to generate the initial population. Each individual in the population represents a potential network architecture. The initial population can be generated randomly or initialized based on some prior knowledge (such as previous research on network structure). Each network architecture can be represented by a set of discrete hyperparameters, such as the number of layers, the number of neurons in each layer, the size of the convolution kernel, the connection method, etc. These hyperparameters will be used as genetic information in the evolutionary algorithm for optimization in subsequent iterations. The network architecture in our model is represented by the β parameter and the alpha parameter.

[0249] (2) Definition of loss function module and loss calculation: In the process of architecture search, the loss function module is the core part of model optimization. Through the loss function designed based on three inference rules (mutually exclusive rule, combination rule and decomposition rule), the network architecture can be deeply optimized so that it can better handle the hierarchical relationship and interaction between components in complex image scenes. Mutually exclusive rule loss (L e ): Ensure the distinction between mutually exclusive categories and avoid the model from mistakenly predicting multiple high-confidence categories at the same time. Combined rule loss (L C ): Process the reasoning relationship between parent and child nodes, especially when the child node confidence is low, to enhance the reasoning influence of the parent node on the child node. Decomposition rule loss (L d ): describes how to decompose a complex object into multiple simple parts and optimize the reasoning of the parent node based on the information of the child node. The final loss function combination is:

[0250] L total =L e +L c +L d

[0251] These loss function modules will be dynamically optimized during the training process to guide the selection and optimization of the network architecture.

[0252] (3) Dynamically optimize β and α: In each round of training, by calculating the loss function L total, after back-propagation and updating the network weights, the β and α parameters are dynamically adjusted. This process ensures that the optimization of the architecture not only takes into account the overall performance of the network, but also dynamically adjusts the importance of each component according to its relationship.

[0253] The update formula is:

[0254]

[0255] Where η is the learning rate, and is the gradient of the loss function with respect to β and α.

[0256] (4) After back propagation, enter the hierarchical information inference rule module: In the conventional neural network training process, back propagation calculates the gradient of each parameter and updates it. But here, we propose a different approach - after back propagation is completed, the feature vector of each component is introduced into the hierarchical information inference rule module for calculation;

[0257] Feature vector extraction: The feature vectors generated by each component will be passed to the hierarchical information inference rule module. These feature vectors represent the output of each component in the network during the forward propagation process.

[0258] Inference rule module: The hierarchical information inference rule module is responsible for further calculating the feature vectors of each component based on three inference rules (mutually exclusive rule, combination rule, and decomposition rule). These inference rules not only deal with the relationship between components, but also dynamically adjust the role of each component in the image according to the adjustment of the loss function. The feature vector and segmentation score of each component will be optimized through the inference rule module;

[0259] Mutual Exclusion Rule: Ensure that at each pixel location, there is enough difference between multiple high-confidence categories to avoid misclassification.

[0260] Combination rules: According to the relationship between parent nodes and child nodes, adjust the feature vector and reasoning process to ensure that the child node can accurately reflect the characteristics of the parent node.

[0261] Decomposition rules: Optimize the decomposition and reasoning process of each object by adjusting the maximum score of child nodes and the characteristics of parent nodes.

[0262] Calculate new feature vectors and segmentation scores: Update the feature vectors and corresponding segmentation scores of each component based on the calculation of the hierarchical information inference rule module. The new feature vectors will help the model better understand the dependencies between components, further adjust the weights in the loss function, and optimize the network architecture.

[0263] (5) Selection, crossover, and mutation selection of evolutionary algorithms: Individuals (i.e., network architectures) with higher fitness in the current population perform better in optimizing loss functions and in the inference process. The selected individuals will enter the next generation and form the parent generation. Through the crossover operation, some features of the selected parent architecture are combined into a new architecture. This combination is based on the network units (cells) and operation types of the parent architecture to generate a new architecture and put it into the next generation population. In order to introduce new diversity, the mutation operation will randomly adjust the current architecture, such as changing the number of neurons in a certain layer, adding or reducing certain operations (such as convolutional layers, pooling layers, etc.). The mutated architecture will be further evaluated and optimized.

[0264] (6) Iteration and convergence: Through multiple generations of evolution, the network architecture will gradually converge to an optimal solution. In each generation, the evolutionary algorithm dynamically adjusts the architecture selection and operation type according to the output of the loss function and the hierarchical information reasoning rule module, so that the model can better handle the hierarchical component interactions in complex images. Finally, the algorithm will output an optimized network architecture that adapts to the multi-level component relationships and complex image scene expressions. In step 4, by introducing the loss function module and the hierarchical information reasoning rule module based on three inference rules, combined with the evolutionary algorithm, the model can dynamically optimize the parameters β and α during the architecture search process and optimize each part of the network architecture. Through these steps, the network architecture can adapt to the hierarchical relationships in complex image scenes, enhance the ability to handle multi-level component interactions, and thus improve the effect of image understanding and reasoning.

[0265] The step 5 is specifically as follows:

[0266] In step 5, the network architecture is generated through the architecture search phase, and the evolutionary algorithm and loss function module in step 4 have obtained an optimized network architecture. The goal of step 5 is to further train the optimized network architecture. In particular, in this stage, the network retraining module will further integrate the improved loss function module and information exchange module to accelerate the convergence process of the network.

[0267] In step 4, the evolutionary algorithm completes the architecture search and finds an optimized architecture. This architecture has been optimized many times in the loss function and hierarchical optimization process, and adapts to the hierarchical relationship and component interaction in complex images. The optimal path is the network structure generated during the architecture search process. Through the evolutionary algorithm, the network architecture has fully considered the mutual exclusion, combination and decomposition relationships between image components. The optimal network path found in the architecture search phase is brought into the retraining phase to ensure that the optimal architecture obtained during the optimization process of the network structure can be fully utilized in the subsequent training process. The goal of this stage is to further optimize the network weights and accelerate convergence.

[0268] In the retraining phase, the improved loss function module designed based on the three inference rules (mutually exclusive rules, combination rules, and decomposition rules) in step 3 is integrated, and the weights of these rules are dynamically optimized in each round of training. Based on the performance of each network architecture, the weights in the loss function are dynamically adjusted to focus on optimizing the mutual exclusion, combination, and decomposition rules at different training stages. At the beginning of network training, a lower weight value can be selected, and the weight of each rule can be gradually increased to help the network gradually adapt to the complex relationship between components and improve accuracy.

[0269] The role of the information exchange module is to accelerate the convergence process of the network, especially in the interaction process between multiple levels and components. The information exchange module can optimize the flow of information and improve the coordination and performance of various components. The information exchange module can dynamically adjust the path of the information flow so that the network can more efficiently transmit key information during reasoning. In each round of training, the information flow of each component in the network is optimized through the information exchange module, so that key components (such as core objects and important scene elements) can receive more attention during the reasoning process. The network dynamically adjusts the information exchange weights between components during training to effectively combine information at different levels. The information exchange module helps the network to effectively integrate across different levels (for example, from low-level features to high-level semantic information), reduce redundancy, and improve learning efficiency.

[0270] In step 5, the AsymmetricLossOptimized loss function will be introduced into the network retraining phase to replace the traditional OhemCELoss, specifically to deal with the class imbalance problem. By asymmetrically adjusting the positive and negative samples of different categories, this loss function effectively reduces the neglect of low-frequency categories and enhances the influence of low-frequency categories on network training.

[0271] Loss function formula:

[0272] L asym (p,y)=-α(y)·(1-p) γ ·log(p)

[0273] in:

[0274] α(y) is the weight associated with category y, which specifically provides higher weights for low-frequency categories and reduces the influence of high-frequency categories.

[0275] (1-p) γ It is an adjustment factor used to increase the loss weight of low-confidence positive samples, so that the network pays more attention to samples that are difficult to classify.

[0276] γ is a hyperparameter that adjusts the difficulty of samples. A larger value makes the network pay more attention to low-confidence samples.

[0277] During the retraining process, the AsymmetricLossOptimized loss function adjusts the weights to ensure that the network can balance the processing of high-frequency and low-frequency categories, thereby improving the learning effect of low-frequency categories.

[0278] Through the joint optimization of the loss function module and the information exchange module, the network can be retrained more efficiently, especially after the optimal architecture is obtained, the training speed and effect of the network are significantly improved. The core of retraining is to accelerate the convergence of the network and improve its reasoning ability. First, in the initial stage, the optimal path (optimal architecture) from the architecture search stage is used as the initial network structure and the weights are initialized. Usually, these weights are first pre-trained to obtain a better initialization. By gradually optimizing the weights of each part in the loss function, emphasizing the learning of the architecture at the beginning, gradually introducing the information exchange module for optimization, and finally accelerating the convergence of the network through multi-task learning and cross-layer information fusion. During the retraining process, the loss function of the three inference rules is combined to optimize each component of the network. These rules will enable the network to better capture the relationship between components and make fine adjustments to the network according to the loss function, so that the architecture can achieve higher accuracy in actual tasks. After the back propagation is completed, the feature vector of each component is passed to the hierarchical information inference rule module. This module further adjusts the feature vector according to the three inference rules (mutually exclusive rule, combination rule and decomposition rule) to optimize the performance of each component. Through the hierarchical information inference rule module, the network can calculate new segmentation scores based on the feature vector of each component. These scores help the network to more accurately identify and classify each component in the image in further training.

[0279] As the network architecture is continuously optimized in each round of training, the final model will converge to an optimal solution, which not only has high reasoning capabilities, but also can accurately understand and distinguish the relationship between different components when processing complex image scenes. Ultimately, through the integrated optimization of the improved loss function module and the information exchange module, the network can converge quickly and show strong performance during reasoning. By retraining the optimal architecture, the final model can not only quickly adapt to different image scenes, but also show higher accuracy and lower error in complex environments.

[0280] In step 5, the optimal network generated in the architecture search phase is further trained, combined with the improved loss function module and information exchange module, to accelerate the convergence process of the network. During the retraining process, by dynamically adjusting the weight of the loss function and introducing the hierarchical information inference rule module to calculate and update the feature vector, it is ensured that the network not only has high accuracy when processing complex images, but also can quickly adapt to different image structures and hierarchical information, ultimately improving the efficiency and effect of reasoning. The action relationship between each module is mainly reflected in the transmission of information flow and the interaction between modules:

[0281] The image component hierarchy building module extracts and analyzes the input image information, and the output hierarchical structure is passed to the hierarchical information inference rule module.

[0282] The hierarchical information reasoning rule module defines and transmits the reasoning relationship information between various components based on the hierarchical structure of the input.

[0283] The loss function module between component information is used in both the architecture search and retraining stages to optimize the model through defined rules.

[0284] The architecture search module and the network retraining module jointly use these rules to gradually find the optimal network architecture and improve model performance through multiple iterations.

[0285] Combined with the attached figure, the interactive relationship between these modules clearly describes the entire process from image input to the final output of the network structure. Through information sharing and collaboration, each module forms a top-down information transmission and bottom-up feedback adjustment, continuously optimizing the network architecture and model performance.

[0286] like Figure 1 As shown in the figure, it includes the information transmission paths and interactive relationships between all modules. The arrows in the figure indicate the entire process from image input to model training, architecture search, and final output.

[0287] like Figure 2As shown in the figure, the construction process of the image component hierarchy is shown. The various components in the input image are identified and constructed into a tree-like hierarchical structure. With important components (such as decks, hulls, etc.) as root nodes, and other subcomponents (such as masts, cabins, etc.) as child nodes or leaf nodes, a complete hierarchical relationship is formed. The components of an object can be divided into different levels according to their importance in the overall structure. Important components (such as decks and hulls) are root nodes, reflecting the core characteristics of the object, while less important components (such as propellers, bridges, cabins, etc.) constitute hierarchical child nodes or leaf nodes. The importance of a component is not only related to its function, but also comes from people's general understanding of the object or expert knowledge. For example, the deck and hull are considered to be the key features of a ship, while other subcomponents such as carrier-based aircraft and masts further refine the specific functions of the hull. Through the construction of a tree-like hierarchical structure, the composition relationship of complex objects can be better expressed, making object analysis, recognition and classification more intuitive and efficient. This hierarchical relationship can be applied to computer vision tasks such as object detection, classification recognition and semantic segmentation, and provides a basic framework for step-by-step processing and structured analysis.

[0288] like Figure 3 As shown in the figure, the specific application of mutually exclusive rules, combination rules and decomposition rules is shown, and how to transfer hierarchical information between components. For example, it shows the connection relationship between parent nodes and child nodes, the information transmission mechanism between sibling nodes, etc. The figure shows how decomposition, combination and mutual exclusion rules effectively manage the relationship between components in the hierarchy, forming a clear and controllable information transmission and functional collaboration framework. The parent node and the child node realize the two-way transmission of information through decomposition rules (from parent to child) and combination rules (from child to parent), reflecting the relationship from the whole to the part and from the part to the whole. Mutually exclusive rules are used to avoid conflicts between sibling nodes to ensure that only one component is activated or selected in a specific scenario. Decomposition rules, combination rules and mutual exclusion rules jointly build a systematic hierarchical management framework, which is conducive to the efficient design and optimization of complex systems. This mechanism can be used for task decomposition, module management and functional coordination, and is suitable for many fields such as product design, logical reasoning, and computer vision.

[0289] like Figure 4As shown in the figure, this figure shows the loss function module designed for the interaction between components, especially how to achieve better modeling by adjusting the weights between components. Including the comparative application of the traditional OhemCELoss loss function and the AsymmetricLossOptimized loss function. The loss function is: Asl+loss_logic and an information exchange is added. After the introduction of Asl+loss_logic and information exchange, the model's ability to model complex interactions between components is significantly improved. Compared with the traditional loss function, the optimization effect is better and the convergence is faster, which is suitable for complex scenarios. OhemCELoss: The traditional method converges quickly, but the modeling ability of complex component interactions is limited, and the decline tends to be gentle in the later stage. Asl+loss_logic: Combining asymmetric loss (ASL) and logical loss (loss_logic), and introducing an information exchange, a smoother and more efficient optimization is achieved, which is suitable for complex component interaction modeling. The improved loss function strengthens the differentiated processing of component weights and highlights the modeling of key components. Information exchange improves the modeling accuracy of the relationship between components and makes the model more efficient. The weight distribution has been optimized and the dynamic expression ability of component interaction has been enhanced.

Claims

1. A neural network architecture search system based on image component hierarchical structure information, characterized in that: It includes an image component hierarchy building module, a hierarchy information inference rule module, a loss function module between component information, an architecture search module and a network retraining module; The image component hierarchical structure building module is used to extract and analyze the input image information and build the image component hierarchical structure; The image component hierarchical structure is constructed by analyzing each component of the object in the image to construct a tree structure based on the components of the object; The hierarchical information reasoning rule module defines and transmits the reasoning relationship information between each component based on the hierarchical structure of the image components; each component refers to the different components contained in the object in the data set; The loss function module between component information is used to optimize the architecture search module and the retraining module according to the defined rules during the architecture search and retraining phases; The architecture search module and the network retraining module jointly use the rules defined by the hierarchical information inference rule module to gradually find the optimal network architecture and improve the model performance through multiple iterations.

2. A search method for a neural network architecture search system based on image component hierarchical structure information according to claim 1, comprising the following steps: Step 1: constructing an image component hierarchical structure using the image component hierarchical structure construction module, and extracting features of each image component; Step 2: using the hierarchical information inference rule module, taking the component features as input of the hierarchical information inference rule module, implementing hierarchical reasoning through classification scores and feature sharing in the inference rule module, dynamically weighting component features at different levels, and transmitting hierarchical information between components; Step 3: Use the loss function module between component information to assign and optimize the weights of the dynamically weighted component features to model the interaction between components and dynamically adjust the influence weights; Step 4: In the architecture search phase, the hierarchical information inference rule module and the loss function module are combined to perform an optimization search for the network architecture through an evolutionary algorithm; Step 5: In the architecture retraining phase, the network convergence process is further accelerated by integrating the improved loss function module and information exchange module.

3. Based on the search method of a neural network architecture search system based on image component hierarchical information as described in claim 2, in step 1, the image component hierarchy is a tree structure, and the image component hierarchy construction module defines the root node and child nodes of the tree structure, the root node represents the most important component or information in the image, and the child node represents the relatively less important component or local feature.

4. Based on the search method of the neural network architecture search system based on image component hierarchical structure information as described in claim 2, the specific steps of step 1 are: Step (1): Preprocess the complex image, including denoising, normalization and scale adjustment, to extract more detailed and multi-level features; Step (2): extracting global and local features of the preprocessed image through a deep learning model; Step (3): Use semantic segmentation technology to divide the global and local features of the image into different regions, separate the objects and parts in the image, and assign semantic labels to each region; Step (4): Based on the semantic labels, objects and components are organized into a tree structure according to their importance. The root node represents the most core object or scene, and the child nodes represent less important objects or local features. Step (5): Dynamically optimize the weights and information flows between different components through an adaptive adjustment mechanism.

5. Based on the search method of the neural network architecture search system based on the hierarchical structure information of image components as claimed in claim 4, in said step 2, the logic rules of the hierarchical information inference rule module include: Mutually exclusive rules, combination rules and decomposition rules are used to describe the relationship between parent and child nodes, between sibling nodes, and between child nodes in the hierarchical structure; the hierarchical information reasoning rule module uses the component information extracted in step 1 for reasoning; Among them, the combination rule refers to the reasoning process of the less important components to the important components, and the more complex object or scene structure is derived from the less important components through the relationship with the important components; Decomposition rules refer to the process of decomposing important components into less important components, breaking down important objects or areas into smaller, less important parts or details in order to further understand their internal structure or relationships.

6. Based on the search method of the neural network architecture search system based on image component hierarchical structure information as claimed in claim 5, the mutually exclusive rule is: At a pixel point, if two or more categories have scores higher than the threshold, the exclusive constraint is strengthened between these high-scoring categories; the modified logic is expressed as follows: in: T is the confidence threshold, which is used to determine which categories are considered high confidence; is an indicator function that indicates whether the category score exceeds the threshold; Indicates that only pixels containing two or more high-confidence categories are exclusively optimized; Loss function of mutually exclusive rule: When there are two or more high confidence categories in a node (pixel position), the mutually exclusive rule is defined as follows: In this formula: s k [v] represents the score of category v on the kth pixel; T is the confidence threshold, which is used to determine whether the score is high enough; and is the indicator function, ensuring that only those categories with confidence exceeding T are calculated into the exclusive loss; The calculation of the mutually exclusive rule loss function is as follows: Multi-category constraints: The formula introduces similarity constraints between multiple high-confidence categories, and the exclusive loss suppresses the similarities between these high-confidence categories to enhance the distinction between categories; High confidence screening: The indicator function ensures that only categories with confidence higher than the threshold T are affected; Loss calculation: By minimizing L E value, making the similarities between multiple high-confidence categories smaller for better distinction; The combination rules are specifically: In the inference formula, 1-s k [v] indicates that the lower the confidence value of node v for the kth sample, the more it depends on the influence of its parent node pv. k [pv] represents the confidence of the parent node pv for the kth sample. The calculation formula of the loss function of the combination rule is defined as follows: s k [v] and s k [pv] also represents the confidence of node v and parent node pv for the kth sample; q is the adjustment index used to control the influence between each node; The final loss function is written as: Where |V| is the number of all nodes and |VL| is the number of leaf nodes. The decomposition rules are specifically: The relationship between the parent node and the child node is combined by introducing (1-MCMB)*MCMA.max(...), where MCMB represents the prediction result of the parent node and MCMA represents the prediction results of multiple child nodes; For the combination of multiple nodes, a similar mechanism is used to merge the prediction results of all child nodes into the parent node; For each non-leaf node v∈V\V, the new D-rule loss is: where GD(v) is given by: In the formula, the value of node v is adjusted to s_k[v] and is connected to the maximum value of its child node max(s k [cn v ]) q Combine and take the maximum value among the child nodes to modify the parent node; For k [v] and the maximum value of the child nodes are adjusted differently, introducing a new power operation q, max({s k [cn v ]}) q It is a power operation of the maximum value of the child node, which helps to strengthen the influence of the child node on the parent node or make other nonlinear adjustments; s k [v] represents the predicted value of the parent node v, max({s k [cn v ]}) q is the maximum value among the predicted values ​​of the child nodes, and this maximum value is adjusted by the power q. The formula represents the transfer relationship between the parent node and the child node, which is specifically calculated by the value s of the parent node. k The reasoning is performed by combining [v] with the maximum value of the child nodes.

7. Based on the search method of the neural network architecture search system based on image component hierarchical structure information as claimed in claim 6, the step 3 is specifically: Based on the loss function design of mutually exclusive rules, combination rules and decomposition rules, the loss functions constructed by these three rules are integrated into the optimization process of framework search, and the framework parameters β and α are optimized in each round of training to dynamically adjust the structure and performance of the network; In the framework search process, β and α represent two key optimization parameters, β: controls the selection strategy of cells in the framework and affects the network structure during the search process; α: controls the importance of the selected operation in the search process and determines the selection weight of each operation type in the network architecture. Based on the dynamic optimization of these two parameters, the framework search adjusts the selection of the network architecture through each round of training. At the same time, the loss function provides guidance for weight adjustment and inference optimization in the optimization process.

8. A search method for a neural network architecture search system based on image component hierarchical structure information according to claim 7, comprising the following specific steps: (1) Initialize the framework parameters β and α. β controls the strategy of selecting different cells. During the framework search process, β affects which cells are selected and form the basic modules of the network architecture. α controls the weights of selecting different operation types and adjusts the selection and combination strategy of operations. (2) Calculation and adjustment of the loss function: Based on the loss function in step 3, in each round of training, the loss function will calculate the error of each component based on the current network architecture; (3) Calculate the loss function and adjust β and α: Through back propagation, the network calculates the error of each component according to the current loss function and gradually adjusts β and α so that the network architecture can better adapt to the relationship between components in the image; Optimize β: β controls the strategy for selecting cells in the frame search. If the currently selected cell performs poorly when processing the image, you need to adjust β to select a more suitable cell. Optimize α: α controls the weight of selecting different operations in the framework search. The current operation type may have a greater impact on some inference rules. If an operation performs poorly, a more suitable operation can be selected by adjusting α; (4) Update network parameters and architecture: At the end of each round of training, the gradient of the loss function is calculated through back propagation, and the network parameters are updated. At the same time, the structure and operation type of the network architecture are readjusted according to the optimized β and α. Through multiple rounds of training, the model gradually converges and the loss function gradually decreases. In each round of training, the framework search optimizes β and α based on the performance of the current network. (5) Convergence and final model architecture: As training progresses, framework search and dynamic adjustment of the loss function will cause the model to eventually converge to an optimized network architecture.

9. Based on the search method of the neural network architecture search system based on image component hierarchical structure information as claimed in claim 8, the step 4 is specifically: (1) Initialize the initial population for architecture search; each individual in the population represents a potential network architecture; (2) Define the loss function module and loss calculation; design the loss function based on three inference rules; Mutually exclusive rule loss (L e ): Ensure the discrimination between mutually exclusive categories and avoid the model from mistakenly predicting multiple high-confidence categories at the same time; Combination rule loss (L C ): handles the inference relationship between parent and child nodes; Decomposition rule loss (L d ): describes how to decompose a complex object into multiple simple parts and optimize the reasoning of the parent node based on the information of the child node. The final loss function combination is: L total =L e +L c +L d (3) Dynamically optimize β and α: In each round of training, by calculating the loss function L total , after back-propagation and updating the network weights, the β and α parameters are dynamically adjusted; The update formula is: Where η is the learning rate, and is the gradient of the loss function with respect to β and α; (4) In the conventional neural network training process, back propagation calculates the gradient of each parameter and updates it; (5) Selection, crossover, and mutation selection of evolutionary algorithms: Individuals with higher fitness in the current population, whose architectures perform better in optimizing the loss function and inference process, will be selected to enter the next generation and form the parent generation. Through the crossover operation, some features of the selected parent generation architecture will be combined into a new architecture. (6) Iteration and convergence: Through multiple generations of evolution, the network architecture will gradually converge to an optimal solution; The step (4) is specifically: After the back-propagation is completed, the feature vector of each component is introduced into the hierarchical information inference rule module for calculation; Feature vector extraction: The feature vectors generated by each component will be passed to the hierarchical information inference rule module. These feature vectors represent the output of each component in the network during the forward propagation process. Inference rule module: The hierarchical information inference rule module is responsible for further calculation of the feature vectors of each component based on three inference rules; Mutual Exclusion Rule: Ensure that at each pixel location, there is enough difference between multiple high-confidence categories to avoid misclassification; Combination rules: According to the relationship between parent nodes and child nodes, adjust the feature vector and reasoning process to ensure that the child node can accurately reflect the characteristics of the parent node; Decomposition rules: Optimize the decomposition and reasoning process of each object by adjusting the maximum score of the child nodes and the characteristics of the parent nodes; Calculate new feature vectors and segmentation scores: Update the feature vector of each component and the corresponding segmentation score based on the calculation of the hierarchical information inference rule module.

10. Based on the search method of the neural network architecture search system based on image component hierarchical structure information as claimed in claim 9, the step 5 is specifically: The AsymmetricLossOptimized loss function will be introduced into the network retraining phase; Loss function formula: L asym (p,y)=-α(y) (1-p) γ log(p) in: α(y) is the weight associated with category y, which specifically provides higher weights for low-frequency categories and reduces the influence of high-frequency categories; (1-p) γ It is a regulation factor used to increase the loss weight of low-confidence positive samples, so that the network pays more attention to samples that are difficult to classify; γ is a hyperparameter for adjusting the difficulty of samples. A larger value makes the network pay more attention to low-confidence samples. During the retraining process, the AsymmetricLossOptimized loss function adjusts the weights to ensure that the network can balance the processing of high-frequency and low-frequency categories, thereby improving the learning effect of low-frequency categories.

Images