Patellar X-ray image recognition method based on lightweight network architecture search under constraints

Abstract

The present invention discloses a method for patella X-ray image recognition using a lightweight network architecture search under constraints, comprising: 1. Collecting and preprocessing labeled patella X-ray image data, and dividing it into a training set and a validation set; 2. Initializing and constructing a convolutional neural network population; 3. Training individuals in the population using the X-ray dataset, using the fitness values ​​of the constrained fitness function as the fitness values ​​of the individuals; 4. Selecting parent individuals based on the fitness values ​​of the individuals in the population, generating a child population through filter pruning with an adaptive pruning rate and crossover mutation, and finally performing environment selection; 5. Iteratively optimizing the network structure until predetermined conditions are met, and extracting the optimal architecture as the patella X-ray image recognition model. The present invention can automatically search for a convolutional neural network architecture suitable for the patella X-ray image dataset, thereby achieving accurate classification of patella X-ray images, particularly in application scenarios with device hardware constraints.

Description

Technical Field

[0001] The present invention belongs to the field of computer and machine learning, in particular to the sub-fields of automated machine learning and neural network architecture search. Specifically, the present invention is a method for patellar X-ray image recognition using a lightweight network architecture search technology under constraints. Background Art

[0002] Convolutional neural networks (CNNs) are becoming a key component of computer vision and artificial intelligence in the field of medical image analysis. They play a vital role in key medical tasks such as image classification, anomaly detection, image segmentation, and reconstruction, aiming to improve recognition accuracy and efficiency. In the field of medical image processing, carefully designed CNNs can significantly improve image recognition accuracy. Furthermore, given the multiple limitations that medical devices may face in real-world applications, such as storage space and computational efficiency, it is crucial to develop CNNs that are lightweight, efficient, and able to meet specific resource constraints. This not only affects the performance of the model, but also its practicality and feasibility in real-world settings. Therefore, researching and designing CNNs that maintain excellent performance under resource-constrained conditions is crucial for promoting the application of AI in medical image recognition.

[0003] In the field of medical X-ray image recognition, neural network design methods are primarily divided into manual design and automated neural architecture search. While hand-designed networks such as VGG16 and ResNet perform well in image classification tasks, they require extensive domain knowledge and manual tuning, making them challenging for most practitioners. While NAS methods can automatically discover high-performance network designs, they also have some drawbacks.

[0004] Designing networks using neural architecture search is based on three main search strategies: reinforcement learning, gradient optimization, and evolutionary algorithms.

[0005] First, while reinforcement learning-based neural architecture search can automatically explore network structures when designing X-ray image recognition networks, this approach is computationally expensive and carries risks of instability and overfitting. This can lead to models overfitting to the training data and poor generalization to new, unseen data in real applications.

[0006] Secondly, while gradient-based neural architecture search methods are computationally efficient, they can be limited to a predefined set of operations and can be challenging to optimize for discrete decisions. This means that for a specific patellar X-ray image recognition task, it may not be possible to find the most suitable network architecture for the task.

[0007] Furthermore, the neural architecture search method based on evolutionary computation has shown its unique advantages. This method can effectively explore a wide range of search spaces to discover network architectures with excellent performance by simulating the process of natural selection. However, this method still has some challenges and limitations when applied to specific tasks. For example, the process may be very time-consuming, and the generated network structure may be too complex. To overcome these challenges, researchers have begun to explore incorporating constraints into the NAS process to guide the search process in a direction that is more suitable for X-ray image tasks. For example, by introducing geometric and topological constraints, the accuracy of segmentation tasks can be improved, and this method can also be used to guide the recognition and classification of patellar X-ray images.

[0008] In summary, although NAS based on evolutionary computation shows potential in patellar X-ray image recognition, further research and optimization are still needed to address its challenges in computational efficiency, network complexity, and adaptability to the specific needs of medical X-ray images. Summary of the Invention

[0009] The present invention aims to overcome the limitations of the existing technology and provides a patellar X-ray image recognition method based on a lightweight network architecture search under constraints, in order to find a more effective neural network structure that better meets resource constraints for the recognition and classification of patellar X-ray images, thereby improving the accuracy of patellar X-ray image classification and solving the problem of difficulty in deploying classification models due to constraints such as hardware storage of medical equipment.

[0010] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0011] The characteristic of the patella X-ray image recognition method of the present invention using a lightweight network architecture search under constraints is that it is performed according to the following steps:

[0012] Step 1: Collect and preprocess patellar X-ray images with I real class labels, obtain the preprocessed patellar X-ray image set and divide it into training set D tra and validation set D val ;

[0013] Step 2: Construct a convolutional neural network consisting of several MobileNetV3 convolution modules, several maximum pooling modules, and several average pooling modules; each convolution module includes a point-by-point convolution layer, a depth-separable convolution layer, an SE attention layer, and an inverted residual connection, wherein the number of filters in the depth-separable convolution layer is in the range of 1 to 512; let the structural length L of the convolutional neural network be the total number of convolution modules, maximum pooling modules, and average pooling modules;

[0014] Step 3: Construct a population with convolutional neural networks as individuals and perform iterative evolution to obtain the individual with the best performance as the optimal patellar X-ray image recognition model A. best ;

[0015] Step 3.1, parameter definition and initialization, including: population size is N, initial pruning rate is , the total number of iterations is T, the current number of iterations is t, and the training cycle is , the crossover probability is , the mutation probability is ;

[0016] Step 3.2, initialize t=1;

[0017] Encode the convolutional neural network to obtain an individual, and the length of the individual is the total number of modules in the convolutional neural network. Each number in the individual corresponds to a module in the convolutional neural network, where the convolution module is represented by a number between [1,512], the average pooling module is represented by a number greater than 0.5, and the maximum pooling module is represented by a number less than 0.5. The order of the numbers in the individual represents the connection order between layers in the convolutional neural network;

[0018] Randomly initialize N convolutional neural networks to obtain the t-th generation population P consisting of N individuals t ;

[0019] Step 3.3, the t-th generation population P t After decoding the N individuals in the t-th generation population P, t N convolutional neural networks in ;

[0020] Step 3.4: Set the training set D tra Enter the t-generation population P respectively t The N convolutional neural networks in the t-th generation are processed to obtain the t-th generation population P t The predicted category labels of the N convolutional neural networks in the training set D tra The true category label constructs the cross entropy loss function, so as to t The N convolutional neural networks in are trained to obtain the t-th generation population P after training. t ;

[0021] The validation set D val Input the t-th generation population P after training respectively t The N convolutional neural networks are processed to obtain the predicted category labels of N convolutional neural networks and compared with the validation set D val The true category label of the t-th generation population P is calculated t The classification accuracy of each individual;

[0022] Step 3.5: Calculate the t-th generation population P using formula (1) t The penalty term CV of the i-th individual t,i ;

[0023] (1)

[0024] In formula (1), Params t,i represents the t-th generation population P t The number of parameters in the i-th individual, Flops t,i represents the t-th generation population P t The floating-point number of the i-th individual; Params max Indicates the upper limit of the number of parameters for each given individual, Flops max Indicates the upper limit of floating-point operations for each given individual; L t,i represents the t-th generation population P t The structural length of the i-th individual; 、 and Represent three penalty factors respectively;

[0025] Step 3.6: Calculate the t-th generation population P using formula (2) t The fitness value F of the i-th individual t,i ;

[0026] (2)

[0027] In formula (2), Accuracy t,i Represents the t-generation population P t The classification accuracy of the i-th individual;

[0028] Step 3.7: Use the binary tournament selection strategy to select the tth generation population P t Each time two parent individuals are selected, and the fitness values ​​of the two selected parent individuals are compared, and the parent individual with higher fitness value is retained to join the tth generation mating pool P t ', until the mating pool P t 'Until the number of individuals in is N;

[0029] Step 3.8: Based on the adaptive pruning rate, the mating pool P of the tth generation t 'After pruning, the pruned t-th generation population AP is obtained t ';

[0030] Step 3.9, for the t-th generation mating pool P t 'Perform crossover mutation to obtain the tth generation population C after mutation t':

[0031] Step 3.10: AP t ' and C t 'Merge into the t-th generation offspring population Q of size N t ;

[0032] Step 3.11, P t With Q t After merging, a t-generation mixed population M with a size of 2N is formed t , and according to M t The fitness value of each individual in M ​​is selected by the roulette wheel selection mechanism. t N individuals are selected from the population to form the t+1 generation population P t+1 ;

[0033] Step 3.12: After assigning t+1 to t, if t>T, then the T-th generation population P is obtained. T , and execute step 3.13, otherwise, return to step 3.3 and execute sequentially;

[0034] Step 3.13: For the T-th generation population P T The fitness values ​​of the N individuals are sorted in descending order, and the individual with the largest fitness value is taken as the optimal patellar X-ray image recognition model A. best , which is used to identify the input patellar X-ray image and obtain the best predicted category label.

[0035] The method for patella X-ray image recognition using a lightweight network architecture search under constraints described in the present invention is also characterized in that step 3.8 is performed as follows:

[0036] Step 3.8.1, for the t-generation mating pool P t The individuals in 'are arranged in descending order according to their own classification accuracy, and the first population of the tth generation after arrangement is obtained;

[0037] Step 3.8.2, for the t-generation mating pool P t The individuals in 'are arranged in descending order according to their own fitness functions to obtain the second population of the tth generation after arrangement;

[0038] Step 3.8.3: Calculate the mating pool P of the tth generation using formula (3) t The complexity of the i-th individual in ' t,i , thus, for the t-th generation mating pool P t The individuals in 'are arranged in ascending order according to their own complexity, and the third population of the tth generation is obtained after arrangement;

[0039] (3)

[0040] Step 3.8.4: Select the first k individuals from the first population of the tth generation, the second population of the tth generation, and the third population of the tth generation after permutation and form the temporary population AP of the tth generation. t ;

[0041] Step 3.8.5: Calculate the adaptive pruning rate of the tth generation using formula (4) :

[0042] (4)

[0043] Step 3.8.6, based on , for AP t The number of filters of each convolution module of each individual in is changed to obtain the pruned t-th generation population AP t '.

[0044] Furthermore, the step 3.9 is performed as follows:

[0045] Step 3.9.1: Select the t-th generation mating pool P through a binary tournament. t 'Select two non-repeated parent individuals p t,1 and p t,2 and use the crossover probability a to p t,1 and p t,2 Perform single-point crossover to obtain two offspring individuals o of the tth generation t,1 and o t,1 Join the tth generation candidate offspring population C t In, until C t The number of neutron generation individuals is N-3k;

[0046] Step 3.9.2, use mutation probability b to change C t Each offspring individual in the mutation operation is performed to obtain the mutated population C t '.

[0047] The electronic device of the present invention includes a memory and a processor, and is characterized in that the memory is used to store a program that supports the processor to execute the patella X-ray image recognition method, and the processor is configured to execute the program stored in the memory.

[0048] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program executes the steps of the patella X-ray image recognition method when the computer program is run by a processor.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] 1. The present invention designs a highly flexible variable-length coding scheme to provide an automated network architecture search strategy for the task of patellar X-ray image recognition. This scheme can penetrate into each layer of the network and freely search for any number of filters to comprehensively cover every possible dimension of the network architecture, thereby achieving efficient recognition of patellar X-ray images. Through this broad and in-depth search, the present invention can identify network architectures with excellent performance. At the same time, by maintaining population diversity, the present invention effectively improves the generalization ability of the model, reduces the risk of overfitting, and ensures the stability and robustness of the recognition method.

[0051] 2. The present invention proposes a filter pruning strategy with adaptive pruning rate, which brings significant advantages to patellar X-ray image recognition. First, the strategy achieves an organic combination of global optimization and local optimization by automatically adjusting the pruning rate with evolutionary iterations, thereby more effectively exploring potential optimal models during the search process. Secondly, this strategy also helps to reduce the model size and computational complexity, making the model more suitable for deployment on resource-constrained devices. Ultimately, this innovative pruning strategy significantly improves the efficiency and practicality of patellar X-ray image recognition without sacrificing classification accuracy.

[0052] 3. The present invention proposes a constraint-based fitness function. In addition to the classification accuracy, the number of model parameters, floating-point calculations, and architecture length are incorporated into the optimization objective as constraint penalty terms. This function can dynamically adjust the search direction and optimize the network architecture while satisfying resource constraints. When the network architecture satisfies the resource constraints, the fitness function focuses on improving the classification accuracy; otherwise, the penalty terms are used to optimize the model parameters and computational complexity, guiding the network to develop in a more lightweight direction. Such a design effectively balances the classification performance and resource consumption of the model. Therefore, under the action of this fitness function, a more lightweight and efficient network architecture can be found for the task of patellar X-ray image recognition.

[0053] 4. The present invention can effectively search for an adapted convolutional neural network architecture for any resource constraint. This architecture not only excels in learning and identifying image features from a patellar X-ray image dataset, achieving high accuracy in classifying abnormalities in patellar X-ray images, but also demonstrates excellent performance in terms of parameter size and floating-point operations. Thanks to its small model parameters and efficient computing power, the network structure of the present invention can be well deployed on medical devices, thereby achieving rapid and accurate classification of patellar X-ray images. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of the method of the present invention;

[0055] Figure 2The convolutional neural network structure diagram designed for the present invention;

[0056] Figure 3 Schematic diagram of each category of X-ray films in the osteoporosis dataset used in the present invention. DETAILED DESCRIPTION

[0057] In this embodiment, Figure 1 As shown in FIG, a patella X-ray image recognition method based on a lightweight network architecture search under constraints is performed in the following steps:

[0058] Step 1: Collect n patellar X-ray image datasets with real category labels and obtain the training set D after preprocessing. tra and validation set D val , and the patellar X-ray images are divided into two categories of normal and abnormal labels, such as Figure 3 As shown;

[0059] Step 1.1. Obtain a total of n X-ray images. Each image is represented as (h0, w0, c, 0), where h0 and w0 represent the original height and width of the image, respectively, c represents the number of channels of the image, and 0 represents the label category corresponding to the image. In this experiment, h and w are set to 48, and c=3.

[0060] Step 1.2: Pass the X-ray image from step 0.1 to the preprocessing function R(h, w) to perform the resize operation, thereby obtaining the preprocessed X-ray image (h, w, c, 0). Here, h and w define the height and width of the resized image, respectively.

[0061] Step 1.3: Feed n X-ray images into the partitioning function S(r), which first shuffles the image data and then divides the data set into the training set D according to the ratio r. tra and validation set D val .

[0062] Step 2. Construct a convolutional neural network consisting of several MobileNetV3 convolutional modules, several maximum pooling modules, and several average pooling modules; each convolutional module includes a point-by-point convolutional layer, a depth-wise separable convolutional layer, an SE attention layer, and an inverted residual connection, where the number of filters in the depth-wise separable convolutional layer is in the range of 1 to 512; let the structural length L of the convolutional neural network be the total number of convolutional modules, maximum pooling modules, and average pooling modules;

[0063] For example Figure 2 The network architecture shown in the example is composed of four convolutional layers and two pooling layers stacked together.

[0064] Step 3: Construct a population with convolutional neural networks as individuals and perform iterative evolution to obtain the individual with the best performance as the optimal patellar X-ray image recognition model A. best ;

[0065] Step 3.1, parameter definition, including: initial population size N = 20, initial pruning rate =20%, the total number of evolutionary iterations is T =20, the current iteration is t=1, and the training cycle is =450, the crossover probability is =0.9, the mutation probability is =0.2, number of categories I=2.

[0066] Step 3.2, initialize t=1;

[0067] Encode the convolutional neural network to obtain an individual, and the length of the individual is the total number of modules in the convolutional neural network. Each number in the individual corresponds to a module in the convolutional neural network, where the convolution module is represented by a number between [1,512], the average pooling module is represented by a number greater than 0.5, and the maximum pooling module is represented by a number less than 0.5. The order of the numbers in the individual represents the connection order between layers in the convolutional neural network; Figure 2 The network architecture encoding representation of the example is 234 - 126 - 0.4 - 69 - 0.8 - 108;

[0068] Randomly initialize N convolutional neural networks to obtain the t-th generation population P consisting of N individuals t .

[0069] Step 3.3, the t-th generation population P t After decoding the N individuals in the t-th generation population P, t N convolutional neural networks in ;

[0070] Step 3.4: Set the training set D tra Enter the t-generation population P respectively t The N convolutional neural networks in the t-th generation are processed to obtain the t-th generation population P t The predicted category labels of the N convolutional neural networks in the training set D tra The true category label constructs the cross entropy loss function, so as to t The N convolutional neural networks in are trained to obtain the t-th generation population P after training. t ;

[0071] The validation set D val Input the t-th generation population P after training respectively tThe N convolutional neural networks are processed to obtain the predicted category labels of N convolutional neural networks and compared with the validation set D val The true category label of the t-th generation population P is calculated t The classification accuracy of each individual.

[0072] Step 3.5: Calculate the t-th generation population P using formula (1) t The penalty term CV of the i-th individual t,i ;

[0073] (1)

[0074] In formula (1), Params t,i represents the t-th generation population P t The number of parameters in the i-th individual, Flops t,i represents the t-th generation population P t The floating-point number of the i-th individual; Params max Indicates the upper limit of the number of parameters for each given individual, which is 0.6M, Flops max Indicates the upper limit of floating-point operation for each given individual, which is 150M; L t,i represents the t-th generation population P t The structural length of the i-th individual; 、 and They represent three penalty factors, with values ​​of 0.5, 0.5, and 0.001 respectively.

[0075] Step 3.6: Calculate the t-th generation population P using formula (2) t The fitness value F of the i-th individual t,i ;

[0076] (2)

[0077] In formula (2), Accuracy t,i Represents the t-generation population P t The classification accuracy of the i-th individual; the constraint-based fitness function can guide the direction of individual evolution in the population. When the network architecture meets the resource constraints, the fitness function focuses on improving the classification accuracy; otherwise, the penalty term is used to optimize the model parameters and computational complexity, guiding the network to develop in a more lightweight direction.

[0078] Step 3.7: Use the binary tournament selection strategy to select the tth generation population P t Each time two parent individuals are selected, and the fitness values ​​of the two selected parent individuals are compared, and the parent individual with higher fitness value is retained to join the tth generation mating pool P t', until the mating pool P t 'Until the number of individuals in is N;

[0079] Step 3.8: Based on the adaptive pruning rate, the mating pool P of the tth generation t 'After pruning, the pruned t-th generation population AP is obtained t ';

[0080] Step 3.8.1, for the t-generation mating pool P t The individuals in ' are arranged in descending order according to their classification accuracy to obtain the first population of the tth generation after arrangement; classification accuracy is the key goal in the optimization process, and the top-ranked network architectures often show more significant performance improvement potential after filter pruning.

[0081] Step 3.8.2, for the t-generation mating pool P t The individuals in ' are arranged in descending order according to their own fitness functions to obtain the second population of the tth generation after arrangement; its fitness value is a comprehensive consideration of the classification accuracy of the network architecture and the degree of constraint violation.

[0082] Step 3.8.3: Calculate the mating pool P of the tth generation using formula (3) t The complexity of the i-th individual in ' t,i , thus, for the t-th generation mating pool P t The individuals in ' are arranged in ascending order according to their own complexity to obtain the third population of the tth generation after arrangement; the smaller the complexity, the closer the network architecture is to the given resource constraints.

[0083] (3)

[0084] Step 3.8.4: Select the first k individuals from the first population of the tth generation, the second population of the tth generation, and the third population of the tth generation after permutation and form the temporary population AP of the tth generation. t .

[0085] Step 3.8.5: Calculate the adaptive pruning rate of the tth generation using formula (4) :

[0086] (4)

[0087] As the number of search generations increases, the filter pruning rate is adaptively adjusted. This strategy allows for rapid convergence in the early stages of network architecture search while retaining sufficient flexibility to accommodate subsequent fine-tuning. As the search progresses, a more focused search is performed on smaller, more refined areas. This not only improves search efficiency but also helps discover more refined and superior network structures.

[0088] Step 3.8.6, based on , for AP t The number of filters of each convolution module of each individual in is changed to obtain the pruned t-th generation population AP t '.

[0089] Step 3.9, for the t-th generation mating pool P t 'Perform crossover mutation to obtain the tth generation population C after mutation t ':

[0090] Step 3.9.1: Select the t-th generation mating pool P through a binary tournament. t 'Select two non-repeated parent individuals p t,1 and p t,2 and use the crossover probability a to p t,1 and p t,2 Perform single-point crossover to obtain two offspring individuals o of the tth generation t,1 and o t,1 Join the tth generation candidate offspring population C t In, until C t The number of neutron generation individuals is N-3k;

[0091] Step 3.9.2, use mutation probability b to change C t Each offspring individual in the mutation operation is performed to obtain the mutated population C t ';

[0092] Four types of mutation operators are defined: 1. Insertion: randomly adds a random pooling module or a convolution module with random initialization values ​​at a selected position; 2. Delete: randomly deletes a module at a selected position; 3. Modification: modifies the pooling module type and the number of convolution module filters; and 4. Replacement: replaces a pooling module with a convolution module with random initialization values, or replaces a convolution module with a random pooling module. The mutation operator is randomly selected to obtain the mutated population Ct'.

[0093] Step 3.10: AP t ' and C t 'Merge into the t-th generation offspring population Q of size N t ;

[0094] Step 3.11, P t With Q t After merging, a t-generation mixed population M with a size of 2N is formed t , and according to M t The fitness value of each individual in M ​​is selected by the roulette wheel selection mechanism. t N individuals are selected from the population to form the t+1 generation population Pt+1 If the individuals with the highest classification accuracy or the largest fitness value are not selected in the selection process, the elite retention strategy is adopted: directly add these elite individuals to the new generation population P t+1 and replace the individuals with the lowest classification accuracy and fitness value.

[0095] Step 3.12: After assigning t+1 to t, if t>T, then the T-th generation population P is obtained. T , and execute step 3.13, otherwise, return to step 3.3 and execute sequentially;

[0096] Step 3.13: For the T-th generation population P T The fitness values ​​of the N individuals are sorted in descending order, and the individual with the largest fitness value is taken as the optimal patellar X-ray image recognition model A. best , which is used to identify the input patellar X-ray image and obtain the best predicted category label.

[0097] Step 4. Save the trained neural network model A best And the corresponding weight parameters, a patella X-ray image recognition model is obtained, thereby realizing the classification prediction of patella X-ray images.

[0098] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0099] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are executed.

[0100] Table 1

[0101]

[0102] As shown in Table 1, the constrained lightweight network architecture search method (hereinafter referred to as OURS) used in this paper was compared with several other methods for the patellar X-ray pattern recognition task. These comparison algorithms include the hand-designed classic convolutional neural network ResNet50, the representative reinforcement learning-based neural architecture search algorithm in the field of automated neural architecture search, the ENAS method proposed by Pham and Hieu in the paper "Efficient neural architecture search via parameters sharing.", the gradient optimization-based neural architecture search algorithm, the DARTS method proposed by Liu and Hanxiao in the paper "Darts: Differentiable architecture search.", and the evolutionary algorithm-based neural architecture search algorithm, the NSGANet method proposed by Lu and Zhichao in the paper "Nsga-net: neural architecture search using multi-objective genetic algorithm." This invention demonstrates superior performance in patellar X-ray image recognition. By optimizing the network structure through constraints and adaptive pruning, it not only surpasses manual design methods and automated search algorithms in accuracy, but also improves computational efficiency by streamlining model parameters and floating-point operations. These advantages make the invention particularly effective in resource-constrained environments, facilitating deployment on medical devices and edge computing equipment, achieving breakthroughs in both theoretical performance and practical application efficiency.

Claims

1. A patellar X-ray image recognition method based on a lightweight network architecture search under constraints, characterized in that: The steps are as follows: Step 1: Collect and preprocess patellar X-ray images with I real class labels, obtain the preprocessed patellar X-ray image set and divide it into training set D tra and validation set D val ; Step 2: Construct a convolutional neural network consisting of several MobileNetV3 structure convolution modules, several maximum pooling modules, and several average pooling modules; Step 3: Construct a population with convolutional neural networks as individuals and perform iterative evolution to obtain the individual with the best performance as the optimal patellar X-ray image recognition model A. best ; Step 3.1, parameter definition and initialization, including: population size is N, initial pruning rate is , the total number of iterations is T, the current number of iterations is t, and the training cycle is , the crossover probability is , the mutation probability is ; Step 3.2: Randomly initialize N convolutional neural networks to obtain the t-th generation population P consisting of N individuals. t ; Step 3.3, the t-th generation population P t After decoding the N individuals in the t-th generation population P, t N convolutional neural networks in ; Step 3.4: Set the training set D tra Enter the t-generation population P respectively t The N convolutional neural networks in the t-th generation are processed to obtain the t-th generation population P t The predicted category labels of the N convolutional neural networks in the training set D tra The true category label constructs the cross entropy loss function, so as to t The N convolutional neural networks in are trained to obtain the t-th generation population P after training t ; The validation set D val Input the t-th generation population P after training respectively t The N convolutional neural networks are processed to obtain the predicted category labels of N convolutional neural networks and compared with the validation set D val The true category label of the t-th generation population P is calculated t The classification accuracy of each individual; Step 3.5: Calculate the t-th generation population P t The penalty term CV of the i-th individual t,i ; Step 3.6: Calculate the population P of the tth generation t The fitness value F of the i-th individual t,i ; Step 3.7: Use the binary tournament selection strategy to select the tth generation population P t Each time two parent individuals are selected, and the fitness values ​​of the two selected parent individuals are compared, and the parent individual with higher fitness value is retained to join the tth generation mating pool P t ', until the mating pool P t 'Until the number of individuals in is N; Step 3.8: Based on the adaptive pruning rate, the mating pool P of the tth generation t 'After pruning, the pruned t-th generation population AP is obtained t '; Step 3.9, for the t-th generation mating pool P t 'Perform crossover mutation to obtain the tth generation population C after mutation t ': Step 3.10: AP t ' and C t 'Merge into the t-th generation offspring population Q of size N t ; Step 3.11, P t With Q t After merging, a t-generation mixed population M with a size of 2N is formed t , and according to M t The fitness value of each individual in M ​​is selected by the roulette wheel selection mechanism. t N individuals are selected from the population to form the t+1 generation population P t+1 ; Step 3.12: After assigning t+1 to t, if t>T, then the T-th generation population P is obtained. T , and execute step 3.13, otherwise, return to step 3.3 and execute sequentially; Step 3.13: For the T-th generation population P T The fitness values ​​of the N individuals are sorted in descending order, and the individual with the largest fitness value is taken as the optimal patellar X-ray image recognition model A. best , which is used to identify the input patellar X-ray image and obtain the best predicted category label.

2. The method for patella X-ray image recognition based on a lightweight network architecture search under constraints according to claim 1, characterized in that: In step 2, each convolution module includes a point-by-point convolution layer, a depth-separable convolution layer, an SE attention layer, and an inverted residual connection, wherein the number of filters in the depth-separable convolution layer is in the range of 1 to 512; let the structural length L of the convolutional neural network be the total number of convolution modules, maximum pooling modules, and average pooling modules; In step 3.2, t=1 is first initialized; then the convolutional neural network is encoded to obtain an individual, and the length of the individual is the total number of modules in the convolutional neural network. Each number in the individual corresponds to a module in the convolutional neural network, where the convolution module is represented by a number between [1,512], the average pooling module is represented by a number greater than 0.5, and the maximum pooling module is represented by a number less than 0.

5. The order of the numbers in the individual represents the connection order between layers in the convolutional neural network; In step 3.5, the t-th generation population P is calculated using formula (1). t The penalty term CV of the i-th individual t,i ; (1) In formula (1), Params t,i represents the t-th generation population P t The number of parameters in the i-th individual, Flops t,i represents the t-th generation population P t The floating-point number of the i-th individual; Params max Indicates the upper limit of the number of parameters for each given individual, Flops max Indicates the upper limit of floating-point operations for each given individual; L t,i represents the t-th generation population P t The structural length of the i-th individual; 、 and Represent three penalty factors respectively; In step 3.6, the t-th generation population P is calculated using formula (2). t The fitness value F of the i-th individual t,i ; (2) In formula (2), Accuracy t,i Represents the t-generation population P t The classification accuracy of the i-th individual.

3. The method for patella X-ray image recognition based on a lightweight network architecture search under constraints according to claim 2, characterized in that: The step 3.8 is performed as follows: Step 3.8.1, for the t-generation mating pool P t The individuals in 'are arranged in descending order according to their own classification accuracy, and the first population of the tth generation after arrangement is obtained; Step 3.8.2, for the t-generation mating pool P t The individuals in 'are arranged in descending order according to their own fitness functions to obtain the second population of the tth generation after arrangement; Step 3.8.3: Calculate the mating pool P of the tth generation using formula (3) t The complexity of the i-th individual in ' t,i , thus, for the t-th generation mating pool P t The individuals in 'are arranged in ascending order according to their own complexity, and the third population of the tth generation is obtained after arrangement; (3) Step 3.8.4: Select the first k individuals from the first population of the tth generation, the second population of the tth generation, and the third population of the tth generation after permutation and form the temporary population AP of the tth generation. t ; Step 3.8.5: Calculate the adaptive pruning rate of the tth generation using formula (4) : (4) Step 3.8.6, based on , for AP t The number of filters of each convolution module of each individual in is changed to obtain the pruned t-th generation population AP t '.

4. The method for patella X-ray image recognition based on a lightweight network architecture search under constraints according to claim 3, characterized in that: The step 3.9 is performed as follows: Step 3.9.1: Select the t-th generation mating pool P through a binary tournament. t 'Select two non-repeated parent individuals p t,1 and p t,2 and use the crossover probability a to p t,1 and p t,2 Perform single-point crossover to obtain two offspring individuals o of the tth generation t,1 and o t,1 Join the tth generation candidate offspring population C t In, until C t The number of neutron generation individuals is N-3k; Step 3.9.2, use mutation probability b to change C t Each offspring individual in the mutation operation is performed to obtain the mutated population C t '.

5. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the patella X-ray image recognition method according to any one of claims 1 to 4, and the processor is configured to execute the program stored in the memory.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the patella X-ray image recognition method according to any one of claims 1 to 4 are executed.

Images