Patella X-ray image recognition method for lightweight network architecture search under constraint conditions

Abstract

The invention discloses a patella X-ray image recognition method for lightweight network architecture search under constraint conditions, which comprises the following steps of: 1, collecting marked patella X-ray image data, preprocessing, and dividing into a training set and a verification set; 2, constructing a Shenconvolution neural network population through initialization; 3, training individuals in the population by using an X-ray data set, and using fitness function values based on constraints as fitness values of the individuals; 4, parent individuals are selected according to the fitness values of the individuals in the population, a filial population is generated through filter trimming and crossover variation of a self-adaptive trimming rate, and finally environment selection is carried out; and 5, iteratively optimizing the network structure until a preset condition is met, and extracting an optimal framework as a patella X-ray image recognition model. According to the method, the convolutional neural network architecture adapted to the patella X-ray image data set can be automatically searched, so that accurate classification of the patella X-ray images is realized, especially in application scenes with equipment hardware constraints.

Description

Technical Field

[0001] The present invention belongs to the field of computer and machine learning, in particular to the sub-fields of automated machine learning and neural network architecture search. Specifically, the present invention is a method for using a lightweight network architecture search technology under constraints to recognize patellar X-ray images. Background Art

[0002] Convolutional neural networks are gradually becoming a key component in the field of computer vision and artificial intelligence in the field of medical image analysis. They play an important role in key medical tasks such as image classification, anomaly detection, image segmentation and reconstruction, aiming to improve the accuracy and efficiency of recognition. In the field of medical image processing, a well-designed convolutional neural network can significantly improve the accuracy of image recognition. In addition, considering the multiple limitations such as storage space and computing efficiency that medical devices may face in practical applications, it is crucial to develop lightweight, efficient convolutional neural networks that can meet specific resource constraints. This is not only related to the performance of the model, but also its practicality and feasibility in practical environments. Therefore, studying and designing convolutional neural networks that can still maintain excellent performance under resource-constrained conditions is of great significance to promote the application of AI in the field of medical image recognition.

[0003] In the field of medical X-ray image recognition, neural network design methods are mainly divided into manual design and automated neural architecture search. Manually designed networks such as VGG16 and ResNet, although they perform well in image classification tasks, they require a lot of domain knowledge and manual adjustments, which is a challenge for most practitioners. Although the NAS method can automatically discover high-performance network designs, it also has some shortcomings.

[0004] Designing networks using neural architecture search is mainly based on three search strategies: reinforcement learning, gradient optimization, and evolutionary algorithms.

[0005] First, although reinforcement learning-based neural architecture search can automatically explore network structures when designing X-ray image recognition networks, this method has high computational costs and risks of instability and overfitting. This may lead to overfitting of the model to the training data in practical applications, and poor generalization ability to new, unseen data.

[0006] Secondly, although the neural architecture search method based on gradient optimization is computationally efficient, it may be limited to a predefined set of operations and may be challenging to optimize for discrete decisions. This means that in a specific patellar X-ray image recognition task, it may not be possible to find the network structure that best suits the task.

[0007] Furthermore, the neural architecture search method based on evolutionary computation has shown its unique advantages. This method can effectively explore a wide search space to discover network architectures with excellent performance by simulating the process of natural selection. However, this method still has some challenges and limitations when applied to specific tasks. For example, the process may be very time-consuming and the generated network structure may be too complex. In order to overcome these challenges, researchers have begun to explore incorporating constraints into the NAS process to guide the search process in a direction that is more suitable for X-ray image tasks. For example, by introducing geometric and topological constraints, the accuracy of segmentation tasks can be improved, and this method can also be used to guide the recognition and classification of patellar X-ray images.

[0008] In summary, although NAS based on evolutionary computation shows potential in patellar X-ray image recognition, further research and optimization are still needed to address its challenges in computational efficiency, network complexity, and adaptability to the specific needs of medical X-ray images. Summary of the invention

[0009] The present invention aims to overcome the limitations of the prior art and provides a patella X-ray image recognition method for lightweight network architecture search under constraints, in order to find a more effective neural network structure that better meets resource constraints for the recognition and classification of patella X-ray images, thereby improving the accuracy of patella X-ray image classification and solving the problem of difficulty in deploying classification models due to constraints such as hardware storage of medical equipment.

[0010] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0011] The characteristic of the patella X-ray image recognition method of the present invention by searching a lightweight network architecture under a constraint condition is that it is performed according to the following steps:

[0012] Step 1: Collect and preprocess patella X-ray images with I real class labels to obtain a set of preprocessed patella X-ray images and divide them into a training set D tra and validation set D val ;

[0013] Step 2, construct a convolutional neural network consisting of several convolutional modules of MobileNetV3 structure, several maximum pooling modules and several average pooling modules; each convolutional module includes a point-by-point convolutional layer, a depth-separable convolutional layer, an SE attention layer and an inverted residual connection, wherein the number of filters in the depth-separable convolutional layer is in the range of 1 to 512; let the structure length L of the convolutional neural network be the total number of convolutional modules, maximum pooling modules and average pooling modules;

[0014] Step 3: Construct a population with convolutional neural networks as individuals and perform iterative evolution to obtain the individual with the best performance as the optimal patellar X-ray image recognition model A best ;

[0015] Step 3.1: Parameter definition and initialization, including: population size is N, initial pruning rate is , the total number of iterations is T, the current number of iterations is t, and the training cycle is , the crossover probability is , the mutation probability is ;

[0016] Step 3.2, initialize t=1;

[0017] Encode the convolutional neural network to obtain an individual, and the length of the individual is the total number of modules in the convolutional neural network. Each number in the individual corresponds to a module in the convolutional neural network, wherein the convolution module is represented by a number between [1,512], the average pooling module is represented by a number greater than 0.5, and the maximum pooling module is represented by a number less than 0.5. The order of the numbers in the individual represents the order of connections between layers in the convolutional neural network;

[0018] Randomly initialize N convolutional neural networks to obtain the t-th generation population P consisting of N individuals t ;

[0019] Step 3.3: Set the tth generation population P t After decoding the N individuals in the tth generation population P t N convolutional neural networks in;

[0020] Step 3.4: Set the training set D tra Enter the tth generation population P respectively t The N convolutional neural networks in the t-th generation are processed to obtain the t-th generation population P t The predicted category labels of the N convolutional neural networks in are compared with the training set D tra The real category label constructs the cross entropy loss function, so as to t The N convolutional neural networks in are trained to obtain the tth generation population P after training. t ;

[0021] The validation set D val Input the tth generation population P after training respectively t The N convolutional neural networks are processed to obtain the predicted category labels of N convolutional neural networks and compared with the validation set D val The true category label of the t-th generation population P is calculated t The classification accuracy of each individual;

[0022] Step 3.5: Calculate the t-th generation population P using formula (1) t The penalty term CV of the i-th individual t,i ;

[0023] (1)

[0024] In formula (1), Params t,i represents the t-th generation population P t The number of parameters in the ith individual, Flops t,i represents the t-th generation population P t The floating point number of the i-th individual; Params max Indicates the upper limit of the number of parameters for each individual, Flops max Indicates the upper limit of the floating-point number of operations for each given individual; L t,i represents the t-th generation population P t The structural length of the i-th individual; , and Represent three penalty factors respectively;

[0025] Step 3.6: Use formula (2) to calculate the t-th generation population P t The fitness value F of the i-th individual t,i ;

[0026] (2)

[0027] In formula (2), Accuracy t,i Represents the tth generation population P t The classification accuracy of the i-th individual;

[0028] Step 3.7: Use the binary tournament selection strategy to select the tth generation population P t Each time two parent individuals are selected, and the fitness values ​​of the two selected parent individuals are compared, and the parent individual with a higher fitness value is retained and added to the t-th generation mating pool P t ', until the mating pool P t 'Until the number of individuals in is N;

[0029] Step 3.8: Based on the adaptive pruning rate, the mating pool P of the tth generation t 'After pruning, the pruned t-th generation population AP is obtained t ';

[0030] Step 3.9: For the t-th generation mating pool P t 'Perform crossover mutation to obtain the tth generation population C after mutation t':

[0031] Step 3.10: AP t ' and C t 'Merge into the t-th generation offspring population Q of size N t ;

[0032] Step 3.11: P t With Q t After merging, a t-generation mixed population M with a size of 2N is formed t , and according to M t The fitness value of each individual in M ​​is selected by the roulette wheel selection mechanism. t N individuals are selected from the population to form the t+1th generation population P t+1 ;

[0033] Step 3.12: After assigning t+1 to t, if t>T holds, it means that the Tth generation population P is obtained. T , and execute step 3.13, otherwise, return to step 3.3 and execute sequentially;

[0034] Step 3.13: For the T-th generation population P T The fitness values ​​of the N individuals are sorted in descending order, and the individual with the largest fitness value is taken as the optimal patella X-ray image recognition model A. best , which is used to identify the input patellar X-ray image and obtain the best predicted category label.

[0035] The characteristic of the patella X-ray image recognition method of the light-weight network architecture search under the constraint condition described in the present invention is that the step 3.8 is performed as follows:

[0036] Step 3.8.1: For the t-th generation mating pool P t The individuals in 'are arranged in descending order according to their own classification accuracy, and the first population of the tth generation is obtained after arrangement;

[0037] Step 3.8.2: For the t-th generation mating pool P t The individuals in 'are arranged in descending order according to their own fitness functions to obtain the second population of the tth generation after arrangement;

[0038] Step 3.8.3: Calculate the t-generation mating pool P using formula (3): t The complexity of the i-th individual in ' t,i , thus, for the t-th generation mating pool P t The individuals in 'are arranged in ascending order according to their own complexity, and the third population of the tth generation is obtained after arrangement;

[0039] (3)

[0040] Step 3.8.4: Select the first k individuals from the arranged first population of the tth generation, the second population of the tth generation, and the third population of the tth generation to form the temporary population AP of the tth generation. t ;

[0041] Step 3.8.5: Calculate the adaptive pruning rate of the tth generation using formula (4): :

[0042] (4)

[0043] Step 3.8.6, based on , for AP t The number of filters in each convolution module of each individual in is changed to obtain the pruned t-th generation population AP t '.

[0044] Further, the step 3.9 is performed as follows:

[0045] Step 3.9.1: Select the t-th generation mating pool P through a binary tournament. t 'Select two non-repeating parent individuals p t,1 and p t,2 , and use the crossover probability a to p t,1 and p t,2 Perform single-point crossover to obtain two offspring individuals o of the tth generation t,1 and t,1 Join the tth generation candidate offspring population C t In, until C t The number of neutron generation individuals reaches N-3k;

[0046] Step 3.9.2: Change C with mutation probability b t Each offspring individual in the mutated population C is obtained by performing mutation operation t '.

[0047] The electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the patella X-ray image recognition method, and the processor is configured to execute the program stored in the memory.

[0048] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the patella X-ray image recognition method when the computer program is executed by a processor.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] 1. The present invention designs a highly flexible variable-length coding scheme, aiming to provide an automated network architecture search strategy for the recognition task of patellar X-ray images. The scheme can go deep into each layer of the network and freely search for any number of filters to fully cover every possible dimension of the network architecture, thereby achieving efficient recognition of patellar X-ray images. Through this extensive and in-depth search, the present invention can identify network architectures with excellent performance. At the same time, by maintaining population diversity, the present invention effectively improves the generalization ability of the model, reduces the risk of overfitting, and ensures the stability and robustness of the recognition method.

[0051] 2. The present invention proposes a filter pruning strategy with an adaptive pruning rate, which brings significant advantages to patella X-ray image recognition. First, the strategy achieves an organic combination of global optimization and local optimization by automatically adjusting the pruning rate with evolutionary iterations, thereby more effectively exploring potential optimal models during the search process. Secondly, this strategy also helps to reduce the model size and reduce the computational complexity, making the model more suitable for deployment on resource-constrained devices. Ultimately, this innovative pruning strategy significantly improves the efficiency and practicality of patella X-ray image recognition without sacrificing classification accuracy.

[0052] 3. The present invention proposes a constraint-based fitness function. In addition to the classification accuracy, the model's parameter quantity, floating-point operation number and architecture length are incorporated into the optimization target as constraint penalty items. The function can dynamically adjust the search direction and optimize the network architecture under the premise of satisfying resource constraints. When the network architecture meets the resource constraints, the fitness function focuses on improving the classification accuracy; otherwise, the model parameters and calculation amount are optimized through the penalty items to guide the network to develop in a lighter direction. Such a design effectively balances the classification performance and resource consumption of the model. Therefore, under the action of this fitness function, a more lightweight and efficient network architecture can be found for the task of patellar X-ray image recognition.

[0053] 4. The present invention can effectively search for an adapted convolutional neural network architecture for any resource constraint. This architecture not only performs well in learning and identifying the image features of the patellar X-ray image dataset, and achieves high accuracy in the abnormal categories of patellar X-ray images, but also shows excellent performance in parameter quantity and floating-point operation; thanks to its small model parameters and efficient computing power, the network structure of the present invention can be well deployed on medical equipment, thereby achieving fast and accurate classification of patellar X-ray images. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of the method of the present invention;

[0055] Figure 2The structure diagram of the convolutional neural network designed for the present invention;

[0056] Figure 3 This is a schematic diagram of each category of X-ray films in the osteoporosis dataset used in the present invention. DETAILED DESCRIPTION

[0057] In this embodiment, Figure 1 As shown, a patella X-ray image recognition method based on a lightweight network architecture search under constraints is performed in the following steps:

[0058] Step 1: Collect n patellar X-ray image datasets with real category labels and obtain the training set D after preprocessing operations. tra and validation set D val , and the patellar X-ray images are divided into two categories of normal and abnormal labels, such as Figure 3 As shown;

[0059] Step 1.1. Obtain a total of n X-ray images, each image is represented as (h0, w0, c, 0), where h0 and w0 represent the original height and width of the image, c represents the number of channels of the image, and 0 represents the label category corresponding to the image. In this experiment, h and w are set to 48, and c=3.

[0060] Step 1.2: Pass the X-ray image of step 0.1 to the preprocessing function R(h, w) to perform the resize operation, thereby obtaining the preprocessed X-ray image (h, w, c, 0). Here, h and w define the height and width of the resized image, respectively.

[0061] Step 1.3: n X-ray images are fed into the partitioning function S(r), which first shuffles the image data randomly and then partitions the data set into the training set D according to the ratio r. tra and validation set D val .

[0062] Step 2: Construct a convolutional neural network consisting of several MobileNetV3 structure convolution modules, several maximum pooling modules and several average pooling modules; each convolution module includes a point-by-point convolution layer, a depth-wise separable convolution layer, an SE attention layer and an inverted residual connection, wherein the number of filters in the depth-wise separable convolution layer is in the range of 1 to 512; let the structure length L of the convolutional neural network be the total number of convolution modules, maximum pooling modules and average pooling modules;

[0063] For example Figure 2 The network architecture shown in the example is composed of four convolutional layers and two pooling layers stacked together.

[0064] Step 3: Construct a population with convolutional neural networks as individuals and perform iterative evolution to obtain the individual with the best performance as the optimal patellar X-ray image recognition model A best ;

[0065] Step 3.1, parameter definition, including: initial population size N = 20, initial pruning rate =20%, the total number of evolutionary iterations is T =20, the current number of iterations is t=1, and the training cycle is =450, the crossover probability is =0.9, the mutation probability is =0.2, number of categories I=2.

[0066] Step 3.2, initialize t=1;

[0067] Encode the convolutional neural network to obtain an individual, and the length of the individual is the total number of modules in the convolutional neural network. Each number in the individual corresponds to a module in the convolutional neural network, where the convolution module is represented by a number between [1,512], the average pooling module is represented by a number greater than 0.5, and the maximum pooling module is represented by a number less than 0.5. The order of numbers in the individual represents the connection order between layers in the convolutional neural network; Figure 2 The network architecture encoding of the example is 234 - 126 - 0.4 - 69 - 0.8 - 108;

[0068] Randomly initialize N convolutional neural networks to obtain the t-th generation population P consisting of N individuals t .

[0069] Step 3.3: Set the tth generation population P t After decoding the N individuals in the tth generation population P t N convolutional neural networks in;

[0070] Step 3.4: Set the training set D tra Enter the tth generation population P respectively t The N convolutional neural networks in the t-th generation are processed to obtain the t-th generation population P t The predicted category labels of the N convolutional neural networks in are compared with the training set D tra The real category label constructs the cross entropy loss function, so as to t The N convolutional neural networks in are trained to obtain the tth generation population P after training. t ;

[0071] The validation set D val Input the tth generation population P after training respectively tThe N convolutional neural networks are processed to obtain the predicted category labels of N convolutional neural networks and compared with the validation set D val The true category label of the t-th generation population P is calculated t The classification accuracy of each individual.

[0072] Step 3.5: Calculate the t-th generation population P using formula (1) t The penalty term CV of the i-th individual t,i ;

[0073] (1)

[0074] In formula (1), Params t,i represents the t-th generation population P t The number of parameters in the ith individual, Flops t,i represents the t-th generation population P t The floating point number of the i-th individual; Params max Indicates the upper limit of the number of parameters for each individual, which is 0.6M, Flops max Indicates the upper limit of floating point number of each given individual, which is 150M; L t,i represents the t-th generation population P t The structural length of the i-th individual; , and They represent three penalty factors, with values ​​of 0.5, 0.5 and 0.001 respectively.

[0075] Step 3.6: Use formula (2) to calculate the t-th generation population P t The fitness value F of the i-th individual t,i ;

[0076] (2)

[0077] In formula (2), Accuracy t,i Represents the tth generation population P t The classification accuracy of the ith individual in the population can be calculated by the constraint-based fitness function. When the network architecture meets the resource constraints, the fitness function focuses on improving the classification accuracy. Otherwise, the penalty term is used to optimize the model parameters and computational complexity, guiding the network to develop in a lighter direction.

[0078] Step 3.7: Use the binary tournament selection strategy to select the tth generation population P t Each time two parent individuals are selected, and the fitness values ​​of the two selected parent individuals are compared, and the parent individual with a higher fitness value is retained and added to the t-th generation mating pool P t', until the mating pool P t 'Until the number of individuals in is N;

[0079] Step 3.8: Based on the adaptive pruning rate, the mating pool P of the tth generation t 'After pruning, the pruned t-th generation population AP is obtained t ';

[0080] Step 3.8.1: For the t-th generation mating pool P t The individuals in ' are arranged in descending order according to their classification accuracy, and the first population of the tth generation after arrangement is obtained; classification accuracy is the key goal in the optimization process, and the top-ranked network architectures often show more significant performance improvement potential after filter pruning.

[0081] Step 3.8.2: For the t-th generation mating pool P t The individuals in ' are arranged in descending order according to their own fitness functions to obtain the second population of the tth generation after arrangement; its fitness value is a comprehensive consideration of the classification accuracy and constraint violation of the network architecture.

[0082] Step 3.8.3: Calculate the t-generation mating pool P using formula (3): t The complexity of the i-th individual in ' t,i , thus, for the t-th generation mating pool P t The individuals in ' are arranged in ascending order according to their own complexity to obtain the third population of the tth generation after arrangement; the smaller the complexity, the closer the network architecture is to the given resource constraints.

[0083] (3)

[0084] Step 3.8.4: Select the first k individuals from the arranged first population of the tth generation, the second population of the tth generation, and the third population of the tth generation to form the temporary population AP of the tth generation. t .

[0085] Step 3.8.5: Calculate the adaptive pruning rate of the tth generation using formula (4): :

[0086] (4)

[0087] As the number of search generations increases, the filter pruning rate is adjusted adaptively. This strategy allows for rapid convergence in the early stages of network architecture search while retaining sufficient flexibility to accommodate subsequent fine-tuning. As the search progresses, more focus is placed on small-scale refined searches, which not only improves search efficiency but also helps discover more refined and superior network structures.

[0088] Step 3.8.6, based on , for AP t The number of filters in each convolution module of each individual in is changed to obtain the pruned t-th generation population AP t '.

[0089] Step 3.9: For the t-th generation mating pool P t 'Perform crossover mutation to obtain the tth generation population C after mutation t ':

[0090] Step 3.9.1: Select the t-th generation mating pool P through a binary tournament. t 'Select two non-repeating parent individuals p t,1 and p t,2 , and use the crossover probability a to p t,1 and p t,2 Perform single-point crossover to obtain two offspring individuals o of the tth generation t,1 and t,1 Join the tth generation candidate offspring population C t In, until C t The number of neutron generation individuals reaches N-3k;

[0091] Step 3.9.2: Change C with mutation probability b t Each offspring individual in the mutated population C is obtained by performing mutation operation t ';

[0092] Four types of mutation operators are defined: 1. Insertion operation: randomly add a random pooling module or a convolution module with a random initial value at the selected position; 2. Delete operation: randomly delete a module at the selected position; 3. Modification operation: including modification of the pooling module type and the number of convolution module filters; 4. Replacement operation: replace the pooling module with a convolution module with a random initial value, or replace the convolution module with a random pooling module. The mutation operation is selected in the form of random selection to obtain the mutated population Ct'.

[0093] Step 3.10: AP t ' and C t 'Merge into the t-th generation offspring population Q of size N t ;

[0094] Step 3.11: P t With Q t After merging, a t-generation mixed population M with a size of 2N is formed t , and according to M t The fitness value of each individual in M ​​is selected by the roulette wheel selection mechanism. t N individuals are selected from the population to form the t+1th generation population Pt+1 ; If the individuals with the highest classification accuracy or the largest fitness value are not selected in the selection process, the elite retention strategy is adopted: directly add these elite individuals to the new generation population P t+1 And replace the individuals with the lowest classification accuracy and fitness value.

[0095] Step 3.12: After assigning t+1 to t, if t>T holds, it means that the Tth generation population P is obtained. T , and execute step 3.13, otherwise, return to step 3.3 and execute sequentially;

[0096] Step 3.13: For the T-th generation population P T The fitness values ​​of the N individuals are sorted in descending order, and the individual with the largest fitness value is taken as the optimal patella X-ray image recognition model A. best , which is used to identify the input patellar X-ray image and obtain the best predicted category label.

[0097] Step 4: Save the trained neural network model A best And the corresponding weight parameters, a patella X-ray image recognition model is obtained, thereby realizing the classification prediction of patella X-ray images.

[0098] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0099] In this embodiment, a computer-readable storage medium stores a computer program on the computer-readable storage medium, and the computer program executes the steps of the above method when executed by a processor.

[0100] Table 1

[0101]

[0102] As shown in Table 1, in the task of patellar X-ray pattern recognition, the patellar X-ray image recognition method of the constrained lightweight network architecture search adopted by the present invention (hereinafter referred to as ours) was compared with other methods. These comparison algorithms include the hand-designed classic convolutional neural network ResNet50, and the neural architecture search algorithm based on reinforcement learning, which is representative in the field of automated neural architecture search, the ENAS method proposed by Pham, Hieu in the paper "Efficient neural architecture search via parameters sharing.", and the neural architecture search algorithm based on gradient optimization, the DARTS method proposed by Liu, Hanxiao in the paper "Darts: Differentiable architecture search.", and the neural architecture search algorithm based on evolutionary algorithm, the NSGANet method proposed by Lu, Zhichao in the paper "Nsga-net: neural architecture search using multi-objective genetic algorithm." The present invention has demonstrated excellent performance in patellar X-ray image recognition. It optimizes the network structure through constraints and adaptive pruning mechanisms, not only surpassing the manual design method and automated search algorithm in terms of accuracy, but also improving the computational efficiency by streamlining the number of model parameters and floating-point operations. These advantages make the present invention particularly prominent in resource-constrained environments, and it is easy to deploy on medical devices and edge computing devices, achieving a double breakthrough in theoretical performance and practical application efficiency.

Claims

1. A patella X-ray image recognition method based on a lightweight network architecture search under constraints, characterized in that: The steps are as follows: Step 1: Collect and preprocess patella X-ray images with I real class labels to obtain a set of preprocessed patella X-ray images and divide them into a training set D tra and validation set D val ; Step 2, construct a convolutional neural network consisting of several convolutional modules of MobileNetV3 structure, several maximum pooling modules and several average pooling modules; each convolutional module includes a point-by-point convolutional layer, a depth-separable convolutional layer, an SE attention layer and an inverted residual connection, wherein the number of filters in the depth-separable convolutional layer is in the range of 1 to 512; let the structure length L of the convolutional neural network be the total number of convolutional modules, maximum pooling modules and average pooling modules; Step 3: Construct a population with convolutional neural networks as individuals and perform iterative evolution to obtain the individual with the best performance as the optimal patellar X-ray image recognition model A. best ; Step 3.1: Parameter definition and initialization, including: population size is N, initial pruning rate is , the total number of iterations is T, the current number of iterations is t, and the training cycle is , the crossover probability is , the mutation probability is ; Step 3.2, initialize t=1; Encode the convolutional neural network to obtain an individual, and the length of the individual is the total number of modules in the convolutional neural network. Each number in the individual corresponds to a module in the convolutional neural network, wherein the convolution module is represented by a number between [1,512], the average pooling module is represented by a number greater than 0.5, and the maximum pooling module is represented by a number less than 0.

5. The order of the numbers in the individual represents the order of connections between layers in the convolutional neural network; Randomly initialize N convolutional neural networks to obtain the t-th generation population P consisting of N individuals t ; Step 3.3: Set the tth generation population P t After decoding the N individuals in the tth generation population P t N convolutional neural networks in; Step 3.4: Set the training set D tra Enter the tth generation population P respectively t The N convolutional neural networks in the tth generation are processed to obtain the tth generation population P t The predicted category labels of the N convolutional neural networks in are compared with the training set D tra The real category label constructs the cross entropy loss function, so as to t The N convolutional neural networks in are trained to obtain the tth generation population P after training. t ; The validation set D val Input the tth generation population P after training respectively t The N convolutional neural networks are processed to obtain the predicted category labels of N convolutional neural networks and compared with the validation set D val The true category label of the t-th generation population P is calculated t The classification accuracy of each individual; Step 3.5: Calculate the t-th generation population P using formula (1) t The penalty term CV of the i-th individual t,i。 2. (1) In formula (1), Params t,i represents the t-th generation population P t The number of parameters in the ith individual, Flops t,i represents the t-th generation population P t The floating point number of the i-th individual; Params max Indicates the upper limit of the number of parameters for each individual, Flops max Indicates the upper limit of the floating-point number of operations for each given individual; L t,i represents the t-th generation population P t The structural length of the i-th individual; , and Represent three penalty factors respectively; Step 3.6: Use formula (2) to calculate the t-th generation population P t The fitness value F of the i-th individual t,i ; (2) In formula (2), Accuracy t,i Represents the tth generation population P t The classification accuracy of the i-th individual; Step 3.7: Use the binary tournament selection strategy to select the tth generation population P t Each time two parent individuals are selected, and the fitness values ​​of the two selected parent individuals are compared, and the parent individual with a higher fitness value is retained and added to the t-th generation mating pool P t ', until the mating pool P t 'Until the number of individuals in is N; Step 3.8: Based on the adaptive pruning rate, the mating pool P of the tth generation t 'After pruning, the pruned t-th generation population AP is obtained t '; Step 3.9: For the t-th generation mating pool P t 'Perform crossover mutation to obtain the tth generation population C after mutation t ': Step 3.10: AP t ' and C t 'Merge into the t-th generation offspring population Q of size N t ; Step 3.11: P t With Q t After merging, a t-generation mixed population M with a size of 2N is formed t , and according to M t The fitness value of each individual in M ​​is selected by the roulette wheel selection mechanism. t N individuals are selected from the population to form the t+1th generation population P t+1 ; Step 3.12: After assigning t+1 to t, if t>T holds, it means that the Tth generation population P is obtained. T , and execute step 3.13, otherwise, return to step 3.3 and execute sequentially; Step 3.13: For the T-th generation population P T The fitness values ​​of the N individuals are sorted in descending order, and the individual with the largest fitness value is taken as the optimal patella X-ray image recognition model A. best , which is used to identify the input patellar X-ray image and obtain the best predicted category label.

3. The patella X-ray image recognition method based on a lightweight network architecture search under constraints according to claim 1, characterized in that: The step 3.8 is performed as follows: Step 3.8.1: For the t-th generation mating pool P t The individuals in 'are arranged in descending order according to their own classification accuracy, and the first population of the tth generation is obtained after arrangement; Step 3.8.2: For the t-th generation mating pool P t The individuals in 'are arranged in descending order according to their own fitness functions to obtain the second population of the tth generation after arrangement; Step 3.8.3: Calculate the t-generation mating pool P using formula (3): t The complexity of the i-th individual in ' t,i , thus, for the t-th generation mating pool P t The individuals in 'are arranged in ascending order according to their own complexity, and the third population of the tth generation is obtained after arrangement; (3) Step 3.8.4: Select the first k individuals from the arranged first population of the tth generation, the second population of the tth generation, and the third population of the tth generation to form the temporary population AP of the tth generation. t ; Step 3.8.5: Calculate the adaptive pruning rate of the tth generation using formula (4): : (4) Step 3.8.6, based on , for AP t The number of filters in each convolution module of each individual in is changed to obtain the pruned t-th generation population AP t '.

4. The patella X-ray image recognition method based on a lightweight network architecture search under constraints according to claim 2, characterized in that: The step 3.9 is performed as follows: Step 3.9.1: Select the t-th generation mating pool P through a binary tournament. t 'Select two non-repeating parent individuals p t,1 and p t,2 , and use the crossover probability a to p t,1 and p t,2 Perform single-point crossover to obtain two offspring individuals o of the tth generation t,1 and t,1 Join the tth generation candidate offspring population C t In, until C t The number of neutron generation individuals reaches N-3k; Step 3.9.2: Change C with mutation probability b t Each offspring individual in the mutated population C is obtained by performing mutation operation t '.

5. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the patella X-ray image recognition method described in any one of claims 1-3, and the processor is configured to execute the program stored in the memory.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the patella X-ray image recognition method according to any one of claims 1 to 3 are executed.

Images