A crack image recognition method based on constraint modulation and related equipment

Abstract

The application relates to the technical field of crack identification, and provides a crack image identification method based on constraint modulation and related equipment, which comprises the following steps: a crack identification model with a multilayer network is constructed, and a plurality of target crack images are obtained; the crack identification model is initialized to obtain an initial structure generation distribution of the crack identification model; a fine line analysis constraint function, a resolution constraint function and a hardware constraint function are designed, and the initial structure generation distribution is modulated and sampled by using the fine line analysis constraint function, the resolution constraint function and the hardware constraint function to obtain a modulation structure population; the crack identification model is iteratively updated based on the modulation structure population and all the target crack images to obtain a final crack identification model; and the final crack identification model is used for crack identification of a to-be-identified crack image to obtain a crack identification result of the to-be-identified crack image. The method can improve the accuracy of crack image identification.

Description

Technical Field

[0001] This application relates to the field of crack recognition technology, and in particular to a method and related equipment for crack image recognition based on constraint modulation. Background Technology

[0002] Crack identification, a key technical problem in structural health monitoring and industrial inspection, focuses on accurately extracting subtle and weakly continuous crack features from complex backgrounds. Since cracks typically exhibit elongated structures with small widths and low contrast, they are highly susceptible to resolution degradation and noise interference during image acquisition and subsequent processing, thus placing high demands on the model's structural representation capabilities.

[0003] For crack recognition tasks, existing methods mainly rely on manually designed convolutional neural network structures or automatically generated network structures through neural network architecture search methods. For the former, deep networks with fixed structures are usually used for training. However, such methods are difficult to balance the ability to express fine-grained features with computational resource constraints during the structure design process, which can easily lead to insufficient ability of the model to preserve crack details, thereby affecting the recognition accuracy.

[0004] For neural network architecture search methods, existing techniques typically employ structure generation mechanisms based on random search, reinforcement learning, or evolutionary algorithms to generate and filter candidate networks. These methods generally first generate candidate network structures in a predefined search space, and then eliminate structures that do not meet the conditions through performance evaluation and constraint determination. However, these methods do not consider task characteristics and constraints during the generation phase, leading to a large number of generated structures being deemed invalid in subsequent stages, thus reducing search efficiency.

[0005] For example, in some existing technologies, network structures are generated through evolutionary algorithms, and individuals are selected based on performance indicators or resource constraints. However, the constraint mechanism mainly affects the selection process after generation and does not participate in the generation process itself. This approach has the following drawbacks: because the constraints are not involved in the construction of the generation probability, the generation of candidate structures still has strong randomness, which can easily lead to a large number of network structures that do not meet the requirements of thin-line structure expression or resolution, resulting in low efficiency in the search process and increased consumption of computational resources.

[0006] Furthermore, for crack recognition tasks, multi-layer convolutional structures and downsampling structures are generally used for feature extraction. However, this type of method causes a gradual decrease in spatial resolution during continuous downsampling, resulting in the gradual disappearance of fine crack features in deep networks, thus affecting the final recognition performance. Simultaneously, in practical deployment, the model must also meet the limitations of computational complexity and storage resources. Existing methods typically treat these constraints as independent conditions for posterior judgment, making it difficult to effectively control them during the structure generation stage, thus leading to uncertainty in the feasibility of the generated structure.

[0007] In summary, existing technologies generally suffer from the following problems in network structure generation: First, the generation process does not reflect task-related constraints, resulting in insufficient matching between the structure and the requirements for crack feature representation; second, the constraint mechanism mainly operates in the posterior selection stage, making it difficult to influence the generation process itself and reducing search efficiency; third, there is a lack of a unified modeling mechanism between resolution preservation and resource constraints, which easily leads to an imbalance between performance and feasibility in the generated structure. These problems collectively result in low accuracy in crack image recognition. Summary of the Invention

[0008] This application provides a constraint modulation-based crack image recognition method and related equipment, which can solve the problem of low accuracy in crack image recognition.

[0009] In a first aspect, embodiments of this application provide a crack image recognition method based on constraint modulation, the crack image recognition method comprising: Construct a crack recognition model with a multi-layer network and acquire multiple target crack images; The crack recognition model is initialized to obtain the initial structure generation distribution of the crack recognition model; the initial structure generation distribution is used to describe the structure generation probability of each layer of the network in the crack recognition model. We design a fine-line analytical constraint function, a resolution constraint function, and a hardware constraint function, and use these functions to modulate and sample the initial structure generation distribution to obtain a modulated structure population. The fine-line analytical constraint function is used to constrain the crack recognition model from ignoring fine-line cracks, the resolution constraint function is used to constrain the crack recognition model from annihilating crack features at different spatial resolutions, and the hardware constraint function is used to constrain the storage usage of the crack recognition model from exceeding the device capacity. Based on the modulation structure population and all target crack images, the crack recognition model is iteratively updated to obtain the final crack recognition model; The final crack recognition model is used to identify cracks in the crack image to obtain the crack recognition result.

[0010] Optionally, the analytical constraint function for the thin line is: ; in, This represents the analytical constraint function value for the thin line. This indicates the number of network layers in the crack recognition model. Represents the global modulation intensity base value. Indicates the first The layer attenuation coefficient of a layered network. Indicates the first The initial structure of the layered network, This represents the crack width tolerance threshold. For the first The nonlinear mapping error function of the local receptive field of the layer network relative to the input crack width.

[0011] Optionally, the resolution constraint function is: ; in, This represents the value of the resolution constraint function. This represents the overall cumulative spatial reduction ratio of the crack recognition model. This represents the tolerance threshold for lossless analysis. This represents the critical threshold for crack feature annihilation. Indicates the decay curvature, Indicates the initial decay rate. This represents a very small constant.

[0012] Optionally, the hardware constraint function is: ; in, This represents the value of the hardware constraint function. and For modulation intensity parameters, This represents the upper limit of computational complexity. Indicates the maximum memory capacity. Indicates the total computational complexity. The maximum active tensor usage of memory resources: ; ; in, Indicates the first The computational cost of layered networks, Indicates the peak activation memory usage function. This represents the network structure encoding.

[0013] Optionally, the initial structure generation distribution is modulated and sampled using a thin-line analytical constraint function, a resolution constraint function, and a hardware constraint function to obtain a modulated structure population, including: The initial structure generation distribution is modulated using the fine-line analytical constraint function, the resolution constraint function, and the hardware constraint function to obtain the constraint-driven generation distribution; The constraint-driven generation distribution is sampled to obtain the modulation structure population.

[0014] Optionally, the initial structure generation distribution is modulated using a thin-line analytical constraint function, a resolution constraint function, and a hardware constraint function to obtain a constraint-driven generation distribution, including: Through the formula: ; Computational constraint-driven generation distribution ; in, This represents the initial structure generation distribution. This represents the normalization constant.

[0015] Optionally, the modulation structure population includes multiple structure individuals, and the structure individuals include the network structure of each layer of the network in the crack recognition model. Based on the modulation structure population and all target crack images, the crack recognition model is iteratively updated to obtain the final crack recognition model, including: Based on each individual structure in the modulation structure population, the network structure of each layer of the crack recognition model is set, and the crack recognition model is used to perform crack classification tasks based on all target crack images. Calculate the fitness of the crack identification model for the crack classification task; Sort all fitness values ​​from largest to smallest, and select the top fitness values ​​of the individual structures as elite structures. The modulation structure population is iteratively updated based on all elite structures to obtain the final structure population. Based on the final structural population, the network structure of each layer in the crack recognition model is set to obtain the final crack recognition model.

[0016] Optionally, the modulation structure population is iteratively updated based on all elite structures to obtain the final structure population, including: Increment the iteration count by 1; The generation probability parameters of each layer of the network are calculated based on all elite structures, and the structure population is calculated based on all generation probability parameters. Determine if the number of iterations is greater than or equal to the iteration threshold; If so, then the structural population will be taken as the final structural population; Otherwise, the structural population is used as the modulation structural population, and the steps of setting the network structure of each layer in the crack recognition model according to the modulation structural population are returned, and the crack recognition model is used to perform crack classification tasks based on all target crack images.

[0017] Optionally, the generation probability parameters for each layer of the network are calculated based on all elite architectures, including: Through the formula: ; Calculate the first The second iteration Generation probability parameters of layered networks ; in, Indicates the smooth update rate. Indicates the first The second iteration Generation probability parameters of layered networks Indicates the number of elite structures. Represents a set of elite structures. For indicator functions, The value is 1 at time. The value is 0 at this time. This indicates the candidate operation options.

[0018] Secondly, embodiments of this application provide a crack image recognition device based on constraint modulation, comprising: The building module is used to construct a crack recognition model with a multi-layer network and acquire multiple target crack images; The initialization module is used to initialize the crack recognition model and obtain the initial structure generation distribution of the crack recognition model. The initial structure generation distribution is used to describe the structure generation probability of each layer of the network in the crack recognition model. The modulation module is used to design the fine-line analytical constraint function, resolution constraint function, and hardware constraint function, and to modulate and sample the initial structure generation distribution using these functions to obtain a modulated structure population. The fine-line analytical constraint function is used to constrain the crack recognition model from ignoring fine-line cracks, the resolution constraint function is used to constrain the crack recognition model from annihilating crack features at different spatial resolutions, and the hardware constraint function is used to constrain the storage usage of the crack recognition model from exceeding the device capacity. The iterative module is used to iteratively update the crack recognition model based on the modulation structure population and all target crack images to obtain the final crack recognition model. The identification module is used to identify cracks in the crack image to be identified using the final crack identification model, and obtain the crack identification result of the crack image to be identified.

[0019] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the aforementioned constraint modulation-based crack image recognition method.

[0020] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned constraint modulation-based crack image recognition method.

[0021] The above-mentioned solution in this application has the following beneficial effects: In the embodiments of this application, a crack recognition model with a multi-layer network is constructed, and multiple target crack images are acquired. Then, the crack recognition model is initialized to obtain the initial structure generation distribution of the crack recognition model. Then, a fine-line analytical constraint function, a resolution constraint function, and a hardware constraint function are designed. The initial structure generation distribution is modulated and sampled using the fine-line analytical constraint function, the resolution constraint function, and the hardware constraint function to obtain a modulated structure population. Then, based on the modulated structure population and all target crack images, the crack recognition model is iteratively updated to obtain the final crack recognition model. Finally, the final crack recognition model is used to perform crack recognition on the crack image to be identified to obtain the crack recognition result of the crack image to be identified. Specifically, by designing fine-line analytical constraint functions, resolution constraint functions, and hardware constraint functions, and using these constraints to modulate the generation distribution, constraints are imposed on the model in three aspects: attention level of fine-line cracks, feature analysis under different spatial resolutions, and storage usage. This allows for the introduction of guidance on crack feature expression capabilities and resource limitations during the model structure generation stage. Based on the modulated structure population, the crack recognition model is iteratively updated, enabling the generation distribution to gradually converge towards structural regions with high fitness and satisfying the constraint conditions. This improves the final crack recognition model's ability to analyze crack images and its feature mining capabilities. Using the final crack recognition model to identify crack images can effectively improve the accuracy of crack image recognition.

[0022] Other beneficial effects of this application will be described in detail in the following detailed description section. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1A flowchart of a constraint modulation-based crack image recognition method provided in an embodiment of this application; Figure 2 A schematic diagram of the structure of a constraint modulation-based crack image recognition device provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0025] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0026] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0027] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0028] To address the low accuracy of existing crack image recognition methods, this application provides a constraint modulation-based crack image recognition method. This method constructs a crack recognition model with a multi-layer network and acquires multiple target crack images. The crack recognition model is then initialized to obtain an initial structure generation distribution. Fine-line analytical constraint functions, resolution constraint functions, and hardware constraint functions are designed and used to modulate and sample the initial structure generation distribution, resulting in a modulation structure population. Based on the modulation structure population and all target crack images, the crack recognition model is iteratively updated to obtain a final crack recognition model. Finally, the final crack recognition model is used to identify cracks in the crack images to be identified, yielding the crack recognition results. Specifically, by designing fine-line analytical constraint functions, resolution constraint functions, and hardware constraint functions, constraints are imposed on the model in three aspects: attention level of fine-line cracks, feature analysis under different spatial resolutions, and storage usage. The constraints are used to modulate the generation distribution, which can guide the crack feature expression ability and resource constraints in the model structure generation stage. Based on the modulated structure population, the crack recognition model is iteratively updated, so that the generation distribution gradually converges to the structure region with high fitness and satisfying the constraint conditions. This improves the final crack recognition model's ability to analyze crack images and its feature mining ability. Using the final crack recognition model to identify crack images can effectively improve the accuracy of crack image recognition.

[0029] The following is an exemplary description of the constraint modulation-based crack image recognition method provided in this application.

[0030] like Figure 1 As shown, the constraint modulation-based crack image recognition method provided in this application includes the following steps: Step 11: Construct a crack recognition model with a multi-layer network and acquire multiple target crack images.

[0031] The aforementioned target crack image can be an image of a concrete pavement, etc.

[0032] For example, the crack recognition model described above has multiple interconnected networks, and each network layer can be composed of computational units such as convolution.

[0033] Step 12: Initialize the crack identification model to obtain the initial structure generation distribution of the crack identification model.

[0034] The aforementioned initial structure generation distribution is used to describe the structure generation probability of each layer of the network in the crack recognition model. The network structure includes convolution kernel size, stride, channel size, etc.

[0035] For example, initialize a set of distribution parameters ,in For the first The initial distribution parameters of the layer network, The number of network layers is represented, and the probability distribution is generated by defining the structure based on the distribution parameters. ,in For the first Structural variables of the layer network (such as kernel size, stride, channel size, etc.).

[0036] Step 13: Design the thin-line analytical constraint function, resolution constraint function, and hardware constraint function, and use the thin-line analytical constraint function, resolution constraint function, and hardware constraint function to modulate and sample the initial structure generation distribution to obtain the modulated structure population.

[0037] The aforementioned fine-line analytical constraint function is used to constrain the crack recognition model from ignoring fine-line cracks, the resolution constraint function is used to constrain the crack recognition model from annihilating crack features at different spatial resolutions, and the hardware constraint function is used to constrain the storage usage of the crack recognition model from exceeding the device's capacity.

[0038] In some embodiments of this application, the steps of designing the fine-line analytical constraint function, resolution constraint function, and hardware constraint function, and using the fine-line analytical constraint function, resolution constraint function, and hardware constraint function to modulate and sample the initial structure generation distribution to obtain the modulated structure population include: The first step is to design the fine-line analytical constraint function, the resolution constraint function, and the hardware constraint function.

[0039] Specifically, the analytical constraint function for the thin line mentioned above is: ; in, This represents the analytical constraint function value for the thin line. This indicates the number of network layers in the crack recognition model. Represents the global modulation intensity base value. Indicates the first The layer attenuation coefficient of a layered network. , As the attenuation control factor, Indicates the first The initial structure of the layered network, This represents the crack width tolerance threshold. For the first The nonlinear mapping error function of the local receptive field of the layer network relative to the input crack width.

[0040] The resolution constraint function mentioned above is: ; in, This represents the value of the resolution constraint function. This represents the overall cumulative spatial reduction ratio of the crack recognition model. This represents the tolerance threshold for lossless analysis. This represents the critical threshold for crack feature annihilation. Indicates the decay curvature, Indicates the initial decay rate. This represents a very small constant.

[0041] The above hardware constraint function is: ; in, This represents the value of the hardware constraint function. and For modulation intensity parameters, This represents the upper limit of computational complexity. Indicates the maximum memory capacity. Indicates the total computational complexity. The maximum active tensor usage of memory resources: ; ; in, Indicates the first The computational cost of layered networks, Indicates the peak activation memory usage function. This represents the network structure encoding.

[0042] The second step involves modulating the initial structure generation distribution using the fine-line analytical constraint function, the resolution constraint function, and the hardware constraint function to obtain the constraint-driven generation distribution.

[0043] Specifically, through the formula: ; Computational constraint-driven generation distribution .

[0044] in, This represents the initial structure generation distribution. This represents the normalization constant.

[0045] The third step is to sample the constraint-driven generation distribution to obtain the modulation structure population.

[0046] The aforementioned modulation structure population includes multiple structural individuals, which include the network structure of each layer of the crack recognition model.

[0047] For example, the constraint-driven generation distribution is sampled to obtain the adjusted network structure of each layer, and then integrated to obtain the modulation structure population.

[0048] Step 14: Based on the modulation structure population and all target crack images, iteratively update the crack recognition model to obtain the final crack recognition model.

[0049] In some embodiments of this application, the steps of iteratively updating the crack recognition model based on the modulation structure population and all target crack images to obtain the final crack recognition model include: The first step is to define the network structure of each layer of the crack recognition model according to each individual structure in the modulation structure population, and then use the crack recognition model to perform crack classification based on all target crack images.

[0050] For example, based on the network structure of each layer in the structural individuals of the modulation structure population, the corresponding network layer in the crack recognition model is set. For instance, if the network structure in the structural individual is: 3*3 convolutional kernel, stride 2, then the convolutional unit of the corresponding network layer in the crack recognition model is set to 3*3 convolutional kernel, stride 2. This process is performed on each layer of the network in the crack recognition model. Then, each target crack image is input into the crack recognition model to perform the crack classification task, and the crack recognition result of each target crack image is output (which may be a label, etc., used to describe whether a crack exists in the target crack image).

[0051] The second step is to calculate the fitness of the crack recognition model in the crack classification task.

[0052] For example, a fitness function is introduced. For each network structure in the modulation structure population, it is decoded into a complete crack recognition model. The model is then trained for a short time (e.g., 5 training epochs) using all target crack images. The F1 score or accuracy of its crack classification is then calculated on the validation set, and the calculated performance index is used as the fitness value of the network structure.

[0053] The third step is to sort all fitness values ​​from largest to smallest, and then select the top fitness values ​​of the individual structures as elite structures.

[0054] For example, the number of elite structures is a preset value.

[0055] The fourth step is to iteratively update the modulation structure population based on all elite structures to obtain the final structure population.

[0056] Specifically, increment the iteration count by 1; the initial iteration count is 0.

[0057] The generation probability parameters of each layer of the network are calculated based on all elite structures, and the structure population is calculated based on all generation probability parameters; the structure population includes the network structure of each layer.

[0058] Determine whether the number of iterations is greater than or equal to the iteration number threshold.

[0059] If so, the structural population will be taken as the final structural population.

[0060] Otherwise, the structural population is used as the modulation structural population, and the steps of setting the network structure of each layer in the crack recognition model according to the modulation structural population are returned, and the crack recognition model is used to perform crack classification tasks based on all target crack images.

[0061] For example, through the formula: ; Calculate the first The second iteration Generation probability parameters of layered networks .

[0062] in, Indicates the smooth update rate. , Indicates the first The second iteration Generation probability parameters of layered networks Indicates the number of elite structures. Represents an elite structure set. For indicator functions, The value is 1 at time. The value is 0 at this time. Indicates the candidate operation options, i.e., the first An index of all possible structural configurations in a layered network. This indicates that the candidate operation option is in the elite structure. It exists in.

[0063] Then, according to the structure generation probability distribution defined in step 12, the structure population is calculated based on all generation probability parameters, i.e., using the formula: ; Calculate the probability of structure generation The structure population is obtained by modulating the generated distribution using fine-line analytical constraint functions, resolution constraint functions, and hardware constraint functions, and then sampling the modulated generated distribution.

[0064] The fifth step is to define the network structure of each layer in the crack recognition model based on the final structural population, thus obtaining the final crack recognition model.

[0065] For example, the individual with the highest fitness can be selected from the final structure population as the final structure individual, and the corresponding network layer in the crack recognition model can be set according to the network structure of each network layer in the final structure individual.

[0066] Step 15: Use the final crack recognition model to perform crack recognition on the crack image to be identified, and obtain the crack recognition result of the crack image to be identified.

[0067] The above images of cracks to be identified are images that need to be identified, such as images of concrete pavements that need to be identified to determine whether cracks exist.

[0068] For example, the image of the crack to be identified is input into the final crack identification model for processing, and the crack identification result of the image to be identified is output. This result can indicate whether there is a crack in the image to be identified.

[0069] It is worth mentioning that by designing fine-line analytical constraint functions, resolution constraint functions, and hardware constraint functions, and using these constraints to modulate the generated distribution, guidance on crack feature expression capabilities and resource constraints can be introduced during the model structure generation stage. Based on the modulated structure population, the crack recognition model is iteratively updated, enabling the generated distribution to gradually converge towards structural regions with high fitness and satisfying the constraint conditions. This improves the final crack recognition model's ability to analyze crack images and its feature mining capabilities. Using the final crack recognition model to identify crack images can effectively improve the accuracy of crack image recognition.

[0070] The method of this application will be illustrated below with a specific example.

[0071] In this embodiment, the maximum number of layers in the convolutional neural network to be searched is defined. Each floor The structure variable encoding consists of four discrete options, forming a set containing... The search space for candidate operations is shown in Table 1.

[0072] Table 1 Encoded Content ; Therefore, the first The structural variables of a layer are represented as one-hot encoded combinations. Total coding length of candidate individuals .

[0073] Based on prior physical knowledge of crack detection and the limitations of the target hardware, this embodiment constrains the modulation function. Perform a specific instantiation.

[0074] 1. Analytic constraint function for thin lines Specific calculations: It acts on local receptive field structures, emphasizing shallow, fine-grained expressive power, targeting input images with a width of approximately For minute cracks in pixels, this embodiment sets a crack width tolerance threshold. Dimensionless (meaning the relative deviation between the receptive field and the target receptive field is allowed to be no more than 50%); global modulation intensity baseline. Hierarchical attenuation control factor .

[0075] Among them, the local receptive field error function Specifically, it is a function of the cumulative stride of this layer and the kernel size, calculated as follows: ; ; Here Set to twice the input crack width (i.e., 10 pixels). When the... The layer uses a step size The operation, It will increase dramatically, especially if it occurs in the shallow layer ( When it is smaller), because A value close to 1 results in a larger penalty weight, thus limiting the number of shallow downsampling operations to preserve fine lines. Simultaneously, the target receptive field is adaptively determined based on the input crack width: the wider the crack, the more shallow downsampling operations are allowed to reduce computational load; the finer the crack, the fewer shallow downsampling operations are allowed to preserve fine lines. Therefore, the number of shallow downsampling operations is adaptively adjusted according to the input crack width.

[0076] 2. Resolution constraint function Specific calculations: This applies to global spatial resolution constraints and controls the overall feature map size variation. It defines the cumulative spatial reduction ratio for the entire network. (That is, the ratio of the input image size to the final feature map size). Lossless parsing tolerance threshold. (i.e., downsampling by 8 times or less, such as) Crack feature annihilation critical threshold. (That is, downsampling exceeds 32 times, such as when the feature map size is smaller than...) Transition range control parameters: , Minimal dead zone probability .

[0077] Calculation example: If a network structure contains 5 layers with a stride of 2, then .at this time , The output value is The weight of this structure in the probability distribution will approach zero, achieving hard truncation of the search space.

[0078] 3. Hardware constraint functions Specific calculations: The target hardware platform is a low-power edge computing platform, whose equivalent computing power is characterized by the upper limit of computational complexity and the upper limit of peak active memory. Therefore, the constraints are set as follows: Upper limit of computational complexity (Unit: MFLOPs, millions of floating-point operations). Maximum memory capacity. (Unit: MB, peak active memory). Modulation intensity , .

[0079] computational complexity The estimation formula uses standard convolution FLOPs for calculation: ; in Channel expansion factor The decision, specifically, is the first Number of input channels of the layer The number of output channels in the previous layer is determined by the number of output channels in the previous layer. ,in Based on the current layer channel expansion factor Sure, It varies with the step size.

[0080] Additional notes: For different operator types, the computational complexity adopts the corresponding computational model: when the operator type is standard convolution, the standard convolution calculation formula is used; when it is depthwise separable convolution, the computational complexity consists of two parts: channel-wise convolution and pointwise convolution; when it is an identity mapping, its computational complexity is approximately zero.

[0081] Apart from the aforementioned constraint modulation function, all unreasonable network structures generated by the algorithm in this embodiment, such as identity skip connections, are considered invalid and will not be evaluated further if the feature map sizes do not match during the generation process.

[0082] This embodiment uses population size. Number of elite individuals Maximum iteration algebra .

[0083] Step 1: Initialization phase (Generation 0).

[0084] First, for the maximum number of layers The search space, each layer contains Combining candidate structure operations to initialize the distribution parameter set. The probability of each operation being selected within each layer follows a uniform distribution, i.e., the initial probability. Based on this, the original generated distribution is constructed. and will sequentially with the thin line analytical constraint function Resolution constraint function and hardware constraint functions Multiplying yields the constraint-driven generation distribution. The unnormalized weight form; to avoid global normalization constants The solution is difficult, so this step adopts a layer-by-layer independent normalization approximation method. That is, the modulated weights of each candidate option within each layer are normalized to determine the sampling probability, thus preserving the modulating effect of the constraint function on the generation probability. This layer-by-layer normalization is an approximate implementation of the original distribution. Subsequently, the roulette wheel selection method is used to drive the distribution based on this constraint. Sub-independent sampling, generating a collection of The initial population of structurally encoded individuals .

[0085] Step 2: Fitness assessment.

[0086] In each generation of evolution, each individual in the population... Decoded into a specific PyTorch convolutional neural network model, where the operator type, channel size, kernel size, and stride of each layer strictly follow the encoding. The specified structural variables. This embodiment uses a dataset of concrete pavement crack images collected in a real industrial application scenario. This dataset contains 2000 grayscale images with a resolution of 224×224 pixels, each labeled with a binary classification label of cracked / no crack by professionals. The dataset is divided into a training set (1200 images), a validation set (200 images), and a test set (600 images) in a 6:1:3 ratio. Before inputting into the model, all images are uniformly scaled to 224×224 pixels and normalized. To complete the fitness evaluation within a limited time, this embodiment randomly selects a dataset of the original size from the training set. The proxy training subset is used to perform the following for each candidate model: Fast training for 10 epochs, and calculating the crack binary classification F1-Score on the corresponding validation subset as the main fitness function. Furthermore, this embodiment incorporates a secondary hardware filtering mechanism during the fitness evaluation phase: for each candidate model, its inference latency (Latency) is measured and peak memory usage is monitored under actual hardware simulation conditions; if the Latency exceeds... If a memory overflow error occurs within milliseconds, the fitness of that individual is directly set to 0. This further eliminates non-deployable structures, ensuring the engineering feasibility of the final output model.

[0087] Step 3: Elite selection and distribution parameter update.

[0088] After calculating the fitness of all individuals in the current generation, according to Sort the population by value from high to low, and select the top-ranked population. The highest fitness individuals constitute the elite subset. Subsequently, for each individual in the elite subset, the first... Statistical analysis was performed on the layer structure selection results, and the candidate operation options in each layer were statistically analyzed. exist The number of times it appears in an elite individual. Distribution parameter set. The update follows a smooth update rule; specifically, for the first... Layer Generation probability parameter of class structure option , its first The value of the generation is determined by the first We obtain the following by weighted fusion of historical parameters of the generation and the empirical frequencies of the current elite subset: ; After the update is complete, the new set of distribution parameters will be... Resubstitute the original generated distribution And again with the constraint function , and Combined, a constraint-driven generation distribution for the next generation is generated, which is then used for subsequent population sampling.

[0089] Step 4: Convergence determination and output.

[0090] Repeat steps 2 and 3 until the number of generations reaches the preset maximum number of iterations. Or, if the entropy of the distribution parameter set is detected to decrease to a preset threshold. The search is terminated early when stability is maintained. After the search process is completed, the structure encoding individual with the highest fitness value is selected from all historical evaluation individuals, and it is decoded into a complete convolutional neural network architecture. This architecture is used as the optimal network structure searched by the method of this invention for the current crack image classification task and the target hardware platform, and is output. After decoding the optimal network structure into a complete model, it is trained on the model for 25 training epochs using all training data, and its crack classification performance is evaluated on an independent test set. The final F1 score of the optimal structure is obtained as the final performance index of the method of this invention on this task. To verify the effectiveness of the method of this application, this embodiment is compared with the following two baseline search algorithms under the same search time budget: 1. Random Search + Hard Filtering: Randomly generate structures, directly discarding models that exceed memory limits or have excessive downsampling, training only compliant models. During comparison, the total number of searches is the product of the population size and the maximum number of iterations. In this embodiment, the number of times is 1200.

[0091] 2. Standard Evolutionary Algorithm + Penalty Term: The standard regular evolutionary algorithm adds hardware constraints as a fitness penalty term to the loss function, using the same population size and maximum number of iterations as the method in this application.

[0092] The comparison results are shown in Table 2: Table 2. Mean results and standard deviation of the 5 experiments ; FLOPs represents floating-point numbers.

[0093] Experimental data show that the proposed method, by directly applying three types of constraint functions to the structure generation probability distribution, achieves an effective structure sampling rate approaching 100%, significantly higher than the comparative methods (0.71%, 52.08%), thus avoiding wasted computational resources on undeployable or ineffective candidate architectures. Simultaneously, the optimal architecture found has the lowest computational complexity (FLOPs) and peak activation memory, meeting the deployment requirements of the target hardware platform. Furthermore, the number of shallow downsampling operations for the optimal structure is adaptively determined by the target crack width—in this embodiment, the crack width in the dataset is relatively large, thus allowing for more shallow downsampling to compress computation; for fine crack scenarios with smaller crack widths, the fine-line analytical constraint function limits the number of shallow downsampling operations with a smaller target receptive field to preserve fine cracks, demonstrating the adaptability of the constraint modulation mechanism to crack width.

[0094] The following is an exemplary description of the constraint modulation-based crack image recognition device provided in this application.

[0095] like Figure 2 As shown, this application provides a constraint modulation-based crack image recognition device 200, which includes: Module 201 is used to build a crack recognition model with a multi-layer network and acquire multiple target crack images; Initialization module 202 is used to initialize the crack recognition model and obtain the initial structure generation distribution of the crack recognition model; the initial structure generation distribution is used to describe the structure generation probability of each layer of the network in the crack recognition model. The modulation module 203 is used to design the fine-line analytical constraint function, resolution constraint function, and hardware constraint function, and to modulate and sample the initial structure generation distribution using the fine-line analytical constraint function, resolution constraint function, and hardware constraint function to obtain a modulated structure population. The fine-line analytical constraint function is used to constrain the crack recognition model from ignoring fine-line cracks, the resolution constraint function is used to constrain the crack recognition model from annihilating crack features at different spatial resolutions, and the hardware constraint function is used to constrain the storage usage of the crack recognition model from exceeding the device capacity. The iteration module 204 is used to iteratively update the crack recognition model based on the modulation structure population and all target crack images to obtain the final crack recognition model. The identification module 205 is used to identify cracks in the crack image to be identified using the final crack identification model, and obtain the crack identification result of the crack image to be identified.

[0096] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0097] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0098] like Figure 3 As shown, an embodiment of this application provides a terminal device, wherein the terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 3 The diagram shows only one processor, a memory D101, and a computer program D102 stored in the memory D101 and executable on at least one processor D100. When the processor D100 executes the computer program D102, it implements the steps in any of the above-described method embodiments.

[0099] Specifically, when processor D100 executes computer program D102, it constructs a crack recognition model with a multi-layer network and acquires multiple target crack images. Then, it initializes the crack recognition model to obtain the initial structure generation distribution. Next, it designs fine-line analytical constraint functions, resolution constraint functions, and hardware constraint functions, and uses these functions to modulate and sample the initial structure generation distribution to obtain a modulated structure population. Based on the modulated structure population and all target crack images, it iteratively updates the crack recognition model to obtain the final crack recognition model. Finally, it uses the final crack recognition model to perform crack recognition on the crack image to obtain the crack recognition result. Specifically, by designing fine-line analytical constraint functions, resolution constraint functions, and hardware constraint functions, constraints are imposed on the model in three aspects: attention level of fine-line cracks, feature analysis under different spatial resolutions, and storage usage. The constraints are used to modulate the generation distribution, which can guide the crack feature expression ability and resource constraints in the model structure generation stage. Based on the modulated structure population, the crack recognition model is iteratively updated, so that the generation distribution gradually converges to the structure region with high fitness and satisfying the constraint conditions. This improves the final crack recognition model's ability to analyze crack images and its feature mining ability. Using the final crack recognition model to identify crack images can effectively improve the accuracy of crack image recognition.

[0100] The processor D100 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0101] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may be an external storage device of the terminal device D10, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device D10. Furthermore, the memory D101 may include both internal and external storage units of the terminal device D10. The memory D101 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory D101 can also be used to temporarily store data that has been output or will be output.

[0102] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0103] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0104] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to the constraint modulation-based crack image recognition method apparatus / terminal device, recording media, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0105] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0106] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0107] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention.

Claims

1. A crack image recognition method based on constraint modulation, characterized in that, include: Construct a crack recognition model with a multi-layer network and acquire multiple target crack images; The crack recognition model is initialized to obtain the initial structure generation distribution of the crack recognition model; the initial structure generation distribution is used to describe the structure generation probability of each layer of the network in the crack recognition model; Design a fine-line analytical constraint function, a resolution constraint function, and a hardware constraint function, and use these functions to modulate and sample the initial structure generation distribution to obtain a modulated structure population. The fine-line analytical constraint function is used to constrain the crack recognition model from ignoring fine-line cracks, the resolution constraint function is used to constrain the crack recognition model from annihilating crack features at different spatial resolutions, and the hardware constraint function is used to constrain the storage usage of the crack recognition model from exceeding the device capacity. Based on the modulation structure population and all target crack images, the crack recognition model is iteratively updated to obtain the final crack recognition model. The final crack recognition model is used to identify cracks in the image to be identified, and the crack recognition result of the image to be identified is obtained.

2. The crack image recognition method according to claim 1, characterized in that, The analytical constraint function for the thin line is: ； in, This represents the analytical constraint function value for the thin line. This indicates the number of network layers in the crack recognition model. Represents the global modulation intensity base value. Indicates the first The layer attenuation coefficient of a layered network. Indicates the first The initial structure of the layered network, This represents the crack width tolerance threshold. For the first The nonlinear mapping error function of the local receptive field of the layer network relative to the input crack width.

3. The crack image recognition method according to claim 2, characterized in that, The resolution constraint function is: ； in, This represents the value of the resolution constraint function. This represents the overall cumulative spatial reduction ratio of the crack recognition model. This represents the tolerance threshold for lossless analysis. This represents the critical threshold for crack feature annihilation. Indicates the decay curvature. Indicates the initial decay rate. This represents a very small constant.

4. The crack image recognition method according to claim 3, characterized in that, The hardware constraint function is: ； in, This represents the value of the hardware constraint function. and For modulation intensity parameters, This represents the upper limit of computational complexity. Indicates the maximum memory capacity. This represents the total computational complexity. The maximum active tensor usage of memory resources: ； ； in, Indicates the first The computational cost of layered networks, Indicates the peak activation memory usage function. This represents the network structure encoding.

5. The crack image recognition method according to claim 4, characterized in that, The process of modulating and sampling the initial structure generation distribution using the thin-line analytical constraint function, resolution constraint function, and hardware constraint function to obtain a modulated structure population includes: The initial structure generation distribution is modulated using the fine-line analytical constraint function, resolution constraint function, and hardware constraint function to obtain a constraint-driven generation distribution; The constraint-driven generation distribution is sampled to obtain the modulation structure population.

6. The crack image recognition method according to claim 5, characterized in that, The step of modulating the initial structure generation distribution using the fine-line analytical constraint function, resolution constraint function, and hardware constraint function to obtain a constraint-driven generation distribution includes: Through the formula: ； Computational constraint-driven generation distribution ; in, This represents the initial structure generation distribution. This represents the normalization constant.

7. The crack image recognition method according to claim 1, characterized in that, The modulation structure population includes multiple structure individuals, and each structure individual includes the network structure of each layer of the network in the crack recognition model. The step of iteratively updating the crack recognition model based on the modulation structure population and all target crack images to obtain the final crack recognition model includes: Based on each individual structure in the modulation structure population, the network structure of each layer of the crack recognition model is set, and the crack recognition model is used to perform crack classification tasks based on all target crack images. Calculate the fitness of the crack recognition model in the crack classification task; Sort all fitness values ​​from largest to smallest, and select the top fitness values ​​of the individual structures as elite structures. The modulation structure population is iteratively updated based on all elite structures to obtain the final structure population. Based on the final structure population, the network structure of each layer in the crack recognition model is set to obtain the final crack recognition model.

8. The crack image recognition method according to claim 7, characterized in that, The iterative update of the modulation structure population based on all elite structures to obtain the final structure population includes: Increment the iteration count by 1; The generation probability parameters of each layer of the network are calculated based on all elite structures, and the structure population is calculated based on all generation probability parameters. Determine whether the number of iterations is greater than or equal to the iteration number threshold; If so, then the aforementioned structural population shall be taken as the final structural population; Otherwise, the structural population is used as the modulation structural population, and the steps of setting the network structure of each layer of the crack recognition model according to the modulation structural population, and performing crack classification tasks using the crack recognition model based on all target crack images are returned.

9. The crack image recognition method according to claim 8, characterized in that, The calculation of generation probability parameters for each layer of the network based on all elite structures includes: Through the formula: ； Calculate the first The second iteration Generation probability parameters of layered networks ; in, Indicates the smooth update rate. Indicates the first The second iteration Generation probability parameters of layered networks Indicates the number of elite structures. Represents an elite structure set. For indicator functions, The value is 1 at time. The value is 0 at this time. Indicates candidate operation options.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the constraint modulation-based crack image recognition method as described in any one of claims 1 to 9.

Images