Multi-scale defect visual inspection method for high-pressure fuel pipes of motor vehicles based on digital twins

Abstract

The present invention relates to a multi-scale defect visual detection method for a motor vehicle high-pressure oil pipe based on digital twins, which belongs to the field of motor vehicle technology and solves the problems of high computational complexity, low efficiency and precision of existing defect detection algorithms. The physical entity of the motor vehicle high-pressure oil pipe to be detected is determined and its environmental information is collected; a digital twin is constructed according to the physical entity, wherein a high-pressure oil pipe three-dimensional model construction module constructs a high-pressure oil pipe three-dimensional model, a motor vehicle high-pressure oil pipe operating environment simulation module performs environmental simulation on the model according to the real-time load of the high-pressure oil pipe and the ambient temperature data, and a motor vehicle high-pressure oil pipe defect evolution prediction module uses an intelligent defect detection algorithm to identify the surface defects of the high-pressure oil pipe at each moment of the model, obtains the category, location information and size information of the defects, and also performs evolution simulation and risk factor analysis. The present invention realizes real-time online, high-precision and high-efficiency detection of surface defects of motor vehicle high-pressure oil pipes, and improves the service life of high-pressure oil pipes.

Description

Technical Field

[0001] The present invention relates to the field of motor vehicle technology, and in particular to a multi-scale defect visual detection method for a motor vehicle high-pressure oil pipe based on digital twins. Background Art

[0002] In the working process of fuel vehicles, the engine is one of its key parts and the core power source of the vehicle. What directly affects the working efficiency of the engine is the engine high-pressure oil pipe connecting the injection pump and the injector. The engine high-pressure oil pipe is responsible for delivering high-pressure fuel to the engine. Once the high-pressure oil pipe has a fault, such as oil leakage, it may cause engine failure at the least, and may cause the vehicle to spontaneously combust and cause major traffic accidents at the worst. Since the high-pressure oil pipe often undergoes various bending and stretching deformations, needs to withstand a variety of variable loads and sudden temperature increases, it often has defects such as pitting, cracks, and mesh defects. In this working environment, each defect may cause a major oil leakage accident. Therefore, defect identification and risk assessment of the engine oil pipe without affecting its normal operation are of great significance to improving the life of the high-pressure oil pipe and ensuring driving safety.

[0003] As for defect detection technology, with the development of machine vision automatic detection technology, machine vision detection has gradually replaced traditional manual detection. The machine vision detection system can avoid damage to the paint surface of the automobile oil pipe by obtaining the surface image of the automobile oil pipe; and the visual detection system can replace workers to work in complex and dangerous environments, thereby reducing the probability of safety accidents; at the same time, the visual automatic detection system also has the advantages of fast detection speed, high efficiency, and stable accuracy. However, since the high-pressure oil pipe is in a closed mechanical structure, it is difficult to observe its surface and interior when it is working, so it is difficult to realize real-time monitoring of the damage changes of the automobile engine high-pressure oil pipe during operation. In order to overcome the limitations of traditional defect detection methods and realize real-time monitoring of defects, deep learning target detection methods based on convolutional neural networks (CNN) have gradually emerged. It obtains image depth features through autonomous learning to identify defect features. The engine high-pressure oil pipe has the characteristics of multiple types, multiple scales, and irregular shapes. Therefore, designing a detection algorithm suitable for this type of defect is of great significance for real-time status monitoring of high-pressure oil pipes.

[0004] However, the existing two-stage visual inspection algorithm has high computational complexity and slow calculation speed; the single-stage inspection algorithm has low accuracy in detecting multi-scale, multi-type and irregular-shaped defects in automobile engine high-pressure oil pipes, and is very likely to miss tiny defects and easily confuse the edges of defects of various shapes, making it difficult to accurately classify defects. Summary of the invention

[0005] In order to solve the problems of high computational complexity, low efficiency and accuracy of existing defect detection algorithms, the present invention provides a multi-scale defect visual detection method for motor vehicle high-pressure oil pipes based on digital twins. The method realizes real-time monitoring of defects generated during the operation of motor vehicle high-pressure oil pipes by constructing a digital twin of the motor vehicle high-pressure oil pipe, and has higher detection accuracy and efficiency.

[0006] To solve the above problems, the present invention adopts the following technical solutions:

[0007] A multi-scale defect visual inspection method for high-pressure fuel pipes of motor vehicles based on digital twins, the method comprising the following steps:

[0008] Step 1: determine the physical entity of the motor vehicle high-pressure fuel pipe to be tested, use a force sensor and a temperature sensor to collect environmental information of the motor vehicle high-pressure fuel pipe and transmit the environmental information to the software port;

[0009] Step 2: Based on the physical entity of the motor vehicle high-pressure fuel pipe, a digital twin of the high-pressure fuel pipe is constructed, wherein the digital twin includes an engine high-pressure fuel pipe three-dimensional model construction module, an engine high-pressure fuel pipe operating environment simulation module and an engine high-pressure fuel pipe defect evolution prediction module, wherein the engine high-pressure fuel pipe three-dimensional model construction module is used to construct a corresponding engine high-pressure fuel pipe three-dimensional model according to the physical entity of the engine high-pressure fuel pipe, the engine high-pressure fuel pipe operating environment simulation module is used to perform environmental simulation on the high-pressure fuel pipe three-dimensional model through Comsol simulation software according to the real-time load data of the engine high-pressure fuel pipe and the temperature data of the environment in which it is located acquired through the software port, the engine high-pressure fuel pipe defect evolution prediction module is used to identify the surface defects of the high-pressure fuel pipe at each moment in the three-dimensional model of the engine high-pressure fuel pipe by using an intelligent defect detection algorithm, obtain the category, location information and size information of the surface defects of the high-pressure fuel pipe, and perform evolution simulation on the surface defects of the high-pressure fuel pipe on a time scale to obtain an evolution curve of defect size and shape characteristics, and perform a risk factor analysis on the evolution trend of the surface defects of the high-pressure fuel pipe;

[0010] The intelligent defect detection algorithm includes a backbone network, a pixel-level global attention module, a neck network and a head network. The feature map obtained after the original high-pressure oil pipe surface image is processed by the convolution layer and the pooling layer is input into the backbone network. After the backbone network performs feature extraction operations, feature maps of different scales are output in four stages respectively. The feature maps output from the first stage to the third stage are directly input into the neck network. The feature map output in the fourth stage is first input into the pixel-level global attention module. The pixel-level global attention module is connected in series with the channel dimension polarization self-attention mechanism and the space dimension polarization self-attention mechanism to extract the input feature map to obtain the final output feature map. The output feature map is input into the neck network. The neck network adopts a feature pyramid network. The feature pyramid network performs multi-scale feature extraction and then outputs a feature map set to the head network. The head network adopts a T-head structure to realize the output of the prediction result, wherein the prediction result includes the category and position of the surface defect of the high-pressure oil pipe.

[0011] Beneficial effects of the present invention: The digital twin-based multi-scale defect visual detection method for motor vehicle high-pressure oil pipes proposed by the present invention realizes real-time online detection of surface defects of motor vehicle high-pressure oil pipes during operation by constructing digital twins of motor vehicle high-pressure oil pipes, thereby improving the service life of motor vehicle high-pressure oil pipes and ensuring driving safety. An intelligent defect detection algorithm for multi-scale, multi-type, and irregularly shaped surface defects of motor vehicle high-pressure oil pipes is designed. A low-parameter pixel-level global attention module is introduced into the network structure of the intelligent defect detection algorithm based on the characteristics of surface defects of motor vehicle high-pressure oil pipes, which can realize high-precision and high-efficiency detection of surface defects of motor vehicle high-pressure oil pipes, laying a foundation for the formulation of maintenance strategies for motor vehicle high-pressure oil pipes. The present invention uses digital twin technology to visualize the working status of motor vehicle high-pressure oil pipes in a closed environment, and performs defect detection and risk assessment on them, effectively solving the problem of difficulty in detecting damage to components in a closed and harsh working environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a schematic diagram of a multi-scale defect visual detection method for a motor vehicle high-pressure oil pipe based on digital twins according to an embodiment of the present invention;

[0013] Figure 2 It is a schematic diagram of the network structure of the intelligent defect detection algorithm;

[0014] Figure 3 Schematic diagram of the network structure of the pixel-level global attention module. DETAILED DESCRIPTION

[0015] In order to realize real-time monitoring of the motor vehicle engine high-pressure oil pipe during operation without affecting the actual working state of the motor vehicle engine high-pressure oil pipe, the present invention provides an intelligent solution based on digital twins, that is, a multi-scale defect visual detection method for motor vehicle engine high-pressure oil pipes based on digital twins. The method utilizes digital twin technology to construct a 1:1 three-dimensional entity in a virtual space, and construct the same environment through simulation software, while synchronously updating environmental changes, visualizing the working state of the target entity, obtaining more operating information, and predicting the subsequent conditions that may occur in the motor vehicle engine high-pressure oil pipe, so that relevant personnel can take appropriate maintenance measures in time, thereby improving the service life of the motor vehicle engine high-pressure oil pipe and ensuring driving safety.

[0016] The present invention proposes a multi-scale defect visual detection method for a motor vehicle high-pressure oil pipe based on digital twins, which mainly includes the following steps:

[0017] Step 1: determine the physical entity of the motor vehicle high-pressure fuel pipe to be tested, use a force sensor and a temperature sensor to collect environmental information of the motor vehicle high-pressure fuel pipe and transmit the environmental information to the software port;

[0018] Step 2: Based on the physical entity of the motor vehicle high-pressure fuel pipe, a digital twin of the high-pressure fuel pipe is constructed, and the digital twin includes a high-pressure fuel pipe three-dimensional model construction module, a high-pressure fuel pipe operation environment simulation module and a high-pressure fuel pipe defect evolution prediction module, wherein the high-pressure fuel pipe three-dimensional model construction module is used to construct a corresponding high-pressure fuel pipe three-dimensional model according to the physical entity of the high-pressure fuel pipe, the high-pressure fuel pipe operation environment simulation module is used to perform environmental simulation on the high-pressure fuel pipe three-dimensional model through Comsol simulation software based on the real-time load data of the high-pressure fuel pipe and the temperature data of the environment in which it is located obtained through the software port, and the high-pressure fuel pipe defect evolution prediction module is used to use an intelligent defect detection algorithm to identify the surface defects of the high-pressure fuel pipe at each moment in the three-dimensional model of the high-pressure fuel pipe, obtain the category, location information and size information of the surface defects of the high-pressure fuel pipe, and simulate the evolution of the surface defects of the high-pressure fuel pipe on a time scale to obtain the evolution curve of the defect size and shape characteristics, and perform a hazard coefficient analysis on the evolution trend of the surface defects of the high-pressure fuel pipe;

[0019] The intelligent defect detection algorithm includes a backbone network, a pixel-level global attention module, a neck network and a head network. The feature map obtained after the original high-pressure oil pipe surface image is processed by the convolution layer and the pooling layer is input into the backbone network. After the backbone network performs feature extraction operations, feature maps of different scales are output in four stages. The feature maps output from the first to the third stages are directly input into the neck network. The feature map output from the fourth stage is first input into the pixel-level global attention module. The pixel-level global attention module is connected in series with the channel dimension polarization self-attention mechanism and the space dimension polarization self-attention mechanism to extract the input feature map and obtain the final output feature map. The output feature map is input into the neck network. The neck network adopts a feature pyramid network. The feature pyramid network performs multi-scale feature extraction and then outputs a feature map set to the head network. The head network adopts a T-head structure to output the prediction results, where the prediction results include the category and location of the surface defects of the high-pressure oil pipe.

[0020] Specifically, Figure 1As shown, the multi-scale defect visual detection method of motor vehicle high-pressure oil pipe based on digital twin proposed by the present invention includes three parts: physical entity, software control management and virtual entity. The method is based on physical entity, and uses physical entity to guide the construction of virtual entity. The environmental information of high-pressure oil pipe is collected through multiple sensors and transmitted to the software port (the control end for controlling virtual entity). Professionals review and correct the data and then update the parameters of the high-pressure oil pipe operation environment simulation module in the virtual environment. After the motor vehicle high-pressure oil pipe defect evolution prediction module obtains the evolution trend and risk factor analysis of the defect, the relevant information is transmitted to the software port. Professionals construct a preventive maintenance plan based on the information and perform preventive maintenance on the high-pressure oil pipe based on the plan. Among them, the virtual entity includes three modules: engine high-pressure oil pipe three-dimensional model construction module, engine high-pressure oil pipe operation environment simulation module and engine high-pressure oil pipe defect evolution prediction module. The engine high-pressure oil pipe three-dimensional model construction module is used to construct a 1:1 engine high-pressure oil pipe three-dimensional model of the entity according to the physical entity of the engine high-pressure oil pipe, and the geometric features and physical features of the entity are integrated. Catia software or solidworks software can be used to construct the engine high-pressure oil pipe three-dimensional model. The engine high-pressure oil pipe operation environment simulation module is used to simulate the environment of the three-dimensional virtual entity through the Comsol simulation software based on the real-time load data of the engine high-pressure oil pipe obtained by the force sensor and the temperature sensor and the temperature data of the environment, so as to simulate the actual working state of the engine high-pressure oil pipe during operation. The engine high-pressure oil pipe defect evolution prediction module mainly has the following functions: first, the intelligent defect detection algorithm is used to identify the surface defects of the high-pressure oil pipe at each moment in the three-dimensional model of the high-pressure oil pipe, and the category information and size information of the defects are read. The size information of the defects mainly refers to the large, medium and small in a broad sense, which is obtained based on the picture input into the algorithm and the definition of the defect scale by the international organization SPIE. When the ratio of the pixel blocks contained in the defect to the pixel blocks of the entire image is less than 0.12%, it is a small-scale target, and the rest are medium or large defects; secondly, the evolution of the surface defects of the high-pressure oil pipe is simulated on the time scale to obtain the evolution curve of the defect size and shape characteristics. Finally, the evolution trend of this defect is analyzed according to the evolution curve, so as to predict the overall danger of the high-pressure oil pipe parts. The engine high-pressure oil pipe defect evolution prediction module also transmits the results of the risk factor analysis to the software port. After analysis and discussion, professionals can obtain a maintenance plan for the surface defects of the high-pressure oil pipe. According to the above process, the digital twin method can be used to avoid sudden ruptures of the high-pressure oil pipe in subsequent work.

[0021] In order to realize online nondestructive detection of surface defects of high-pressure oil pipes, the present invention adopts a high-precision intelligent defect detection algorithm that can detect multi-scale, multi-type and irregular-shaped defects of high-pressure oil pipes. The overall architecture of the algorithm is as follows: Figure 2 As shown in the figure, it is divided into three parts: feature extraction (backbone network), feature fusion (neck network) and defect classification and positioning (head network). Among them, the backbone network uses the classic feature extraction network ResNet-50 neural network. The ResNet-50 neural network is a classic feature extraction model. By adding the BatchNorm layer and the residual structure, it can ensure that the network is deep enough while maintaining high detection accuracy. A standard residual block in the ResNet-50 neural network consists of two convolutional layers, and RELU is the activation function. In the feature extraction part, the original high-pressure oil pipe surface image is processed by a convolutional layer and a pooling layer to obtain a feature map with a size of 64 channels, a height of h, and a width of w. The feature map of size (64, h, w) is input into the backbone network for feature extraction. In the ResNet50 processing process, it goes through four stages respectively. The backbone network adopts the ResNet-50 neural network. The original high-pressure oil pipe surface image is processed by the convolution layer and the pooling layer to obtain a feature map of size (64, h, w). The feature map of size (64, h, w) is input into the ResNet-50 neural network for feature extraction. After four stages of feature extraction, four feature maps are output respectively. The feature maps output from the first to the third stages are directly input into the neck network. The feature map output from the fourth stage is first input into the pixel-level global attention module. The pixel-level global attention module extracts the input feature map by serially fusing the channel dimension polarization self-attention mechanism and the space dimension polarization self-attention mechanism to obtain the final output feature map, which is then input into the neck network.

[0022] In order to realize the detection of multi-scale defects in high-pressure oil pipes, the neck network of the model uses Feature Pyramid Network (FPN) for multi-scale feature extraction. Feature detection methods are mainly divided into three categories: single feature layer detection, multi-scale input detection and multi-scale feature map detection. In the convolutional network, the information contained in feature maps at different levels is different. The deep feature map contains rich semantic information, that is, category information, but due to the low pixel count, it contains less spatial information. The shallow feature map contains rich spatial information, but less semantic information. The backbone network can obtain feature maps of different scales output in four stages. The semantic information of the output feature map is gradually enriched and the spatial information is gradually reduced. In order to better utilize the information features at different scales, the feature maps of different scales obtained by feature extraction are combined, and the high-level feature map is upsampled and fused with the feature map of the previous layer, so that the feature map of this layer contains both high-level semantic information and low-level spatial information, so as to achieve efficient and accurate prediction of feature maps of different scales and complete the detection of minor defects. In the FPN network, the high-level feature map is upsampled by 2 times and added to the feature map of the previous layer, and then the fused feature map is upsampled by 2 times and added to the feature map of the next layer, and so on. At the same time, in order to meet the head network's requirements for the scale characteristics of the input feature map, two downsamplings are required to obtain feature maps with spatial sizes of (256, h / 16, w / 16) and (256, h / 32, w / 32) respectively. The feature maps obtained by each fusion constitute the neck output feature map set (or fused feature map), denoted as The head network will complete the prediction of defect type and defect location based on the feature maps in the set.

[0023] For the model head network, the T-head structure is used to achieve the output of prediction results, where the prediction results include the category and location of the surface defects of the high-pressure oil pipe. The T-head structure is different from the traditional single-stage detection output head. This structure realizes the interaction between the classification task and the position positioning task through the alignment task module to reduce the conflict between the results of the two types of tasks and achieve more accurate classification prediction and position positioning. The T-head structure consists of two parts: the feature conversion structure and the task alignment predictor (TAP).

[0024] like Figure 3As shown in the figure, in order to adapt to the characteristics of multi-type, multi-scale and irregular-shaped defects of high-pressure oil pipes, ensure the unity of classification tasks and location positioning tasks, and realize efficient comprehensive detection of complex defects of high-pressure oil pipes, a low parameter-pixel levelglobal (LPPLG) attention module is added between the backbone network and the neck network. The LPPLG attention module uses polarization operation to achieve channel and spatial pixel-by-pixel feature extraction while maintaining low parameter volume. At the same time, the halo structure is parallelized to extract the global features of the feature map, and then combined with the channel features and spatial features respectively to achieve accurate extraction of global features and local features. The aforementioned module optimizes the feature extraction process to complete the accurate extraction of irregular-shaped defects, and performs targeted fusion attention processing on the four scale feature maps output by ResNet-50, highlighting the spatial features and channel features of feature maps of different scales, so as to enhance the algorithm's recognition ability for feature maps of different scales. In view of the fact that the receptive field of each pixel in the output feature map of the fourth stage is large, contains more defect information, and has a more urgent requirement for pixel-by-pixel regression tasks, the LPPLG attention module is proposed. This module completes the pixel-by-pixel regression task with a low number of parameters by fusing the polarized self-attention module and the halo attention module. At the same time, it accurately processes the detailed features of the feature map and realizes efficient use of the advantages of the feature map output in the fourth stage.

[0025] The polarized self-attention mechanism includes two modules: the channel dimension polarized self-attention mechanism and the spatial dimension polarized self-attention mechanism. The present invention realizes the fusion of the two-dimensional self-attention by a series connection method. Common pixel-level tasks are usually complex and have a high number of parameters, but the polarized self-attention mechanism used in the present invention, through the polarization filtering method, ensures that the spatial dimension and the channel dimension are not compressed to a great extent while achieving high-precision pixel-by-pixel regression tasks. The polarization filtering mechanism is based on the light transmission principle similar to that of an optical lens, which only allows light orthogonal to the lateral direction to pass through. Similarly, when migrated to the algorithm, this mechanism can achieve complete compression of the features of one dimension while maintaining high resolution of the orthogonal dimension. In the channel dimension polarized self-attention mechanism used in the present invention, first, the input feature map X is subjected to dimensionality reduction processing through a convolution layer with a convolution kernel size of 1×1, so that the number of channels is halved, reducing the number of parameters in the subsequent process; secondly, the feature map is respectively reduced through a 1×1 convolution layer. The characteristic matrix With the weight matrix Multiply them to get new feature maps Q and V, where the channel dimension of Q is completely compressed to 1, and the channel dimension of V remains at a high level. Since the channel dimension of Q is completely compressed, in order to achieve the subsequent fusion of Q and V, it is necessary to enhance the information of Q through the Sigmoid function and expand the dimension. Finally, matrix multiplication is performed on Q and V to obtain the comprehensive weight , and restore the number of channels of the feature map to C / 2 through a 1×1 convolution layer to obtain the channel polarization self-attention weight The channel attention weight not only includes the pixel-level channel attention weight calculation of global information, but also needs to extract subtle channel features. The present invention uses the halo attention mechanism to extract subtle features. The halo attention mechanism converts the feature map into Extract features in parts, and perform SA (Blocked Local Self-Attention) operation on each part to extract subtle features. At the same time, before performing SA calculation, perform haloing operation to enlarge the receptive field of each part, integrate the edge information around the part, reduce the isolation of each part feature map, and achieve intercommunication between feature maps of each part. After extracting subtle features through halo attention, obtain the halo attention weight , further making the halo attention weight With channel polarization self-attention weight Perform matrix multiplication to obtain the comprehensive channel branch attention weight , the calculation method is shown in formula (1). Similarly, the spatial dimension is operated similarly to the channel dimension. First, a 1×1 convolution layer is used to convert the attention weights of the comprehensive channel branches into The feature map is converted into feature map Q and feature map V. For the Q feature, GlobalPooling is used to compress the spatial dimension and convert it into a size of 1×1; while the spatial dimension of the V feature remains at a relatively large level (H×W). Since the spatial dimension of the feature map Q is compressed, Softmax is used to enhance the information of the feature map Q. Then matrix multiplication is performed on the feature map Q and the feature map V, and then Reshape and Sigmoid are connected to keep all parameters between 0-1, and finally the spatial polarization self-attention weight is obtained. After extracting subtle features through halo attention, we get the halo attention weight , further making the halo attention weight With spatially polarized self-attention weights Perform matrix multiplication to obtain the comprehensive spatial branch attention weights , the calculation method is shown in formula (2). Finally, the final feature map is obtained according to formula (3) .

[0026] W ch ( X ') = W c × W ha 1 = F [ W | θ 1 (( σ 1 ( WV ( X '))) × F ( σ 2 ( WQ ( X '))))] × W ha 1 (1)

[0027] in, Represents the comprehensive weight of feature map Q and feature map V in the channel dimension, represents the parameters in the convolution, represents the Softmax function, represents the Sigmoid function, and are tensor deformation operators, and Represents the feature maps The corresponding feature matrix and weight matrix.

[0028] W sp ( X ') = W s × W ha 2 = F sg [ σ 5 F sm ( σ 3 ( F GP ( W Q ( X '))) × σ 4 ( W V ( X ')))] × W ha 2 (2)

[0029] in, is the Global Pooling function, , , are three tensor deformation operators.

[0030] (3)

[0031] in, represents the polarized attention weight, is the spatial dimension multiplication operator, is the channel dimension multiplication operator.

[0032] The digital twin-based multi-scale defect visual detection method for motor vehicle high-pressure oil pipes proposed in the present invention realizes real-time online detection of surface defects of motor vehicle high-pressure oil pipes during operation by constructing digital twins of motor vehicle high-pressure oil pipes, thereby improving the service life of motor vehicle high-pressure oil pipes and ensuring driving safety. An intelligent defect detection algorithm for multi-scale, multi-type and irregular-shaped surface defects of motor vehicle high-pressure oil pipes is designed. A low-parameter pixel-level global attention module is introduced into the network structure of the intelligent defect detection algorithm based on the characteristics of surface defects of motor vehicle high-pressure oil pipes, which can realize high-precision and high-efficiency detection of surface defects of motor vehicle high-pressure oil pipes and lay a foundation for the formulation of maintenance strategies for motor vehicle high-pressure oil pipes. The present invention uses digital twin technology to visualize the working state of motor vehicle high-pressure oil pipes in a closed environment, and performs defect detection and risk assessment on them, effectively solving the problem of difficulty in detecting damage to parts in a closed and harsh working environment.

[0033] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0034] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A multi-scale defect visual inspection method for high-pressure fuel pipes of motor vehicles based on digital twins, characterized in that: The following steps are involved: Step 1: determine the physical entity of the motor vehicle high-pressure fuel pipe to be tested, use a force sensor and a temperature sensor to collect environmental information of the motor vehicle high-pressure fuel pipe and transmit the environmental information to the software port; Step 2: Based on the physical entity of the motor vehicle high-pressure fuel pipe, a digital twin of the high-pressure fuel pipe is constructed, wherein the digital twin includes an engine high-pressure fuel pipe three-dimensional model construction module, an engine high-pressure fuel pipe operating environment simulation module and an engine high-pressure fuel pipe defect evolution prediction module, wherein the engine high-pressure fuel pipe three-dimensional model construction module is used to construct a corresponding engine high-pressure fuel pipe three-dimensional model according to the physical entity of the engine high-pressure fuel pipe, the engine high-pressure fuel pipe operating environment simulation module is used to perform environmental simulation on the high-pressure fuel pipe three-dimensional model through Comsol simulation software according to the real-time load data of the engine high-pressure fuel pipe and the temperature data of the environment in which it is located acquired through the software port, the engine high-pressure fuel pipe defect evolution prediction module is used to identify the surface defects of the high-pressure fuel pipe at each moment in the three-dimensional model of the engine high-pressure fuel pipe by using an intelligent defect detection algorithm, obtain the category, location information and size information of the surface defects of the high-pressure fuel pipe, and perform evolution simulation on the surface defects of the high-pressure fuel pipe on a time scale to obtain an evolution curve of the defect size and shape characteristics, and perform a risk factor analysis on the evolution trend of the surface defects of the high-pressure fuel pipe; The intelligent defect detection algorithm includes a backbone network, a pixel-level global attention module, a neck network and a head network. The feature map obtained after the original high-pressure oil pipe surface image is processed by the convolution layer and the pooling layer is input into the backbone network. After the backbone network performs feature extraction operations, feature maps of different scales are output in four stages respectively. The feature maps output from the first stage to the third stage are directly input into the neck network. The feature map output in the fourth stage is first input into the pixel-level global attention module. The pixel-level global attention module is connected in series with the channel dimension polarization self-attention mechanism and the space dimension polarization self-attention mechanism to extract the input feature map to obtain the final output feature map. The output feature map is input into the neck network. The neck network adopts a feature pyramid network. The feature pyramid network performs multi-scale feature extraction and then outputs a feature map set to the head network. The head network adopts a T-head structure to realize the output of the prediction result, wherein the prediction result includes the category and position of the surface defect of the high-pressure oil pipe.

2. The multi-scale defect visual inspection method for high-pressure fuel pipes of motor vehicles based on digital twins according to claim 1 is characterized in that: The backbone network uses the ResNet-50 neural network. The original high-pressure oil pipe surface image is processed by the convolution layer and the pooling layer to obtain a feature map of size (64, h, w). The feature map of size (64, h, w) is input into the ResNet-50 neural network for feature extraction. After four stages of feature extraction, four feature maps are output respectively.

3. The multi-scale defect visual inspection method for high-pressure fuel pipes of motor vehicles based on digital twins according to claim 1 or 2 is characterized in that: In the channel dimension polarized self-attention mechanism, the input feature map is firstly processed through a convolution layer with a convolution kernel size of 1×1. Perform dimensionality reduction to obtain a feature map with half the number of channels ; The feature maps are respectively transformed through 1×1 convolution layers The characteristic matrix With the weight matrix The new feature map Q and feature map V are obtained by multiplication, where the channel dimension of feature map Q is completely compressed to 1, and the feature map Q is enhanced. After the dimension is expanded, the feature map Q and the feature map V are matrix multiplied to obtain the comprehensive weight , and restore the number of channels of the feature map to C / 2 through a 1×1 convolution layer to obtain the channel polarization self-attention weight ; Use the halo attention mechanism to extract subtle features. The halo attention mechanism converts the feature map into Extract features in parts, and perform SA operation on each part to extract subtle features. At the same time, before performing SA calculation, perform haloing operation to extract subtle features through halo attention and obtain halo attention weight. , so that it is consistent with the channel polarization self-attention weight Perform matrix multiplication to obtain the comprehensive channel branch attention weight .

4. The multi-scale defect visual inspection method for high-pressure fuel pipes of motor vehicles based on digital twins according to claim 3 is characterized in that: Comprehensive channel branch attention weights The calculation formula is as follows: （1） in, Represents the comprehensive weight of feature map Q and feature map V in the channel dimension, represents the parameters in the convolution, represents the Softmax function, represents the Sigmoid function, and are tensor deformation operators, and Represents the feature maps The corresponding feature matrix and weight matrix.

5. The method for multi-scale defect visual inspection of motor vehicle high-pressure oil pipes based on digital twin according to claim 4 is characterized in that: Attention weights with comprehensive channel branches The feature map of the channel dimension is subjected to the same spatial dimension feature extraction operation as the channel dimension to obtain the spatial polarization self-attention weight After extracting subtle features through halo attention, we get the halo attention weight , which is consistent with the spatial polarization self-attention weight Perform matrix multiplication to obtain the comprehensive spatial branch attention weights .

6. The method for multi-scale defect visual inspection of motor vehicle high-pressure oil pipes based on digital twins according to claim 5 is characterized in that: Comprehensive channel branch attention weights The calculation formula is as follows: （2） in, is the Global Pooling function, , , are three tensor deformation operators.

7. The method for multi-scale defect visual inspection of motor vehicle high-pressure oil pipes based on digital twins according to claim 6 is characterized in that: The calculation formula of the polarized attention weight of the pixel-level global attention module is as follows: （3） in, represents the polarized attention weight, is the spatial dimension multiplication operator, is the channel dimension multiplication operator.

8. The multi-scale defect visual inspection method for high-pressure fuel pipes of motor vehicles based on digital twins according to claim 1 or 2 is characterized in that: The T-head structure includes a feature transformation structure and a task alignment predictor.

9. The multi-scale defect visual inspection method for high-pressure fuel pipes of motor vehicles based on digital twins according to claim 1 or 2 is characterized in that: The high-pressure oil pipe defect evolution prediction module is also used to transmit the result of the risk factor analysis to the software port to obtain a maintenance plan for the surface defects of the high-pressure oil pipe.

10. The multi-scale defect visual inspection method for high-pressure fuel pipes of motor vehicles based on digital twins according to claim 1 or 2, characterized in that: Use Catia software or solidworks software to build a three-dimensional model of the high-pressure oil pipe.

Images