A kind of engineering vehicle anti-collision method and system based on AC-DETR

Abstract

An AC-DETR-based engineering vehicle anti-collision method and system, wherein the method includes: S1, constructing an engineering vehicle operation obstacle data set; S2, constructing an AC-DETR model, i.e., a target detection model; S3, using the engineering vehicle operation obstacle data set to train the target detection model; S4, using knowledge distillation technology to optimize the trained target detection model, and deploying the model on the engineering vehicle equipment side; S5, when the engineering vehicle is operating, the position sensor obtains the distance information between the engineering vehicle and the obstacle, and combines the image information output by multiple visual sensors, the distance information output by the position sensor, and the prediction results, and then inputs them into the Kalman filter model to finally obtain the obstacle information; S6, the control module processes the received obstacle information and distance information, and issues an obstacle avoidance instruction. The present invention has a stronger ability to capture long-distance dependencies, and its detection performance is excellent in complex environments.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle obstacle avoidance, and in particular to an AC-DETR-based engineering vehicle anti-collision method and system. Background Art

[0002] In various engineering transportation operation scenarios such as mines and tunnels, how to avoid collision accidents and ensure driving safety is an important issue to protect the lives of operators and maintain the operation and production of enterprises. The operating environment of engineering vehicles is complex, with various obstacles, such as construction equipment, material equipment, and personnel. Through obstacle target detection, engineering vehicles can detect and avoid obstacles in time, prevent collisions, ensure the safety of vehicles and personnel, and reduce maintenance costs caused by damage to vehicles and facilities. At the same time, due to the tall body and many blind spots of engineering vehicles, it is difficult for drivers to observe all situations around the vehicle in a timely and accurate manner, which may lead to accidents. The multi-sensor obstacle target detection system can monitor blind spots and remind drivers in time when obstacles enter, reducing the risk of accidents, and even assisting in unmanned driving and improving transportation efficiency. Traditional anti-collision methods rely on mechanical devices and sensors, and have low detection performance in complex environments, which are prone to false detection and missed detection. Summary of the invention

[0003] The present invention provides an AC-DETR-based engineering vehicle collision avoidance method and system to solve the technical problems mentioned in the background technology.

[0004] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0005] The present invention provides an engineering vehicle anti-collision method based on AC-DETR, comprising the following steps:

[0006] S1. Multiple visual sensors and position sensors are installed on the engineering vehicle. The visual sensors are used to collect scene images, and then the scene images are processed and labeled to construct an engineering vehicle operation obstacle dataset.

[0007] S2. Construct an AC-DETR model. The AC-DETR model is an improved target detection model. The AC-DETR model includes a backbone network, an efficient encoder, and a decoder connected in sequence.

[0008] S3, using the engineering vehicle operation obstacle data set to train the AC-DETR model to obtain a trained AC-DETR model;

[0009] S4. Use knowledge distillation technology to optimize the trained AC-DETR model, and deploy the optimized AC-DETR model on the engineering vehicle equipment side;

[0010] S5. When the engineering vehicle is operating, the position sensor obtains the distance information between the engineering vehicle and the obstacle, and then uses the optimized AC-DETR model in the engineering vehicle equipment end to perform real-time detection and output the prediction results. The image information output by multiple visual sensors and the distance information output by the position sensor are combined with the prediction results and input into the Kalman filter model to finally obtain the obstacle information;

[0011] S6. The control module processes the received obstacle information and distance information, issues an obstacle avoidance command, and the engineering vehicle performs real-time obstacle avoidance according to the obstacle avoidance command.

[0012] Furthermore, the S1 specifically includes the following steps:

[0013] S11. Use industrial cameras as visual sensors, and place two industrial cameras on the front and rear of the engineering vehicle to obtain real-time environmental obstacle information; use laser radar as a position sensor, and then install the laser radar on the top of the engineering vehicle;

[0014] S12, using two industrial cameras to collect multiple scene images in real time, then processing and labeling all the collected scene images, and constructing a data set of obstacles for engineering vehicle operations;

[0015] S13. Divide the engineering vehicle operation obstacle dataset into a training set and a validation set.

[0016] Furthermore, the AC-DETR model uses the RT-DETR model as a baseline model; the RT-DETR model is the original target detection model;

[0017] Compared with the original backbone network in the RT-DETR model, the backbone network in the AC-DETR model replaces the MaxPool2d layer in the original backbone network with the WaveletPool layer; and replaces the Basicblock module of the original backbone network with the CIRM module to improve the AC-DETR model's ability to capture distance dependencies and reduce the computational overhead of the AC-DETR model;

[0018] Compared with the RT-DETR model, the AC-DETR model introduces the SCOM module between the first-layer CIRM module of the backbone network and the feature cross-scale fusion layer CCFM in the efficient encoder to increase the richness of feature extraction and improve the generalization ability of the model to detect targets of different scales.

[0019] Furthermore, the CIRM module uses the reverse residual movement module IRMB as a baseline model, and replaces the extended window multi-head self-attention mechanism in the reverse residual movement module IRMB with a cascade group attention mechanism;

[0020] The cascade group attention mechanism is used to split all features and then connect them to each attention head to improve the diversity of feature maps. The cascade group attention mechanism can be expressed in mathematical form as follows:

[0021] ;

[0022] in, For the The feature is passed through The output of the attention head is, For attention operations; For feature slicing; 、 、 The layers are three different information conversion and interaction layers; , , Represents the feature projections obtained at different layers of the input feature map; For splicing operation, is a linear layer; Represents the output features of the cascade group attention mechanism; Represents the total number of attention heads;

[0023] The SCOM module includes an SPDConv module and an OKM module. The input end of the SPDConv module is connected to the first-layer CIRM module of the backbone network in the AC-DETR model. The output end of the SPDConv module is connected to the input end of the OKM module in the feature cross-scale fusion layer CCFM through a splicing operation. The output end of the OKM module is connected to the RepC3 module in the feature cross-scale fusion layer CCFM.

[0024] Further, the OKM module includes three branches: local branch, large branch and global branch;

[0025] Among them, the global branch is composed of a dual-domain channel attention module DCAM and a frequency-based spatial attention module FSAM. The dual-domain channel attention module DCAM includes frequency channel attention FCA and spatial channel attention SCA;

[0026] The mathematical expression of frequency channel attention FCA is:

[0027] ;

[0028] in, and are the fast Fourier transform and its inverse operation, is the output of frequency channel attention FCA, is a convolution layer with a 1×1 convolution kernel. GAP is the global average pooling, Represents element-wise multiplication operation; Represents the feature map input to the frequency channel attention FCA;

[0029] The mathematical expression of the dual-domain channel attention module DCAM is:

[0030] ;

[0031] in, is the output of the dual-domain channel attention module DCAM; It is a convolution layer with a 1×1 convolution kernel;

[0032] The mathematical expression of the spatial attention module FSAM is:

[0033] ;

[0034] in, is the output of the spatial attention module FSAM; and It is a convolution of two different 1×1 convolution kernels.

[0035] Furthermore, the S3 specifically includes the following steps:

[0036] S31, select a scene image from the training set and input it into the backbone network of the AC-DETR model. The backbone network extracts features from the scene image and outputs feature maps of different scales;

[0037] S32, inputting feature maps of different scales into the efficient encoder of the AC-DETR model, the efficient encoder performs cross-scale fusion on feature maps of different scales, and encodes the spatial information and semantic information of the target in the scene image to obtain multi-scale features;

[0038] S33, input the multi-scale features into the decoder of the AC-DETR model, the decoder performs feature fusion and prediction on the multi-scale features, and outputs the prediction box of the target object with category and confidence;

[0039] S34, construct loss function 1, calculate the loss based on the prediction box with category and confidence, loop S31 to S34, minimize loss function 1, until loss function 1 converges, use the validation set for verification, update the weights of the AC-DETR model through back propagation, and obtain the trained AC-DETR model.

[0040] Furthermore, the S4 specifically includes the following steps:

[0041] S41, use the RT-DETR-r50 model as the teacher model and the trained AC-DETR model as the student model;

[0042] S42, training the teacher model using the engineering vehicle operation obstacle data set to obtain a trained teacher model;

[0043] S43. Select two scene images from the engineering vehicle operation obstacle data set and input them into the trained teacher model and the trained student model respectively, and obtain the outputs of the teacher model and the student model respectively. Then, use the outputs of the teacher model and the student model to calculate the losses of the teacher model and the student model respectively, so as to obtain the loss function of the teacher model. , and the loss function of the student model ;

[0044] The loss function of the teacher model is The calculation formula is as follows:

[0045] ;

[0046] in, is the KL divergence, The teacher neural network A smoothed value, The student neural network A smoothed value;

[0047] S44. Use knowledge distillation to distill the trained student model and calculate the distillation balance factor , distillation balance factor That is, the influence ratio of knowledge distillation in the student model training process, and then construct the loss function of the student model after distillation , the calculation formula is as follows:

[0048] ;

[0049] The formula for knowledge distillation is as follows:

[0050] ;

[0051] in It is The value of Logits, which is the output of the last layer of the teacher model. is the distillation temperature, is the probability value, exp represents the natural exponential function;

[0052] S45, loop S43 to S44 until the loss function of the student model after distillation Converge and update the weight parameters of the trained student model to obtain the optimized AC-DETR model;

[0053] S46. Import the optimized AC-DETR model into the edge computing device of the engineering vehicle.

[0054] Furthermore, the S5 specifically includes the following steps:

[0055] S51, the laser radar scans the surrounding environment of the engineering vehicle at a set frequency by emitting a laser beam and receiving a reflected signal, and obtains the position where the laser radar beam intersects with the surface of an object in the environment, thereby obtaining the distance information between the engineering vehicle and the obstacle;

[0056] S52, using a coordinate system conversion formula to convert the laser radar coordinate system into the engineering vehicle coordinate system, so as to unify the laser radar and the engineering vehicle's own coordinate systems;

[0057] S53, integrating the distance information from the laser radar, the image information from the industrial camera, and the prediction results output by the optimized AC-DETR model in the edge computing device in the same spatial coordinate system, and inputting these data into the Kalman filter model as initial estimated values;

[0058] S54, the Kalman filter model continuously corrects the estimated value through the prediction and update steps, and makes the best estimate of the position state of the obstacle, thereby obtaining the position state of the obstacle; the position state of the obstacle and the prediction result output by the optimized AC-DETR model constitute the obstacle information.

[0059] Furthermore, the coordinate system conversion formula in S52 is specifically as follows:

[0060] ;

[0061] in, The x, y and z axis coordinate values ​​of the coordinate point in the engineering vehicle coordinate system; are the x, y and z axis coordinate values ​​of the coordinate point in the laser radar coordinate system; is the coordinate of the installation position of the laser radar on the engineering vehicle relative to the center of mass of the engineering vehicle in the engineering vehicle coordinate system; is the rotation matrix of the laser radar coordinate system relative to the engineering vehicle coordinate system.

[0062] Another aspect of the present invention further provides an AC-DETR-based engineering vehicle collision avoidance system, comprising a computer device, the computer device being programmed or configured to execute the above engineering vehicle collision avoidance method;

[0063] Or, a computer program programmed or configured to execute the above engineering vehicle collision avoidance method is stored in the memory of the computer device.

[0064] Beneficial effects of the present invention:

[0065] 1. The present invention discloses an AC-DETR-based engineering vehicle collision avoidance method. The method applies the AC-DETR model to engineering vehicle track obstacle detection. The AC-DETR model realizes true end-to-end detection. Unlike previous target detectors, it does not rely on artificial priors and does not require NMS (Non-Maximum Suppression) post-processing steps. The AC-DETR model is based on the Transformer architecture design, has a stronger ability to capture long-distance dependencies, and has excellent detection performance in complex environments.

[0066] 2. The CIRM module in the AC-DETR model of the present invention makes a secondary innovative improvement on the IRMB module, introducing a cascade group attention mechanism, reducing computational redundancy and improving feature richness.

[0067] 3. The AC-DETR model in the present invention introduces the SCOM module, which enhances the multi-scale feature fusion capability of the model to solve the difficulty of obstacle detection model, especially in small target detection in complex engineering environments. A new SPDConv module is added to the neck of the SCOM module to splice the shallow features containing more small target information in the feature layer with the deep features, and multi-scale fusion is performed in the OKM module, which enhances the ability of the AC-DETR model to detect small targets;

[0068] 4. The present invention uses knowledge distillation technology to optimize the AC-DETR model. During the knowledge distillation process, the teacher model (i.e., the RT-DETR-r50 model) provides the prior knowledge of obstacles in the data set to the student model (i.e., the AC-DETR model). The AC-DETR model converges faster and has higher accuracy in detecting obstacles. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 A layout diagram of multiple visual sensors and position sensors in an engineering vehicle in the present invention;

[0070] Figure 2 It is the overall structural block diagram of the AC-DETR model in the present invention;

[0071] Figure 3 It is a structural block diagram of the cascade group attention mechanism in the present invention;

[0072] Figure 4It is a structural block diagram of the CIRM module in the present invention;

[0073] Figure 5 It is a structural block diagram of the SCOM module in the present invention;

[0074] Figure 6 It is a working schematic diagram of the SPD-Conv module in the SCOM module of the present invention;

[0075] Figure 7 This is a flow chart of knowledge distillation in the present invention;

[0076] Figure 8 This is the overall structural diagram of the RT-DETR model. DETAILED DESCRIPTION

[0077] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Preferred embodiments of the present invention are provided in the drawings. However, the present invention can be implemented in many other different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0078] Reference Figure 1 The embodiment of the present application provides an AC-DETR-based engineering vehicle collision avoidance method, comprising the following steps:

[0079] S1. Multiple visual sensors and position sensors are installed on the engineering vehicle. The visual sensors are used to collect scene images, and then the scene images are processed and labeled to construct an engineering vehicle operation obstacle dataset.

[0080] S2. Construct an AC-DETR model, which is an improved target detection model. The AC-DETR model includes a backbone network, an efficient encoder, and a decoder connected in sequence. A and C in the AC-DETR model represent autonomous driving and a cascade group attention mechanism, respectively. DETR refers to the abbreviation of the Detection TRansformer end-to-end target detection model. That is, the AC-DETR model is a target detection model for autonomous driving based on a cascade group attention mechanism.

[0081] S3, using the engineering vehicle operation obstacle data set to train the AC-DETR model to obtain a trained AC-DETR model;

[0082] S4. Use knowledge distillation technology to optimize the trained AC-DETR model, and deploy the optimized AC-DETR model on the engineering vehicle equipment side;

[0083] S5. When the engineering vehicle is operating, the position sensor obtains the distance information between the engineering vehicle and the obstacle, and then uses the optimized AC-DETR model in the engineering vehicle equipment end to perform real-time detection and output the prediction results. The image information output by multiple visual sensors and the distance information output by the position sensor are combined with the prediction results and input into the Kalman filter model to finally obtain the obstacle information;

[0084] S6. The control module processes the received obstacle information and distance information, issues an obstacle avoidance command, and the engineering vehicle performs real-time obstacle avoidance according to the obstacle avoidance command.

[0085] In some embodiments, the S1 specifically includes the following steps:

[0086] S11. Use industrial cameras as visual sensors and place two industrial cameras on the front and rear of the engineering vehicle to obtain real-time environmental obstacle information. Use laser radar as position sensor and install the laser radar on the top of the engineering vehicle. Figure 1 ;

[0087] S12, using two industrial cameras to collect multiple scene images in real time, then processing and labeling all the collected scene images, and constructing a data set of obstacles for engineering vehicle operations;

[0088] S13. Divide the engineering vehicle operation obstacle dataset into a training set and a validation set.

[0089] In some embodiments, the AC-DETR model uses the RT-DETR model as a baseline model; the RT-DETR model is the original target detection model; wherein the RT in the RT-DETR model is the abbreviation of Real Time.

[0090] Reference Figure 2 and Figure 8 , the backbone network in the AC-DETR model replaces the MaxPool2d layer in the original backbone network with the WaveletPool layer compared to the original backbone network in the RT-DETR model; and replaces the Basicblock module of the original backbone network with the CIRM module to improve the AC-DETR model's ability to capture distance dependencies and reduce the computational overhead of the AC-DETR model;

[0091] Compared with the RT-DETR model, the AC-DETR model introduces the SCOM module between the first-layer CIRM module of the backbone network and the feature cross-scale fusion layer CCFM in the efficient encoder to increase the richness of feature extraction and improve the generalization ability and robustness of the model in detecting targets of different scales. The structure of the AC-DETR model is as follows: Figure 2shown.

[0092] In some embodiments, the CIRM module uses the reverse residual movement module IRMB as the baseline model, and replaces the extended window multi-head self-attention mechanism (EW-MHSA) in the reverse residual movement module IRMB with a cascade group attention mechanism; the structural block diagram of the cascade group attention mechanism is shown in Figure 3 As shown;

[0093] The cascaded group attention mechanism is used to segment all features and then connect them to each attention head to improve the diversity of feature maps. Feature segmentation is used instead of complete features during calculation, which is more efficient and reduces overhead. On the other hand, in order to avoid feature segmentation affecting feature richness, the output of each attention head is connected to the next attention head using a cascade method. At the same time, a Token interaction layer is used after the Q layer, so that the attention head can not only capture local information, but also obtain all information from the previous attention head. The capacity and diversity of feature information are improved, and attention redundancy is reduced. The cascaded group attention mechanism can be expressed in mathematical form as follows:

[0094] ;

[0095] in, For the The feature is passed through The output of the attention head is, For attention operations; For feature slicing; 、 、 The layers are three different information conversion and interaction layers; , , Represents the feature projections obtained at different layers of the input feature map; For splicing operation, is a linear layer; Represents the output features of the cascade group attention mechanism; Represents the total number of attention heads; the CIRM module structure is as follows Figure 4 As shown;

[0096] The SCOM module includes an SPDConv module and an OKM module. The input end of the SPDConv module is connected to the first layer CIRM module of the backbone network in the AC-DETR model. The output end of the SPDConv module is connected to the input end of the OKM module in the feature cross-scale fusion layer CCFM through a splicing operation. The output end of the OKM module is connected to the RepC3 module in the feature cross-scale fusion layer CCFM.

[0097] The SPD-Conv module consists of a space-to-depth (SPD) layer and a non-strided convolution layer. It enhances the feature representation by converting the spatial dimension of the feature map into the depth dimension, which can make up for the problem of insufficient information in low-resolution images, such as Figure 6 , a series of sub-feature maps are cut out from the intermediate feature map F of size S × S × C1, and each sub-map downsamples the feature map F by the scale factor scale. When scale = 2, F is downsampled twice to obtain four sub-feature maps of shape (S / 2, S / 2, C1). The four sub-feature maps are concatenated according to the channel dimension to obtain an intermediate feature map F' of size (S / 2, S / 2, 4C1). Compared with the original feature map, its spatial dimension is reduced to 1 / scale of the original, and the channel dimension is increased to scale 2 times, which can preserve the spatial and fine-grained information in the channel dimension without information loss. The role of the final non-strided convolutional layer is to reduce the number of channels and retain all the discriminative information as much as possible.

[0098] In some embodiments, the OKM module includes three branches: a local branch, a large branch, and a global branch. The three branches process features respectively to enhance multi-scale feature representation; the local branch only integrates local information by a simple and efficient deep convolution;

[0099] The local branch consists of only a simple and efficient deep convolution to integrate local information; the large branch is composed of three parallel branches of deep convolutions with different convolution kernel sizes. The deep convolution with a kernel size of 31×31 can expand the receptive field as much as possible to more completely restore the image features. In addition, 1×31 and 31×1 deep convolutions are used to capture strip context information.

[0100] Among them, the global branch is composed of a dual-domain channel attention module DCAM and a frequency-based spatial attention module FSAM. The dual-domain channel attention module DCAM includes frequency channel attention FCA and spatial channel attention SCA;

[0101] The mathematical expression of frequency channel attention FCA is:

[0102] ;

[0103] in, and are the fast Fourier transform and its inverse operation, is the output of frequency channel attention FCA, is a convolution layer with a 1×1 convolution kernel. GAP is the global average pooling, Represents element-wise multiplication operation; Represents the feature map input to the frequency channel attention FCA;

[0104] The mathematical expression of the dual-domain channel attention module DCAM is:

[0105] ;

[0106] in, is the output of the dual-domain channel attention module DCAM; It is a convolution layer with a 1×1 convolution kernel;

[0107] The mathematical expression of the spatial attention module FSAM is:

[0108] ;

[0109] in, is the output of the spatial attention module FSAM; and is the convolution of two different 1×1 convolution kernels. Through the spatial attention module FSAM, the model can pay more attention to the frequency part with rich information. The SCOM module effectively supplements the shallow feature information, allowing the model to learn and capture richer feature expressions. The SCOM module structure is as follows: Figure 5 .

[0110] In some embodiments, S3 specifically includes the following steps:

[0111] S31, select a scene image from the training set and input it into the backbone network of the AC-DETR model. The backbone network extracts features from the scene image and outputs feature maps of different scales;

[0112] S32, inputting feature maps of different scales into the efficient encoder of the AC-DETR model, the efficient encoder performs cross-scale fusion on feature maps of different scales, and encodes the spatial information and semantic information of the target in the scene image to obtain multi-scale features;

[0113] S33, input the multi-scale features into the decoder of the AC-DETR model, the decoder performs feature fusion and prediction on the multi-scale features, and outputs the prediction box of the target object with category and confidence;

[0114] S34, construct loss function 1, calculate the loss based on the prediction box with category and confidence, loop S31 to S34, minimize loss function 1, until loss function 1 converges, use the validation set for verification, update the weights of the AC-DETR model through back propagation, and obtain the trained AC-DETR model.

[0115] In some embodiments, the S4 specifically includes the following steps:

[0116] S41, use the RT-DETR-r50 model as the teacher model and the trained AC-DETR model as the student model;

[0117] S42, training the teacher model using the engineering vehicle operation obstacle data set to obtain a trained teacher model;

[0118] S43. Select two scene images from the engineering vehicle operation obstacle data set and input them into the trained teacher model and the trained student model respectively, and obtain the outputs of the teacher model and the student model respectively. Then, use the outputs of the teacher model and the student model to calculate the losses of the teacher model and the student model respectively, so as to obtain the loss function of the teacher model. , and the loss function of the student model ;

[0119] The loss function of the teacher model is The calculation formula is as follows:

[0120] ;

[0121] in, is the KL divergence, The teacher neural network A smoothed value, The student neural network A smoothed value;

[0122] S44. Use knowledge distillation to distill the trained student model and calculate the distillation balance factor , distillation balance factor That is, the influence ratio of knowledge distillation in the student model training process, and then construct the loss function of the student model after distillation , the calculation formula is as follows:

[0123] ;

[0124] Reference Figure 7, the distillation method is to extract the knowledge learned from the teacher model to guide the student model to train. The role of the teacher model is equivalent to helping the student model obtain prior information about the data set and improve the accuracy of the student neural network. The formula for knowledge distillation is as follows:

[0125] ;

[0126] in It is The value of Logits, which is the output of the last layer of the teacher model. is the distillation temperature, is the probability value, exp represents the natural exponential function; when When , it is the ordinary SoftMax (normalized exponential function) formula. tend to hour, tends to 0, all probability values They will also tend to be equal, that is, high temperature will eliminate the original differences caused by different numerical values.

[0127] S45, loop S43 to S44 until the loss function of the student model after distillation Converge and update the weight parameters of the trained student model to obtain the optimized AC-DETR model;

[0128] S46. Import the optimized AC-DETR model to the edge computing device of the engineering vehicle. The edge computing platform uses Jetson Nano developed by NVIDIA.

[0129] In some embodiments, S5 specifically includes the following steps:

[0130] S51. The laser radar scans the surrounding environment of the engineering vehicle at a set frequency (10-20 scans per second) by emitting a laser beam and receiving a reflected signal, and obtains the position where the laser radar beam intersects with the surface of an object in the environment, thereby obtaining the distance information between the engineering vehicle and the obstacle;

[0131] S52, using a coordinate system conversion formula to convert the laser radar coordinate system into the engineering vehicle coordinate system, so as to unify the laser radar and the engineering vehicle's own coordinate systems;

[0132] S53, integrating the distance information from the laser radar, the image information from the industrial camera, and the prediction results output by the optimized AC-DETR model in the edge computing device in the same spatial coordinate system, and inputting these data into the Kalman filter model as initial estimated values;

[0133] S54, the Kalman filter model continuously corrects the estimated value through the prediction and update steps, and makes the best estimate of the position state of the obstacle, thereby obtaining the position state of the obstacle; the position state of the obstacle and the prediction result output by the optimized AC-DETR model constitute the obstacle information.

[0134] In some embodiments, the coordinate system conversion formula in S52 is specifically as follows:

[0135] ;

[0136] in, The x, y and z axis coordinate values ​​of the coordinate point in the engineering vehicle coordinate system; are the x, y and z axis coordinate values ​​of the coordinate point in the laser radar coordinate system; is the coordinate of the installation position of the laser radar on the engineering vehicle relative to the center of mass of the engineering vehicle in the engineering vehicle coordinate system; is the rotation matrix of the laser radar coordinate system relative to the engineering vehicle coordinate system.

[0137] The present invention discloses an engineering vehicle collision avoidance method based on AC-DETR. The method applies the AC-DETR model to engineering vehicle track obstacle detection. The AC-DETR model realizes true end-to-end detection. Unlike previous target detectors, it does not rely on artificial priors and does not require NMS post-processing steps. The AC-DETR model is based on the Transformer architecture design, has a stronger long-distance dependency capture capability, and has excellent detection performance in complex environments.

[0138] Another aspect of the present invention further provides an AC-DETR-based engineering vehicle collision avoidance system, comprising a computer device, the computer device being programmed or configured to execute the above engineering vehicle collision avoidance method;

[0139] Or, a computer program programmed or configured to execute the above engineering vehicle collision avoidance method is stored in the memory of the computer device.

[0140] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered within the protection scope of the present invention. In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An AC-DETR-based engineering vehicle collision avoidance method, characterized in that: The steps include: S1. Multiple visual sensors and position sensors are installed on the engineering vehicle. The visual sensors are used to collect scene images, and then the scene images are processed and labeled to construct an engineering vehicle operation obstacle dataset. S2. Construct an AC-DETR model. The AC-DETR model is an improved target detection model. The AC-DETR model includes a backbone network, an efficient encoder, and a decoder connected in sequence. S3, using the engineering vehicle operation obstacle data set to train the AC-DETR model to obtain a trained AC-DETR model; S4. Use knowledge distillation technology to optimize the trained AC-DETR model, and deploy the optimized AC-DETR model on the engineering vehicle equipment side; S5. When the engineering vehicle is operating, the position sensor obtains the distance information between the engineering vehicle and the obstacle, and then uses the optimized AC-DETR model in the engineering vehicle equipment end to perform real-time detection and output the prediction results. The image information output by multiple visual sensors and the distance information output by the position sensor are combined with the prediction results and input into the Kalman filter model to finally obtain the obstacle information; S6, the control module processes the received obstacle information and distance information, issues an obstacle avoidance command, and the engineering vehicle performs obstacle avoidance in real time according to the obstacle avoidance command; The AC-DETR model uses the RT-DETR model as a baseline model; the RT-DETR model is the original target detection model; The backbone network in the AC-DETR model is different from the original backbone network in the RT-DETR model in that the MaxPool2d layer in the original backbone network is replaced by the WaveletPool layer; and the Basicblock module in the original backbone network is replaced by the CIRM module; Compared with the RT-DETR model, the AC-DETR model introduces the SCOM module between the first-layer CIRM module of the backbone network and the feature cross-scale fusion layer CCFM in the efficient encoder; The CIRM module uses the reverse residual movement module IRMB as the baseline model, and replaces the extended window multi-head self-attention mechanism in the reverse residual movement module IRMB with a cascade group attention mechanism; The SCOM module includes an SPDConv module and an OKM module. The input end of the SPDConv module is connected to the first-layer CIRM module of the backbone network in the AC-DETR model. The output end of the SPDConv module is connected to the input end of the OKM module in the feature cross-scale fusion layer CCFM through a splicing operation. The output end of the OKM module is connected to the RepC3 module in the feature cross-scale fusion layer CCFM.

2. The anti-collision method for an engineering vehicle according to claim 1, characterized in that: The S1 specifically includes the following steps: S11. Use industrial cameras as visual sensors, and place two industrial cameras on the front and rear of the engineering vehicle to obtain real-time environmental obstacle information; use laser radar as a position sensor, and then install the laser radar on the top of the engineering vehicle; S12, using two industrial cameras to collect multiple scene images in real time, then processing and labeling all the collected scene images, and constructing a data set of obstacles for engineering vehicle operations; S13. Divide the engineering vehicle operation obstacle dataset into a training set and a validation set.

3. The anti-collision method for an engineering vehicle according to claim 2, characterized in that: The cascade group attention mechanism is used to split all features and then connect them to each attention head to improve the diversity of feature maps; the cascade group attention mechanism is mathematically expressed as: ； in, For the The feature is passed through The output of the attention head is, For attention operations; is feature slice; 、 、 The layers are three different information conversion and interaction layers; , , Represents the feature projections obtained at different layers of the input feature map; For splicing operation, is a linear layer; Represents the output features of the cascade group attention mechanism; Represents the total number of attention heads.

4. The anti-collision method for an engineering vehicle according to claim 3, characterized in that: The OKM module includes three branches: local branch, large branch and global branch; Among them, the global branch is composed of a dual-domain channel attention module DCAM and a frequency-based spatial attention module FSAM. The dual-domain channel attention module DCAM includes frequency channel attention FCA and spatial channel attention SCA; The mathematical expression of frequency channel attention FCA is: ； in, and are the fast Fourier transform and its inverse operation, is the output of frequency channel attention FCA, is a convolution layer with a 1×1 convolution kernel. GAP is the global average pooling, Represents element-wise multiplication operation; Represents the feature map input to the frequency channel attention FCA; The mathematical expression of the dual-domain channel attention module DCAM is: ； in, is the output of the dual-domain channel attention module DCAM; It is a convolution layer with a 1×1 convolution kernel; The mathematical expression of the spatial attention module FSAM is: ； in, is the output of the spatial attention module FSAM; and It is a convolution of two different 1×1 convolution kernels.

5. The anti-collision method for an engineering vehicle according to claim 4, characterized in that: The S3 specifically includes the following steps: S31, select a scene image from the training set and input it into the backbone network of the AC-DETR model. The backbone network extracts features from the scene image and outputs feature maps of different scales; S32, inputting feature maps of different scales into the efficient encoder of the AC-DETR model, the efficient encoder performs cross-scale fusion on feature maps of different scales, and encodes the spatial information and semantic information of the target in the scene image to obtain multi-scale features; S33, input the multi-scale features into the decoder of the AC-DETR model, the decoder performs feature fusion and prediction on the multi-scale features, and outputs the prediction box of the target object with category and confidence; S34, construct loss function 1, calculate the loss based on the prediction box with category and confidence, loop S31 to S34, minimize loss function 1, until loss function 1 converges, use the validation set for verification, update the weights of the AC-DETR model through back propagation, and obtain the trained AC-DETR model.

6. The anti-collision method for an engineering vehicle according to claim 1, characterized in that: The S4 specifically includes the following steps: S41, use the RT-DETR-r50 model as the teacher model and the trained AC-DETR model as the student model; S42, training the teacher model using the engineering vehicle operation obstacle data set to obtain a trained teacher model; S43. Select two scene images from the engineering vehicle operation obstacle data set and input them into the trained teacher model and the trained student model respectively, and obtain the outputs of the teacher model and the student model respectively. Then, use the outputs of the teacher model and the student model to calculate the losses of the teacher model and the student model respectively, so as to obtain the loss function of the teacher model. , and the loss function of the student model ; The loss function of the teacher model is The calculation formula is as follows: ； in, is the KL divergence, The teacher neural network A smoothed value, The student neural network A smoothed value; S44. Use knowledge distillation to distill the trained student model and calculate the distillation balance factor , distillation balance factor That is, the influence ratio of knowledge distillation in the student model training process, and then construct the loss function of the student model after distillation , the calculation formula is as follows: ； The formula for knowledge distillation is as follows: ； in It is The value of Logits, which is the output of the last layer of the teacher model. is the distillation temperature, is the probability value, exp represents the natural exponential function; S45, loop S43 to S44 until the loss function of the student model after distillation Converge and update the weight parameters of the trained student model to obtain the optimized AC-DETR model; S46. Import the optimized AC-DETR model into the edge computing device of the engineering vehicle.

7. The anti-collision method for an engineering vehicle according to claim 6, characterized in that: The S5 specifically includes the following steps: S51, the laser radar scans the surrounding environment of the engineering vehicle at a set frequency by emitting a laser beam and receiving a reflected signal, and obtains the position where the laser radar beam intersects with the surface of an object in the environment, thereby obtaining the distance information between the engineering vehicle and the obstacle; S52, using a coordinate system conversion formula to convert the laser radar coordinate system into the engineering vehicle coordinate system, so as to unify the laser radar and the engineering vehicle's own coordinate systems; S53, integrating the distance information from the laser radar, the image information from the industrial camera, and the prediction results output by the optimized AC-DETR model in the edge computing device in the same spatial coordinate system, and inputting these data into the Kalman filter model as initial estimated values; S54, the Kalman filter model continuously corrects the estimated value through the prediction and update steps, and makes an optimal estimate of the position state of the obstacle, thereby obtaining the position state of the obstacle; The position status of the obstacle and the prediction results output by the optimized AC-DETR model constitute the obstacle information.

8. The anti-collision method for an engineering vehicle according to claim 7, characterized in that: The coordinate system conversion formula in S52 is as follows: ； in, The x, y and z axis coordinate values ​​of the coordinate point in the engineering vehicle coordinate system; are the x, y and z axis coordinate values ​​of the coordinate point in the laser radar coordinate system; is the coordinate of the installation position of the laser radar on the engineering vehicle relative to the center of mass of the engineering vehicle in the engineering vehicle coordinate system; is the rotation matrix of the laser radar coordinate system relative to the engineering vehicle coordinate system.

9. An AC-DETR-based engineering vehicle collision avoidance system, comprising a computer device, characterized in that: The computer device is programmed or configured to execute the engineering vehicle collision avoidance method according to any one of claims 1 to 8; Or, a computer program programmed or configured to execute the engineering vehicle collision avoidance method according to any one of claims 1 to 8 is stored in the memory of the computer device.

Images