Detection method of surface defects of gunpowder column based on large language model and 3D Gaussian sputtering

Abstract

The present invention relates to a technique for detecting apparent defects in powder columns, specifically a method for detecting apparent defects in powder columns based on a large language model and 3D Gaussian sputtering. The method is implemented using the following steps: Step 1: Acquire multi-view images of the powder column; Step 2: Generate visual features; Step 3: Generate language embedding tags; Step 4: Generate text features; Step 5: Generate feature map A; Step 6: Generate a Gaussian parameter set; Step 7: Generate feature map B; Step 8: Generate a multi-layer feature map; Step 9: Generate a feature pyramid with rich multi-scale information; Step 10: Generate bounding box information of the apparent defects in the powder column; Step 11: Output a bounding box prediction result of the apparent defects in the powder column. The present invention effectively solves the problems of poor detection accuracy and poor detection comprehensiveness of existing methods for detecting apparent defects in powder columns, and is suitable for detecting apparent defects in powder columns.

Description

Technical Field

[0001] The present invention relates to a gunpowder column surface defect detection technology, in particular to a gunpowder column surface defect detection method based on a large language model and 3D Gaussian sputtering. Background Art

[0002] The gunpowder column is an important component of ammunition, and its quality has a significant impact on the performance and safety of ammunition. In order to ensure the quality of the gunpowder column, it is necessary to promptly detect the apparent defects of the gunpowder column (such as pits, cracks, etc.). However, due to the limitations of their own principles, the existing gunpowder column apparent defect detection method has the following problems: First, the existing gunpowder column apparent defect detection method is unable to detect the depth information of the gunpowder column apparent defect, resulting in poor detection accuracy. Second, when detecting tiny gunpowder column apparent defects, the existing gunpowder column apparent defect detection method is prone to missed detection, resulting in poor detection comprehensiveness. Based on this, it is necessary to invent a gunpowder column apparent defect detection method based on a large language model and 3D Gaussian sputtering to solve the problems of poor detection accuracy and poor detection comprehensiveness of the existing gunpowder column apparent defect detection method. Summary of the Invention

[0003] In order to solve the problems of poor detection accuracy and poor detection comprehensiveness of existing methods for detecting apparent defects of gunpowder columns, the present invention provides a method for detecting apparent defects of gunpowder columns based on a large language model and 3D Gaussian sputtering.

[0004] The present invention is achieved by adopting the following technical solutions:

[0005] A method for detecting surface defects of gunpowder columns based on a large language model and 3D Gaussian sputtering is implemented using the following steps:

[0006] Step 1: Use a camera to capture multi-view images of the gunpowder column;

[0007] Step 2: Input the multi-view image of the gunpowder column into the visual encoder to generate visual features;

[0008] Step 3: Input the visual features into the vision-language connector to generate language embedding tokens;

[0009] Step 4: Input the language embedding tags and the text information describing the apparent defect characteristics of the gunpowder column into the pre-trained large language model to generate text features;

[0010] Step 5: The multi-view images of the gunpowder column are input into the backbone network and the FPN neck network for feature extraction, thereby generating a feature map A;

[0011] Step 6: Input the feature map A into the 3D Gaussian generator to generate a Gaussian parameter set;

[0012] Step 7: Input the Gaussian parameter set into the gridding module to generate the feature map B;

[0013] Step 8: Input the feature map B and text features into the 3D backbone network for feature extraction, thereby generating a multi-layer feature map;

[0014] Step 9: Input the multi-layer feature map into the 3D FPN neck network to generate a feature pyramid with rich multi-scale information;

[0015] Step 10: Input the feature pyramid into the 3D region proposal network head to generate the bounding box information of the apparent defect of the gunpowder column;

[0016] Step 11: Input the bounding box information of the apparent defect of the gunpowder column into the non-maximum suppression module for screening, thereby outputting the bounding box prediction result of the apparent defect of the gunpowder column.

[0017] Furthermore, in step six, the specific steps of generating the Gaussian parameter set are as follows: first, the depth head of the 3D Gaussian generator uses the intrinsic parameter set of the camera to derive the 3D point information of the feature map A, the 3D compensation head of the 3D Gaussian generator refines the 3D point information of the feature map A by adding a small 3D displacement, the rotation head of the 3D Gaussian generator uses the intrinsic parameter set of the camera to retrieve the egocentric rotation information of the feature map A, the scale head of the 3D Gaussian generator extracts the scale information of the feature map A, the opacity head of the 3D Gaussian generator extracts the opacity information of the feature map A, and the feature head of the 3D Gaussian generator extracts other feature information of the feature map A; then, the extrinsic parameter set of the camera is used to convert the refined 3D point information of the feature map A and the egocentric rotation information of the feature map A from the camera reference system to the world reference system, and the conversion result is connected with the scale information of the feature map A, the opacity information of the feature map A, and the other feature information of the feature map A, thereby generating a Gaussian parameter set.

[0018] Furthermore, in step eight, each layer of the multi-layer feature map contains feature information of different scales and levels.

[0019] Furthermore, in step nine, the specific steps of generating a feature pyramid with rich multi-scale information are as follows: first, upsampling the high-level feature map so that its resolution matches that of the lower-level feature map; then, fusing the upsampled high-level feature map with the lower-level feature map to generate a feature pyramid with rich multi-scale information.

[0020] Furthermore, in step 10, the specific steps for generating the bounding box information of the apparent defect of the gunpowder column are as follows: first, 3D anchor frames of different aspect ratios and sizes are placed in each layer of the feature pyramid; then, through two 1×1×1 convolutional layers, the offset and score of the object in each 3D anchor frame are obtained respectively, thereby generating the bounding box information of the apparent defect of the gunpowder column.

[0021] Compared with the existing method for detecting apparent defects of gunpowder columns, the present invention, on the one hand, utilizes the characteristics of 3D Gaussian sputtering to achieve smooth and efficient three-dimensional reconstruction to efficiently reconstruct and render the apparent defects of gunpowder columns. On the other hand, it utilizes the characteristics of large language models to understand and generate natural language text to enhance the description of the apparent defects of gunpowder columns. In this way, on the one hand, it realizes the detection of depth information of the apparent defects of gunpowder columns, and on the other hand, it avoids missed detection when detecting tiny apparent defects of gunpowder columns, thereby effectively improving the detection accuracy on the one hand, and effectively improving the comprehensiveness of detection on the other hand.

[0022] The present invention effectively solves the problems of poor detection accuracy and poor detection comprehensiveness of existing methods for detecting apparent defects of gunpowder columns, and is suitable for detecting apparent defects of gunpowder columns. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of the principle of the present invention. DETAILED DESCRIPTION

[0024] A method for detecting surface defects of gunpowder columns based on a large language model and 3D Gaussian sputtering is implemented using the following steps:

[0025] Step 1: Use a camera to capture multi-view images of the gunpowder column;

[0026] Step 2: Input the multi-view image of the gunpowder column into the visual encoder to generate visual features;

[0027] Step 3: Input the visual features into the vision-language connector to generate language embedding tokens;

[0028] Step 4: Input the language embedding tags and the text information describing the apparent defect characteristics of the gunpowder column into the pre-trained large language model to generate text features;

[0029] Step 5: The multi-view images of the gunpowder column are input into the backbone network and the FPN neck network for feature extraction, thereby generating a feature map A;

[0030] Step 6: Input the feature map A into the 3D Gaussian generator to generate a Gaussian parameter set;

[0031] Step 7: Input the Gaussian parameter set into the gridding module to generate the feature map B;

[0032] Step 8: Input the feature map B and text features into the 3D backbone network for feature extraction, thereby generating a multi-layer feature map;

[0033] Step 9: Input the multi-layer feature map into the 3D FPN neck network to generate a feature pyramid with rich multi-scale information;

[0034] Step 10: Input the feature pyramid into the 3D region proposal network head to generate the bounding box information of the apparent defect of the gunpowder column;

[0035] Step 11: Input the bounding box information of the apparent defect of the gunpowder column into the non-maximum suppression module for screening, thereby outputting the bounding box prediction result of the apparent defect of the gunpowder column.

[0036] In step six, the specific steps of generating the Gaussian parameter set are as follows: first, the depth head of the 3D Gaussian generator uses the intrinsic parameter set of the camera to derive the 3D point information of the feature map A, the 3D compensation head of the 3D Gaussian generator refines the 3D point information of the feature map A by adding a small 3D displacement, the rotation head of the 3D Gaussian generator uses the intrinsic parameter set of the camera to retrieve the egocentric rotation information of the feature map A, the scale head of the 3D Gaussian generator extracts the scale information of the feature map A, the opacity head of the 3D Gaussian generator extracts the opacity information of the feature map A, and the feature head of the 3D Gaussian generator extracts other feature information of the feature map A; then, the extrinsic parameter set of the camera is used to convert the refined 3D point information and the egocentric rotation information of the feature map A from the camera reference system to the world reference system, and the conversion result is connected with the scale information of the feature map A, the opacity information of the feature map A, and the other feature information of the feature map A, thereby generating the Gaussian parameter set.

[0037] In step eight, each layer of the multi-layer feature map contains feature information of different scales and levels.

[0038] In step nine, the specific steps of generating a feature pyramid with rich multi-scale information are as follows: first, upsampling the high-level feature map so that its resolution matches that of the lower-level feature map; then, fusing the upsampled high-level feature map with the lower-level feature map to generate a feature pyramid with rich multi-scale information.

[0039] In step 10, the specific steps for generating the bounding box information of the apparent defect of the gunpowder column are as follows: first, 3D anchor boxes with different aspect ratios and sizes are placed in each layer of the feature pyramid; then, through two 1×1×1 convolutional layers, the offset and score of the object in each 3D anchor box are obtained respectively, thereby generating the bounding box information of the apparent defect of the gunpowder column.

[0040] Although specific embodiments of the present invention have been described above, those skilled in the art will appreciate that these are merely illustrative and that the scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, and such changes and modifications are intended to fall within the scope of the present invention.

Claims

1. A method for detecting surface defects of a powder column based on a large language model and 3D Gaussian sputtering, characterized by: This method is implemented using the following steps: Step 1: Use a camera to capture multi-view images of the gunpowder column; Step 2: Input the multi-view image of the gunpowder column into the visual encoder to generate visual features; Step 3: Input the visual features into the vision-language connector to generate language embedding tokens; Step 4: Input the language embedding tags and the text information describing the apparent defect characteristics of the gunpowder column into the pre-trained large language model to generate text features; Step 5: The multi-view images of the gunpowder column are input into the backbone network and the FPN neck network for feature extraction, thereby generating a feature map A; Step 6: Input the feature map A into the 3D Gaussian generator to generate a Gaussian parameter set; Step 7: Input the Gaussian parameter set into the gridding module to generate the feature map B; Step 8: Input the feature map B and text features into the 3D backbone network for feature extraction, thereby generating a multi-layer feature map; Step 9: Input the multi-layer feature map into the 3D FPN neck network to generate a feature pyramid with rich multi-scale information; Step 10: Input the feature pyramid into the 3D region proposal network head to generate the bounding box information of the apparent defect of the gunpowder column; Step 11: Input the bounding box information of the apparent defect of the gunpowder column into the non-maximum suppression module for screening, thereby outputting the bounding box prediction result of the apparent defect of the gunpowder column.

2. The method for detecting surface defects of a powder column based on a large language model and 3D Gaussian sputtering according to claim 1, characterized in that: In step six, the specific steps of generating the Gaussian parameter set are as follows: first, the depth head of the 3D Gaussian generator uses the intrinsic parameter set of the camera to derive the 3D point information of the feature map A, the 3D compensation head of the 3D Gaussian generator refines the 3D point information of the feature map A by adding a small 3D displacement, the rotation head of the 3D Gaussian generator uses the intrinsic parameter set of the camera to retrieve the egocentric rotation information of the feature map A, the scale head of the 3D Gaussian generator extracts the scale information of the feature map A, the opacity head of the 3D Gaussian generator extracts the opacity information of the feature map A, and the feature head of the 3D Gaussian generator extracts other feature information of the feature map A; then, the extrinsic parameter set of the camera is used to convert the refined 3D point information and the egocentric rotation information of the feature map A from the camera reference system to the world reference system, and the conversion result is connected with the scale information of the feature map A, the opacity information of the feature map A, and the other feature information of the feature map A, thereby generating the Gaussian parameter set.

3. The method for detecting surface defects of a powder column based on a large language model and 3D Gaussian sputtering according to claim 1, characterized in that: In step eight, each layer of the multi-layer feature map contains feature information of different scales and levels.

4. The method for detecting surface defects of a powder column based on a large language model and 3D Gaussian sputtering according to claim 1, characterized in that: In step nine, the specific steps of generating a feature pyramid with rich multi-scale information are as follows: first, upsampling the high-level feature map so that its resolution matches that of the lower-level feature map; then, fusing the upsampled high-level feature map with the lower-level feature map to generate a feature pyramid with rich multi-scale information.

5. The method for detecting surface defects of a powder column based on a large language model and 3D Gaussian sputtering according to claim 1, characterized in that: In step 10, the specific steps for generating the bounding box information of the apparent defect of the gunpowder column are as follows: first, 3D anchor boxes with different aspect ratios and sizes are placed in each layer of the feature pyramid; then, through two 1×1×1 convolutional layers, the offset and score of the object in each 3D anchor box are obtained respectively, thereby generating the bounding box information of the apparent defect of the gunpowder column.

Images