Fastener nut missing identification method and system based on visual large model secondary segmentation

Abstract

The invention discloses a fastener nut missing identification method and system based on visual large model secondary segmentation. The method comprises the following steps: S1, obtaining a to-be-identified track image; s2, segmenting the to-be-identified track image by using a first visual large model to obtain a fastener image; s3, segmenting the fastener image by using a second visual large model to obtain segmented sub-images and masks corresponding to the segmented sub-images; s4, preliminarily screening out a fastener nut image from the segmented sub-images through polygon features; s5, performing global pooling on masks corresponding to the fastener nut images preliminarily screened out in the S4; s6, calculating the similarity between the feature vector of the to-be-identified nut and the feature vectors of a plurality of normal nuts, and if the similarity between the feature vector of the to-be-identified nut and the feature vector of a certain normal nut is smaller than a preset threshold value, determining that the fastener nut image preliminarily screened out in S4 has the problem of fastener nut missing; according to the method and the device, the missing condition of the fastener nut can be accurately and efficiently identified under the condition that the missing samples of the fastener nut are rare.

Description

Technical Field

[0001] This invention relates to the field of rail transit technology, and more specifically to a method and system for identifying missing fastener nuts based on secondary segmentation of a large visual model. Background Technology

[0002] Currently, rail transit, as a key infrastructure of modern cities, plays a crucial role in the city's vigorous development and the convenient travel of residents through its safe operation. Track fasteners, as core components ensuring the stability of the connection between rails and sleepers, are directly related to the safety of the track structure, especially the integrity of the fastener nuts. However, fastener nuts may become missing due to long-term vibration, material fatigue, or improper installation. Failure to detect and address this issue in a timely manner can potentially lead to changes in track geometry, reduced track structural stability, and even serious safety accidents such as train derailment. Therefore, monitoring and timely maintenance of the integrity of fastener nuts is a critical link in ensuring the safe operation of rail transit.

[0003] However, with the rapid expansion of the rail transit network and the continuous increase in operating mileage, traditional manual inspection methods are no longer sufficient to meet the demands of large-scale, high-frequency inspections. Furthermore, manual inspection is not only inefficient but also highly susceptible to subjective factors and environmental conditions, making it difficult to guarantee the consistency and accuracy of the results. While existing automated inspection methods (such as classification methods) have improved inspection efficiency to some extent, they also struggle to guarantee accuracy when faced with a limited number of missing fastener / nut samples, thus failing to effectively meet actual inspection needs.

[0004] Therefore, how to provide a method and system for identifying missing fastener nuts that can accurately and efficiently identify missing fastener nuts when samples are scarce is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method and system for identifying missing fastener nuts based on secondary segmentation of a large visual model.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] Firstly, a method for identifying missing fastener nuts based on secondary segmentation of a large visual model is provided, including the following steps:

[0008] S1: Obtain an image of the track to be identified; wherein, the image of the track to be identified includes the track bed, rails, and several fasteners;

[0009] S2: The track image to be identified is segmented using the first visual large model to obtain the fastener image P. iWhere i = 1, 2, ..., I; I represents the number of fastener images obtained by segmenting the track image to be identified; the fastener image P i Includes spring clips, screws, fasteners, nuts, and metal parts;

[0010] S3: Utilize the second visual large model to analyze the fastener image P i Perform segmentation to obtain segmented subgraph P i j and its corresponding mask F i j Among them, P i j This indicates the fastener image P. i The j-th segmented sub-image obtained by segmentation; j = 1, 2, ..., J; J represents the segmentation of the fastener image P i The number of sub-images obtained from the segmentation process;

[0011] S4: From the segmented subgraph P using polygon features i j Images of fastener nuts were initially selected;

[0012] S5: Perform global pooling on the mask corresponding to the fastener nut image initially selected in S4;

[0013] S6: Calculate the similarity between the feature vector of the nut to be identified and the feature vectors of several normal nuts. If the similarity between the feature vector of the nut to be identified and the feature vector of a certain normal nut is less than a preset threshold, then the fastener nut image initially screened in S4 has the problem of missing fastener nuts. The feature vector of the nut to be identified is the channel feature vector obtained by global pooling of the mask corresponding to the fastener nut image initially screened in S4.

[0014] Preferably, a certain normal nut feature vector is obtained from the plurality of normal nut feature vectors based on the following steps;

[0015] S61: Obtain a normal track image; wherein, the normal track image includes the track bed, rails, and several fasteners;

[0016] S62: Segment the normal track image using the first visual large model to obtain the fastener image Q. m Where m = 1, 2, ..., M; M represents the number of fastener images obtained by segmenting the normal track image; the fastener image Q m Includes spring clips, screws, fasteners, nuts, and metal parts;

[0017] S63: Utilize the second visual large model to analyze the fastener image Q m Perform segmentation to obtain segmented subgraphs. and its corresponding mask in, This indicates the fastener image Q. m The nth segmented sub-image obtained by segmentation; n = 1, 2, ..., N; N represents the sum of the segments of the fastener image Q. m The number of sub-images obtained from the segmentation process;

[0018] S64: From the segmented subgraph using polygon features Images of fastener nuts were initially selected;

[0019] S65: Manually verify the fastener and nut images initially selected in S64 to obtain the actual fastener and nut images;

[0020] S66: Perform global pooling on the mask corresponding to the real fastener nut image to obtain a normal nut feature vector among the plurality of normal nut feature vectors; wherein, the normal nut feature vector among the plurality of normal nut feature vectors is a channel feature vector obtained by performing global pooling on the mask corresponding to the real fastener nut image.

[0021] Preferably, the first visual large model is a Segment Anything model fine-tuned using labeled negative samples; wherein, the labeled negative samples are track images with missing fastener nuts.

[0022] Preferably, the second large visual model is a trained and optimized Mobile Segment Anything model; wherein, the second large visual model is trained and optimized using a "teacher-student" paradigm, and the Segment Anything model fine-tuned with labeled negative samples is used as the teacher model in the "teacher-student" paradigm.

[0023] Preferably, the industrial control computer is installed in the equipment compartment of the track defect comprehensive detection vehicle via a shock absorption device.

[0024] In a second aspect, a fastener nut missing identification system based on visual large model secondary segmentation is provided to implement the fastener nut missing identification method described in the first aspect. The system includes a track image acquisition unit and an industrial control computer. The industrial control computer includes a fastener image acquisition unit, a segmented sub-image and its corresponding mask acquisition unit, a fastener nut image preliminary screening unit, a global pooling unit, and a unit for determining whether a fastener nut is missing.

[0025] The track image acquisition unit is used to acquire the track image to be identified; wherein, the track image to be identified includes the track bed, rails and several fasteners;

[0026] The fastener image acquisition unit is used to segment the track image to be identified using a first visual large model to obtain the fastener image P. i Where i = 1, 2, ..., I; I represents the number of fastener images obtained by segmenting the track image to be identified; the fastener image P i Includes spring clips, screws, fasteners, nuts, and metal parts;

[0027] The segmentation sub-image and its corresponding mask acquisition unit are used to obtain the fastener image P using the second visual large model. i Perform segmentation to obtain segmented subgraph P i j and its corresponding mask F i j Among them, P i j This indicates the fastener image P. i The j-th segmented sub-image obtained by segmentation; j = 1, 2, ..., J; J represents the segmentation of the fastener image P i The number of sub-images obtained from the segmentation process;

[0028] The fastener nut image preliminary screening unit is used to filter the segmented sub-image P by using polygon features. i j Images of fastener nuts were initially selected;

[0029] The global pooling unit is used to perform global pooling on the mask corresponding to the fastener nut image initially screened by the fastener nut image preliminary screening unit.

[0030] The unit for determining whether a fastener nut is missing is used to calculate the similarity between the feature vector of the nut to be identified and the feature vectors of several normal nuts. If the similarity between the feature vector of the nut to be identified and the feature vector of a certain normal nut is less than a preset threshold, then the fastener nut image initially screened by the fastener nut image preliminary screening unit has a problem of missing fastener nuts. The feature vector of the nut to be identified is a channel feature vector obtained by global pooling of the mask corresponding to the fastener nut image initially screened by the fastener nut image preliminary screening unit.

[0031] Preferably, the track image acquisition unit to be identified includes a track defect comprehensive detection vehicle, a data acquisition beam, a high-definition vision sensor, a signal processing circuit, an analog-to-digital conversion unit, and a digital signal acquisition device;

[0032] The high-definition vision sensor is fixed on the data acquisition beam;

[0033] The data acquisition beam is fixed to the bottom of the track defect comprehensive detection vehicle by a shock absorption device;

[0034] The high-definition vision sensor is electrically connected to the industrial control computer in sequence through the signal processing circuit, the analog-to-digital conversion unit, and the digital signal acquisition unit;

[0035] The high-definition vision sensor is used to acquire simulated orbit images;

[0036] The signal processing circuit is used to perform a first preprocessing on the simulated track image; wherein, the first preprocessing includes signal amplification, filtering, level conversion, and anti-interference;

[0037] The analog-to-digital conversion unit is used to convert the preprocessed analog track image into a digital track image;

[0038] The digital signal acquisition device is used to perform a second preprocessing on the digital track image to obtain the track image to be identified; wherein, the second preprocessing includes caching, sorting, packaging, format conversion, error detection and verification;

[0039] The digital signal acquisition device is used to transmit the track image to be identified at high speed to the industrial control computer.

[0040] Preferably, the number of high-definition vision sensors is 5.

[0041] Thirdly, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the fastener nut missing identification method based on visual large model secondary segmentation described in the first aspect.

[0042] Fourthly, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, it implements the fastener nut missing identification method based on visual large model secondary segmentation described in the first aspect.

[0043] Fifthly, a computer program product is provided, including a computer program that, when executed by a processor, implements the fastener nut missing identification method based on visual large model secondary segmentation described in the first aspect.

[0044] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a method and system for identifying missing fastener nuts based on visual large model secondary segmentation. The present invention can accurately and efficiently identify the missing fastener nut situation when there are few missing fastener nut samples. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0046] Figure 1 A flowchart of a fastener nut missing identification method based on visual large model secondary segmentation provided by the present invention;

[0047] Figure 2 This is a schematic diagram of an electronic device provided by the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Firstly, such as Figure 1 As shown in the figure, this invention discloses a method for identifying missing fastener nuts based on secondary segmentation of a large visual model, including the following steps:

[0050] S1: Obtain an image of the track to be identified; wherein, the image of the track to be identified includes the track bed, rails, and several fasteners;

[0051] S2: The track image to be identified is segmented using the first visual large model to obtain the fastener image P. i Where i = 1, 2, ..., I; I represents the number of fastener images obtained by segmenting the track image to be identified; the fastener image P i Includes spring clips, screws, fasteners, nuts, and metal parts;

[0052] S3: Utilize the second visual large model to analyze the fastener image P i Perform segmentation to obtain segmented subgraph P i j and its corresponding mask F i j Among them, P i j This indicates the fastener image P. i The j-th segmented sub-image obtained by segmentation; j = 1, 2, ..., J; J represents the segmentation of the fastener image P i The number of sub-images obtained from the segmentation process;

[0053] S4: From the segmented subgraph P using polygon features i j Images of fastener nuts were initially selected;

[0054] It is understood that the shape of the polygonal feature is the same as the shape of the fastener nut;

[0055] S5: Perform global pooling on the mask corresponding to the fastener nut image initially selected in S4;

[0056] S6: Calculate the similarity between the feature vector of the nut to be identified and the feature vectors of several normal nuts. If the similarity between the feature vector of the nut to be identified and the feature vector of a certain normal nut is less than a preset threshold, then the fastener nut image initially screened in S4 has the problem of missing fastener nuts. The feature vector of the nut to be identified is the channel feature vector obtained by global pooling of the mask corresponding to the fastener nut image initially screened in S4.

[0057] In one embodiment, a certain normal nut feature vector among the plurality of normal nut feature vectors is obtained based on the following steps;

[0058] S61: Obtain a normal track image; wherein, the normal track image includes the track bed, rails, and several fasteners;

[0059] It is understood that the method for acquiring the normal orbit image is the same as the method for acquiring the orbit image to be identified.

[0060] S62: Segment the normal track image using the first visual large model to obtain the fastener image Q. m Where m = 1, 2, ..., M; M represents the number of fastener images obtained by segmenting the normal track image; the fastener image Q m Includes spring clips, screws, fasteners, nuts, and metal parts;

[0061] S63: Utilize the second visual large model to analyze the fastener image Q m Perform segmentation to obtain segmented subgraphs. and its corresponding mask in, This indicates the fastener image Q. m The nth segmented sub-image obtained by segmentation; n = 1, 2, ..., N; N represents the sum of the segments of the fastener image Q. m The number of sub-images obtained from the segmentation process;

[0062] S64: From the segmented subgraph using polygon features Images of fastener nuts were initially selected;

[0063] It is understood that the shape of the polygonal feature is the same as the shape of the fastener nut;

[0064] S65: Manually verify the fastener and nut images initially selected in S64 to obtain the actual fastener and nut images;

[0065] S66: Perform global pooling on the mask corresponding to the real fastener nut image to obtain a normal nut feature vector among the plurality of normal nut feature vectors; wherein, the normal nut feature vector among the plurality of normal nut feature vectors is a channel feature vector obtained by performing global pooling on the mask corresponding to the real fastener nut image.

[0066] In one embodiment, the first large visual model is a SegmentAnything model finely tuned using labeled negative samples; wherein the labeled negative samples are track images with missing fastener nuts.

[0067] Understandably, during the fine-tuning process, a small number of labeled negative samples are used, and transfer learning techniques are employed to update only some parameters of the Segment Anything model (these parameters include the preset threshold for similarity, the image size of the input model, the number of iterations, etc.).

[0068] The method for obtaining track images with missing fastener nuts is the same as that for the track images to be identified.

[0069] In one embodiment, the second large visual model is a trained and optimized Mobile Segment Anything model; wherein the second large visual model is trained and optimized using a "teacher-student" paradigm, and the Segment Anything model fine-tuned with labeled negative samples is used as the teacher model in the "teacher-student" paradigm.

[0070] Understandably, during training, the focus is on the model's segmentation accuracy in the fastener nut region. Model performance is optimized by adjusting hyperparameters such as the learning rate and batch size. Simultaneously, model pruning and quantization techniques are employed to further reduce model size and computational complexity, ensuring improved inference speed while maintaining accuracy. During training, model performance is periodically evaluated on the validation set, including metrics such as segmentation accuracy, recall, and inference speed. Training is halted when these metrics stabilize on the validation set and meet preset targets. Finally, a final evaluation is performed on an independent test set to ensure the model exhibits good generalization ability.

[0071] In one embodiment, the industrial control computer is installed in the equipment compartment of the track defect comprehensive detection vehicle via a shock absorption device.

[0072] Secondly, embodiments of the present invention also provide a fastener nut missing identification system based on visual large model secondary segmentation, used to implement the fastener nut missing identification method described in the first aspect, including a track image acquisition unit and an industrial control computer; the industrial control computer includes a fastener image acquisition unit, a segmented sub-image and its corresponding mask acquisition unit, a fastener nut image preliminary screening unit, a global pooling unit, and a unit for determining whether a fastener nut is missing.

[0073] The track image acquisition unit is used to acquire the track image to be identified; wherein, the track image to be identified includes the track bed, rails and several fasteners;

[0074] The fastener image acquisition unit is used to segment the track image to be identified using a first visual large model to obtain the fastener image P. i Where i = 1, 2, ..., I; I represents the number of fastener images obtained by segmenting the track image to be identified; the fastener image P i Includes spring clips, screws, fasteners, nuts, and metal parts;

[0075] The segmentation sub-image and its corresponding mask acquisition unit are used to obtain the fastener image P using the second visual large model. i Perform segmentation to obtain segmented subgraph P i j and its corresponding mask F i j Among them, P i j This indicates the fastener image P. i The j-th segmented sub-image obtained by segmentation; j = 1, 2, ..., J; J represents the segmentation of the fastener image P i The number of sub-images obtained from the segmentation process;

[0076] The fastener nut image preliminary screening unit is used to filter the segmented sub-image P by using polygon features. i j Images of fastener nuts were initially selected;

[0077] The global pooling unit is used to perform global pooling on the mask corresponding to the fastener nut image initially screened by the fastener nut image preliminary screening unit.

[0078] The unit for determining whether a fastener nut is missing is used to calculate the similarity between the feature vector of the nut to be identified and the feature vectors of several normal nuts. If the similarity between the feature vector of the nut to be identified and the feature vector of a certain normal nut is less than a preset threshold, then the fastener nut image initially screened by the fastener nut image preliminary screening unit has a problem of missing fastener nuts. The feature vector of the nut to be identified is a channel feature vector obtained by global pooling of the mask corresponding to the fastener nut image initially screened by the fastener nut image preliminary screening unit.

[0079] In one embodiment, the track image acquisition unit to be identified includes a track defect comprehensive detection vehicle, a data acquisition beam, a high-definition vision sensor, a signal processing circuit, an analog-to-digital conversion unit, and a digital signal acquisition device.

[0080] The high-definition vision sensor is fixed on the data acquisition beam;

[0081] The data acquisition beam is fixed to the bottom of the track defect comprehensive detection vehicle by a shock absorption device;

[0082] It is understandable that the data acquisition beam is connected to the track defect comprehensive inspection vehicle through a vibration damping device, which can reduce the impact of vehicle vibration on image acquisition.

[0083] The high-definition vision sensor is electrically connected to the industrial control computer in sequence through the signal processing circuit, the analog-to-digital conversion unit, and the digital signal acquisition unit;

[0084] The high-definition vision sensor is used to acquire simulated orbit images;

[0085] The signal processing circuit is used to perform a first preprocessing on the simulated track image; wherein, the first preprocessing includes signal amplification, filtering, level conversion, and anti-interference;

[0086] The analog-to-digital conversion unit is used to convert the preprocessed analog track image into a digital track image;

[0087] The digital signal acquisition device is used to perform a second preprocessing on the digital track image to obtain the track image to be identified; wherein, the second preprocessing includes caching, sorting, packaging, format conversion (converting to a data format that can be recognized and processed by an industrial control computer), error detection and verification;

[0088] The digital signal acquisition device is used to transmit the track image to be identified at high speed to the industrial control computer.

[0089] It is understood that the signal processing circuit is connected to the analog-to-digital conversion unit via a high-speed data cable.

[0090] In one embodiment, the number of high-definition vision sensors is five.

[0091] Understandably, the installation positions and angles of the five high-definition vision sensors were precisely calculated to ensure full coverage of the track surface and avoid blind spots.

[0092] The track image to be identified, the normal track image, and the track image with missing fastener nuts are all stitched images of simulated track images collected by five high-definition vision sensors (the stitching process is implemented by the industrial control computer).

[0093] Thirdly, embodiments of the present invention also provide an electronic device, such as... Figure 2 As shown, the electronic device may include a processor 201, a communication interface 202, a memory 203, and a communication bus 204. The processor 201, communication interface 202, and memory 203 communicate with each other via the communication bus 204. The processor 201 can call logical instructions from the memory 203 to execute a fastener nut missing identification method based on visual large model secondary segmentation.

[0094] Furthermore, the logical instructions in the aforementioned memory 203 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0095] Fourthly, embodiments of the present invention also provide a computer program product, the computer program product including a computer program, the computer program being stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer is able to execute the fastener nut missing identification method based on visual large model secondary segmentation provided by the above methods.

[0096] Fifthly, embodiments of the present invention also provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the fastener nut missing identification method based on visual large model secondary segmentation provided by the above methods.

[0097] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0098] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0099] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0100] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying missing fastener nuts based on secondary segmentation of a large visual model, characterized in that, Includes the following steps: S1: Obtain an image of the track to be identified; wherein, the image of the track to be identified includes the track bed, rails, and several fasteners; S2: The track image to be identified is segmented using the first visual large model to obtain the fastener image P. i Where i = 1, 2, ..., I; I represents the number of fastener images obtained by segmenting the track image to be identified; the fastener image P i Includes spring clips, screws, fasteners, nuts, and metal parts; S3: Utilize the second visual large model to analyze the fastener image P i Perform segmentation to obtain segmented subgraph P i j and its corresponding mask F i j Among them, P i j This indicates the fastener image P. i The j-th segmented sub-image obtained by segmentation; j = 1, 2, ..., J; J represents the segmentation of the fastener image P i The number of sub-images obtained from the segmentation process; S4: From the segmented subgraph P using polygon features i j Images of fastener nuts were initially selected; S5: Perform global pooling on the mask corresponding to the fastener nut image initially selected in S4; S6: Calculate the similarity between the feature vector of the nut to be identified and the feature vectors of several normal nuts. If the similarity between the feature vector of the nut to be identified and the feature vector of a certain normal nut is less than a preset threshold, then the fastener nut image initially screened in S4 has the problem of missing fastener nuts. The feature vector of the nut to be identified is the channel feature vector obtained by global pooling of the mask corresponding to the fastener nut image initially screened in S4.

2. The method for identifying missing fastener nuts based on visual large model secondary segmentation according to claim 1, characterized in that, The following steps are used to obtain a specific normal nut feature vector from the plurality of normal nut feature vectors; S61: Obtain a normal track image; wherein, the normal track image includes the track bed, rails, and several fasteners; S62: Segment the normal track image using the first visual large model to obtain the fastener image Q. m Where m = 1, 2, ..., M; M represents the number of fastener images obtained by segmenting the normal track image; the fastener image Q m Includes spring clips, screws, fasteners, nuts, and metal parts; S63: Utilize the second visual large model to analyze the fastener image Q m Perform segmentation to obtain segmented subgraphs. and its corresponding mask in, This indicates the fastener image Q. m The nth segmented sub-image obtained by segmentation; n = 1, 2, ..., N; N represents the sum of the segments of the fastener image Q. m The number of sub-images obtained from the segmentation process; S64: From the segmented subgraph using polygon features Images of fastener nuts were initially selected; S65: Manually verify the fastener and nut images initially selected in S64 to obtain the actual fastener and nut images; S66: Perform global pooling on the mask corresponding to the real fastener nut image to obtain a normal nut feature vector among the plurality of normal nut feature vectors; wherein, the normal nut feature vector among the plurality of normal nut feature vectors is a channel feature vector obtained by performing global pooling on the mask corresponding to the real fastener nut image.

3. The method for identifying missing fastener nuts based on secondary segmentation of a large visual model according to claim 2, characterized in that, The first large visual model is a Segment Anything model fine-tuned using labeled negative samples; wherein, the labeled negative samples are track images with missing fastener nuts.

4. The method for identifying missing fastener nuts based on secondary segmentation of a large visual model according to claim 3, characterized in that, The second large-scale visual model is a trained and optimized Mobile Segment Anything model. The second large-scale visual model is trained and optimized using a "teacher-student" paradigm, and the SegmentAnything model fine-tuned with labeled negative samples is used as the teacher model in the "teacher-student" paradigm.

5. A fastener nut missing identification system based on visual large model secondary segmentation, characterized in that, The method for identifying missing fastener nuts according to any one of claims 1-4 includes a track image acquisition unit and an industrial control computer; the industrial control computer includes a fastener image acquisition unit, a segmented sub-image and its corresponding mask acquisition unit, a fastener nut image preliminary screening unit, a global pooling unit, and a unit for determining whether a fastener nut is missing. The track image acquisition unit is used to acquire the track image to be identified; wherein, the track image to be identified includes the track bed, rails and several fasteners; The fastener image acquisition unit is used to segment the track image to be identified using a first visual large model to obtain the fastener image P. i Where i = 1, 2, ..., I; I represents the number of fastener images obtained by segmenting the track image to be identified; the fastener image P i Includes spring clips, screws, fasteners, nuts, and metal parts; The segmentation sub-image and its corresponding mask acquisition unit are used to obtain the fastener image P using the second visual large model. i Perform segmentation to obtain segmented subgraph P i j and its corresponding mask F i j Among them, P i j This indicates the fastener image P. i The j-th segmented sub-image obtained by segmentation; j = 1, 2, ..., J; J represents the segmentation of the fastener image P i The number of sub-images obtained from the segmentation process; The fastener nut image preliminary screening unit is used to filter the segmented sub-image P by using polygon features. i j Images of fastener nuts were initially selected; The global pooling unit is used to perform global pooling on the mask corresponding to the fastener nut image initially screened by the fastener nut image preliminary screening unit. The unit for determining whether a fastener nut is missing is used to calculate the similarity between the feature vector of the nut to be identified and the feature vectors of several normal nuts. If the similarity between the feature vector of the nut to be identified and the feature vector of a certain normal nut is less than a preset threshold, then the fastener nut image initially screened by the fastener nut image preliminary screening unit has a problem of missing fastener nuts. The feature vector of the nut to be identified is a channel feature vector obtained by global pooling of the mask corresponding to the fastener nut image initially screened by the fastener nut image preliminary screening unit.

6. The fastener nut missing identification system based on visual large model secondary segmentation according to claim 5, characterized in that, The track image acquisition unit to be identified includes a track defect comprehensive detection vehicle, a data acquisition beam, a high-definition vision sensor, a signal processing circuit, an analog-to-digital conversion unit, and a digital signal acquisition device. The high-definition vision sensor is fixed on the data acquisition beam; The data acquisition beam is fixed to the bottom of the track defect comprehensive detection vehicle by a shock absorption device; The high-definition vision sensor is electrically connected to the industrial control computer in sequence through the signal processing circuit, the analog-to-digital conversion unit, and the digital signal acquisition unit; The high-definition vision sensor is used to acquire simulated orbit images; The signal processing circuit is used to perform a first preprocessing on the simulated track image; wherein, the first preprocessing includes signal amplification, filtering, level conversion, and anti-interference; The analog-to-digital conversion unit is used to convert the preprocessed analog track image into a digital track image; The digital signal acquisition device is used to perform a second preprocessing on the digital track image to obtain the track image to be identified; wherein, the second preprocessing includes caching, sorting, packaging, format conversion, error detection and verification; The digital signal acquisition device is used to transmit the track image to be identified at high speed to the industrial control computer.

7. The fastener nut missing identification system based on visual large model secondary segmentation according to claim 6, characterized in that, The number of high-definition vision sensors is 5.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the fastener nut missing identification method based on visual large model secondary segmentation as described in any one of claims 1 to 4.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the fastener nut missing identification method based on visual large model secondary segmentation as described in any one of claims 1 to 4.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the fastener nut missing identification method based on visual large model secondary segmentation as described in any one of claims 1 to 4.

Images