AI-based rail transit industry material identity detection method and system

Abstract

The invention discloses an AI-based rail transit industry material identity detection method and system, and the method comprises the steps: collecting the image and weight information of different types of materials, and carrying out the preprocessing; constructing a multi-modal training data set based on the preprocessed material image, the weight information and the marked material position information in the image; training a pre-constructed material identification model by using the multi-modal training data set to obtain a trained material identification model; pre-processing material images and weight information acquired on site, inputting the pre-processed material images and weight information into the trained material identification model, and outputting category information of the materials. The method for identifying the identity information of the materials belongs to a non-contact mode, the materials can be protected to the maximum degree, and high practicability is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of rail transit, and specifically relates to a method and system for detecting the identity of materials in the rail transit industry based on AI. Background Art

[0002] At present, in the field of rail transit, the identity recognition of materials mainly uses methods such as visual inspection, barcodes, and RFID to identify and track materials. It relies more on the experience of personnel. And in the management and recognition methods of barcodes and RFID, a large amount of preliminary work is required, such as the printing or information injection of barcodes and RFID cards, and the operation process is relatively cumbersome. The more commonly used RFID technology belongs to a contact method. In the current content of material management in the rail transit industry, there are a large number of rubber parts, metal parts, and irregular items. There is a risk of dropping when RFID is bound to materials, and the problem of metal resistance has a greater impact on the recognition accuracy of RFID. The application of RFID technology requires the consumption of a large number of RFID cards, with a high cost. And in the application process, a large amount of initial preparation work also needs to be done: RFID information entry, binding and pasting of RFID to materials, supporting RFID scanning equipment, etc. Summary of the Invention

[0003] The present application provides a method and system for detecting the identity of materials in the rail transit industry based on AI to solve the problems existing in the identity recognition method of materials in the existing rail transit field, such as high usage cost, large workload, and cumbersome operation process.

[0004] According to a first aspect, in one embodiment, a method for detecting the identity of materials in the rail transit industry based on AI is provided. The method includes:

[0005] Collect images and weight information of different categories of materials and perform preprocessing;

[0006] Construct a multi-modal training dataset based on the preprocessed material images, weight information, and the marked material position information in the images;

[0007] Use the multi-modal training dataset to train a pre-constructed material recognition model to obtain a trained material recognition model;

[0008] Input the preprocessed material images and weight information collected on-site into the trained material recognition model to output the category information of the materials.

[0009] Further, collecting images and weight information of different categories of materials and performing preprocessing specifically includes:

[0010] For each model of each type of material, collect material images I from multiple directions through one or more cameras;

[0011] For each material of each model, collect the material weight information m through a weight sensor, convert the weight information m into an image size, and obtain the weight information M of a two-dimensional vector.

[0012] Furthermore, construct a multi-modal training dataset, specifically including:

[0013] The data format in the training set is [I, M, L, box], where:

[0014] I: Material image;

[0015] M: Weight information of the material;

[0016] L: Category information of the material corresponding to I and M;

[0017] box: Material position information annotated in the image I.

[0018] Furthermore, use the multi-modal training dataset to train a pre-constructed material recognition model to obtain a trained material recognition model, specifically including:

[0019] Construct a framework training model based on yolov5 for material recognition.

[0020] Furthermore, use the multi-modal training dataset to train a pre-constructed material recognition model to obtain a trained material recognition model, specifically including:

[0021] Input: 4-channel image;

[0022] Build a Focus structure: Use the Focus slicing method to convert the original 4-channel image into a 16-channel image, and at the same time reduce the resolution of the image to half of the original;

[0023] Skeleton network backbone: Incorporate the Cross Stage Partial Network (CSPnet) structure, and the Cspnet structure contains the Residual Network (Resnet) structure;

[0024] Neck network neck: Use the Feature Pyramid Network (FPN) + Path Aggregation Network (PAN) structure to fuse features;

[0025] Head network head: Use an end-to-end structure and directly adopt double-branch prediction, namely classification prediction and regression of position information.

[0026] Furthermore, use the multi-modal training dataset to train a pre-constructed material recognition model to obtain a trained material recognition model, specifically including:

[0027] Select a cost function during training:

[0028] Loss = Wbox *L box +W cls *L cls

[0029] Among them: W box : is the weighting coefficient of the material position information;

[0030] L box : the cost function of the position information of the material;

[0031] W cls : the weighting coefficient of the material category information;

[0032] L cls : the cost function of the material category information.

[0033] Furthermore, training the pre-constructed material recognition model with the multi-modal training dataset to obtain the trained material recognition model, specifically including:

[0034] The cost function of the position information box of the material is as follows, which integrates the area of the intersection and union set of the position information:

[0035]

[0036] Among them:

[0037]

[0038] w, h: the width and height of the predicted box;

[0039] w gt ,h gt : the width and height of the real box;

[0040] θ: is an empirical value;

[0041] IoU: the intersection and union ratio of the predicted box and the real box;

[0042] Among them: Ω characterizes the shape difference of the position information; Δ characterizes the distance difference of the position information; the cost of the box corrects the model by constraining the intersection and union area, shape difference, and distance difference of the position information.

[0043] Furthermore, training the pre-constructed material recognition model with the multi-modal training dataset to obtain the trained material recognition model, specifically including:

[0044] The cost function of the material category information is as follows:

[0045]

[0046] S: the number of anchor points in the image;

[0047] B: The number of categories;

[0048] Whether there is an object at position (i, j). If so, this value is 1; if not, this value is 0;

[0049] The probability that the predicted category of the object at position (i, j) is cls;

[0050] Calculate the cost information only for the positions where there are objects.

[0051] Furthermore, the material image and weight information collected on-site are preprocessed and then input into the trained material recognition model to output the category information of the material, specifically including:

[0052] After preprocessing the image information I collected on-site and the weight information m of the materials, input them into the trained material recognition model Model to output the category information of the materials;

[0053] According to the category information of the materials, automatically map to obtain the detailed information of the materials, and report the detailed information of the materials to the management system in real time to achieve automatic data filling.

[0054] According to the second aspect, in one embodiment, an AI-based material identity detection system for the rail transit industry is provided. The system includes:

[0055] A data acquisition module, configured to collect the images and weight information of different categories of materials and perform preprocessing;

[0056] A training set construction module, configured to construct a multi-modal training data set based on the material images, weight information, and the labeled material position information in the images obtained by preprocessing;

[0057] A model training module, configured to train the pre-constructed material recognition model using the multi-modal training data set to obtain a trained material recognition model;

[0058] A material recognition module, configured to input the material image and weight information collected on-site after preprocessing into the trained material recognition model to output the category information of the materials.

[0059] This application provides an AI-based method and system for detecting the identity of materials in the rail transit industry, which collect images and weight information of different categories of materials and perform preprocessing; construct a multi-modal training dataset based on the material images, weight information, and the marked material position information in the images obtained from the preprocessing; train a pre-constructed material recognition model using the multi-modal training dataset to obtain a trained material recognition model; and input the material images and weight information collected on-site after preprocessing into the trained material recognition model to output the category information of the materials. The method for identifying the identity information of materials proposed in the present invention is contactless and can protect the materials to the greatest extent; this method is highly feasible and only requires installing image sensors and lighting equipment at appropriate positions; the method of the present invention uses multi-modal information and fully integrates the image information and weight information of the materials; the algorithm model constructed by this method supports the input of multi-modal information; for material identity recognition, this method belongs to an end-to-end manner, greatly simplifying the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 FIG. 6 is a flowchart of an AI-based method for detecting the identity of materials in the rail transit industry provided by an embodiment of the present invention;

[0061] Figure 2 FIG. 10 is a YOLO network architecture diagram in an AI-based method for detecting the identity of materials in the rail transit industry provided by an embodiment of the present invention;

[0062] Figure 3 FIG. 14 is a schematic logical structure diagram of an AI-based system for detecting the identity of materials in the rail transit industry provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0063] The present invention will be further described in detail below in conjunction with the drawings through specific embodiments. Similar components in different embodiments are labeled with related similar reference numerals. In the following embodiments, many detailed descriptions are provided to enable a better understanding of the present application. However, those skilled in the art can easily recognize that some of the features can be omitted in different situations, or can be replaced by other components, materials, or methods. In some cases, some operations related to the present application are not shown or described in the specification to avoid overwhelming the core part of the present application with excessive descriptions. For those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0064] In addition, the features, operations or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can also be reordered or adjusted in a manner obvious to those skilled in the art. Therefore, the various sequences in the specification and drawings are only for clearly describing a certain embodiment and do not mean a necessary sequence, unless it is stated that a certain sequence must be followed.

[0065] A method for detecting the identity of materials in the rail transit industry based on AI provided by the first embodiment of the present invention will be described in detail below in conjunction with Figure 1 this.

[0066] As Figure 1 shown, in step S100, images and weight information of different categories of materials are collected and preprocessed.

[0067] The above steps specifically include:

[0068] Image acquisition: For each type and model of material, images of the material are collected from multiple directions through one or more cameras;

[0069] Weight information acquisition: For different models of materials, the collected images may not change much, so a weight sensor is introduced here. For each different model of material, its weight information is obtained.

[0070] Data preprocessing: For the collected weight information data m, the padding method is used to convert the data into the size of the image to obtain the preprocessed weight information M of the material, and M is a two-dimensional vector.

[0071] As Figure 1 shown, in step S200, based on the preprocessed material images, weight information, and the position information of the materials in the labeled images, a multi-modal training dataset is constructed.

[0072] The above steps specifically include:

[0073] For the collected image data I, the target detection method is used to label the position information of the materials and save it as box.

[0074] Construct a multi-modal training dataset, and the data format in the training set is [I, M, L, box], where:

[0075] I: Material image;

[0076] M: Weight information of the material;

[0077] L: Category information of the material corresponding to I and M;

[0078] box: Position information of the materials labeled in image I.

[0079] As Figure 1 shown, in step S300, the pre-constructed material recognition model is trained using the multi-modal training dataset to obtain the trained material recognition model.

[0080] The above steps specifically include:

[0081] Construct a multi-modal algorithm model Model. In this embodiment, a deep learning model is selected. Here, the yolov5 framework training model is selected, and the input of the model needs to be modified to four channels.

[0082] (1) Build the Focus structure

[0083] Adopt the focus slicing method to convert the original 4-channel image into 16-channel information, and at the same time reduce the resolution of the image to half of the original. In this way, without losing information, the computational complexity of the algorithm is reduced. The way Focus reduces resolution: separate the odd and even rows and columns in the image and reconstruct the image.

[0084] (2) Incorporate the CSPnet structure in the backbone

[0085] Its structure is as Figure 2 . Incorporate the CSPnet structure in the backbone, which contains the residual structure of Resnet, facilitating convergence during the model optimization process. The specific connection method: separate the input into two inputs according to the channels; one branch enters the denset branch, and finally at the transition structure, merge the other input branch and the output branch of denset. In addition, the CSP structure is also introduced to fuse the information of multiple communications and make full use of the extracted feature information.

[0086] (3) In the neck stage, adopt the FPN+PAN structure to fuse the features

[0087] In this embodiment, FPN is selected to extract the location information from the deep semantic information and fuse it with the shallow location information to solve the feature extraction of small targets; the PAN structure is selected, which adopts a bottom-up structure and fuses the information with the feature layer of the same resolution in the previous layer in an add way.

[0088] (4) In the head layer, adopt an end-to-end structure and directly use double-branch prediction, which are classification prediction and regression of location information respectively. In the head layer, adopt a multi-scale regression method to simultaneously consider the detection of large and small targets.

[0089] Train the model:

[0090] When training, select the cost function:

[0091] Loss = W box *L box +W cls *L cls

[0092] Where: W box : is the weighting coefficient of the material position information;

[0093] L box : is the cost function of the position information of the material;

[0094] W cls : is the weighting coefficient of the material category information;

[0095] L cls : is the cost function of the material category information.

[0096] The cost function of the position information box of the material is as follows, which integrates the area of the intersection and union set of the position information:

[0097]

[0098] Where:

[0099]

[0100] w, h: are the width and height of the predicted box;

[0101] w gt ,h gt : are the width and height of the true box;

[0102] θ: is an empirical value;

[0103] IoU: is the intersection-over-union ratio of the predicted box and the true box;

[0104] Where: Ω represents the shape difference of the position information; Δ represents the distance difference of the position information; the cost of the box corrects the model by constraining the intersection and union area, shape difference, and distance difference of the position information.

[0105] The cost function of the material category information is as follows:

[0106]

[0107] S: is the number of anchors in the image;

[0108] B: is the number of categories;

[0109] Whether there is an object at the position (i, j), if so, this value is 1; if not, this value is 0;

[0110] The probability that the predicted class of the target object at position (i, j) is cls;

[0111] Calculate the cost information only at the positions where the target object exists.

[0112] Finally, the model uses the common Adam optimizer to optimize the model.

[0113] Such as Figure 1 As shown, in step S400, the material image and weight information collected on-site are preprocessed and then input into the trained material recognition model to output the category information of the material.

[0114] Specifically, the usage of the material recognition model is as follows:

[0115] R = Model(I, M; θ)

[0116] Where: Model is the established recognition model;

[0117] I: The collected image data;

[0118] M: The preprocessed weight information;

[0119] θ: The trained model parameter information;

[0120] In actual use, the on-site collected image information I and the weight information m of the collected materials are preprocessed to obtain M, which is input into the above Model model, and the category information of the materials is output. According to the category information of this data, the detailed information of the materials is mapped. The mapped detailed information is reported to the management system in real time to achieve automatic data filling.

[0121] It should be noted that in this embodiment, the image information and weight information are fused. As an option, the weight information can be converted into three-dimensional laser information, but the real-time performance and cost are slightly higher. In addition, during the model training process, for each type of material, the clustering method can be used to set the prior box.

[0122] Corresponding to the above-disclosed AI-based method for detecting the identity of materials in the rail transit industry, an embodiment of the present invention also discloses an AI-based system for detecting the identity of materials in the rail transit industry, such as Figure 3 As shown, it specifically includes:

[0123] A data acquisition module, configured to acquire the images and weight information of different types of materials and perform preprocessing;

[0124] A training set construction module, configured to construct a multi-modal training data set based on the material images, weight information, and the marked material position information in the images obtained through preprocessing;

[0125] A model training module, configured to train a pre-constructed material recognition model using the multi-modal training data set to obtain a trained material recognition model;

[0126] A material recognition module, configured to input the preprocessed material images and weight information collected on-site into the trained material recognition model and output the category information of the material.

[0127] It should be noted that for the detailed description of a fine characterization system for sandstone and mudstone based on waveform phased array technology combined with model inversion provided in an embodiment of the present invention, reference can be made to the related description of an AI-based method for detecting the identity of materials in the rail transit industry provided in an embodiment of the present application, which will not be elaborated here.

[0128] Those skilled in the art can understand that all or part of the functions of the above methods can be implemented in a hardware manner or in a computer program manner. When all or part of the functions in the above embodiments are implemented in a computer program manner, the program can be stored in a computer-readable storage medium. The storage medium may include: read-only memory, random access memory, magnetic disk, optical disk, hard disk, etc. The above functions can be realized by a computer executing the program. For example, the program is stored in the memory of the device, and when the processor executes the program in the memory, the above all or part of the functions can be realized. In addition, when all or part of the functions in the above embodiments are implemented in a computer program manner, the program can also be stored in a storage medium such as a server, another computer, magnetic disk, optical disk, flash drive, or mobile hard disk, downloaded or copied and saved to the memory of the local device, or the system of the local device is updated in version. When the processor executes the program in the memory, the above all or part of the functions in the above embodiments can be realized.

[0129] The above uses specific examples to elaborate on the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art of the present invention, based on the idea of the present invention, several simple deductions, deformations, or substitutions can be made.

Claims

1. A material identity detection method for the rail transit industry based on AI, characterized in that: The method comprises: Collect images and weight information of different types of materials and perform pre-processing; Construct a multimodal training dataset based on the preprocessed material images, weight information, and material position information in the annotated images; Using the multimodal training data set to train a pre-built material recognition model to obtain a trained material recognition model; The material images and weight information collected on site are preprocessed and input into the trained material recognition model to output the material category information.

2. The AI-based material identity detection method for rail transit industry as claimed in claim 1, characterized in that: Collect images and weight information of different types of materials and perform preprocessing, including: For each type of material, one or more cameras are used to collect material images I from multiple directions; For each type of material, the material weight information m is collected by a weight sensor, and the weight information m is converted into an image size to obtain the weight information M of a two-dimensional vector.

3. The AI-based material identity detection method for rail transit industry as claimed in claim 1, characterized in that: Construct a multimodal training dataset, including: The data format in the training set is [I,M,L,box], where: I: material image; M: Material weight information; L: Category information of materials corresponding to I and M; box: the material location information marked in image I.

4. The AI-based material identity detection method for rail transit industry as claimed in claim 1, characterized in that: The pre-built material recognition model is trained using the multimodal training data set to obtain a trained material recognition model, specifically including: Build a framework training model based on yolov5 for material recognition.

5. The AI-based material identity detection method for rail transit industry as claimed in claim 4, characterized in that: The pre-built material recognition model is trained using the multimodal training data set to obtain a trained material recognition model, specifically including: Input: 4-channel image; Build the Focus structure: Use focus slicing to convert the original 4-channel image into 16-channel, and reduce the image resolution to half of the original; Backbone network: It integrates the cross-stage local network CSPnet structure, which contains the residual network Resnet structure; Neck network: FPN+PAN structure is used to fuse features; Head network: It adopts an end-to-end structure and directly uses dual-branch prediction, which is classification prediction and position information regression.

6. The AI-based material identity detection method for rail transit industry as claimed in claim 1, characterized in that: The pre-built material recognition model is trained using the multimodal training data set to obtain a trained material recognition model, specifically including: The cost function used during training is: Loss=W box *L box +W cls *L cls Where: W box : is the weighting coefficient of material location information; L box : The cost function of the location information of materials; W cls : Weighting coefficient of material category information; L cls : The cost function of the category information of materials.

7. The AI-based material identity detection method for rail transit industry as claimed in claim 6, characterized in that: The pre-built material recognition model is trained using the multimodal training data set to obtain a trained material recognition model, specifically including: The cost function of the material location information box is as follows, which combines the area of ​​the intersection and union of the location information: in: w,h: predict the width and height of the box; w gt ,h gt : The width and height of the real box; θ: is an empirical value; IoU: The intersection-over-union ratio between the predicted box and the real box; Among them: Ω represents the shape difference of the position information; Δ represents the distance difference of the position information; the cost of the box corrects the model by constraining the intersection area, shape difference and distance difference of the position information.

8. The AI-based material identity detection method for rail transit industry as claimed in claim 6, characterized in that: The pre-built material recognition model is trained using the multimodal training data set to obtain a trained material recognition model, specifically including: The cost function of the material category information is as follows: S: the number of anchor points in the image; B: number of categories; Is there a target at position (i, j)? If so, this value is 1; if not, this value is 0; At position (i, j), the probability that the predicted target category is cls; The cost information is calculated only for the locations where the target objects exist.

9. The AI-based material identity detection method for rail transit industry as claimed in claim 1, characterized in that: The material images and weight information collected on site are pre-processed and input into the trained material recognition model to output the material category information, including: After preprocessing, the image information I and the weight information m of the materials collected on site are input into the trained material recognition model Model, and the category information of the materials is output; According to the category information of materials, the detailed information of materials is automatically mapped and submitted to the management system in real time to realize automatic data filling.

10. An AI-based material identity detection system for the rail transit industry, characterized in that: The system comprises: Data acquisition module, used to collect images and weight information of different types of materials and perform preprocessing; A training set construction module is used to construct a multimodal training data set based on the preprocessed material images, weight information, and material position information in the annotated images; A model training module, used to train a pre-built material recognition model using the multimodal training data set to obtain a trained material recognition model; The material identification module is used to input the material images and weight information collected on site into the trained material identification model after preprocessing, and output the material category information.

Images