Communication cabinet based on multi-modal information and internal equipment BIM detection method

Abstract

The invention relates to a communication cabinet and internal equipment BIM detection method based on multi-modal information, and the method comprises the steps: building a TOF optical pulse-temperature propagation velocity compensation model, improving a measurement algorithm of a TOF sensor, and then measuring the cabinet, and obtaining the size data of the cabinet; the method comprises the following steps: acquiring massive images of various types of equipment in a cabinet, marking to obtain a first data set, performing YOLOv10 model training, and outputting position information of the equipment in the cabinet; acquiring massive images of equipment labels of various types of equipment in the cabinet, marking to obtain a second data set, performing YOLOv10 model training, selecting the positions of the equipment labels, and identifying character information of the equipment labels; multi-modal information is collected for comparison, and detection is completed. According to the method, the size of the railway communication cabinet and the information in the label of the internal equipment are collected and then compared with the BIM model, rapid detection of digital engineering is achieved, the BIM detection efficiency is improved, and the accuracy of the BIM model is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of railway communication BIM detection technology, and in particular to a communication cabinet and internal equipment BIM detection method based on multimodal information. Background Art

[0002] The railway system is becoming increasingly large and complex, and its operation and maintenance model urgently needs to be transformed and upgraded to improve operation and maintenance efficiency. BIM (Building Information Modeling), as a digital model containing rich building facility information, has been applied in the entire life cycle of railway projects. Its model verification has become a key link in ensuring the quality of railway projects and the efficiency of operation and maintenance.

[0003] At present, the common BIM detection method is implemented through manual inspection or software based on predetermined rules. Manual inspection is more intuitive, but the efficiency is low, and errors may occur due to differences in the professional level and subjective judgment standards of auditors. For software detection methods, the detection speed is faster, but it relies too much on pre-set rules and algorithms, and has poor flexibility.

[0004] Therefore, it is necessary to propose a new method to overcome the above defects. Summary of the invention

[0005] The purpose of the present invention is to provide a BIM detection method for communication cabinets and internal equipment based on multimodal information to solve the problems of low efficiency in manual inspection and poor flexibility in software detection.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] A BIM detection method for communication cabinets and internal equipment based on multimodal information, the method comprising:

[0008] Establish a TOF light pulse-temperature propagation velocity compensation model to improve the measurement algorithm of the TOF sensor, thereby compensating for the error of the results obtained by the TOF sensor at different temperatures;

[0009] The cabinet is measured using a TOF sensor with an improved measurement algorithm to obtain the cabinet's dimension data;

[0010] Collect a large number of images of equipment inside various cabinets, and obtain the first data set after annotation;

[0011] Use the massive images of equipment inside the cabinet and the first data set to train the YOLOv10 model and output the location information of the equipment inside the cabinet;

[0012] Collect a large number of images of equipment labels inside various cabinets, and obtain the second data set after annotation;

[0013] Use the massive images of equipment signs and the second data set to train the YOLOv10 model, select the location of the equipment sign, and recognize the text information on the equipment sign;

[0014] Collect the cabinet's dimensional data, the location information of the equipment inside the cabinet, and the text information on the equipment label, and compare them with the design information in the BIM model to complete the inspection.

[0015] Furthermore, the cabinet dimensions include cabinet length, cabinet width, and cabinet height.

[0016] Furthermore, a TOF light pulse-temperature propagation velocity compensation model is established, including:

[0017] Taking the temperature into account, calculate the refractive index n of light in air:

[0018]

[0019] in:

[0020] T is absolute temperature;

[0021] S is pressure;

[0022] Get the distance d measured by the TOF sensor each time i :

[0023]

[0024] in:

[0025] v is the speed of light in air;

[0026] c is the speed of light in vacuum;

[0027] n is the refractive index;

[0028] t i The time of each measurement by the TOF sensor;

[0029] Computer cabinet size correction data:

[0030]

[0031] in:

[0032] d 1 and d 2 The distance measured twice by the TOF sensor;

[0033] θ is the angle between two measurements by the TOF sensor.

[0034] Furthermore, a large number of images of various types of equipment inside the cabinets are collected and annotated to obtain the first data set, including:

[0035] The images of various equipment inside the cabinets are collected by cameras and annotated using the LabelImg image annotation tool. After the annotation is completed, the "txt" format file is exported as the first data set.

[0036] Furthermore, the location information of the equipment inside the cabinet is output, including:

[0037] Improve the upper left corner coordinates and lower right corner coordinates of the bounding box of the original output to the center point coordinates;

[0038]

[0039]

[0040] in:

[0041] x 1 and 1 is the coordinate of the upper left corner of the bounding box;

[0042] x 2 and 2 is the coordinate of the lower right corner of the bounding box;

[0043] x c and c is the coordinate of the center point of the bounding box;

[0044] x c Compare the position relationship of the equipment in the horizontal direction, y c Compare the positional relationship of the devices in the vertical direction to form the position distribution information of the devices inside the cabinet.

[0045] Furthermore, a large number of images of equipment labels of various equipment inside the cabinets are collected and annotated to obtain the second data set, including:

[0046] The equipment label images of various equipment inside the cabinets are collected by the camera, and the LabelImg image annotation tool is used to annotate them. After the annotation is completed, the "txt" format file is exported as the second data set.

[0047] Furthermore, the text information on the equipment label is identified, including:

[0048] Configure the deep learning framework Padd1ePadd1 version to 2.1.2 or above, and select ch PP-OCR v 4 rec It is used as a text recognition model to recognize the text information on equipment labels and further obtain equipment information.

[0049] Furthermore, further device information is obtained, including:

[0050] Cut content based on the common characters on the equipment label;

[0051] Extract the device model obtained after cutting;

[0052] Query the device model in the device information database to obtain device information.

[0053] Furthermore, the cabinet size data, the location information of the equipment inside the cabinet, and the text information of the equipment label are collected and compared with the design information in the BIM model, including:

[0054] Combine the queried equipment information with the collected cabinet size data, cabinet internal equipment location information, and equipment label text information, and compare them with the attribute information in the BIM model;

[0055] The comparison results are written into the document through the python-docx library to generate a detection report.

[0056] On the other hand, a BIM detection system for communication cabinets and internal equipment based on multimodal information is provided, and the system is used to implement the method described, including:

[0057] The temperature compensation module is used to establish a TOF light pulse-temperature propagation speed compensation model, improve the measurement algorithm of the TOF sensor, and thus compensate for the error of the results obtained by the TOF sensor at different temperatures;

[0058] A cabinet dimension data acquisition module is used to acquire the cabinet dimension data;

[0059] The first data set acquisition module is used to collect a large number of images of various types of internal equipment in the cabinets, and obtain the first data set after annotation;

[0060] The first model training module is used to perform YOLOv10 model training using a large number of images of equipment inside the cabinet and a first data set, and output position information of equipment inside the cabinet;

[0061] The second data set acquisition module is used to collect a large number of images of equipment labels of various types of equipment inside the cabinets, and obtain the second data set after annotation;

[0062] The second model training module is used to train the YOLOv10 model using the massive images of the equipment sign and the second data set, select the equipment sign location, and recognize the equipment sign text information;

[0063] The collection detection module is used to collect the cabinet size data, the location information of the equipment inside the cabinet, and the text information of the equipment label, and compare them with the design information in the BIM model to complete the detection.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] The present invention provides a BIM detection method for communication cabinets and internal equipment based on multimodal information. Through image processing technology and text processing technology, the dimensions of railway communication cabinets and information on internal equipment labels are collected, and then compared with the BIM model to achieve rapid detection of digital engineering, thereby improving the efficiency of BIM detection and ensuring the accuracy of the BIM model. It does not rely too much on pre-set rules and algorithms, has strong flexibility, and provides guarantee for the digital operation and maintenance of the four electrical equipment of railways. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, drawings of other embodiments can be obtained based on these drawings without paying creative work.

[0067] Figure 1 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0068] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. The preferred embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thoroughly understood.

[0069] It should be noted that the terms "comprises", "comprising", "includes", "comprises" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0070] In addition, steps "S1", "S2" and the like involved in the embodiments are only for the convenience of describing the technical solutions and understanding the specific steps and contents of the embodiments, and should not be understood as limitations on the order of the steps. Any changes to the order of the steps should be within the scope of protection of the present invention.

[0071] Meanwhile, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance. Of course, such objects can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0072] At present, most of the methods for BIM detection of communication cabinets and internal equipment are based on rules specified by experts and rely on pre-set rules and algorithms. In view of such defects, the present invention introduces a deep learning model into the method to improve the flexibility of detection. Specifically, the present invention provides a BIM detection method for communication cabinets and internal equipment based on multimodal information, such as Figure 1 , the method specifically comprises the following steps:

[0073] S1: A TOF light pulse-temperature propagation velocity compensation model is established to improve the measurement algorithm of the TOF sensor, thereby compensating for the errors in the results obtained by the TOF sensor at different temperatures.

[0074] Since the ambient temperature will affect the propagation speed of the signal, the present invention establishes a temperature-compensated TOF algorithm in order to obtain more accurate measurement results. For optical pulse TOF ranging, the change of light speed in the air is mainly affected by the refractive index of the air, and the refractive index will change with the changes of temperature, pressure and other factors.

[0075] S101: Considering the temperature, calculate the refractive index n of light in air:

[0076]

[0077] in:

[0078] T is absolute temperature;

[0079] S is pressure;

[0080] S102: Obtain the distance d measured by the TOF sensor each time i :

[0081]

[0082] in:

[0083] v is the speed of light in air;

[0084] c is the speed of light in vacuum;

[0085] n is the refractive index;

[0086] t i The time of each measurement by the TOF sensor;

[0087] S103: Calculate cabinet size correction data s:

[0088]

[0089] in:

[0090] d 1 and d2 The distance measured twice by the TOF sensor;

[0091] θ is the angle between two measurements by the TOF sensor.

[0092] The cabinet dimensions include the cabinet length, cabinet width, and cabinet height. The cabinet dimension correction data calculated by the above process is the cabinet dimension data.

[0093] S2: Use the TOF sensor with improved measurement algorithm to measure the cabinet and obtain the size data of the cabinet.

[0094] S3: Collect a large number of images of various types of equipment inside the cabinets, and obtain the first data set after annotation. Including:

[0095] The images of various equipment inside the cabinets are collected by cameras and annotated using the LabelImg image annotation tool. After the annotation is completed, the "txt" format file is exported as the first data set.

[0096] S4: Use the massive images of the equipment inside the cabinet and the first data set to train the YOLOv10 model and output the location information of the equipment inside the cabinet.

[0097] The present invention improves the output coordinates, and improves the upper left corner coordinates and the lower right corner coordinates of the original output boundary box into the center point coordinates;

[0098]

[0099]

[0100] in:

[0101] x 1 and 1 is the coordinate of the upper left corner of the bounding box;

[0102] x 2 and 2 is the coordinate of the lower right corner of the bounding box;

[0103] x c and c is the coordinate of the center point of the bounding box;

[0104] x c Compare the position relationship of the equipment in the horizontal direction, y c Compare the positional relationship of the devices in the vertical direction to form the position distribution information of the devices inside the cabinet.

[0105] S5: Collect a large number of images of equipment labels of various types of equipment inside cabinets, and obtain the second data set after annotation.

[0106] Specifically include:

[0107] The equipment label images of various equipment inside the cabinets are collected by the camera, and the LabelImg image annotation tool is used to annotate them. After the annotation is completed, the "txt" format file is exported as the second data set.

[0108] S6: Use the massive images of equipment signs and the second data set to train the YOLOv10 model, select the location of the equipment sign, and recognize the text information on the equipment sign. Specifically include:

[0109] Configure the deep learning framework Padd1ePadd1 version to 2.1.2 or above, and select ch PP-OCR v 4 rec Used as a text recognition model to recognize text information on equipment labels.

[0110] After identification, further device information can be obtained through query, including:

[0111] S601: Cutting content based on the common character identification on the equipment label;

[0112] S602: extracting the device model obtained after cutting;

[0113] S603: Query the device model in the device information database to obtain device information.

[0114] S7: Collect the cabinet size data, the location information of the equipment inside the cabinet, and the text information of the equipment label, and compare them with the design information (attribute information) in the BIM model in combination with the queried equipment information to complete the inspection. The comparison results are written into the document through the python-docx library to generate the inspection report.

[0115] On the other hand, the present invention also provides a BIM detection system for communication cabinets and internal equipment based on multimodal information, which is used to implement the above method, including:

[0116] A temperature compensation module is used to establish a TOF light pulse-temperature propagation speed compensation model, improve the measurement algorithm of the TOF sensor, and thus compensate for the error of the results obtained by the TOF sensor at different temperatures, corresponding to S1 of the above method;

[0117] A cabinet size data acquisition module, used to acquire cabinet size data, corresponding to S2 of the above method;

[0118] A first data set acquisition module is used to acquire a large number of images of various types of internal equipment in the cabinets, and obtain a first data set after annotation, corresponding to S3 of the above method;

[0119] A first model training module is used to perform YOLOv10 model training using a large number of images of equipment inside the cabinet and a first data set, and output position information of equipment inside the cabinet, corresponding to S4 of the above method;

[0120] A second data set acquisition module is used to acquire a large number of images of equipment labels of various types of equipment inside the cabinets, and obtain a second data set after labeling, which corresponds to S5 of the above method;

[0121] A second model training module is used to perform YOLOv10 model training using a large number of images of equipment signs and a second data set, select equipment sign locations, and recognize equipment sign text information, corresponding to S6 of the above method;

[0122] The collection detection module is used to collect the cabinet size correction data, the location information of the equipment inside the cabinet, and the text information of the equipment label, and compare them with the design information in the BIM model, which corresponds to S7 of the above method.

[0123] The method of the present invention is described in detail below through specific embodiments:

[0124] Firstly, the cabinet is measured using a TOF sensor with an improved measurement algorithm to obtain the size data of the cabinet.

[0125] After that, use the camera to collect a large number of images of various types of equipment inside the cabinet, such as 1,000 images, including transmission equipment, switching equipment, power supply equipment, data communication equipment, wiring equipment, etc. Use the LabelImg image annotation tool to export the "txt" format file as the data set after the annotation is completed. In order to train and evaluate the performance of the model, the data set is divided into 7:2:1.

[0126] The training directory of the model is set to be a large number of images of various types of internal equipment in the cabinet and the annotated "txt" files. The mean average precision (mAP@0.5) is used as the evaluation indicator of the model's target detection effect. The output coordinates of the model are improved, and the coordinates of the upper left corner and lower right corner of the original output bounding box are improved to the coordinates of the center point. After the model training accuracy is achieved, the "quantization" toolkit is used to quantize the model.

[0127] Next, use the camera to collect a large number of equipment images with labels, such as 1,000 images, and use the LabelImg image annotation tool to select the location of the label on the equipment. After the data standard is completed, the data set is divided into 7:2:1.

[0128] The training directory of the model is set to be a large number of equipment images with labels and annotated "txt" files. The average precision (mAP@0.5) is used as the evaluation indicator of the model's target detection effect. After the model training accuracy is achieved, the "quantization" toolkit is used to quantize the model.

[0129] Then, configure the deep learning framework PaddlePaddl version to 2.1.2 or above, select "ch_PP-OCRv4_rec" as the text recognition model, and then quantify the model through the PaddleSlim tool.

[0130] Since the information in the label is usually composed of "<attribute: attribute value", the cutting content can be marked by ":". The device model obtained after cutting is extracted and queried in the device information library to obtain more information about the device itself.

[0131] Finally, the cabinet size correction data, cabinet internal equipment location information, equipment label text information are collected, combined with the queried equipment information, and compared with the design information (attribute information) in the BIM model to complete the inspection. The comparison results are written into the document through the python-docx library to generate a test report.

[0132] Those skilled in the art will appreciate that all or part of the functions of the embodiments of the present invention may be implemented by hardware or by computer programs. When all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory, a random access memory, a disk, an optical disk, a hard disk, etc., and the program is executed by a computer to implement the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above-mentioned functions can be implemented. In addition, when all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program may also be stored in a storage medium such as a server, another computer, a disk, an optical disk, a flash disk or a mobile hard disk, and is downloaded or copied and saved in the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0133] The above specific examples are used to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art, according to the concept of the present invention, some simple deductions, modifications or substitutions can be made.

Claims

1. A BIM detection method for communication cabinets and internal equipment based on multimodal information, characterized in that: The method comprises: Establish a TOF light pulse-temperature propagation velocity compensation model to improve the measurement algorithm of the TOF sensor, thereby compensating for the error of the results obtained by the TOF sensor at different temperatures; The cabinet is measured using a TOF sensor with an improved measurement algorithm to obtain the cabinet's dimension data; Collect a large number of images of equipment inside various cabinets, and obtain the first data set after annotation; Use the massive images of equipment inside the cabinet and the first data set to train the YOLOv10 model and output the location information of the equipment inside the cabinet; Collect a large number of images of equipment labels inside various cabinets, and obtain the second data set after annotation; Use the massive images of equipment signs and the second data set to train the YOLOv10 model, select the location of the equipment sign, and recognize the text information on the equipment sign; Collect the cabinet's dimensional data, the location information of the equipment inside the cabinet, and the text information on the equipment label, and compare them with the design information in the BIM model to complete the inspection.

2. The method for BIM detection of communication cabinets and internal equipment based on multimodal information according to claim 1 is characterized in that: The cabinet dimensions include cabinet length, cabinet width, and cabinet height.

3. The method for BIM detection of communication cabinets and internal equipment based on multimodal information according to claim 2 is characterized in that: Establish a TOF optical pulse-temperature propagation velocity compensation model, including: Taking the temperature into account, calculate the refractive index n of light in air: in: T is absolute temperature; S is pressure; Get the distance d measured by the TOF sensor each time i : i=1 or 2 in: v is the speed of light in air; c is the speed of light in vacuum; n is the refractive index; t i The time of each measurement by the TOF sensor; Computer cabinet size correction data: in: d1 and d2 are the distances measured twice by the TOF sensor; θ is the angle between two measurements by the TOF sensor.

4. The method for BIM detection of communication cabinets and internal equipment based on multimodal information according to claim 3 is characterized in that: Collect a large number of images of various types of equipment inside cabinets, and obtain the first data set after annotation, including: The images of various equipment inside the cabinets are collected by cameras and annotated using the LabelImg image annotation tool. After the annotation is completed, the "txt" format file is exported as the first data set.

5. The method for BIM detection of communication cabinets and internal equipment based on multimodal information according to claim 4 is characterized in that: Output the location information of the equipment inside the cabinet, including: Improve the upper left corner coordinates and lower right corner coordinates of the bounding box of the original output to the center point coordinates; in: x1 and y1 are the coordinates of the upper left corner of the bounding box; x2 and y2 are the coordinates of the lower right corner of the bounding box; x c and c is the coordinate of the center point of the bounding box; x c Compare the position relationship of the equipment in the horizontal direction, y c Compare the positional relationship of the devices in the vertical direction to form the position distribution information of the devices inside the cabinet.

6. The method for BIM detection of communication cabinets and internal equipment based on multimodal information according to claim 5 is characterized in that: Collect a large number of images of equipment labels inside various cabinets, and obtain the second data set after annotation, including: The equipment label images of various equipment inside the cabinets are collected by the camera, and the LabelImg image annotation tool is used to annotate them. After the annotation is completed, the "txt" format file is exported as the second data set.

7. The method for BIM detection of communication cabinets and internal equipment based on multimodal information according to claim 6 is characterized in that: Identify the text information on the equipment label, including: Configure the deep learning framework PaddlePaddl version to 2.1.2 or above, select ch_PP-OCRv4_rec as the text recognition model to recognize the text information on the equipment sign, and further obtain the equipment information.

8. The method for BIM detection of communication cabinets and internal equipment based on multimodal information according to claim 7 is characterized in that: Further obtain device information, including: Cut content based on the common characters on the equipment label; Extract the device model obtained after cutting; Query the device model in the device information database to obtain device information.

9. The method for BIM detection of communication cabinets and internal equipment based on multimodal information according to claim 8 is characterized in that: Collect the cabinet size data, the location information of the equipment inside the cabinet, and the text information of the equipment label, and compare them with the design information in the BIM model, including: Combine the queried equipment information with the collected cabinet size data, cabinet internal equipment location information, and equipment label text information, and compare them with the attribute information in the BIM model; The comparison results are written into the document through the python-docx library to generate a detection report.

10. The BIM detection system for communication cabinets and internal equipment based on multimodal information is characterized by: The system is used to implement the method according to claim 1, comprising: The temperature compensation module is used to establish a TOF light pulse-temperature propagation speed compensation model, improve the measurement algorithm of the TOF sensor, and thus compensate for the error of the results obtained by the TOF sensor at different temperatures; A cabinet dimension data acquisition module is used to acquire the cabinet dimension data; The first data set acquisition module is used to collect a large number of images of various types of internal equipment in the cabinets, and obtain the first data set after annotation; The first model training module is used to perform YOLOv10 model training using a large number of images of equipment inside the cabinet and a first data set, and output position information of equipment inside the cabinet; The second data set acquisition module is used to collect a large number of images of equipment labels of various types of equipment inside the cabinets, and obtain the second data set after annotation; The second model training module is used to train the YOLOv10 model using the massive images of the equipment sign and the second data set, select the equipment sign location, and recognize the equipment sign text information; The collection detection module is used to collect the cabinet size data, the location information of the equipment inside the cabinet, and the text information of the equipment label, and compare them with the design information in the BIM model to complete the detection.

Images