A ceiling disease early warning method and image acquisition device

By acquiring and analyzing the original and depth images of the subway station ceiling and automatically detecting them using the recognition and segmentation model, the warning problems of looseness and drop of the ceiling are solved, and high-precision disease identification and size measurement are achieved, reducing the risk of high-altitude operations.

CN117953192BActive Publication Date: 2024-12-03BEIJING DITIE ARCHITECTURE INSTALL ENG CO
View PDF 10 Cites 0 Cited by

Patent Information

Application Number
CN202410033498.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-01-09
Publication Date
2024-12-03
Estimated Expiration
2044-01-09

AI Technical Summary

Technical Problem

After long-term use, the ceiling of the subway station loosens and falls due to environmental vibration and piston wind. The existing detection methods have subjective impacts and high-altitude operation risks, making it difficult to effectively warn and detect.

Method used

A ceiling disease warning method is adopted to obtain the original image and depth image of the target station, and use the recognition model and segmentation model to identify the location and size of the disease, combined with the depth image to conduct early warning, to realize automated detection and reduce manual intervention and high-altitude risks.

Benefits of technology

It realizes high-precision detection of subway ceiling working conditions, can identify diseases such as deformation, dislocation, cracking, etc., and the calculation accuracy of the disease size reaches the millimeter level, solving the secondary damage and subjective impact problems of traditional detection methods, and reducing the risk of high-altitude operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117953192B_ABST
    Figure CN117953192B_ABST
Patent Text Reader

Abstract

The embodiments of the present disclosure disclose a method for early warning of ceiling defects and an image acquisition device. A specific implementation of the method includes: obtaining the original image and depth image of the ceiling of the target station; inputting the original image into the recognition model to obtain the defect location of the ceiling of the target station; inputting the defect location and the original image into the segmentation model to obtain the defect size of the ceiling of the target station; determining the defect early warning of the ceiling of the target station according to the defect location, defect size and depth image. This implementation realizes the detection of the working condition of the subway ceiling, and the types of defects that can be identified include surface defects such as deformation, dislocation, cracking, sagging, and tilting. The calculation accuracy of the defect size can reach the millimeter level, which can solve the problems existing in traditional detection methods such as secondary damage to the ceiling caused by contact detection, subjective influence on the accuracy of manual detection, and the risk of high-altitude operations.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a method, device, electronic device, computer-readable medium, and image acquisition device for early warning of suspended ceiling diseases. Background Art

[0002] As subway stations age, affected by environmental vibrations and piston wind generated during the operation of subway vehicles, station ceilings may become loose or fall off, affecting the personal safety of passengers. At present, there have been many cases in China where passengers were injured due to the detachment of subway station ceilings. This problem has become an urgent issue for subway operators to solve.

[0003] At present, daily inspection of suspended ceilings is mainly done by visual inspection, which can reveal the locations of suspended ceiling defects. At these locations, the suspended ceilings are prone to loosening and falling off, and further manual touch inspection is required using high ladders, scaffolding, etc. The existing inspection methods have problems such as the inspection results being greatly affected by subjective factors and the existence of risks in high-altitude operations. Summary of the invention

[0004] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0005] Some embodiments of the present disclosure provide a ceiling disease early warning method, device, electronic device and computer-readable medium to solve the technical problems mentioned in the above background technology section.

[0006] In a first aspect, some embodiments of the present disclosure provide a method for warning of ceiling defects, the method comprising: acquiring an original image and a depth image of a target station ceiling; inputting the original image into a recognition model to obtain a defect location of the target station ceiling; inputting the defect location and the original image into a segmentation model to obtain a defect size of the target station ceiling; and determining a defect warning of the target station ceiling based on the defect location, the defect size and the depth image.

[0007] In a second aspect, some embodiments of the present disclosure provide a device for warning of ceiling defects, the device comprising: an acquisition unit, configured to acquire an original image and a depth image of a target station ceiling; a recognition unit, configured to input the original image into a recognition model to obtain a defect position of the target station ceiling; a segmentation unit, configured to input the defect position and the original image into a segmentation model to obtain a defect size of the target station ceiling; and a determination unit, configured to determine a defect warning of the target station ceiling according to the defect position, the defect size and the depth image.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, which network device includes: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation method in the first aspect.

[0009] In a fourth aspect, an embodiment of the present application provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0010] In the fifth aspect, the present application provides an image acquisition device, including: a mobile base, a lifting slide rail, a power supply and control device, and a measuring device. The mobile base is equipped with triangular wheels and a three-axis shock absorber for realizing push-pull movement of the equipment; the lifting slide rail is used to adjust the height of the measuring device from the surface of the ceiling of the target station; the power supply and control device is used to supply power to the measuring device and control the lifting height of the measuring device; the measuring device is used to acquire the original image and depth image of the ceiling of the target station.

[0011] One of the above-mentioned embodiments of the present disclosure has the following beneficial effects: it realizes the detection of the working condition of subway ceilings, and can identify the types of defects including surface defects such as deformation, dislocation, cracking, sagging, and tilting. The calculation accuracy of the defect size can reach the millimeter level, which can solve the problems existing in traditional detection methods such as contact detection causing secondary damage to the ceiling, manual detection accuracy being affected by subjective factors, and the existence of risks in high-altitude operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0013] Figure 1is a schematic diagram of an application scenario of a suspended ceiling disease early warning method according to some embodiments of the present disclosure;

[0014] Figure 2 is a flow chart of some embodiments of the suspended ceiling disease early warning method according to the present disclosure;

[0015] Figure 3 It is a structural schematic diagram of some embodiments of the suspended ceiling disease early warning device according to the present disclosure;

[0016] Figure 4 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0018] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0019] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0020] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0022] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0023] Figure 1 It is a schematic diagram of an application scenario of the suspended ceiling disease early warning method according to some embodiments of the present disclosure.

[0024] like Figure 1As shown, the server 101 can obtain the original image 102 and the depth image 103 of the target station ceiling, input the original image 102 into the recognition model, and obtain the defect position 104 of the target station ceiling; input the defect position 104 and the original image 102 into the segmentation model to obtain the defect size 105 of the target station ceiling; determine the defect warning 106 of the target station ceiling according to the defect position 104, the defect size 105 and the depth image 103.

[0025] It is understandable that the ceiling disease early warning method can be executed by a terminal device, or it can also be executed by a server 101. The execution subject of the above method can also include a device formed by integrating the above terminal device and the above server 101 through a network, or it can also be executed by various software programs. Among them, the terminal device can be various electronic devices with information processing capabilities, including but not limited to smart phones, tablet computers, e-book readers, laptop portable computers and desktop computers, etc. The execution subject can also be embodied as a server 101, software, etc. When the execution subject is software, it can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules for providing distributed services, for example, or it can be implemented as a single software or software module. No specific limitation is made here.

[0026] It should be understood that Figure 1 The number of servers in the example is only for illustration purposes. Any number of servers may be provided as required.

[0027] Continue to refer Figure 2 , shows a process 200 of some embodiments of the suspended ceiling disease early warning method according to the present disclosure. The suspended ceiling disease early warning method comprises the following steps:

[0028] Step 201, obtaining an original image and a depth image of a target station ceiling.

[0029] In some embodiments, the execution subject of the ceiling disease early warning method (for example Figure 1 The server shown in the figure) can obtain the original image and depth image of the ceiling of the target station by connecting with the above-mentioned image acquisition device. It should be noted that the above-mentioned wireless connection method may include but is not limited to 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0030] Here, the original image is usually a two-dimensional color image, and the depth image is usually a three-dimensional image or a three-dimensional model. The target station ceiling is usually a station ceiling that the user needs to determine whether it has defects.

[0031] Step 202: input the original image into a recognition model to obtain the defect location of the ceiling of the target station.

[0032] In some embodiments, the above execution entities (e.g. Figure 1 The server shown in the figure) can input the above original image into the recognition model to obtain the defect location of the ceiling of the above target station.

[0033] Here, the above recognition model is usually used to characterize the correspondence between the original image and the disease location. As an example, the above recognition model can be a correspondence table obtained by researchers based on a large amount of data. As another example, the above recognition model can also be an artificial intelligence network with target recognition capabilities, such as DETR (DEtection Transformer, target detection network) and other models.

[0034] In some optional implementations of some embodiments, the above-mentioned recognition model is obtained by training the above-mentioned initial model using the YOLOv5 model as the initial model, the sample original image as input, the sample disease position corresponding to the sample original image as the expected output, and the pre-trained YOLOv8 model as the teacher model using the knowledge distillation algorithm.

[0035] In some optional implementations of some embodiments, the above-mentioned recognition model includes 3 1×1 convolutional layers, the number of channels of the above-mentioned convolutional layers is 512, and the extracted features are fused into the feature map by using a feature fusion method.

[0036] In some optional implementations of some embodiments, the above-mentioned recognition model also includes a small target detection layer, and the above-mentioned small target detection layer is used to splice and detect the shallow feature map and the deep feature map of the above-mentioned original image.

[0037] Specifically, adding a small target detection layer to YOLOv8 to detect after splicing the shallow feature map with the deep feature map can make the network pay more attention to the detection of small targets and improve the detection effect. However, after adding the detection layer, the problem is that the amount of calculation increases, resulting in a decrease in the reasoning detection speed. This method requires instant detection results, so it is necessary to increase the detection efficiency after adding small target detection. According to the analysis of the yolov8 network framework, this method optimizes the network structure, loss function and training data while improving the detection rate:

[0038] 1. In the network structure, a deep and narrow network structure is selected, and the impact of different feature fusion methods is studied:

[0039] In order to balance the effect and speed of the algorithm, a deep and narrow network structure is adopted. The number of convolution channels is reduced from 1024 in the Yolo algorithm to 512; in order to achieve a wider width, three 1×1 convolution layers are added at the end. The feature fusion method is adopted to fuse the features extracted from the first few layers into the feature maps of the later layers. In the fusion process, the stacking method is adopted, that is, the large-size feature map is resized first and then fused with the small-size feature map;

[0040] 2. In the loss function design, the distillation loss function and FM-NMS method are improved to adapt to the one-stage algorithm of yolo:

[0041] In order to optimize the loss function and improve detection efficiency, the knowledge distillation method is used to reduce the number of parameters and computational costs. Knowledge distillation is a training method for target detection models, which aims to identify and locate specific targets in images or videos. As a transfer learning technique, knowledge distillation uses a model that has been trained on a large dataset (called a teacher model) to guide the training of another model (called a student model). Knowledge distillation is used to improve the efficiency of the student model.

[0042] Specifically, we use YOLOv8 knowledge distillation to use a pre-trained YOLOv8 model as the teacher model, and then use a smaller-scale YOLOv5 model as the student model. The student model learns how to better predict the location and category of the target by observing the output of the teacher model. This knowledge distillation process can improve the accuracy and generalization ability of the student model, while reducing the number of parameters and computational costs, and improving the performance and efficiency of the target detection model.

[0043] 3. Both labeled and unlabeled data were used during training:

[0044] Training data optimization By using the distillation algorithm, it is very convenient to use labeled data and unlabeled data during training. If there is labeled data, the full distillation loss function is used. If there is no labeled data, only the distillation loss part of the distillation loss function is used.

[0045] Step 203: input the above-mentioned defect location and the above-mentioned original image into a segmentation model to obtain the defect size of the above-mentioned target station ceiling.

[0046] In some embodiments, the execution entity may input the defect location and the original image into a segmentation model to obtain the defect size of the target station ceiling.

[0047] Here, the above segmentation model is usually used to characterize the location of the disease and the correspondence between the original image and the size of the disease.

[0048] In some optional implementations of some embodiments, the above-mentioned segmentation model is obtained by training the above-mentioned initial model with the detection2 model as the initial model, the sample disease location and the sample original image as input, and the sample disease size corresponding to the sample disease location and the sample original image as the expected output.

[0049] Generally speaking, physical segmentation can meet the detection efficiency and accuracy of large and small targets by modifying the yolov8 network. However, this method also needs to calculate the defects of the identified ceiling surface conditions. Therefore, this method uses detection2 to segment the detected targets and describe the contours of the detected targets (diseases). Segmentation is a pixel-level description of the image, which gives each pixel category meaning. It is suitable for scenes with high understanding requirements, such as segmenting the defects of the ceiling surface conditions, and can accurately and effectively calculate the defect size of the ceiling surface conditions.

[0050] Step 204: determining a defect warning for the ceiling of the target station according to the defect location, the defect size and the depth image.

[0051] In some embodiments, the execution entity may determine a defect warning for the ceiling of the target station according to the defect location, the defect size and the depth image.

[0052] Specifically, when the above-mentioned execution subject detects that the physical quantity obtained from the surface defect size and defect location information of the ceiling such as dislocation, deformation, flatness, etc. is greater than or equal to the preset threshold control value, a threshold control value warning is issued; if the converted physical quantity is greater than or equal to the preset first-level threshold warning value and less than the preset threshold control value, a first-level threshold warning value warning is issued; if the converted physical quantity is greater than or equal to the preset second-level threshold warning value and less than the preset first-level threshold warning value, a second-level threshold warning value warning is issued.

[0053] In addition, when the warning value or alarm threshold is reached, the above-mentioned execution entity can issue an alarm, and the personnel will subsequently verify the detection results. If the personnel's judgment is the same as the judgment of the above-mentioned execution entity, the alarm information, picture, alarm type, location area, time and other information at this location will be saved; if they are different, the alarm will be canceled.

[0054] As an example, the threshold warning is as follows:

[0055] 1) The setting of the threshold warning value and threshold control value of the suspended ceiling working condition should be based on the design documents, relevant specifications, combined with the material properties of the suspended ceiling and the rules of historical data, and can be reasonably revised based on sufficient demonstration according to actual use;

[0056] 2) The threshold warning value warning function is divided into two levels of management, namely the second threshold warning value warning (yellow) and the first threshold warning value warning (red). The second threshold warning value can be set at 60% of the threshold control value, and the first threshold warning value can be set at 80% of the threshold control value;

[0057] 3) The setting of threshold warning value shall comply with the requirements of suspended ceiling safety control indicators.

[0058] As for threshold warning, if the warning level is the second-level threshold warning value warning, the system technical person in charge of the project department will be notified to verify the warning according to the warning cycle, warning object, and warning medium; if the warning level is the first-level threshold warning value warning, the dispatch will be notified according to the warning cycle, warning object, and warning medium, and the dispatch room will notify the system technical person in charge of the project department by telephone to verify the warning, organize the work area inspection personnel to conduct on-site verification, determine whether the warning information is true, and report the verification situation to the dispatch room within 24 hours.

[0059] In addition, the above-mentioned execution entity can usually store the ID of the collector, two-dimensional image data, three-dimensional depth image data, collection time, location, disease size, disease location and other parameters as data records in the database identified by a unique serial number.

[0060] Image data can be stored in BLOB format, including binary objects of 2D images and 3D depth images, while the timestamp indicates the time of data acquisition. Disease size and disease location parameters are recorded in floating point form and can contain multiple specific parameter information. This data design enables the collected information to be efficiently stored, managed and retrieved in the database, providing valuable resources for subsequent analysis and application.

[0061] The use of the database ensures data consistency, security and easy accessibility, thus providing support for the entire process. Based on the design of this process and the realization of functional effects, it can effectively optimize the efficiency and accuracy of disease detection and analysis, enter the formulated work plan into the database, and realize the review and tracking of the inspection plan; receive feedback on the execution of the plan, and supervise and review it.

[0062] One of the above-mentioned embodiments of the present disclosure has the following beneficial effects: it realizes the detection of the working condition of subway ceilings, and can identify the types of defects including surface defects such as deformation, dislocation, cracking, sagging, and tilting. The calculation accuracy of the defect size can reach the millimeter level, which can solve the problems existing in traditional detection methods such as contact detection causing secondary damage to the ceiling, manual detection accuracy being affected by subjective factors, and the existence of risks in high-altitude operations.

[0063] On the other hand, the present application provides an image acquisition device in an embodiment, including: a mobile base, a lifting rail, a power supply and control device, and a measuring device. The mobile base is equipped with triangular wheels and three-axis shock absorbers for realizing push-pull movement of the equipment; the lifting rail is used to adjust the height of the measuring device from the surface of the ceiling of the target station; the power supply and control device is used to supply power to the measuring device and control the lifting height of the measuring device; the measuring device is used to acquire the original image and depth image of the ceiling of the target station.

[0064] In some optional implementations of some embodiments, the measuring device includes: a structured light emitter, a structured light camera, an RGB camera and a laser dot matrix, the structured light emitter is used to emit line structured light; the structured light camera is used to collect the depth image of the ceiling of the target station; the RGB camera is used to collect the original image of the ceiling of the target station; the laser dot matrix is ​​used to determine the distance between the measuring device and the ceiling of the target station, and to perform height calibration on the plane of the measuring device.

[0065] First, the laser rangefinder senses the distance between the ceiling to be tested and the equipment to determine the effective field of view. At the same time, the initialization parameter information and detection time series are obtained based on the type and material of the ceiling to complete the minimum movement distance determination. After the above process is completed, the structured light camera and RGB camera will collect the original data of the ceiling to be tested.

[0066] In addition, the above device may also include a display terminal for displaying the collected original image and depth image.

[0067] The above-mentioned image acquisition equipment can use laser dot matrix, structured light camera and RGB camera to obtain the relative height between the ceiling surface and the ground, original color image and depth image of target stations of different heights and types, and transmit them to the executor of the ceiling disease early warning method.

[0068] Specifically, the mobile base is equipped with triangular wheels to ensure that the movement of the equipment is not affected by steps, and a three-axis shock absorber is installed to reduce the impact of jitter during the measurement process on the measurement effect; the lifting slide rail controls the lifting height to ensure that the detection range is within the effective detection range of the structured light camera.

[0069] In the actual ceiling acquisition process, since the ground of the platform, station hall and transfer passage cannot be guaranteed to be consistently level, in order to ensure the measurement accuracy and eliminate the inconsistency between the depth calculation results of the ceiling surface flatness, staggered joints, etc. caused by uneven ground and the actual manual measurement results (manual detection is between adjacent ceiling panels, and structured light detection is relative to the ground + adjacent ceiling panels), this equipment uses a dot matrix high-precision laser ranging sensor to adjust the error of the depth calculation of the ceiling depth map to ensure that the detection calculation results are consistent with the actual detection results; use every three sensors to detect a plane, and use the difference between the ranging surface and the depth map of the structured light camera to perform coordinate fitting to eliminate the deviation of the detection results from the actual due to uneven ground.

[0070] As an example, the above-mentioned image acquisition device can control the lifting slide rail and read the value of the high-precision laser rangefinder. The lifting slide rail stops when the distance between the device and the ceiling surface reaches the specified range; the RGB camera collects the color image of the ceiling surface to be measured, and the structured light camera collects the depth image at the same position. Combined with the moving distance of the device, it is automatically collected every n seconds.

[0071] Afterwards, the above-mentioned image acquisition device can send the acquired depth image and color image to the execution subject of the suspended ceiling disease early warning method for identification and determination of the disease size and location. When the above-mentioned disease size and disease location reach the warning value or alarm threshold, the execution subject of the suspended ceiling disease early warning method will issue an alarm, and the personnel will verify the detection result. If the personnel judgment is the same as the system judgment, the alarm information, picture, alarm type, location area, time and other information at this location will be saved; if they are not the same, the alarm will be canceled. The structured light is projected onto the surface of the subway station ceiling to be detected, and it is photographed to obtain the corresponding original image of the subway station ceiling.

[0072] An image acquisition device provided by an embodiment of the present invention can realize acquisition while moving without the need for repeated manual operations. It adopts high-precision laser dot matrix, automatic lifting control, and structured light camera. It can be raised and lowered to a position 1.5 to 2 meters away from the ceiling in different subway stations. The structured light camera acquires the surface condition of the ceiling. The image acquisition device combines the laser dot matrix to supplement the depth information of the acquired image and realizes precision measurement.

[0073] Further references Figure 3 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a suspended ceiling disease early warning device. These device embodiments are similar to Figure 2 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0074] like Figure 3As shown, some embodiments of the suspended ceiling defect warning device 300 include: an acquisition unit 301, an identification unit 302, a segmentation unit 303 and a determination unit 304. The acquisition unit 301 is configured to acquire the original image and the depth image of the suspended ceiling of the target station; the identification unit 302 is configured to input the original image into the identification model to obtain the defect position of the suspended ceiling of the target station; the segmentation unit 303 is configured to input the defect position and the original image into the segmentation model to obtain the defect size of the suspended ceiling of the target station; the determination unit 304 is configured to determine the defect warning of the suspended ceiling of the target station according to the defect position, the defect size and the depth image.

[0075] It is understood that the units described in the device 300 are similar to those described in the reference Figure 2 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 300 and the units contained therein, and will not be described in detail here.

[0076] One of the above-mentioned embodiments of the present disclosure has the following beneficial effects: it realizes the detection of the working condition of subway ceilings, and can identify the types of defects including surface defects such as deformation, dislocation, cracking, sagging, and tilting. The calculation accuracy of the defect size can reach the millimeter level, which can solve the problems existing in traditional detection methods such as contact detection causing secondary damage to the ceiling, manual detection accuracy being affected by subjective factors, and the existence of risks in high-altitude operations.

[0077] Reference below Figure 4 , which shows an electronic device (eg, Figure 1 A schematic diagram of the structure of the server 400. Figure 4 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0078] like Figure 4 As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0079] Typically, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 4 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0080] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 409, or installed from the storage device 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0081] It should be noted that the computer-readable medium described above in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0082] In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.

[0083] The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0084] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0085] The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains the original image and depth image of the target station ceiling; inputs the original image into the recognition model to obtain the defect location of the target station ceiling; inputs the defect location and the original image into the segmentation model to obtain the defect size of the target station ceiling; determines the defect warning of the target station ceiling according to the defect location, the defect size and the depth image.

[0086] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0087] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0088] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor includes an acquisition unit, an identification unit, a segmentation unit, and a determination unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the acquisition unit may also be described as a "unit for acquiring an original image and a depth image of the ceiling of a target station".

[0089] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0090] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A method for early warning of suspended ceiling diseases, include: Obtain the original image and depth image of the ceiling of the target station; Inputting the original image into a recognition model to obtain the defect location of the ceiling of the target station; Input the defect location and the original image into a segmentation model to obtain the defect size of the target station ceiling, wherein the segmentation model is obtained by training the initial model with the detection2 model as the initial model, the sample defect location and the sample original image as input, and the sample defect size corresponding to the sample defect location and the sample original image as the expected output; Determine a defect warning of the target station ceiling according to the defect location, the defect size and the depth image; The recognition model is obtained by training the initial model using a YOLOv5 model as an initial model, a sample original image as an input, a sample disease position corresponding to the sample original image as an expected output, a pre-trained YOLOv8 model as a teacher model, and a knowledge distillation algorithm; The recognition model includes three 1×1 convolutional layers, the number of channels of the convolutional layers is 512, and the extracted features are fused into the feature map by using feature fusion; The recognition model also includes a small target detection layer, which is used to splice and detect the shallow feature map and deep feature map of the original image.

2. An image acquisition device for use in the method as claimed in claim 1, include: Mobile base, lifting slide rails, power supply and control device and measuring device, The mobile base is equipped with triangular wheels and three-axis shock absorbers for realizing push-pull movement of the equipment; The lifting slide rail is used to adjust the height of the measuring device from the ceiling surface of the target station; The power supply and control device are used to supply power to the measuring device and control the lifting height of the measuring device; The measuring device is used to collect the original image and depth image of the ceiling of the target station; The measuring device comprises: a structured light emitter, a structured light camera, an RGB camera and a laser dot matrix. The structured light emitter is used to emit line structured light; The structured light camera is used to collect a depth image of the ceiling of the target station; The RGB camera is used to collect the original image of the ceiling of the target station; The laser dot matrix is ​​used to determine the distance between the measuring device and the ceiling of the target station, and to perform height calibration on the plane of the measuring device.

3. A ceiling disease early warning device, include: An acquisition unit is configured to acquire an original image and a depth image of a ceiling of a target station; A recognition unit is configured to input the original image into a recognition model to obtain the defect location of the ceiling of the target station; A segmentation unit is configured to input the defect position and the original image into a segmentation model to obtain the defect size of the target station ceiling, wherein the segmentation model is obtained by training the initial model with the detection2 model as an initial model, the sample defect position and the sample original image as input, and the sample defect size corresponding to the sample defect position and the sample original image as the expected output; A determination unit, configured to determine a defect warning of the target station ceiling according to the defect location, the defect size and the depth image; The recognition model is obtained by training the initial model using a YOLOv5 model as an initial model, a sample original image as an input, a sample disease position corresponding to the sample original image as an expected output, a pre-trained YOLOv8 model as a teacher model, and a knowledge distillation algorithm; The recognition model includes three 1×1 convolutional layers, the number of channels of the convolutional layers is 512, and the extracted features are fused into the feature map by using feature fusion; The recognition model also includes a small target detection layer, which is used to splice and detect the shallow feature map and deep feature map of the original image.

4. An electronic device, include: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as claimed in claim 1.

5. A computer readable medium having a computer program stored thereon, in, When the program is executed by a processor, the method according to claim 1 is implemented.

Citation Information

Patent Citations

  • Lightweight small target detection method in combination with attention mechanism

    CN113065558A

  • Detection method, detection system, equipment and storage medium

    CN113627257A

  • Melon and fruit vegetable size measurement method based on depth camera and key points

    CN113932712A

  • Fish resource rapid identification method and system based on deep learning

    CN115100688A

  • Intention object prediction method based on multi-path network structure and multi-depth supervision mechanism

    CN115171050A