Bridge structure disease determination method and device, unmanned aerial vehicle, medium and product

Abstract

The invention discloses a bridge structure disease determination method and device, an unmanned aerial vehicle, a medium and a product. The method comprises the steps that in the process of controlling an unmanned aerial vehicle to fly along a bridge, the unmanned aerial vehicle is controlled to conduct structure disease recognition on the surface of the bridge; shooting the recognized bridge structure disease area to obtain at least one bridge structure disease image; and carrying out structure disease type identification on each bridge structure disease image to obtain a structure disease type corresponding to each bridge structure disease image. Through the technical scheme of the invention, the problems of high operation difficulty, inaccurate image recognition and screening and the like when a conventional unmanned aerial vehicle is configured for inspection operation can be solved, unnecessary power consumption in the inspection process of the unmanned aerial vehicle is reduced, invalid shooting time is shortened to increase the endurance of the unmanned aerial vehicle, so that the detection of an extra-large bridge is completed more smoothly, and the inspection efficiency is improved. Meanwhile, the threshold and time of manual auditing are reduced through an effective disease screening mechanism, manpower and material resources are saved, and the detection efficiency of bridge structure diseases is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of image processing and image classification, and in particular to a method, device, drone, medium and product for determining bridge structure defects. Background Art

[0002] In traditional bridge inspection work, the main method adopted is a combination of regular inspection and daily patrol inspection. Traditional inspection methods have many disadvantages such as time-consuming and labor-intensive, highly subjective, and large blind spots. For bridges under some special geographical conditions, there are great safety hazards, and it is difficult to ensure the comprehensiveness and accuracy of the inspection.

[0003] With the rise of drone technology, drone technology has gradually been applied to bridge inspection. At present, drone inspection is mostly carried out by pilots manually controlling the drone inspection method. The drones are photographed and inspected by manual control, and the key image information of the inspection parts is collected. After that, the data information is comprehensively analyzed and processed to efficiently complete the detection and identification of bridge defects. However, the drone inspection method based on manual control requires pilots to have high flight experience. At the same time, the pilots need to identify structural defects in time to improve the shooting efficiency during the process of controlling the drone shooting. They need to have a certain reserve of knowledge related to the bridge structure, which makes this method have a high threshold and is not universal. On the other hand, due to the complex and changeable spatial environment of the bridge structure, the pilots are easily disturbed by trees, water pipes and cables, high-voltage power grids, etc. around the bridge when controlling the drone close to the bridge structure, which greatly limits the efficiency and entry threshold of drone bridge inspection. However, for some drone automatic inspections currently available on the market, the automatic disease identification technology for image data acquisition images is not yet mature, and it is impossible to complete the real-time dynamic analysis of video image data, and the identification accuracy needs to be improved. During the automatic inspection process of drones, especially for the inspection of super-large bridges across the sea and rivers, the battery life of the drone is an issue that cannot be ignored. Long-path inspections consume power quickly. At the same time, due to its inability to quickly screen and identify key structural defects, a large amount of unscreened image information is transmitted to the terminal in real time through the drone, which greatly increases the time and difficulty of manual review and greatly reduces its detection efficiency.

[0004] Therefore, in the field of drone automatic inspection and disease identification, there is an urgent need for an image recognition method that can quickly identify key diseases. Summary of the invention

[0005] The embodiments of the present invention provide a method, device, drone, medium and product for determining bridge structure defects, so as to reduce invalid shooting time to increase its endurance, and reduce the threshold and time of manual review through an effective defect screening mechanism.

[0006] According to one aspect of the present invention, a method for determining bridge structure defects is provided, comprising:

[0007] In the process of controlling the UAV to fly along the bridge, controlling the UAV to identify structural defects on the surface of the bridge;

[0008] Photographing the identified bridge structure defect area to obtain at least one bridge structure defect image;

[0009] Structural defect type identification is performed on each of the bridge structural defect images to obtain the structural defect type corresponding to each bridge structural defect image.

[0010] In some embodiments of the present invention, before controlling the drone to identify structural defects on the bridge surface, the method further includes:

[0011] Obtain a sample set of bridge structure disease images;

[0012] A dictionary learning function is constructed, and based on the dictionary learning function, dictionary learning is performed according to the bridge structure defect image sample set to obtain an overcomplete dictionary.

[0013] In some embodiments of the present invention, controlling the drone to identify structural defects on the bridge surface includes:

[0014] The drone is controlled to determine whether there is a bridge structure defect area on the bridge surface whose similarity with the bridge structure defect image samples corresponding to the overcomplete dictionary is greater than a preset threshold.

[0015] In some embodiments of the present invention, identifying the structural damage type of each bridge structure damage image includes:

[0016] Determine a dictionary corresponding to each bridge structure defect type based on the overcomplete dictionary;

[0017] An image set corresponding to each bridge structure disease type is determined according to each of the bridge structure disease images and a dictionary corresponding to each bridge structure disease type.

[0018] In some embodiments of the present invention, determining an image set corresponding to each bridge structure defect type according to each bridge structure defect image and a dictionary corresponding to each bridge structure defect type includes:

[0019] Classification is performed according to the reconstruction error size of each bridge structure defect image on the dictionary corresponding to each bridge structure defect type, so as to obtain an image set corresponding to each bridge structure defect type.

[0020] In some embodiments of the present invention, determining a dictionary corresponding to each bridge structure defect type based on the overcomplete dictionary includes:

[0021] A sparse coefficient expression in the dictionary learning function is constructed, and a dictionary corresponding to each bridge structure defect type is determined based on the overcomplete dictionary and the sparse coefficient expression.

[0022] According to another aspect of the present invention, a bridge structure defect determination device is provided, the device comprising:

[0023] A control module, used to control the UAV to identify structural defects on the bridge surface during the process of controlling the UAV to fly along the bridge;

[0024] A shooting module, used to shoot the identified bridge structure defect area to obtain at least one bridge structure defect image;

[0025] The identification module is used to identify the structural damage type of each bridge structure damage image to obtain the structural damage type corresponding to each bridge structure damage image.

[0026] According to another aspect of the present invention, a drone is provided, the drone comprising: a drone body, a camera, and a light conversion mirror; the drone body is connected to the camera, and the drone body is connected to the light conversion mirror;

[0027] The drone body includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the bridge structure defect determination method described in any embodiment of the present invention.

[0028] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the bridge structure defect determination method described in any embodiment of the present invention when executed.

[0029] According to another aspect of the present invention, an embodiment of the present invention further provides a computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, it implements the bridge structure defect determination method described in any embodiment of the present invention.

[0030] The embodiment of the present invention controls the drone to identify structural defects on the bridge surface during the process of controlling the drone to fly along the bridge, photographs the identified bridge structural defect area, obtains at least one bridge structural defect image, identifies the structural defect type of each bridge structural defect image, and obtains the structural defect type corresponding to each bridge structural defect image. The technical solution of the present invention can solve the problems of high operating difficulty and inaccurate image recognition screening when conventionally configured drones are used for inspection operations, reduce unnecessary power consumption during drone inspections, reduce invalid shooting time to increase its endurance, so as to more smoothly complete the inspection of super-large bridges, and at the same time reduce the threshold and time of manual review through an effective defect screening mechanism, save manpower and material resources, and improve the detection efficiency of bridge structural defects.

[0031] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0033] Figure 1 is a flow chart of a method for determining bridge structure defects in an embodiment of the present invention;

[0034] Figure 2 is a schematic structural diagram of a bridge structure defect determination device in an embodiment of the present invention;

[0035] Figure 3 It is a structural schematic diagram of an unmanned aerial vehicle for implementing the bridge structure defect determination method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and their meanings are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0039] Embodiment 1

[0040] Figure 1 is a flow chart of a method for determining a bridge structure defect in an embodiment of the present invention. This embodiment is applicable to the case where a UAV is used to determine a bridge structure defect. The method can be executed by a device for determining a bridge structure defect in an embodiment of the present invention. The device can be implemented in software and / or hardware, such as Figure 1 As shown, the method specifically comprises the following steps:

[0041] S101. In the process of controlling the drone to fly along the bridge, the drone is controlled to identify structural defects on the surface of the bridge.

[0042] In this embodiment, the structural defects on the bridge surface are mainly identified by drones. The structural defects on the bridge surface generally refer to damage and defects that affect the integrity and safety of the bridge structure. For example, the structural defects may be cracks, damage to the bridge deck pavement, expansion joint problems, loose or damaged connectors, etc.

[0043] In the implementation process, it is necessary to control the drone to fly along the bridge surface and identify structural defects on the bridge surface during the flight. Specifically, the height at which the drone flies from the bridge surface is not limited in this embodiment, and the user can set it comprehensively according to factors such as the bridge structure, the surrounding environment, and the performance of the drone.

[0044] S102: photograph the identified bridge structure defect area to obtain at least one bridge structure defect image.

[0045] The bridge structure defect area may be an area on the bridge surface where structural defects exist, and the bridge structure defect image may be an image containing the bridge structure defect area obtained after photographing the bridge structure defect area by a drone.

[0046] Specifically, when the drone is flying along the bridge, if it identifies a structural defect area on the bridge surface, it will take a photo of the area to obtain a bridge structure defect image. In actual operation, the drone can take photos while flying, and can take only one photo or multiple photos continuously, which can be set by the user. After the photo is taken, the photo of the bridge structure defect can be stored in the drone or uploaded to the cloud synchronously. The user can directly view the bridge structure defect image taken by the drone in the cloud.

[0047] S103, identifying the structural defect type of each bridge structural defect image to obtain the structural defect type corresponding to each bridge structural defect image.

[0048] In actual operation, there are many types of structural defects in bridges, and different types of structural defects may correspond to different structural defect images. After the drone obtains the structural defect image, it can identify the structural defect type of each bridge structural defect image and obtain the structural defect type corresponding to each bridge structural defect image.

[0049] The embodiment of the present invention controls the drone to identify structural defects on the bridge surface during the process of controlling the drone to fly along the bridge, photographs the identified bridge structural defect area, obtains at least one bridge structural defect image, identifies the structural defect type of each bridge structural defect image, and obtains the structural defect type corresponding to each bridge structural defect image. The technical solution of the present invention can solve the problems of high operating difficulty and inaccurate image recognition screening when conventionally configured drones are used for inspection operations, reduce unnecessary power consumption during drone inspections, reduce invalid shooting time to increase its endurance, so as to more smoothly complete the inspection of super-large bridges, and at the same time reduce the threshold and time of manual review through an effective defect screening mechanism, save manpower and material resources, and improve the detection efficiency of bridge structural defects.

[0050] Optionally, before controlling the drone to identify structural defects on the bridge surface, the following steps are also included:

[0051] Get a sample set of bridge structure defect images.

[0052] It should be noted that the bridge structure defect image sample set may be a sample set consisting of several known typical bridge structure defect images. In this embodiment, the bridge structure defect image sample set may be divided into training samples and test samples.

[0053] Specifically, a large number of known typical structural defect images of bridges are obtained to form a sample set of bridge structural defect images, and some of them are selected as training samples for dictionary learning. In this embodiment, the training sample set composed of the sample set of bridge structural defect images can be expressed as X = {x 1 ,x 2 ,…,x N}∈RM.

[0054] A dictionary learning function is constructed, and based on the dictionary learning function, dictionary learning is performed according to a sample set of bridge structure defect images to obtain an overcomplete dictionary.

[0055] It can be known that the dictionary learning function refers to a mathematical function or algorithm used to optimize the dictionary and sparse representation, and the overcomplete dictionary is a concept in dictionary learning, which refers to a dictionary containing more atoms (or basis vectors) than the data dimension. In signal processing and machine learning, the overcomplete dictionary is used to sparsely represent the signal, that is, to represent the signal with as few atoms as possible. In this embodiment, the overcomplete dictionary can be used to represent the structural disease image signal of the bridge surface.

[0056] Specifically, in this embodiment, an overcomplete dictionary D is first established using typical bridge structural disease image samples. Assuming that the training sample set is X = {x 1 ,x 2 ,…,x N}∈RM, the dictionary learning form is as follows:

[0057]

[0058] Where D = {d 1 ,d 2 ,…,d K}∈R M×N is the dictionary to be sought, i.e., the overcomplete dictionary in this embodiment, A={α 1 ,α 2 ,…,α N}∈R K×N is the sparse reconstruction coefficient under the sparse representation constraint, the sparse regularization term is used to constrain the sparsity of each type of data, and the parameter λ is used to adjust the constraint weight.

[0059] It should be noted that the dictionary obtained by calculating the above method can adaptively represent various signals, but cannot be directly used for classification tasks. That is, based on the current method, the drone can only determine whether there are structural defects on the bridge surface, but cannot directly determine the specific type of structural defects.

[0060] Through self-learning of typical structural disease image samples using dictionaries, drones can quickly identify and detect typical diseases in images. In the later stage, by improving the model and collecting more training data, the recognition rate of key diseases by drones can be further improved. In addition, for atypical structural diseases, transfer learning can be used to solve the learning problem of little or no labeled data in some target areas. The basic idea of ​​transfer learning is to use existing knowledge to train new models. This method relaxes the requirement that training data and test data follow the same distribution, and reduces the requirements of the classification model on the scale of training data.

[0061] Optionally, control the drone to identify structural defects on the bridge surface, including:

[0062] The drone is controlled to determine whether there is a bridge structure defect area on the bridge surface whose similarity with the bridge structure defect image samples corresponding to the overcomplete dictionary is greater than a preset threshold.

[0063] Among them, the preset threshold can be pre-set according to actual conditions, and is a threshold of the similarity between the structural damage area on the bridge surface and the bridge structure damage image samples included in the over-complete dictionary. This embodiment does not limit the specific value of the preset threshold, and the user can set it according to experience.

[0064] Specifically, in the process of controlling the drone to fly along the bridge, the drone is controlled to detect in real time whether there is a bridge structure defect area on the bridge surface whose similarity with the bridge structure defect image sample corresponding to the overcomplete dictionary is greater than a preset threshold. If so, it can be determined that there is a bridge structure defect on the bridge surface, and the drone takes a photo of the bridge structure defect area; if not, it can be determined that there is no bridge structure defect on the bridge surface, and the drone continues to fly along the bridge.

[0065] Dictionary learning is actually a self-learning method that adapts to different signals. The core of dictionary learning is to construct a bunch of atoms that can fully express the characteristics of the source signal, and sparsely represent the signal through the combination and arrangement of atoms. In order to establish a dictionary for classification tasks, a joint dictionary is adopted to learn a dictionary for each category of data, and then classification is achieved based on the reconstruction error size of the data to be classified on different dictionaries, which reduces the time complexity of the algorithm.

[0066] Optionally, the structural damage type is identified for each bridge structural damage image, including:

[0067] The dictionary corresponding to each bridge structure defect type is determined based on the overcomplete dictionary.

[0068] In this embodiment, after establishing an overcomplete dictionary, the drone can determine whether there is a structural defect on the bridge surface, but cannot directly determine the specific type of the structural defect. Therefore, it is necessary to learn a dictionary for each bridge structural defect category, and then implement image classification based on the reconstruction error size of each bridge structural defect image on different dictionaries to reduce the time complexity of the algorithm.

[0069] An image set corresponding to each bridge structure disease type is determined according to each bridge structure disease image and a dictionary corresponding to each bridge structure disease type.

[0070] It should be noted that the image set may be a bridge structure disease type image set obtained by classifying images of different bridge structure disease types.

[0071] Specifically, the image categories of all bridge structure defect images can be classified by calculating the reconstruction error size of each bridge structure defect image on the dictionary corresponding to different bridge structure defect types, and an image set corresponding to each bridge structure defect type can be obtained.

[0072] For the photos of key diseases that have been identified, data cleaning is performed to obtain the target renderings. When facing image classification tasks, the accuracy of traditional algorithms is easily affected by noise data. For this reason, this embodiment adopts the transfer learning theory mentioned in the second step to initially divide the recognition sample data, and then migrate a batch of samples from the source domain to the center of each category, and re-cluster the newly formed sample set, and then form an over-complete dictionary for each category through a machine algorithm, and then update the sample category labels according to the principle of minimizing reconstruction errors. In the process of continuous data migration, only some samples close to the center point are selected to re-establish a more representative sample set, and because the influence of noise data is eliminated, the newly established samples re-clustered using the corresponding center representative points are more accurate than the initial center representative points.

[0073] Optionally, determining an image set corresponding to each bridge structure defect type according to each bridge structure defect image and a dictionary corresponding to each bridge structure defect type includes:

[0074] Classify each bridge structure defect image according to the size of the reconstruction error on the dictionary corresponding to each bridge structure defect type, and obtain an image set corresponding to each bridge structure defect type.

[0075] It is known that reconstruction error is an important concept in signal processing, image processing, machine learning, and data compression. It measures the difference between the original signal or data and the signal or data reconstructed by a certain model or algorithm. In the context of dictionary learning and sparse representation, reconstruction error is often used to evaluate the quality of a sparse representation, that is, how to accurately represent a signal with the least information.

[0076] Specifically, the reconstruction error of each bridge structure defect image in the dictionary corresponding to different bridge structure defect types can be calculated, and the category label can be updated according to the principle of minimizing the reconstruction error to achieve image category classification of all bridge structure defect images, and obtain an image set corresponding to each bridge structure defect type. This embodiment does not specifically limit the process of calculating the reconstruction error.

[0077] Optionally, a dictionary corresponding to each bridge structure defect type is determined based on an overcomplete dictionary, including:

[0078] A sparse coefficient expression in the dictionary learning function is constructed, and the dictionary corresponding to each bridge structure defect type is determined based on the overcomplete dictionary and the sparse coefficient expression.

[0079] In this embodiment, the sparse coefficient expression of the typical bridge structural defect image test sample under the overcomplete dictionary D can be:

[0080]

[0081] Where D = {d 1 ,d 2 ,…,d K}∈R M×N is an over-complete dictionary, the parameter λ is used to adjust the constraint weight, and the parameter y represents the image in the image set corresponding to each bridge structure disease type.

[0082] Specifically, a dictionary is learned for each bridge structure defect type, and then classification is achieved according to the reconstruction error size of each bridge structure defect image on different dictionaries, which reduces the time complexity of the algorithm.

[0083] The technical solution of the embodiment of the present invention, based on sparse learning, solves the problems of difficult operation and inaccurate image recognition and screening when conventionally configured drones are used for inspection to the greatest extent, and reduces unnecessary power consumption of drones during bridge inspections, so as to more smoothly complete the inspection of super-large bridges. The technical solution of the embodiment of the present invention, after constructing an overcomplete dictionary, updates the category labels according to the principle of minimizing reconstruction errors, which greatly reduces the redundant and disordered memory of the data collected by the drones; at the same time, the key disease data set is classified according to the principle of minimizing reconstruction errors using the final overcomplete dictionary, which can effectively reduce the repetitive operations of the inspection technicians and improve the inspection efficiency. The effective capture of typical diseases also provides more intuitive data support for the subsequent bridge repair and maintenance of the management and maintenance units, and can provide timely warnings of key diseases to the management and maintenance units or inspection technicians.

[0084] Embodiment 2

[0085] Figure 2 is a schematic diagram of a bridge structure defect determination device in an embodiment of the present invention. This embodiment can be applied to the case where a drone is used to determine bridge structure defects. The device can be implemented in software and / or hardware. The device can be integrated into any device that provides the function of determining bridge structure defects, such as Figure 2 As shown, the bridge structure defect determination device specifically includes: a control module 201, a shooting module 202 and a recognition module 203.

[0086] The control module 201 is used to control the drone to identify structural defects on the bridge surface during the process of controlling the drone to fly along the bridge;

[0087] A photographing module 202 is used to photograph the identified bridge structure defect area to obtain at least one bridge structure defect image;

[0088] The identification module 203 is used to identify the structural damage type of each bridge structure damage image to obtain the structural damage type corresponding to each bridge structure damage image.

[0089] Optionally, the device further comprises:

[0090] An acquisition module is used to acquire a sample set of bridge structure disease images;

[0091] The construction and learning module is used to construct a dictionary learning function, and based on the dictionary learning function, perform dictionary learning according to the bridge structure defect image sample set to obtain an overcomplete dictionary.

[0092] Optionally, the control module 201 is specifically used for:

[0093] The drone is controlled to determine whether there is a bridge structure defect area on the bridge surface whose similarity with the bridge structure defect image samples corresponding to the overcomplete dictionary is greater than a preset threshold.

[0094] Optionally, the identification module 203 includes:

[0095] A first determining unit, configured to determine a dictionary corresponding to each bridge structure defect type based on the overcomplete dictionary;

[0096] The second determining unit is used to determine an image set corresponding to each bridge structure disease type according to each of the bridge structure disease images and a dictionary corresponding to each bridge structure disease type.

[0097] Optionally, the second determining unit is specifically configured to:

[0098] Classification is performed according to the reconstruction error size of each bridge structure defect image on the dictionary corresponding to each bridge structure defect type, so as to obtain an image set corresponding to each bridge structure defect type.

[0099] Optionally, the first determining unit is specifically configured to:

[0100] A sparse coefficient expression in the dictionary learning function is constructed, and a dictionary corresponding to each bridge structure defect type is determined based on the overcomplete dictionary and the sparse coefficient expression.

[0101] The above-mentioned product can execute the bridge structure defect determination method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0102] Embodiment 3

[0103] Figure 3 A schematic diagram of the structure of a drone 123 that can be used to implement an embodiment of the present invention is shown. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0104] like Figure 3 As shown, the drone 123 includes: a camera 10, a light conversion mirror 20 and a drone body 30; the drone body 30 is connected to the camera 10, and the drone body 30 is connected to the light conversion mirror 20. Exemplarily, in this embodiment, the camera 10 can be a three-axis gimbal camera, which can stably take 48 million pixel photos, but the camera captures a viewing angle of -90° to +24° (the viewing angle is 0° when viewed horizontally, negative when viewed downward, and positive when viewed upward). Therefore, a light conversion mirror 20 and a corresponding stabilizing device are designed in front of the camera ( Figure 3The light conversion mirror 20 can be a trapezoidal reflector in front of the lens, which is at an angle of 45° to the vertical plane.

[0105] The drone body 30 includes: at least one processor 31, and a memory connected to the at least one processor 31, such as a read-only memory (ROM) 32, a random access memory (RAM) 33, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 31 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 32 or the computer program loaded from the storage unit 38 to the random access memory (RAM) 33. In the RAM 33, various programs and data required for the operation of the drone 123 can also be stored. The processor 31, the ROM 32, and the RAM 33 are connected to each other through a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0106] A number of components in the drone 123 are connected to the I / O interface 35, including: an input unit 36, such as a keyboard, a mouse, etc.; an output unit 37, such as various types of displays, speakers, etc.; a storage unit 38, such as a disk, an optical disk, etc.; and a communication unit 39, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 39 allows the drone 123 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0107] The processor 31 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 31 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 31 executes the various methods and processes described above, such as the bridge structure disease determination method:

[0108] In the process of controlling the UAV to fly along the bridge, controlling the UAV to identify structural defects on the surface of the bridge;

[0109] Photographing the identified bridge structure defect area to obtain at least one bridge structure defect image;

[0110] Structural defect type identification is performed on each of the bridge structural defect images to obtain the structural defect type corresponding to each bridge structural defect image.

[0111] In some embodiments, the bridge structure disease determination method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 38. In some embodiments, part or all of the computer program can be loaded and / or installed on the drone 123 via the ROM 32 and / or the communication unit 39. When the computer program is loaded into the RAM 33 and executed by the processor 31, one or more steps of the bridge structure disease determination method described above can be performed. Alternatively, in other embodiments, the processor 31 can be configured to execute the bridge structure disease determination method in any other appropriate manner (for example, by means of firmware).

[0112] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0113] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0114] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0115] To provide interaction with a user, the systems and techniques described herein may be implemented on a drone having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the drone. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0116] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0117] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0118] In one embodiment, the embodiment of the present invention further includes a computer program product, the computer program product includes a computer program, and the computer program, when executed by a processor, implements the bridge structure defect determination method of any embodiment of the present invention.

[0119] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0120] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0121] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for determining bridge structure defects, characterized in that: include: In the process of controlling the UAV to fly along the bridge, controlling the UAV to identify structural defects on the surface of the bridge; Photographing the identified bridge structure defect area to obtain at least one bridge structure defect image; Structural defect type identification is performed on each of the bridge structural defect images to obtain the structural defect type corresponding to each bridge structural defect image.

2. The method according to claim 1, characterized in that Before controlling the UAV to identify structural defects on the bridge surface, the method further includes: Obtain a sample set of bridge structure disease images; A dictionary learning function is constructed, and based on the dictionary learning function, dictionary learning is performed according to the bridge structure defect image sample set to obtain an overcomplete dictionary.

3. The method according to claim 2, characterized in that Controlling the drone to identify structural defects on the bridge surface includes: The drone is controlled to determine whether there is a bridge structure defect area on the bridge surface whose similarity with the bridge structure defect image samples corresponding to the overcomplete dictionary is greater than a preset threshold.

4. The method according to claim 2, characterized in that: Identifying the structural damage type of each bridge structure damage image includes: Determine a dictionary corresponding to each bridge structure defect type based on the overcomplete dictionary; An image set corresponding to each bridge structure disease type is determined according to each of the bridge structure disease images and a dictionary corresponding to each bridge structure disease type.

5. The method according to claim 4, characterized in that Determining an image set corresponding to each bridge structure disease type according to each of the bridge structure disease images and a dictionary corresponding to each bridge structure disease type includes: Classification is performed according to the reconstruction error size of each bridge structure defect image on the dictionary corresponding to each bridge structure defect type, so as to obtain an image set corresponding to each bridge structure defect type.

6. The method according to claim 4, characterized in that Determining a dictionary corresponding to each bridge structure defect type based on the overcomplete dictionary includes: A sparse coefficient expression in the dictionary learning function is constructed, and a dictionary corresponding to each bridge structure defect type is determined based on the overcomplete dictionary and the sparse coefficient expression.

7. A bridge structure defect determination device, characterized in that: include: A control module, used to control the UAV to identify structural defects on the bridge surface during the process of controlling the UAV to fly along the bridge; A shooting module, used to shoot the identified bridge structure defect area to obtain at least one bridge structure defect image; The identification module is used to identify the structural damage type of each bridge structure damage image to obtain the structural damage type corresponding to each bridge structure damage image.

8. A drone, characterized in that: The drone comprises: a drone body, a camera and a light conversion mirror; the drone body is connected to the camera, and the drone body is connected to the light conversion mirror; The drone body includes: at least one processor; and a memory that is communicatively connected to the at least one processor; wherein the memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the bridge structure defect determination method described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the bridge structure defect determination method according to any one of claims 1 to 6 when executed.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method for determining bridge structure defects according to any one of claims 1 to 6 is implemented.

Images