Bridge health status detection and early warning method, device and medium based on object imaging

Abstract

The present application discloses a bridge health status detection and early warning method, device and medium based on object imaging, which can be widely used in the field of bridge monitoring technology. After real-time video data of the real-time status of the river surface section corresponding to the monitored bridge is collected by a dual-wavelength camera, the real-time video data is subjected to image comparison processing to obtain an image comparison result, and then the observation target image data in the real-time video data is subjected to object imaging processing to obtain an image offset according to the image comparison result, and the bridge health status is judged according to the image offset, and then bridge risk early warning information is generated according to the bridge health status judgment result, thereby effectively reducing the data processing amount of bridge status monitoring to improve data processing efficiency, thereby improving the real-time nature of bridge monitoring and early warning, and at the same time, the image monitoring accuracy is improved by setting an observation target, thereby improving the accuracy of bridge monitoring and early warning.

Description

Technical Field

[0001] The present application relates to the technical field of bridge monitoring, and in particular to a bridge health status detection and early warning method, device and medium based on object imaging. Background Art

[0002] In the relevant technologies, bridges are the core hubs of transportation networks, and their stability and safety are of vital importance. Currently, camera sensors are used to monitor bridges in real time and judge their safety status. However, the monitoring and judgment process of camera sensors requires complex algorithms and processing capabilities, which cannot meet the real-time requirements in bridge collision avoidance scenarios. In addition, camera sensors are easily affected by external factors such as weather, resulting in low clarity of the monitored images, thereby reducing the accuracy of bridge monitoring.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a bridge health status detection and early warning method, device and medium based on object imaging, which can effectively improve the real-time and accuracy of bridge monitoring and early warning.

[0005] To achieve the above-mentioned purpose, an embodiment of the present application provides a bridge health status detection and early warning method based on object imaging, the method comprising the following steps:

[0006] Acquire real-time video data from a dual-wavelength camera, wherein the real-time video data includes the real-time status of a river section corresponding to a monitored bridge, wherein an observation target is provided at a preset position of the monitored bridge;

[0007] Performing image comparison processing on the real-time video data to obtain an image comparison result;

[0008] Performing object imaging processing on the observation target image data in the real-time video data according to the image comparison result to obtain an image offset;

[0009] judging the health status of the bridge according to the image offset;

[0010] Generate bridge risk warning information based on the bridge health status judgment results.

[0011] In some embodiments, performing image comparison processing on the real-time video data to obtain an image comparison result includes:

[0012] Performing feature detection on each frame of the real-time video data to obtain target feature points;

[0013] Performing feature description on the target feature points to obtain a descriptor;

[0014] Calculating a color histogram of an area of ​​a preset size around the target feature point;

[0015] Fusion the descriptor with the color histogram to obtain a composite descriptor;

[0016] Calculating the distance between any two of the composite descriptors;

[0017] Determine a set of matching points according to the distance;

[0018] Calculating a translation vector between each matching point in the matching point set;

[0019] Calculate the total camera offset according to the translation vector;

[0020] If it is determined according to the total camera offset that the camera image has not shifted, the center point of the camera image is used as a reference point; if it is determined according to the total camera offset that the camera image has shifted, an image compensation amount is generated.

[0021] In some embodiments, the performing feature detection on each frame of the real-time video data to obtain target feature points includes:

[0022] Convert each frame of the real-time video data into a Gaussian difference image;

[0023] Calculate the second-order partial derivative matrix of each pixel point of the Gaussian difference image at different scales;

[0024] Detecting candidate feature points from the real-time video data using the second-order partial derivative matrix;

[0025] The target feature point is determined from the candidate feature points using a non-maximum suppression method.

[0026] In some embodiments, the step of performing feature description on the target feature point to obtain a descriptor includes:

[0027] Calculate the gradient direction and magnitude of the area around each target feature point;

[0028] The gradient direction and the magnitude are quantized as descriptors.

[0029] In some embodiments, when the reference point is determined, performing object imaging processing on the observed target image data in the real-time video data according to the image comparison result to obtain the image offset includes:

[0030] Acquire observation target image data from the real-time video data;

[0031] Acquire key point coordinate data in the observed target image data;

[0032] Converting the key point coordinate data into relative coordinate data according to the reference point;

[0033] Calculate the relative offset according to the relative coordinate data;

[0034] The relative offset is mapped to the actual coordinates to obtain the actual offset as the image offset.

[0035] In some embodiments, the dual-wavelength cameras include four cameras, which are respectively arranged at preset upstream and downstream positions on both sides of the monitored bridge, and the four dual-wavelength cameras are powered by a combination of solar panels and batteries.

[0036] In some embodiments, the control system of the four dual-wavelength cameras is equipped with polarized light filters and wipers.

[0037] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application provides a bridge health status detection and early warning device based on object imaging, the device comprising:

[0038] The first module is used to obtain real-time video data from a dual-wavelength camera, wherein the real-time video data includes the real-time status of the river section corresponding to the monitored bridge, and an observation target is provided at a preset position of the monitored bridge;

[0039] The second module is used to perform image comparison processing on the real-time video data to obtain an image comparison result;

[0040] The third module is used to perform object imaging processing on the observation target image data in the real-time video data according to the image comparison result to obtain an image offset;

[0041] A fourth module is used to determine the health status of the bridge according to the image offset;

[0042] The fifth module is used to generate bridge risk warning information based on the bridge health status judgment results.

[0043] To achieve the above objective, another aspect of the present application provides a computer device, including:

[0044] at least one processor;

[0045] at least one memory for storing at least one program;

[0046] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0047] To achieve the above objective, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0048] The embodiments of the present application include at least the following beneficial effects: The present application provides a bridge health status detection and early warning method, device and medium based on object imaging. The scheme uses a dual-wavelength camera to collect real-time video data of the real-time conditions of the river surface section corresponding to the monitored bridge, and then performs image comparison processing on the real-time video data to obtain an image comparison result. Then, according to the image comparison result, object imaging processing is performed on the observation target image data in the real-time video data to obtain an image offset, and the bridge health status is judged according to the image offset. Then, bridge risk early warning information is generated according to the bridge health status judgment result, thereby effectively reducing the data processing volume of bridge status monitoring to improve data processing efficiency, thereby improving the real-time nature of bridge monitoring and early warning, and at the same time, the image monitoring accuracy is improved by setting an observation target, thereby improving the accuracy of bridge monitoring and early warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of a bridge health status detection and early warning method based on object imaging provided in an embodiment of the present application;

[0050] Figure 2 is a schematic diagram of an observation target provided in an embodiment of the present application;

[0051] Figure 3 is a flowchart of a process for determining whether the camera lens frame has moved as a whole, provided in an embodiment of the present application;

[0052] Figure 4 is a processing flow chart of an object imaging algorithm provided in an embodiment of the present application;

[0053] Figure 5 This is an effect diagram of object imaging and target extraction provided by an embodiment of the present application;

[0054] Figure 6 This is a key yellow pixel extraction effect diagram provided in an embodiment of the present application;

[0055] Figure 7 This is a key red pixel extraction effect diagram provided in an embodiment of the present application;

[0056] Figure 8 It is a structural schematic diagram of a bridge health status detection and early warning device based on object imaging provided in an embodiment of the present application;

[0057] Fig. 9 It is a schematic diagram of the hardware structure of the computer device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application, they are only examples of devices and methods consistent with some aspects of the embodiments of the present application.

[0059] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0060] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0062] In the embodiment of the present application, a bridge health status detection and early warning method, device and medium based on object imaging are provided. After real-time video data of the real-time status of the river surface section corresponding to the monitored bridge is collected in real time by a dual-wavelength camera, image comparison processing is performed on the real-time video data to obtain an image comparison result, and then object imaging processing is performed on the observation target image data in the real-time video data according to the image comparison result to obtain an image offset, and the bridge health status is judged according to the image offset, and then bridge risk early warning information is generated according to the bridge health status judgment result, thereby effectively reducing the data processing volume of bridge status monitoring to improve data processing efficiency, thereby improving the real-time nature of bridge monitoring and early warning, and at the same time, the image monitoring accuracy is improved by setting an observation target, thereby improving the accuracy of bridge monitoring and early warning.

[0063] The bridge health status detection and early warning method based on object imaging provided in the embodiment of the present application relates to the field of bridge monitoring technology. The bridge health status detection and early warning method based on object imaging provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements a bridge health status detection and early warning method based on object imaging, etc., but is not limited to the above forms.

[0064] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0065] Figure 1 is an optional flow chart of a bridge health status detection and early warning method based on object imaging provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S110 to S150:

[0066] Step S110, acquiring real-time video data of the dual-wavelength camera, wherein the real-time video data includes the real-time status of the river section corresponding to the monitored bridge, and an observation target is set at a preset position of the monitored bridge;

[0067] Step S120, performing image comparison processing on the real-time video data to obtain an image comparison result;

[0068] Step S130, performing object imaging processing on the observation target image data in the real-time video data according to the image comparison result to obtain an image offset;

[0069] Step S140, judging the health status of the bridge according to the image offset;

[0070] Step S150: Generate bridge risk warning information according to the bridge health status judgment result.

[0071] In the embodiment of the present application, before bridge monitoring and early warning are performed, a number of observation targets are arranged on the monitored bridge in a preset interval size, wherein the preset interval size may be 20 meters or 30 meters, which may be set according to actual conditions. Figure 2 As shown, the observation target can be set to a square, and its size can be 100cm*100cm. Different grids are divided on the observation target, and the square grid can be 40cm*40cm, and different colors are painted in the grid for easy image recognition. The number of dual-wavelength cameras can be 4, and the 4 dual-wavelength cameras are respectively set at the upstream and downstream preset positions on both sides of the monitored bridge through poles, so that the monitored bridge can be monitored in all directions. At the same time, in order to maintain the stable operation of the control system where the 4 dual-wavelength cameras are located, the control system where the dual-wavelength cameras are located is powered by a green energy supply solution combining solar panels and batteries to achieve self-sufficiency and low-carbon operation of the equipment.

[0072] In addition, in order to further adapt to small bridges and reduce costs, the present embodiment can also deploy one camera each in the direction of the upstream and downstream navigation ports of the bridge, for a total of two dual-wavelength cameras. The cameras can be installed on poles to monitor the bridge status of two key navigation ports respectively, so as to monitor the navigation-direction bridge section with the highest probability of accidents with the least deployment.

[0073] In an embodiment of the present application, in order to address the problem of image overexposure caused by strong lighting environments (such as direct sunlight, strong light reflection, etc.), this embodiment additionally configures a polarized light filter on the control system of the dual-wavelength camera. The filter effectively reduces the intensity of light entering the camera lens by selectively absorbing or reflecting light in a specific direction, thereby alleviating the adverse effects of strong light on image quality to a certain extent and improving the stability and clarity of the image.

[0074] In order to deal with the obstruction of vision caused by rain and dust pollution, this embodiment further integrates an intelligent wiper on the basis of the polarized light filter. The control system of the dual-wavelength camera can automatically sense the ambient humidity and dust concentration, and start the wiper in time to remove rain or dust on the lens surface, ensuring the cleanliness of the camera lens and the clearness of the field of vision, effectively overcoming the interference of severe weather conditions on monitoring work, and improving the reliability and stability of the system.

[0075] In the embodiment of the present application, since the dual-wavelength camera is easily affected by wind and rain during real-time monitoring, causing the camera lens frame to shift, which in turn leads to detection errors, the present embodiment uses an image matching method to determine whether the camera lens frame has moved as a whole. The monitored bridge is determined to be shifted based on the lens frame movement judgment result and the object imaging processing process.

[0076] It is understandable that if Figure 3 As shown, the process of determining whether the camera lens frame has moved as a whole includes but is not limited to the following steps:

[0077] Step S610: Perform feature detection on each frame of the real-time video data to obtain target feature points;

[0078] It is understandable that the present embodiment can use the SURF feature detector for feature detection. First, the image is Gaussian blurred by a Gaussian filter to obtain a Gaussian difference image, and a Gaussian pyramid scale space is constructed to detect feature points of different sizes in the Gaussian difference image, and the second-order partial derivative matrix (Hessian) of the pixels corresponding to each feature point at different scales is calculated, and the candidate feature points are detected from the real-time video data through the second-order partial derivative matrix. Among them, the determinant of the Hessian matrix is ​​defined as follows:

[0079] ;

[0080] In the formula, , and is the Gaussian difference image in and The first and second derivatives in the direction, Represents the scale parameter of the Gaussian kernel, which is used to control the width of the Gaussian filter.

[0081] Specifically, for each candidate feature point, this embodiment may use non-maximum suppression (NMS) to select the local maximum value as the target feature point.

[0082] Step S620, performing feature description on the target feature point to obtain a descriptor;

[0083] It can be understood that this embodiment describes the detected target feature points by using SIFT descriptors. Specifically, for each target feature point, the gradient direction and magnitude of the area around each target feature point are calculated, and the gradient direction is quantified using a Gaussian weighting function, and the gradient direction and magnitude are quantized into a descriptor, and the formula is as follows:

[0084] ;

[0085] In the formula, is the Gaussian weight function, is the gradient direction quantization function, Represents the gradient of image I at position (x, y), where (x, y) represents the position of the target feature point.

[0086] Step S630, calculating a color histogram of a preset size area around the target feature point;

[0087] It is understandable that in order to further improve the robustness and distinguishing ability of feature description, this embodiment introduces color information. Specifically, a region of a preset size is extracted around each target feature point, and then the color histogram of the region is calculated:

[0088] ;

[0089] In the formula, is the count of the i-th bin of the color histogram, is the Dirac delta function, It's pixels The color value of is the center value of the i-th color interval.

[0090] Step S640, fusing the descriptor with the color histogram to obtain a composite descriptor;

[0091] It can be understood that the fusion process of this embodiment is as follows:

[0092] ;

[0093] In the formula, Indicates that it matches the descriptor. is the SIFT descriptor, is the color histogram, and It is the weight used to balance SIFT features and color features.

[0094] Step S650, calculating the distance between any two composite descriptors, and determining a matching point set according to the distance;

[0095] It is understandable that in this embodiment, the most similar feature point pairs between two images can be found through the FLANN algorithm, and these feature point pairs are combined into a matching point set. The process of the FLANN algorithm is as follows:

[0096] ;

[0097] In the formula, represents the matching degree between two compound descriptors; M represents the set of compound descriptors to be matched, Represents a composite descriptor With composite descriptor The distance metric function between (Euclidean distance).

[0098] Step S660, calculating the translation vector between each matching point in the matching point set;

[0099] It is understandable that, assuming that the optimal set of matching points has been obtained ,in Feature points in the first image, is the corresponding feature point in the second image. Each feature point is converted into homogeneous coordinate form and the translation matrix is ​​estimated using the least squares method. . Then estimate the translation vector :

[0100] ;

[0101] In the formula, T represents the translation matrix of the matching point.

[0102] Step S670, calculating the total camera offset according to the translation vector;

[0103] It can be understood that this embodiment determines the total offset of the camera by analyzing the translation amount:

[0104] ;

[0105] In the formula, and represents the horizontal and vertical components of the translation component t.

[0106] Step S680: If it is determined according to the total camera offset that the camera image has not shifted, the center point of the camera image is used as a reference point; if it is determined according to the total camera offset that the camera image has shifted, an image compensation amount is generated.

[0107] It can be understood that this embodiment determines whether the camera frame has moved as a whole based on the calculated total camera offset. If there is no movement, the center point of the camera frame is determined to be unchanged and used as a reference point to assist subsequent object imaging to calculate the target key coordinate points and determine bridge changes.

[0108] In the embodiment of the present application, after obtaining the real-time panoramic bridge monitoring video data from the dual-wavelength camera, the object imaging algorithm is used to complete the monitoring of the equidistant observation targets on the side of the bridge, and then determine whether the bridge has health problems. Specifically, Figure 4As shown, the object imaging algorithm is to obtain the convolution kernel parameters containing the features of the object of interest through neural network model training, and to construct a feature filter through the convolution kernel parameters. With the help of the feature filter, the specific image in the complex image is screened and extracted, that is, the identification of the object of interest is completed, which avoids the image containing irrelevant information and background information during imaging. For example, after obtaining the real-time monitoring image of the bridge, the corresponding imaging effect diagram can be obtained after processing by the object imaging algorithm, and the imaging effect diagram filters out irrelevant background information.

[0109] It is understandable that object imaging can be implemented based on various types of convolutional neural networks. Considering the similarity between object imaging tasks and segmentation tasks, the classic U-Net network is selected for implementation. U-Net is a convolutional neural network (CNN) for image segmentation, which is mostly used for medical image segmentation tasks. The structure of U-Net is a symmetrical encoder-decoder structure, in which the encoder adopts a stacked convolutional layer structure similar to VGG, and each encoder stage contains a convolution layer and a pooling layer. The decoder is symmetrical with the encoder, and each decoder stage contains an upsampling operation (deconvolution or interpolation) and a convolution layer. In the decoder stage, skip connections are also introduced to merge the feature map of the encoder with the feature map of the decoder to improve the accuracy of the model. The advantages of U-Net are that it can handle relatively small data sets, has high segmentation accuracy, and can be applied to different image segmentation tasks. It is widely used in medical images, such as the segmentation of lungs, livers, and blood vessels.

[0110] In the embodiment of the present application, when processing real-time video data, firstly, an object imaging algorithm is used to obtain the following from the real-time video data: Figure 5 The pure target data in the full scene shown is used as the observation target image data, and then these data are handed over to the subsequent algorithm to accurately calculate the key coordinate points, match them with the coordinate points of the past state, determine the offset and then evaluate the deformation and state change of the bridge. In practical applications, in order to further reduce the computing pressure of the full-scene image processing of the industrial control board and optimize resource utilization and processing efficiency, this embodiment processes the extracted full-scene target data one by one in sequence, focusing on the in-depth analysis and precise measurement of a single target. These two strategies complement each other and ensure the balanced optimization of the system in terms of speed, accuracy and computing power.

[0111] In the embodiment of the present application, since the color features of the observed target are rich, the present embodiment analyzes the HSV color space of the observed target image data and accurately extracts the pixels of the two key colors of red and yellow according to the preset color threshold. Figure 6 and Figure 7As shown. Then, after obtaining the key point coordinate data in the observed target image data, the key point coordinate data is converted into relative coordinate data according to the reference point, that is, the relative coordinates of the five points (four corner points and the target center point) relative to the frame center reference point are calculated by pixel value. Then, the relative coordinate data is compared with the previous information, and the relative offset between the five points is obtained according to the linear relationship, and then the relative offset is mapped to the actual coordinate system to obtain the actual offset as the image offset.

[0112] In the embodiment of the present application, after the actual offset is obtained, three levels of alarm are divided according to the offset to distinguish the severity of the bridge damage. If the actual coordinates are found to be offset at the centimeter level according to the actual offset, and it is determined that the entire frame of the lens is not offset, that is, the situation of coordinate offset caused by the shaking of the lens due to wind is avoided, then the system can accurately determine that the bridge may have health hazards, such as tilting, structural damage, etc.

[0113] It is understandable that this embodiment handles the corresponding alarm events through three-level alarm classification. When a low-risk event is judged to have occurred based on the actual offset, the alarm information is uploaded to the backend competent department in a timely manner. When a medium-risk or above event occurs, an automatic alarm can be realized and an emergency reminder can be given to vehicles and pedestrians at the scene in the first time, such as starting the road warning LED display screen to play "Traffic control, please detour" and other warning words (the content can be adjusted). The bridge management unit can analyze the incident situation and determine that the bridge is safe to pass or the emergency disposal is completed, then lift the on-site alarm and restore the on-site traffic in an orderly manner. And automatically start the bridge deck warning lights and horn equipment, and the red and blue warning lights under the bridge head LED screen to remind passing vehicles in time. And install a 1㎡ road warning LED display screen on the shoulder of the approach road at both ends of the bridge by means of poles, so that drivers can view the warning content in time.

[0114] When a high-risk alarm event is judged to have occurred based on the actual offset, such as a preliminary judgment that the bridge may collapse at any time, such as collapse, collapse, serious damage to the main structure of the bridge, and other events that may cause casualties, it is necessary to close the bridge deck access road and remind vehicles to stop passing. At the same time, the relevant personnel should be reported by phone within 3 minutes, and a written report should be submitted in the work group in accordance with the initial information report format. Pictures reflecting the occurrence of the incident and the condition of the bridge should be submitted to the work group within 5 minutes, and video materials reflecting the occurrence of the incident and the condition of the bridge should be submitted to the work group within 10 minutes. After the warning event occurs, the maritime and waterway departments should be informed depending on the development of the situation. A 1㎡ road warning LED display screen should be installed on the shoulders of the approach roads at both ends of the bridge using a pole. The pole height should be sufficient and the screen should be eye-catching, so that the driver can view the warning content in time.

[0115] In summary, the embodiment of the present application uses a dual-wavelength camera on the river bank to monitor the real-time status of the road section above the river surface of the bridge around the clock, and uses an object imaging processing algorithm to realize the spatial calculation of the bridge observation target, thereby detecting the health status of the bridge (deformation, tilt, collapse, etc.), and transmitting the data to the decision center. In the edge computing decision center, the received data is deeply processed and evaluated using the preset anti-collision safety rules, and finally it is determined whether there is a health problem with the bridge and divided into three levels of alarm. At the same time, this embodiment can also automatically send alarm information to the bridge road drivers and passengers through the bridge deck LED screen display, sound and light alarm, etc., and promptly report to the back-end competent department to ensure the long-term safety of the bridge structure and the smooth passage of traffic flow.

[0116] Reference Figure 8 The embodiment of the present application provides a bridge health status detection and early warning device based on object imaging, the device comprising:

[0117] The first module 1310 is used to obtain real-time video data from a dual-wavelength camera, wherein the real-time video data includes the real-time status of the river section corresponding to the monitored bridge, and an observation target is set at a preset position of the monitored bridge;

[0118] The second module 1320 is used to perform image comparison processing on the real-time video data to obtain an image comparison result;

[0119] The third module 1330 is used to perform object imaging processing on the observation target image data in the real-time video data according to the image comparison result to obtain an image offset;

[0120] The fourth module 1340 is used to determine the health status of the bridge according to the image offset;

[0121] The fifth module 1350 is used to generate bridge risk warning information according to the bridge health status judgment result.

[0122] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0123] The embodiment of the present application also provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned bridge health status detection and early warning method based on object imaging when executing the computer program. The computer device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0124] It can be understood that the contents of the above method embodiments are all applicable to the computer device embodiments, the functions specifically implemented by the computer device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0125] See also Fig. 9 , Fig. 9 The hardware structure of a computer device according to another embodiment is shown, and the computer device includes:

[0126] The processor 1410 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0127] The memory 1420 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1420 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1420, and the processor 1410 calls and executes the bridge health status detection and early warning method based on object imaging in the embodiment of this application;

[0128] Input / output interface 1430, used to implement information input and output;

[0129] Communication interface 1440, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0130] The bus 1450 transmits information between the various components of the device (e.g., the processor 1410, the memory 1420, the input / output interface 1430, and the communication interface 1440);

[0131] The processor 1410 , the memory 1420 , the input / output interface 1430 , and the communication interface 1440 are connected to each other in communication within the device via a bus 1450 .

[0132] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned bridge health status detection and early warning method based on object imaging is implemented.

[0133] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0134] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0135] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0136] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0137] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0138] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0139] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0140] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0141] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0142] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0143] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0144] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0145] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.

Claims

1. A bridge health status detection and early warning method based on object imaging, characterized in that: The method comprises the following steps: Acquire real-time video data of a dual-wavelength camera, the real-time video data including the real-time status of the river section corresponding to the monitored bridge, an observation target is provided at a preset position of the monitored bridge, a yellow block and a red block are provided on the observation target, and the red block is arranged inside the yellow block; a polarized light filter and a wiper are provided on the control coefficient of the dual-wavelength camera; Performing image comparison processing on the real-time video data to obtain an image comparison result; Performing object imaging processing on the observation target image data in the real-time video data according to the image comparison result to obtain an image offset; judging the health status of the bridge according to the image offset; Generate bridge risk warning information based on the bridge health status judgment results combined with the three-level alarm; The performing image comparison processing on the real-time video data to obtain an image comparison result includes: Performing feature detection on each frame of the real-time video data to obtain target feature points, wherein the target feature points are target feature points of each frame of the image at different scales; Performing feature description on the target feature points at each scale to obtain a descriptor; Calculate the color histogram of a preset size area around the target feature point at each scale; Merging all the descriptors with all the color histograms to obtain a composite descriptor; Calculating the distance between any two of the composite descriptors; Determine a set of matching points according to the distance; Calculating a translation vector between each matching point in the matching point set; Calculate the total camera offset according to the translation vector; If it is determined according to the total camera offset that the camera image has not been offset, the center point of the camera image is used as a reference point; if it is determined according to the total camera offset that the camera image has been offset, an image compensation amount is generated; When the reference point is determined, the object imaging processing is performed on the observation target image data in the real-time video data according to the image comparison result to obtain the image offset, including: Acquire observation target image data from the real-time video data, analyze the HSV color space of the observation target image data, and accurately extract pixel points of two key colors, red and yellow, according to a preset color threshold; the observation target image data is pure target data; Acquire key point coordinate data in the observation target image data according to the pixel points of the two key colors of red and yellow; Converting the key point coordinate data into relative coordinate data according to the reference point; Calculate the relative offset according to the relative coordinate data; The relative offset is mapped to the actual coordinates to obtain the actual offset as the image offset.

2. The method according to claim 1, characterized in that The performing feature detection on each frame of the real-time video data to obtain target feature points includes: Convert each frame of the real-time video data into a Gaussian difference image; Calculate the second-order partial derivative matrix of each pixel point of the Gaussian difference image at different scales; Detecting candidate feature points from the real-time video data using the second-order partial derivative matrix; The target feature point is determined from the candidate feature points using a non-maximum suppression method.

3. The method according to claim 1, characterized in that The step of performing feature description on the target feature point to obtain a descriptor includes: Calculate the gradient direction and magnitude of the area around each target feature point; The gradient direction and the magnitude are quantized as descriptors.

4. The method according to claim 1, characterized in that: The dual-wavelength cameras include four, which are respectively arranged at preset upstream and downstream positions on both sides of the monitored bridge, and are powered by a combination of solar panels and batteries.

5. A bridge health status detection and early warning device based on object imaging, characterized in that: The device comprises: The first module is used to obtain real-time video data of a dual-wavelength camera, wherein the real-time video data includes the real-time status of the river section corresponding to the monitored bridge, an observation target is provided at a preset position of the monitored bridge, a yellow block and a red block are provided on the observation target, and the red block is provided inside the yellow block; a polarized light filter and a wiper are provided on the control coefficient of the dual-wavelength camera; The second module is used to perform image comparison processing on the real-time video data to obtain an image comparison result; The third module is used to perform object imaging processing on the observation target image data in the real-time video data according to the image comparison result to obtain an image offset; A fourth module is used to determine the health status of the bridge according to the image offset; The fifth module is used to generate bridge risk warning information based on the bridge health status judgment results combined with the three-level alarm; The performing image comparison processing on the real-time video data to obtain an image comparison result includes: Performing feature detection on each frame of the real-time video data to obtain target feature points, wherein the target feature points are target feature points of each frame of the image at different scales; Performing feature description on the target feature points at each scale to obtain a descriptor; Calculate the color histogram of a preset size area around the target feature point at each scale; Merging all the descriptors with all the color histograms to obtain a composite descriptor; Calculating the distance between any two of the composite descriptors; Determine a set of matching points according to the distance; Calculating a translation vector between each matching point in the matching point set; Calculate the total camera offset according to the translation vector; If it is determined according to the total camera offset that the camera image has not been offset, the center point of the camera image is used as a reference point; if it is determined according to the total camera offset that the camera image has been offset, an image compensation amount is generated; When the reference point is determined, the object imaging processing is performed on the observation target image data in the real-time video data according to the image comparison result to obtain the image offset, including: Acquire observation target image data from the real-time video data, analyze the HSV color space of the observation target image data, and accurately extract pixel points of two key colors, red and yellow, according to a preset color threshold; the observation target image data is pure target data; Acquire key point coordinate data in the observation target image data according to the pixel points of the two key colors of red and yellow; Converting the key point coordinate data into relative coordinate data according to the reference point; Calculate the relative offset according to the relative coordinate data; The relative offset is mapped to the actual coordinates to obtain the actual offset as the image offset.

6. A computer device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images