Intelligent image assessment identification method for forcible entry assessment

Through intelligent image assessment and recognition methods, the problem of inability to quantitatively evaluate technical and tactical training in earthquake rescue training was solved, and the automatic evaluation of the demolition effect was achieved, and the training effect and the skill level of the emergency rescue team were improved.

CN119068039APending Publication Date: 2024-12-03BEIJING AEROSPACE INST FOR METROLOGY & MEASUREMENT TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410904887.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-08
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing earthquake rescue training lacks intelligent equipment and cannot quantify the evaluation of technical and tactical training such as search and rescue, lifting, and demolition, making it difficult to detect shortcomings and conduct targeted training, affecting the improvement of training results.

Method used

Intelligent image assessment and recognition methods are adopted to achieve automated evaluation of the demolition effect through image preprocessing, demolition area profile extraction, dimension restoration and reference parameter evaluation, and comprehensive judgment is made in combination with referee monitoring.

Benefits of technology

Intelligent assessment of geological disaster rescue training has been realized, the pertinence and effectiveness of training have been improved, and the skill level and training quality of the emergency rescue team have been enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068039A_ABST
    Figure CN119068039A_ABST
Patent Text Reader

Abstract

The invention provides an intelligent image assessment identification method for forcible entry assessment. The method is used for remotely controlling and measuring the horizontal or vertical forcible entry cutting or chiseling effect of a concrete slab. Comprising the following steps: preprocessing a color image of a broken area; extracting a forcible entry area contour from the preprocessed image; according to the distance between the camera and the wall body, size reduction is conducted on the forcible entry size, and the real forcible entry size is obtained; the size comprises a forcible entry area and size data in each direction of forcible entry; judging the trafficability of the forcible entry size; and according to preset reference parameters, in combination with monitoring and judgment of a referee on the operation process, the forcible entry achievement effect is evaluated, and evaluation is completed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of emergency rescue training, and in particular relates to an image assessment and recognition method. Background Art

[0002] my country is one of the countries in the world with the most frequent and most severe geological disasters, which are characterized by wide distribution, concentrated time periods, and severe losses. In 2021 alone, 4,772 geological disasters occurred in my country, causing 26,000 casualties, posing a major threat to the property and life safety of the Chinese people.

[0003] In order to reduce the losses caused by geological disasters and improve rescue capabilities, the country's emergency rescue teams and fire brigades have actively carried out geological disaster rescue training to enhance the skills of rescue personnel. However, at present, my country's earthquake rescue training mainly relies on earthwork facilities such as collapsed buildings, tilted buildings, and demolition platforms. There is no intelligent training equipment, and it can only be judged and evaluated through cameras or manual methods. The degree of intelligence is low, and the main technical and tactical training such as search and rescue, jacking, and demolition cannot be quantified. It is difficult to find deficiencies in techniques and tactics, and it is impossible to carry out targeted training, which affects the further improvement of training effects. Summary of the invention

[0004] In view of this, the present invention proposes an intelligent image assessment and recognition method for demolition assessment, which is used to remotely measure the horizontal or vertical demolition cutting or chiseling effect of concrete slabs, and automatically generate side length or boundary feature values ​​through graphic processing to judge the passability of personnel. According to the pre-set benchmark parameters, combined with the monitoring and judgment of the referee on the operation process, the demolition effect can be evaluated.

[0005] The specific technical solutions are as follows:

[0006] An intelligent image assessment and recognition method for demolition assessment is used for remotely measuring the horizontal or vertical demolition cutting and chiseling effects of concrete slabs, comprising the following steps:

[0007] Step 1: pre-process the color image of the broken area;

[0008] Step 2: extracting the demolition area contour from the preprocessed image;

[0009] Step 3: restore the demolition size according to the distance between the camera and the wall to obtain the real demolition size; the size includes the demolition area and the size data of the demolition in each direction;

[0010] Step 4: Determine the passability of the demolition size;

[0011] Step 5: Based on the pre-set benchmark parameters and the referee's monitoring and judgment of the operation process, the demolition effect is evaluated to complete the assessment.

[0012] Furthermore, the preprocessing includes converting to a grayscale image and performing binarization on the grayscale image.

[0013] Furthermore, the demolition area contour is extracted through machine learning, and comprehensive learning sampling and analysis are conducted on prefabricated panels in different states caused by climate reasons.

[0014] Furthermore, the size restoration of the demolition size to obtain the real demolition size features includes:

[0015] Eliminate the distortion of the collected images through the distortion model;

[0016] The camera image coordinates and world coordinates are converted, and the camera pixel accuracy is obtained through on-site chessboard calibration plate calibration, and the physical size conversion is performed.

[0017] Furthermore, the camera calibration process is as follows:

[0018] 1) Print a chessboard with a known size and stick it on a plane as a calibration object;

[0019] 2) Take some photos of the calibration object in different directions by adjusting the direction of the calibration object or the camera;

[0020] 3) Extract the checkerboard corner points from the photo for detection, obtain the pixel coordinate values ​​of the calibration plate corner points, and calculate the physical coordinate values ​​of the calibration plate corner points based on the known checkerboard size and the origin of the world coordinate system;

[0021] 4) Estimate the internal and external parameter matrix of the camera under ideal distortion-free conditions through the distortion model;

[0022] 5) Apply the least squares method to estimate the distortion coefficient under actual radial distortion.

[0023] 6) Maximum likelihood method, optimizes estimation and improves estimation accuracy.

[0024] Beneficial Effects

[0025] 1. The present invention is used for national emergency rescue teams, fire brigades and other units to carry out intelligent rescue training for geological disasters, and can conduct special intelligent training and assessment for demolition, jacking, search and rescue, etc. in geological disaster rescue training.

[0026] 2. Due to the interference of ambient light in actual experiments, various ambient light sources form various projections on the object to be detected, affecting the contour expression and feature extraction of the object to be detected. The present invention improves the accuracy of detection by converting RGB images into HSL format images.

[0027] 3. Due to the influence of the actual construction environment on site, the camera and the prefabricated board to be inspected are installed at a non-standard vertical angle, resulting in a certain degree of distortion in the collected pictures. The present invention uses a calibration plate for multi-angle calibration on site to model the imaging system and calculate the internal and external parameter matrix of the camera. The distortion model is used to eliminate the influence of distortion by bringing it into the calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 、System overall block diagram

[0029] Figure 2 , Flow chart of the method of the present invention DETAILED DESCRIPTION

[0030] The demolition size assessment system consists of an image acquisition unit, an image processing unit, accessories and a demolition size assessment software system.

[0031] 1) Image acquisition unit

[0032] Dual-channel CameraLink acquisition card, supports multiple camera formats to acquire images, pixel clock frequency 85MHZ, Frame Buffer 256MB

[0033] The industrial camera uses a 5-megapixel high-end area array camera. The camera transmits real-time uncompressed images through the Camera Link interface. Four configuration modes are available: Base, Medium, Full, and 80-bit. The maximum frame rate can reach 140 frames.

[0034] 2) Image processing unit and evaluation software

[0035] Control operation: realize mechanical operation to obtain the best shooting angle and position.

[0036] Image processing: Adapt to different operation modes such as regular and irregular edges, and apply relevant image processing algorithms to realize edge analysis. Measurement: According to the processed image, the demolition size measurement is realized, and the test error is less than 5 mm, and the system response time is less than 5 seconds. Data communication: According to the data transmission protocol, the measured demolition size is uploaded to the central control system. Interface display and control: The interface is friendly, and some functions such as display, operation of machinery, measurement display and judgment can be realized. Independently developed with programs such as C++ / Python / Java / Unity3D.

[0037] The image recognition processing system uses optical technology and computer technology to extract image features and convert them into a format that can be accepted by computers and understood by humans. The collected images are subjected to image preprocessing, layout processing, image segmentation, feature extraction, matching and model training, matching, image geometry processing, arithmetic processing, image enhancement, image restoration, image reconstruction, image coding, segmentation, grayscale processing and image understanding and recognition, and finally algorithm matching with the learned feature images is performed to achieve the correct recognition of the demolition contour.

[0038] In an image, the processing of color information increases the workload of the computer. In addition, many efficient algorithms are based on grayscale images, so converting color images into grayscale images is a necessary part of preprocessing. After generating the grayscale image, the image is binarized, feature contours are extracted, and edge calculations are performed. The main process is shown in the figure.

[0039] After image processing, the outline of the demolition area is extracted, and the demolition size is restored according to the distance between the camera and the wall, so as to obtain the real demolition size characteristics. The size characteristics include: demolition area, demolition size evaluation data in all directions, and then judge the passability of the demolition size to complete the evaluation.

[0040] Main technical indicators and characteristics

[0041] It is used to remotely measure the horizontal or vertical demolition cutting or chiseling effect of concrete slabs. It automatically generates the side length or boundary feature value through graphic processing, automatically determines the passability of personnel, and conducts automatic demolition assessment. It can automatically evaluate the demolition effect based on the pre-set benchmark parameters and the monitoring and judgment of the referee on the operation process.

[0042] 1. Image acquisition card:

[0043] The frame grabber has four independent GigE ports for data transmission and can power four GigE cameras using Power over Ethernet (PoE).

[0044] The supported data transmission rate is up to 1Gbit / s and the cable length can reach 100 meters. Longer distance transmission can be achieved through optical fiber forwarding.

[0045] 2. Camera:

[0046] The photosensitive chip size is 5.7mm x4.3mm;

[0047] Horizontal / vertical resolution: 2590px x 1942px;

[0048] Resolution: 5MP

[0049] Horizontal / vertical pixel size: 2.2μm x 2.2μm;

[0050] Frame rate: 14fps

[0051] Data interface: GigE long distance transmission

[0052] 3. Lens:

[0053] Lens structure: fixed focal length; filter diameter: 11 mm; lens interface: C-mount; aperture: F1.4–F16.0; wide angle: standard; focal length: standard.

[0054] 4. Lighting system:

[0055] LED surface light source. Voltage: multiple voltages; Working wavelength: 3200-6500K (±200K); Power: 10W; Frequency: 20Hz; Quality standard: national standard.

[0056] 5. Installation platform:

[0057] Change to wall mounting according to the on-site environment, and adapt the guide rails and pan / tilt to the on-site environment.

[0058] 6. Test control and graphics processing system:

[0059] Image processing: Adapt to different operation modes such as regular and irregular edges, and apply relevant image processing algorithms to realize edge analysis. Measurement: According to the processed image, the demolition size is measured, the test error is less than 5 mm, and the system response time is less than 5 seconds. Data communication: According to the data transmission protocol, the measured demolition size is uploaded to the central control system. Interface display and control: The interface is friendly and can display and operate machinery, measure display and judge some functions.

[0060] 7. Development platform: independently developed with C++ / C# / Java / Unity3D and other programs.

[0061] The specific process of the present invention is as follows:

[0062] 1. In actual experiments, there is interference from ambient light. Various ambient light sources form various projections on the objects to be tested, affecting the outline and feature extraction of the objects to be tested. By setting the camera acquisition format, the RGB image is converted into an HSL format image, and the influence of ambient light on the detection is reduced by numerical processing of the L channel. By adding multi-angle artificial light sources to cover up the natural light projection of the objects to be tested, the success rate of machine vision detection is increased again.

[0063] 2. Because the site is located in the southwest region with a humid climate, some prefabricated panels to be tested have moss on the surface and internal steel bars are corroded. This interferes with the accuracy of the test. Through machine learning technology, comprehensive learning sampling analysis is performed on prefabricated panels in different states. The visual analysis model is advanced to realize the demolition and detection of prefabricated panels in various states.

[0064] 3. Due to the influence of the actual construction environment on site, the camera and the prefabricated board to be inspected are installed at a non-standard vertical angle, resulting in a certain degree of distortion in the collected images. The imaging system is modeled and the internal and external parameter matrices of the camera are calculated by using a calibration plate for multi-angle calibration on site. The distortion model is used to eliminate the influence of distortion.

[0065] The radial distortion formula (3rd order) is as follows:

[0066]

[0067] The tangential distortion formula is as follows:

[0068]

[0069] 4. Because the final goal is to calculate the area, perimeter, diagonal length, and minimum circumscribed rectangle area of ​​the broken area, it is necessary to convert the camera image coordinates and world coordinates. Calibrate the camera pixel accuracy through the on-site chessboard calibration board and convert the physical size. The specific calibration calculation formula is as follows:

[0070]

[0071] Among them, (U, V, W) is the physical coordinates of a point in the world coordinate system, (u, v) is the pixel coordinates corresponding to the point in the pixel coordinate system, and Z is the scale factor.

[0072] We will matrix:

[0073]

[0074] It is called the intrinsic parameter matrix of the camera, which depends on the internal parameters of the camera. Among them, f is the image distance, dX, dY represent the physical length of a pixel on the camera photosensitive plate in the X and Y directions respectively (that is, how many millimeters a pixel is on the photosensitive plate), u0, v0 represent the coordinates of the center of the camera photosensitive plate in the pixel coordinate system, and θ represents the angle between the horizontal and vertical edges of the photosensitive plate (90° means no error).

[0075] We will matrix: It is called the extrinsic parameter matrix of the camera. The extrinsic parameter matrix depends on the relative position of the camera coordinate system and the world coordinate system. R represents the rotation matrix and T represents the translation vector.

[0076] That is, the single-point distortion-free camera imaging model is as follows:

[0077]

[0078] Camera calibration steps:

[0079] 1. Print a chessboard with a known size and stick it on a flat surface as a calibration object.

[0080] 2. Take some photos of the calibration object in different directions by adjusting the direction of the calibration object or the camera.

[0081] 3. Extract the checkerboard corner points from the photo for detection, and obtain the pixel coordinate values ​​of the corner points of the calibration plate. According to the known size of the checkerboard and the origin of the world coordinate system, calculate the physical coordinate values ​​of the corner points of the calibration plate;

[0082] 4. Estimate five intrinsic parameters and six extrinsic parameters under ideal distortion-free conditions.

[0083] 5. Use the least squares method to estimate the distortion coefficient under actual radial distortion.

[0084] 6. Maximum likelihood method, optimize estimation and improve estimation accuracy.

Claims

1. An intelligent image assessment and recognition method for demolition assessment, used for remotely measuring the horizontal or vertical demolition cutting and chiseling effects of concrete slabs, characterized by: Step 1: pre-process the color image of the broken area; Step 2: extracting the demolition area contour from the preprocessed image; Step 3: restore the demolition size according to the distance between the camera and the wall to obtain the real demolition size; the size includes the demolition area and the size data of the demolition in each direction; Step 4: Determine the passability of the demolition size; Step 5: Based on the pre-set benchmark parameters and the referee's monitoring and judgment of the operation process, the demolition effect is evaluated to complete the assessment.

2. According to the intelligent image assessment and recognition method for demolition assessment described in claim 1, it is characterized by: Preprocessing includes converting to grayscale image and binarizing the grayscale image.

3. According to the intelligent image assessment and recognition method for demolition assessment described in claim 2, it is characterized by: The demolition area contour is extracted through machine learning, and comprehensive learning sampling and analysis are conducted on prefabricated panels in different states caused by climate reasons.

4. According to claim 3, an intelligent image assessment and recognition method for demolition assessment is characterized by: The said performing size restoration on the demolition size to obtain the real demolition size features includes: Eliminate the distortion of the collected images through the distortion model; The camera image coordinates and world coordinates are converted, and the camera pixel accuracy is obtained through on-site chessboard calibration plate calibration, and the physical size conversion is performed.

5. According to any one of the intelligent image assessment and recognition methods for demolition assessment described in claims 1 to 4, it is characterized by: The camera calibration process is as follows: 1) Print a chessboard with a known size and stick it on a plane as a calibration object; 2) Take some photos of the calibration object in different directions by adjusting the direction of the calibration object or the camera; 3) Extract the checkerboard corner points from the photo for detection, obtain the pixel coordinate values ​​of the calibration plate corner points, and calculate the physical coordinate values ​​of the calibration plate corner points based on the known checkerboard size and the origin of the world coordinate system; 4) Estimate the internal and external parameter matrix of the camera under ideal distortion-free conditions through the distortion model; 5) Apply the least squares method to estimate the distortion coefficient under actual radial distortion. 6) Maximum likelihood method, optimizes estimation and improves estimation accuracy.