Laser forcible entry dynamic optimization method and system based on machine vision guidance

Abstract

The invention discloses a laser forcible entry dynamic optimization method based on machine vision guidance, and the method comprises the steps: S1, calculating a color histogram of a cutting head region in a video image after the initial state is calibrated; s2, based on the similarity of color histograms, determining the current center position of a cutting head area in a # imgabs0 # frame video image; s3, identifying a current cutting state of a cutting head in a # imgabs1 # frame video image by adopting a deep learning network; s4, on the basis of the current center position and the current cutting state, a next center position and next output power are obtained through prediction; and S5, based on the next center position and the next output power, the movement and the output power of a cutting head are controlled, and precise forcible entry is achieved. The machine vision technology is adopted, and effective positioning of the cutting head and effective recognition of the cutting state are achieved; and on the basis of state prediction of the cutting head, feedback control is conducted on the motion and output power of the cutting head, and accurate forcible entry is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of emergency rescue, and in particular to a dynamic optimization method and system for laser demolition guided by machine vision. Background Art

[0002] Laser cutting can complete the cutting of hard materials in a relatively short time, which is crucial for emergency rescue. Laser cutting is to heat and vaporize materials by using high-energy laser beams, so as to achieve material cutting; as a new rescue technology, laser cutting can reduce secondary damage to the surrounding environment and secondary injuries to personnel.

[0003] Laser cutting is a complex process. The material, thickness, shape of the material to be cut, and the output power of the cutting head will all affect the cutting state. Therefore, in the process of laser cutting, it is very useful to adjust the movement of the cutting head in real time through feedback control to ensure the accuracy of cutting, enhance the cutting stability, and improve the cutting quality.

[0004] However, the existing feedback control systems in the field of laser cutting mainly position and control the cutting head through sensors, and there are technical problems such as low positioning accuracy, poor timeliness, and inability to accurately control the cutting state of the cutting head. Summary of the Invention

[0005] In view of this, based on machine vision technology, the present invention analyzes and processes to obtain the center position of the cutting head area and the cutting state of the cutting head, and performs feedback control on the movement and output power of the cutting head to achieve precise demolition. The technical solution proposed by the present invention is a dynamic optimization method for laser demolition guided by machine vision, including the following steps: S1: In the initial state, calibrate the video image with the actual position of the cutting head, and calculate the color histogram of the cutting head area in the video image; S2: Based on the similarity of the color histogram of the cutting head area, determine the current center position of the cutting head area in the th frame of the video image; S3: Use a deep learning network to identify the current cutting state of the cutting head in the th frame of the video image; S4: Based on the current center position and the current cutting state, predict the center position of the cutting head area and the output power of the cutting head in the next frame of the video image to obtain the next center position and the next output power; S5: Based on the next center position and the next output power, control the movement and output power of the cutting head to achieve precise demolition.

[0006] Optionally, in step S1, in the initial state, calibrate the calibration video image with the actual position of the cutting head, and calculate the color histogram of the cutting head area in the video image, including: S11: In the initial state, the electric lens calibrates the calibration video image with the actual position of the cutting head by translation and tilting to ensure that the cutting head is in the video image; S12: In the video image, determine the cutting head area and calculate the color histogram of the cutting head area: ; Among them, represents the color histogram of the cutting head area, represents the color value, represents the probability distribution of the color in the cutting head area.

[0007] Optionally, in step S2, based on the similarity of the color histogram of the cutting head area, determine the current center position of the cutting head area in the th frame of the video image, including: S21: In the th frame of the video image, determine the boundary values of the cutting head candidate area: ; Among them, represents the length range of the cutting head candidate area, represents the width range of the cutting head candidate area, represents the center position of the th frame of the video image, represents the size of the cutting head candidate area; S22: Calculate the color histogram of the cutting head candidate area: ; Among them, represents the color histogram of the cutting head candidate area, represents the probability distribution of the color in the cutting head candidate area; S23: Measure the similarity between the color histogram of the cutting head candidate area and the color histogram of the cutting head area: ; Among them, represents the similarity coefficient; S24: If , it is considered that the color histogram of the cutting head candidate area is similar to the color histogram of the cutting head area, and the current center position of the cutting head area is determined as: ; Among them, represents the center position of the cutting head area in the frame of the video image; If , it is considered that the color histogram of the cutting head candidate area and the color histogram of the cutting head area do not have similarity; Update , , where represents the length offset, represents the width offset; S25: Repeat steps S22, S23 and S24 until .

[0008] Optionally, in the step S3, a deep learning network is used to identify the current cutting state of the cutting head in the frame of the video image, including: Input the frame of the video image into the trained deep learning network to identify the current cutting state of the cutting head; The cutting states include: normal cutting, piercing, overheating cutting, underburning cutting, jitter cutting, offset cutting; Among them, in the normal cutting state, the position of the cutting head changes according to a predetermined route; in the piercing state, the cutting head heats and pierces the material at the current position, and the position of the cutting head remains unchanged; in the overheating cutting and underburning cutting states, the cutting speed of the cutting head slows down, and the position of the cutting head changes slowly according to a predetermined route; in the jitter cutting state, the cutting head will have unstable jitter, and the position of the cutting head has a small offset relative to the predetermined route; in the offset cutting state, due to the system nowhere and material movement, cutting offset misalignment occurs, and the position of the cutting head has a large offset relative to the predetermined route.

[0009] Optionally, in the step S4, based on the current center position and the current cutting state, predict the center position of the cutting head area and the output power of the cutting head in the next frame of the video image, and obtain the next center position and the next output power, including: S41: Based on the current center position, construct a cutting head motion state vector: ; where represents the motion state vector of the cutting head in the frame of the video image, represents the transpose matrix of the matrix , represents the center position of the cutting head area in the frame of the video image, represents the Output power of the cutting head in the frame video image, Indicates Offset speed of the cutting head area in the frame video image: ; in, Indicates The center position of the cutting head area in the frame video image; S42: Based on the current cutting state, predict the motion state vector of the cutting head in the next frame of video image to obtain the next center position and the next output power: ; in, Indicates The motion state vector prediction value of the cutting head in the frame video image, the state transfer matrix Related to the current cutting state, select a suitable state transfer matrix according to the current cutting state ; The first two elements of are the next center position ; The last element is the next output power ; In the normal cutting state, the cutting head moves at a constant speed along the predetermined route, and the state transfer matrix is: ; In the perforation state, the cutting head heats and perforates the material at the current position, and the position of the cutting head remains unchanged. The state transfer matrix is: ; Under overheat cutting and underburn cutting conditions, the state transfer matrix is: ; In the jitter cutting state, the cutting speed of the cutting head will slow down, the cutting head will decelerate along the predetermined route, and the position of the cutting head will change slowly along the predetermined route; state transfer matrix: ; in, Indicates that the cutting head is The velocity coefficient of the direction, Indicates that the cutting head is Speed ​​coefficient of direction; In the offset cutting state, due to system errors and material movement, the cutting offset is inaccurate, and the position of the cutting head deviates greatly from the predetermined route. The state transfer matrix is: ; in, Indicates the speed offset coefficient of the cutting head in the direction, Indicates the speed offset coefficient of the cutting head in the direction.

[0010] Optionally, in step S5, based on the next center position and the next output power, controlling the movement and output power of the cutting head to achieve precise demolition includes: S51: Based on the next output power , output the cutting power; S52: Based on the next center position , control the movement of the cutting head to achieve precise demolition.

[0011] The present invention also provides a laser demolition dynamic optimization system based on machine vision guidance, including: Cutting head area color histogram module: used to calibrate the video image and the actual position of the cutting head in the initial state, and calculate the color histogram of the cutting head area in the video image; Cutting head area center position determination module: used to determine the current center position of the cutting head area in the t-th frame video image; Cutting head cutting state determination module: used to identify the current cutting state of the cutting head in the t-th frame video image based on a deep learning network; Prediction module: used to predict the center position of the cutting head area and the output power of the cutting head in the next frame video image based on the current center position and the current cutting state, and obtain the next center position and the next output power; Precise demolition module: used to control the movement and output power of the cutting head based on the next center position and the next output power to achieve precise demolition.

[0012] Beneficial effects The present invention adopts machine vision technology, based on the color histogram of the cutting head area, realizes effective positioning of the cutting head in the video image, improves the tracking efficiency and accuracy of the cutting head; on the basis of tracking the cutting head, effectively identifies the cutting head state; based on different cutting head states, realizes precise prediction of the center position of the cutting head area and the output power of the cutting head in the next frame video image; through feedback control of the movement and output power of the cutting head, realizes precise demolition.

[0013] The present invention makes full use of the morphological features of the cutting head region in the video image to measure the similarity between the color histogram of the cutting head candidate region and the color histogram of the cutting head region; and combines the region offset to accurately obtain the center position of the cutting head region; based on the center position, offset speed of the cutting head region and the output power of the cutting head, fully considering the motion characteristics of the cutting head, constructs the motion state vector of the cutting head; based on the cutting state of the cutting head, selects the state transition matrix to realize the prediction of the motion state vector of the cutting head. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 FIG. is a schematic flow chart of a dynamic optimization method for laser demolition guided by machine vision provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0015] The present invention will be further described below with reference to the drawings, but the present invention is not limited in any way. Any transformation or replacement made based on the teachings of the present invention falls within the protection scope of the present invention.

[0016] Embodiment 1: A dynamic optimization method for laser demolition guided by machine vision, as Figure 1 shown, includes the following steps: S1: In the initial state, calibrate the video image with the actual position of the cutting head, and calculate the color histogram of the cutting head region in the video image; It can be understood that by calibrating the video image, the display position of the cutting head is aligned with its true physical position to ensure that the cutting head accurately appears in the specified area of the video image. Then, identify the pixel color distribution in this area, calculate and generate the color histogram of the cutting head region (the color histogram is a histogram used to reflect the color composition distribution of the image), providing a color feature data basis for subsequent image matching and tracking.

[0017] Optionally, in step S1, in the initial state, calibrating the video image with the actual position of the cutting head and calculating the color histogram of the cutting head region in the video image includes: S11: In the initial state, the electric lens calibrates the video image with the actual position of the cutting head through translation and tilting to ensure that the cutting head is in the video image; S12: In the video image, determine the cutting head region and calculate the color histogram of the cutting head region: ; Wherein, represents the color histogram of the cutting head region, represents the color value, represents the probability distribution of the color in the cutting head region.

[0018] S2: Based on the similarity of the color histogram of the cutting head area, determine the current center position of the cutting head area in the frame video image; It can be understood that by comparing the color histograms of each area in the frame video image with the color histogram of the cutting head area, the area with the most similar color features can be found, so as to determine the area where the cutting head is located in the current frame, and calculate the center position of the cutting head area to obtain the current center position of the cutting head area.

[0019] Optionally, in step S2, based on the similarity of the color histogram of the cutting head area, determine the current center position of the cutting head area in the frame video image, including: S21: In the frame video image, determine the boundary values of the cutting head candidate area: ; Among them, represents the length range of the cutting head candidate area, represents the width range of the cutting head candidate area, represents the center position of the frame video image, represents the size of the cutting head candidate area; S22: Calculate the color histogram of the cutting head candidate area: ; Among them, represents the color histogram of the cutting head candidate area, represents the probability distribution of the color in the cutting head candidate area; S23: Measure the similarity between the color histogram of the cutting head candidate area and the color histogram of the cutting head area: ; Among them, represents the similarity coefficient; S24: If , it is considered that the color histogram of the cutting head candidate area is similar to the color histogram of the cutting head area, and the determined current center position of the cutting head area is: ; Among them, represents the center position of the cutting head area in the frame video image; If , it is considered that the color histogram of the cutting head candidate area is not similar to the color histogram of the cutting head area; Update , , where represents the length offset, represents the width offset; S25: Repeat steps S22, S23, and S24 until .

[0020] S3: Use a deep learning network to identify the current cutting state of the cutting head in the th frame of the video image; Optionally, in step S3, use a deep learning network to identify the current cutting state of the cutting head in the th frame of the video image, including: Input the th frame of the video image into the trained deep learning network to identify the current cutting state of the cutting head; In the embodiments of the present invention, the cutting states include: normal cutting, perforation, overheat cutting, underburn cutting, jitter cutting, and offset cutting; among them, in the normal cutting state, the position of the cutting head changes along a predetermined route; in the perforation state, the cutting head heats and perforates the material at the current position, and the position of the cutting head remains unchanged; in the overheat cutting and underburn cutting states, the cutting speed of the cutting head will slow down, and the position of the cutting head changes slowly along a predetermined route; in the jitter cutting state, the cutting head will experience unstable jitter, and the position of the cutting head has a small offset relative to the predetermined route; in the offset cutting state, due to system errors and material movement, cutting offset inaccuracy occurs, and the position of the cutting head has a large offset relative to the predetermined route.

[0021] S4: Based on the current center position and the current cutting state, predict the center position and the output power of the cutting head area in the next frame of the video image to obtain the next center position and the next output power; Optionally, in step S4, based on the current center position and the current cutting state, predict the center position and the output power of the cutting head area in the next frame of the video image to obtain the next center position and the next output power, including: S41: Based on the current center position, construct a cutting head motion state vector: ; where represents the motion state vector of the cutting head in the th frame of the video image, represents the transpose matrix of the matrix , represents the center position of the cutting head area in the th frame of the video image, represents the The output power of the cutting head in the -th frame video image, ; wherein, represents the center position of the cutting head area in the -th frame video image; ; wherein, represents the predicted value of the motion state vector of the cutting head in the -th frame video image, the state transition matrix is related to the current cutting state, and a suitable state transition matrix is selected according to the current cutting state The first two elements of are the next center position The last element of is the next output power In the embodiments of the present invention, in the normal cutting state, the cutting head moves at a constant speed along a predetermined route, and the state transition matrix: ; In the piercing state, the cutting head heats and pierces the material at the current position, and the position of the cutting head remains unchanged. The state transition matrix: ; In the overheat cutting and underburn cutting states, the state transition matrix: ;

[0022] In the jitter cutting state, the cutting speed of the cutting head will slow down, and the cutting head moves at a reduced speed along a predetermined route, and the position of the cutting head changes slowly along a predetermined route; the state transition matrix: ; wherein, represents the velocity coefficient of the cutting head in the direction, represents the velocity coefficient of the cutting head in the direction; In the offset cutting state, due to system errors and material movement, cutting offset misalignment occurs, and the position of the cutting head has a large offset relative to the predetermined route. The state transition matrix: ; Among them, represents the speed offset coefficient of the cutting head in the direction, represents the speed offset coefficient of the cutting head in the direction; It can be understood that through the current cutting state, a state transition matrix is selected; based on the current central position and the current cutting state, through the state transition matrix, the motion state vector of the cutting head in the next frame of video image is predicted, and the next central position and the next output power are obtained.

[0023] S5: Based on the next central position and the next output power, control the motion and output power of the cutting head to achieve precise demolition; Optionally, in the step S5, based on the next central position and the next output power, controlling the motion and output power of the cutting head to achieve precise demolition includes: S51: Based on the next output power , output the cutting power; S52: Based on the next central position , control the motion of the cutting head to achieve precise demolition.

[0024] Embodiment 2: The present invention also provides a laser demolition dynamic optimization system based on machine vision guidance, including the following five modules: Cutting head region color histogram module: used to calibrate the video image and the actual position of the cutting head in the initial state, and calculate the color histogram of the cutting head region in the video image; Cutting head region central position determination module: used to determine the current central position of the cutting head region in the th frame of video image; Cutting head cutting state determination module: used to identify the current cutting state of the cutting head in the th frame of video image based on a deep learning network; Prediction module: used to predict the central position of the cutting head region and the output power of the cutting head in the next frame of video image based on the current central position and the current cutting state, and obtain the next central position and the next output power; Precise demolition module: used to control the motion and output power of the cutting head based on the next central position and the next output power to achieve precise demolition.

[0025] For the specific limitations of a dynamic optimization system for laser demolition guided by machine vision, reference may be made to the limitations of a dynamic optimization method for laser demolition guided by machine vision in the foregoing text, which will not be elaborated herein. Each module in the above-mentioned dynamic optimization system for laser demolition guided by machine vision can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so as to facilitate the processor to call and execute the operations corresponding to each of the above modules.

[0026] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments. And the term "including", "comprising" or any other variant thereof in this article is intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, device, article or method. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, device, article or method including that element.

[0027] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0028] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structural or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A laser demolition dynamic optimization method based on machine vision guidance, characterized in that: The method comprises: S1: In the initial state, calibrate the video image and the actual position of the cutting head, and calculate the color histogram of the cutting head area in the video image; S2: Based on the similarity of the color histogram of the cutting head area, In the frame video image, determine the current center position of the cutting head area; S3: Using deep learning network to identify The current cutting state of the cutting head in the frame video image; S4: Based on the current center position and the current cutting state, predict the center position of the cutting head area in the next frame of video image and the output power of the cutting head to obtain the next center position and the next output power; S5: Based on the next center position and the next output power, the movement and output power of the cutting head are controlled to achieve precise demolition.

2. The laser demolition dynamic optimization method based on machine vision guidance according to claim 1 is characterized in that: The step S1 comprises: S11: In the initial state, the electric lens calibrates the video image and the actual position of the cutting head by translation and tilt to ensure that the cutting head is in the video image; S12: In the video image, determine the cutting head area, and calculate the color histogram of the cutting head area: ； in, The color histogram representing the cutting head area, Indicates the color value, Indicates the color in the cutting head area The probability distribution of .

3. The laser demolition dynamic optimization method based on machine vision guidance according to claim 2 is characterized in that: The step S2 comprises: S21: In the frame video image, determine the boundary value of the cutting head candidate area: ； in, Indicates the length range of the candidate area for the cutting head, Indicates the width range of the candidate area for the cutting head. Indicates The center position of the frame video image, Indicates the size of the candidate area for the cutting head; S22: Calculate the color histogram of the candidate area of ​​the cutting head: ； in, The color histogram representing the candidate area of ​​the cutting head, Indicates the color in the candidate area of ​​the cutting head The probability distribution of S23: Measure the similarity between the color histogram of the cutting head candidate area and the color histogram of the cutting head area: ； in, represents the similarity coefficient; S24: If , it is considered that the color histogram of the cutting head candidate region is similar to the color histogram of the cutting head region, and the current center position of the cutting head region is determined as: ； in, Indicates The center position of the cutting head area in the frame video image; like , it is considered that the color histogram of the cutting head candidate region is not similar to the color histogram of the cutting head region; renew , ,in, Indicates the length offset, Indicates width offset; S25: Repeat steps S22, S23 and S24 until .

4. The laser demolition dynamic optimization method based on machine vision guidance according to claim 3 is characterized in that: The step S3 comprises: The first The frame video image is input into a trained deep learning network to identify the current cutting state of the cutting head.

5. The laser demolition dynamic optimization method based on machine vision guidance according to claim 1 is characterized in that: The step S4 comprises: S41: Based on the current center position, construct a cutting head motion state vector: ； in, Indicates The motion state vector of the cutting head in the frame video image, Representation Matrix The transposed matrix of Indicates The center position of the cutting head area in the frame video image, Indicates Output power of the cutting head in the frame video image, Indicates Offset speed of the cutting head area in the frame video image: ； in, Indicates The center position of the cutting head area in the frame video image; S42: Based on the current cutting state, predict the motion state vector of the cutting head in the next frame of video image to obtain the next center position and the next output power: ； in, Indicates The motion state vector prediction value of the cutting head in the frame video image, the state transfer matrix Related to the current cutting state, select a suitable state transfer matrix according to the current cutting state ; The first two elements of are the next center position ; The last element of is the next output power .

6. The laser demolition dynamic optimization method based on machine vision guidance according to claim 5 is characterized in that: The step S5 comprises: S51: Based on the next output power , output cutting power; S52: Based on the next center position , control the movement of the cutting head to achieve precise demolition.

7. A laser demolition dynamic optimization system based on machine vision guidance, characterized in that: include: Cutting head area color histogram module: used to calibrate the video image and the actual position of the cutting head in the initial state, and calculate the color histogram of the cutting head area in the video image; The cutting head area center position determination module is used to determine the current center position of the cutting head area in the t-th frame video image; Cutting head cutting state determination module: used to identify the current cutting state of the cutting head in the t-th frame video image based on the deep learning network; Prediction module: used for predicting the center position of the cutting head area and the output power of the cutting head in the next frame of video image based on the current center position and the current cutting state, and obtaining the next center position and the next output power; Precision demolition module: used to control the movement and output power of the cutting head based on the next center position and the next output power to achieve precision demolition; To realize a dynamic optimization method of laser demolition based on machine vision guidance as described in any one of claims 1-6.

Images