A dynamic optimization method and system for laser demolition based on machine vision guidance

Abstract

The present invention discloses a dynamic optimization method for laser demolition guided by machine vision, comprising the following steps: S1: after initial state calibration, calculating the color histogram of the cutting head area in the video image; S2: based on the similarity of the color histograms, determining the current center position of the cutting head area in the frame of the video image; S3: using a deep learning network to identify the current cutting state of the cutting head in the frame of the video image; S4: based on the current center position and the current cutting state, predicting the next center position and the next output power; S5: based on the next center position and the next output power, controlling the movement and output power of the cutting head to achieve precise demolition. The present invention uses machine vision technology to achieve effective positioning of the cutting head and effective identification of the cutting state; based on the prediction of the cutting head state, feedback control is performed on the movement and output power of the cutting head to achieve precise demolition.

Description

Technical Field

[0001] The present invention relates to the technical field of emergency rescue, and in particular to a laser demolition dynamic optimization method and system based on machine vision guidance. Background Art

[0002] Laser cutting can cut through hard materials in a short time, which is crucial for emergency rescue. Laser cutting uses a high-energy laser beam to heat and vaporize the material, thus achieving the desired effect. As a new rescue technology, laser cutting can reduce secondary damage to the surrounding environment and secondary injuries to personnel.

[0003] Laser cutting is a complex process. The material, thickness, and shape of the material being cut, as well as the output power of the cutting head, all affect the cutting process. Therefore, during the laser cutting process, using feedback control to adjust the cutting head's motion in real time is very useful for ensuring cutting accuracy, enhancing cutting stability, and improving cutting quality.

[0004] However, the existing feedback control systems in the field of laser cutting mainly use sensors to position and control the motion of the cutting head, which has technical problems such as low positioning accuracy, poor timeliness, and inability to accurately control the cutting state of the cutting head. Summary of the Invention

[0005] In view of this, the present invention is based on machine vision technology. By analyzing and processing the center position of the cutting head area and the cutting state of the cutting head, feedback control is performed on the movement and output power of the cutting head to achieve precise demolition. The technical solution proposed in this invention is a dynamic optimization method for laser demolition based on machine vision guidance, which includes the following steps:

[0006] S1: In the initial state, calibrate the video image and the actual position of the cutting head, and calculate the color histogram of the cutting head area in the video image;

[0007] S2: Based on the similarity of the color histogram of the cutting head area, In the frame video image, determine the current center position of the cutting head area;

[0008] S3: Using deep learning network to identify The current cutting status of the cutting head in the frame video image;

[0009] S4: Based on the current center position and the current cutting state, predict the center position of the cutting head area and the output power of the cutting head in the next frame of video image to obtain the next center position and the next output power;

[0010] S5: Based on the next center position and the next output power, the movement and output power of the cutting head are controlled to achieve precise demolition.

[0011] Optionally, in step S1, in the initial state, calibrating the video image and the actual position of the cutting head, and calculating the color histogram of the cutting head area in the video image includes:

[0012] S11: In the initial state, the motorized lens calibrates the video image and the actual position of the cutting head by translation and tilting to ensure that the cutting head is in the video image;

[0013] S12: In the video image, determine the cutting head area and calculate the color histogram of the cutting head area:

[0014] ;

[0015] in, Represents the color histogram of the cutting head area, Represents the color value, Indicates the color in the cutting head area The probability distribution of .

[0016] Optionally, in step S2, based on the similarity of the color histogram of the cutting head area, In the frame video image, determine the current center position of the cutting head area, including:

[0017] S21: In the frame video image, determine the boundary value of the candidate area of ​​the cutting head:

[0018] ;

[0019] in, Indicates the length range of the candidate area for the cutting head, Indicates the width range of the candidate area of ​​the cutting head, Indicates the The center position of the frame video image, Indicates the size of the candidate area for the cutting head;

[0020] S22: Calculate the color histogram of the candidate area of ​​the cutting head:

[0021] ;

[0022] in, The color histogram representing the candidate area of ​​the cutting head, Indicates the color in the candidate area of ​​the cutting head The probability distribution of

[0023] S23: Measure the similarity between the color histogram of the candidate cutting head region and the color histogram of the cutting head region:

[0024] ;

[0025] in, represents the similarity coefficient;

[0026] S24: If , it is considered that the color histogram of the cutting head candidate region is similar to the color histogram of the cutting head region, and the current center position of the cutting head region is determined as:

[0027] ;

[0028] in, Indicates the The center position of the cutting head area in the frame video image;

[0029] like ,It is believed that the color histogram of the cutting head candidate region is not similar to the color histogram of the cutting head region;

[0030] renew , ,in, Indicates the length offset, Indicates width offset;

[0031] S25: Repeat steps S22, S23 and S24 until .

[0032] Optionally, in step S3, a deep learning network is used to identify the The current cutting status of the cutting head in the frame video image, including:

[0033] The first Frame video images are input into a trained deep learning network to identify the current cutting state of the cutting head;

[0034] Cutting status includes: normal cutting, perforation, overheat cutting, underburn cutting, jitter cutting, and offset cutting;

[0035] Among them, in the normal cutting state, the position of the cutting head changes according to the predetermined route; in the perforation state, the cutting head heats and perforates the material at the current position, and the position of the cutting head remains unchanged; in the overheating cutting and underburning cutting states, the cutting speed of the cutting head will slow down, and the position of the cutting head will slowly change according to the predetermined route; in the shaking cutting state, the cutting head will shake unstably, and the position of the cutting head will deviate slightly relative to the predetermined route; in the offset cutting state, the cutting offset is inaccurate due to the system and material movement, and the cutting head position will deviate greatly relative to the predetermined route.

[0036] Optionally, in step S4, based on the current center position and the current cutting state, predicting the center position of the cutting head area and the output power of the cutting head in the next frame of video image to obtain the next center position and the next output power includes:

[0037] S41: Based on the current center position, construct a cutting head motion state vector:

[0038] ;

[0039] in, Indicates the The motion state vector of the cutting head in the frame video image, Representation matrix

[0040] The transposed matrix of Indicates the The center position of the cutting head area in the frame video image, Indicates the Output power of the cutting head in the frame video image, Indicates the Offset speed of the cutting head area in the frame video image:

[0041] ;

[0042] in, Indicates the The center position of the cutting head area in the frame video image;

[0043] S42: Based on the current cutting state, predict the motion state vector of the cutting head in the next frame of video image to obtain the next center position and the next output power:

[0044] ;

[0045] in, Indicates the The motion state vector prediction value of the cutting head in the frame video image, the state transfer matrix Related to the current cutting state, select the appropriate state transfer matrix according to the current cutting state ;

[0046] The first two elements of are the next center position ;

[0047] The last element is the next output power ;

[0048] In the normal cutting state, the cutting head moves at a constant speed along the predetermined route, and the state transfer matrix is:

[0049] ;

[0050] In the perforation state, the cutting head heats and perforates the material at the current position. The position of the cutting head remains unchanged. The state transfer matrix is:

[0051] ;

[0052] Under overheat cutting and underburn cutting conditions, the state transfer matrix is:

[0053] ;

[0054] In the jitter cutting state, the cutting speed of the cutting head will slow down, the cutting head will decelerate along the predetermined route, and the position of the cutting head will change slowly along the predetermined route; the state transfer matrix:

[0055] ;

[0056] in, Indicates that the cutting head is The velocity coefficient in the direction, Indicates that the cutting head is Speed ​​coefficient of direction;

[0057] In the offset cutting state, due to system errors and material movement, the cutting offset is inaccurate, and the position of the cutting head deviates greatly from the predetermined path. The state transfer matrix is:

[0058] ;

[0059] in, Indicates that the cutting head is The velocity deviation coefficient in the direction, Indicates that the cutting head is The speed offset factor in the direction.

[0060] Optionally, in step S5, based on the next center position and the next output power, controlling the movement and output power of the cutting head to achieve precise demolition includes:

[0061] S51: Based on the next output power , output cutting power;

[0062] S52: Based on the next center position , control the movement of the cutting head to achieve precise demolition.

[0063] The present invention also provides a laser demolition dynamic optimization system based on machine vision guidance, comprising:

[0064] Cutting head area color histogram module: used to calibrate the video image and the actual position of the cutting head in the initial state, and calculate the color histogram of the cutting head area in the video image;

[0065] Cutting head area center position determination module: used to determine the current center position of the cutting head area in the t-th frame video image;

[0066] Cutting head cutting state determination module: used to identify the current cutting state of the cutting head in the t-th frame video image based on the deep learning network;

[0067] Prediction module: used for predicting the center position of the cutting head area and the output power of the cutting head in the next frame of video image based on the current center position and the current cutting state, and obtaining the next center position and the next output power;

[0068] Precision demolition module: used to control the movement and output power of the cutting head based on the next center position and the next output power to achieve precision demolition.

[0069] Beneficial effects

[0070] The present invention adopts machine vision technology and realizes effective positioning of the cutting head in the video image based on the color histogram of the cutting head area, thereby improving the efficiency and accuracy of cutting head tracking; based on tracking the cutting head, the cutting head state is effectively identified; based on different cutting head states, the center position of the cutting head area in the next frame of video image and the output power of the cutting head are accurately predicted; and precise demolition is achieved through feedback control of the movement and output power of the cutting head.

[0071] The present invention makes full use of the morphological features of the cutting head area in the video image to measure the similarity between the color histogram of the candidate cutting head area and the color histogram of the cutting head area; and combines the area offset to accurately obtain the center position of the cutting head area; based on the center position of the cutting head area, the offset speed and the output power of the cutting head, the motion characteristics of the cutting head are fully considered to construct the motion state vector of the cutting head; based on the cutting state of the cutting head, the state transfer matrix is ​​selected to realize the prediction of the motion state vector of the cutting head. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 A schematic flow chart of a dynamic optimization method for laser demolition based on machine vision guidance provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0073] The present invention will be further described below with reference to the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention fall within the scope of protection of the present invention.

[0074] Example 1:

[0075] A dynamic optimization method for laser demolition based on machine vision guidance, such as Figure 1 As shown, the following steps are included:

[0076] S1: In the initial state, calibrate the video image and the actual position of the cutting head, and calculate the color histogram of the cutting head area in the video image;

[0077] As you can understand, calibrating the video image aligns the displayed position of the cutting head with its actual physical location, ensuring that the cutting head appears accurately in the designated area of ​​the video image. Next, the color distribution of the pixels in this area is identified, and a color histogram of the cutting head area is calculated and generated (a color histogram is a histogram used to reflect the distribution of image color components). This provides the color feature data foundation for subsequent image matching and tracking.

[0078] Optionally, in step S1, in the initial state, calibrating the video image and the actual position of the cutting head, and calculating the color histogram of the cutting head area in the video image includes:

[0079] S11: In the initial state, the motorized lens calibrates the video image and the actual position of the cutting head by translation and tilting to ensure that the cutting head is in the video image;

[0080] S12: In the video image, determine the cutting head area and calculate the color histogram of the cutting head area:

[0081] ;

[0082] in, Represents the color histogram of the cutting head area, Represents the color value, Indicates the color in the cutting head area The probability distribution of .

[0083] S2: Based on the similarity of the color histogram of the cutting head area, In the frame video image, determine the current center position of the cutting head area;

[0084] It can be understood that by comparing the color histogram of each area in each frame of the video image with the color histogram of the cutting head area, the area with the most similar color features can be found, thereby determining the area where the cutting head is located in the current frame, and calculating the center position of the cutting head area to obtain the current center position of the cutting head area.

[0085] Optionally, in step S2, based on the similarity of the color histogram of the cutting head area, In the frame video image, determine the current center position of the cutting head area, including:

[0086] S21: In the frame video image, determine the boundary value of the candidate area of ​​the cutting head:

[0087] ;

[0088] in, Indicates the length range of the candidate area for the cutting head, Indicates the width range of the candidate area of ​​the cutting head, Indicates the The center position of the frame video image, Indicates the size of the candidate area for the cutting head;

[0089] S22: Calculate the color histogram of the candidate area of ​​the cutting head:

[0090] ;

[0091] in, The color histogram representing the candidate area of ​​the cutting head, Indicates the color in the candidate area of ​​the cutting head The probability distribution of

[0092] S23: Measure the similarity between the color histogram of the candidate cutting head region and the color histogram of the cutting head region:

[0093] ;

[0094] in, represents the similarity coefficient;

[0095] S24: If , it is considered that the color histogram of the cutting head candidate region is similar to the color histogram of the cutting head region, and the current center position of the cutting head region is determined as:

[0096] ;

[0097] in, Indicates the The center position of the cutting head area in the frame video image;

[0098] like ,It is believed that the color histogram of the cutting head candidate region is not similar to the color histogram of the cutting head region;

[0099] renew , ,in, Indicates the length offset, Indicates width offset;

[0100] S25: Repeat steps S22, S23 and S24 until .

[0101] S3: Using deep learning network to identify The current cutting status of the cutting head in the frame video image;

[0102] Optionally, in step S3, a deep learning network is used to identify the The current cutting status of the cutting head in the frame video image, including:

[0103] The first Frame video images are input into a trained deep learning network to identify the current cutting state of the cutting head;

[0104] In an embodiment of the present invention, the cutting states include: normal cutting, perforation, overheating cutting, underfire cutting, jitter cutting, and offset cutting; wherein, in the normal cutting state, the position of the cutting head changes according to a predetermined route; in the perforation state, the cutting head heats and perforates the material at the current position, and the position of the cutting head remains unchanged; in the overheating cutting and underfire cutting states, the cutting speed of the cutting head will slow down, and the position of the cutting head will slowly change according to a predetermined route; in the jitter cutting state, the cutting head will experience unstable jitter, and the position of the cutting head will have a small offset relative to the predetermined route; in the offset cutting state, the cutting offset is inaccurate due to the system being out of position and the movement of the material, and the position of the cutting head will have a large offset relative to the predetermined route.

[0105] S4: Based on the current center position and the current cutting state, predict the center position of the cutting head area and the output power of the cutting head in the next frame of video image to obtain the next center position and the next output power;

[0106] Optionally, in step S4, based on the current center position and the current cutting state, predicting the center position of the cutting head area and the output power of the cutting head in the next frame of video image to obtain the next center position and the next output power includes:

[0107] S41: Based on the current center position, construct a cutting head motion state vector:

[0108] ;

[0109] in, Indicates the The motion state vector of the cutting head in the frame video image, Representation matrix

[0110] The transposed matrix of Indicates the The center position of the cutting head area in the frame video image, Indicates the Output power of the cutting head in the frame video image, Indicates the Offset speed of the cutting head area in the frame video image:

[0111] ;

[0112] in, Indicates the The center position of the cutting head area in the frame video image;

[0113] S42: Based on the current cutting state, predict the motion state vector of the cutting head in the next frame of video image to obtain the next center position and the next output power:

[0114] ;

[0115] in, Indicates the The motion state vector prediction value of the cutting head in the frame video image, the state transfer matrix Related to the current cutting state, select the appropriate state transfer matrix according to the current cutting state ;

[0116] The first two elements of are the next center position ;

[0117] The last element is the next output power ;

[0118] In the embodiment of the present invention, in the normal cutting state, the cutting head moves at a constant speed along a predetermined route, and the state transition matrix is:

[0119] ;

[0120] In the perforation state, the cutting head heats and perforates the material at the current position. The position of the cutting head remains unchanged. The state transfer matrix is:

[0121] ;

[0122] Under overheat cutting and underburn cutting conditions, the state transfer matrix is:

[0123] ;

[0124] In the jitter cutting state, the cutting speed of the cutting head will slow down, the cutting head will decelerate along the predetermined route, and the position of the cutting head will change slowly along the predetermined route; the state transfer matrix:

[0125] ;

[0126] in, Indicates that the cutting head is The velocity coefficient in the direction, Indicates that the cutting head is Speed ​​coefficient of direction;

[0127] In the offset cutting state, due to system errors and material movement, the cutting offset is inaccurate, and the position of the cutting head deviates greatly from the predetermined path. The state transfer matrix is:

[0128] ;

[0129] in, Indicates that the cutting head is The velocity deviation coefficient in the direction, Indicates that the cutting head is Velocity deviation coefficient of direction;

[0130] It can be understood that the state transfer matrix is ​​selected through the current cutting state; based on the current center position and the current cutting state, the motion state vector of the cutting head in the next frame of video image is predicted through the state transfer matrix to obtain the next center position and the next output power.

[0131] S5: Based on the next center position and the next output power, controlling the movement and output power of the cutting head to achieve precise demolition;

[0132] Optionally, in step S5, based on the next center position and the next output power, controlling the movement and output power of the cutting head to achieve precise demolition includes:

[0133] S51: Based on the next output power , output cutting power;

[0134] S52: Based on the next center position , control the movement of the cutting head to achieve precise demolition.

[0135] Example 2: The present invention also provides a laser demolition dynamic optimization system based on machine vision guidance, which includes the following five modules:

[0136] Cutting head area color histogram module: used to calibrate the video image and the actual position of the cutting head in the initial state, and calculate the color histogram of the cutting head area in the video image;

[0137] Cutting head area center position determination module: used in the In the frame video image, determine the current center position of the cutting head area;

[0138] Cutting head cutting state determination module: used to identify the cutting state based on deep learning network The current cutting status of the cutting head in the frame video image;

[0139] Prediction module: used for predicting the center position of the cutting head area and the output power of the cutting head in the next frame of video image based on the current center position and the current cutting state, and obtaining the next center position and the next output power;

[0140] Precision demolition module: used to control the movement and output power of the cutting head based on the next center position and the next output power to achieve precision demolition.

[0141] The specific definition of a machine vision-guided laser demolition dynamic optimization system can be found in the definition of a machine vision-guided laser demolition dynamic optimization method described above and will not be repeated here. Each module in the aforementioned machine vision-guided laser demolition dynamic optimization system can be implemented in whole or in part through software, hardware, or a combination thereof. Each of these modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each of these modules.

[0142] It should be noted that the serial numbers of the above-mentioned embodiments of the present invention are for descriptive purposes only and do not represent the advantages or disadvantages of the embodiments. In addition, the terms "including", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method comprising the element.

[0143] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0144] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A dynamic optimization method for laser demolition based on machine vision guidance, characterized in that: The method comprises: S1: In the initial state, calibrate the video image and the actual position of the cutting head, and calculate the color histogram of the cutting head area in the video image; the step S1 includes: S11: In the initial state, the motorized lens calibrates the video image and the actual position of the cutting head by translation and tilting to ensure that the cutting head is in the video image; S12: In the video image, determine the cutting head area and calculate the color histogram of the cutting head area: ； in, Represents the color histogram of the cutting head area, Represents the color value, Indicates the color in the cutting head area The probability distribution of S2: Based on the similarity of the color histogram of the cutting head area, In the frame video image, the current center position of the cutting head area is determined; the step S2 includes: S21: In the frame video image, determine the boundary value of the candidate area of ​​the cutting head: ； in, Indicates the length range of the candidate area for the cutting head, Indicates the width range of the candidate area of ​​the cutting head, Indicates the The center position of the frame video image, Indicates the size of the candidate area for the cutting head; S22: Calculate the color histogram of the candidate area of ​​the cutting head: ； in, The color histogram representing the candidate area of ​​the cutting head, Indicates the color in the candidate area of ​​the cutting head The probability distribution of S23: Measure the similarity between the color histogram of the candidate cutting head region and the color histogram of the cutting head region: ； in, represents the similarity coefficient; S24: If , it is considered that the color histogram of the cutting head candidate region is similar to the color histogram of the cutting head region, and the current center position of the cutting head region is determined as: ； in, Indicates the The center position of the cutting head area in the frame video image; like ,It is believed that the color histogram of the cutting head candidate region is not similar to the color histogram of the cutting head region; renew , ,in, Indicates the length offset, Indicates width offset; S25: Repeat steps S22, S23 and S24 until ; S3: Using deep learning network to identify The current cutting status of the cutting head in the frame video image; S4: Based on the current center position and the current cutting state, predict the center position of the cutting head area and the output power of the cutting head in the next frame of video image to obtain the next center position and the next output power; S5: Based on the next center position and the next output power, the movement and output power of the cutting head are controlled to achieve precise demolition.

2. The laser demolition dynamic optimization method based on machine vision guidance according to claim 1 is characterized in that: The step S3 comprises: The first The frame video image is input into the trained deep learning network to identify the current cutting state of the cutting head.

3. The laser demolition dynamic optimization method based on machine vision guidance according to claim 1 is characterized in that: The step S4 comprises: S41: Based on the current center position, construct a cutting head motion state vector: ； in, Indicates the The motion state vector of the cutting head in the frame video image, Representation matrix The transposed matrix of Indicates the The center position of the cutting head area in the frame video image, Indicates the Output power of the cutting head in the frame video image, Indicates the Offset speed of the cutting head area in the frame video image: ； in, Indicates the The center position of the cutting head area in the frame video image; S42: Based on the current cutting state, predict the motion state vector of the cutting head in the next frame of video image to obtain the next center position and the next output power: ； in, Indicates the The motion state vector prediction value of the cutting head in the frame video image, the state transfer matrix Related to the current cutting state, select the appropriate state transfer matrix according to the current cutting state ; The first two elements of are the next center position ; The last element is the next output power .

4. The laser demolition dynamic optimization method based on machine vision guidance according to claim 3 is characterized in that: The step S5 comprises: S51: Based on the next output power , output cutting power; S52: Based on the next center position , control the movement of the cutting head to achieve precise demolition.

5. A laser demolition dynamic optimization system based on machine vision guidance, characterized in that: include: Cutting head area color histogram module: used to calibrate the video image and the actual position of the cutting head in the initial state, and calculate the color histogram of the cutting head area in the video image; Cutting head area center position determination module: used to determine the current center position of the cutting head area in the t-th frame video image; Cutting head cutting state determination module: used to identify the current cutting state of the cutting head in the t-th frame video image based on the deep learning network; Prediction module: used for predicting the center position of the cutting head area and the output power of the cutting head in the next frame of video image based on the current center position and the current cutting state, and obtaining the next center position and the next output power; Precision demolition module: used to control the movement and output power of the cutting head based on the next center position and the next output power to achieve precise demolition; To realize a dynamic optimization method of laser demolition based on machine vision guidance as described in any one of claims 1-4.

Images