Laser equipment target state evaluation method, system, equipment and medium

Abstract

The invention relates to the technical field of target detection, and discloses a laser equipment target state evaluation method and system, equipment and a medium. The method comprises the following steps: acquiring a video sequence of a moving target in a moving process by adopting laser equipment; obtaining the category of a moving target in each frame of image of the video sequence and a detection frame used for indicating the position and the size of the moving target in the image through a target detection model; separating the background part and the foreground part in the video sequence by using the low-rank characteristic of the background part in the multi-frame image of the video sequence and the sparse characteristic of the foreground part corresponding to the moving target; in combination with the attribute feature of the moving target obtained through the target detection model and the attribute feature of the moving target obtained through segmenting the background part and the foreground part of the image, the state of the moving target is evaluated, and the evaluation effect is better.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a state assessment method, system, device and medium for a laser device target. Background Art

[0002] As an advanced optical system, laser equipment is widely used in industrial processing, communication, medical treatment and scientific research due to its high precision, fast response and energy controllability. However, in practical applications, laser equipment faces the challenge of target state assessment. Complex environmental factors, such as background clutter, illumination changes, small target size and fast speed, increase the difficulty of target state assessment.

[0003] In laser equipment applications, target status assessment refers to a comprehensive analysis and judgment of the real-time status of the target being irradiated or processed by the laser. This assessment involves dynamic characteristics such as the position and speed of the target, as well as static characteristics such as material integrity and surface condition. Accurate status assessment is crucial for adjusting laser power, optimizing processing paths, and ensuring processing quality and equipment safety.

[0004] For the state assessment of the target, traditional multi-attribute decision-making methods, such as the analytic hierarchy process (AHP) and grey relational analysis, have limitations when facing rapidly changing dynamic targets. They rely too much on static features and ignore the motion information of the target, which may lead to distorted assessment results. Summary of the invention

[0005] The object of the present invention is to provide a method, system, device and medium for evaluating the state of a laser device target, which can realize effective evaluation of the target state.

[0006] To solve the above technical problems, an embodiment of the present invention provides a state assessment method for a laser device target, comprising the following steps:

[0007] Use laser equipment to obtain video sequences of moving targets during movement;

[0008] Input each frame of the video sequence into the trained target detection model to obtain the category of the moving target in each frame and the detection frame used to indicate the position and size of the moving target in the image;

[0009] By utilizing the low-rank characteristics of the background part and the sparse characteristics of the foreground part corresponding to the moving target in the multi-frame images of the video sequence, the background part and the foreground part in the video sequence are separated;

[0010] According to the change of the detection frame position of the continuous frame images in the video sequence, the moving speed and moving direction of the moving target are obtained, and the moving speed, moving direction, category and size of the moving target are used as the first attribute features of the moving target;

[0011] According to the foreground part separated from the video sequence and the size of the detection frame, the shape feature and color feature of the moving target are obtained as the second attribute feature of the moving target;

[0012] The state of the moving target is evaluated by combining the first attribute feature and the second attribute feature of the moving target.

[0013] Optionally, the network structure of the target detection model uses YOLOX-Tiny as the backbone network, YOLOX-Tiny includes a feature extraction layer and a detection head, and multiple convolution blocks of the feature extraction layer in YOLOX-Tiny are replaced by SwinTransformer modules;

[0014] The step of inputting each frame of the video sequence into the trained target detection model to obtain the category of the moving target in each frame and the detection frame used to indicate the position and size of the moving target in the image includes:

[0015] For each frame of image, the feature extraction layer divides the image into multiple non-overlapping small blocks, each small block is converted into a vector of fixed length through linear mapping, and multiple Swin Transformer modules in the feature extraction layer extract features for each vector; for each vector, the extracted features are divided into multiple non-overlapping windows, and the self-attention mechanism is applied in each window to extract the local features of each window; the shift window strategy is adopted to change the division method of multiple non-overlapping windows between adjacent Swin Transformer modules to transfer information across windows and obtain multi-level features of the image;

[0016] The detection head adopts YOLOX-Tiny's decoupling head design and sample matching strategy to detect the multi-level features of each frame image, and obtain the category of the moving target in each frame image and the detection frame used to indicate the position and size of the moving target in the image.

[0017] Optionally, utilizing the low-rank characteristic of the background part and the sparse characteristic of the foreground part corresponding to the moving target in the multi-frame images of the video sequence to separate the background part from the foreground part in the video sequence includes:

[0018] Convert the video sequence into a three-dimensional tensor D, and the three-dimensional tensor D includes a background part B and a foreground part F corresponding to the moving target;

[0019] Using a non-convex rank approximation algorithm, by minimizing the objective function Separate the background part B and the foreground part F in the video sequence;

[0020] Among them, rank(B) represents the rank constraint of the background part B, ||F||_0 represents the L0 norm constraint of the foreground part F, Represents the three-dimensional total variation constraint of the foreground part F.

[0021] Optionally, the rank constraint of the non-convex rank approximation algorithm adopts Schatten-p norm ||B||Sp=(Σ_iσ_i^p)^(1 / p);

[0022] The non-convex rank approximation algorithm is used to minimize the objective function Separating the background part B and the foreground part F in the video sequence includes:

[0023] The objective function Refactoring into objective function Among them, σ_i is the singular value of B, 0 <p≤1;

[0024] Combining the augmented Lagrangian method and the alternating direction multiplier method, by introducing auxiliary variables and Lagrangian multipliers, the minimization problem of the reconstructed objective function is decomposed into multiple sub-problems, and the background part B, the foreground part F and the auxiliary variables are optimized alternately. After reaching the preset number of iterations, the separated background part B and foreground part F are obtained.

[0025] In each iteration, singular value decomposition is performed on the background part B, and the singular value is updated through a soft threshold operation; for the foreground part F, a hard threshold operation is used to process the L0 norm constraint, and a soft threshold operation is used to process the three-dimensional total variation constraint.

[0026] Optionally, the combining the first attribute feature and the second attribute feature of the moving target to perform a state assessment on the moving target includes:

[0027] Based on the multi-attribute decision-making TOPSIS algorithm, the decision matrix is ​​constructed using the first attribute characteristics and the second attribute characteristics of the moving target, and different weights are set for different first attribute characteristics and second attribute characteristics according to the degree of influence of different attribute characteristics on the state of the moving target;

[0028] According to the decision matrix and the set weights, the positive ideal solution and the negative ideal solution are calculated, and the weighted Euclidean distance between the moving target and the positive ideal solution and the negative ideal solution is calculated;

[0029] According to the calculated weighted Euclidean distance, the relative closeness of the moving target is obtained;

[0030] The state of the moving target is evaluated by the relative proximity of the moving target.

[0031] Optionally, the performing state assessment on the moving target based on the relative proximity of the moving target includes:

[0032] If the relative closeness of the moving target is less than the first preset threshold, the moving target is in a normal state;

[0033] If the relative proximity of the moving target is greater than or equal to a first preset threshold and less than or equal to a second preset threshold, the moving target is in a warning state;

[0034] If the relative proximity of the moving target is greater than a second preset threshold, the moving target is in a dangerous state.

[0035] Optionally, acquiring shape features of the moving target according to the foreground portion separated from the video sequence and the size of the detection frame includes:

[0036] The total number of pixels in the foreground is taken as the area of ​​the foreground, the number of pixels at the boundary is taken as the perimeter of the foreground, and the circularity of the foreground is calculated using the formula 4π*area / perimeter^2 to describe the regularity of the shape of the foreground.

[0037] According to the height and / or width of the detection frame, the aspect ratio of the moving target is obtained, and according to the total number of pixels in the foreground part and the area of ​​the detection frame, the filling rate of the moving target is obtained;

[0038] The circularity of the foreground part, the aspect ratio and the filling rate of the moving object are used as the shape features of the moving object.

[0039] An embodiment of the present invention further provides a state assessment system for a laser device target, comprising:

[0040] An image acquisition module is used to acquire a video sequence of a moving target during its motion using a laser device;

[0041] The target detection module is used to input each frame of the video sequence into the trained target detection model to obtain the category of the moving target in each frame and the detection frame used to indicate the position and size of the moving target in the image;

[0042] An image segmentation module is used to separate the background part from the foreground part in the video sequence by utilizing the low-rank characteristics of the background part and the sparse characteristics of the foreground part corresponding to the moving target in multiple frames of the video sequence;

[0043] A feature extraction module is used to obtain the moving speed and moving direction of the moving target according to the change of the detection frame position of the continuous frame images in the video sequence, and use the moving speed, moving direction, category and size of the moving target as the first attribute feature of the moving target;

[0044] According to the foreground part separated from the video sequence and the size of the detection frame, the shape feature and color feature of the moving target are obtained as the second attribute feature of the moving target;

[0045] The state evaluation module is used to evaluate the state of the moving target by combining the first attribute feature and the second attribute feature of the moving target.

[0046] An embodiment of the present invention also provides a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned laser device target state assessment method.

[0047] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the above-mentioned method for evaluating the state of a laser device target.

[0048] The state assessment method of a laser device target provided by the present invention has at least the following beneficial effects:

[0049] After obtaining a video sequence of a moving target in motion through a laser device, first, a target detection model is used to detect the moving target in each frame of the video sequence, and the category of the moving target and the detection frame used to indicate the position and size of the moving target in the image are obtained; then, the low-rank characteristics of the background part in multiple frames of the video sequence and the sparse characteristics of the foreground part corresponding to the moving target are utilized to separate the background part from the foreground part in the video sequence, and the foreground part (i.e., the moving target) in the video sequence is obtained; finally, the information of the moving target obtained by these two methods is combined to evaluate the state of the moving target, and more comprehensive properties of the moving target are extracted, which provides a rich data basis for accurately evaluating the state of the moving target and improves the accuracy of the state evaluation of the moving target.

[0050] Moreover, the movement speed and direction of the moving target, that is, the dynamic characteristics of the moving target, can be obtained through the changes in the detection box position of consecutive frame images in the video sequence, while the category, size, shape characteristics and color characteristics of the moving target are the static characteristics of the moving target. Taking into account the dynamic changes of the moving target, the accuracy of the moving target state assessment is further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] One or more embodiments are exemplarily described by the pictures in the corresponding drawings, and these exemplary descriptions do not constitute limitations on the embodiments.

[0052] Figure 1 is a flow chart of a method for evaluating a state of a laser device target provided according to an embodiment of the present invention;

[0053] Figure 2 is a processing flow chart of a target detection model provided according to an embodiment of the present invention;

[0054] Figure 3 It is a schematic diagram of a state assessment of a laser device target provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the embodiments of the present invention, many technical details are proposed in order to enable the reader to better understand the present invention. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present invention can be implemented. The division of the following embodiments is for the convenience of description and should not constitute any limitation on the specific implementation of the present invention. The various embodiments can be combined and referenced with each other without contradiction.

[0056] First, the status assessment of laser equipment targets in related technologies is introduced:

[0057] In terms of target detection:

[0058] Deep learning technology, especially convolutional neural networks (CNN), has made significant progress in the field of object detection. Single-stage object detection algorithms, such as the YOLO series (YOLOv3, YOLOv4, YOLOX, etc.), have attracted widespread attention for their fast and efficient detection performance. However, the detection accuracy of these models for small targets in complex backgrounds still needs to be improved, and it is difficult to fully meet the needs of real-time applications.

[0059] Visual Transformers have also made breakthrough progress in the field of object detection. For example, applying Transformers to object detection has achieved end-to-end detection. And Swin Transformer has greatly improved computational efficiency by introducing a sliding window attention mechanism. Models such as PVTv2 have further optimized the structure of Transformers and achieved state-of-the-art performance on multiple visual tasks. However, these models generally have problems with large number of parameters and high computational complexity, making them difficult to run in real time on resource-constrained devices.

[0060] In the segmentation of moving objects

[0061] Deep learning-based methods, such as achieving end-to-end video object segmentation through spatiotemporal convolutional networks, or introducing multi-object tracking mechanisms to improve the temporal consistency of segmentation. However, these methods usually require a lot of computing resources and are difficult to meet real-time requirements.

[0062] In terms of target status assessment:

[0063] Traditional multi-attribute decision-making methods such as the Analytic Hierarchy Process (AHP) and grey relational analysis have limitations when dealing with dynamically changing target attributes. These methods usually assume that the attribute weights are fixed and are difficult to adapt to real-time changing scenarios. In addition, they mainly focus on static features and ignore the motion information of the target, which may lead to distorted evaluation results.

[0064] An embodiment of the present invention relates to a state assessment method for a laser device target. The implementation details of the state assessment method for a laser device target of this embodiment are specifically described below. The following content is only the implementation details provided for easy understanding and is not necessary for implementing this solution.

[0065] The specific process of the state assessment method of the laser device target in this embodiment is as follows: Figure 1 As shown, including:

[0066] Step 101: Use a laser device to acquire a video sequence of a moving target during its movement.

[0067] Step 102, input each frame of the video sequence into the trained target detection model to obtain the category of the moving target in each frame of the image and the detection frame used to indicate the position and size of the moving target in the image.

[0068] Specifically, the target detection model (LVT-YOLOX) of this embodiment has a network structure with YOLOX-Tiny as the backbone network. YOLOX-Tiny includes a feature extraction layer and a detection head, and multiple convolution blocks of the feature extraction layer in YOLOX-Tiny are replaced by Swin Transformer modules. That is, the LVT-YOLOX model is based on the overall architecture of YOLOX-Tiny and integrates Swin Transformer to form a lightweight model suitable for laser device target detection.

[0069] For each frame in the video sequence, Figure 2 The process shown realizes target detection: the feature extraction layer of YOLOX-Tiny divides the image into multiple non-overlapping small patches, each of which is converted into a fixed-length vector patchtokens through linear mapping processing, and the multiple SwinTransformer modules that replace the convolutional blocks in the feature extraction layer of YOLOX-Tiny perform feature extraction of different scales on multiple vectors through self-attention mechanism and cross-window information transmission to extract multi-level features of the image. Among them, when the multiple Swin Transformer modules perform feature extraction of different scales on multiple vectors through self-attention mechanism and cross-window information transmission, first, multiple Swin Transformer modules perform feature extraction on each vector, and for each vector, the extracted features are divided into multiple non-overlapping windows with a window size of 4x4 or 8x8, and the self-attention mechanism is applied in each window, and the attention weight is generated by calculating the similarity between the query, key and value to realize the effective extraction of local features; then, the shift window strategy is adopted to change the division method of multiple non-overlapping windows between adjacent Swin Transformer modules to perform cross-window information transmission to obtain multi-level features of the image.

[0070] Finally, the model retains the decoupling head design and sample matching strategy of YOLOX-Tiny. The detection head uses the decoupling head design and sample matching strategy to detect multi-level features, obtain the category of the moving target in the image, and the detection box used to indicate the position and size of the moving target in the image, which can ensure the efficiency and accuracy of the detection. Among them, the decoupling head design decomposes the target detection task into subtasks such as category prediction, bounding box regression, and target prediction. Each subtask is processed using an independent convolutional layer, and a dynamic k matching algorithm is used for sample matching to calculate the quality score of each real target and all prediction boxes; the sample matching strategy is based on the combination of IoU and category confidence, and the first k prediction boxes with the highest quality scores are selected as positive samples. The k value is adaptively adjusted according to the target size and complexity, and the remaining prediction boxes are regarded as negative samples or ignored. The matching results are used to guide the loss function calculation and optimize the model parameters; finally, the category of the moving target in the image and the detection box used to indicate the position and size of the moving target in the image are obtained.

[0071] In addition, in order to achieve lightweight design, the model reduces the number of layers of the SwinTransformer block compared to the original SwinTransformer, reduces the number of layers of the Transformer block from the original 4 layers to 2 layers, and adopts a smaller attention calculation window, which effectively reduces the overall parameter amount and computational complexity. In addition, the model introduces spatial attention mechanism and channel attention mechanism on the feature map of each scale, which enhances the attention to the target and the ability to express features. The spatial attention mechanism is mainly used to highlight the features of the target and strengthen the attention to the target. It identifies and highlights important areas through the spatial information on the feature map, so that the model can detect the target more accurately. The channel attention mechanism is used to improve the expression of effective features. By strengthening the feature information of important channels, the model can more accurately distinguish between the target and the background, and improve the stability and accuracy of detection.

[0072] Through the design of the above network structure, the LVT-YOLOX model inherits the lightweight and efficient characteristics of YOLOX-Tiny while obtaining the powerful feature extraction capability of Swin Transformer. It can realize efficient and accurate detection of small targets in complex backgrounds while meeting the needs of real-time detection.

[0073] In one example, in order to enhance the model's ability to learn difficult samples, the fuzzy loss function FocalLoss was introduced. By adjusting specific parameters, the model pays more attention to samples that are difficult to classify. This improvement enables LVT-YOLOX to improve the accuracy of small target detection in complex backgrounds while maintaining high-speed detection.

[0074] Step 103: Utilize the low-rank property of the background part and the sparse property of the foreground part corresponding to the moving object in multiple frames of the video sequence to separate the background part and the foreground part in the video sequence.

[0075] Specifically, first convert the video sequence into a three-dimensional tensor D, and the three-dimensional tensor D includes the background part B and the foreground part F corresponding to the moving object. For each frame of the image, arrange its pixel values according to the height and width, and then stack consecutive t frames to form D ∈ R^(m×n×t), where m and n respectively represent the height and width of the image, and t represents the time dimension.

[0076] Then, adopt a non-convex rank approximation algorithm to separate the background part B and the foreground part F in the video sequence by minimizing the objective function where rank(B) represents the rank constraint of the background part B, and ||F||_0 represents the L0 norm constraint (sparsity constraint) of the foreground part F, represents the three-dimensional total variation constraint (spatial-temporal smoothness constraint) of the foreground part F.

[0077] In the specific implementation, to better approximate the actual rank function and reduce the computational complexity, the non-convex rank approximation algorithm uses the Schatten-p norm ||B||Sp = (Σ_iσ_i^p)^(1 / p) to replace the traditional nuclear norm (i.e., the rank constraint), where σ_i is the singular value of B and 0 < p ≤ 1. At this time, the objective function can be reconstructed as the objective function s.t. D = B + F. Combining the augmented Lagrangian method and the alternating direction method of multipliers, by introducing auxiliary variables and Lagrange multipliers, the minimization problem of the reconstructed objective function is decomposed into multiple sub-problems, alternately optimizing the background part B, the foreground part F, and the auxiliary variables, and after reaching the preset number of iterations, the separated background part B and foreground part F are obtained. Among them, in each iteration, perform singular value decomposition on the background part B and update the singular values through soft thresholding operations; for the foreground part F, use hard thresholding operations to handle the L0 norm constraint and soft thresholding operations to handle the three-dimensional total variation constraint to achieve efficient solution.

[0078] Among them, the original problem (i.e., the original objective function) aims to decompose the video sequence D into a low-rank background B and a sparse foreground F, expressed as s.t. D = B + F. The term ||B||_Sp^p represents the low-rank constraint of the background, and the term ||F||_0 represents the sparse constraint of the foreground, The term represents the spatio-temporal continuity constraint of the foreground. This embodiment uses the alternating direction method of multipliers (ADMM) for optimization and solution. First, introduce auxiliary variables Z and Y, and transform the original problem into stB=Z,F=Y,D=B+F, and then construct the augmented Lagrangian function. The purpose of introducing auxiliary variables is to separate the coupled constraints so that each sub-problem is easy to solve. The optimization process is carried out by alternating the update of each variable: when updating B, a least squares problem is solved, which mainly involves matrix operations; when updating F, soft threshold and hard threshold operations are performed to deal with sparsity and continuity constraints; Z is updated by singular value thresholding method to deal with low rank constraints; Y is updated by hard threshold operation to further enhance sparsity; Lagrangian multipliers and penalty parameters are updated at the same time. These operations have closed solutions or efficient algorithms, so that the entire optimization process can converge quickly. This process is iterated repeatedly until convergence or the maximum number of iterations is reached. In this way, the complex original problem is decomposed into multiple relatively simple sub-problems, each of which has an efficient solution method, so that the algorithm can converge quickly and meet the requirements of real-time detection. In each iteration, the background part B and the foreground part F are gradually optimized, and finally the low-rank background and sparse foreground decomposition results of the video sequence are obtained.

[0079] After the foreground part F is obtained through segmentation, it is binarized and morphologically processed to obtain the final moving target segmentation result. That is, for each input video frame, it is compared with the estimated background part B to obtain the preliminary foreground part. Then, the final moving target segmentation result is obtained through morphological operations and connected region analysis.

[0080] The moving target segmentation algorithm (NR-TV3D) based on non-convex rank approximation and three-dimensional total variation of this embodiment improves the ability to characterize low-rank structures through non-convex rank approximation, and at the same time uses three-dimensional total variation constraints to enhance the spatiotemporal consistency of the segmentation results, effectively realizing the separation of background and moving targets in video sequences. In addition, the image separation method of this embodiment has strong robustness and can effectively cope with the interference of illumination changes and dynamic backgrounds, which is mainly due to the non-convex rank approximation and three-dimensional total variation constraints adopted by the algorithm. The non-convex rank approximation describes the low-rank characteristics of the background more accurately through the Schatten-p norm, so that the algorithm can better adapt to the slow changes of the background, such as illumination changes; and the three-dimensional total variation constraint takes into account the continuity of the foreground target in the time dimension, which helps to suppress the interference caused by the dynamic background and improve the recognition accuracy of the real moving target. The combination of these two key technologies enables the algorithm to achieve accurate segmentation of moving targets under complex backgrounds in practical applications. At the same time, the algorithm has high computational efficiency and is suitable for real-time segmentation scenarios. This is because the alternating direction multiplier method (ADMM) is used to decompose the complex optimization problem into multiple easy-to-solve sub-problems, each of which has an efficient solution method. In addition, many operations in the algorithm, such as singular value thresholding and soft and hard threshold operations, have closed solutions or fast algorithms, which further improves the calculation speed. Through these designs, the algorithm achieves efficient calculation while ensuring segmentation accuracy, enabling it to meet the needs of real-time processing.

[0081] Step 104, according to the change of the detection frame position of the continuous frame images in the video sequence, the moving speed and moving direction of the moving target are obtained, and the moving speed, moving direction, category and size of the moving target are used as the first attribute feature of the moving target.

[0082] Step 105 , according to the foreground part separated from the video sequence and the size of the detection frame, the shape feature and the color feature of the moving object are obtained as the second attribute feature of the moving object.

[0083] Specifically, when obtaining the shape features of the moving target, the total number of pixels in the foreground part is taken as the area of ​​the foreground part, and the number of pixels at the boundary position is taken as the perimeter of the foreground part. The formula 4π*area / perimeter^2 is used to calculate the circularity of the foreground part to describe the regularity of the shape of the foreground part; the aspect ratio of the moving target is obtained according to the height and / or width of the detection frame, and the fill rate of the moving target is obtained according to the total number of pixels in the foreground part and the area of ​​the detection frame; the circularity of the foreground part, the aspect ratio of the moving target and the fill rate are used as the shape features of the moving target.

[0084] Step 106 , combining the first attribute feature and the second attribute feature of the moving target to perform a state assessment on the moving target.

[0085] In the specific implementation, based on the multi-attribute decision TOPSIS algorithm, the decision matrix is ​​constructed using the first attribute characteristics and the second attribute characteristics of the moving target, and different weights are set for different first attribute characteristics and second attribute characteristics according to the degree of influence of different attribute characteristics on the state of the moving target; according to the decision matrix and the set weights, the positive ideal solution and the negative ideal solution are calculated, and the weighted Euclidean distance between the moving target and the positive ideal solution and the negative ideal solution is calculated; according to the calculated weighted Euclidean distance, the relative closeness of the moving target is obtained; the state of the moving target is evaluated by the relative closeness of the moving target. Among them, for the dynamic attribute characteristics of the moving target such as the moving speed and moving direction, first, the Kalman filter is used to predict and update the dynamic attribute characteristics of the target such as the position and speed, and then the decision matrix is ​​constructed.

[0086] When evaluating the state of a moving target through its relative proximity, if the relative proximity of the moving target is less than the first preset threshold, the moving target is in a normal state; if the relative proximity of the moving target is greater than or equal to the first preset threshold and less than or equal to the second preset threshold, the moving target is in a warning state; if the relative proximity of the moving target is greater than the second preset threshold, the moving target is in a dangerous state. Among them, the first preset threshold and the second preset threshold take into account factors such as historical data analysis, expert experience, safety margin, scene adaptability and dynamic adjustment, determine the statistical distribution of different states of the moving target by analyzing a large amount of historical data, determine a reasonable threshold range based on the advice of experts in the field, and adjust the threshold according to different application scenarios (such as airports, borders, etc.), and continuously optimize the threshold setting according to feedback during the operation of the system.

[0087] For example, the following thresholds are set: the normal state corresponds to a relative closeness less than 0.6, the warning state corresponds to between 0.6 and 0.8, and the dangerous state corresponds to greater than 0.8. For example: Assume that a moving target is detected, and its attributes are {category: aircraft, position: (100,200), size: (50,100), speed: 5m / s, direction: 45°, circularity: 0.7, aspect ratio: 2, filling rate: 0.8}. After calculation by the TOPSIS algorithm, the relative closeness is 0.75. According to the above thresholds, the state of the moving target is evaluated as "warning".

[0088] Through the above process, this embodiment can obtain the state evaluation results of each detected target, including target ID, state level (such as normal, warning, danger), relative closeness value (evaluation score between 0-1) and key attribute values ​​(such as current speed, position, shape characteristics, etc.). These results can be used for subsequent target tracking, behavior analysis or alarm triggering and other applications. This state evaluation method combines the static characteristics (such as size, shape) and dynamic characteristics (such as speed and position changes) of the target to provide more accurate and comprehensive evaluation results. By introducing a dynamic attribute update mechanism (i.e., using Kalman filtering to predict and update dynamic attribute characteristics such as the position and speed of the target), this method has higher real-time and adaptability, and can cope with dynamic changes in target status. Compared with traditional methods, this method shows better robustness and accuracy in complex environments, and is suitable for target state evaluation needs in complex environments.

[0089] In this embodiment, after obtaining a video sequence of a moving target in motion by a laser device, first, a target detection model is used to detect the moving target in each frame of the video sequence, and the category of the moving target and the detection frame used to indicate the position and size of the moving target in the image are obtained; then, the background part and the foreground part in the video sequence are separated by using the low-rank characteristics of the background part in the multi-frame image of the video sequence and the sparse characteristics of the foreground part corresponding to the moving target, and the foreground part (i.e., the moving target) in the video sequence is obtained; finally, the information of the moving target obtained by these two methods is combined to evaluate the state of the moving target, and a more comprehensive moving target attribute is extracted, which provides a rich data basis for accurately evaluating the state of the moving target and improves the accuracy of the moving target state evaluation. In addition, the moving speed and direction of the moving target, i.e., the dynamic characteristics of the moving target, can be obtained through the changes in the position of the detection frame of the continuous frame images in the video sequence, while the category, size, shape characteristics, and color characteristics of the moving target are static characteristics of the moving target, taking into account the dynamic change scene of the moving target, and further improving the accuracy of the moving target state evaluation.

[0090] In one embodiment, the state assessment method of the laser device target of the present invention is implemented as follows: Figure 3As shown in the figure, the complete data processing flow is as follows: first input the current frame image, then apply the LVT-YOLOX model to detect the target, and obtain the category and detection frame information. Then use the NR-TV3D algorithm to segment the foreground and background. In the attribute extraction stage, take the center coordinates of the detection frame as the position, record the width and height of the detection frame as the size, compare the target position in adjacent frames to calculate the velocity vector, calculate the shape features such as area, perimeter, circularity, aspect ratio and fill rate based on the foreground, and calculate the average RGB value of the foreground as the color feature. Then apply the improved TOPSIS algorithm for status evaluation, and finally generate an evaluation report containing target ID, status level, relative closeness and key attributes.

[0091] The step division of the various methods above is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the protection scope of the present invention. Adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the protection scope of the invention.

[0092] Another embodiment of the present invention relates to a state assessment system for a laser device target. The implementation details of the state assessment system for a laser device target of this embodiment are described in detail below. The following content is only provided for the convenience of understanding the implementation details and is not necessary for implementing this solution. The state assessment system for a laser device target of this embodiment includes:

[0093] An image acquisition module is used to acquire a video sequence of a moving target during its motion using a laser device;

[0094] The target detection module is used to input each frame of the video sequence into the trained target detection model to obtain the category of the moving target in each frame and the detection frame used to indicate the position and size of the moving target in the image;

[0095] An image segmentation module is used to separate the background part from the foreground part in the video sequence by utilizing the low-rank characteristics of the background part and the sparse characteristics of the foreground part corresponding to the moving target in multiple frames of the video sequence;

[0096] A feature extraction module is used to obtain the moving speed and moving direction of the moving target according to the change of the detection frame position of the continuous frame images in the video sequence, and use the moving speed, moving direction, category and size of the moving target as the first attribute feature of the moving target;

[0097] According to the foreground part separated from the video sequence and the size of the detection frame, the shape feature and color feature of the moving target are obtained as the second attribute feature of the moving target;

[0098] The state evaluation module is used to evaluate the state of the moving target by combining the first attribute feature and the second attribute feature of the moving target.

[0099] It is not difficult to find that this embodiment is a system embodiment corresponding to the above method embodiment, and this embodiment can be implemented in conjunction with the above method embodiment. The relevant technical details and technical effects mentioned in the above embodiment are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiment.

[0100] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of the present invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by the present invention, but this does not mean that there are no other units in this embodiment.

[0101] Another embodiment of the present invention relates to a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the state assessment method of the laser device target in the above-mentioned embodiments.

[0102] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0103] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0104] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0105] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as: ROM), random access memory (Random Access Memory, referred to as: RAM), disk or optical disk and other media that can store program codes.

[0106] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present invention, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present invention.

Claims

1. A method for evaluating the state of a laser device target, characterized in that: The method comprises: Use laser equipment to obtain video sequences of moving targets during movement; Input each frame of the video sequence into the trained target detection model to obtain the category of the moving target in each frame and the detection frame used to indicate the position and size of the moving target in the image; By utilizing the low-rank characteristics of the background part and the sparse characteristics of the foreground part corresponding to the moving target in the multi-frame images of the video sequence, the background part and the foreground part in the video sequence are separated; According to the change of the detection frame position of the continuous frame images in the video sequence, the moving speed and moving direction of the moving target are obtained, and the moving speed, moving direction, category and size of the moving target are used as the first attribute features of the moving target; According to the foreground part separated from the video sequence and the size of the detection frame, the shape feature and color feature of the moving target are obtained as the second attribute feature of the moving target; The state of the moving target is evaluated by combining the first attribute feature and the second attribute feature of the moving target.

2. The state assessment method of a laser device target according to claim 1, characterized in that: The network structure of the target detection model uses YOLOX-Tiny as the backbone network. YOLOX-Tiny includes a feature extraction layer and a detection head, and multiple convolution blocks of the feature extraction layer in YOLOX-Tiny are replaced by Swin Transformer modules. The step of inputting each frame of the video sequence into the trained target detection model to obtain the category of the moving target in each frame and the detection frame used to indicate the position and size of the moving target in the image includes: For each frame of image, the feature extraction layer divides the image into multiple non-overlapping small blocks, each small block is converted into a vector of fixed length through linear mapping, and multiple Swin Transformer modules in the feature extraction layer extract features for each vector; for each vector, the extracted features are divided into multiple non-overlapping windows, and the self-attention mechanism is applied in each window to extract the local features of each window; the shift window strategy is adopted to change the division method of multiple non-overlapping windows between adjacent Swin Transformer modules to transfer information across windows and obtain multi-level features of the image; The detection head adopts YOLOX-Tiny's decoupling head design and sample matching strategy to detect the multi-level features of each frame image, and obtain the category of the moving target in each frame image and the detection frame used to indicate the position and size of the moving target in the image.

3. The state assessment method of a laser device target according to claim 1, characterized in that: The method utilizes the low-rank characteristics of the background part and the sparse characteristics of the foreground part corresponding to the moving target in the multi-frame images of the video sequence to separate the background part from the foreground part in the video sequence, including: Convert the video sequence into a three-dimensional tensor D, and the three-dimensional tensor D includes a background part B and a foreground part F corresponding to the moving target; Using a non-convex rank approximation algorithm, by minimizing the objective function Separate the background part B and the foreground part F in the video sequence; Among them, rank(B) represents the rank constraint of the background part B, ||F||_0 represents the L0 norm constraint of the foreground part F, represents the three-dimensional total variation constraint of the foreground part F.

4. The state assessment method of a laser device target according to claim 3, characterized in that: The rank constraint of the non-convex rank approximation algorithm adopts the Schatten-p norm ||B||Sp=(Σ_iσ_i^p)^(1 / p); The non-convex rank approximation algorithm is used to minimize the objective function Separating the background part B and the foreground part F in the video sequence includes: The objective function Refactoring into objective function Among them, σ_i is the singular value of B, 0 <p≤1; Combining the augmented Lagrangian method and the alternating direction multiplier method, by introducing auxiliary variables and Lagrangian multipliers, the minimization problem of the reconstructed objective function is decomposed into multiple sub-problems, and the background part B, the foreground part F and the auxiliary variables are optimized alternately. After reaching the preset number of iterations, the separated background part B and foreground part F are obtained. In each iteration, singular value decomposition is performed on the background part B, and the singular value is updated through a soft threshold operation; for the foreground part F, a hard threshold operation is used to process the L0 norm constraint, and a soft threshold operation is used to process the three-dimensional total variation constraint.

5. The state assessment method of a laser device target according to claim 1, characterized in that: The step of evaluating the state of the moving target by combining the first attribute feature and the second attribute feature of the moving target includes: Based on the multi-attribute decision-making TOPSIS algorithm, the decision matrix is ​​constructed using the first attribute characteristics and the second attribute characteristics of the moving target, and different weights are set for different first attribute characteristics and second attribute characteristics according to the degree of influence of different attribute characteristics on the state of the moving target; According to the decision matrix and the set weights, the positive ideal solution and the negative ideal solution are calculated, and the weighted Euclidean distance between the moving target and the positive ideal solution and the negative ideal solution is calculated; According to the calculated weighted Euclidean distance, the relative closeness of the moving target is obtained; The state of the moving target is evaluated by the relative proximity of the moving target.

6. The method for evaluating the state of a laser device target according to claim 5, characterized in that: The state evaluation of the moving target by the relative proximity of the moving target includes: If the relative closeness of the moving target is less than the first preset threshold, the moving target is in a normal state; If the relative proximity of the moving target is greater than or equal to a first preset threshold and less than or equal to a second preset threshold, the moving target is in a warning state; If the relative proximity of the moving target is greater than a second preset threshold, the moving target is in a dangerous state.

7. The state assessment method of a laser device target according to claim 1, characterized in that: The step of obtaining the shape features of the moving target according to the foreground portion separated from the video sequence and the size of the detection frame includes: The total number of pixels in the foreground is taken as the area of ​​the foreground, the number of pixels at the boundary is taken as the perimeter of the foreground, and the circularity of the foreground is calculated using the formula 4π*area / perimeter^2 to describe the regularity of the shape of the foreground. According to the height and / or width of the detection frame, the aspect ratio of the moving target is obtained, and according to the total number of pixels in the foreground part and the area of ​​the detection frame, the filling rate of the moving target is obtained; The circularity of the foreground part, the aspect ratio and the filling rate of the moving object are used as the shape features of the moving object.

8. A state assessment system for a laser device target, characterized in that: The system comprises: An image acquisition module is used to acquire a video sequence of a moving target during its motion using a laser device; The target detection module is used to input each frame of the video sequence into the trained target detection model to obtain the category of the moving target in each frame and the detection frame used to indicate the position and size of the moving target in the image; An image segmentation module is used to separate the background part from the foreground part in the video sequence by utilizing the low-rank characteristics of the background part and the sparse characteristics of the foreground part corresponding to the moving target in multiple frames of the video sequence; A feature extraction module is used to obtain the moving speed and moving direction of the moving target according to the change of the detection frame position of the continuous frame images in the video sequence, and use the moving speed, moving direction, category and size of the moving target as the first attribute feature of the moving target; According to the foreground part separated from the video sequence and the size of the detection frame, the shape feature and color feature of the moving target are obtained as the second attribute feature of the moving target; The state evaluation module is used to evaluate the state of the moving target by combining the first attribute feature and the second attribute feature of the moving target.

9. A computer device, characterized in that: include: at least one processor; And, a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the state assessment method of the laser device target according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the state assessment method of a laser device target according to any one of claims 1 to 7 is implemented.

Images