A method, system, device and medium for evaluating the state of a target of a laser device

Abstract

The application relates to the technical field of target detection, and discloses a state evaluation method, system, device and medium for a target of a laser device. In the method, a video sequence of a moving target in a moving process is acquired by using a laser device; a category of the moving target in each frame of image of the video sequence and a detection frame for indicating the position and size of the moving target in the image are acquired by using a target detection model; the background part and the foreground part in the video sequence are separated by using the low-rank characteristics of the background part in the multiple frames of image of the video sequence and the sparse characteristics of the foreground part corresponding to the moving target; and the moving target is subjected to state evaluation by combining the attribute features of the moving target obtained by using the target detection model and the attribute features of the moving target obtained by segmenting the background part and the foreground part of the image, so that the evaluation effect is better.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and in particular relates to a laser device target state evaluation method, system, device and medium. BACKGROUND

[0002] As an advanced optical system, laser devices are widely used in industrial processing, communication, medical treatment and scientific research due to their high precision, fast response and controllable energy. However, in practical applications, laser devices face the challenge of target state evaluation. Complex environmental factors, such as cluttered background, light changes, small target size and high speed, all increase the difficulty of target state evaluation.

[0003] In laser device applications, target state evaluation refers to a comprehensive analysis and judgment of the real-time status of the target irradiated or processed by the laser. This evaluation involves dynamic characteristics such as target position and speed, as well as static characteristics such as material integrity and surface condition. Accurate state evaluation is crucial for adjusting laser power, optimizing processing path, ensuring processing quality and equipment safety.

[0004] For target state evaluation, traditional multi-attribute decision-making methods such as analytic hierarchy process (AHP) and grey correlation analysis have limitations when facing rapidly changing dynamic targets, relying too much on static characteristics and ignoring target motion information, which may lead to distorted evaluation results. SUMMARY

[0005] The purpose of the present application is to provide a laser device target state evaluation method, system, device and medium, which can realize effective evaluation of target state.

[0006] To solve the above technical problems, an embodiment of the present application provides a laser device target state evaluation method, comprising the following steps:

[0007] Using a laser device to obtain a video sequence of a moving target in a motion process;

[0008] Inputting each frame of image in the video sequence into a trained target detection model respectively to obtain the category of the moving target in each frame of image and a detection box indicating the position and size of the moving target in the image;

[0009] Separating the background part and the foreground part in the video sequence by using the low-rank characteristics of the background part and the sparse characteristics of the foreground part corresponding to the moving target in multiple frames of images of the video sequence;

[0010] According to the change of the detection box position of the continuous frame images in the video sequence, the motion speed and the motion direction of the motion target are obtained, and the motion speed, the motion direction, the category and the size of the motion target are taken as the first attribute features of the motion target;

[0011] According to the size of the detection box and the separated foreground part in the video sequence, the shape feature and the color feature of the motion target are obtained as the second attribute features of the motion target;

[0012] The state of the motion target is evaluated in combination with the first attribute features and the second attribute features of the motion target.

[0013] Optionally, the network structure of the target detection model takes YOLOX-Tiny as the backbone network, YOLOX-Tiny includes a feature extraction layer and a detection head, and each convolution block of the feature extraction layer in YOLOX-Tiny is replaced by a SwinTransformer module;

[0014] The method comprises the following steps:

[0015] For each frame image, the image is divided into a plurality of non-overlapping small blocks by the feature extraction layer, each small block is converted into a fixed-length vector through linear mapping processing, and a plurality of Swin Transformer modules in the feature extraction layer are used to extract features of each vector; for each vector, the extracted features are divided into a plurality of non-overlapping windows, and a self-attention mechanism is applied in each window to extract local features of each window; a shift window strategy is used to change the division mode of the plurality of non-overlapping windows between adjacent Swin Transformer modules to perform cross-window information transmission, thereby obtaining multi-level features of the image;

[0016] The multi-level features of each frame image are detected by the decoupled head design and the sample matching strategy of YOLOX-Tiny to obtain the category of the motion target in each frame image and the detection box indicating the position and size of the motion target in the image.

[0017] Optionally, the background part and the foreground part in the video sequence are separated by using the low-rank characteristics of the background part and the sparse characteristics of the foreground part corresponding to the motion target in the plurality of frame images of the video sequence, and the method comprises the following steps:

[0018] The video sequence is converted into a three-dimensional tensor D, and the three-dimensional tensor D includes a background part B and a foreground part F corresponding to the motion target;

[0019] The non-convex rank approximation algorithm is used to minimize an objective function The background part B and the foreground part F in the video sequence are separated.

[0020] Wherein, rank(B) represents the rank constraint of the background part B, ||F||_0 represents the L0 norm constraint of the foreground part F, represents the three-dimensional total variation constraint of the foreground part F.

[0021] Optionally, the rank constraint of the non-convex rank approximation algorithm adopts Schatten-p norm ||B||_Sp=(Σ_iσ_i^p)^(1 / p).

[0022] The non-convex rank approximation algorithm is used to minimize an objective function The background part B and the foreground part F in the video sequence are separated, comprising:

[0023] The objective function is reconstructed into an objective function Wherein, σ_i is the singular value of B, and 0<p≤1.

[0024] By introducing auxiliary variables and Lagrange multipliers, the minimization problem of the reconstructed objective function is decomposed into multiple sub-problems by combining the augmented Lagrange method and the alternating direction multiplier method, and the background part B, the foreground part F and the auxiliary variables are alternately optimized, and after a preset number of iterations is reached, the separated background part B and the foreground part F are obtained.

[0025] Wherein, in each iteration, singular value decomposition is performed on the background part B, and the singular values are updated by a soft threshold operation; for the foreground part F, the L0 norm constraint is processed by a hard threshold operation, and the three-dimensional total variation constraint is processed by a soft threshold operation.

[0026] Optionally, the state of the moving target is evaluated by combining the first attribute feature and the second attribute feature of the moving target, comprising:

[0027] Based on the multi-attribute decision TOPSIS algorithm, a decision matrix is constructed by using the first attribute feature and the second attribute feature of the moving target, and different weights are set for different first attribute features and second attribute features according to the influence degree of different attribute features on the state of the moving target.

[0028] According to the decision matrix and the set weight, the positive ideal solution and the negative ideal solution are calculated, and the weighted Euclidean distance between the moving target and the positive ideal solution and the negative ideal solution is calculated.

[0029] According to the calculated weighted Euclidean distance, the relative closeness of the moving target is obtained.

[0030] The state of the moving target is evaluated according to the relative closeness of the moving target.

[0031] Optionally, the state of the moving target is evaluated according to the relative closeness of the moving target, and the state evaluation includes:

[0032] If the relative closeness of the moving target is less than a first preset threshold, the moving target is in a normal state.

[0033] If the relative closeness of the moving target is greater than or equal to the first preset threshold and less than or equal to a second preset threshold, the moving target is in a warning state.

[0034] If the relative closeness of the moving target is greater than the second preset threshold, the moving target is in a dangerous state.

[0035] Optionally, the shape feature of the moving target is obtained according to the separated foreground part in the video sequence and the size of the detection frame, and the shape feature includes:

[0036] The total pixel number of the foreground part is taken as the area of the foreground part, the pixel number at the boundary position is taken as the perimeter of the foreground part, the circularity of the foreground part is calculated by using a formula 4π*area / perimeter^2 to describe the regularity of the shape of the foreground part;

[0037] The aspect ratio of the moving target is obtained according to the height and the width of the detection frame, and the filling rate of the moving target is obtained according to the total pixel number of the foreground part and the area of the detection frame.

[0038] The circularity of the foreground part, the aspect ratio of the moving target and the filling rate of the moving target are taken as the shape feature of the moving target.

[0039] Embodiments of the present application also provide a state evaluation system of a laser equipment target, including:

[0040] An image acquisition module is configured to acquire a video sequence of the moving target in a moving process by using a laser equipment.

[0041] A target detection module is configured to input each frame of image in the video sequence into a trained target detection model respectively to obtain a category of the moving target in each frame of image and a detection frame for indicating the position and size of the moving target in the image.

[0042] An image segmentation module is configured to separate a background part and a foreground part in the video sequence by using the low-rank characteristics of the background part and the sparse characteristics of the foreground part corresponding to the moving target in multiple frames of image of the video sequence.

[0043] The feature extraction module is configured to acquire a motion speed and a motion direction of the moving target according to a change of the detection frame position of the continuous frame images in the video sequence, and take the motion speed, the motion direction, the category and the size of the moving target as first attribute features of the moving target.

[0044] According to the separated foreground part in the video sequence and the size of the detection frame, shape features and color features of the moving target are acquired as second attribute features of the moving target.

[0045] The state evaluation module is configured to evaluate the state of the moving target in combination with the first attribute features and the second attribute features of the moving target.

[0046] Embodiments of the present application also provide a computer device, comprising at least one processor, and a memory in communication connection with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the state evaluation method of the laser equipment target.

[0047] Embodiments of the present application also provide a computer readable storage medium storing a computer program, and the computer program is executed by a processor to implement the state evaluation method of the laser equipment target.

[0048] The state evaluation method of the laser equipment target provided by the present application has at least the following beneficial effects:

[0049] After the video sequence of the moving target in the motion process is acquired by the laser equipment, first, a target detection model is used to detect the moving target in each frame image in the video sequence to obtain the category of the moving target and a detection frame indicating the position and size of the moving target in the image; then, the low-rank characteristics of the background part in the multiple frame images of the video sequence and the sparse characteristics of the foreground part corresponding to the moving target are used to separate the background part and the foreground part in the video sequence to obtain the foreground part (i.e. the moving target) in the video sequence; finally, the information of the moving target obtained by the two methods is combined to evaluate the state of the moving target, and more comprehensive attributes of the moving target are extracted to provide rich data basis for accurately evaluating the state of the moving target, and the accuracy of the state evaluation of the moving target is improved.

[0050] Moreover, the motion speed and the motion direction of the moving target, i.e. the dynamic features of the moving target, can be acquired through the change of the detection frame position of the continuous frame images in the video sequence, and the category, the size, the shape features and the color features of the moving target are static features of the moving target, which considers the dynamic change scene of the moving target and further improves the accuracy of the state evaluation of the moving target. BRIEF DESCRIPTION OF DRAWINGS

[0051] One or more embodiments are illustrated by way of example in the figures that form a part of this disclosure and which do not limit the scope of the embodiments in any way. In the figures:

[0052] Figure 1 is a flow chart of a state evaluation method of a laser equipment target according to an embodiment of the present application;

[0053] Figure 2 is a processing flow chart of a target detection model according to an embodiment of the present application;

[0054] Figure 3 is a schematic diagram of a state evaluation of a laser equipment target according to an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the drawings. However, those skilled in the art can understand that in the embodiments of the present application, many technical details are proposed in order to make the reader better understand the present application. However, even without these technical details and based on various changes and modifications of the following embodiments, the technical solutions claimed by the present application can be implemented. The division of the following embodiments is for the convenience of description and should not constitute any limitation on the specific implementation of the present application. The embodiments can be combined and referred to each other without contradiction.

[0056] First, the state evaluation of the laser equipment target in the related art is introduced:

[0057] In the target detection detection aspect:

[0058] Deep learning technology, especially Convolutional Neural Networks (CNN), has made significant progress in target detection. Single-stage target detection algorithms, such as YOLO series (YOLOv3, YOLOv4, YOLOX, etc.), have attracted widespread attention for their fast and efficient detection performance. However, the detection accuracy of these models for small targets in complex backgrounds still needs to be improved, and it is difficult to fully meet the real-time application requirements.

[0059] Vision Transformers have also made breakthroughs in the field of object detection. For example, applying Transformers to object detection enables end-to-end detection. Swin Transformer significantly improves computational efficiency by introducing a sliding window attention mechanism. Models such as PVTv2 further optimize the structure of Transformers and achieve state-of-the-art performance on multiple visual tasks. However, these models generally have large parameter quantities and high computational complexity, making it difficult to run in real time on resource-constrained devices.

[0060] In the aspect of motion target segmentation

[0061] Deep learning-based methods, such as through spatio-temporal convolution networks, achieve end-to-end video object segmentation, or introduce multi-object tracking mechanisms to improve temporal consistency of segmentation. However, these methods generally require a large amount of computational resources, making it difficult to meet real-time requirements.

[0062] In the aspect of target state evaluation

[0063] Traditional multi-attribute decision-making methods such as the Analytic Hierarchy Process (AHP) and grey relational analysis have limitations in handling dynamically changing target attributes. These methods usually assume that attribute weights are fixed, making it difficult to adapt to real-time changing scenarios. In addition, they mainly focus on static features, ignoring the motion information of the target, which may lead to distorted evaluation results.

[0064] One embodiment of the present application relates to a method for evaluating the state of a laser device target. The implementation details of the method for evaluating the state of the laser device target in this embodiment are described below. The following content provides implementation details for easy understanding, and is not necessary for implementing this solution.

[0065] The specific process of the method for evaluating the state of the laser device target in this embodiment is shown in Figure 1 , which includes:

[0066] Step 101: Use a laser device to obtain a video sequence of a moving target during its movement.

[0067] Step 102: Input each frame of the video sequence into a trained target detection model to obtain the class of the moving target in each frame and a detection box indicating the position and size of the moving target in the image.

[0068] Specifically, the target detection model (LVT-YOLOX) of the embodiment has a network structure taking YOLOX-Tiny as a backbone network. The YOLOX-Tiny includes a feature extraction layer and a detection head, and each convolution block of the feature extraction layer in the YOLOX-Tiny is replaced by a Swin Transformer module. That is, the LVT-YOLOX model is based on the overall architecture of YOLOX-Tiny, and simultaneously fuses Swin Transformer, forming a lightweight model suitable for laser equipment target detection.

[0069] For each image in the video sequence, target detection is implemented through the flow as shown in Figure 2 The feature extraction layer of the YOLOX-Tiny divides the image into multiple non-overlapping patches, each patch is converted into a fixed-length vector patchtokens through linear mapping processing, and the multiple Swin Transformer modules in the convolution block of the YOLOX-Tiny feature extraction layer are replaced by a self-attention mechanism and cross-window information transmission. Different scales of feature extraction are performed on multiple vectors to extract multi-level features of the image. When the self-attention mechanism and cross-window information transmission are performed between the multiple Swin Transformer modules, different scales of feature extraction are performed on multiple vectors. First, the multiple Swin Transformer modules perform feature extraction on each vector. For each vector, the extracted features are divided into multiple non-overlapping windows, the window size is 4x4 or 8x8, and the self-attention mechanism is applied in each window to generate attention weights by calculating the similarity between queries, keys and values, thereby achieving effective extraction of local features. Then, a shift window strategy is used to change the division method of multiple non-overlapping windows between adjacent Swin Transformer modules to perform cross-window information transmission, thereby obtaining multi-level features of the image.

[0070] Finally, the model retains the decoupled head design and sample matching strategy of YOLOX-Tiny. By using the decoupled head design and sample matching strategy in the detection head, the multi-level features are detected to obtain the category of the moving target in the image and the detection box indicating the position and size of the moving target in the image, which can ensure the efficiency and accuracy of the detection. Among them, the decoupled head design divides the target detection task into sub-tasks such as category prediction, bounding box regression, and target prediction, each of which uses independent convolutional layers for processing, and uses a dynamic k matching algorithm for sample matching to calculate the quality score of each real target and all predicted boxes; the sample matching strategy selects the top k predicted boxes with the highest quality score as positive samples based on the combination of IoU and class confidence, and adjusts the k value according to the target size and complexity, and considers the remaining predicted boxes as negative samples or ignores them, and uses the matching results to guide the loss function calculation to optimize the model parameters; finally, the category of the moving target in the image and the detection box indicating the position and size of the moving target in the image are obtained.

[0071] In addition, in order to realize the lightweight design, the model reduces the number of SwinTransformer blocks compared to the original SwinTransformer, reduces the number of Transformer blocks from the original 4 layers to 2 layers, and uses a smaller attention calculation window, effectively reducing the overall parameter quantity and computational complexity. Moreover, the model introduces spatial attention mechanism and channel attention mechanism on each scale of feature map, enhancing the attention degree and feature expression ability of the target. Spatial attention mechanism is mainly used to highlight the features of the target and strengthen the attention degree of the target. It identifies and highlights important regions through spatial information on the feature map, so that the model can more accurately detect the target. Channel attention mechanism is used to enhance the expression of effective features by strengthening the feature information of important channels, so that the model can more accurately distinguish the target from the background and improve the stability and accuracy of the detection.

[0072] Through the design of the above network structure, the LVT-YOLOX model inherits the lightweight and efficient characteristics of YOLOX-Tiny while obtaining the powerful feature extraction capability of Swin Transformer, enabling efficient and accurate detection of small targets in complex backgrounds while meeting the real-time detection requirements.

[0073] In one example, in order to enhance the learning ability of the model for difficult samples, a fuzzy loss function FocalLoss is introduced, which adjusts certain parameters to make the model pay more attention to difficult-to-classify samples. This improvement enables LVT-YOLOX to maintain high-speed detection while improving the accuracy of small target detection in complex backgrounds.

[0074] At step 103, the background part and the foreground part in the video sequence are separated by using the low-rank characteristics of the background part and the sparse characteristics of the foreground part corresponding to the moving object in the multiple frames of the video sequence.

[0075] Specifically, the video sequence is first converted into a three-dimensional tensor D, and the three-dimensional tensor D includes the background part B and the foreground part F corresponding to the moving object. For each frame image, the pixel values thereof are arranged according to the height and the width, and then the continuous t frames are stacked to form D ∈ R^(m×n×t), where m and n represent the height and the width of the image respectively, and t represents the time dimension.

[0076] Then, a non-convex rank approximation algorithm is used to minimize the objective function to separate the background part B and the foreground part F in the video sequence; where rank(B) represents the rank constraint of the background part B, ||F||_0 represents the L0 norm constraint (sparsity constraint) of the foreground part F, and ||F||_3 represents the three-dimensional total variation constraint (spatial-temporal smoothness constraint) of the foreground part F.

[0077] In a specific implementation, to better approximate the actual rank function and reduce the calculation complexity, the non-convex rank approximation algorithm uses the Schatten-p norm ||B||_Sp=(Σ_iσ_i^p)^(1 / p) to replace the traditional nuclear norm (i.e., the rank constraint), where σ_i is the singular value of B, and 0<p≤1. At this time, the objective function can be reconstructed as the objective function s.t.D=B+F. By combining the augmented Lagrangian method and the alternating direction multiplier method, the minimization problem of the reconstructed objective function is decomposed into multiple sub-problems by introducing auxiliary variables and Lagrange multipliers, and the background part B, the foreground part F and the auxiliary variables are alternately optimized. After a preset number of iterations is reached, the separated background part B and the foreground part F are obtained. In each iteration, the singular value decomposition is performed on the background part B, and the singular values are updated by the soft threshold operation; for the foreground part F, the L0 norm constraint is processed by the hard threshold operation, and the three-dimensional total variation constraint is processed by the soft threshold operation, so as to achieve efficient solution.

[0078] Wherein, the original problem (i.e., the original objective function) aims to decompose the video sequence D into a low-rank background B and a sparse foreground F, and is expressed as s.t.D=B+F. Wherein, the term ||B||_Sp^p represents the low-rank constraint of the background, the term ||F||_0 represents the sparsity constraint of the foreground, and the term ||F||_3 represents the spatio-temporal continuity constraint of the foreground. In this embodiment, the alternating direction multiplier method (ADMM) is used for optimization and solution, and first auxiliary variables Z and Y are introduced to convert the original problem into s.t. B = Z, F = Y, D = B + F, and then construct an augmented Lagrangian function. The purpose of introducing auxiliary variables is to decouple the coupled constraints, so that each subproblem is easy to solve. The optimization process is carried out by alternately updating each variable: updating B solves a least squares problem, mainly involving matrix operations; updating F performs soft thresholding and hard thresholding operations to handle sparsity and continuity constraints; updating Z is solved by the singular value thresholding method to handle the low-rank constraint; updating Y is solved by the hard thresholding operation to further enhance sparsity; and updating the Lagrange multiplier and penalty parameter at the same time. These operations have closed-form solutions or efficient algorithms, so that the entire optimization process can quickly converge. This process is iterated until convergence or the maximum number of iterations is reached. In this way, the complex original problem is decomposed into multiple relatively simple subproblems, each of which has an efficient solution method, so that the algorithm can quickly converge to meet the real-time detection requirements. In each iteration, the background part B and the foreground part F are gradually optimized, and finally the low-rank background and sparse foreground decomposition results of the video sequence are obtained.

[0079] After the foreground part F is segmented, the foreground part F is binarized and morphologically processed to obtain the final moving target segmentation result. That is, for each input video frame, it is compared with the estimated background part B to obtain a preliminary foreground part. Then, through morphological operations and connected region analysis, the final moving target segmentation result is obtained.

[0080] The motion target segmentation algorithm (NR-TV3D) based on non-convex rank approximation and three-dimensional total variation of the embodiment improves the description ability of low rank structure through non-convex rank approximation, and enhances the spatio-temporal consistency of the segmentation result by using three-dimensional total variation constraint, effectively realizing the separation of background and motion target in the video sequence. In addition, the image separation method of the embodiment has strong robustness and can effectively cope with the interference of light changes and dynamic backgrounds, which is mainly due to the non-convex rank approximation and three-dimensional total variation constraint adopted by the algorithm. The non-convex rank approximation more accurately describes the low rank characteristics of the background through the Schatten-p norm, so that the algorithm can better adapt to the slow changes of the background, such as light changes; while the three-dimensional total variation constraint considers the continuity of the foreground target in the time dimension, which helps to suppress the interference of dynamic background and improve the recognition accuracy of real motion targets. The combination of these two key technologies enables the algorithm to achieve accurate segmentation of motion targets in complex backgrounds in practical applications. At the same time, the algorithm has high computational efficiency and is suitable for real-time segmentation scenarios, because the alternating direction multiplier method (ADMM) is used to decompose the complex optimization problem into multiple sub-problems that are easy to solve, each sub-problem has an efficient solution method, and many operations in the algorithm, such as singular value thresholding and soft and hard thresholding operations, have closed-form solutions or fast algorithms, further improving the calculation speed. Through these designs, the algorithm not only ensures the segmentation accuracy, but also realizes efficient calculation, so that it can meet the real-time processing requirements.

[0081] In step 104, the motion speed and direction of the motion target are obtained according to the change of the detection frame position of the continuous frame images in the video sequence, and the motion speed, direction, category and size of the motion target are taken as the first attribute features of the motion target.

[0082] In step 105, the shape feature and color feature of the motion target are obtained according to the separated foreground part and the size of the detection frame in the video sequence, as the second attribute features of the motion target.

[0083] Specifically, when obtaining the shape feature of the motion target, the total pixel number of the foreground part is taken as the area of the foreground part, and the pixel number at the boundary position is taken as the perimeter of the foreground part. The circularity of the foreground part is calculated by using the formula 4π*area / perimeter^2 to describe the regularity of the shape of the foreground part; the aspect ratio of the motion target is obtained according to the height and width of the detection frame, and the fill rate of the motion target is obtained according to the total pixel number of the foreground part and the area of the detection frame; the circularity of the foreground part, the aspect ratio of the motion target and the fill rate are taken as the shape features of the motion target.

[0084] In step 106, the state of the motion target is evaluated by combining the first attribute features and the second attribute features of the motion target.

[0085] In a specific implementation, based on the multi-attribute decision TOPSIS algorithm, a decision matrix is constructed using the first attribute feature and the second attribute feature of the moving target, and different weights are set for different first attribute features and second attribute features according to the influence degree of different attribute features on the state of the moving target; the positive ideal solution and the negative ideal solution are calculated according to the decision matrix and the set weights, and the weighted Euclidean distance of the moving target from the positive ideal solution and the negative ideal solution is calculated; the relative closeness of the moving target is obtained according to the calculated weighted Euclidean distance; the state of the moving target is evaluated through the relative closeness of the moving target. For dynamic attribute features such as the moving speed and the moving direction of the moving target, first, Kalman filtering is used to predict and update the dynamic attribute features such as the position and the speed of the target, and then the decision matrix is constructed.

[0086] When the state of the moving target is evaluated through the relative closeness of the moving target, if the relative closeness of the moving target is less than a first preset threshold, the moving target is in a normal state; if the relative closeness of the moving target is greater than or equal to the first preset threshold and less than or equal to a second preset threshold, the moving target is in a warning state; if the relative closeness of the moving target is greater than the second preset threshold, the moving target is in a dangerous state. The first preset threshold and the second preset threshold consider factors such as historical data analysis, expert experience, safety margin, scene adaptability, and dynamic adjustment, the statistical distribution of different states of the moving target is determined by analyzing a large amount of historical data, a reasonable threshold range is determined in combination with the suggestions of experts in the field, the threshold is adjusted according to different application scenarios (such as airports, borders, etc.), and the threshold setting is continuously optimized according to feedback during system operation.

[0087] For example, the following thresholds are set: the normal state corresponds to a relative closeness less than 0.6, the warning state corresponds to between 0.6 and 0.8, and the dangerous state corresponds to greater than 0.8. As an example, suppose a moving target is detected, with attributes {category: airplane, position: (100, 200), size: (50, 100), speed: 5 m / s, direction: 45°, circularity: 0.7, aspect ratio: 2, fill rate: 0.8}. After TOPSIS algorithm calculation, the relative closeness is 0.75. According to the above thresholds, the state evaluation of the moving target is “warning”.

[0088] Through the above process, the state evaluation result of each detection target can be obtained in this embodiment, including target ID, state level (such as normal, warning, danger), relative closeness value (evaluation score between 0-1) and key attribute value (such as current speed, position, shape feature, etc.). These results can be used for subsequent target tracking, behavior analysis or alarm triggering applications. The state evaluation method combines the static features (such as size, shape) and dynamic features (such as speed, position change) of the target, providing more accurate and comprehensive evaluation results. By introducing the attribute dynamic updating mechanism (i.e. using Kalman filter to predict and update the dynamic attribute features such as position and speed of the target), the method has higher real-time and adaptability, and can cope with the dynamic changes of the target state. Compared with traditional methods, this method shows better robustness and accuracy in complex environments, and is suitable for target state evaluation requirements in complex environments.

[0089] In this embodiment, after obtaining the video sequence of the moving target in the movement process through the laser device, first, the target detection model is used to detect the moving target in each frame of image in the video sequence, and the category of the moving target and the detection box indicating the position and size of the moving target in the image are obtained; then, the low-rank characteristics of the background part in the multiple frames of images of the video sequence and the sparse characteristics of the corresponding foreground part of the moving target are used to separate the background part and the foreground part in the video sequence, and the foreground part (i.e. the moving target) in the video sequence is obtained; finally, the state of the moving target is evaluated by combining the information of the moving target obtained by the two ways, and more comprehensive attributes of the moving target are extracted, which provides rich data basis for accurate evaluation of the state of the moving target and improves the accuracy of the state evaluation of the moving target. Moreover, through the change of the position of the detection box in the continuous frame images in the video sequence, the movement speed and direction of the moving target, i.e. the dynamic characteristics of the moving target, can be obtained, and the category, size, shape features and color features of the moving target are the static characteristics of the moving target, which considers the dynamic change scene of the moving target, further improving the accuracy of the state evaluation of the moving target.

[0090] In one embodiment, the implementation mode of the state evaluation method of the target of the laser device of the present application is as follows Figure 3As shown, the complete data processing process is as follows: first, input the current frame image, then apply the LVT-YOLOX model to detect the target to obtain the category and detection box information. Then use the NR-TV3D algorithm to segment the foreground and background. In the attribute extraction stage, take the detection box center coordinates as the position, record the detection box width and height as the size, compare the target position in adjacent frames to calculate the velocity vector, calculate the area, perimeter, circularity, aspect ratio and filling rate of the foreground based on the shape features, and calculate the average RGB value of the foreground as the color feature. Then the improved TOPSIS algorithm is applied for state evaluation, and finally the evaluation report containing target ID, state level, relative closeness and key attributes is generated.

[0091] The step division of the above various methods is only for the purpose of clear description, and in implementation, one step can be combined or some steps can be split and decomposed into multiple steps, as long as the same logical relationship is included, and all are within the protection scope of the present application. Irrelevant modifications or irrelevant designs are added to the algorithm or process, but the core design of the algorithm and process is not changed, and all are within the protection scope of the present application.

[0092] Another embodiment of the present application relates to a state evaluation system of a laser equipment target, and the implementation details of the state evaluation system of the laser equipment target of the present embodiment will be specifically described below. The following content is only provided for the implementation details for the convenience of understanding, and is not essential for implementing the present solution. The state evaluation system of the laser equipment target of the present embodiment comprises:

[0093] An image acquisition module is configured to acquire a video sequence of a moving target in a motion process by using a laser equipment.

[0094] A target detection module is configured to input each frame of image in the video sequence into a trained target detection model respectively to obtain a category of the moving target in each frame of image and a detection box for indicating a position and a size of the moving target in the image.

[0095] An image segmentation module is configured to separate a background part and a foreground part in the video sequence by using a low-rank characteristic of the background part and a sparse characteristic of the foreground part corresponding to the moving target in multiple frames of image of the video sequence.

[0096] A feature extraction module is configured to acquire a motion speed and a motion direction of the moving target according to a change condition of the detection box position of the continuous frames of image in the video sequence, and take the motion speed, the motion direction, the category and the size of the moving target as first attribute features of the moving target.

[0097] According to the separated foreground part in the video sequence and the size of the detection box, shape features and color features of the moving target are acquired as second attribute features of the moving target.

[0098] The state evaluation module is configured to evaluate the state of the moving target in combination with the first attribute feature and the second attribute feature of the moving target.

[0099] It can be found that the embodiment is a system embodiment corresponding to the above-mentioned method embodiment, and the embodiment can be implemented in cooperation with the above-mentioned method embodiment. The related technical details and technical effects mentioned in the above-mentioned embodiments are still valid in the embodiment. In order to reduce repetition, they will not be described here. Accordingly, the related technical details mentioned in the embodiment can also be applied in the above-mentioned embodiments.

[0100] It is worth mentioning that each module involved in the embodiment is a logical module. In actual application, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of the application, units not closely related to solving the technical problems proposed in the application are not introduced in the embodiment, but this does not mean that there are no other units in the embodiment.

[0101] Another embodiment of the application relates to a computer device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the state evaluation method of the laser device target in each of the above-mentioned embodiments.

[0102] The memory and the processor are connected in a bus manner, the bus can include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripheral devices, voltage stabilizers and power management circuits together, which are well known in the art, and therefore, they will not be further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements such as multiple receivers and transmitters, which provide a unit for communicating with various other devices on the transmission medium. The data processed by the processor is transmitted on the wireless medium through the antenna, and further, the antenna also receives data and transmits the data to the processor.

[0103] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interface, voltage regulation, power management and other control functions. And the memory can be used to store the data used by the processor in the execution operation.

[0104] Another embodiment of the present application relates to a computer readable storage medium storing a computer program. The computer program, when executed by a processor, implements the method embodiments described above.

[0105] That is, those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by programs instructing relevant hardware, the programs are stored in a storage medium, and the programs include a plurality of instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media capable of storing program codes.

[0106] Those skilled in the art can understand that the above-mentioned embodiments are specific embodiments for implementing the present application, and in actual applications, various changes can be made in form and details without departing from the spirit and scope of the present application.

Claims

1. A method for assessing the state of a laser device target, characterized in that, The method includes: Using laser equipment to acquire video sequences of moving targets during their motion; Each frame of the video sequence is input into the trained target detection model to obtain the category of moving targets in each frame and the detection box used to indicate the position and size of moving targets in the image; By utilizing the low-rank characteristics of the background portion and the sparsity characteristics of the foreground portion corresponding to the moving target in the multi-frame images of the video sequence, the background portion and the foreground portion in the video sequence are separated. Based on the changes in the position of the detection box in consecutive frames of the video sequence, the motion speed and direction of the moving target are obtained, and the motion speed, direction, category, and size of the moving target are used as the first attribute features of the moving target. Based on the foreground portion separated from the video sequence and the size of the detection box, the shape and color features of the moving target are obtained as the second attribute features of the moving target; The state of the moving target is assessed by combining its first and second attribute characteristics. The target detection model uses YOLOX-Tiny as its backbone network. YOLOX-Tiny includes a feature extraction layer and a detection head. Furthermore, multiple convolutional blocks in the feature extraction layer of YOLOX-Tiny are replaced by Swing Transformer modules. The step of inputting each frame of the video sequence into the trained target detection model to obtain the category of moving targets in each frame and the detection box used to indicate the position and size of the moving targets in the image includes: For each frame of the image, the image is divided into multiple non-overlapping blocks by a feature extraction layer. Each block is transformed into a fixed-length vector through linear mapping, and multiple Swing Transformer modules in the feature extraction layer extract features from each vector. For each vector, the extracted features are divided into multiple non-overlapping windows, and a self-attention mechanism is applied within each window to extract local features. A shift window strategy is adopted to change the division method of multiple non-overlapping windows between adjacent Swing Transformer modules to perform cross-window information transfer and obtain multi-level features of the image. By employing the YOLOX-Tiny decoupled head design and sample matching strategy, the detection head detects multi-level features of each frame of image, obtaining the category of moving targets in each frame of image and detection boxes used to indicate the position and size of moving targets in the image; The process of combining the first and second attribute features of the moving target to perform state assessment includes: Based on the TOPSIS algorithm for multi-attribute decision-making, a decision matrix is ​​constructed using the first and second attribute features of the moving target. Different weights are assigned to different first and second attribute features according to the degree of influence of different attribute features on the state of the moving target. Based on the decision matrix and the set weights, calculate the positive ideal solution and the negative ideal solution, and calculate the weighted Euclidean distance between the moving target and the positive ideal solution and the negative ideal solution; The relative proximity of the moving target is obtained based on the calculated weighted Euclidean distance. The state of a moving target is assessed by its relative proximity to the target.

2. The method for assessing the state of a laser device target according to claim 1, characterized in that, The method of separating the background and foreground portions in a video sequence by utilizing the low-rank characteristics of the background portion and the sparsity characteristics of the foreground portion corresponding to the moving target in multiple frames of the video sequence includes: The video sequence is converted into a three-dimensional tensor D, and the three-dimensional tensor D contains a background part B and a foreground part F corresponding to the moving target; The non-convex rank approximation algorithm is used to separate the background part B and the foreground part F in the video sequence by minimizing the objective function min_{B,F}rank(B)+λ||F||_0+μ||∇_3DF||_1; Where rank(B) represents the rank constraint of the background part B, ||F||_0 represents the L0 norm constraint of the foreground part F, and ||∇_3DF||_1 represents the three-dimensional total variation constraint of the foreground part F.

3. The method for assessing the state of a laser device target according to claim 2, characterized in that, The rank constraint of the non-convex rank approximation algorithm adopts the Schatten-p norm ||B||Sp=(Σ_iσ_i^p)^(1 / p); The non-convex rank approximation algorithm is used to separate the background part B and the foreground part F in the video sequence by minimizing the objective function min_{B,F}rank(B)+λ||F||_0+μ||∇_3DF||_1, including: The objective function min_{B,F}rank(B)+λ||F||_0+μ||∇_3DF||_1 is reconstructed into the objective function min_{B,F}||B||_Sp^p+λ||F||_0+μ||∇_3DF||_1; where σ_i are the singular values ​​of B, 0 <p≤1; By combining the augmented Lagrange method and the alternating direction multiplier method, and by introducing auxiliary variables and Lagrange multipliers, the minimization problem of the reconstructed objective function is decomposed into multiple subproblems. The background part B, the foreground part F, and the auxiliary variables are alternately optimized, and the separated background part B and foreground part F are obtained after reaching the preset number of iterations. In each iteration, singular value decomposition is performed on the background part B, and the singular values ​​are updated through soft thresholding. For the foreground part F, hard thresholding is used to handle L0 norm constraints, and soft thresholding is used to handle 3D total variation constraints.

4. The method for assessing the state of a laser device target according to claim 1, characterized in that, The process of assessing the state of a moving target based on its relative proximity includes: If the relative proximity of the moving target is less than the first preset threshold, the moving target is in a normal state. If the relative proximity of the moving target is greater than or equal to the first preset threshold and less than or equal to the second preset threshold, the moving target is in a warning state. If the relative proximity of the moving target is greater than the second preset threshold, the moving target is in a dangerous state.

5. The method for assessing the state of a laser device target according to claim 1, characterized in that, The step of obtaining the shape features of the moving target based on the foreground portion separated from the video sequence and the size of the detection box includes: The area of ​​the foreground is defined by the total number of pixels in the foreground and the perimeter is defined by the number of pixels at the boundary. The circularity of the foreground is calculated using the formula 4π * area / perimeter^2 to describe the regularity of the shape of the foreground. Based on the height and / or width of the detection box, the aspect ratio of the moving target is obtained, and based on the total number of pixels in the foreground and the area of ​​the detection box, the fill rate of the moving target is obtained. The roundness of the foreground, the aspect ratio of the moving target, and the fill rate are used as the shape features of the moving target.

6. A state assessment system for a laser device target, characterized in that, The system includes: The image acquisition module is used to acquire video sequences of moving targets during their motion using laser equipment; The target detection module is used to input each frame of the video sequence into the trained target detection model to obtain the category of moving targets in each frame and the detection box used to indicate the position and size of the moving targets in the image; The image segmentation module is used to separate the background and foreground parts in a video sequence by utilizing the low-rank characteristics of the background part and the sparsity characteristics of the foreground part corresponding to the moving target in the multi-frame images of the video sequence. The feature extraction module is used to obtain the motion speed and direction of the moving target based on the changes in the position of the detection box in consecutive frames of the video sequence, and to use the motion speed, direction, category and size of the moving target as the first attribute feature of the moving target; Based on the foreground portion separated from the video sequence and the size of the detection box, the shape and color features of the moving target are obtained as the second attribute features of the moving target; The state assessment module is used to assess the state of a moving target by combining its first and second attribute features. The target detection model uses YOLOX-Tiny as its backbone network. YOLOX-Tiny includes a feature extraction layer and a detection head. Furthermore, multiple convolutional blocks in the feature extraction layer of YOLOX-Tiny are replaced by Swing Transformer modules. The target detection module is also used for: For each frame of the image, the image is divided into multiple non-overlapping blocks by a feature extraction layer. Each block is transformed into a fixed-length vector through linear mapping, and multiple Swing Transformer modules in the feature extraction layer extract features from each vector. For each vector, the extracted features are divided into multiple non-overlapping windows, and a self-attention mechanism is applied within each window to extract local features. A shift window strategy is adopted to change the division method of multiple non-overlapping windows between adjacent Swing Transformer modules to perform cross-window information transfer and obtain multi-level features of the image. By employing the YOLOX-Tiny decoupled head design and sample matching strategy, the detection head detects multi-level features of each frame of image, obtaining the category of moving targets in each frame of image and detection boxes used to indicate the position and size of moving targets in the image; The status assessment module is also used for: Based on the TOPSIS algorithm for multi-attribute decision-making, a decision matrix is ​​constructed using the first and second attribute features of the moving target. Different weights are assigned to different first and second attribute features according to the degree of influence of different attribute features on the state of the moving target. Based on the decision matrix and the set weights, calculate the positive ideal solution and the negative ideal solution, and calculate the weighted Euclidean distance between the moving target and the positive ideal solution and the negative ideal solution; The relative proximity of the moving target is obtained based on the calculated weighted Euclidean distance. The state of a moving target is assessed by its relative proximity to the target.

7. A computer device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the state assessment method for a laser device target as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the state assessment method for the laser device target as described in any one of claims 1 to 5.

Images