Emergency drill efficiency monitoring method, device and equipment based on machine vision

Abstract

The invention relates to the technical field of machine vision, and provides an emergency drill efficiency monitoring method, device and equipment based on machine vision, and the method comprises the steps: obtaining an emergency drill scene original image, and carrying out the preprocessing of the emergency drill scene original image, and obtaining an emergency drill scene image; skeleton extraction is carried out on the emergency drill scene image, skeleton lines are determined, and skeleton nodes are determined according to the skeleton lines; the extracted skeleton nodes are analyzed, and the offset skeleton nodes are corrected according to an analysis result; according to the corrected position information of the skeleton nodes, gaps and fractures in the skeleton structure are reconnected, and a final skeleton structure is obtained; and monitoring and analyzing the drilling condition according to the skeleton structure. According to the invention, the problem of poor accuracy of multi-target identification and skeleton extraction in emergency drill efficiency monitoring in the prior art is solved, and the improvement of skeleton extraction and identification precision and accuracy in a complex environment is realized.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a method, device and equipment for monitoring the effectiveness of emergency drills based on machine vision. Background Art

[0002] According to relevant regulations, at least one comprehensive emergency plan drill or special emergency plan drill should be organized every year, and at least one on-site disposal plan drill should be organized every six months. At present, the traditional emergency drill effectiveness evaluation is completed manually by the organizers, relying on the scores of observation groups at all levels, and then summarizing them to form an emergency drill effectiveness rating. Although this method can achieve the rating of emergency drill effectiveness, it is highly subjective, and many companies' emergency drills are just formalities, and cannot be objectively scored based on the actual reactions and real physical participation of the drill personnel. Therefore, they will be in a hurry when an emergency occurs.

[0003] With the progress of society and the development of science and technology, machine vision technology is being used more and more widely in various fields, especially in the monitoring of emergency drill effectiveness, where its importance is becoming increasingly prominent. Emergency drills are an important means to improve the ability to respond to emergencies, and the introduction of machine vision technology makes it possible to monitor, record and analyze the drill process in real time, thereby providing strong data support for the evaluation and improvement of drills.

[0004] In the practical application of machine vision technology, multi-target recognition and skeleton extraction are two core links. At present, the traditional methods commonly used in the industry face many challenges in dealing with these tasks. For example, changes in lighting conditions, mutual occlusion between targets, and interference from complex backgrounds may affect the accuracy of recognition and extraction. Especially when performing skeleton extraction, traditional methods often produce skeleton line clusters with larger widths, which not only reduces the accuracy of recognition, but may also mislead subsequent data analysis.

[0005] However, although the application of machine vision technology in emergency drill effectiveness monitoring has made significant progress, there are still defects such as poor target recognition and poor skeleton extraction accuracy. Summary of the invention

[0006] The present invention provides a machine vision-based emergency drill effectiveness monitoring method, device and equipment, which are used to solve the problem of poor accuracy in multi-target recognition and skeleton extraction in emergency drill effectiveness monitoring in the prior art, and achieve improvement in skeleton extraction recognition accuracy and stability in complex environments.

[0007] The present invention provides a method for monitoring the effectiveness of emergency drills based on machine vision, comprising the following steps: Acquire an original image of an emergency drill scene, and preprocess the original image of the emergency drill scene to obtain an emergency drill scene image; Performing skeleton extraction on the emergency drill scene image to determine skeleton lines, and determining skeleton nodes according to the skeleton lines; Analyze the extracted skeleton nodes, and correct the offset skeleton nodes according to the analysis results; Reconnecting gaps and breaks in the skeleton structure according to the corrected skeleton node position information to obtain a final skeleton structure; The drill situation is monitored and analyzed based on the skeleton structure.

[0008] According to the machine vision-based emergency drill effectiveness monitoring method provided by the present invention, the skeleton extraction of the emergency drill scene image is performed to determine the skeleton line, and the skeleton nodes are determined according to the skeleton line, which specifically includes: acquiring the emergency drill scene image, performing gradient calculation on the emergency drill scene image through a differential edge unbiased operator to obtain the gradient of each pixel in the image in the horizontal and vertical directions; determining the differential edge unbiased skeleton operator of each pixel according to the gradient of each pixel in the horizontal and vertical directions; determining the skeleton line according to the differential edge unbiased skeleton operator; and determining the position of the skeleton node by calculating the distance function value from the pixel point on the skeleton line to other skeleton lines.

[0009] According to the machine vision-based emergency drill effectiveness monitoring method provided by the present invention, the position of the skeleton node is determined by calculating the distance function value from the pixel point on the skeleton line to other skeleton lines, which specifically includes: traversing each skeleton line in the emergency drill scene image, calculating the first shortest distance from each pixel point on the current skeleton line to all other skeleton lines; traversing the second shortest distance from the neighborhood pixel point of each pixel point on the current skeleton line to all other skeleton lines; if the first shortest distance is not less than the second shortest distance, marking the pixel point on the current skeleton line corresponding to the first shortest distance as a skeleton node.

[0010] According to the machine vision-based emergency drill effectiveness monitoring method provided by the present invention, the extracted skeleton nodes are analyzed, and the offset skeleton nodes are corrected according to the analysis results, specifically including: obtaining the position, connection angle and relative distance of each skeleton node with its adjacent skeleton nodes; based on geometric and topological rules, judging whether the current skeleton node is an offset node according to the position, connection angle and relative distance of each skeleton node with its adjacent skeleton nodes; determining a correction strategy according to the offset node, and correcting the skeleton node according to the correction strategy.

[0011] According to the machine vision-based emergency drill effectiveness monitoring method provided by the present invention, the gaps and breaks in the skeleton structure are reconnected according to the position information of the corrected skeleton nodes to obtain the final skeleton structure, which specifically includes: obtaining the position information and connection relationship of the skeleton nodes; identifying the gaps and breaks in the skeleton line according to the position information and connection relationship of the skeleton nodes; determining a corresponding reconnection strategy according to the gaps or breaks; and reconnecting the gaps and breaks according to the reconnection strategy.

[0012] According to the machine vision-based emergency drill effectiveness monitoring method provided by the present invention, the preprocessing of the original image of the emergency drill scene to obtain the emergency drill scene image specifically includes: denoising the original image of the emergency drill scene; performing image enhancement on the denoised image; performing edge detection on the enhanced image; and using the image obtained after edge detection as the emergency drill scene image.

[0013] According to the machine vision-based emergency drill effectiveness monitoring method provided by the present invention, the skeleton node is an intersection point, branch point or endpoint on the skeleton line; the intersection point is a point where two or more skeleton lines intersect or overlap; the branch point is a position where multiple branches are separated from the skeleton line; the endpoint is the starting point or ending point of the skeleton line.

[0014] The present invention also provides an emergency drill effectiveness monitoring device based on machine vision, comprising the following modules: An image preprocessing module is used to obtain an original image of an emergency drill scene, and preprocess the original image of the emergency drill scene to obtain an emergency drill scene image; A skeleton extraction module, used to perform skeleton extraction on the emergency drill scene image, determine skeleton lines, and determine skeleton nodes according to the skeleton lines; An offset correction module is used to analyze the extracted skeleton nodes and correct the offset skeleton nodes according to the analysis results; A reconnection module, used to reconnect the gaps and breaks in the skeleton structure according to the corrected position information of the skeleton nodes to obtain a final skeleton structure; The monitoring and analysis module is used to monitor and analyze the drill situation according to the skeleton structure.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, an emergency drill effectiveness monitoring method based on machine vision as described above is implemented.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements any of the above-mentioned emergency drill effectiveness monitoring methods based on machine vision.

[0017] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned emergency drill effectiveness monitoring methods based on machine vision.

[0018] The present invention provides a machine vision-based emergency drill effectiveness monitoring method, device and equipment, which can significantly improve the accuracy of drill situation monitoring by extracting the skeleton of the emergency drill scene image, correcting the offset skeleton nodes, and reconnecting the gaps and breaks in the skeleton structure. By using machine vision technology, image preprocessing, skeleton extraction, node analysis and correction, and reconnection of the skeleton structure can be automatically completed, greatly reducing manual intervention and improving the degree of automation of monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flow chart of the emergency drill effectiveness monitoring method based on machine vision provided by the present invention.

[0021] Figure 2 It is a structural schematic diagram of the emergency drill effectiveness monitoring device based on machine vision provided by the present invention.

[0022] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The existing technology has the following defects: 1. Subjectivity and reliance on manual evaluation: Existing emergency drill effectiveness monitoring methods often rely on subjective evaluations by experts and use subjective evaluation indicators to measure the effectiveness of drills. This method is easily affected by individual subjective factors, and the evaluation results may have subjective biases.

[0025] 2. Lack of real-time and timely feedback: Existing emergency drill effectiveness monitoring methods often take a long time to collect, process and analyze data, resulting in delayed feedback of evaluation results. This delay reduces the timely monitoring and adjustment of drill effectiveness and cannot provide real-time feedback and guidance.

[0026] 3. Lack of precise behavioral analysis: Existing methods have limitations in analyzing participants’ behaviors. They may only be able to detect and track some simple movements or position changes, but cannot provide more in-depth behavioral assessments, such as the accuracy, coordination, and timing of movements.

[0027] 4. Lack of automation and intelligence: Existing methods may still rely on manual operation and judgment in emergency drill effectiveness monitoring, lacking automation and intelligence. This may increase the demand for human resources and also limit the efficiency and accuracy of monitoring methods.

[0028] 5. Poor environmental adaptability: Existing methods are greatly affected by on-site environmental conditions, and monitoring accuracy is easily reduced in complex emergency drill scenarios. Machine vision methods can better cope with complex factors such as lighting changes and occlusions by optimizing algorithms, thereby enhancing the environmental adaptability of the overall system.

[0029] In view of the above-mentioned defects, the present invention proposes a method for monitoring the effectiveness of emergency drills based on machine vision, aiming to solve the problems of insufficient accuracy and poor stability of existing machine vision technology in multi-target recognition and skeleton extraction in emergency drill effectiveness monitoring. Specifically, it is necessary to solve the problems of excessive skeleton line width and position offset generated by traditional differential edge detection methods, as well as to improve the recognition accuracy and stability in complex environments. The method mainly uses a differential edge unbiased skeleton operator for skeleton extraction. By checking the distance function value along the skeleton line, the biased influence of the local maximum value of the distance function on the formation of an ideal skeleton is solved. This method ensures that after the skeleton of multiple targets is offset, the odd pixel width can be reduced from the two pixel width of the conventional method to the unit width, and is accurately located in the correct position to achieve proper reconnection.

[0030] Combine the following Figure 1-Figure 3 Embodiments of the present invention are described.

[0031] Figure 1 is one of the flow charts of the emergency drill effectiveness monitoring method based on machine vision provided by the present invention, such as Figure 1As shown, the method includes the following: S110: Acquire an original image of an emergency drill scene, and pre-process the original image of the emergency drill scene to obtain an image of the emergency drill scene.

[0032] According to the machine vision-based emergency drill effectiveness monitoring method provided by the present invention, the original image of the emergency drill scene is preprocessed to obtain the emergency drill scene image, which specifically includes: denoising the original image of the emergency drill scene; performing image enhancement on the denoised image; performing edge detection on the enhanced image; and using the image obtained after edge detection as the emergency drill scene image.

[0033] Specifically, a high-definition camera or other image capture device is used to capture the data of the original image of the emergency drill scene at the emergency drill site. The image acquisition device of the present invention is in a fixed position, focusing on the object to be identified being in motion, and the group is disordered and non-directional, so it is difficult to capture the skeleton in real time. If the skeleton feature extraction is not performed with a single pixel width, a large amount of aliasing and misjudgment will occur. These image data generally contain various noises and unnecessary details, so preprocessing is required to improve the image quality.

[0034] Apply image processing algorithms, such as median filtering, Gaussian filtering or bilateral filtering, to remove noise from the image. These filters can effectively reduce salt and pepper noise, Gaussian noise, etc. in the image, making the image clearer.

[0035] After denoising, image enhancement is performed to improve the visual effect of the image, for example by adjusting the contrast, brightness or applying techniques such as histogram equalization. This can highlight the useful information in the image and make subsequent image analysis more accurate.

[0036] Use algorithms such as Canny edge detection, Sobel operator, Prewitt operator or Laplacian operator to detect edges of the enhanced image. Edge detection is an important step in machine vision. It can highlight the edge information in the image and facilitate the subsequent recognition and tracking of the target object in the image.

[0037] After edge detection, the edge information of the image is enhanced, which helps to more accurately identify and track the target in subsequent machine vision analysis. The image obtained after edge detection is marked as an emergency drill scene image and used for further performance monitoring and analysis.

[0038] Through denoising and image enhancement processing, the image quality can be significantly improved, making the image clearer and easier to analyze in subsequent machine vision. Edge detection strengthens the edge information in the image, which is critical for identifying key objects and dynamics in emergency drill scenes, and improves the recognition accuracy of the machine vision system. High-quality pre-processed images provide a better data basis for subsequent emergency effectiveness drill monitoring, which can more accurately evaluate the effectiveness of the drill and promptly identify and correct problems. By pre-processing the image, the computational complexity of the machine vision system is reduced, thereby improving the real-time performance and response speed of the system.

[0039] S120 , performing skeleton extraction on the emergency drill scene image, determining skeleton lines, and determining skeleton nodes according to the skeleton lines.

[0040] According to the machine vision-based emergency drill effectiveness monitoring method provided by the present invention, skeleton extraction is performed on the emergency drill scene image to determine the skeleton line, and the skeleton nodes are determined according to the skeleton line. Specifically, the method includes: acquiring the emergency drill scene image, performing gradient calculation on the emergency drill scene image by a differential edge unbiased operator to obtain the gradient of each pixel in the image in the horizontal and vertical directions; determining the differential edge unbiased skeleton operator of each pixel according to the gradient of each pixel in the horizontal and vertical directions; determining the skeleton line according to the differential edge unbiased skeleton operator; and determining the position of the skeleton node by calculating the distance function value from the pixel on the skeleton line to other skeleton lines.

[0041] Specifically, the algorithm of the differential marginal unbiased operator is as follows: First, calculate the image gradient: use the standard Sobel operator

[0042] =

[0043] =

[0044] Where I is the gray value of the image, is the horizontal gradient; is the vertical gradient.

[0045] Then, the differential edge unbiased skeleton operator D(x, y) is calculated based on the gradients in both directions

[0046] D(x,y) =

[0047] This operator can eliminate skeleton deviation caused by local maximum.

[0048] By traversing each pixel (x, y) in the image, if D(x, y) , then mark the point as a point on the skeleton line. In this way, the complete skeleton line can be extracted.

[0049] According to the machine vision-based emergency drill effectiveness monitoring method provided by the present invention, the position of the skeleton node is determined by calculating the distance function value from the pixel point on the skeleton line to other skeleton lines, which specifically includes: traversing each skeleton line in the emergency drill scene image, calculating the first shortest distance from each pixel point on the current skeleton line to all other skeleton lines; traversing the second shortest distance from the neighborhood pixel point of each pixel point on the current skeleton line to all other skeleton lines; if the first shortest distance is not less than the second shortest distance, marking the pixel point on the current skeleton line corresponding to the first shortest distance as a skeleton node.

[0050] According to the machine vision-based emergency drill effectiveness monitoring method provided by the present invention, a skeleton node is an intersection point, a branch point or an endpoint on a skeleton line; an intersection point is a point where two or more skeleton lines intersect or overlap; a branch point is a location where multiple branches are separated from a skeleton line; and an endpoint is a starting point or an ending point of a skeleton line.

[0051] Specifically, a high-definition camera or other image capture device is used to obtain real-time images of the emergency drill scene. Appropriate differential operators (such as Sobel, Prewitt, Laplacian, etc.) are selected to perform convolution operations on the acquired images. Through the convolution operation, the edge information in the image is enhanced to obtain the differential operation result. Based on the differential operation result, the skeleton line of the target is extracted. The skeleton line can be regarded as a simplified representation of the target shape, which retains the important topological structure of the shape. The differential edge unbiased skeleton operator can effectively deal with the local maximum problem of the distance function and avoid the formation of an ideal skeleton bias.

[0052] After we have the skeleton line, we can extract the specific location of the skeleton nodes. This can be achieved by checking the distance function value along the skeleton line: The distance here refers to the Euclidean distance between two coordinate points: =

[0053] Traverse each skeleton line in the image and store the coordinates of all points on it in a list. Each element is a list containing (x, y) coordinates. For each point (x, y) on the skeleton line, calculate its shortest distance d(x, y) to all other skeleton lines. Traverse the neighborhood pixels of (x, y), including its 8 neighborhood pixels above, below, left, right, and diagonal lines. Check whether d(x, y) is greater than or equal to the d value of the other 8 pixels in these 9 pixels. If d(x, y) is indeed the maximum value among these 9 pixels, it can be considered a local maximum point. Mark the coordinate (x, y) marked as a local maximum as a skeleton node.

[0054] Based on the connection of the skeleton nodes, identify intersections, branch points, and endpoints. An intersection is a point where two or more skeleton lines intersect or overlap. A branch point is a location where multiple branches are separated from a skeleton line. An endpoint is the starting point or ending point of a skeleton line.

[0055] Through differential operators and edge detection algorithms, the edges of the target can be accurately extracted, and then accurate skeleton lines can be obtained. The use of unbiased skeleton operators can efficiently extract skeleton lines, and quickly locate skeleton nodes by calculating distance function values. In emergency effectiveness drills, real-time monitoring and analysis of drill scenes through machine vision technology can provide strong data support for the evaluation and improvement of drills. This method is not only suitable for the analysis of static images, but can also be extended to the processing of video streams to achieve real-time monitoring and analysis.

[0056] S130, analyzing the extracted skeleton nodes, and correcting the offset skeleton nodes according to the analysis results.

[0057] According to the machine vision-based emergency drill effectiveness monitoring method provided by the present invention, the extracted skeleton nodes are analyzed, and the offset skeleton nodes are corrected according to the analysis results, specifically including: obtaining the position, connection angle and relative distance of each skeleton node with its adjacent skeleton nodes; based on geometric and topological rules, judging whether the current skeleton node is an offset node according to the position, connection angle and relative distance of each skeleton node with its adjacent skeleton nodes; determining a correction strategy according to the offset node, and correcting the skeleton node according to the correction strategy.

[0058] Specifically, for each skeleton node, its precise position (coordinates), connection angle with adjacent nodes, and relative distance are recorded.

[0059] First, assuming there is a current skeleton node P(x1, y1), calculate the relative distance and connection angle between adjacent skeleton nodes. The distance here refers to the Euclidean distance between two coordinates. The calculation formula is as follows: =

[0060] Then calculate the connection angles of adjacent skeleton nodes: Calculate the angle θ between the current node and the two adjacent skeleton nodes Q(x2, y2) and R(x3, y3), and calculate the angle θ between the vectors PQ and PR.

[0061] Vector PQ = (x2- x1, y2- y1), vector PR = (x3- x1, y3- y1).

[0062] Calculate the magnitude of a vector: |PQ| =

[0063] |PR| =

[0064] Compute the vector dot product: PQ · PR = (x2- x1) * (x3- x1) + (y2- y1) * (y3- y1)

[0065] Calculate the angle θ using vector dot product and modulus length: θ = arccos((PQ · PR) / (|PQ| * |PR|)).

[0066] Using geometric and topological rules, the position, angle and distance of each skeleton node are analyzed. Each normal skeleton node should meet the following geometric and topological characteristics: 1. The angle change between adjacent nodes should not be too large, usually less than a certain threshold θ max

[0067] 2. The distance between adjacent nodes should not be too large or too small, but should be within a reasonable range [d min , d max ] Inside; 3. If a node does not meet the above rules, it can be considered as an offset node.

[0068] If there are offset nodes, the offset nodes can be corrected by adjusting the angle and distance between the nodes to satisfy the set and topological characteristics.

[0069] By comparing the relative positions and angles between adjacent nodes, abnormal nodes that do not conform to normal human posture or movement can be detected and used as offset nodes. For the detected offset nodes, corresponding correction strategies are formulated according to the degree and direction of their offset. Correction strategies include adjusting the position of the node, recalculating the connection angle, or correcting the relative distance with the adjacent nodes. According to the determined correction strategy, the offset skeleton nodes are corrected to accurately reduce the original two-pixel-wide bone clusters to a unit pixel width. During the correction process, the position and influence of the surrounding nodes need to be considered to ensure the coherence and accuracy of the overall skeleton. After the correction is completed, the correction effect is verified by comparing the skeleton structure before and after correction.

[0070] By correcting the offset nodes, the accuracy of skeleton extraction can be significantly improved, providing a more reliable data basis for subsequent human posture analysis, action recognition, etc. By reducing the skeleton width to unit pixels, the measurement of skeleton width is more accurate, which helps to improve the overall performance of the machine vision system. Through machine vision technology, the single pixel width objectively records the actual physical reaction of each exercise participant, objectively evaluates actual combat exercises and desktop exercises, and evaluates the effectiveness of different types of exercises in combination with each other. For multi-target scenarios, this solution can effectively handle the problem of skeleton offset and enhance the robustness of the system.

[0071] S140. Reconnect the gaps and breaks in the skeleton structure according to the corrected position information of the skeleton nodes to obtain a final skeleton structure.

[0072] According to the machine vision-based emergency drill effectiveness monitoring method provided by the present invention, the gaps and breaks in the skeleton structure are reconnected according to the position information of the corrected skeleton nodes to obtain the final skeleton structure, which specifically includes: obtaining the position information and connection relationship of the skeleton nodes; identifying the gaps and breaks in the skeleton line according to the position information and connection relationship of the skeleton nodes; determining the corresponding reconnection strategy according to the gaps or breaks; and reconnecting the gaps and breaks according to the reconnection strategy.

[0073] Specifically, the skeleton nodes in the image are identified and extracted, and the position coordinates of each node are recorded. At the same time, according to the connection mode of the skeleton nodes in the image, the connection relationship between the skeleton nodes is determined to form a complete skeleton line.

[0074] Using the position information and connection relationship of the skeleton nodes, detect whether there are gaps or breaks in the skeleton line. A gap refers to a missing part of the skeleton line, that is, there is no connection between two nodes that should be connected. A break refers to a skeleton node that should have been a continuous node being mistakenly identified as two adjacent nodes, and the distance between the two nodes exceeds the preset threshold.

[0075] According to the detected gaps or breaks, a corresponding reconnection strategy is formulated. For gaps, the missing parts can be filled by interpolation or other image processing techniques. For breaks, if the distance between two adjacent nodes is slightly above the threshold, the two nodes can be reconnected; if the distance is larger, other algorithms can be used to determine how to connect.

[0076] According to the determined reconnection strategy, the gaps are filled or the breaks are connected. During the reconnection process, the positions of nodes are adjusted or new nodes are added to ensure the continuity and smoothness of the skeleton line.

[0077] Specifically, for each pair of adjacent nodes (x i , y i ) and ( , ): The distance between two nodes is: d i =

[0078] If d i >Preset distance threshold d th It is considered that there is a fracture; if d i <d th , but if there is an unconnected gap between two nodes, it is considered that there is a gap.

[0079] For the identified gap, new nodes are inserted into the gap using an interpolation algorithm based on the position information of the nodes at both ends of the gap to fill the gap. Linear interpolation can be used.

[0080] x, y: coordinates of the new endpoint to be optimized

[0081] x' = + t * ( - )

[0082] y' = + t * ( - )

[0083] Where t is the interpolation parameter, and its value range is [0, 1]

[0084] For the identified fractures, determine whether they can be directly connected based on the positions and connection information of the nodes at both ends of the fracture. If they cannot be directly connected, an optimization algorithm can be used to recalculate the positions of the nodes at both ends of the fracture so that they can be successfully connected: f(x, y)= + The objective function represents the sum of the squared distances between the new endpoint coordinates and the original endpoint coordinates, and we hope to minimize this value.

[0085] The constraints must be met <d th , this constraint requires that the distance between the new endpoint and the original endpoint i+1 is less than the preset threshold d th .

[0086] The algorithm has the ability and strategy to reconnect gaps and breaks in the skeleton structure, capture the skeleton positions of emergency drills and evacuees at different movement speeds and directions, and predict and splice strategies in real time as occlusion, disappearance, and appearance occur, with good stability.

[0087] By reconnecting gaps and breaks, the problem of incomplete skeleton structure caused by image quality, occlusion or other factors can be repaired, thereby improving the accuracy of skeleton extraction. In practical applications, machine vision systems often encounter various complex scenes and interference factors. Through the processing of this solution, the system can better adapt to these situations and improve robustness. Accurate skeleton structure is the basis for subsequent target tracking, behavior analysis, posture recognition and other tasks. The implementation of this solution can provide more reliable data support for these tasks.

[0088] S150. Monitor and analyze the drill situation based on the skeleton structure.

[0089] Specifically, the extracted skeleton structure is time-synchronized with the drill video or real-time video stream to ensure that the skeleton data of each frame corresponds to the video content. According to the purpose and requirements of the drill, a series of monitoring indicators are defined, such as action completion, action standardization, reaction time, etc. The detection indicators are calculated through parameters such as node position, speed and acceleration in the skeleton structure. During the drill, the changes in the skeleton structure are monitored in real time, and the data related to the monitoring indicators are recorded. Abnormal or critical events (such as falls, rapid movements, etc.) are specially marked and recorded.

[0090] After the drill, the collected skeleton data is analyzed in depth to evaluate the participants’ performance by comparing them with preset standard movements or behavior patterns, and to analyze the participants’ movement fluency, coordination, and reaction speed during the drill.

[0091] By analyzing the drill situation through skeleton structure data, the performance of participants can be evaluated more objectively and the influence of subjective judgment can be reduced. Based on the analysis results of skeleton data, detailed guidance and suggestions can be provided to participants to help them master the essentials of movements and drill techniques more accurately. Through real-time monitoring and feedback, errors or deficiencies in drills can be discovered and corrected in a timely manner, improving drill efficiency while reducing safety risks caused by irregular movements. In-depth analysis of skeleton data can help discover potential problems and improvement points in the drill process, thereby promoting innovation and improvement of related technologies.

[0092] The following is a description of the emergency drill effectiveness monitoring device based on machine vision provided by the present invention. The emergency drill effectiveness monitoring device based on machine vision described below and the emergency drill effectiveness monitoring method based on machine vision described above can be referenced to each other.

[0093] A machine vision-based emergency drill effectiveness monitoring method, such as Figure 2 As shown, it includes the following modules: An image preprocessing module 210 is used to obtain an original image of an emergency drill scene, and preprocess the original image of the emergency drill scene to obtain an emergency drill scene image; A skeleton extraction module 220 is used to perform skeleton extraction on the emergency drill scene image, determine skeleton lines, and determine skeleton nodes according to the skeleton lines; The offset correction module 230 is used to analyze the extracted skeleton nodes and correct the offset skeleton nodes according to the analysis results; A reconnection module 240, configured to reconnect gaps and breaks in the skeleton structure according to the corrected position information of the skeleton nodes to obtain a final skeleton structure; The monitoring and analysis module 250 is used to monitor and analyze the drill situation according to the skeleton structure.

[0094] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the emergency drill effectiveness monitoring method based on machine vision, the method comprising: obtaining the original image of the emergency drill scene, pre-processing the original image of the emergency drill scene to obtain the emergency drill scene image; performing skeleton extraction on the emergency drill scene image, determining the skeleton line, and determining the skeleton node according to the skeleton line; analyzing the extracted skeleton nodes, and correcting the offset skeleton nodes according to the analysis results; reconnecting the gaps and breaks in the skeleton structure according to the position information of the corrected skeleton nodes to obtain the final skeleton structure; monitoring and analyzing the drill situation according to the skeleton structure.

[0095] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0096] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the machine vision-based emergency drill effectiveness monitoring method provided by the above-mentioned methods, the method including: acquiring an original image of the emergency drill scene, preprocessing the original image of the emergency drill scene, and obtaining an emergency drill scene image; performing skeleton extraction on the emergency drill scene image, determining skeleton lines, and determining skeleton nodes based on the skeleton lines; analyzing the extracted skeleton nodes, and correcting the offset skeleton nodes based on the analysis results; reconnecting the gaps and breaks in the skeleton structure based on the position information of the corrected skeleton nodes to obtain the final skeleton structure; and monitoring and analyzing the drill situation based on the skeleton structure.

[0097] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the machine vision-based emergency drill effectiveness monitoring method provided by the above-mentioned methods, the method comprising: acquiring an original image of the emergency drill scene, preprocessing the original image of the emergency drill scene, and obtaining an emergency drill scene image; performing skeleton extraction on the emergency drill scene image, determining skeleton lines, and determining skeleton nodes based on the skeleton lines; analyzing the extracted skeleton nodes, and correcting the offset skeleton nodes based on the analysis results; reconnecting the gaps and breaks in the skeleton structure based on the position information of the corrected skeleton nodes to obtain a final skeleton structure; and monitoring and analyzing the drill situation based on the skeleton structure.

[0098] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0099] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on such an understanding, the above technical solution can essentially or in other words be embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for monitoring the effectiveness of emergency drills based on machine vision, characterized in that: include: Acquire an original image of an emergency drill scene, and preprocess the original image of the emergency drill scene to obtain an emergency drill scene image; Performing skeleton extraction on the emergency drill scene image to determine skeleton lines, and determining skeleton nodes according to the skeleton lines; Analyze the extracted skeleton nodes, and correct the offset skeleton nodes according to the analysis results; Reconnecting gaps and breaks in the skeleton structure according to the corrected skeleton node position information to obtain a final skeleton structure; The drill situation is monitored and analyzed based on the skeleton structure.

2. The method for monitoring the effectiveness of emergency drills based on machine vision according to claim 1, characterized in that: The step of extracting a skeleton from the emergency drill scene image, determining a skeleton line, and determining a skeleton node according to the skeleton line specifically includes: Acquire the emergency drill scene image, perform gradient calculation on the emergency drill scene image by using a differential edge unbiased operator, and obtain the gradient of each pixel in the image in the horizontal direction and the vertical direction; Determine the differential edge unbiased skeleton operator of each pixel point according to the gradient of each pixel point in the horizontal direction and the vertical direction; determining a skeleton line according to the differential edge unbiased skeleton operator; The position of the skeleton node is determined by calculating the distance function value from the pixel points on the skeleton line to other skeleton lines.

3. The method for monitoring the effectiveness of emergency drills based on machine vision according to claim 2, characterized in that: The method of determining the position of the skeleton node by calculating the distance function value from the pixel point on the skeleton line to other skeleton lines specifically includes: Traversing each skeleton line in the emergency drill scene image, and calculating the first shortest distance from each pixel point on the current skeleton line to all other skeleton lines; Traverse the second shortest distance from the neighboring pixel of each pixel on the current skeleton line to all other skeleton lines; If the first shortest distance is not less than the second shortest distance, the pixel point on the current skeleton line corresponding to the first shortest distance is marked as a skeleton node.

4. The method for monitoring the effectiveness of emergency drills based on machine vision according to claim 1, characterized in that: The step of analyzing the extracted skeleton nodes and correcting the offset skeleton nodes according to the analysis results specifically includes: Get the position, connection angle and relative distance of each skeleton node to its adjacent skeleton nodes; Based on geometric and topological rules, judging whether the current skeleton node is an offset node according to the position, connection angle and relative distance of each skeleton node to its adjacent skeleton nodes; A correction strategy is determined according to the offset nodes, and the skeleton nodes are corrected according to the correction strategy.

5. The method for monitoring the effectiveness of emergency drills based on machine vision according to claim 1, characterized in that: The step of reconnecting gaps and breaks in the skeleton structure according to the corrected position information of the skeleton nodes to obtain the final skeleton structure specifically includes: Get the location information and connection relationship of skeleton nodes; According to the position information and connection relationship of the skeleton nodes, identifying gaps and breaks in the skeleton line; Determine the appropriate reconnection strategy based on the gap or break; Gaps and breaks are reconnected according to the reconnection strategy.

6. The method for monitoring the effectiveness of emergency drills based on machine vision according to claim 1, characterized in that: The preprocessing of the original image of the emergency drill scene to obtain the emergency drill scene image specifically includes: Performing denoising processing on the original image of the emergency drill scene; Perform image enhancement on the denoised image; Perform edge detection on the image after image enhancement; The image obtained after edge detection is used as the emergency drill scene image.

7. The method for monitoring the effectiveness of emergency drills based on machine vision according to claim 1, characterized in that: The skeleton nodes are intersections, branch points or endpoints on the skeleton lines; The intersection point is a point where two or more skeleton lines intersect or overlap; The branch point is a position where multiple branches are separated from the skeleton line; The endpoint is the starting point or the ending point of the skeleton line.

8. An emergency drill effectiveness monitoring device based on machine vision, characterized in that: include: An image preprocessing module is used to obtain an original image of an emergency drill scene, and preprocess the original image of the emergency drill scene to obtain an emergency drill scene image; A skeleton extraction module, used to perform skeleton extraction on the emergency drill scene image, determine skeleton lines, and determine skeleton nodes according to the skeleton lines; An offset correction module is used to analyze the extracted skeleton nodes and correct the offset skeleton nodes according to the analysis results; A reconnection module, used to reconnect the gaps and breaks in the skeleton structure according to the corrected position information of the skeleton nodes to obtain a final skeleton structure; The monitoring and analysis module is used to monitor and analyze the drill situation according to the skeleton structure.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the machine vision-based emergency drill effectiveness monitoring method as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for monitoring the effectiveness of emergency drills based on machine vision as described in any one of claims 1 to 7 is implemented.

Images