An intelligent inspection method and system for a special operation physical examination site

Abstract

The application discloses an intelligent inspection method for a special operation physical examination room, and relates to the field of path planning.The method comprises the following steps: acquiring examination room data and environmental parameters in real time, and establishing a real scene navigation map based on the examination room data, wherein the real scene navigation map is used for real-time linkage with the examination room data; acquiring examination room images in real time through an inspection device; extracting image features of the examination room images, and dividing an inspection path into n regions based on the image features, wherein n is a positive integer; based on the real scene navigation map and the image features, a dynamic path planning strategy is executed to generate an inspection path of the inspection device; the environmental parameters and the image features are fused and analyzed to determine a congestion index on the inspection path; based on the congestion index, a dangerous region is determined in the n regions, and the dangerous region is visualized. The application can effectively improve the accuracy of inspection on the special operation physical examination room, thereby effectively reducing the risk hidden danger of the examination room.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of path planning, and in particular to an intelligent inspection method and system for a special operation physical examination room. BACKGROUND

[0002] The inspection work of a special operation examination room (such as a high-risk industry such as electric power, chemical industry, and construction) is of great significance to ensure operation safety and prevent accidents. The traditional inspection method mainly relies on inspection personnel to periodically check the examination room equipment and environment to ensure that they meet the safety standards.

[0003] Specifically, the inspection personnel need to check the equipment and environment in the examination room one by one according to a fixed route. The accuracy of manual inspection is highly dependent on the experience and skills of the inspection personnel. However, as the scale of the examination room expands, the complexity of the equipment increases, and the safety requirements improve, especially in high-risk operation environments such as high-altitude operations and welding operations, in order to avoid safety hazards to personnel, manual inspection often fails to fully cover the examination room area, resulting in inspection blind spots. If a problem occurs in a device in the examination room, not only does it reduce the accuracy of manual inspection, but it also poses a risk to the personnel in the examination room.

[0004] Currently, there is no effective solution to the above problems. SUMMARY

[0005] Embodiments of the present application provide an intelligent inspection method and system for a special operation physical examination room, which effectively improves the accuracy of inspection of the special operation physical examination room, thereby effectively reducing the risk of the examination room.

[0006] To achieve the above object, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, an intelligent inspection method for a special operation physical examination room is provided, applied to a central control device, the method comprising:

[0008] real-time acquisition of examination room data and environmental parameters, and establishment of a real scene navigation map based on the examination room data, wherein the real scene navigation map is used for real-time linkage with the examination room data;

[0009] real-time acquisition of examination room images by an inspection device, wherein the inspection device is wirelessly or wiredly connected to the central control device, the inspection device is provided with a camera, and the camera is used for collecting the examination room images;

[0010] extraction of image features of the examination room images, and division of an inspection path into n regions based on the image features, n being a positive integer;

[0011] based on the real scene navigation map and the image features, a dynamic path planning strategy is executed to generate an inspection path of the inspection device;

[0012] fusing the environmental parameters with the image features to determine a congestion index on the inspection path;

[0013] based on the congestion index, determining a dangerous area in the n areas and visualizing the dangerous area.

[0014] In a possible implementation manner of the first aspect, the examination room data includes three-dimensional space data, and the real-scene navigation map is established based on the examination room data, including:

[0015] converting the three-dimensional space data into a grid map;

[0016] adopting a pre-constructed neural network model to perform semantic segmentation on the three-dimensional space data to obtain a semantic segmentation result;

[0017] fusing the semantic segmentation result with the grid map to obtain the real-scene navigation map including semantic information.

[0018] In another possible implementation manner of the first aspect, the dynamic path planning strategy includes:

[0019] obtaining historical inspection data;

[0020] calculating a global optimal path based on an improved A* algorithm, wherein the improved A* algorithm is improved based on a heuristic function;

[0021] monitoring a crowd density and an obstacle distribution on the inspection path in real time;

[0022] determining whether there is a path blocking area and / or a dangerous area in the global optimal path based on the crowd density and the obstacle distribution;

[0023] in a case where there is a path blocking and / or a dangerous area in the global optimal path, performing a local path re-planning strategy and generating a final path;

[0024] wherein the local path re-planning strategy includes:

[0025] identifying boundary coordinates of the path blocking area and / or the dangerous area;

[0026] assigning a high passing value to a grid node of the path blocking area and / or the dangerous area in the real-scene navigation map;

[0027] adopting a D*Lite algorithm to calculate a local optimal path with a current position as a starting point and a target inspection point as an ending point, wherein the target inspection point is a grid node closest to the grid node of the path blocking area and / or the dangerous area outside the boundary coordinates;

[0028] When multiple continuous path blockage areas and / or dangerous areas are detected, a segmented planning strategy is adopted to divide the path into multiple sub-paths;

[0029] An ant colony optimization algorithm is applied to each sub-path to generate a locally optimal sub-path;

[0030] All locally optimal sub-paths are connected to obtain a locally re-planned path.

[0031] In another possible implementation of the first aspect, based on the real scene navigation map and the image features, a dynamic path planning strategy is performed to generate the inspection path of the inspection device, comprising:

[0032] The real scene navigation map is divided into multiple grid nodes, and the passage cost value of each grid node is calculated;

[0033] In response to a setting instruction of the user on the real scene navigation map, an inspection point in the real scene navigation map and a weight coefficient corresponding to each inspection point are determined;

[0034] An improved A* algorithm is used to calculate an initial global path;

[0035] The real-time acquired environmental parameters and image features are used as input parameters of a pre-constructed multi-feature fusion model to dynamically update the passage cost value of each grid node;

[0036] When the change amount of the passage cost value exceeds a preset threshold, a preset path re-planning algorithm is used to generate a new inspection path;

[0037] When the change amount of the passage cost value does not exceed the preset threshold, the initial global path is used as the inspection path of the inspection device.

[0038] In another possible implementation of the first aspect, based on the image features, the inspection path is divided into n regions, comprising:

[0039] Based on a K-means clustering algorithm, the image features are clustered to obtain at least one clustering region;

[0040] The feature vectors of each clustering region are calculated, and the similarity between the feature vectors is calculated;

[0041] According to the similarity between the feature vectors, the inspection path is divided into n continuous regions, wherein each region is assigned a unique identifier.

[0042] In another possible implementation of the first aspect, the environmental parameters and the image features are fused and analyzed to determine the congestion index on the inspection path, comprising:

[0043] The personnel density, the equipment distribution and the operation behavior are extracted from the image features by the YOLOv8 model;

[0044] The congestion index is output by a multi-layer perceptron neural network model, wherein an input layer of the multi-layer perceptron neural network model includes the environment parameter, the personnel density, the equipment distribution and the operation behavior, and an output layer is the congestion index.

[0045] In a possible implementation manner of the first aspect, the dangerous area is determined in the n areas based on the congestion index, including:

[0046] When the congestion index of the area is lower than a preset first congestion index threshold T1, the area is marked as a safe area;

[0047] When the congestion index of the area is between the first congestion index threshold T1 and a second congestion index threshold T2, the area is marked as a warning area;

[0048] When the congestion index of the area is higher than the second congestion index threshold T2, the area is marked as a dangerous area, wherein 0 < T1 < T2 < 100.

[0049] In a possible implementation manner of the first aspect, the camera is further configured to collect the examination room video, the examination room video including a plurality of video frames, and the method further includes:

[0050] The video frame is input into a pre-constructed YOLOv8 model to identify a predefined type of violation behavior;

[0051] For each type of violation behavior, a start time and an end time of the violation behavior are recorded;

[0052] In a preset time interval, key video frames are extracted at a frequency of m frames per second, and a timestamp, a violation type label and a violation area bounding box are added to each key video frame;

[0053] The key video frames are sorted in chronological order, a violation behavior evidence package is generated, the violation behavior evidence package is stored in a security database of the central control device, and a violation behavior index of the security database is constructed.

[0054] In a possible implementation manner of the first aspect, the inspection device transmits the examination room image to the central control device, including:

[0055] The inspection device performs H.264 encoding compression on the examination room image, establishes an RTMP streaming channel, and pushes the encoded data stream to the central control device;

[0056] The encoded data stream is subjected to AES-256 encryption to obtain an encrypted data stream;

[0057] When the inspection device detects a network interruption, the encrypted data stream is stored in a local cache, and after detecting network recovery, the encrypted data stream is transmitted to the control device.

[0058] In a second aspect, the application provides an inspection system, comprising:

[0059] The control device is used for executing the intelligent inspection method for the special operation physical examination room described above.

[0060] The inspection device is wirelessly or wiredly connected to the control device.

[0061] Through the above technical solution, the control device acquires examination room data and environmental parameters in real time, and establishes a real scene navigation map based on the examination room data. The real scene navigation map is in real-time linkage with the examination room data, providing accurate spatial positioning and navigation support for inspection. Secondly, the inspection device acquires examination room images in real time, and the inspection device is wirelessly or wiredly connected to the control device. The inspection device is provided with a camera for collecting examination room images, so that the inspection device can quickly cover all areas of the examination room, avoiding the time-consuming problem of manual inspection. At the same time, by extracting image features of the examination room images and dividing the inspection path into n regions based on the image features, n is a positive integer, the inspection can comprehensively cover the examination room area, further improving the inspection efficiency. In addition, based on the real scene navigation map and the image features, a dynamic path planning strategy is executed to generate an inspection path for the inspection device. This scheme can adjust the inspection route in real time according to the actual situation of the examination room, ensuring the comprehensiveness and pertinence of the inspection. At the same time, the environmental parameters and the image features are fused and analyzed to determine the congestion index on the inspection path, which can more accurately judge the state of the examination room, reduce the subjectivity and error of human judgment. Finally, based on the congestion index, a dangerous area is determined in the n regions, and the dangerous area is visualized. This can enable management personnel to quickly understand the state of the examination room and develop targeted improvement measures, thereby effectively preventing accidents. For high-risk areas, this scheme uses inspection devices (such as drones, robots, etc.) to enter the high-risk areas for inspection, avoiding the blind area problem of manual inspection, and significantly expanding the coverage range of the examination room inspection. In addition, this scheme uses image feature extraction technology to accurately identify changes in examination room equipment and environment. By fusing and analyzing the image features and environmental parameters, the state of the examination room can be more accurately judged, reducing the subjectivity and error of human judgment. Not only does it improve the efficiency and accuracy of inspection, but it also expands the coverage range of inspection, effectively reducing the risk of the examination room, thereby effectively preventing accidents and ensuring the safety of the examination room personnel.

[0062] Other features and advantages of the embodiments of the application will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1A flowchart of an intelligent inspection method for a special operation physical examination room provided by an embodiment of the present application is shown in the figure;

[0064] Figure 2 A structural diagram of an inspection system provided by an embodiment of the present application is shown in the figure;

[0065] Figure 3 A structural diagram of an inspection device provided by an embodiment of the present application is shown in the figure;

[0066] Figure 4 A setting diagram of a real scene navigation map inspection point provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0067] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. It should be understood that the specific embodiments described herein are merely used to explain and illustrate the embodiments of the present application and should not be used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort fall within the scope of protection of the present application.

[0068] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are merely used to explain the relative positional relationship, movement condition, etc. between components in a certain posture (as shown in the drawings), and if the certain posture changes, the directional indications also change accordingly.

[0069] In addition, if the embodiments of the present application involve descriptions of “first”, “second”, etc., the descriptions of “first”, “second”, etc. are merely for description purposes and should not be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features limited by “first” and “second” can explicitly or implicitly include at least one of the features. In addition, the technical solutions of the various embodiments can be combined with each other, but it must be based on the fact that a person of ordinary skill in the art can realize it, and when the combination of technical solutions appears to be contradictory or unachievable, it should be considered that the combination of technical solutions does not exist and is not within the scope of protection claimed by the present application.

[0070] The technical solutions of the present application will be described clearly and completely below in connection with the drawings and specific embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort fall within the scope of protection of the present application.

[0071] Figure 1 Fig. 1 shows a flowchart of an intelligent inspection method for a special operation physical examination room according to an embodiment of the present application. As shown in Fig. 1, the present application provides an intelligent inspection method for a special operation physical examination room, which is applied to a central control device. The method can include the following steps. Figure 1

[0072] S110, real-time acquisition of examination room data and environmental parameters, and establishment of a real scene navigation map based on the examination room data, wherein the real scene navigation map is used for real-time linkage with the examination room data;

[0073] S120, real-time acquisition of examination room images by an inspection device, wherein the inspection device is wirelessly or wiredly connected to the central control device, the inspection device is provided with a camera, and the camera is used for collecting examination room images;

[0074] S130, extraction of image features of the examination room images, and division of an inspection path into n regions based on the image features, wherein n is a positive integer;

[0075] S140, execution of a dynamic path planning strategy based on the real scene navigation map and the image features, to generate an inspection path of the inspection device;

[0076] S150, fusion analysis of the environmental parameters and the image features, to determine a congestion index on the inspection path;

[0077] S160, determination of a dangerous region in the n regions based on the congestion index, and visualization of the dangerous region.

[0078] Figure 2 Fig. 2 shows a structural diagram of an inspection system according to an embodiment of the present application. As shown in Fig. 2, the central control device and the inspection device are wirelessly or wiredly connected for communication. Hereinafter, the wireless connection is taken as an example. As shown in Fig. 2, the inspection system includes a central control device and an inspection device, and the central control device and the inspection device are wirelessly connected for communication. Figure 2 The inspection device is provided with a camera for collecting examination room images. The central control device acquires examination room data and environmental parameters, and establishes a real scene navigation map. The examination room images are transmitted to the central control device, image features are extracted, and an inspection path region is divided. Based on the real scene navigation map and the image features, a dynamic path planning is executed, to generate an inspection path. The environmental parameters and the image features are subjected to fusion analysis, to determine a congestion index. Based on the congestion index, a dangerous region is determined and visualized.

[0079] ​In this embodiment, the central control device can be a tablet computer, a desktop computer, a laptop computer, a handheld computer, a wearable device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, or the like device with a processor. Of course, the central control device can also be a server. The embodiments of the present application do not specially limit the specific form of the central control device.

[0080] The central control device obtains examination room data and environmental parameters in real time through various sensors and data acquisition devices. The examination room data can include, but is not limited to, three-dimensional space data of the examination room, layout information of the examination room, device distribution information, personnel distribution information, and the like. The environmental parameters can include, but are not limited to, temperature, humidity, noise, light intensity, gas concentration, and the like.

[0081] In a possible implementation, the examination room data can be obtained by the following methods: 1. Obtain three-dimensional point cloud data of the examination room by LiDAR scanning; 2. Obtain real-time video stream of the examination room by fixed cameras; 3. Obtain basic layout information of the examination room by a pre-established CAD model of the examination room; 4. Obtain the position information of personnel and devices in the examination room by RFID tags or Bluetooth beacons.

[0082] The environmental parameters can be obtained by an environmental sensor network distributed throughout the examination room. These sensors are connected to the central control device through wired or wireless means and transmit data in real time.

[0083] Based on the obtained examination room data, the central control device establishes a real scene navigation map. The real scene navigation map is a high-precision map that integrates actual scene information and can reflect the real-time state of the examination room. The real scene navigation map is in real-time linkage with the examination room data, and when the state of the examination room changes, the real scene navigation map will also be updated accordingly.

[0084] In a possible implementation, the establishment process of the real scene navigation map includes: 1. Convert the three-dimensional point cloud data into a grid map, and each grid represents an area in the examination room; 2. Integrate the scene information extracted from the video stream with the grid map; 3. Superimpose the static layout information in the CAD model onto the map; 4. Dynamically update the position information of personnel and devices onto the map.

[0085] The real scene navigation map can adopt a multi-layer structure, and different levels represent different types of information, for example: the basic layer represents the physical layout of the examination room, the dynamic layer represents the real-time positions of personnel and devices, and the semantic layer represents the functional attributes of different areas, and the like.

[0086] The inspection device acquires real-time images of the examination room. The inspection device can be a mobile robot, a drone, or other movable intelligent device equipped with a high-definition camera for capturing real-time images of the examination room. The inspection device is connected to the central control device through a wireless network, which can be Wi-Fi, 5G, or a dedicated wireless communication network.

[0087] Figure 3 The structure of the inspection device provided by the embodiments of the present application is shown in the schematic diagram as Figure 3 As shown in a possible implementation, the inspection device can include: 1. Mobile chassis: provides mobility, which can be wheeled, tracked, or other forms; 2. Camera system: including high-definition cameras, gimbals, etc., for capturing images of the examination room; 3. Communication module: for wireless communication with the central control device; 4. Navigation system: for autonomous navigation and positioning; 5. Power supply system: provides the energy needs of the inspection device.

[0088] The camera can be fixed or gimbal, and the gimbal camera can achieve horizontal 360-degree rotation and vertical -90 degrees to +90 degrees tilt to obtain a wider field of view. The resolution of the camera is not less than 1080p, and the frame rate is not less than 30fps to ensure image quality.

[0089] The images captured by the inspection device are transmitted in real time to the central control device through a wireless network, and video encoding techniques such as H.264 or H.265 can be used for compression during transmission to reduce bandwidth usage.

[0090] The central control device processes the examination room images received from the inspection device, extracts image features, and divides the inspection path into n regions based on these features, where n is a positive integer.

[0091] Image feature extraction can use a variety of computer vision techniques, including but not limited to: 1. Edge detection: using Canny, Sobel, etc. operator to detect edges in the image; 2. Corner detection: using Harris, FAST, etc. algorithm to detect corners in the image; 3. Feature descriptor: using SIFT, SURF, ORB, etc. algorithm to extract local features of the image; 4. Deep learning features: using pre-trained convolutional neural network (CNN) to extract high-level semantic features of the image.

[0092] In a possible implementation, the process of image feature extraction includes: 1. Preprocessing the image, including denoising, illumination equalization, etc.; 2. Using a pre-trained deep neural network such as ResNet-50 to extract the depth features of the image; 3. Using principal component analysis (PCA) to reduce the dimensionality of the features, retaining the main information; 4. Associating the extracted features with geographic location information to form a feature map with spatial attributes.

[0093] Based on the extracted image features, the central control equipment divides the inspection path into n regions. Region division can be based on the following factors: 1. Similarity of image features: regions with similar features can be grouped together; 2. Spatial continuity: spatially continuous regions tend to be grouped together; 3. Functional attributes: regions with the same function can be grouped together; 4. Risk level: regions with similar risk levels can be grouped together.

[0094] In one possible implementation, region partitioning can be achieved using the K-means clustering algorithm, grouping regions with similar characteristics into one cluster. The clustering steps include: 1. Initializing K cluster centers; 2. Assigning each region to the nearest cluster center; 3. Recalculating the center of each cluster; 4. Repeating steps 2 and 3 until the cluster centers no longer change or the maximum number of iterations is reached.

[0095] Each partitioned region is assigned a unique identifier for subsequent route planning and risk assessment.

[0096] In one implementation, the central control device executes a dynamic path planning strategy based on a real-view navigation map and extracted image features to generate the optimal inspection path for the inspection equipment.

[0097] Dynamic path planning strategies consider a variety of factors, including but not limited to: 1. Examination room layout: avoid inaccessible areas, such as walls, large equipment, etc.; 2. Inspection coverage: ensure that all important areas are inspected; 3. Path length: shorten the path length as much as possible while meeting inspection requirements; 4. Risk factors: avoid high-risk areas or prioritize the inspection of high-risk areas; 5. Real-time obstacles: adjust the path according to obstacles detected in real time.

[0098] In one possible implementation, the dynamic path planning process includes: 1. Converting the real-world navigation map into a cost map, where each grid cell has a cost value representing the difficulty of traversing that area; 2. Dynamically updating the cost map based on image features and environmental parameters; 3. Calculating the globally optimal path using an improved A* algorithm; 4. Dynamically adjusting the path based on real-time information acquired during the inspection process.

[0099] The improved A* algorithm is an optimized version of the traditional A* algorithm. Its heuristic function considers multiple factors, such as distance, risk level, and inspection priority. The core formula of the algorithm is:

[0100] f (n) = g (n) + h (n) + r (n) + p (n);

[0101] Wherein: f(n) is the total evaluation value of node n; g(n) is the actual cost from the starting point to node n; h(n) is the estimated cost from node n to the target; r(n) is the risk factor of node n; p(n) is the inspection priority factor of node n.

[0102] When the test environment changes, such as the appearance of new obstacles or the gathering of people, the system will recalculate the path in the affected area to ensure that the inspection equipment can safely and efficiently complete the inspection task.

[0103] In one possible implementation, the central control device fuses and analyzes the environmental parameters and image features to calculate the congestion index of each area on the inspection path. The congestion index is a comprehensive indicator that measures the degree of congestion and potential risk of the area.

[0104] The fusion analysis of environmental parameters and image features can use various methods, including but not limited to: 1. Feature-level fusion: fuse environmental parameters and image features at the feature level; 2. Decision-level fusion: make decisions based on environmental parameters and image features respectively, then fuse the decision results; 3. Model-level fusion: build a fusion model that can handle environmental parameters and image features simultaneously.

[0105] In one possible implementation, the process of fusion analysis includes: 1. Extracting information such as personnel density, equipment distribution, and operation behavior from image features; 2. Standardizing environmental parameters to make them comparable with image features; 3. Building a multi-layer perceptron neural network model, the input layer includes environmental parameters and image features, and the output layer is the congestion index; 4. Training the neural network model using historical data to enable it to accurately predict the congestion index.

[0106] The calculation formula of the congestion index can be expressed as:

[0107] CI = f(ED, PD, ED, OB, EP).

[0108] Wherein: CI is the congestion index; ED is the equipment distribution density; PD is the personnel distribution density; OB is the operation behavior risk score; EP is the environmental parameter comprehensive score; f is the function represented by the neural network model.

[0109] The congestion index has a value range of 0 to 100, and the larger the value, the higher the congestion level and the greater the risk.

[0110] Based on the calculated congestion index, the central control device determines the dangerous area among the n areas and visualizes the dangerous area on the real scene navigation map.

[0111] The determination of the dangerous area can be based on a congestion index threshold, for example: when the congestion index of an area is lower than the threshold T1, the area is marked as a safe area; when the congestion index of an area is between the thresholds T1 and T2, the area is marked as a warning area; when the congestion index of an area is higher than the threshold T2, the area is marked as a dangerous area.

[0112] wherein 0 < T1 < T2 < 100, and the specific thresholds can be adjusted according to actual application scenarios.

[0113] In a possible implementation, T1 can be set to 30 and T2 can be set to 70, that is, an area with a congestion index lower than 30 is a safe area, an area with a congestion index between 30 and 70 is a warning area, and an area with a congestion index higher than 70 is a dangerous area.

[0114] The visualization of the dangerous area can be achieved by the following methods: 1. using different colors to mark areas with different risk levels on a real scene navigation map, for example, green for safe areas, yellow for warning areas, and red for dangerous areas; 2. using a heat map to show the distribution of congestion indexes, with colors from blue to red representing congestion indexes from low to high; 3. adding flashing or other visual effects to the dangerous area to attract the attention of the operator; 4. superimposing risk type icons such as personnel gathering and equipment anomaly on the dangerous area.

[0115] The visualization result can be displayed in real time on the display interface of the central control device, or can be remotely viewed through a mobile terminal or a Web interface. Based on the visualization result, the operator can timely discover potential risks and take appropriate intervention measures.

[0116] Through the above steps, the intelligent inspection method provided by the embodiments of the present application can realize comprehensive monitoring and risk assessment of the special operation physical examination center, improve the efficiency and accuracy of the examination center safety management, and reduce the probability of safety accidents.

[0117] In actual application, the method can be flexibly adjusted and optimized according to the specific examination center environment and needs to adapt to different application scenarios.

[0118] The embodiment obtains examination room data and environmental parameters in real time through the central control device, establishes a real scene navigation map based on the examination room data, and the real scene navigation map is linked in real time with the examination room data to provide accurate spatial positioning and navigation support for the inspection. Secondly, the examination room image is obtained in real time through the inspection device, the inspection device is wirelessly or wiredly connected with the central control device, the inspection device is provided with a camera, and the camera is used to collect the examination room image, so that the inspection device can quickly cover each area of the examination room, avoiding the time-consuming problem of manual inspection. At the same time, the image features of the examination room image are extracted, and the inspection path is divided into n regions based on the image features, n being a positive integer, so that the examination room area is comprehensively covered, and the inspection efficiency is further improved. In addition, based on the real scene navigation map and the image features, a dynamic path planning strategy is executed to generate the inspection path of the inspection device. The scheme can adjust the inspection route in real time according to the actual situation of the examination room, ensure the comprehensiveness and pertinence of the inspection, and fuse and analyze the environmental parameters and the image features to determine the congestion index on the inspection path, so as to more accurately judge the state of the examination room and reduce the subjectivity and error of human judgment. Finally, based on the congestion index, a dangerous area is determined in the n regions, and the dangerous area is visualized, so that the management personnel can quickly understand the state of the examination room and formulate targeted improvement measures, thereby effectively preventing accidents. For the high-risk area, the scheme uses the inspection device (such as a drone, a robot, etc.) to enter the high-risk area for inspection, avoiding the blind area problem of manual inspection, so that the coverage range of the examination room inspection is significantly expanded. In addition, the scheme uses the image feature extraction technology to accurately identify the changes of the examination room equipment and environment, and through the fusion analysis of the image features and the environmental parameters, the state of the examination room can be more accurately judged, the subjectivity and error of human judgment can be reduced, the inspection efficiency and accuracy can be improved, the inspection coverage range can be expanded, the risk hidden danger of the examination room can be effectively reduced, and accidents can be effectively prevented, thereby ensuring the safety of the examination room personnel.

[0119] In one of the implementation manners of the embodiment, the examination room data includes three-dimensional space data, and the real scene navigation map is established based on the examination room data, including the following steps:

[0120] S210, converting the three-dimensional space data into a grid map;

[0121] S220, performing semantic segmentation on the three-dimensional space data by using a pre-constructed neural network model to obtain a semantic segmentation result;

[0122] S230, fusing the semantic segmentation result with the grid map to obtain a real scene navigation map including semantic information.

[0123] The exam room data includes three-dimensional spatial data, which is usually obtained by devices such as laser radar (LiDAR), depth camera or three-dimensional scanner. The three-dimensional spatial data contains information such as the geometric shape, size and spatial position of the exam room environment, existing in the form of point cloud, grid or voxel. These data can accurately reflect the physical layout of the exam room, including the position and shape of fixed facilities such as walls, doors and windows, equipment, desks and chairs, providing basic data support for subsequent navigation map construction. The collection process of three-dimensional spatial data needs to consider factors such as sampling density, accuracy and coverage range to ensure the integrity and accuracy of the data. In special operation physical exam rooms, multi-station scanning is usually used to collect three-dimensional spatial data, and the data of multiple stations are registered and fused to form a complete three-dimensional model of the exam room. These data are usually stored in point cloud file formats (such as.pcd,.ply, etc.) or three-dimensional model formats (such as.obj,.stl, etc.), and preprocessed by data processing software, including denoising, downsampling and outlier filtering, to improve data quality and processing efficiency. High-quality three-dimensional spatial data is a key prerequisite for constructing accurate and realistic navigation maps, directly affecting the effectiveness of subsequent navigation and inspection.

[0124] The process of converting three-dimensional spatial data into a grid map is to discretize continuous three-dimensional spatial information into a regular grid structure, which is convenient for computer processing and path planning algorithm application. In specific implementation, the resolution of the grid, i.e. the physical size of each grid cell, needs to be determined, which is usually determined according to the size of the inspection device, the navigation accuracy requirement and the calculation resource limitation. For example, the grid size can be set to 10cm x 10cm, which can ensure sufficient accuracy without causing excessive calculation. In the conversion process, the three-dimensional spatial data is projected onto the horizontal plane to form a two-dimensional occupancy grid map. For each grid cell, the occupancy probability value is calculated according to the distribution of the corresponding spatial point cloud. The specific calculation method can use the Bayesian update formula. When the point cloud density in a certain height range (usually between 10cm and 200cm above the ground) exceeds the threshold value, the grid is marked as occupied; when the point cloud density is below the threshold value, the grid is marked as free; when there is not enough observation data, the grid is marked as unknown. In addition, the height value of each grid cell can be assigned according to the height information of the point cloud to form a 2.5D elevation map, which better represents the height changes of the environment. The generation process of the grid map also needs to consider the filtering of dynamic obstacles, which are identified and removed through multiple scanning comparisons, leaving only the static environmental structure. The converted grid map is stored in matrix form, with each element representing the state (occupied, free or unknown) of the corresponding grid, providing a basis for subsequent path planning and navigation.

[0125] The pre-constructed neural network model is used to perform semantic segmentation on the three-dimensional spatial data to obtain the semantic segmentation result, so as to understand the functional attributes of different objects and regions in the environment.

[0126] Semantic segmentation is a task in computer vision that aims to assign each pixel or point in an image or point cloud to a predefined semantic class.

[0127] In the special operation physical examination room scene, common semantic categories include ground, wall, door, window, table and chair, equipment, personnel passage, dangerous area, etc. The pre-constructed neural network model usually adopts a deep learning architecture such as PointNet++, SparseConvNet or MinkowskiNet, which is specially designed for processing three-dimensional point cloud data. The training process of the model requires a large amount of three-dimensional data with semantic annotation, which can use the transfer learning method to first pre-train on a large-scale public data set (such as S3DIS, ScanNet) and then fine-tune on the specific examination room data.

[0128] In practical applications, the three-dimensional point cloud data is first pre-processed into a format acceptable to the model, including point cloud downsampling, normal vector calculation and feature extraction steps. Then, the processed point cloud is input into the neural network model, which learns the local and global features of the point cloud through multiple layers of feature extraction and nonlinear transformation, and finally predicts the semantic label for each point. The output of the model is a vector with the same size as the input point cloud, and each element represents the probability distribution of the corresponding point belonging to each semantic category. By taking the class corresponding to the maximum probability as the final prediction result, the semantic segmentation task is completed.

[0129] To improve the segmentation accuracy, multi-view information can be combined to fuse RGB images and point cloud data, taking advantage of their complementary advantages. In addition, post-processing methods such as Conditional Random Fields (CRF) can be applied to optimize the spatial consistency of the segmentation results. The results of semantic segmentation are stored in the form of class labels for each point, providing semantic information for subsequent navigation map construction. Through semantic segmentation, the functional structure of the environment can be better understood, and different types of regions and objects can be distinguished, providing higher-level environmental understanding capabilities for intelligent inspection.

[0130] Fusing the semantic segmentation results with the grid map to obtain a real-world navigation map that includes semantic information is a key step in constructing advanced navigation maps. The fusion process first needs to establish a spatial correspondence between the semantic segmentation results and the grid map, which is usually achieved through coordinate transformation. For each grid cell, the number of points of each semantic class falling into the grid is counted, and the majority voting method is used to determine the main semantic class of the grid. For example, if 60% of the points in a grid are classified as "equipment", 30% are classified as "ground", and 10% are classified as "wall", the semantic label of the grid is determined as "equipment". To handle the uncertainty in semantic segmentation, a semantic probability distribution can be maintained for each grid instead of just a single label, which allows the uncertainty of semantics to be considered in subsequent path planning. In the fusion process, the functional attributes of different semantic classes also need to be considered, such as assigning a traversability cost to each semantic class, which represents the difficulty or risk of the inspection device passing through the area of that class. Generally, the traversability cost of "ground" and "passage" classes is lower, and the traversability cost of "equipment" and "wall" classes is higher or non-traversable. In addition, different inspection priorities can also be set according to the semantic class, such as "dangerous area" and "critical equipment" having a higher inspection priority. The fused real-world navigation map is stored in a multi-layer structure, including occupancy layer, semantic layer, cost layer, and priority layer, etc., each layer representing different types of information. To facilitate visualization and human-computer interaction, different colors can be assigned to different semantic classes for intuitive display in two-dimensional or three-dimensional views. The real-world navigation map also needs to support real-time updating, which can dynamically adjust the map content when the environment changes or new observation data is obtained. The real-world navigation map that fuses semantic information not only contains the geometric structure of the environment, but also contains the functional attributes of the environment, which can support more intelligent path planning and decision-making, and improve the efficiency and safety of inspection.

[0131] The embodiment converts three-dimensional spatial data into a gridded map, uses a pre-constructed neural network model for semantic segmentation, and fuses the semantic segmentation results with the gridded map, finally constructing a real scene navigation map containing rich semantic information. This advanced navigation map not only represents the geometric structure of the examination environment, but also contains the functional attributes of different regions and objects in the environment, providing a solid foundation for intelligent inspection. Based on this real scene navigation map, the inspection system can understand the semantic structure of the environment, distinguish different types of regions such as dangerous areas, equipment areas, and personnel passages, and thus achieve more intelligent path planning and decision making. For example, the system can prioritize inspection of dangerous areas, avoid impassable areas, and reasonably arrange the inspection path to improve inspection efficiency. At the same time, semantic information can also help the system better understand environmental changes and identify abnormal situations, such as detecting obstacles in an originally empty area, which may indicate potential safety hazards. In addition, the multi-layer structure design of the real scene navigation map enables it to support multiple application scenarios and meet different levels of navigation and decision-making needs. Overall, this real scene navigation map that integrates semantic information significantly improves the environmental understanding, path planning, and safety monitoring capabilities of the special operation real scene examination intelligent inspection system, providing strong support for ensuring examination safety.

[0132] In one embodiment of the present embodiment, the dynamic path planning strategy includes the following steps:

[0133] S310, obtaining historical inspection data;

[0134] S320, calculating a global optimal path based on an improved A* algorithm, wherein the improved A* algorithm is improved based on a heuristic function;

[0135] S330, monitoring the crowd density and obstacle distribution on the inspection path in real time;

[0136] S340, determining whether there are path blocking areas and / or dangerous areas in the global optimal path based on the crowd density and obstacle distribution;

[0137] S350, in the case that there are path blocking and / or dangerous areas in the global optimal path, executing a local path re-planning strategy and generating a final path;

[0138] The local path re-planning strategy includes:

[0139] S1, identifying the boundary coordinates of the path blocking area and / or the dangerous area;

[0140] S2, assigning a high passage cost value to the grid nodes of the path blocking area and / or the dangerous area in the real scene navigation map;

[0141] S3, taking the current position as the starting point and the target inspection point as the ending point, applying the D* Lite algorithm to calculate the locally optimal path, wherein the target inspection point is the grid node closest to the grid node of the distance path blockage area and / or dangerous area outside the boundary coordinates;

[0142] S4, when detecting multiple continuous path blockage areas and / or dangerous areas, adopting a segmented planning strategy to decompose the path into multiple sub-paths;

[0143] S5, applying an ant colony optimization algorithm to each sub-path to generate a locally optimal sub-path;

[0144] S6, connecting all locally optimal sub-paths to obtain a locally re-planned path.

[0145] Historical inspection data typically includes inspection path records, inspection point access order, road segment travel time, human flow density variation law, obstacle occurrence frequency and position distribution, and other multi-dimensional information. These data are stored in the form of time series, and each record contains fields such as timestamp, position coordinates, inspection state and environmental parameters. The process of obtaining historical inspection data first needs to extract the original data from the system database, which is usually stored in a structured form in a relational database (such as MySQL) or a time series database (such as InfluxDB). Data extraction can be achieved through SQL query statements.

[0146] It should be noted that the acquisition of historical inspection data is not limited to internal system data, but can also integrate external data sources, such as human flow statistics data from the examination monitoring system, access records from the access control system, etc.

[0147] The improvement of the improved A* algorithm mainly lies in the design of the heuristic function. The heuristic function used by the traditional A* algorithm is usually the Euclidean distance or Manhattan distance, which cannot fully consider the complex environmental characteristics of the special operation physical examination room. The improved A* algorithm introduces a multi-factor weighted heuristic function, which considers distance, historical travel time, congestion probability and safety risk, etc. The specific heuristic function can be expressed as:

[0148] h(n) = ω1·d(n, target node) + ω2·t + ω3·p + ω4·r;

[0149] where d(n, target node) represents the Euclidean distance from node n to the target node, t represents the historical average travel time of node n, p represents the historical congestion probability of node n, r represents the safety risk value of node n, and ω1, ω2, ω3, and ω4 are corresponding weight coefficients that can be dynamically adjusted according to actual needs. For example, during an exam, the value of ω3 can be increased to place more emphasis on avoiding congested areas; near dangerous work areas, the value of ω4 can be increased to place more emphasis on safety. In the implementation process of the improved A* algorithm, a priority queue (usually a binary heap) is used to maintain the nodes to be expanded, and each time the node with the smallest f(n) = g(n) + h(n) value is taken out from the queue for expansion, where g(n) represents the actual cost from the starting point to node n.

[0150] In another embodiment, in order to improve the efficiency of the algorithm, a bidirectional A* search strategy can also be used, starting the search from both the starting point and the end point, and when the two search directions meet, the optimal path is found. In addition, in order to handle large-scale grid maps, a hierarchical A* algorithm (HPA*) can be used, which first plans a rough path at an abstract level and then optimizes the local path at a fine-grained level. The calculated global optimal path is a series of consecutive grid nodes representing the complete path from the starting point to the end point, arranged in the order of access, forming the inspection path. The calculation result of the global optimal path directly affects the inspection efficiency and safety, and is the basis for dynamic path planning.

[0151] In this embodiment, the real-time monitoring system continuously collects and analyzes the dynamic changes of the inspection environment through various sensor fusion technologies. The sensor configuration usually includes a laser radar (LiDAR) on the robot platform, a depth camera, an RGB camera, and fixed monitoring cameras distributed throughout the examination room. The laser radar can accurately measure the distance and profile of surrounding objects by emitting laser beams and receiving reflected signals, forming 360-degree point cloud data suitable for detecting the position and shape of obstacles. The depth camera can generate depth images, providing three-dimensional structural information of the scene, which helps to identify obstacles of different heights. The RGB camera captures color images, which, combined with computer vision algorithms, can identify the types and states of personnel, equipment, and other objects. Fixed monitoring cameras provide a wider field of view, covering the blind area of the robot's own sensors. The pedestrian density monitoring uses deep learning-based object detection and tracking algorithms, such as YOLOv8 to detect personnel, combined with multi-target tracking algorithms such as DeepSORT to track the movement trajectories of personnel.

[0152] The pedestrian density calculation formula is: ρ = N / A, where N is the number of people in the area, and A is the area of the region. According to the density value, the pedestrian flow condition can be divided into four levels: sparse (ρ < 0.2 person / ㎡), normal (0.2 person / ㎡ ≤ ρ < 0.5 person / ㎡), crowded (0.5 person / ㎡ ≤ ρ < 1 person / ㎡), and extremely crowded (ρ ≥ 1 person / ㎡).

[0153] Obstacle detection employs point cloud segmentation and clustering algorithms to divide the point cloud data into different clusters, each representing a potential obstacle. For each detected obstacle, its position coordinates, size, moving speed, and direction, etc. are recorded. The real-time monitoring system updates the environment state at a frequency of 10Hz or higher, ensuring timely capture of environmental changes. Monitoring data is transmitted to the central processing unit through a wireless network, registered and fused with the real-world navigation map, forming a real-time environmental state representation, providing a basis for subsequent path planning decisions.

[0154] The determination of path blockage areas and dangerous areas adopts a multi-index fusion method, considering factors such as pedestrian density, obstacle distribution, passage width, and safety risk. For pedestrian density, when the pedestrian density in a certain area exceeds the preset threshold (usually 0.8 person / ㎡), the area is marked as a potential path blockage area. The pedestrian density threshold can be adjusted according to the functional characteristics of different areas, for example, a higher threshold can be set at the entrance of the examination room, while a lower threshold can be set in the equipment operation area. For obstacle distribution, the spatial range occupied by obstacles is first calculated, and then the width of the remaining passage space is evaluated. When the remaining passage space width is less than the width of the inspection device plus the safety margin (usually 1.5 times the width of the device), the area is marked as a path blockage area. For example, if the width of the inspection robot is 60cm and the safety margin is 30cm, when the passage width is less than 90cm, it is determined as a path blockage area. The determination of dangerous areas considers multiple risk factors, including the degree of personnel concentration, obstacle moving speed, special equipment (such as high-voltage electrical equipment, dangerous chemicals, etc.) in the area, and historical safety event records.

[0155] Risk assessment can use fuzzy logic methods to map various risk factors to risk values in the [0,1] interval, and then calculate the comprehensive risk index by weighted summation. When the comprehensive risk index exceeds the safety threshold, the area is marked as a dangerous area. Time factors also need to be considered in the determination process to distinguish between permanent blockages / dangers and temporary blockages / dangers. For each grid node on the globally optimal path, check whether it is located within the determined path blockage area or dangerous area. If such a node exists, mark the corresponding section in the global path and record the range, degree, and expected duration of the blockage / danger area to provide a basis for subsequent local path re-planning. This multi-dimensional path state assessment ensures the safety and feasibility of the inspection task.

[0156] When detecting the existence of impassable or high-risk areas on the global path, it is necessary to adjust the path in time to avoid these areas while maintaining the overall optimality of the path as much as possible. Local path re-planning first needs to determine the scope of re-planning, which is usually centered on the blocked / dangerous area and extends a certain distance (such as 10 meters) outward as the re-planning area. Within the re-planning area, an improved path planning algorithm such as the D*Lite algorithm or the RRT* algorithm is applied to generate a local alternative path. The D*Lite algorithm is suitable for path re-planning in dynamic environments because it can efficiently handle environmental changes and only recalculate the affected path portion. The RRT* algorithm is suitable for handling environments with complex geometric constraints and can quickly find a feasible path. Local path re-planning also needs to consider the time factor. For temporary blockages (such as short-term personnel gathering), a waiting strategy can be chosen. When the estimated waiting time is shorter than the detour time, choose to wait for the blockage to be eliminated; when the estimated waiting time is longer than the detour time, choose the detour strategy.

[0157] In this embodiment, the detour strategy needs to consider the balance of energy consumption, time and safety, and through a multi-objective optimization method, the detour path with the minimum comprehensive cost is calculated. After local path re-planning, the local alternative path needs to be seamlessly connected with the unaffected part of the original global path to form a new complete path. During the connection process, the smoothness and continuity of the path need to be ensured to avoid sharp turns or unnecessary back-and-forth movements.

[0158] In one embodiment of the present embodiment, a Bezier curve or spline interpolation method can be used to smooth the path connection points. Through local path re-planning, local environmental changes can be flexibly handled while maintaining the overall optimality of the global path, ensuring the continuity and safety of the inspection task.

[0159] The identification of boundary coordinates uses computer vision and point cloud processing technology, combined with real-time monitoring data and real scene navigation maps. First, the detected blocked or dangerous areas are spatially clustered, and adjacent blocked / dangerous grids are merged into continuous areas. The clustering algorithm can use the DBSCAN algorithm to group based on the spatial adjacency relationship and state similarity of the grid.

[0160] For each clustered area, extract its outer contour points as the boundary point set. Boundary extraction can use a contour tracking algorithm such as the Moore neighborhood tracking algorithm, which starts from one boundary point of the area and moves along the area boundary according to certain rules (such as clockwise direction) until it returns to the starting point, recording all the boundary points it passes through.

[0161] To facilitate subsequent processing, a convex hull algorithm can be used to calculate the convex hull of the region as a simplified representation of the region. For regions with complex shapes, a geometric figure such as the minimum circumscribed rectangle or ellipse can be used for approximate representation, further simplifying the boundary description. The boundary coordinates also need to be attached with timestamp and confidence information, indicating the timeliness and reliability of the boundary.

[0162] In this embodiment, the passing cost value reflects the difficulty or risk level of the inspection device passing through a certain grid, which is an important input of the path planning algorithm. The cost allocation uses a multi-level cost model, and different cost values are set according to the degree of obstruction / danger. For completely obstructed areas (such as spaces occupied by fixed obstacles), an infinite cost value (in actual implementation, it can be a large enough number, such as 10000) is set to ensure that the path planning algorithm absolutely avoids these areas.

[0163] For partially obstructed areas (such as areas with high pedestrian density but still passable), a gradient cost value is set according to the degree of obstruction, which can use the following formula:

[0164] Passing cost value = basic passing cost × (1 + α × pedestrian density + β × obstacle density);

[0165] Where the basic passing cost is usually 1, and α and β are weight coefficients used to adjust the influence of pedestrian flow and obstacles on the cost.

[0166] It should be noted that for dangerous areas, the passing cost value not only considers the difficulty of passing, but also needs to consider the safety risk. In order to achieve smooth path planning, the allocation of cost value is not limited to the grids within the obstructed / dangerous area, but also needs to set a cost gradient around the area. The cost gradient can be realized using a distance function, the closer to the boundary of the obstructed / dangerous area, the higher the cost value; the farther, the cost value gradually decreases to the basic cost. This gradient cost allocation can use the following formula:

[0167] d = basic passing cost + (maximum cost value - basic passing cost) × exp(-d 2 / σ 2 );

[0168] Where d is the distance from the grid to the boundary of the obstructed / dangerous area, the maximum cost value is the maximum cost value at the boundary, and σ is a parameter that controls the gradient decay rate. The allocation of cost value also needs to consider the time factor, for temporary obstruction / danger, the cost value will decay with time; for permanent obstruction / danger, the cost value remains unchanged. The update frequency of the cost value is consistent with the environmental monitoring frequency, ensuring that the cost map can timely reflect the environmental changes. By reasonably allocating the passing cost value, the path planning algorithm can naturally avoid high-cost areas and generate safe and efficient inspection paths.

[0169] The D* Lite algorithm is used to calculate a locally optimal path from the current position to the target inspection point, which is the grid node closest to the grid nodes of the distance blocking / dangerous area outside the boundary coordinates. The D* Lite algorithm is an incremental search algorithm suitable for path re-planning in dynamic environments, and can efficiently handle environmental changes without completely re-planning.

[0170] In practical applications, the selection of the target inspection point is crucial, and it needs to be ensured that it is located outside the boundary of the blocking / dangerous area and has the minimum deviation from the original planned path. The grid node with the minimum distance to the original planned path outside the boundary of the blocking / dangerous area can be selected as the target inspection point by calculating the distance. Through the D* Lite algorithm, a locally optimal path from the current position to the target inspection point can be quickly calculated, effectively avoiding the blocking / dangerous area while maintaining the overall optimality of the path.

[0171] When multiple continuous path blocking areas and / or dangerous areas are detected, a segmented planning strategy is adopted, and the path is decomposed into multiple sub-paths, which is an effective method for handling complex environments. Segmented planning first needs to identify all blocking / dangerous areas and analyze their spatial distribution relationship. When the distance between two blocking / dangerous areas is less than a preset threshold (such as 2 times the length of the inspection device), they are considered as continuous regions; when the distance is greater than the threshold, they are considered as independent regions. For continuous blocking / dangerous areas, they can be combined into a large composite area to simplify processing; for independent blocking / dangerous areas, they can be processed separately. Path decomposition uses a key point extraction method to select a series of key points on the original global path as the demarcation points of the sub-paths. The selection of key points considers multiple factors, including the location, shape of the blocking / dangerous area, the topological structure of the environment, and the requirements of the inspection task, etc. Commonly used key points include: entrance and exit points of the blocking / dangerous area, turning points of the path, inspection points, and feature points in the environment (such as doorways, corridor intersection points, etc.). Key point extraction can use a curvature-based method to calculate the curvature of each point on the path and select the curvature peak point as a candidate key point; or use a visibility-based method to start from a key point and proceed along the path until a point is encountered that blocks the line of sight, and the point is taken as the next key point. After determining the key points, the original path is decomposed into multiple sub-paths, each connecting two adjacent key points. The number of sub-paths depends on the complexity of the environment and the distribution of the blocking / dangerous area, and is usually 2-5. Segmented planning also needs to consider the continuity and smoothness between sub-paths to ensure that there are no sharp turns or unnecessary back-and-forth movements at the connection points of the sub-paths. Path smoothing techniques such as Bezier curve interpolation or spline interpolation can be applied at the connection points to make the path transition more natural.

[0172] The advantage of the segmented planning is that the complex global path planning problem is decomposed into multiple simple local path planning problems, which reduces the computational complexity and improves the planning efficiency, and can more flexibly cope with local environmental changes.

[0173] The ant colony optimization algorithm is a swarm intelligence optimization algorithm inspired by the foraging behavior of ants, which is suitable for solving path planning and other combinatorial optimization problems. When applying the ACO algorithm, the area between the start and end points of the sub-path needs to be discretized into a grid, and each grid node is a location that ants can access. The core of the algorithm is the pheromone update mechanism, and the pheromone concentration represents the attractiveness of the path. At the beginning, the pheromone concentration of all paths is equal. In each iteration, multiple "ants" (virtual search agents) start from the starting point and select the next access node based on the pheromone concentration and heuristic information (such as the distance to the target point). After each ant completes the path search, the path quality is evaluated based on the path length and safety, and the higher the quality of the path, the greater the pheromone increment it obtains.

[0174] In one embodiment of the present embodiment, in order to adapt to the characteristics of the special operation physical examination room, the ACO algorithm has been improved in several ways: the introduction of a safety factor, which takes the safety of the grid (distance from blocked / dangerous areas) into account in the heuristic information; the use of an elite ant colony strategy, only ants that find better paths can release pheromones; the introduction of a local search mechanism to optimize the found path locally, such as path smoothing and redundant point deletion. The algorithm iterates until the maximum number of iterations is reached or the path quality does not improve significantly for several consecutive iterations. The highest quality path is finally selected as the locally optimal sub-path. The advantage of the ACO algorithm is that it can effectively handle path planning problems in complex environments, especially in the presence of multiple blocked / dangerous areas, and can find the optimal path that balances distance, safety, and smoothness.

[0175] When there are path blockages and / or dangerous areas in the globally optimal path, a local path replanning strategy needs to be performed to generate a new safe path. The local path replanning strategy includes the following steps: First, identify the boundary coordinates of the path blockage area and / or dangerous area, for example, the boundary coordinates of a dangerous area are (x1, y1) to (x2, y2). Then, in the real scene navigation map, assign a high passing generation value to the grid nodes of these areas, for example, set the passing generation value of the dangerous area to 1000, which is much higher than the generation value of ordinary areas. Next, take the current position as the starting point and the target inspection point as the end point, and apply the D*Lite algorithm to calculate the locally optimal path. The D*Lite algorithm is an incremental path planning algorithm that can quickly update the path when the environment changes. For example, in a certain local path replanning, the D*Lite algorithm generates a shortest path that bypasses the dangerous area. When multiple consecutive path blockage areas and / or dangerous areas are detected, a segmented planning strategy is used to decompose the path into multiple sub-paths, and the ant colony optimization algorithm is applied to each sub-path to generate a locally optimal sub-path. For example, a certain path is decomposed into 3 sub-paths, and the ant colony optimization algorithm generates the optimal solution for each sub-path. Finally, all locally optimal sub-paths are connected to obtain a local replanning path. For example, a certain local replanning path is connected by 3 sub-paths, with a total length of 50 meters, which is 10 meters shorter than the original path.

[0176] It should be noted that connecting all locally optimal sub-paths to obtain a local replanning path is a dynamic path planning that needs to ensure that the connected path is smooth, continuous and feasible.

[0177] The connection of sub-paths first needs to check whether the connection points of adjacent sub-paths are consistent. If not, a transition path segment needs to be generated. The generation of the transition path segment can use the Bezier curve interpolation method, and the selection of the control points considers the tangent direction of the path to ensure smooth transition.

[0178] The connected path may have redundant points and unnecessary twists, which needs to be optimized. Path optimization includes point simplification and smoothing. Point simplification can use the Douglas-Peucker algorithm to reduce the number of path points while maintaining the shape of the path; smoothing can use moving average or spline interpolation methods to eliminate sharp corners and abrupt changes in the path.

[0179] The embodiment can effectively deal with complex changes in the examination room environment through a dynamic path planning strategy, ensure efficient operation of the inspection device and safety of the examination room. The historical inspection data obtained provides a reliable reference for path planning, and the improved A algorithm improves the efficiency and accuracy of global path planning. Real-time monitoring of pedestrian density and obstacle distribution can timely discover path blocking areas and dangerous areas, and provide a basis for local path re-planning. The local path re-planning strategy generates a safe and efficient new path through the D* Lite algorithm and the ant colony optimization algorithm, avoiding the running delay and safety hazards of the inspection device. Not only does it improve the inspection efficiency and accuracy, but also expands the inspection coverage, effectively reduces the risk of the examination room, and provides a strong guarantee for the safety of the examination room personnel.

[0180] In one of the embodiments of the present embodiment, based on the real navigation map and image features, a dynamic path planning strategy is performed to generate an inspection path of the inspection device, including the following steps:

[0181] S410, dividing the real navigation map into a plurality of grid nodes, and calculating the passing generation value of each grid node;

[0182] S420, in response to a setting instruction of a user on the real navigation map, determining an inspection point in the real navigation map and a weight coefficient corresponding to each inspection point;

[0183] S430, calculating an initial global path by using an improved A* algorithm;

[0184] S440, taking the real-time acquired environmental parameters and image features as input parameters of a pre-constructed multi-feature fusion model to dynamically update the passing generation value of each grid node;

[0185] S450, when the change amount of the passing generation value exceeds a preset threshold, a preset path re-planning algorithm is used to generate a new inspection path;

[0186] S460, when the change amount of the passing generation value does not exceed the preset threshold, the initial global path is taken as the inspection path of the inspection device.

[0187] In this embodiment, it is first necessary to divide the real scene navigation map into multiple grid nodes, and calculate the passing generation value of each grid node. Specifically, the real scene navigation map is usually based on high-precision map data of the actual environment, which contains information such as the geometric structure, obstacle distribution, and terrain features of the environment. The real scene navigation map is divided into grids to discretize the continuous space, which is convenient for subsequent path planning algorithm processing. The grid division adopts a uniform grid method, that is, the entire map area is divided into regular square grids according to a preset grid size (such as 0.5 meters x 0.5 meters). Each grid node represents a position point in the actual environment and has a unique coordinate identifier (i, j), where i represents the row index and j represents the column index. The size of the grid needs to be balanced between the calculation complexity and the planning accuracy. If the grid is too large, the path planning accuracy will be insufficient, and if the grid is too small, the calculation burden will be increased.

[0188] For each grid node, the passing generation value needs to be calculated, which reflects the difficulty of the inspection device passing through the node. The calculation of the passing generation value considers multiple factors, including: terrain factors (such as slope, flatness), obstacle factors (such as static obstacle distance, dynamic obstacle predicted position), environmental factors (such as lighting conditions, weather conditions), and task-specific factors (such as device power demand, signal coverage strength).

[0189] In response to the user's setting instruction for the real scene navigation map, the inspection points in the real scene navigation map and the weight coefficients corresponding to each inspection point are determined. The inspection point refers to a key position that the inspection device must reach during the execution of the inspection task, and is usually a device, facility or area that needs to be checked. The user can directly mark the inspection point on the real scene navigation map through an interactive interface, or can batch set by importing a predefined inspection point list. Each inspection point has a clear coordinate position (x, y) on the map and is mapped to the nearest grid node. In actual application, the setting of the inspection point usually considers factors such as device distribution, inspection frequency demand, safety hazard risk, etc., to ensure that the inspection covers all key areas.

[0190] Figure 4 A setting diagram of an inspection point of a real scene navigation map provided by an embodiment of the present application is shown, as shown in Figure 4 The setting diagram of the inspection point of the real scene navigation map shows the overall system architecture of the setting of the inspection point in the real scene navigation map, including: the user marking the inspection point through an interactive interface or importing a predefined list, the storage and processing process of the inspection point data, the process of mapping the coordinates to the grid node, the key attributes (coordinates, weight coefficients, etc.) of the inspection point, and the components of the real scene navigation map.

[0191] In a specific implementation, for each inspection point, the user can also set a corresponding weight coefficient, which reflects the importance or priority of the inspection point. The weight coefficient usually ranges from 0 to 1, and the larger the value, the more important the inspection point. The determination of the weight coefficient can be based on various factors, such as equipment failure history, equipment importance, inspection urgency, etc. For example, for critical equipment in high-risk areas, a higher weight coefficient (such as 0.8-1.0) can be set; for ordinary equipment for routine inspection, a medium weight coefficient (such as 0.4-0.7) can be set; for auxiliary inspection points, a lower weight coefficient (such as 0.1-0.3) can be set.

[0192] For example, if a device recently has abnormal data, the system will automatically increase the weight coefficient of the inspection point corresponding to the device.

[0193] The improved A* algorithm is described above, and the present application will not be repeated here. Through the improved A* algorithm, the initial global path can be calculated.

[0194] The multi-feature fusion model is a deep learning model that can comprehensively process multi-source heterogeneous data, mainly composed of a feature extraction module, a feature fusion module, and a cost prediction module. The feature extraction module is responsible for extracting effective features from data from different sources, including environmental parameters such as temperature, humidity, and illumination obtained from environmental sensors, as well as image data obtained from cameras. For image data, a pre-trained convolutional neural network (such as ResNet-50 or EfficientNet) is used to extract high-level semantic features, which can identify key information such as obstacles, terrain changes, and personnel activities.

[0195] In this embodiment, the feature fusion module uses attention mechanisms and cross-modal fusion techniques to effectively integrate features from different sources. Specifically, first, normalize each type of feature, then calculate the relevance between different features through multi-head attention mechanisms, as follows:

[0196]

[0197] where Q, K, and V are query matrices, key matrices, and value matrices, respectively, converted from different features, d k is the feature dimension. Through the attention mechanism, the model can automatically learn the importance weights of different features and perform weighted fusion.

[0198] The cost prediction module then predicts the passage cost value update amount for each grid node based on the fused features. This module uses a multi-layer perceptron structure, with a ReLU activation function in the last layer to ensure that the output cost value is non-negative. The prediction formula can be:

[0199] AC(i, j) = MLP(FF(i, j));

[0200] where AC(i, j) is the update of the cost value of the grid node (i, j), and FF(i, j) is the fusion feature corresponding to the node.

[0201] In practical applications, the multi-feature fusion model triggers the update calculation at fixed time intervals (such as 1 second) or when a significant change in the environment is detected. The updated cost value is:

[0202] C new (i, j) = C old (i, j) + AC(i, j);

[0203] In this way, real-time perception of environmental changes can be achieved, such as detecting temporary obstacles (such as pedestrians, vehicles), identifying unfavorable terrain conditions (such as slippery ground, temporary water accumulation), or perceiving changes in environmental conditions (such as insufficient light, severe weather), and adjusting the cost value accordingly to ensure the safety and feasibility of the inspection path.

[0204] When the change in the cost value exceeds the preset threshold, a preset path re-planning algorithm is used to generate a new inspection path. First, it is evaluated whether the change in the cost value exceeds the preset threshold.

[0205] When the change exceeds the preset threshold, the path re-planning algorithm is triggered. In order to balance the calculation efficiency and path quality, a hierarchical re-planning strategy is adopted: first, evaluate the impact of the changed area on the current path, if there is no intersection or the impact is small, only local path re-planning is performed; if the changed area has a significant impact on the current path, global path re-planning is performed. Local path re-planning uses the D*Lite algorithm, which is a variant of the A* algorithm designed for path re-planning in dynamic environments, and can efficiently update the affected path segment. Global path re-planning re-executes the improved A* algorithm, but uses the previously calculated results as the initial solution to speed up the convergence process.

[0206] Through the threshold-based path re-planning mechanism, the inspection path can be adjusted in time when the environment changes significantly, ensuring the safety and effectiveness of the inspection task, while avoiding unnecessary waste of computing resources.

[0207] When the change in the cost value does not exceed the preset threshold, the initial global path is used as the inspection path for the inspection device. At this time, the initial global path is maintained, but the following optimization processes are performed: first, perform path validity verification to ensure that there are no new impassable areas on the current path. The verification method is to check whether the cost value of all grid nodes on the path has become infinite, if so, trigger local path adjustment.

[0208] The embodiment divides the real scene navigation map into grid nodes and calculates the passing cost value, combines the user set inspection points and weight coefficients, generates an initial global path by using an improved A* algorithm, dynamically updates the passing cost value based on a multi-feature fusion model, and determines whether to re-plan the path according to the change amount, so as to realize intelligent and adaptive inspection path planning, effectively cope with complex and changeable environment, balance inspection efficiency and safety, reduce waste of computing resources, and improve the completion quality of inspection tasks.

[0209] In one of the embodiments of the present embodiment, the inspection path is divided into n regions based on image features, including the following steps:

[0210] S510, based on the K-means clustering algorithm, the image features are clustered to obtain at least one clustering region;

[0211] S520, calculating the feature vector of each clustering region and calculating the similarity between the feature vectors;

[0212] S530, according to the similarity between the feature vectors, the inspection path is divided into n continuous regions, wherein each region is assigned a unique identifier.

[0213] In actual implementation process, the image features are usually high-dimensional vectors (such as 2048 dimensions), which can effectively capture the semantic information and visual features of the image. In order to reduce the computational complexity, the high-dimensional features are usually reduced, such as using principal component analysis (PCA) to reduce the feature dimension to 128 or 256 dimensions, while retaining most of the effective information.

[0214] After obtaining the image features, the K-means clustering algorithm is applied to cluster these features. K-means algorithm is an iterative clustering method, its core idea is to divide n data points into k clusters, so that each data point belongs to the nearest cluster center. The specific steps of the algorithm include: first, randomly select k points as initial cluster centers; then iteratively execute two steps: (1) assign each data point to the cluster represented by the nearest cluster center, (2) recalculate the center point of each cluster (i.e. the average value of all points in the cluster); when the cluster center point no longer changes significantly or reaches the maximum iteration number, the algorithm terminates.

[0215] In practical applications, the K value can be set according to the needs of the inspection task and the complexity of the environment. For example, for complex and changeable industrial environments, more clustering regions may be needed (e.g., K = 8 to 12); while for relatively simple environments, fewer clustering regions (e.g., K = 3 to 5) may be sufficient. After clustering, each clustering region represents a set of images with similar visual features. These regions may be continuous in physical space (such as corridors of the same type) or scattered (such as equipment areas with similar features distributed in different locations).

[0216] In the previous step, the image features were clustered into multiple regions using the K-means algorithm, with each region containing a set of image points with similar features. To more accurately characterize the features of each clustered region, a representative feature vector needs to be calculated for each region. The feature vector comprehensively considers the feature distribution of all image points within the region. Specifically, the feature vector can be calculated using a weighted average method, that is, by calculating the weighted average of all feature vectors, to reduce the influence of outliers and improve the representativeness of the feature vector.

[0217] After obtaining the feature vectors of each cluster region, it is necessary to calculate the similarity between the feature vectors to assess the degree of association between different regions. Similarity can be calculated using cosine similarity, which measures the similarity in direction between two vectors.

[0218] In the first two steps, K-means clustering was used to obtain clustered regions, and the feature similarity between each region was calculated. However, these clustered regions may be spatially dispersed, while the inspection path needs to be a continuous linear structure. Therefore, it is necessary to map the clustering results onto the inspection path to form a continuous path region partition. This process uses a dynamic programming algorithm, treating the inspection path as a one-dimensional sequence, to find the optimal split point, ensuring high feature similarity within the partitioned regions and significant feature differences between regions.

[0219] The specific implementation method is to first represent the inspection path as a series of ordered location point sequences P = {p1, p2, ..., p...} m Each location point is associated with a label representing its corresponding cluster region. Then, an objective function is defined to evaluate the quality of the path partitioning:

[0220]

[0221] Where S = {S1, S2, ..., S} n} is a path partitioning scheme, S n I(S) represents the nth region. i )Measure area S i Internal similarity (the higher the better), I(S)i ,S i+1 ) measures the similarity between adjacent regions (the lower the better), and λ1 and λ2 are weighting coefficients.

[0222] We use dynamic programming to find the optimal partitioning scheme: Define DP[i][j] as the optimal score for dividing the first i points of the path into j regions, and the state transition equation is:

[0223]

[0224] Where Score(k+1, i) represents the position point p k+1 to p i This is a score for a region. By filling in the DP table, the optimal partitioning scheme can be obtained.

[0225] After identifying n consecutive regions, each region is assigned a unique identifier. Specifically, numerical codes (such as 1, 2, ..., n) can be used as the basic identifier; secondly, the main characteristic type of the region can be combined, such as "A1" representing a densely populated equipment area, "B2" representing a corridor area, "C3" representing an open space, etc.; in addition, the location information of the region can also be encoded in the identifier, such as "N1" representing northern region 1, "SE2" representing southeastern region 2, etc.

[0226] Ultimately, each region not only has a unique identifier but is also associated with a series of metadata, such as a description of the region's main characteristics, a list of key equipment, suggested inspection speeds, and precautions. This information can be stored in a region attribute database for the inspection system to reference when executing tasks. In this way, the inspection path is divided into continuous regions with clear boundaries and rich semantic information, providing a structured spatial framework for subsequent inspection strategy formulation and execution.

[0227] This implementation uses the K-means clustering algorithm to cluster image features, calculates the feature vectors of the clustered regions and their similarity, and divides the inspection path into continuous regions with unique identifiers. This enables intelligent partitioning management of inspection paths in complex environments, achieving environmental understanding based on visual features. The system can identify and distinguish different types of areas (such as equipment areas, corridors, and open spaces), improving environmental perception capabilities. Secondly, the region division optimizes the inspection strategy, allowing for differentiated inspection parameters based on the characteristics of different areas, thus improving inspection efficiency. Furthermore, it simplifies task planning and anomaly localization, enabling operators to accurately reference specific areas, reducing communication costs and improving fault response speed. Region division provides a spatial indexing framework for data management, facilitating the organization and retrieval of inspection data by region and supporting more refined data analysis.

[0228] In one implementation of the embodiment, the environmental parameters are fused with the image features to determine the congestion index on the inspection path, including the following steps:

[0229] S610, extracting the personnel density, equipment distribution and operation behavior from the image features through the YOLOv8 model;

[0230] S620, outputting the congestion index through the multi-layer perceptron neural network model, wherein the input layer of the multi-layer perceptron neural network model includes the environmental parameters, the personnel density, the equipment distribution and the operation behavior, and the output layer is the congestion index.

[0231] In this embodiment, YOLOv8 is a real-time target detection algorithm, and its core principle is to divide the image into grids and simultaneously predict the target bounding box, class probability and confidence in each grid unit in a single forward propagation. Compared with traditional two-stage detectors, YOLOv8 adopts a single-stage detection architecture, greatly improving the detection speed while maintaining high detection accuracy through multiple technical innovations.

[0232] The network architecture of YOLOv8 mainly consists of three parts: backbone, neck and head. The backbone is responsible for extracting multi-scale features of the image, and adopts the CSPDarknet structure to reduce the amount of calculation while maintaining the feature extraction ability through techniques such as depth separable convolution and cross-stage local connection; the neck adopts the PANet (Path Aggregation Network) structure to realize the fusion of features of different scales, enhancing the detection ability of targets of different sizes; the head is responsible for the final target positioning and classification, improving the detection accuracy through multi-scale prediction boxes and dynamic allocation strategies.

[0233] In practical applications, the YOLOv8 model needs to be specially trained for the inspection scene. The training data set contains a large number of annotated industrial environment images, covering various personnel, equipment and operation behavior categories. Specifically, the personnel category can be subdivided into workers, managers, visitors, etc.; the equipment category can include fixed equipment (such as production lines, control cabinets) and mobile equipment (such as forklifts, transport vehicles); the operation behavior can be divided into regular operations (such as equipment inspection, material handling) and special operations (such as maintenance, debugging). During the training process, a transfer learning strategy is adopted, based on a pre-trained YOLOv8 model, to adapt to the specific scene through fine-tuning, while applying data augmentation techniques (such as rotation, scaling, color jitter, etc.) to improve the generalization ability of the model.

[0234] After the model training is completed, during the inspection process, the real-time collected images will be sent to the YOLOv8 model for processing. The model output includes the bounding box coordinates of detected targets of each class, class labels, and confidence scores. Based on these raw outputs, three key indicators are further calculated:

[0235] 1. Personnel density: calculated by counting the number of detected personnel per unit area. The specific calculation formula is:

[0236]

[0237] where n is the total number of detected targets, w i is the weight of the i-th target (which can be determined based on confidence and target size), and I(·) is the indicator function, which takes the value 1 when the i-th target belongs to the personnel class set C 人员 , and 0 otherwise, and A is the actual area covered by the image (in square meters).

[0238] 2. Equipment distribution: using kernel density estimation method, generate equipment distribution heat map, and extract distribution characteristics such as aggregation degree, uniformity, etc. The equipment distribution indicator can be expressed as:

[0239] Equipment distribution = {Equipment density, Aggregation degree, Uniformity, Main equipment type proportion}

[0240] 3. Operation behavior: analyze the interaction between detected personnel and equipment, and identify different types of operation behavior. Through time series analysis and pose estimation, distinguish between static operations (such as observation, recording) and dynamic operations (such as carrying, debugging). The operation behavior indicator can be expressed as:

[0241] Operation behavior = {Operation type distribution, Operation intensity, Operation duration, Abnormal operation proportion}

[0242] In order to improve the stability of feature extraction, usually the analysis results of continuous multiple frames of images are time-smoothed to reduce the influence of single-frame detection fluctuations. At the same time, combined with the position and angle information of the camera, the detection results are spatially corrected to ensure that the data obtained at different positions are comparable.

[0243] The three types of features (personnel density, equipment distribution, and operation behavior) extracted by the YOLOv8 model provide key inputs for subsequent congestion index calculation, which together reflect the activity state and potential congestion risk in the industrial environment. This visual-based feature extraction method has the advantages of wide coverage and rich information compared to traditional sensor monitoring, and can fully capture various factors affecting the inspection path traffic conditions.

[0244] The environmental parameters are fused with the image features extracted in the previous step for analysis, and a congestion index of the inspection path is finally generated. Multi-Layer Perceptron (MLP) is a kind of feedforward neural network, which is composed of an input layer, one or more hidden layers and an output layer, and is connected by full connection between layers, and can learn the complex nonlinear mapping relationship between input and output. In this application, the architecture of the MLP model is carefully designed to effectively fuse the environmental parameters and image features.

[0245] The input layer of the MLP model receives multiple types of feature data. First, the environmental parameters, including temperature, humidity, noise level, light intensity, air flow speed, and other physical environmental indicators, which are usually collected by various sensors distributed along the inspection path in real time. Second, the three types of image features extracted by the YOLOv8 model in the previous step: personnel density, equipment distribution, and operation behavior. In order to effectively fuse different types and dimensions of features, first, Z-score standardization is used to standardize all input features.

[0246] The hidden layer of the MLP model adopts a multi-layer structure, and the typical configuration includes 3-5 hidden layers, each containing 64-256 neurons. The hidden layer uses the ReLU (Rectified Linear Unit) activation function, whose mathematical expression is:

[0247] f(x)=max((0,x);

[0248] ReLU activation function has the advantages of simple calculation and stable gradient, which helps to solve the gradient vanishing problem in deep network training. Between the hidden layers, batch normalization technology can also be applied, which standardizes the input distribution of each layer, speeds up network training and improves model stability. To prevent overfitting, Dropout layers are added between the hidden layers, which randomly discard a certain proportion (usually 0.2-0.5) of neurons, enhancing the generalization ability of the model.

[0249] The output layer of the MLP model contains one neuron, which uses the Sigmoid activation function to map the output to the [0,1] interval, representing the normalized value of the congestion index. The mathematical expression of the Sigmoid function is:

[0250]

[0251] The closer the congestion index value is to 1, the more congested the path is; the closer to 0, the more unobstructed the path is. In order to improve the interpretability of the congestion index, the [0, 1] interval can be divided into multiple levels, such as 0-0.2 for "unobstructed", 0.2-0.4 for "slight congestion", 0.4-0.6 for "moderate congestion", 0.6-0.8 for "serious congestion", and 0.8-1.0 for "extreme congestion".

[0252] The training of the MLP model adopts a supervised learning method, which requires the construction of a training data set containing input features and congestion index labels. Label data can be obtained in several ways: first, expert annotation, in which domain experts evaluate the congestion level based on scene images and environmental data; second, automatic annotation based on rules, in which a preliminary congestion index is calculated based on pre-defined rules such as personnel density threshold and travel time; third, semi-supervised learning, which combines a small amount of labeled data and a large amount of unlabeled data for training. During the training process, mean squared error (MSE) or mean absolute error (MAE) is used as the loss function, and the Adam optimizer is used for parameter updating, with an initial learning rate of 0.001 and a learning rate decay strategy.

[0253] After the model is trained, in actual application, the real-time collected environmental parameters and image features are fed into the MLP model, and the congestion index of the current inspection path is output. This index not only reflects the immediate congestion state of the path, but also can be used to predict the congestion trend in the short term, providing decision basis for inspection path planning and adjustment. Through regular retraining and online learning, the model can continuously adapt to environmental changes and new congestion patterns, maintaining prediction accuracy.

[0254] The technical solution of the present embodiment fuses environmental parameters and image features to determine the congestion index on the inspection path, extracts personnel density, equipment distribution and operation behavior in the image through the YOLOv8 model, and fuses and analyzes these features with environmental parameters through a multi-layer perceptron neural network model, achieving accurate evaluation of the congestion status of the inspection path in the industrial environment. This solution first realizes intelligent fusion of multi-source data, organically combining traditional environmental sensing data with advanced computer vision analysis results, creating a more comprehensive and accurate scene understanding capability, effectively improving congestion prediction accuracy; secondly, it provides real-time congestion index, not only reflecting the current state, but also predicting future congestion trends, making inspection task planning more forward-looking; in addition, through fine-grained congestion analysis, different types of congestion causes (such as personnel gathering, equipment blocking, special operations, etc.) can be identified, and adaptive inspection path optimization is supported, which can dynamically adjust the inspection route and time arrangement according to the real-time congestion index, reducing inspection delay time.

[0255] In one embodiment of the present embodiment, based on the congestion index, a dangerous area is determined in n areas, including the following steps:

[0256] S710, when the congestion index of the area is lower than the preset first congestion index threshold T1, the area is marked as a safe area;

[0257] S720, when the congestion index of the area is between the first congestion index threshold T1 and the second congestion index threshold T2, the area is marked as a warning area;

[0258] S730, when the congestion index of the area is higher than the second congestion index threshold T2, the area is marked as a dangerous area, wherein 0

[0259] When the congestion index of the area is lower than the preset first congestion index threshold T1, the area is marked as a safe area, and the first congestion index threshold T1 is a safe upper limit value determined through scientific analysis and practical verification, representing the maximum congestion degree that the area can maintain normal operation without causing potential risks to personnel safety and equipment operation.

[0260] In the implementation process, the system continuously monitors the congestion index of each area, and immediately marks the area as a safe area when it detects that the congestion index of the area is lower than T1. After being marked as a safe area, the area is usually represented in green on the visualization interface, and the related safety status information is recorded in the database for subsequent statistical analysis and report generation.

[0261] The marking of the safe area has multiple practical meanings: first, it indicates that the current personnel and equipment density in the area is within a controllable range, and normal production and operation activities can proceed smoothly; second, the safe area can serve as a priority evacuation channel or temporary gathering point in emergency situations; third, the proportion and distribution of safe areas can reflect the overall safety of the environment, providing a basis for management decisions.

[0262] It is worth noting that the determination of a safe area is a dynamic process, and as the personnel flow and equipment operation state in the area change, the safety status of the area may change. Therefore, the system needs to update the congestion index calculation results at appropriate frequencies (usually every 5-10 seconds) to ensure the real-time and accuracy of the safety status evaluation. In addition, in order to avoid frequent state switching caused by short-term fluctuations, a time smoothing mechanism can be introduced, that is, only when the congestion index is lower than T1 for multiple consecutive time points (such as 3 consecutive detections), the area is marked as a safe area.

[0263] When the congestion index of a region lies between the first congestion index threshold T1 and the second congestion index threshold T2, the region is marked as a warning region, which represents a transitional state between safety and danger, indicating that the congestion level in the region has exceeded the ideal range but has not yet reached a dangerous level that requires immediate intervention.

[0264] The interval width between the first congestion index threshold T1 and the second congestion index threshold T2 directly affects the sensitivity of the warning mechanism. If the interval is too narrow, the system may be too sensitive and trigger warnings frequently; if the interval is too wide, the warning may lose its timeliness. Therefore, the difference between T1 and T2 is usually set to 15 to 30 percentage points, which can provide sufficient buffer time for preventive intervention without excessively expanding the coverage of the warning state.

[0265] In practical applications, the marking of the warning region is usually accompanied by the initiation of a series of preventive measures. First, on the visual interface, the warning region will be represented in yellow or orange, forming an intuitive visual cue; second, the system will send a warning notification to relevant management personnel, including interface pop-ups, SMS, email, or push messages from dedicated applications; third, for specific types of regions, pre-set response strategies may be triggered, such as adjusting personnel flow in adjacent regions, temporarily limiting new personnel from entering, increasing monitoring frequency, etc.

[0266] To improve the accuracy of the warning mechanism, the rate of change of the congestion index can be introduced as an auxiliary judgment factor. Even if the congestion index has not reached T1, if its growth rate is abnormally fast (e.g., increasing by more than 10 percentage points in a short period of time), a pre-warning may be triggered.

[0267] When the congestion index of a region is higher than the second congestion index threshold T2, the region is marked as a danger region, which is the highest alert level of the region safety state evaluation, indicating that the congestion level in the region has reached a critical level that may threaten personnel safety, equipment operation, or production order, requiring immediate intervention measures. The second congestion index threshold T2 is a critical value determined based on safety risk assessment and emergency management needs, usually set between 60 to 80, with the specific value depending on the characteristics and safety requirements of the region.

[0268] In practical applications, when a certain area is marked as a dangerous area, a series of emergency response measures will be triggered. First, on the visualization interface, the dangerous area will be represented in red or flashing red, possibly accompanied by audible and visual alarm signals; second, the system will immediately send high-priority alerts to safety managers, area supervisors, and relevant emergency teams, ensuring timely communication of information through multiple channels; third, according to the pre-set emergency plan, corresponding intervention measures will be initiated, such as evacuating unnecessary personnel from the area, suspending or adjusting exam activities, activating emergency equipment (such as exhaust systems, fire equipment), etc.; fourth, detailed event logs are recorded, including the start time, duration, maximum congestion index value of the dangerous state, and the response measures taken, providing a basis for subsequent analysis and improvement.

[0269] In addition, the determination of a dangerous area can also be combined with other safety monitoring indicators to form a multi-dimensional risk assessment. For example, by combining environmental parameters such as temperature, gas concentration, noise level, or device operating status, abnormal behavior detection results, etc., a more comprehensive dangerous state assessment model can be constructed:

[0270] Dangerous degree = w1·Congestion index + w2·Environmental risk index + w3·Behavior risk index +...;

[0271] Where w1, w2, w3, etc. are the weight coefficients of each indicator, which can be adjusted according to the characteristics and safety priorities of different areas. When the comprehensive dangerous degree exceeds the pre-set threshold, the dangerous area marking is triggered.

[0272] The timely identification and handling of dangerous areas are of great significance in preventing safety accidents, ensuring personnel safety, and maintaining normal production order. By establishing clear determination criteria and response mechanisms, effective intervention can be made before the risk evolves into actual harm, minimizing potential losses.

[0273] This embodiment establishes a three-level safety state assessment system by setting two key thresholds, achieving accurate classification and timely warning of regional safety conditions.

[0274] In one of the embodiments of the present embodiment, the camera is also used to collect the exam room video, and the exam room video includes a plurality of video frames, and further includes the following steps:

[0275] S810, input the video frame into the pre-constructed YOLOv8 model to identify the pre-defined rule violation behavior type;

[0276] S820, for each rule violation behavior type, record the start time and end time of the rule violation behavior;

[0277] S830, extracting key video frames at a frequency of m frames per second within a preset time interval, and adding a timestamp, a violation type label, and a violation area bounding box to each key video frame;

[0278] S840, sorting the key video frames in chronological order, generating a violation behavior evidence package, storing the violation behavior evidence package in a security database of a central control device, and constructing a violation behavior index of the security database.

[0279] The video frames are input into a pre-constructed YOLOv8 model to identify predefined violation behavior types. The specific content of the YOLOv8 model is described above and will not be repeated here. In actual application, the examination room video captured by the camera is decomposed into a continuous sequence of video frames, each frame being a static image. These images are sent to the pre-trained YOLOv8 model for processing and analysis.

[0280] In the model input stage, each frame of video image is preprocessed, including size adjustment (usually adjusted to 640x640 or 416x416 pixels), normalization (scaling pixel values to the range of 0-1), and channel arrangement adjustment, etc. to meet the input requirements of the YOLOv8 model. The processed image data is sent to the input layer of the model, and then passes through the multi-layer convolutional neural network of the model for feature extraction and analysis.

[0281] The working principle of the YOLOv8 model is to divide the input image into SxS grids, and each grid is responsible for predicting the targets contained therein. For each grid cell, the model predicts B bounding boxes and their confidence scores, as well as the probability distribution of C classes.

[0282] In the examination room monitoring application, the predefined violation behavior types usually include but are not limited to: whispering (communication between two or more examinees), passing objects, using unauthorized electronic devices (such as mobile phones, smart watches), looking at others' papers, using hidden reference materials, leaving the seat, etc. The YOLOv8 model analyzes each frame of image and outputs the detected violation behavior type, location (represented by a bounding box), and confidence score.

[0283] Compared with traditional manual monitoring, the automated recognition system can monitor multiple examination rooms simultaneously and will not miss any violation behavior due to fatigue or distraction, greatly improving the efficiency and fairness of the invigilation. In addition, the recognition method based on deep learning can adapt to different environmental conditions and behavior changes, and through continuous learning and updating, the recognition accuracy can be continuously improved.

[0284] For each type of violation, record the start and end time of the violation. This step is a key link for time dimension tracking and recording of violation events after the YOLOv8 model successfully identifies the violation. In actual implementation, the time record of the violation adopts a continuous tracking mechanism, and the specific process is as follows: when the YOLOv8 model first detects a specific type of violation (such as using a mobile phone) in a frame and the confidence exceeds the preset threshold (usually set to 0.75 or 0.8), the system will record the current timestamp as the start time of the violation. This timestamp usually contains information such as year, month, day, hour, minute, second, and millisecond to ensure the accuracy of time recording. For example, if examinee A is detected using a mobile phone at 09:45:23.456 on June 15, 2023, this time point will be recorded as the start time of the violation.

[0285] After that, the existence of the violation in subsequent frames is continuously tracked. Since the video is a continuous sequence of frames, the violation is usually detected in multiple consecutive frames. To avoid mistakenly considering the violation as ended due to temporary detection failure (such as temporary non-detection due to occlusion, light change, etc.), a time tolerance window (usually 0.5-2 seconds) can be set. Only when the violation is not detected in multiple consecutive frames or exceeds the preset time window, it is considered that the violation has ended, and the timestamp of the last time the behavior is detected is recorded as the end time.

[0286] For multiple possible violations, the system will maintain time records for each type of violation. For example, if an examinee first whispers (09:30:15-09:30:45) and then uses a mobile phone (09:35:20-09:35:40), the start and end times of the two different types of violations are recorded separately. The data structure of the time record usually includes the following fields: violation ID (unique identifier), violation type (such as "using a mobile phone", "whispering", etc.), start time (timestamp accurate to milliseconds), end time (timestamp accurate to milliseconds), duration (end time minus start time), violation subject (such as examinee ID or seat number), confidence score (model's confidence in judging the violation), etc.

[0287] Within the preset time interval, extract key video frames at a frequency of m frames per second, and add timestamps, violation type labels, and violation region bounding boxes to each key video frame, to ensure evidence integrity while avoiding excessive storage and processing of redundant data, improving system efficiency and usability.

[0288] The preset time interval generally refers to a time range of several seconds before the start time to several seconds after the end time of the violation behavior. For example, if the system records that a cross-talking behavior occurs at 09:45:30 to 09:45:50, the preset time interval can be set to 09:45:25 to 09:45:55, i.e. extending 5 seconds before and after the violation behavior to capture the complete behavior process and context.

[0289] The extraction frequency parameter m of the key video frame is an adjustable value, which is generally determined according to the type, duration of the violation behavior and system resource limit. For a rapidly changing violation behavior (such as passing an article), a higher frame rate (such as m = 10, i.e. extracting 10 frames per second) can be required to ensure that the key moment of the behavior is captured; while for a violation behavior with a longer duration and slow change (such as long-time viewing of non-permitted materials), a lower frame rate (such as m = 2) can be used to save storage space. In practical applications, the m value is usually set between 2 and 15 to balance the evidence integrity and system resource consumption.

[0290] The extraction of key video frames can adopt uniform sampling or intelligent sampling. Uniform sampling means that every 1 / m second within the preset time interval, a frame is extracted; while intelligent sampling dynamically adjusts the sampling frequency based on factors such as the degree of change in the frame content, the confidence score of the violation behavior, etc., and extracts more frames at key moments (such as when the violation behavior just starts or is about to end) and fewer frames at relatively static stages.

[0291] For each extracted key video frame, the following labeling information can be added: timestamp, violation type label and violation region bounding box.

[0292] Specifically, the timestamp can accurately record the collection time of the frame, usually including date and time information, and the timestamp can be added directly on the image (usually at the top left corner or top right corner of the image) or stored as metadata together with the image. The violation type label can clearly label the type of violation behavior detected in the frame, such as "using mobile phone", "cross-talking", etc. When multiple violation behaviors are detected in a frame, all detected types will be listed. The label is usually overlaid on the image in text form, located near the corresponding violation region or placed uniformly at a specific position on the image. The violation region bounding box can be precisely marked by drawing a rectangular box on the image to indicate the location where the violation behavior occurs. The bounding box is usually drawn in a prominent color (such as red, yellow), and the line width and style can be adjusted as needed to improve visibility. For different types of violation behaviors, different colored bounding boxes can be used for differentiation. The coordinate information (x, y coordinates of the top left corner and bottom right corner) of the bounding box is also saved as metadata for subsequent analysis and processing.

[0293] In addition, other auxiliary information can be added on the key frame, such as the confidence score of the violation, the examination room number, the examinee information (such as the seat number), etc., to enhance the completeness and traceability of the evidence.

[0294] Finally, the key video frames are sorted in chronological order, and the violation evidence package is generated and stored in the security database of the control equipment, and the violation index of the security database is constructed. Specifically, first, the chronological ordering of key video frames is the basis for constructing a coherent evidence chain. Although the timestamp of each frame has been recorded during the extraction process, due to factors such as parallel processing or network transmission delay, the received frames may be in a time sequence disorder. Therefore, the system will strictly sort the time sequence according to the timestamp information of each frame to ensure the time continuity of the evidence. The sorting algorithm usually uses a preset sorting method to ensure that frames with the same timestamp maintain the original relative order. The sorted frame sequence can clearly show the complete development process of the violation, from the state before the behavior starts, to the occurrence, duration and end of the violation, forming a complete timeline evidence chain.

[0295] Next, the sorted key video frames are organized into a structured violation evidence package. The evidence package is a logical data collection that contains all evidence materials and metadata related to a specific violation event. A typical violation evidence package includes the following components: 1. Evidence package metadata: including unique identifier (UUID), violation type, violation time range (start time and end time), violation location (examination room number), involved personnel information (such as examinee ID or seat number), evidence generation time, evidence package version number, etc. Basic information. 2. Key video frame set: all annotated key video frames arranged in chronological order, each frame containing timestamp, violation type label and violation area bounding box, etc. Information. 3. Violation summary: a brief description of the violation, including behavior type, duration, severity assessment, etc. Information for quick understanding of the violation. 4. Technical verification information: including digital signature, hash value, etc., for verifying the integrity and authenticity of the evidence, preventing evidence tampering. 5. Access control information: defines which roles or users have access to the evidence package, and the operations that can be performed by different permission levels (such as viewing, exporting, deleting, etc.).

[0296] The evidence package is usually organized in a standardized data format (such as JSON, XML or proprietary binary format), and may be compressed and encrypted to reduce storage space requirements and protect sensitive information.

[0297] The generated evidence package of the violation behavior is securely stored in the security database of the central control device. The security database is a database system specially designed for storing sensitive data, with advanced security features such as access control, data encryption, audit logs, etc. During storage, the following operations are performed: 1. Data verification: check the integrity and format correctness of the evidence package, ensure that all necessary fields are filled and conform to the expected format. 2. Data encryption: encrypt the evidence package, usually using high-strength encryption algorithms such as AES-256, to ensure that even if the database is accessed by unauthorized persons, the evidence content will not be leaked. 3. Access control settings: according to the permission control information in the evidence package, set the corresponding database access permissions, ensure that only authorized users can access specific evidence packages. 4. Storage confirmation: after completing the storage operation, generate storage confirmation information, including storage time, storage location, storage status, etc., and record it in the system log.

[0298] Finally, build a violation behavior index for the security database, which is a key data structure for improving evidence retrieval and query efficiency. The violation behavior index can be set up based on the time dimension, that is, based on the occurrence time of the violation behavior, to support time range queries (such as finding all violation behaviors in a specific date or time period).

[0299] The implementation of the index usually uses efficient data structures such as B+ tree, hash table or inverted index to ensure fast queries even on large data sets.

[0300] This embodiment uses the pre-built YOLOv8 deep learning model to analyze video frames in real time, which can accurately identify predefined types of violation behaviors such as whispering, using mobile phones, etc. The labeled key frames are organized in chronological order into violation behavior evidence packages, stored in the security database and constructed with multi-dimensional indexes to ensure the integrity, security and retrievability of the evidence. This greatly improves the efficiency and accuracy of the examination room monitoring, reduces the workload of the invigilators; provides objective and detailed evidence of violation behavior, reducing the possibility of disputes and complaints; through data analysis, it reveals the patterns and trends of violation behavior, providing a basis for optimizing examination management.

[0301] In one of the embodiments of the present embodiment, the inspection device transmits the examination room image to the central control device, including the following steps:

[0302] S910, the inspection device encodes and compresses the examination room image using H.264, establishes an RTMP streaming channel, and pushes the encoded data stream to the central control device;

[0303] S920, the encoded data stream is encrypted using AES-256 to obtain an encrypted data stream;

[0304] S930, when the inspection device detects a network interruption, the encrypted data stream is stored in a local cache, and after detecting a network recovery, the encrypted data stream is transmitted to the control device.

[0305] In the process of H.264 encoding and compression of the examination room image, the inspection device first collects the original image data of the examination room, which is usually in uncompressed YUV or RGB format and has a large amount of data. In order to improve the transmission efficiency, the image is compressed using the H.264 encoding algorithm. The H.264 encoding process includes macroblock division, intra-frame prediction, inter-frame prediction, transform quantization and entropy encoding, etc. Specifically, the image is first divided into 16x16 pixel macroblocks, and then each macroblock is predicted and encoded, including I frame (intra-frame prediction), P frame (forward prediction) and B frame (bidirectional prediction). Intra-frame prediction uses the adjacent pixels of the already encoded current frame for prediction, while inter-frame prediction uses similar areas in the reference frame for motion estimation and compensation. The residual data after prediction is processed by discrete cosine transform (DCT) and quantization to further reduce data redundancy.

[0306] Finally, the quantized coefficients are losslessly compressed by entropy encoding methods such as CAVLC or CABAC. H.264 encoding can reduce the data volume to 1 / 50 to 1 / 100 of the original data while maintaining high image quality. After encoding, the inspection device establishes an RTMP (Real-Time Messaging Protocol) push channel, which is based on TCP and can provide stable streaming media transmission.

[0307] The RTMP channel establishment process includes three stages of handshake, connection and push, through which the encoded data stream is pushed to the control device to realize real-time transmission of the monitoring picture.

[0308] The encoded data stream is encrypted to ensure the security of the examination room monitoring data. Specifically, the H.264 encoded data stream is encrypted using the AES-256 encryption algorithm. AES (Advanced Encryption Standard) is a symmetric encryption algorithm, and its 256-bit key length provides extremely high security. The encryption process first requires generating a 256-bit random key, which can be shared between the inspection device and the central control device through a secure key exchange protocol. During encryption, the data stream is divided into 128-bit (16-byte) data blocks, and each data block undergoes four transformation operations: SubBytes (byte substitution), ShiftRows (row shifting), MixColumns (column mixing), and AddRoundKey (round key addition), for a total of 14 rounds of transformation. The encryption mode uses CBC (Cipher Block Chaining) or GCM (Galois / Counter Mode) mode to enhance encryption strength and prevent replay attacks. Encrypted data cannot be decrypted without the correct key, even if intercepted, thus ensuring the confidentiality and integrity of the examination room monitoring data during transmission. The encrypted data stream maintains the same format structure as the original encoded data, but the content has been randomized, ensuring that only authorized central control devices can correctly decrypt and restore the monitoring picture.

[0309] The network interruption handling mechanism is a key technology to ensure the integrity of the monitoring data. When the inspection device detects a network interruption, it will start the local caching mechanism to ensure that the monitoring data will not be lost due to network problems. Network interruption detection uses multiple mechanisms, including TCP connection state monitoring, ICMP probe packet (ping) detection, and heartbeat packet mechanism. When consecutive multiple (usually 3-5) probes fail or the heartbeat packet times out without response, it is determined that the network is interrupted. After the network interruption, the inspection device redirects the encrypted data stream to the local storage device, which can use a circular buffer (Circular Buffer) structure for storage, which can maximize the storage of the latest monitoring data in a limited storage space. During storage, the timestamp and sequence number of each data packet are recorded to facilitate subsequent retransmission in chronological order. Local storage uses high-speed flash memory or SSD to ensure that the write speed meets the storage needs of real-time monitoring data. At the same time, data integrity verification mechanisms such as CRC32 or MD5 checksum are implemented to prevent data corruption during storage. When the network is restored, the connection is confirmed to be stable through the aforementioned network detection mechanism, and the data retransmission process is started. The retransmission adopts a priority strategy, first transmitting real-time data, and then transmitting buffered data in chronological order to ensure the continuity of monitoring. During the retransmission process, flow control is implemented to avoid network congestion or interruption caused by sudden large traffic. Through this mechanism, even in an unstable network environment, the complete recording and transmission of examination room monitoring data can be ensured.

[0310] The embodiment significantly reduces the bandwidth requirement under the premise of ensuring image quality, and the encryption mechanism ensures that the monitoring data is not accessed or tampered with by unauthorized persons during transmission, thereby ensuring the security and fairness of the examination process. The network interruption processing mechanism solves the problem of network instability in practical applications, ensures the continuity and integrity of the monitoring data, and does not lose key monitoring pictures due to network fluctuations, so that the examination monitoring system has the characteristics of high efficiency, safety and reliability, and can meet the strict monitoring requirements of various examination scenes, thereby providing a solid technical guarantee for the fairness of the examination.

[0311] As shown in Figure 2 The embodiment of the application also provides a patrol inspection system, which comprises:

[0312] A central control device, which is used for executing the intelligent patrol inspection method for the special operation physical examination room.

[0313] A patrol inspection device, which is wirelessly or wiredly connected to the central control device.

[0314] Those skilled in the art should understand that the embodiments of the application can be provided as a method, a system or a computer program product. Therefore, the application can adopt a completely hardware embodiment, a completely software embodiment or an embodiment combining software and hardware aspects. Moreover, the application can adopt the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0315] The application is described with reference to the flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams and the combination of the flows and / or blocks can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The functions specified in one flow or multiple flows and / or blocks

[0316] These computer program instructions can also be stored in a computer readable storage medium, which can guide the computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer readable storage medium produce a product including instruction devices, which implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The functions specified in one flow or multiple flows and / or blocks

[0317] These computer program instructions can also be loaded into computer or other programmable data processing devices to cause a series of operational steps to be performed on the computer or other programmable devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable devices provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 Figure 1 One or more flowchart or block diagrams can be used to describe embodiments of the present application as can be set forth in the accompanying drawings. It will be understood that each block of the flowchart or block diagrams, and combinations of blocks in the flowchart or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable device, implement the functions specified in the flowchart or block diagram block or blocks.

[0318] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0319] The memory can include non-persistent memory and / or persistent memory, such as flash memory, read-only memory (ROM), and / or other types of nonvolatile memory. The memory is an example of computer-readable media.

[0320] Computer-readable media includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0321] It should also be noted that the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0322] ​The above merely provides an example of the present application, and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall fall into the scope of claims of the present application.

Claims

1. An intelligent inspection method for special operation physical examination center, characterized in that, The method applied to the central control device comprises: Real-time acquisition of examination room data and environmental parameters, and establishment of a real scene navigation map based on the examination room data, wherein the real scene navigation map is used for real-time linkage with the examination room data; Real-time acquisition of examination room images by a patrol device, wherein the patrol device is wirelessly or wiredly connected to the central control device, the patrol device is provided with a camera, and the camera is used for collecting the examination room images; Extraction of image features of the examination room images, and division of a patrol path into n regions based on the image features, wherein n is a positive integer; Based on the real scene navigation map and the image features, a dynamic path planning strategy is executed to generate a patrol path of the patrol device; Fusion analysis of the environmental parameters and the image features is performed to determine a congestion index on the patrol path; Based on the congestion index, a dangerous region is determined in the n regions, and the dangerous region is visualized; The dynamic path planning strategy comprises: Acquisition of historical patrol data; Based on an improved A The algorithm calculates a globally optimal path, wherein the improved A The algorithm is improved based on a heuristic function; Real-time monitoring of people flow density and obstacle distribution on the patrol path; Determination of whether there is a path blockage region and / or a dangerous region in a global optimal path based on the people flow density and the obstacle distribution; In the case that there is a path blockage and / or a dangerous region in the global optimal path, a local path re-planning strategy is executed, and a final path is generated; The local path re-planning strategy comprises: Identification of boundary coordinates of the path blockage region and / or the dangerous region; In the real scene navigation map, high passage cost values are assigned to grid nodes of the path blockage region and / or the dangerous region; Dijkstra algorithm is applied to calculate the local optimal path from the current position to the target inspection point Lite algorithm calculates the local optimal path, wherein the target inspection point is the grid node closest to the grid nodes in the distance path blocked area and / or dangerous area outside the boundary coordinates; When multiple continuous path blockage regions and / or dangerous regions are detected, a segmented planning strategy is adopted to decompose the path into multiple sub-paths; An ant colony optimization algorithm is applied to each sub-path to generate a locally optimal sub-path; All the locally optimal sub-paths are connected to obtain a locally re-planned path.

2. The method of claim 1, wherein, The examination room data comprises three-dimensional space data, and the real scene navigation map is established based on the examination room data, comprising: Conversion of the three-dimensional space data into a grid map; Semantic segmentation of the three-dimensional space data is performed by using a pre-constructed neural network model to obtain a semantic segmentation result; The semantic segmentation result is fused with the grid map to obtain a real scene navigation map comprising semantic information.

3. The method of claim 1, wherein, Based on the real scene navigation map and the image features, a dynamic path planning strategy is executed to generate a patrol path of the patrol device, comprising: The real scene navigation map is divided into multiple grid nodes, and passage cost values of each grid node are calculated; In response to a setting instruction of a user on the real scene navigation map, a patrol point in the real scene navigation map and a weight coefficient corresponding to each patrol point are determined; Using the improved A An initial global path is calculated using an algorithm; Real-time acquired environmental parameters and image features are taken as input parameters of a pre-constructed multi-feature fusion model to dynamically update the passage cost values of each grid node; When a change amount of the passage cost values exceeds a preset threshold, a preset path re-planning algorithm is adopted to generate a new patrol path; When the change amount of the passage cost values does not exceed the preset threshold, an initial global path is taken as the patrol path of the patrol device.

4. The method of claim 1, wherein, Based on the image features, the patrol path is divided into n regions, comprising: Based on a K-means clustering algorithm, the image features are clustered to obtain at least one clustering region; Calculate the feature vector of each cluster area, and calculate the similarity between the feature vectors; According to the similarity between the feature vectors, the inspection path is divided into n continuous areas, and each area is assigned a unique identifier.

5. The method of claim 1, wherein, Fusion analysis of environmental parameters and image features to determine the congestion index on the inspection path, including: Extracting personnel density, equipment distribution and operation behavior from image features through YOLOv8 model; Output the congestion index through the multilayer perceptron neural network model, wherein the input layer of the multilayer perceptron neural network model includes environmental parameters, personnel density, equipment distribution and operation behavior, and the output layer is the congestion index.

6. The method of claim 5, wherein, Based on the congestion index, determine the dangerous area in the n areas, including: When the congestion index of an area is lower than a preset first congestion index threshold T1, mark the area as a safe area; When the congestion index of an area is between the first congestion index threshold T1 and the second congestion index threshold T2, mark the area as a warning area; When the congestion index of an area is higher than the second congestion index threshold T2, mark the area as a dangerous area, wherein 0 7. The method of claim 1, wherein, The camera is also used to capture the examination room video, which includes multiple video frames. The method further comprises: Input the video frame into the pre-constructed YOLOv8 model to identify the pre-defined violation behavior type; For each violation behavior type, record the start time and end time of the violation behavior; Within a preset time interval, extract key video frames at a frequency of m frames per second, and add a timestamp, a violation type label and a violation area bounding box to each key video frame; Sort the key video frames in chronological order, generate a violation behavior evidence package, store the violation behavior evidence package in the security database of the central control device, and build a violation behavior index of the security database.

8. The method of claim 1, wherein, The inspection device transmits the examination room image to the central control device, including: The inspection device encodes and compresses the examination room image using H.264, establishes an RTMP streaming channel, and pushes the encoded data stream to the central control device; AES-256 encrypt the encoded data stream to get the encrypted data stream; When the inspection device detects a network interruption, store the encrypted data stream in the local cache, and transmit the encrypted data stream to the central control device after detecting network recovery.

9. A patrol system characterized by comprising: Including: The central control device is used to execute the intelligent inspection method for the special operation physical examination room according to any one of claims 1-8; The inspection device is wirelessly or wired connected to the central control device.

Images