High-risk operation site active management and control method and device based on edge calculation, and electronic equipment

Abstract

The invention discloses a high-risk operation site active management and control method and device based on edge computing and electronic equipment. The method comprises the following steps: for received image video data, generating feature points in a feature pyramid of a backbone network by utilizing Ghost convolution and path aggregation, and outputting image recognition data through a cross-scale attention mechanism; performing dynamic loudness normalization processing on the input audio data to obtain feature data of the audio data; and calculating a fusion weight for the feature data of the image recognition data and the audio data based on a similarity weight, constructing a time sequence reasoning chain, and generating an emergency plan. According to the invention, the effectiveness of safety management is improved, and a better working environment is provided for operators.

Description

Technical Field

[0001] The embodiments of the present application relate to monitoring and safety assurance technologies for temporary work sites, and in particular to an active management and control method and device for high-risk work sites based on edge computing, as well as electronic equipment. Background Art

[0002] In today's industrial production landscape, especially in high-risk or high-risk industries like thermal power plants, petrochemicals, and mining, safety is paramount. All workers entering high-risk work sites are required to exercise safety discipline in accordance with safety standards and requirements. Traditional safety management models, relying primarily on visual monitoring, manual record-keeping, and manual inspections, are not only subject to the risks of human negligence and inaccurate information, but also often lead to a formalized approach to safety management, making effective safety supervision difficult and posing a significant challenge to safe production. In particular, since maintenance sites at power plants and mining operations are often located in specific locations such as oil tanks or in suburban areas, network signals in these areas are often inaccessible. This is especially true in confined work environments, where on-site work footage cannot be transmitted, leading to blind spots for safety inspections and on-site monitoring. Consequently, violations by workers cannot be detected and stopped promptly. At the same time, on-site management is prone to human negligence, and safety supervision is difficult to be in place, and safety systems and processes are difficult to implement effectively; manual recording and paper retention of on-site information not only have the hidden dangers of human negligence and inaccurate information, but are also difficult to quickly query and trace; evidence of violations is difficult to record and retain, and there is no data and image support, resulting in insufficient evidence for subsequent assessments, causing disputes and difficulties in closed-loop management; multi-level management linkage and coordination are difficult, and often one link is missing and events are fragmented.

[0003] In particular, temporary work sites, such as energy facility maintenance and road emergency repairs, have their own particularities that are different from fixed industrial scenarios:

[0004] 1. Spatiotemporal uncertainty: The operation area is subject to dynamic changes. For example, the scope of emergency repair operations for oil and gas pipelines may expand by 30-50 meters per day (see data from the "2023 China Energy Industry Emergency Operations White Paper");

[0005] 2. Environmental complexity: In typical scenarios, electromagnetic interference (up to 80dBμV / m around the substation), dust (PM2.5 concentration > 300μg / m 3 ), corrosive gases and other multiple interference factors;

[0006] 3. Resource constraints: 75% of temporary operations last less than 72 hours (statistics from the National Safety Production Emergency Rescue Center), requiring equipment to have rapid deployment capabilities.

[0007] The current industry generally adopts a discrete model of video surveillance + portable detectors + paper records, which has the following technical bottlenecks:

[0008] Data coordination failure: The independent operation of each subsystem leads to inconsistent data time scales. Actual measurements show that when a gas detector alarm sounds, it takes the video system an average of 6.2 seconds to retrieve the corresponding image.

[0009] Inefficient human-computer interaction: Operators need to switch between multiple terminals. A maintenance case study at a thermal power plant showed that workers walked an average of 8.7 kilometers per day.

[0010] Insufficient scalability: The existing system interface protocols are fragmented, such as the mixed use of Modbus, HTTP / HTTPS, MQTT and other protocols, and new devices require customized development interfaces.

[0011] The 4G / 5G-based cloud management platforms that have emerged in recent years have fundamental flaws:

[0012] Network dependency: In scenarios such as tunnels and underground pipeline corridors, network latency can fluctuate by 500-2000ms.

[0013] Cost: HD video streaming generates approximately 1.2TB / point of traffic per month.

[0014] Response delay: The end-to-end delay from data collection to cloud-returned instructions is typically 3-5 seconds, which cannot meet the emergency response requirement of within 1 second specified in GB 30871-2022.

[0015] Therefore, there is an urgent need to supervise operating behaviors in special operating scenarios to ensure operational safety, especially for temporary sites of special operations such as power plants and mines. An automatic supervision system that automatically monitors their operating behaviors is needed. Unfortunately, there is currently no relevant technology available for reference. Summary of the Invention

[0016] In view of this, embodiments of the present application provide a method and device, as well as electronic equipment, for active management and control of high-risk work sites based on edge computing.

[0017] According to a first aspect of an embodiment of the present application, a method for proactively managing and controlling high-risk work sites based on edge computing is provided, comprising:

[0018] Receiving image and video data, and performing convolution on the image and video data using a backbone network in a depthwise separable convolution and a pointwise convolution manner;

[0019] Dynamic pruning based on channel importance in the backbone network is used to sort channels and retain the top n channels. Ghost convolution and path aggregation are used to generate feature points in the backbone network's feature pyramid. Feature points from different levels are dynamically fused through a cross-scale attention mechanism. The input image size is adjusted based on the fused feature points, and the overlap threshold between the predicted and ground-truth boxes for target detection is adjusted based on the feature density of the target object.

[0020] Based on the overlap determination threshold, irrelevant targets at the work site are filtered out and image recognition data is output;

[0021] Perform dynamic loudness normalization on the input audio data, perform multi-scale time-domain convolution calculation on the normalization result, separate the noise component through adversarial training, perform differential pulse detection on the noise-separated audio data, and obtain the feature data of the audio data;

[0022] The feature data of image recognition data and audio data are aligned based on the attention mechanism, a cross-modal attention matrix is ​​generated based on the aligned data, the similarity weights between modalities are calculated, the fusion weights are calculated based on the similarity weights, a temporal reasoning chain is constructed, and the risk transmission path is modeled based on the graph neural network to determine the warning level and generate an emergency plan.

[0023] In some feasible embodiments, the received image and video data are convolved using a backbone network in a depthwise separable convolution and a pointwise convolution manner, including:

[0024] Input tensor X∈RH×W×Cin, convolution kernel Kdepth∈Rk×k×Cin×Cout, output Ydepth∈RH′×W′×Cout, where k×k is the spatial size of the convolution kernel, Cin is the number of input channels, aligned with the input tensor, Cout is the number of output channels, which determines the depth of the output tensor, H is the height of the input feature map, W is the width of the input feature map, H′ is the height of the output feature map, H′=H or H′=H / s, s is the stride, W′ is the width of the output feature map, W′=W or W′=W / s;

[0025] Based on the image and video data, the convolution calculation is performed using the following input convolution calculation formula:

[0026]

[0027] For the depth convolution calculation results, the number of output channels is adjusted by the following point-by-point convolution method:

[0028] Input Ydepth∈RH′×W′×Cin, convolution kernel Kpoint∈R1×1×Cin×Cout, output Y(h′,w′,c out)∈RH′×W′×Cout:

[0029]

[0030] Let Y(h′,w′,c out ) as the convolution result.

[0031] In some feasible embodiments, the Ghost convolution and path aggregation are used to generate feature points in the feature pyramid of the backbone network, and the feature points at different levels are dynamically integrated through the cross-scale attention mechanism, including:

[0032] For the feature map Xi∈RH×W×C of the i-th layer of the backbone network, some inherent features Y′=Xi*W′ are generated, where W′∈Rk×k×C×mk is the convolution kernel size, m is the number of output channels, and m<<C;

[0033] Perform depth convolution on Y′ channel by channel to generate redundant features Y ij :

[0034] Y ij =Φ j (Y′ i ),j=1,…,v

[0035] v is the number of Ghosts of each set of inherent features, Φj() is the linear transformation operation;

[0036] Output Ghost feature map Yi = Concat([Y′,Y11,...,Ymv])∈RH×W×C′;

[0037] Upsample the high-level features and add them element-wise to the low-level features:

[0038] P i =Upsample(P i+1 )+Y i

[0039] Downsample low-level features and fuse them with high-level features:

[0040] Q i =Conv 1×1 (Y i )+Downsample(Q i-1 )

[0041] Perform bidirectional aggregation on the features after upsampling fusion and downsampling fusion:

[0042] in, Indicates channel splicing, F i ∈RH×W×D;

[0043] For the features after bidirectional aggregation, the heat map is generated by the following convolution:

[0044] H i =σ(Conv 1×1 (F i )), where σ() is the Sigmoid activation function;

[0045] Perform non-maximum suppression (NMS) and threshold filtering on the heat map to obtain the coordinates of the feature points.

[0046] In some feasible embodiments, adjusting the overlap determination threshold between the predicted frame and the real frame of target detection based on the feature density of the target object, filtering out irrelevant targets at the work site based on the overlap determination threshold, and outputting image recognition data includes:

[0047] The target detection model outputs a set of predicted boxes B = {b1, b2, ..., bn}, where each box bi = (x, y, w, h, s), (x, y) represents the coordinates, w, h, s represent the width, height and confidence respectively;

[0048] For each predicted box bi, calculate the density of other boxes in the surrounding k neighborhood as follows:

[0049]

[0050] Where σ is the neighborhood radius hyperparameter;

[0051] Normalize the prediction box density:

[0052]

[0053] For the normalized prediction box density Set the basic IoU threshold τ base Set as the set threshold, and perform dynamic threshold τ by the following formula i Adjustment:

[0054] α is the adjustment coefficient, which is a set constant;

[0055] Based on the predicted box bi, the real box G = {g1,...,gm}, and the dynamic threshold τi, the IoU is calculated as follows:

[0056]

[0057] If gj∈G makes IoU(bi,gj)≥τi, then bi is retained, otherwise it is an irrelevant target.

[0058] In some feasible embodiments, the step of performing dynamic loudness normalization on the input audio data, performing multi-scale time-domain convolution calculation on the normalization result, separating noise components through adversarial training, and performing differential pulse detection on the noise-separated audio data to obtain feature data of the audio data includes:

[0059] Eliminate the loudness difference of the audio data x(t) and calculate the RMS energy within the sliding window w:

[0060] Among them, E RMS [n] represents the RMS energy value of the nth frame in the audio data;

[0061] To E RMS [n] Perform dynamic gain adjustment as follows:

[0062] Among them, x norm (t) is the normalized output audio data, Ltarget is the target loudness level, and ∈ is a small constant to prevent zero division;

[0063] x norm (t) Parallel multi-branch convolution captures different time domain features. The output of the k-th branch is as follows:

[0064]

[0065] Among them, L k is the convolution kernel length of the kth branch, w k (τ) is the convolution kernel weight of the kth branch, b k is the bias term of the k-th branch, Y(t) is the output result of multi-scale feature concatenation, Y(t) = [y1(t); y2(t); ...; yK(t)];

[0066] The output result Y(t) of multi-scale feature concatenation is subjected to adversarial training noise separation as follows:

[0067]

[0068] M(t) is the output noise mask, Estimated value of the separated audio.

[0069] In some feasible embodiments, the step of aligning the feature data of the image recognition data and the audio data based on an attention mechanism includes:

[0070] Use the two-stream Transformer architecture to process image and audio features separately, and build a cross-modal attention mechanism in the feature space as follows:

[0071] Among them, Falign is the aligned fusion feature matrix, Q = Wq, Fimg is the image feature query, Wq∈Rdk×dimg; K = Wk, Faudio is the audio feature key matrix, Wk∈Rdk×daudio; V = Wv, Faudio is the audio eigenvalue matrix, Wv∈Rdv×daudio, dk is the key vector dimension;

[0072] The computing power support unit generates a cross-modal attention matrix based on the aligned data and calculates the inter-modal similarity weight, including:

[0073]

[0074] where αij is the attention between the i-th image feature and the j-th audio feature, sim(·) is the cosine similarity function, τ is the temperature coefficient, and fiimg is the i-th image feature vector.

[0075] In some feasible embodiments, the method further includes:

[0076] In response to receiving the first control operation instruction, parsing the first control operation instruction to obtain at least location information and control period information of the operation control;

[0077] Obtaining current location information, planning a travel route based on the current location information and location information of the operation control, and determining a speed and duration for the individual soldier control system to travel to the operation control location based on the travel route;

[0078] In response to receiving a travel instruction, or based on the arrival of an operation time limit, the individual soldier control system is triggered to start the drive motor and move along the travel route to the operation control position.

[0079] According to a second aspect of the present application, there is provided an active control device for high-risk work sites based on edge computing, comprising:

[0080] The convolution unit is used to perform depth-wise separable convolution and point-wise convolution on the received image and video data using the backbone network;

[0081] The image recognition unit is used to sort channels based on dynamic pruning of channel importance in the backbone network and retain the top n channels. It uses Ghost convolution and path aggregation to generate feature points in the feature pyramid of the backbone network and dynamically fuses feature points from different levels through a cross-scale attention mechanism. The input image size is adjusted based on the fused feature points. The overlap determination threshold between the predicted box and the true box of the target detection is adjusted according to the feature density of the target object. Based on the overlap determination threshold, irrelevant targets at the work site are filtered out and the image recognition data is output.

[0082] An audio feature extraction unit is used to perform dynamic loudness normalization on the input audio data, perform multi-scale time-domain convolution calculation on the normalization result, separate the noise component through adversarial training, and perform differential pulse detection on the audio data after noise separation to obtain feature data of the audio data;

[0083] The emergency plan generation unit is used to align the feature data of image recognition data and audio data based on the attention mechanism, generate a cross-modal attention matrix based on the aligned data, calculate the similarity weights between modalities, calculate the fusion weights based on the similarity weights, build a temporal reasoning chain, determine the warning level based on the risk transmission path modeled by the graph neural network, and generate an emergency plan.

[0084] In some feasible embodiments, the convolution unit is further configured to:

[0085] Input tensor X∈RH×W×Cin, convolution kernel Kdepth∈Rk×k×Cin×Cout, output Ydepth∈RH′×W′×Cout, where k×k is the spatial size of the convolution kernel, Cin is the number of input channels, aligned with the input tensor, Cout is the number of output channels, which determines the depth of the output tensor, H is the height of the input feature map, W is the width of the input feature map, H′ is the height of the output feature map, H′=H or H′=H / s, s is the stride, W′ is the width of the output feature map, W′=W or W′=W / s;

[0086] Based on the image and video data, the convolution calculation is performed using the following input convolution calculation formula:

[0087]

[0088] For the depth convolution calculation results, the number of output channels is adjusted by the following point-by-point convolution method:

[0089] Input Ydepth∈RH′×W′×Cin, convolution kernel Kpoint∈R1×1×Cin×Cout, output Y(h′,w′,c out )∈RH′×W′×Cout:

[0090]

[0091] Let Y(h′,w′,c out ) as the convolution result.

[0092] In some feasible embodiments, the image recognition unit is further configured to:

[0093] For the feature map Xi∈RH×W×C of the i-th layer of the backbone network, some inherent features Y′=Xi*W′ are generated, where W′∈Rk×k×C×mk is the convolution kernel size, m is the number of output channels, and m<<C;

[0094] Perform depth convolution on Y′ channel by channel to generate redundant features Y ij :

[0095] Y ij =Φ j (Y′ i ),j=1,…,v

[0096] v is the number of Ghosts of each set of inherent features, Φj() is the linear transformation operation;

[0097] Output Ghost feature map Yi = Concat([Y′,Y11,...,Ymv])∈RH×W×C′;

[0098] Upsample the high-level features and add them element-wise to the low-level features:

[0099] P i =Upsample(P i+1 )+Y i

[0100] Downsample low-level features and fuse them with high-level features:

[0101] Q i =Conv 1×1 (Y i )+Downsample(Q i-1 )

[0102] Perform bidirectional aggregation on the features after upsampling fusion and downsampling fusion:

[0103] ,in, Indicates channel splicing, F i ∈RH×W×D;

[0104] For the features after bidirectional aggregation, the heat map is generated by the following convolution:

[0105] H i =σ(Conv 1×1 (F i )), where σ() is the Sigmoid activation function;

[0106] Perform non-maximum suppression (NMS) and threshold filtering on the heat map to obtain the coordinates of the feature points.

[0107] In some feasible embodiments, the image recognition unit is further configured to:

[0108] The target detection model outputs a set of predicted boxes B = {b1, b2, ..., bn}, where each box bi = (x, y, w, h, s), (x, y) represents the coordinates, w, h, s represent the width, height and confidence respectively;

[0109] For each predicted box bi, calculate the density of other boxes in the surrounding k neighborhood as follows:

[0110]

[0111] Where σ is the neighborhood radius hyperparameter;

[0112] Normalize the prediction box density:

[0113]

[0114] For the normalized prediction box density Set the basic IoU threshold τ base Set as the set threshold, and perform dynamic threshold τ by the following formula i Adjustment:

[0115] α is the adjustment coefficient, which is a set constant;

[0116] Based on the predicted box bi, the real box G = {g1,...,gm}, and the dynamic threshold τi, the IoU is calculated as follows:

[0117]

[0118] If gj∈G makes IoU(bi,gj)≥τi, then bi is retained, otherwise it is an irrelevant target.

[0119] In some feasible embodiments, the audio feature extraction unit is further configured to:

[0120] Eliminate the loudness difference of the audio data x(t) and calculate the RMS energy within the sliding window w:

[0121] Among them, E RMS [n] represents the RMS energy value of the nth frame in the audio data;

[0122] To E RMS [n] Perform dynamic gain adjustment as follows:

[0123] Among them, x norm (t) is the normalized output audio data, Ltarget is the target loudness level, and ∈ is a small constant to prevent zero division;

[0124] x norm (t) Parallel multi-branch convolution captures different time domain features. The output of the k-th branch is as follows:

[0125]

[0126] Among them, L k is the convolution kernel length of the kth branch, w k (τ) is the convolution kernel weight of the kth branch, b k is the bias term of the k-th branch, Y(t) is the output result of multi-scale feature concatenation, Y(t) = [y1(t); y2(t); ...; yK(t)];

[0127] The output result Y(t) of multi-scale feature concatenation is subjected to adversarial training noise separation as follows:

[0128]

[0129] M(t) is the output noise mask, Estimated value of the separated audio.

[0130] In some feasible embodiments, the plan generating unit is further configured to:

[0131] Use the two-stream Transformer architecture to process image and audio features separately, and build a cross-modal attention mechanism in the feature space as follows:

[0132] Among them, Falign is the aligned fusion feature matrix, Q = Wq, Fimg is the image feature query, Wq∈Rdk×dimg; K = Wk, Faudio is the audio feature key matrix, Wk∈Rdk×daudio; V = Wv, Faudio is the audio eigenvalue matrix, Wv∈Rdv×daudio, dk is the key vector dimension;

[0133] Correspondingly, generating a cross-modal attention matrix based on the aligned data and calculating the inter-modal similarity weights include:

[0134]

[0135] where αij is the attention between the i-th image feature and the j-th audio feature, sim(·) is the cosine similarity function, τ is the temperature coefficient, and fiimg is the i-th image feature vector.

[0136] In some feasible embodiments, the apparatus further includes:

[0137] a parsing unit, configured to, in response to receiving a first control operation instruction, parse the first control operation instruction to obtain at least location information and control period information of the operation control;

[0138] a path planning unit, configured to obtain current location information, plan a travel route based on the current location information and the location information of the operation control, and determine a speed and duration for the individual soldier control system to travel to the location of the operation control based on the travel route;

[0139] The trigger unit is used to trigger the individual soldier control system to start the drive motor in response to receiving a travel instruction or based on the arrival of an operation time limit, and move to the operation control position along the travel route.

[0140] According to a third aspect of the present application, there is provided an electronic device, comprising:

[0141] at least one processor; and

[0142] a memory communicatively connected to the at least one processor; wherein,

[0143] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the high-risk work site active management and control method based on edge computing.

[0144] Compared with traditional safety management methods, the technical solution of the embodiment of the present application adopts a pre-loaded Linux real-time kernel, which greatly shortens the cold start time. The video analysis of the embodiment of the present application adopts a pruned YOLOv5s model, and the gas sensor data is jointly processed by Kalman filtering + LSTM time series prediction. By automatically collecting videos or images, through the active management mode, workers involved in high-risk operations must perform their operations in accordance with the relevant requirements of construction safeguards and risk treatment plans. Through the biometric recognition and identity recognition of the workers, it can more intuitively assist enterprise safety management personnel to fully grasp the safety status of the work site, form an effective management method, and urge management and execution personnel at all levels to consciously practice the "three current management", namely the management principles of on-site, actual objects, and reality, and take the initiative to go deep into the site, thereby maximizing the safety management efficiency, strictly ensuring the safety of high-risk work sites and their periphery, improving the effectiveness of safety management, and providing workers with a better working environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0145] To more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be derived from these drawings without inventive effort.

[0146] Figure 1 A schematic diagram of the structure of an active control system for high-risk work sites based on edge computing according to an embodiment of the present application is shown;

[0147] Figure 2 This is a flow chart of a method for proactively managing and controlling high-risk work sites based on edge computing according to an embodiment of the present application;

[0148] Figure 3 This is a schematic diagram of the structure of an active control device for high-risk work sites based on edge computing according to an embodiment of the present application;

[0149] Figure 4 This is a schematic diagram of the composition structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0150] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0151] The essence of the technical solution of the embodiment of the present application is explained in detail below with reference to the accompanying drawings.

[0152] Figure 1 The schematic diagram of the structure of the high-risk operation site active management and control system based on edge computing in the embodiment of the present application is shown as follows: Figure 1 As shown, the high-risk operation site active management and control system based on edge computing of the embodiment of the present application can be used as a single-soldier management and control system. It is a portable box with rollers and a built-in modular cabin. Based on this single-soldier management and control system, active management and control of high-risk operation sites can be achieved. This high-risk operation site active management and control system based on edge computing integrates the following components:

[0153] Human-computer interaction unit: Includes a liftable touchscreen and voiceprint recognition module, supporting biometric authentication and remote intercom. The liftable display uses a worm gear drive to achieve the display's rise and retraction. The worm is supported by a sliding slot and moves within the slot, enabling the display to move and support its rise and retraction functions.

[0154] Computing support unit: A CPU / GPU heterogeneous architecture deploys an edge computing engine to enable multimodal data processing. For example, the CPU / GPU heterogeneous architecture uses an Intel i7-1185G7 (4 cores / 8 threads) to process routine business logic and an NVIDIA Jetson Xavier NX (384 CUDA cores) for sensor fusion computing. The two are interconnected via PCIe 3.0 x 4 channels.

[0155] Environmental perception unit: includes a control ball array, gas sensors, and infrared thermal imagers to collect environmental data.

[0156] Security management unit: includes electronic fence, access control counter and security situation assessment model.

[0157] Communication unit: 5G multi-link aggregation module, supporting edge-cloud collaboration.

[0158] Power management unit: hot-swappable battery packs and multi-source power adapters.

[0159] Mobile inspection unit: including smart helmets, drones and other dynamic monitoring equipment.

[0160] The circuit boards in the communication unit and computing power support unit feature a conformal coating. Interfaces utilize silicone seals and drain valves for protection. Sensor signal transmission utilizes a differential RS-485 bus, and video cables are equipped with magnetic rings to suppress interference in the 100MHz-1GHz frequency band.

[0161] The individual control system of the embodiment of the present application may include a main box frame, a rapid deployment mechanism, a damping roller assembly, an equipment installation cabin, a stress dispersion structure, etc. The main box frame of the mobile load-bearing unit is made of aviation aluminum alloy, and its box body is provided with reinforcing ribs; the rapid deployment mechanism has a four-link hydraulic system and an automatic locking device, the hydraulic cylinder thrust can be 1200N, and the automatic locking device has an electromagnetic latch and redundant mechanical clips. The equipment installation cabin has a modular slot, wherein the computing power support unit cabin is provided with a heat dissipation fin, the sensor cabin is an EMI shielding cabin, and the battery cabin is an explosion-proof structure. The rapid deployment mechanism adopts a hinge structure, such as a hinge structure composed of a stainless steel pin shaft; the roller on the main box frame is installed through a flange mounting plate. The computing power support unit cabin is designed with a slide rail and a guide pin positioning method. A phase change material insulation layer is provided between the battery cabin and the computing power support unit cabin.

[0162] The security management unit of the application embodiment includes: a multi-band positioning system of the electronic fence, which is used to perform real-time detection of personnel and work tools entering and leaving the site, and to alarm for objects entering and leaving non-working areas; a personnel density warning unit, which detects the number and density of personnel on site based on the fusion results of access control counting and video analysis, and outputs alarm information when the requirements are not met; an emergency plan database, which associates risk types, generates disposal plans through risk warning models, and sends them to the human-computer interaction unit and the background control center.

[0163] The multi-band geo-fence positioning system utilizes multi-source positioning fusion, such as RFID, UWB, and Bluetooth 5.1. RFID uses passive tags deployed on tools and helmets, UWB employs the TDOA positioning algorithm, and Bluetooth AoA uses phase difference positioning. The positioning engine utilizes a particle filter algorithm to fuse multi-source data and dynamically assign weights. The crowd density warning unit uses dual card readers for tailgating prevention (through time difference detection), uploads data over LoRaWAN, utilizes YOLOv5s and DeepSORT for real-time head detection and tracking, and employs the CSRNet network to generate crowd heat maps.

[0164] The communication unit of the embodiment of the present application includes: a channel quality perception module that dynamically selects 5G / satellite / ad hoc network links; a data hierarchical transmission strategy that prioritizes high-bandwidth channels for critical alarm data; and an end-to-end encryption module that supports national secret algorithms and quantum key distribution.

[0165] The power management unit of the embodiment of the present application includes: a battery health monitoring system for predicting the remaining life based on impedance spectrum analysis; a fast battery replacement mechanism for supporting the replacement of battery modules without power outage; and an intelligent distribution box for supporting automatic correction of three-phase imbalance.

[0166] Figure 2 This is a flow chart of the active control method for high-risk work sites based on edge computing according to an embodiment of the present application. Figure 2 As shown, the high-risk operation site active management and control method based on edge computing in the embodiment of the present application includes:

[0167] In step 201 , the received image and video data are convolved using a backbone network in a depthwise separable convolution and a pointwise convolution manner.

[0168] Specifically, the input tensor X∈R H×W×Cin , convolution kernel Output Among them, k×k is the spatial size of the convolution kernel, C in is the number of input channels, aligned with the input tensor, C outis the number of output channels, which determines the depth of the output tensor. H is the height of the input feature map, W is the width of the input feature map, H′ is the height of the output feature map, H′=H or H′=H / s, s is the step size, and W′ is the width of the output feature map, W′=W or W′=W / s;

[0169] Based on the image and video data, the convolution calculation is performed using the following input convolution calculation formula:

[0170]

[0171] For the depth convolution calculation results, the number of output channels is adjusted by the following point-by-point convolution method:

[0172] Enter Y depth ∈R H′×W′×Cin , convolution kernel K point ∈R 1×1×Cin×Cout , output Y(h′,w′,c out )∈R H ′×W′×Cout :

[0173]

[0174] Let Y(h′,w′,c out ) as the convolution result.

[0175] Step 202: Dynamically prune the channels based on their importance in the backbone network and retain the first n channels. Ghost convolution and path aggregation are used to generate feature points in the feature pyramid of the backbone network. Feature points at different levels are dynamically fused through a cross-scale attention mechanism. The input image size is adjusted based on the fused feature points. The overlap threshold between the predicted box and the true box of the target detection is adjusted according to the feature density of the target object. Based on the overlap threshold, irrelevant targets at the work site are filtered out and image recognition data is output.

[0176] Among them, Ghost convolution and path aggregation are used to generate feature points in the feature pyramid of the backbone network, and the cross-scale attention mechanism is used to dynamically fuse feature points at different levels, including:

[0177] For the feature map X of the i-th layer of the backbone network i ∈R H×W×C , generate some inherent features Y′=X i *W′,W′∈R k ×k×C×m k is the convolution kernel size, m is the number of output channels, m< <C;

[0178] Perform depth convolution on Y′ channel by channel to generate redundant features Y ij :

[0179] Y ij =Φ j (Y′ i ),j=1,…,v

[0180] v is the number of Ghosts of each set of inherent features, Φ j () is a linear transformation operation;

[0181] Output Ghost feature map Y i =Concat([Y′,Y 11 ,...,Y mv ])∈R H×W×C′ ;

[0182] Upsample the high-level features and add them element-wise to the low-level features:

[0183] P i =Upsample(P i+1 )+Y i

[0184] Downsample low-level features and fuse them with high-level features:

[0185] Q i =Conv 1×1 (Y i )+Downsample(Q i-1 )

[0186] Perform bidirectional aggregation on the features after upsampling fusion and downsampling fusion:

[0187] ,in, Indicates channel splicing, F i ∈R H×W×D ;

[0188] For the features after bidirectional aggregation, the heat map is generated by the following convolution:

[0189] H i =σ(Conv 1×1 (F i )), where σ() is the Sigmoid activation function;

[0190] Perform non-maximum suppression (NMS) and threshold filtering on the heat map to obtain the coordinates of the feature points.

[0191] According to the feature density of the target object, the overlap threshold between the predicted frame and the real frame of the target detection is adjusted. Based on the overlap threshold, irrelevant targets at the work site are filtered out and the image recognition data is output, including:

[0192] The prediction box set B output by the target detection model is {b1,b2,...,b n}, where each box b i =(x, y, w, h, s), (x, y) represents the coordinates, w, h, s represent the width, height and confidence respectively;

[0193] For each prediction box b i , calculate the density of other boxes in the surrounding k neighborhood as follows:

[0194]

[0195] Where σ is the neighborhood radius hyperparameter;

[0196] Normalize the prediction box density:

[0197]

[0198] For the normalized prediction box density Set the basic IoU threshold τ base Set as the set threshold, and perform dynamic threshold τ by the following formula i Adjustment:

[0199] α is the adjustment coefficient, which is a set constant;

[0200] Based on the predicted box b i , ground truth box G = {g1,...,g m}, dynamic threshold τ i , calculate IoU as follows:

[0201]

[0202] If g j ∈G such that IoU(b i ,g j )≥τ i , then keep b i , otherwise it is irrelevant target.

[0203] In step 203, dynamic loudness normalization is performed on the input audio data, multi-scale time-domain convolution calculation is performed on the normalization result, noise components are separated through adversarial training, and differential pulse detection is performed on the audio data after noise separation to obtain feature data of the audio data.

[0204] Eliminate the loudness difference of the audio data x(t) and calculate the RMS energy within the sliding window w:

[0205] Among them, E RMS[n] represents the RMS energy value of the nth frame in the audio data;

[0206] To E RMS [n] Perform dynamic gain adjustment as follows:

[0207] Among them, x norm (t) is the normalized output audio data, L target is the target loudness level, ∈ is a small constant to prevent zero;

[0208] x norm (t) Parallel multi-branch convolution captures different time domain features. The output of the k-th branch is as follows:

[0209]

[0210] Among them, L k is the convolution kernel length of the kth branch, w k (τ) is the convolution kernel weight of the kth branch, b k is the bias term of the k-th branch, Y(t) is the output result of multi-scale feature concatenation, Y(t) = [y1(t); y2(t); ...; y K (t)];

[0211] The output result Y(t) of multi-scale feature concatenation is subjected to adversarial training noise separation as follows:

[0212]

[0213] M(t) is the output noise mask, Estimated value of the separated audio.

[0214] In step 204, feature alignment is performed on the feature data of the image recognition data and the audio data based on the attention mechanism, a cross-modal attention matrix is ​​generated based on the aligned data, the similarity weights between the modalities are calculated, the fusion weights are calculated based on the similarity weights, a temporal reasoning chain is constructed, and the risk transmission path is modeled based on the graph neural network to determine the warning level, and an emergency plan is generated, and the aligned data is sent to the communication unit.

[0215] Among them, feature alignment of the feature data of image recognition data and audio data is performed based on the attention mechanism, including:

[0216] Use the two-stream Transformer architecture to process image and audio features separately, and build a cross-modal attention mechanism in the feature space as follows:

[0217] Among them, F align is the aligned fusion feature matrix, Q = W q , Fimg is the image feature query, K=W k , F audio is the audio feature key matrix, V=Wv,F audio is the audio eigenvalue matrix, d k is the key vector dimension;

[0218] Among them, the cross-modal attention matrix is ​​generated based on the aligned data, and the inter-modal similarity weight is calculated, including:

[0219]

[0220] Among them, α ij is the attention between the i-th image feature and the j-th audio feature, sim(·) is the cosine similarity function, τ is the temperature coefficient, and f i img is the i-th image feature vector.

[0221] The video analysis of the embodiment of the present application adopts the pruned YOLOv5s model, and the gas sensor data is jointly processed by Kalman filtering + LSTM time series prediction. By automatically collecting videos or images, through the active management mode, workers involved in high-risk operations must perform their operations in accordance with the relevant requirements of construction safeguards and risk management plans. Through the biometric recognition and identity recognition of the workers, it can more intuitively assist the enterprise safety management personnel to fully grasp the safety status of the work site, form an effective management method, and urge management and execution personnel at all levels to consciously practice the "three current management", namely the management principles of on-site, actual objects, and reality, and take the initiative to go deep into the site, thereby maximizing the safety management efficiency, strictly ensuring the safety of high-risk work sites and their periphery, improving the effectiveness of safety management, and providing workers with a better working environment.

[0222] Figure 3 This is a structural diagram of an active control device for high-risk work sites based on edge computing according to an embodiment of the present application. Figure 3 As shown, the high-risk operation site active management and control device based on edge computing in an embodiment of the present application includes:

[0223] The convolution unit 30 is used to perform convolution on the received image and video data using a backbone network in a depthwise separable convolution and a pointwise convolution manner;

[0224] Image recognition unit 31, used to perform channel sorting based on dynamic pruning of channel importance in the backbone network, retain the top n channels, generate feature points in the feature pyramid of the backbone network using Ghost convolution and path aggregation, dynamically fuse feature points from different levels through a cross-scale attention mechanism, adjust the input image size based on the fused feature points, and adjust the overlap determination threshold between the predicted box and the true box of target detection based on the feature density of the target object. Based on the overlap determination threshold, irrelevant targets at the work site are filtered out and image recognition data is output;

[0225] An audio feature extraction unit 32 is configured to perform dynamic loudness normalization on the input audio data, perform multi-scale time-domain convolution calculation on the normalization result, separate the noise component through adversarial training, and perform differential pulse detection on the audio data after noise separation to obtain feature data of the audio data;

[0226] The emergency plan generation unit 33 is used to perform feature alignment on the feature data of image recognition data and audio data based on the attention mechanism, generate a cross-modal attention matrix based on the aligned data, calculate the similarity weight between modalities, calculate the fusion weight based on the similarity weight, build a temporal reasoning chain, determine the warning level based on the risk transmission path modeled by the graph neural network, and generate an emergency plan.

[0227] In some feasible embodiments, the convolution unit 30 is further configured to:

[0228] Input tensor X∈RH×W×Cin, convolution kernel Kdepth∈Rk×k×Cin×Cout, output Ydepth∈RH′×W′×Cout, where k×k is the spatial size of the convolution kernel, Cin is the number of input channels, aligned with the input tensor, Cout is the number of output channels, which determines the depth of the output tensor, H is the height of the input feature map, W is the width of the input feature map, H′ is the height of the output feature map, H′=H or H′=H / s, s is the stride, W′ is the width of the output feature map, W′=W or W′=W / s;

[0229] Based on the image and video data, the convolution calculation is performed using the following input convolution calculation formula:

[0230]

[0231] For the depth convolution calculation results, the number of output channels is adjusted by the following point-by-point convolution method:

[0232] Input Ydepth∈RH′×W′×Cin, convolution kernel Kpoint∈R1×1×Cin×Cout, output Y(h′,w′,c out )∈RH′×W′×Cout:

[0233]

[0234] Let Y(h′,w′,c out ) as the convolution result.

[0235] In some feasible embodiments, the image recognition unit 31 is further configured to:

[0236] For the feature map Xi∈RH×W×C of the i-th layer of the backbone network, some inherent features Y′=Xi*W′ are generated, where W′∈Rk×k×C×mk is the convolution kernel size, m is the number of output channels, and m<<C;

[0237] Perform depth convolution on Y′ channel by channel to generate redundant features Y ij :

[0238] Y ij =Φ j (Y′ i ),j=1,…,v

[0239] v is the number of Ghosts of each set of inherent features, Φj() is the linear transformation operation;

[0240] Output Ghost feature map Yi = Concat([Y′,Y11,...,Ymv])∈RH×W×C′;

[0241] Upsample the high-level features and add them element-wise to the low-level features:

[0242] P i =Upsample(P i+1 )+Y i

[0243] Downsample low-level features and fuse them with high-level features:

[0244] Q i =Conv 1×1 (Y i )+Downsample(Q i-1 )

[0245] Perform bidirectional aggregation on the features after upsampling fusion and downsampling fusion:

[0246] ,in, Indicates channel splicing, F i ∈RH×W×D;

[0247] For the features after bidirectional aggregation, the heat map is generated by the following convolution:

[0248] Hi =σ(Conv 1×1 (F i )), where σ() is the Sigmoid activation function;

[0249] Perform non-maximum suppression (NMS) and threshold filtering on the heat map to obtain the coordinates of the feature points.

[0250] In some feasible embodiments, the image recognition unit 31 is further configured to:

[0251] The target detection model outputs a set of predicted boxes B = {b1, b2, ..., bn}, where each box bi = (x, y, w, h, s), (x, y) represents the coordinates, w, h, s represent the width, height and confidence respectively;

[0252] For each predicted box bi, calculate the density of other boxes in the surrounding k neighborhood as follows:

[0253]

[0254] Where σ is the neighborhood radius hyperparameter;

[0255] Normalize the prediction box density:

[0256]

[0257] For the normalized prediction box density Set the basic IoU threshold τ base Set as the set threshold, and perform dynamic threshold τ by the following formula i Adjustment:

[0258] α is the adjustment coefficient, which is a set constant;

[0259] Based on the predicted box bi, the real box G = {g1,...,gm}, and the dynamic threshold τi, the IoU is calculated as follows:

[0260]

[0261] If gj∈G makes IoU(bi,gj)≥τi, then bi is retained, otherwise it is an irrelevant target.

[0262] In some feasible embodiments, the audio feature extraction unit 32 is further configured to:

[0263] Eliminate the loudness difference of the audio data x(t) and calculate the RMS energy within the sliding window w:

[0264] Among them, E RMS[n] represents the RMS energy value of the nth frame in the audio data;

[0265] To E RMS [n] Perform dynamic gain adjustment as follows:

[0266] Among them, x norm (t) is the normalized output audio data, Ltarget is the target loudness level, and ∈ is a small constant to prevent zero division;

[0267] x norm (t) Parallel multi-branch convolution captures different time domain features. The output of the k-th branch is as follows:

[0268]

[0269] Among them, L k is the convolution kernel length of the kth branch, w k (τ) is the convolution kernel weight of the kth branch, b k is the bias term of the k-th branch, Y(t) is the output result of multi-scale feature concatenation, Y(t) = [y1(t); y2(t); ...; yK(t)];

[0270] The output result Y(t) of multi-scale feature concatenation is subjected to adversarial training noise separation as follows:

[0271]

[0272] M(t) is the output noise mask, Estimated value of the separated audio.

[0273] In some feasible embodiments, the plan generating unit 33 is further configured to:

[0274] Use the two-stream Transformer architecture to process image and audio features separately, and build a cross-modal attention mechanism in the feature space as follows:

[0275] Among them, Falign is the aligned fusion feature matrix, Q = Wq, Fimg is the image feature query, Wq∈Rdk×dimg; K = Wk, Faudio is the audio feature key matrix, Wk∈Rdk×daudio; V = Wv, Faudio is the audio eigenvalue matrix, Wv∈Rdv×daudio, dk is the key vector dimension;

[0276] Correspondingly, generating a cross-modal attention matrix based on the aligned data and calculating the inter-modal similarity weights include:

[0277]

[0278] where αij is the attention between the i-th image feature and the j-th audio feature, sim(·) is the cosine similarity function, τ is the temperature coefficient, and fiimg is the i-th image feature vector.

[0279] In some feasible embodiments, the apparatus further includes:

[0280] Parsing unit ( Figure 3 (not shown) for parsing the first control operation instruction in response to receiving the first control operation instruction, and obtaining at least the location information and the control period information of the operation control;

[0281] Path planning unit ( Figure 3 (not shown) for obtaining current position information, planning a travel route based on the current position information and the position information of the operation control, and determining the speed and duration of travel to the operation control position for the individual soldier control system based on the travel route;

[0282] Trigger unit ( Figure 3 (not shown) is used to trigger the individual soldier control system to start the drive motor in response to receiving a travel instruction or based on the arrival of the operation time limit, and move to the operation control position along the travel route.

[0283] In an exemplary embodiment, the aforementioned units may be implemented by one or more central processing units (CPU), graphics processing units (GPU), application-specific integrated circuits (ASIC), DSPs, programmable logic devices (PLD), complex programmable logic devices (CPLD), field-programmable gate arrays (FPGA), general-purpose processors, controllers, microcontrollers (MCU), microprocessors, or other electronic components.

[0284] Regarding the device in the above embodiment, the specific manner in which each module and unit performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0285] Figure 4This is a schematic diagram of the structure of the electronic device according to the embodiment of the present application. Figure 4 As shown, the electronic device 800 supports multi-screen output, and the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0286] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 802 may include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

[0287] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0288] The power component 806 provides power to the various components of the electronic device 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0289] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0290] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0291] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0292] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the electronic device 800. The sensor assembly 814 can also detect changes in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and temperature changes of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0293] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as Wi-Fi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0294] In an exemplary embodiment, the electronic device 800 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the steps of the high-risk work site active management and control method based on edge computing of the above-mentioned embodiment.

[0295] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is further provided, such as a memory 804 including instructions. The instructions can be executed by the processor 820 of the electronic device 800 to complete the steps of the high-risk job site active management and control method based on edge computing of the above embodiment. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0296] It should be understood that “one embodiment” or “an embodiment” mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, “in one embodiment” or “in an embodiment” appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The serial numbers of the embodiments of the present application mentioned above are for description only and do not represent the advantages and disadvantages of the embodiments.

[0297] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0298] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is merely a logical functional division. In actual implementation, other division methods may be used, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not exist.

[0299] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0300] The above description is merely an embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A high-risk operation site active management and control method based on edge computing, characterized in that: The method comprises: Receiving image and video data, and performing convolution on the image and video data using a backbone network in a depthwise separable convolution and a pointwise convolution manner; Dynamic pruning based on channel importance in the backbone network is used to sort channels and retain the top n channels. Ghost convolution and path aggregation are used to generate feature points in the backbone network's feature pyramid. Feature points from different levels are dynamically fused through a cross-scale attention mechanism. The input image size is adjusted based on the fused feature points, and the overlap threshold between the predicted and ground-truth boxes for target detection is adjusted based on the feature density of the target object. Based on the overlap determination threshold, irrelevant targets at the work site are filtered out and image recognition data is output; Perform dynamic loudness normalization on the input audio data, perform multi-scale time-domain convolution calculation on the normalization result, separate the noise component through adversarial training, perform differential pulse detection on the noise-separated audio data, and obtain the feature data of the audio data; The feature data of image recognition data and audio data are aligned based on the attention mechanism, a cross-modal attention matrix is ​​generated based on the aligned data, the similarity weights between modalities are calculated, the fusion weights are calculated based on the similarity weights, a temporal reasoning chain is constructed, and the risk transmission path is modeled based on the graph neural network to determine the warning level and generate an emergency plan.

2. The method according to claim 1, characterized in that The received image and video data are convolved using a backbone network in a depthwise separable convolution and a pointwise convolution manner, including: Input tensor X∈RH×W×Cin, convolution kernel Kdepth∈Rk×k×Cin×Cout, output Ydepth∈RH′×W′×Cout, where k×k is the spatial size of the convolution kernel, Cin is the number of input channels, aligned with the input tensor, Cout is the number of output channels, which determines the depth of the output tensor, H is the height of the input feature map, W is the width of the input feature map, H′ is the height of the output feature map, H′=H or H′=H / s, s is the stride, W′ is the width of the output feature map, W′=W or W′=W / s; Based on the image and video data, the convolution calculation is performed using the following input convolution calculation formula: For the depth convolution calculation results, the number of output channels is adjusted by the following point-by-point convolution method: Input Ydepth∈RH′×W′×Cin, convolution kernel Kpoint∈R1×1×Cin×Cout, output Y(h′,w′,c out )∈RH′×W′×Cout: Let Y(h′,w′,c out ) as the convolution result.

3. The method according to claim 1, characterized in that The method uses Ghost convolution and path aggregation to generate feature points in the feature pyramid of the backbone network, and dynamically fuses feature points at different levels through a cross-scale attention mechanism, including: For the feature map Xi∈RH×W×C of the i-th layer of the backbone network, some inherent features Y′=Xi*W′ are generated, where W′∈Rk×k×C×mk is the convolution kernel size, m is the number of output channels, and m< <C; Perform depth convolution on Y′ channel by channel to generate redundant features Y ij : Y ij =Φ j (Y i ′),j=1,…,v v is the number of Ghosts of each set of inherent features, Φj() is the linear transformation operation; Output Ghost feature map Yi = Concat([Y′,Y11,...,Ymv])∈RH×W×C′; Upsample the high-level features and add them element-wise to the low-level features: P i =Upsample(P i+1 )+Y i Downsample low-level features and fuse them with high-level features: Q i =Conv 1×1 (Y i )+Downsample(Q i-1 ) Perform bidirectional aggregation on the features after upsampling fusion and downsampling fusion: in, Indicates channel splicing, F i ∈RH×W×D; For the features after bidirectional aggregation, the heat map is generated by the following convolution: H i =σ(Conv 1×1 (F i )), where σ() is the Sigmoid activation function; Perform non-maximum suppression (NMS) and threshold filtering on the heat map to obtain the coordinates of the feature points.

4. The method according to claim 1, wherein The method adjusts the overlap determination threshold between the predicted frame and the real frame of target detection according to the feature density of the target object, filters out irrelevant targets at the work site based on the overlap determination threshold, and outputs image recognition data, including: The target detection model outputs a set of predicted boxes B = {b1, b2, ..., bn}, where each box bi = (x, y, w, h, s), (x, y) represents the coordinates, w, h, s represent the width, height and confidence respectively; For each predicted box bi, calculate the density of other boxes in the surrounding k neighborhood as follows: Where σ is the neighborhood radius hyperparameter; Normalize the prediction box density: For the normalized prediction box density Set the basic IoU threshold τ base Set as the set threshold, and perform dynamic threshold τ by the following formula i Adjustment: α is the adjustment coefficient, which is a set constant; Based on the predicted box bi, the real box G = {g1,...,gm}, and the dynamic threshold τi, the IoU is calculated as follows: If gj∈G makes IoU(bi,gj)≥τi, then bi is retained, otherwise it is an irrelevant target.

5. The method according to claim 1, characterized in that The method includes performing dynamic loudness normalization processing on the input audio data, performing multi-scale time domain convolution calculation on the normalization processing result, separating the noise component through adversarial training, and performing differential pulse detection on the audio data after noise separation to obtain feature data of the audio data, including: Eliminate the loudness difference of the audio data x(t) and calculate the RMS energy within the sliding window w: Among them, E RMS [n] represents the RMS energy value of the nth frame in the audio data; To E RMS [n] Perform dynamic gain adjustment as follows: Among them, x norm (t) is the normalized output audio data, Ltarget is the target loudness level, and ∈ is a small constant to prevent zero division; x norm (t) Parallel multi-branch convolution captures different time domain features. The output of the k-th branch is as follows: Among them, L k is the convolution kernel length of the kth branch, w k (τ) is the convolution kernel weight of the kth branch, b k is the bias term of the k-th branch, Y(t) is the output result of multi-scale feature concatenation, Y(t) = [y1(t); y2(t); ...; yK(t)]; The output result Y(t) of multi-scale feature concatenation is subjected to adversarial training noise separation as follows: M(t) is the output noise mask, Estimated value of the separated audio.

6. The method according to claim 1, characterized in that The feature alignment of the feature data of the image recognition data and the audio data based on the attention mechanism includes: Use the two-stream Transformer architecture to process image and audio features separately, and build a cross-modal attention mechanism in the feature space as follows: Among them, Falign is the aligned fusion feature matrix, Q = Wq, Fimg is the image feature query, Wq∈Rdk×dimg; K = Wk, Faudio is the audio feature key matrix, Wk∈Rdk×daudio; V = Wv, Faudio is the audio eigenvalue matrix, Wv∈Rdv×daudio, dk is the key vector dimension; The generation of a cross-modal attention matrix based on the aligned data and calculation of the inter-modal similarity weights include: where αij is the attention between the i-th image feature and the j-th audio feature, sim(·) is the cosine similarity function, τ is the temperature coefficient, and fiimg is the i-th image feature vector.

7. The method according to claim 1, characterized in that The method further comprises: In response to receiving the first control operation instruction, parsing the first control operation instruction to obtain at least location information and control period information of the operation control; Obtaining current location information, planning a travel route based on the current location information and location information of the operation control, and determining a speed and duration for the individual soldier control system to travel to the operation control location based on the travel route; In response to receiving a travel instruction, or based on the arrival of an operation time limit, the individual soldier control system is triggered to start the drive motor and move along the travel route to the operation control position.

8. An active control device for high-risk work sites based on edge computing, characterized in that: The device comprises: The convolution unit is used to perform depth-wise separable convolution and point-wise convolution on the received image and video data using the backbone network; The image recognition unit is used to sort channels based on dynamic pruning of channel importance in the backbone network and retain the top n channels. It uses Ghost convolution and path aggregation to generate feature points in the feature pyramid of the backbone network and dynamically fuses feature points from different levels through a cross-scale attention mechanism. The input image size is adjusted based on the fused feature points. The overlap determination threshold between the predicted box and the true box of the target detection is adjusted according to the feature density of the target object. Based on the overlap determination threshold, irrelevant targets at the work site are filtered out and the image recognition data is output. An audio feature extraction unit is used to perform dynamic loudness normalization on the input audio data, perform multi-scale time-domain convolution calculation on the normalization result, separate the noise component through adversarial training, and perform differential pulse detection on the audio data after noise separation to obtain feature data of the audio data; The emergency plan generation unit is used to align the feature data of image recognition data and audio data based on the attention mechanism, generate a cross-modal attention matrix based on the aligned data, calculate the similarity weights between modalities, calculate the fusion weights based on the similarity weights, build a temporal reasoning chain, determine the warning level based on the risk transmission path modeled by the graph neural network, and generate an emergency plan.

9. The device according to claim 8, characterized in that The convolution unit is further used to: Input tensor X∈RH×W×Cin, convolution kernel Kdepth∈Rk×k×Cin×Cout, output Ydepth∈RH′×W′×Cout, where k×k is the spatial size of the convolution kernel, Cin is the number of input channels, aligned with the input tensor, Cout is the number of output channels, which determines the depth of the output tensor, H is the height of the input feature map, W is the width of the input feature map, H′ is the height of the output feature map, H′=H or H′=H / s, s is the stride, W′ is the width of the output feature map, W′=W or W′=W / s; Based on the image and video data, the convolution calculation is performed using the following input convolution calculation formula: For the depth convolution calculation results, the number of output channels is adjusted by the following point-by-point convolution method: Input Ydepth∈RH′×W′×Cin, convolution kernel Kpoint∈R1×1×Cin×Cout, output Y(h′,w′,c out )∈RH′×W′×Cout: Let Y(h′,w′,c out ) as the convolution result.

10. The device according to claim 8, characterized in that The image recognition unit is further configured to: For the feature map Xi∈RH×W×C of the i-th layer of the backbone network, some inherent features Y′=Xi*W′ are generated, where W′∈Rk×k×C×mk is the convolution kernel size, m is the number of output channels, and m<<C; Perform depth convolution on Y′ channel by channel to generate redundant features Y ij : Y ij =Φ j (Y i ′),j=1,…,v v is the number of Ghosts of each set of inherent features, Φj() is the linear transformation operation; Output Ghost feature map Yi = Concat([Y′,Y11,...,Ymv])∈RH×W×C′; Upsample the high-level features and add them element-wise to the low-level features: P i =Upsample(P i+1 )+Y i Downsample low-level features and fuse them with high-level features: Q i =Conv 1×1 (Y i )+Downsample(Q i-1 ) Perform bidirectional aggregation on the features after upsampling fusion and downsampling fusion: in, Indicates channel splicing, F i ∈RH×W×D; For the features after bidirectional aggregation, the heat map is generated by the following convolution: H i =σ(Conv 1×1 (F i )), where σ() is the Sigmoid activation function; Perform non-maximum suppression (NMS) and threshold filtering on the heat map to obtain the coordinates of the feature points.

11. The device according to claim 8, characterized in that The image recognition unit is further configured to: The target detection model outputs a set of predicted boxes B = {b1, b2, ..., bn}, where each box bi = (x, y, w, h, s), (x, y) represents the coordinates, w, h, s represent the width, height and confidence respectively; For each predicted box bi, calculate the density of other boxes in the surrounding k neighborhood as follows: Where σ is the neighborhood radius hyperparameter; Normalize the prediction box density: For the normalized prediction box density Set the basic IoU threshold τ base Set as the set threshold, and perform dynamic threshold τ by the following formula i Adjustment: α is the adjustment coefficient, which is a set constant; Based on the predicted box bi, the real box G = {g1,...,gm}, and the dynamic threshold τi, the IoU is calculated as follows: If gj∈G makes IoU(bi,gj)≥τi, then bi is retained, otherwise it is an irrelevant target.

12. The device according to claim 8, characterized in that The audio feature extraction unit is further used to: Eliminate the loudness difference of the audio data x(t) and calculate the RMS energy within the sliding window w: Among them, E RMS [n] represents the RMS energy value of the nth frame in the audio data; To E RMS [n] Perform dynamic gain adjustment as follows: Among them, x norm (t) is the normalized output audio data, Ltarget is the target loudness level, and ∈ is a small constant to prevent zero division; x norm (t) Parallel multi-branch convolution captures different time domain features. The output of the k-th branch is as follows: Among them, L k is the convolution kernel length of the kth branch, w k (τ) is the convolution kernel weight of the kth branch, b k is the bias term of the k-th branch, Y(t) is the output result of multi-scale feature concatenation, Y(t) = [y1(t); y2(t); ...; yK(t)]; The output result Y(t) of multi-scale feature concatenation is subjected to adversarial training noise separation as follows: M(t) is the output noise mask, Estimated value of the separated audio.

13. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the high-risk work site active management and control method based on edge computing as described in any one of claims 1 to 7.

Images