Edge computing-based high-risk operation site active management and control method and device and electronic equipment

Abstract

The application discloses an edge computing-based high-risk operation site active management and control method and device and electronic equipment. The method comprises the following steps: generating feature points in a feature pyramid of a backbone network by using Ghost convolution and path aggregation on received image video data, and outputting image recognition data by a cross-scale attention mechanism; performing dynamic loudness normalization processing on input audio data to obtain feature data of the audio data; calculating a fusion weight based on a similarity weight of image recognition data and feature data of the audio data, constructing a time sequence reasoning chain, and generating an emergency plan. The application improves the effectiveness of safety management and provides a better working environment for workers.

Description

Technical Field

[0001] This application relates to monitoring and safety assurance technologies for temporary work sites, and particularly to a proactive management method, device, and electronic device for high-risk work sites based on edge computing. Background Technology

[0002] In today's industrial production sectors, especially in high-risk or high-danger industries such as thermal power plants, petrochemicals, and mining, safety is paramount. All personnel entering high-risk work sites must adhere to safety regulations and requirements to constrain their own safe behavior. Traditional safety management models primarily rely on visual monitoring, manual recording, and manual inspections. These methods not only harbor the risks of human negligence and inaccurate information but also often render safety management merely a formality, hindering effective safety supervision and posing significant challenges to safe production. In particular, because maintenance sites in power plants and mines are often located in specific locations such as inside oil tanks or in suburban areas, network signal coverage is often lacking in these areas. Especially in enclosed work environments, on-site video feeds cannot be transmitted, creating blind spots for safety inspections and on-site monitoring. Consequently, violations by workers cannot be detected and stopped in a timely manner. Meanwhile, on-site management is prone to human negligence, and safety supervision is difficult to implement effectively, making it difficult to enforce safety systems and procedures. Manual recording and paper-based retention of on-site information not only pose risks of human negligence and inaccurate information, but also make it difficult to quickly search and trace. Recording and retaining evidence of violations is difficult, and the lack of data and image support leads to insufficient evidence for later assessments, disputes, and difficulties in closing the management loop. Multi-level management linkage and coordination are difficult, often resulting in missing links and fragmented events.

[0003] In particular, temporary work sites, such as energy facility maintenance and emergency road repairs, have unique characteristics that distinguish them from fixed industrial sites:

[0004] 1. Spatiotemporal uncertainty: The work area has dynamic changing characteristics. For example, the scope of oil and gas pipeline emergency repair work may expand by 30-50 meters every day (see data in the "2023 China Energy Industry Emergency Operation White Paper");

[0005] 2. Environmental complexity: Typical scenarios simultaneously present electromagnetic interference (up to 80 dBμV / m around substations) and dust (PM2.5 concentration > 300 μg / m³). 3 Multiple interfering factors, including corrosive gases;

[0006] 3. Resource constraints: 75% of temporary operations last less than 72 hours (statistics from the National Emergency Rescue Center for Production Safety), requiring equipment with rapid deployment capabilities.

[0007] The current industry standard employs a discrete model combining video surveillance, portable testing equipment, and paper records, which presents the following technical bottlenecks:

[0008] Data collaboration failure: Independent operation of each subsystem leads to inconsistent data timescales. Actual measurements show that when the gas detector alarms, the video system takes an average of 6.2 seconds to retrieve the corresponding footage.

[0009] Inefficient human-computer interaction: Operators need to switch between multiple terminals. A maintenance case at a thermal power plant showed that staff walked an average of 8.7 kilometers per day.

[0010] Insufficient scalability: Existing system interface protocols are fragmented, such as the mixed use of Modbus, HTTP / HTTPS, MQTT and other protocols, requiring custom interface development for new devices.

[0011] The cloud management platforms based on 4G / 5G that have emerged in recent years have fundamental flaws:

[0012] Network dependency: In scenarios such as tunnels and underground utility tunnels, network latency fluctuations can reach 500-2000ms;

[0013] Cost issue: High-definition video streaming incurs a traffic cost of approximately 1.2TB per node per month;

[0014] Response latency: The typical end-to-end latency from data acquisition to the return of instructions from the cloud is 3-5 seconds, which cannot meet the emergency response requirement of within 1 second specified in GB 30871-2022.

[0015] Therefore, there is an urgent need to supervise work activities in special operating scenarios to ensure operational safety. In particular, for temporary sites of special operations such as power plants and mines, an automatic monitoring system is needed to monitor their operations. Unfortunately, there is currently no relevant technology available for reference. Summary of the Invention

[0016] In view of this, embodiments of this application provide a method, apparatus, and electronic device for proactive management of high-risk work sites based on edge computing.

[0017] According to a first aspect of the embodiments of this application, a proactive management method for high-risk work sites based on edge computing is provided, comprising:

[0018] Receive image and video data, and perform convolution on the image and video data using a backbone network with depthwise separable convolution and point convolution.

[0019] Dynamic pruning based on channel importance in the backbone network sorts channels and retains the top n channels. Using Ghost convolution and path aggregation, feature points are generated in the feature pyramid of the backbone network. Through a cross-scale attention mechanism, feature points at different levels are dynamically fused. The size of the input image is adjusted based on the fused feature points, and the overlap threshold between the predicted bounding box and the ground truth bounding box is adjusted according to the feature density of the target object.

[0020] Based on the overlap determination threshold, irrelevant targets at the work site are filtered out, and image recognition data is output.

[0021] The input audio data is subjected to dynamic loudness normalization, the normalization result is subjected to multi-scale temporal convolution calculation, and the noise component is separated through adversarial training. Differential impulse detection is performed on the noise-separated audio data to obtain the feature data of the audio data.

[0022] Feature alignment is performed on image recognition data and audio data based on an attention mechanism. A cross-modal attention matrix is ​​generated based on the aligned data, inter-modal similarity weights are calculated, fusion weights are calculated based on the similarity weights, a temporal inference chain is constructed, and a risk transmission path is modeled based on a graph neural network to determine the warning level and generate an emergency plan.

[0023] In some feasible embodiments, the step of performing convolution on the received image and video data using a backbone network with depthwise separable convolution and pointwise convolution includes:

[0024] The input tensor X∈RH×W×Cin, the convolution kernel Kdepth∈Rk×k×Cin×Cout, and the output Ydepth∈RH′×W′×Cout, where k×k is the spatial size of the convolution kernel, Cin is the number of input channels, aligned with the input tensor, Cout is the number of output channels, which determines the depth of the output tensor, H is the height of the input feature map, W is the width of the input feature map, H′ is the height of the output feature map, H′=H or H′=H / s, where s is the stride, and W′ is the width of the output feature map, W′=W or W′=W / s;

[0025] Based on the image and video data, convolution is performed using the following input convolution calculation formula:

[0026]

[0027] The number of output channels is then adjusted using the following pointwise convolution method based on the depthwise convolution calculation results:

[0028] Input Ydepth∈RH′×W′×Cin, convolution kernel Kpoint∈R1×1×Cin×Cout, output Y(h′,w′,c out)∈RH′×W′×Cout:

[0029]

[0030] Y(h′,w′,c) out ) as the result of convolution.

[0031] In some feasible embodiments, the generation of feature points in the feature pyramid of the backbone network using Ghost convolution and path aggregation, and the dynamic fusion of feature points from different levels through a cross-scale attention mechanism, includes:

[0032] For the feature map Xi∈RH×W×C of the i-th layer of the backbone network, the generated partial intrinsic feature Y′=Xi*W′, where W′∈Rk×k×C×mk is the convolution kernel size, m is the number of output channels, and m<<C;

[0033] Perform depthwise convolution on Y′ channel by channel to generate redundant features Y. ij :

[0034] Y ij =Φ j (Y′ i ),j=1,…,v

[0035] v represents the number of Ghosts in each set of inherent features, and Φj() represents the linear transformation operation;

[0036] Output Ghost feature map Yi = Concat([Y′,Y11,...,Ymv])∈RH×W×C′;

[0037] Upsample the high-level features and add them element-wise to the low-level features:

[0038] P i =Upsample(P i+1 )+Y i

[0039] Downsampling of low-level features and fusing them with high-level features:

[0040] Q i =Conv 1×1 (Y i )+Downsample(Q i-1 )

[0041] The features obtained from upsampling fusion and downsampling fusion are then bidirectionally aggregated:

[0042] in, Indicates channel splicing, F i ∈RH×W×D;

[0043] Heatmaps are generated from the bidirectional aggregated features using the following convolution:

[0044] H i =σ(Conv 1×1 (F i ), where σ() is the Sigmoid activation function;

[0045] Non-maximum suppression (NMS) and threshold filtering are applied to the heatmap to obtain the coordinates of the feature points.

[0046] In some feasible embodiments, adjusting the overlap threshold between the predicted bounding box and the ground truth bounding box of the target detection based on the feature density of the target object, filtering out irrelevant targets in the work site based on the overlap threshold, and outputting image recognition data includes:

[0047] The set of predicted bounding boxes output by the object detection model is B = {b1, b2, ..., bn}, where each box bi = (x, y, w, h, s), (x, y) represents the coordinates, and w, h, and s represent the width, height, and confidence score, respectively.

[0048] For each predicted box bi, calculate the density of other boxes within its k-neighborhood, as follows:

[0049]

[0050] Where σ is the neighborhood radius hyperparameter;

[0051] Normalize the density of the predicted bounding boxes:

[0052]

[0053] For the normalized prediction box density Set the basic IoU threshold τ base Set a threshold value, and use the following formula to calculate the dynamic threshold τ. i Adjustment:

[0054] α is the adjustment coefficient, which is a set constant;

[0055] Based on the predicted bounding box bi, the ground truth bounding box G={g1,...,gm}, and the dynamic threshold τi, the IoU is calculated as follows:

[0056]

[0057] If gj∈G such that IoU(bi,gj)≥τi, then retain bi; otherwise, it is an irrelevant target.

[0058] In some feasible embodiments, the process of performing dynamic loudness normalization on the input audio data, performing multi-scale temporal convolution calculation on the normalization result, separating noise components through adversarial training, and performing differential impulse detection on the noise-separated audio data to obtain feature data of the audio data includes:

[0059] Loudness difference cancellation is performed on the audio data x(t), and the RMS energy within the sliding window w is calculated:

[0060] Among them, E RMS [n] represents the RMS energy value of the nth frame in the audio data;

[0061] For E RMS [n] performs dynamic gain adjustment as follows:

[0062] , where x norm (t) represents the normalized output audio data, Ltarget represents the target loudness level, and ∈ represents the small constant to prevent zeroing;

[0063] For x norm (t) Parallel multi-branch convolution captures different temporal features. The output of the k-th branch is as follows:

[0064]

[0065] Among them, L k w is the kernel length of the k-th branch. k (τ) represents the convolution kernel weight of the k-th branch, b k Y(t) is the bias term for the k-th branch, and Y(t) is the output of the multi-scale feature concatenation, where Y(t) = [y1(t); y2(t); ...; yK(t)].

[0066] The output Y(t) of the multi-scale feature concatenation is subjected to adversarial training noise separation, as follows:

[0067]

[0068] M(t) is the output noise mask. This is the estimated value of the separated audio.

[0069] In some feasible embodiments, the feature alignment of the image recognition data and audio data based on an attention mechanism includes:

[0070] A two-stream Transformer architecture is used to process image and audio features separately, and a cross-modal attention mechanism is constructed in the feature space as follows:

[0071] Where Falign is the aligned fusion feature matrix, Q = Wq, Fimg is the image feature query, Wq ∈ Rdk × dimg; K = Wk, Faudio is the audio feature key matrix, Wk ∈ Rdk × daudio; V = Wv, Faudio is the audio feature value matrix, Wv ∈ Rdv × daudio, dk is the key vector dimension;

[0072] The computing power support unit generates a cross-modal attention matrix based on the alignment data and calculates the inter-modal similarity weights, including:

[0073]

[0074] Where αij is the attention between the i-th image feature and the j-th audio feature, sim(·) is the cosine similarity function, τ is the temperature coefficient, and fiimg is the i-th image feature vector.

[0075] In some feasible embodiments, the method further includes:

[0076] In response to receiving the first control operation instruction, the first control operation instruction is parsed to obtain at least the location information and control time period information of the operation control;

[0077] Obtain current location information, plan a route based on the current location information and the location information of the operation control, and determine the speed and duration for the individual soldier control system to reach the location of the operation control based on the route;

[0078] In response to receiving a movement command or based on the arrival of a work deadline, the individual soldier management system is triggered to start the drive motor and move along the movement route to the work control position.

[0079] According to a second aspect of this application, a proactive control device for high-risk work sites based on edge computing is provided, comprising:

[0080] The convolutional unit is used to perform depthwise separable convolution and pointwise convolution on the received image and video data using the backbone network.

[0081] The image recognition unit is used to perform channel sorting based on dynamic pruning of channel importance in the backbone network and retain the top n channels. It generates feature points in the feature pyramid of the backbone network using Ghost convolution and path aggregation, and dynamically fuses feature points at different levels through a cross-scale attention mechanism. The size of the input image is adjusted based on the fused feature points, and the overlap threshold between the predicted bounding box and the ground truth bounding box is adjusted according to the feature density of the target object. Based on the overlap threshold, irrelevant targets in the work site are filtered out, and image recognition data is output.

[0082] The audio feature extraction unit is used to perform dynamic loudness normalization on the input audio data, perform multi-scale temporal convolution calculation on the normalization result, separate noise components through adversarial training, and perform differential impulse detection on the noise-separated audio data to obtain the feature data of the audio data.

[0083] The contingency plan generation unit is used to perform feature alignment on the feature data of image recognition data and audio data based on the attention mechanism, generate a cross-modal attention matrix based on the aligned data, calculate the inter-modal similarity weights, calculate the fusion weights based on the similarity weights, construct a temporal inference chain, and determine the warning level based on the risk transmission path modeled by the graph neural network, and generate an emergency plan.

[0084] In some implementable embodiments, the convolutional unit is further configured to:

[0085] The input tensor X∈RH×W×Cin, the convolution kernel Kdepth∈Rk×k×Cin×Cout, and the output Ydepth∈RH′×W′×Cout, where k×k is the spatial size of the convolution kernel, Cin is the number of input channels, aligned with the input tensor, Cout is the number of output channels, which determines the depth of the output tensor, H is the height of the input feature map, W is the width of the input feature map, H′ is the height of the output feature map, H′=H or H′=H / s, where s is the stride, and W′ is the width of the output feature map, W′=W or W′=W / s;

[0086] Based on the image and video data, convolution is performed using the following input convolution calculation formula:

[0087]

[0088] The number of output channels is then adjusted using the following pointwise convolution method based on the depthwise convolution calculation results:

[0089] Input Ydepth∈RH′×W′×Cin, convolution kernel Kpoint∈R1×1×Cin×Cout, output Y(h′,w′,c out )∈RH′×W′×Cout:

[0090]

[0091] Y(h′,w′,c) out ) as the result of convolution.

[0092] In some feasible embodiments, the image recognition unit is further configured to:

[0093] For the feature map Xi∈RH×W×C of the i-th layer of the backbone network, the generated partial intrinsic feature Y′=Xi*W′, where W′∈Rk×k×C×mk is the convolution kernel size, m is the number of output channels, and m<<C;

[0094] Perform depthwise convolution on Y′ channel by channel to generate redundant features Y. ij :

[0095] Y ij =Φ j (Y′ i ),j=1,…,v

[0096] v represents the number of Ghosts in each set of inherent features, and Φj() represents the linear transformation operation;

[0097] Output Ghost feature map Yi = Concat([Y′,Y11,...,Ymv])∈RH×W×C′;

[0098] Upsample the high-level features and add them element-wise to the low-level features:

[0099] P i =Upsample(P i+1 )+Y i

[0100] Downsampling of low-level features and fusing them with high-level features:

[0101] Q i =Conv 1×1 (Y i )+Downsample(Q i-1 )

[0102] The features obtained from upsampling fusion and downsampling fusion are then bidirectionally aggregated:

[0103] ,in, Indicates channel splicing, F i ∈RH×W×D;

[0104] Heatmaps are generated from the bidirectional aggregated features using the following convolution:

[0105] H i =σ(Conv 1×1 (F i ), where σ() is the Sigmoid activation function;

[0106] Non-maximum suppression (NMS) and threshold filtering are applied to the heatmap to obtain the coordinates of the feature points.

[0107] In some feasible embodiments, the image recognition unit is further configured to:

[0108] The set of predicted bounding boxes output by the object detection model is B = {b1, b2, ..., bn}, where each box bi = (x, y, w, h, s), (x, y) represents the coordinates, and w, h, and s represent the width, height, and confidence score, respectively.

[0109] For each predicted box bi, calculate the density of other boxes within its k-neighborhood, as follows:

[0110]

[0111] Where σ is the neighborhood radius hyperparameter;

[0112] Normalize the density of the predicted bounding boxes:

[0113]

[0114] For the normalized prediction box density Set the basic IoU threshold τ base Set a threshold value, and use the following formula to calculate the dynamic threshold τ. i Adjustment:

[0115] α is the adjustment coefficient, which is a set constant;

[0116] Based on the predicted bounding box bi, the ground truth bounding box G={g1,...,gm}, and the dynamic threshold τi, the IoU is calculated as follows:

[0117]

[0118] If gj∈G such that IoU(bi,gj)≥τi, then retain bi; otherwise, it is an irrelevant target.

[0119] In some feasible embodiments, the audio feature extraction unit is further configured to:

[0120] Loudness difference cancellation is performed on the audio data x(t), and the RMS energy within the sliding window w is calculated:

[0121] Among them, E RMS [n] represents the RMS energy value of the nth frame in the audio data;

[0122] For E RMS [n] performs dynamic gain adjustment as follows:

[0123] , where x norm (t) represents the normalized output audio data, Ltarget represents the target loudness level, and ∈ represents the small constant to prevent zeroing;

[0124] For x norm (t) Parallel multi-branch convolution captures different temporal features. The output of the k-th branch is as follows:

[0125]

[0126] Among them, L k w is the kernel length of the k-th branch. k (τ) represents the convolution kernel weight of the k-th branch, b k Y(t) is the bias term for the k-th branch, and Y(t) is the output of the multi-scale feature concatenation, where Y(t) = [y1(t); y2(t); ...; yK(t)].

[0127] The output Y(t) of the multi-scale feature concatenation is subjected to adversarial training noise separation, as follows:

[0128]

[0129] M(t) is the output noise mask. This is the estimated value of the separated audio.

[0130] In some feasible embodiments, the plan generation unit is further configured to:

[0131] A two-stream Transformer architecture is used to process image and audio features separately, and a cross-modal attention mechanism is constructed in the feature space as follows:

[0132] Where Falign is the aligned fusion feature matrix, Q = Wq, Fimg is the image feature query, Wq ∈ Rdk × dimg; K = Wk, Faudio is the audio feature key matrix, Wk ∈ Rdk × daudio; V = Wv, Faudio is the audio feature value matrix, Wv ∈ Rdv × daudio, dk is the key vector dimension;

[0133] Correspondingly, the step of generating a cross-modal attention matrix based on aligned data and calculating inter-modal similarity weights includes:

[0134]

[0135] Where αij is the attention between the i-th image feature and the j-th audio feature, sim(·) is the cosine similarity function, τ is the temperature coefficient, and fiimg is the i-th image feature vector.

[0136] In some feasible embodiments, the apparatus further includes:

[0137] The parsing unit is used to respond to receiving the first control operation instruction, parse the first control operation instruction, and at least obtain the location information and control time period information of the operation control;

[0138] The path planning unit is used to acquire the current location information, plan the route based on the current location information and the location information of the operation control, and determine the speed and duration for the individual soldier control system to reach the location of the operation control based on the route.

[0139] The triggering unit is used to respond to receiving a movement command or to trigger the individual soldier management system to start the drive motor and move along the movement route to the position of operation management based on the arrival of the operation deadline.

[0140] According to a third aspect of this application, an electronic device is provided, characterized in that it comprises:

[0141] At least one processor; and

[0142] A memory communicatively connected to the at least one processor; wherein,

[0143] The memory stores instructions that can be executed by the at least one processor, which enables the at least one processor to perform the steps of the edge computing-based proactive management method for high-risk work sites.

[0144] Compared with traditional safety management methods, the technical solution of this application adopts a pre-loaded Linux real-time kernel, which greatly shortens the cold start time. The video analysis of this application adopts a pruned YOLOv5s model, and the gas sensor data is processed by Kalman filtering + LSTM time series prediction. By automatically acquiring video or images and through active management mode, it ensures that workers involved in high-risk operations must perform their work in accordance with the relevant requirements of construction protection measures and risk handling plans. By identifying the biometrics and identities of workers, it can more intuitively assist enterprise safety management personnel in fully grasping the safety status of the work site, forming an effective management method, and urging management and execution personnel at all levels to consciously practice the "three-real" management principle, namely, on-site, on-object, and real-world management, actively go deep into the site, maximize the effectiveness of safety management, strictly ensure the safety of high-risk work sites and their surroundings, improve the effectiveness of safety management, and provide a better working environment for workers. Attached Figure Description

[0145] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0146] Figure 1 This paper shows a schematic diagram of the structure of a proactive control system for high-risk work sites based on edge computing, according to an embodiment of this application.

[0147] Figure 2 This is a flowchart illustrating the proactive management method for high-risk work sites based on edge computing, as described in an embodiment of this application.

[0148] Figure 3 This is a schematic diagram of the structure of the edge computing-based proactive control device for high-risk work sites according to an embodiment of this application;

[0149] Figure 4 This is a schematic diagram of the composition structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0150] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0151] The essence of the technical solutions of the embodiments of this application will be explained in detail below with reference to the accompanying drawings.

[0152] Figure 1 This application illustrates a schematic diagram of the structure of a proactive control system for high-risk work sites based on edge computing, as shown in the embodiment of this application. Figure 1 As shown in the embodiment of this application, the edge computing-based proactive control system for high-risk work sites can be used as a single-soldier control system. It is a portable case with wheels and a built-in modular cabin. Based on this single-soldier control system, proactive control of high-risk work sites can be achieved. This edge computing-based proactive control system for high-risk work sites integrates the following components:

[0153] Human-computer interaction unit: Includes a height-adjustable touchscreen and a voiceprint recognition module, supporting biometric authentication and remote intercom. The height-adjustable display screen uses a worm gear drive to achieve the raising and lowering of the display screen. The worm gear is supported in a sliding groove and moves within the groove, enabling the display screen to move and supporting its raising and lowering functions.

[0154] Computing support unit: CPU / GPU heterogeneous architecture, deploying an edge computing engine to achieve multimodal data processing. As an example, the CPU / GPU heterogeneous architecture uses an Intel i7-1185G7 (4 cores / 8 threads) to handle routine business logic; it utilizes an NVIDIA Jetson Xavier NX (384-core CUDA) dedicated to sensor fusion computing; the two are interconnected via PCIe 3.0 x4 lanes.

[0155] Environmental sensing unit: includes a sphere array, gas sensors, and infrared thermal imagers to collect environmental data.

[0156] Security Management Unit: Includes electronic fence, access control counter and security situation assessment model.

[0157] Communication unit: 5G multi-link aggregation module, supporting edge-cloud collaboration.

[0158] Power management unit: hot-swappable battery pack and multi-source power adapter.

[0159] Mobile inspection unit: includes dynamic monitoring equipment such as smart safety helmets and drones.

[0160] The circuit boards in the communication unit and computing support unit adopt a conformal coating structure; the interfaces use silicone sealing rings and vent valves to protect them. Sensor signal transmission uses a differential RS-485 bus, and the video cable is equipped with a ferrite core to suppress interference in the 100MHz-1GHz frequency band.

[0161] The individual soldier control system of this application embodiment may include a main housing frame, a rapid deployment mechanism, a damping roller assembly, an equipment mounting compartment, a stress dispersion structure, etc. The main housing frame of the mobile load-bearing unit is made of aviation aluminum alloy, and its interior is equipped with reinforcing ribs. The rapid deployment mechanism has a four-bar hydraulic system and an automatic locking device. The hydraulic cylinder thrust can be up to 1200N, and the automatic locking device has an electromagnetic pin and redundant mechanical latches. The equipment mounting compartment has modular slots. The computing power support unit compartment is equipped with heat dissipation fins, the sensor compartment is an EMI shielded compartment, and the battery compartment is an explosion-proof structure. The rapid deployment mechanism adopts a hinge structure, such as a hinge structure made of stainless steel pins. The rollers on the main housing frame are mounted via flange-type mounting plates. The computing power support unit compartment is designed with a sliding rail and guide pin positioning method. A phase change material heat insulation layer is set between the battery compartment and the computing power support unit compartment.

[0162] The security management unit in the application embodiment includes: a multi-band positioning system for electronic fences, used to detect personnel and tools entering and exiting the site in real time, and to alarm for objects entering and exiting non-work areas; a personnel density early warning unit, which detects the number and density of personnel on site based on the fusion results of access control counting and video analysis, and outputs alarm information when the requirements are not met; and an emergency plan database, which associates risk types, generates disposal plans through risk early warning models, and sends them to the human-computer interaction unit and the back-end centralized control center.

[0163] The electronic fence multi-band positioning system employs multi-source positioning fusion, such as RFID + UWB + Bluetooth 5.1 three-band collaboration: RFID: passive tags are deployed on tools / safety helmets; UWB: TDOA positioning algorithm; Bluetooth AoA: positioning via phase difference; the positioning engine: uses a particle filter algorithm to fuse multi-source data and dynamically allocates weights. The personnel density early warning unit uses dual card readers to prevent tailgating (through time difference detection), uploads data based on LoRaWAN, utilizes YOLOv5s + DeepSORT for real-time head detection and tracking, and uses a CSRNet network to generate crowd heat maps.

[0164] The communication unit in this embodiment includes: a channel quality sensing module that dynamically selects 5G / satellite / ad hoc network links; a data hierarchical transmission strategy that prioritizes high-bandwidth channels for critical alarm data; and an end-to-end encryption module that supports Chinese cryptographic algorithms and quantum key distribution.

[0165] The power management unit in this application embodiment includes: a battery health monitoring system for predicting remaining lifespan based on impedance spectrum analysis; a fast battery swapping mechanism for supporting uninterrupted battery module replacement; and an intelligent distribution box for supporting automatic correction of three-phase imbalance.

[0166] Figure 2 This is a flowchart illustrating the proactive management method for high-risk work sites based on edge computing, as described in this application embodiment. Figure 2 As shown in the figure, the proactive management method for high-risk work sites based on edge computing in this application includes:

[0167] Step 201: The received image and video data are convolved using a backbone network with depth-separable convolution and point convolution.

[0168] Specifically, the input tensor X∈R H×W×Cin convolution kernel Output Where k×k is the spatial size of the convolution kernel, C in It is the number of input channels, aligned with the input tensor, C outThe number of output channels determines the depth of the output tensor. H is the height of the input feature map, W is the width of the input feature map, H′ is the height of the output feature map, H′ = H or H′ = H / s, where s is the stride, and W′ is the width of the output feature map, W′ = W or W′ = W / s.

[0169] Based on the image and video data, convolution is performed using the following input convolution calculation formula:

[0170]

[0171] The number of output channels is then adjusted using the following pointwise convolution method based on the depthwise convolution calculation results:

[0172] Enter Y depth ∈R H′×W′×Cin K convolution kernel K point ∈R 1×1×Cin×Cout Output Y(h′,w′,c) out )∈R H ′×W′×Cout :

[0173]

[0174] Y(h′,w′,c) out ) as the result of convolution.

[0175] Step 202: Based on the dynamic pruning of channel importance in the backbone network, channels are sorted and the top n channels are retained. Using Ghost convolution and path aggregation, feature points are generated in the feature pyramid of the backbone network. Through a cross-scale attention mechanism, feature points at different levels are dynamically fused. The size of the input image is adjusted based on the fused feature points. The overlap threshold between the predicted bounding box and the ground truth bounding box of the target detection is adjusted according to the feature density of the target object. Based on the overlap threshold, irrelevant targets in the work site are filtered out, and the image recognition data is output.

[0176] Specifically, Ghost convolution and path aggregation are used to generate feature points in the feature pyramid of the backbone network, and a cross-scale attention mechanism is used to dynamically fuse feature points from different levels, including:

[0177] For the feature map X of the i-th layer of the backbone network i ∈R H×W×C Generate some inherent features Y′=X i *W′,W′∈R k ×k×C×m k is the kernel size, m is the number of output channels, m < <C;

[0178] Perform depthwise convolution on Y′ channel by channel to generate redundant features Y. ij :

[0179] Y ij =Φ j (Y′ i ),j=1,…,v

[0180] v is the number of Ghosts in each group of inherent features, Φ j () represents a linear transformation operation;

[0181] Output Ghost feature map Y i =Concat([Y′,Y 11 ,...,Y mv ])∈R H×W×C′ ;

[0182] Upsample the high-level features and add them element-wise to the low-level features:

[0183] P i =Upsample(P i+1 )+Y i

[0184] Downsampling of low-level features and fusing them with high-level features:

[0185] Q i =Conv 1×1 (Y i )+Downsample(Q i-1 )

[0186] The features obtained from upsampling fusion and downsampling fusion are then bidirectionally aggregated:

[0187] ,in, Indicates channel splicing, F i ∈R H×W×D ;

[0188] Heatmaps are generated from the bidirectional aggregated features using the following convolution:

[0189] H i =σ(Conv 1×1 (F i ), where σ() is the Sigmoid activation function;

[0190] Non-maximum suppression (NMS) and threshold filtering are applied to the heatmap to obtain the coordinates of the feature points.

[0191] Based on the feature density of the target object, the overlap threshold between the predicted bounding box and the ground truth bounding box of the target detection is adjusted. Based on the overlap threshold, irrelevant targets at the work site are filtered out, and image recognition data is output, including:

[0192] The set of predicted bounding boxes output by the object detection model is B = {b1, b2, ..., b}. n}, where each box b i = (x,y,w,h,s), where (x,y) represents the coordinates, and w, h, and s represent the width, height, and confidence level, respectively;

[0193] For each prediction box b i Calculate the density of other boxes within its k-neighborhood as follows:

[0194]

[0195] Where σ is the neighborhood radius hyperparameter;

[0196] Normalize the density of the predicted bounding boxes:

[0197]

[0198] For the normalized prediction box density Set the basic IoU threshold τ base Set a threshold value, and use the following formula to calculate the dynamic threshold τ. i Adjustment:

[0199] α is the adjustment coefficient, which is a set constant;

[0200] Based on prediction box b i The true bounding box G = {g1,...,g m}, dynamic threshold τ i Calculate IoU as follows:

[0201]

[0202] If g j ∈G such that IoU(b i ,g j )≥τ i Then keep b i Otherwise, it is an irrelevant target.

[0203] Step 203: Perform dynamic loudness normalization on the input audio data, perform multi-scale temporal convolution calculation on the normalization result, separate noise components through adversarial training, and perform differential impulse detection on the audio data after noise separation to obtain the feature data of the audio data.

[0204] Loudness difference cancellation is performed on the audio data x(t), and the RMS energy within the sliding window w is calculated:

[0205] Among them, E RMS[n] represents the RMS energy value of the nth frame in the audio data;

[0206] For E RMS [n] performs dynamic gain adjustment as follows:

[0207] , where x norm (t) represents the normalized output audio data, L target The target loudness level is denoted by ∈, which is a small constant to prevent division by zero.

[0208] For x norm (t) Parallel multi-branch convolution captures different temporal features. The output of the k-th branch is as follows:

[0209]

[0210] Among them, L k w is the kernel length of the k-th branch. k (τ) represents the convolution kernel weight of the k-th branch, b k Y(t) is the bias term of the k-th branch, and Y(t) is the output of the multi-scale feature concatenation, Y(t) = [y1(t); y2(t); ...; y K (t)];

[0211] The output Y(t) of the multi-scale feature concatenation is subjected to adversarial training noise separation, as follows:

[0212]

[0213] M(t) is the output noise mask. This is the estimated value of the separated audio.

[0214] Step 204: Feature alignment is performed on the feature data of image recognition data and audio data based on the attention mechanism. A cross-modal attention matrix is ​​generated based on the aligned data, the inter-modal similarity weights are calculated, the fusion weights are calculated based on the similarity weights, a temporal inference chain is constructed, and the risk transmission path is modeled based on the graph neural network to determine the warning level and generate an emergency plan. The aligned data is then sent to the communication unit.

[0215] Among them, feature alignment of image recognition data and audio data is performed based on an attention mechanism, including:

[0216] A two-stream Transformer architecture is used to process image and audio features separately, and a cross-modal attention mechanism is constructed in the feature space as follows:

[0217] Among them, F align The aligned fusion feature matrix, Q = W q Fimg For image feature query, K = W k F audio For audio feature key matrix, V = Wv, F audio The audio eigenvalue matrix, d k The dimension of the key vector;

[0218] This includes generating a cross-modal attention matrix based on aligned data and calculating inter-modal similarity weights, including:

[0219]

[0220] Where, α ij The attention function is defined for the i-th image feature and the j-th audio feature, where sim(·) is the cosine similarity function, τ is the temperature coefficient, and f is the signal. i img Let be the feature vector of the i-th image.

[0221] The video analysis in this application uses a pruned YOLOv5s model. Gas sensor data is processed by Kalman filtering and LSTM time-series prediction. Through automatic video or image acquisition and active management, high-risk operation personnel are required to perform their work in accordance with the relevant requirements of construction protection measures and risk handling plans. By identifying the biometrics and identities of the personnel, the system can more intuitively assist enterprise safety management personnel in fully grasping the safety status of the work site, forming an effective management tool. It urges management and execution personnel at all levels to consciously practice the "three-real" management principle, namely, on-site, on-object, and real-world management, and to proactively go deep into the site, maximizing the effectiveness of safety management, strictly ensuring the safety of the high-risk work site and its surroundings, improving the effectiveness of safety management, and providing a better working environment for the personnel.

[0222] Figure 3 This is a schematic diagram of the structure of the edge computing-based proactive control device for high-risk work sites according to an embodiment of this application. Figure 3 As shown, the edge computing-based proactive control device for high-risk work sites according to this application includes:

[0223] Convolutional unit 30 is used to perform depth-separable convolution and point convolution on the received image and video data using a backbone network.

[0224] Image recognition unit 31 is used to perform channel sorting based on dynamic pruning of channel importance in the backbone network and retain the top n channels. It generates feature points in the feature pyramid of the backbone network using Ghost convolution and path aggregation, and dynamically fuses feature points at different levels through a cross-scale attention mechanism. Based on the fused feature points, it adjusts the size of the input image and adjusts the overlap judgment threshold between the predicted bounding box and the ground box of the target detection according to the feature density of the target object. Based on the overlap judgment threshold, it filters out irrelevant targets in the work site and outputs image recognition data.

[0225] The audio feature extraction unit 32 is used to perform dynamic loudness normalization on the input audio data, perform multi-scale temporal convolution calculation on the normalization result, separate noise components through adversarial training, and perform differential impulse detection on the audio data after noise separation to obtain the feature data of the audio data.

[0226] The contingency plan generation unit 33 is used to perform feature alignment on the feature data of image recognition data and audio data based on the attention mechanism, generate a cross-modal attention matrix based on the aligned data, calculate the inter-modal similarity weight, calculate the fusion weight based on the similarity weight, construct a temporal inference chain, and determine the warning level based on the risk transmission path modeled by the graph neural network, and generate an emergency plan.

[0227] In some feasible embodiments, the convolutional unit 30 is further configured to:

[0228] The input tensor X∈RH×W×Cin, the convolution kernel Kdepth∈Rk×k×Cin×Cout, and the output Ydepth∈RH′×W′×Cout, where k×k is the spatial size of the convolution kernel, Cin is the number of input channels, aligned with the input tensor, Cout is the number of output channels, which determines the depth of the output tensor, H is the height of the input feature map, W is the width of the input feature map, H′ is the height of the output feature map, H′=H or H′=H / s, where s is the stride, and W′ is the width of the output feature map, W′=W or W′=W / s;

[0229] Based on the image and video data, convolution is performed using the following input convolution calculation formula:

[0230]

[0231] The number of output channels is then adjusted using the following pointwise convolution method based on the depthwise convolution calculation results:

[0232] Input Ydepth∈RH′×W′×Cin, convolution kernel Kpoint∈R1×1×Cin×Cout, output Y(h′,w′,c out )∈RH′×W′×Cout:

[0233]

[0234] Y(h′,w′,c) out ) as the result of convolution.

[0235] In some feasible embodiments, the image recognition unit 31 is further configured to:

[0236] For the feature map Xi∈RH×W×C of the i-th layer of the backbone network, the generated partial intrinsic feature Y′=Xi*W′, where W′∈Rk×k×C×mk is the convolution kernel size, m is the number of output channels, and m<<C;

[0237] Perform depthwise convolution on Y′ channel by channel to generate redundant features Y. ij :

[0238] Y ij =Φ j (Y′ i ),j=1,…,v

[0239] v represents the number of Ghosts in each set of inherent features, and Φj() represents the linear transformation operation;

[0240] Output Ghost feature map Yi = Concat([Y′,Y11,...,Ymv])∈RH×W×C′;

[0241] Upsample the high-level features and add them element-wise to the low-level features:

[0242] P i =Upsample(P i+1 )+Y i

[0243] Downsampling of low-level features and fusing them with high-level features:

[0244] Q i =Conv 1×1 (Y i )+Downsample(Q i-1 )

[0245] The features obtained from upsampling fusion and downsampling fusion are then bidirectionally aggregated:

[0246] ,in, Indicates channel splicing, F i ∈RH×W×D;

[0247] Heatmaps are generated from the bidirectional aggregated features using the following convolution:

[0248] Hi =σ(Conv 1×1 (F i ), where σ() is the Sigmoid activation function;

[0249] Non-maximum suppression (NMS) and threshold filtering are applied to the heatmap to obtain the coordinates of the feature points.

[0250] In some feasible embodiments, the image recognition unit 31 is further configured to:

[0251] The set of predicted bounding boxes output by the object detection model is B = {b1, b2, ..., bn}, where each box bi = (x, y, w, h, s), (x, y) represents the coordinates, and w, h, and s represent the width, height, and confidence score, respectively.

[0252] For each predicted box bi, calculate the density of other boxes within its k-neighborhood, as follows:

[0253]

[0254] Where σ is the neighborhood radius hyperparameter;

[0255] Normalize the density of the predicted bounding boxes:

[0256]

[0257] For the normalized prediction box density Set the basic IoU threshold τ base Set a threshold value, and use the following formula to calculate the dynamic threshold τ. i Adjustment:

[0258] α is the adjustment coefficient, which is a set constant;

[0259] Based on the predicted bounding box bi, the ground truth bounding box G={g1,...,gm}, and the dynamic threshold τi, the IoU is calculated as follows:

[0260]

[0261] If gj∈G such that IoU(bi,gj)≥τi, then retain bi; otherwise, it is an irrelevant target.

[0262] In some feasible embodiments, the audio feature extraction unit 32 is further configured to:

[0263] Loudness difference cancellation is performed on the audio data x(t), and the RMS energy within the sliding window w is calculated:

[0264] Among them, E RMS[n] represents the RMS energy value of the nth frame in the audio data;

[0265] For E RMS [n] performs dynamic gain adjustment as follows:

[0266] , where x norm (t) represents the normalized output audio data, Ltarget represents the target loudness level, and ∈ represents the small constant to prevent zeroing;

[0267] For x norm (t) Parallel multi-branch convolution captures different temporal features. The output of the k-th branch is as follows:

[0268]

[0269] Among them, L k w is the kernel length of the k-th branch. k (τ) represents the convolution kernel weight of the k-th branch, b k Y(t) is the bias term for the k-th branch, and Y(t) is the output of the multi-scale feature concatenation, where Y(t) = [y1(t); y2(t); ...; yK(t)].

[0270] The output Y(t) of the multi-scale feature concatenation is subjected to adversarial training noise separation, as follows:

[0271]

[0272] M(t) is the output noise mask. This is the estimated value of the separated audio.

[0273] In some feasible embodiments, the plan generation unit 33 is further configured to:

[0274] A two-stream Transformer architecture is used to process image and audio features separately, and a cross-modal attention mechanism is constructed in the feature space as follows:

[0275] Where Falign is the aligned fusion feature matrix, Q = Wq, Fimg is the image feature query, Wq ∈ Rdk × dimg; K = Wk, Faudio is the audio feature key matrix, Wk ∈ Rdk × daudio; V = Wv, Faudio is the audio feature value matrix, Wv ∈ Rdv × daudio, dk is the key vector dimension;

[0276] Correspondingly, the step of generating a cross-modal attention matrix based on aligned data and calculating inter-modal similarity weights includes:

[0277]

[0278] Where αij is the attention between the i-th image feature and the j-th audio feature, sim(·) is the cosine similarity function, τ is the temperature coefficient, and fiimg is the i-th image feature vector.

[0279] In some feasible embodiments, the apparatus further includes:

[0280] Analysis unit ( Figure 3 (not shown in the image), used to respond to receiving a first control operation instruction, parse the first control operation instruction, and at least obtain the location information and control time period information of the operation control;

[0281] Path planning unit ( Figure 3 (Not shown in the image) is used to obtain current location information, plan a route based on the current location information and the location information of the operation control, and determine the speed and duration for the individual soldier control system to travel to the location of the operation control based on the route.

[0282] Trigger unit ( Figure 3 (Not shown in the image) is used to trigger the individual soldier management system to start the drive motor and move along the movement route to the position of operation control in response to receiving a movement command or based on the arrival of the operation deadline.

[0283] In an exemplary embodiment, the aforementioned units may be implemented by one or more central processing units (CPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components.

[0284] Regarding the apparatus in the above embodiments, the specific manner in which each module and unit performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0285] Figure 4This is a schematic diagram of the composition structure of the electronic device according to an embodiment of this application, such as... Figure 4 As shown, the electronic device 800 supports multi-screen output and may include one or more of the following components: processing component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input / output (I / O) interface 812, sensor component 814, and communication component 816.

[0286] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0287] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of this data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0288] Power supply component 806 provides power to various components of electronic device 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800.

[0289] Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0290] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0291] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0292] Sensor assembly 814 includes one or more sensors for providing state assessments of various aspects of electronic device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of electronic device 800, changes in position of electronic device 800 or a component of electronic device 800, the presence or absence of user contact with electronic device 800, orientation or acceleration / deceleration of electronic device 800, and temperature changes of electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0293] Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 can access wireless networks based on communication standards, such as Wi-Fi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0294] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the steps of the edge computing-based proactive management method for high-risk work sites described in the above embodiments.

[0295] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of an electronic device 800 to complete the steps of the edge computing-based proactive management method for high-risk work sites described in the above embodiments. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0296] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of the invention, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0297] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0298] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not present.

[0299] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0300] The above description is merely an embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A proactive management and control method for high-risk work sites based on edge computing, characterized in that, The method includes: Receive image and video data, and perform convolution on the image and video data using a backbone network with depthwise separable convolution and point convolution. Dynamic pruning based on channel importance in the backbone network sorts channels and retains the top n channels. Using Ghost convolution and path aggregation, feature points are generated in the feature pyramid of the backbone network. Through a cross-scale attention mechanism, feature points at different levels are dynamically fused. The size of the input image is adjusted based on the fused feature points, and the overlap threshold between the predicted bounding box and the ground truth bounding box is adjusted according to the feature density of the target object. Based on the overlap determination threshold, irrelevant targets at the work site are filtered out, and image recognition data is output. The input audio data is subjected to dynamic loudness normalization, the normalization result is subjected to multi-scale temporal convolution calculation, and the noise component is separated through adversarial training. Differential impulse detection is performed on the noise-separated audio data to obtain the feature data of the audio data. Feature alignment is performed on image recognition data and audio data based on an attention mechanism. A cross-modal attention matrix is ​​generated based on the aligned data, inter-modal similarity weights are calculated, fusion weights are calculated based on the similarity weights, a temporal inference chain is constructed, and a risk transmission path is modeled based on a graph neural network to determine the warning level and generate an emergency plan.

2. The method according to claim 1, characterized in that, The received image and video data are convolved using a backbone network with depthwise separable convolution and pointwise convolution methods, including: The input tensor X∈RH×W×Cin, the convolution kernel Kdepth∈Rk×k×Cin×Cout, and the output Ydepth∈RH′×W′×Cout, where k×k is the spatial size of the convolution kernel, Cin is the number of input channels, aligned with the input tensor, Cout is the number of output channels, which determines the depth of the output tensor, H is the height of the input feature map, W is the width of the input feature map, H′ is the height of the output feature map, H′=H or H′=H / s, where s is the stride, and W′ is the width of the output feature map, W′=W or W′=W / s; Based on the image and video data, convolution is performed using the following input convolution calculation formula: The number of output channels is then adjusted using the following pointwise convolution method based on the depthwise convolution calculation results: Input Ydepth∈RH′×W′×Cin, convolution kernel Kpoint∈R1×1×Cin×Cout, output Y(h′,w′,c out )∈RH′×W′×Cout: Y(h′,w′,c) out ) as the result of convolution.

3. The method according to claim 1, characterized in that, The method utilizes Ghost convolution and path aggregation to generate feature points in the feature pyramid of the backbone network, and dynamically fuses feature points from different levels through a cross-scale attention mechanism, including: For the feature map Xi ∈ RH×W×C of the i-th layer of the backbone network, the generated partial intrinsic feature Y′=Xi*W′, where W′ ∈ Rk×k×C×m, k is the convolution kernel size, m is the number of output channels, and m < <C; Perform depthwise convolution on Y′ channel by channel to generate redundant features Y. ij : Y ij *Φ j (Y i ′),j=1,…,v v represents the number of Ghosts in each set of inherent features, and Φj() represents the linear transformation operation; Output Ghost feature map Yi = Concat([Y′,Y11,...,Ymv])∈RH×W×C′; Upsample the high-level features and add them element-wise to the low-level features: P i =Upsample(P i+1 )+Y i Downsampling of low-level features and fusing them with high-level features: Q i =Conv 1×1 (Y i )+Downsample(Q i-1 ) The features obtained from upsampling fusion and downsampling fusion are then bidirectionally aggregated: in, Indicates channel splicing, F i ∈RH×W×D; Heatmaps are generated from the bidirectional aggregated features using the following convolution: H i =σ(Conv 1×1 (F i ), where σ() is the Sigmoid activation function; Non-maximum suppression (NMS) and threshold filtering are applied to the heatmap to obtain the coordinates of the feature points.

4. The method according to claim 1, characterized in that, The process of adjusting the overlap threshold between the predicted bounding box and the ground truth bounding box based on the feature density of the target object, filtering out irrelevant targets at the work site based on the overlap threshold, and outputting image recognition data includes: The set of predicted bounding boxes output by the object detection model is B = {b1, b2, ..., bn}, where each box bi = (x, y, w, h, s), (x, y) represents the coordinates, and w, h, and s represent the width, height, and confidence score, respectively. For each predicted box bi, calculate the density of other boxes within its k-neighborhood, as follows: Where σ is the neighborhood radius hyperparameter; Normalize the density of the predicted bounding boxes: For the normalized prediction box density The basic IoU threshold τ base Set a threshold value, and use the following formula to calculate the dynamic threshold τ. i Adjustment: α is the adjustment coefficient, which is a set constant; Based on the predicted bounding box bi, the ground truth bounding box G={g1,...,gm}, and the dynamic threshold τi, the IoU is calculated as follows: If gj∈G such that IoU(bi,gj)≥τi, then retain bi; otherwise, it is an irrelevant target.

5. The method according to claim 1, characterized in that, The process involves performing dynamic loudness normalization on the input audio data, multi-scale temporal convolution calculation on the normalization result, separating noise components through adversarial training, and performing differential impulse detection on the noise-separated audio data to obtain the feature data of the audio data, including: Loudness difference cancellation is performed on the audio data x(t), and the RMS energy within the sliding window w is calculated: Among them, E RMS [n] represents the RMS energy value of the nth frame in the audio data; For E RMS [n] performs dynamic gain adjustment as follows: , where x norm (t) represents the normalized output audio data, Ltarget represents the target loudness level, and ∈ represents the small constant to prevent zeroing; For x norm (t) Parallel multi-branch convolution captures different temporal features. The output of the k-th branch is as follows: Among them, L k w is the kernel length of the k-th branch. k (τ) represents the convolution kernel weight of the k-th branch, b k Y(t) is the bias term for the k-th branch, and Y(t) is the output of the multi-scale feature concatenation, where Y(t) = [y1(t); y2(t); ...; yK(t)]. The output Y(t) of the multi-scale feature concatenation is subjected to adversarial training noise separation, as follows: M(t) is the output noise mask. This is the estimated value of the separated audio.

6. The method according to claim 1, characterized in that, The feature alignment of image recognition data and audio data based on an attention mechanism includes: A two-stream Transformer architecture is used to process image and audio features separately, and a cross-modal attention mechanism is constructed in the feature space as follows: Where Falign is the aligned fusion feature matrix, Q = Wq, Fimg is the image feature query, Wq ∈ Rdk × dimg; K = Wk, Faudio is the audio feature key matrix, Wk ∈ Rdk × daudio; V = Wv, Faudio is the audio feature value matrix, Wv ∈ Rdv × daudio, dk is the key vector dimension; The process of generating a cross-modal attention matrix based on aligned data and calculating inter-modal similarity weights includes: Where αij is the attention between the i-th image feature and the j-th audio feature, sim(·) is the cosine similarity function, τ is the temperature coefficient, and fiimg is the i-th image feature vector.

7. The method according to claim 1, characterized in that, The method further includes: In response to receiving the first control operation instruction, the first control operation instruction is parsed to obtain at least the location information and control time period information of the operation control; Obtain current location information, plan a route based on the current location information and the location information of the operation control, and determine the speed and duration for the individual soldier control system to reach the location of the operation control based on the route; In response to receiving a movement command or based on the arrival of a work deadline, the individual soldier management system is triggered to start the drive motor and move along the movement route to the work control position.

8. A proactive control device for high-risk work sites based on edge computing, characterized in that, The device includes: The convolutional unit is used to perform depthwise separable convolution and pointwise convolution on the received image and video data using the backbone network. The image recognition unit is used to perform channel sorting based on dynamic pruning of channel importance in the backbone network and retain the top n channels. It generates feature points in the feature pyramid of the backbone network using Ghost convolution and path aggregation, and dynamically fuses feature points at different levels through a cross-scale attention mechanism. The size of the input image is adjusted based on the fused feature points, and the overlap threshold between the predicted bounding box and the ground truth bounding box is adjusted according to the feature density of the target object. Based on the overlap threshold, irrelevant targets in the work site are filtered out, and image recognition data is output. The audio feature extraction unit is used to perform dynamic loudness normalization on the input audio data, perform multi-scale temporal convolution calculation on the normalization result, separate noise components through adversarial training, and perform differential impulse detection on the noise-separated audio data to obtain the feature data of the audio data. The contingency plan generation unit is used to perform feature alignment on the feature data of image recognition data and audio data based on the attention mechanism, generate a cross-modal attention matrix based on the aligned data, calculate the inter-modal similarity weights, calculate the fusion weights based on the similarity weights, construct a temporal inference chain, and determine the warning level based on the risk transmission path modeled by the graph neural network, and generate an emergency plan.

9. The apparatus according to claim 8, characterized in that, The convolutional unit is also used for: The input tensor X∈RH×W×Cin, the convolution kernel Kdepth∈Rk×k×Cin×Cout, and the output Ydepth∈RH′×W′×Cout, where k×k is the spatial size of the convolution kernel, Cin is the number of input channels, aligned with the input tensor, Cout is the number of output channels, which determines the depth of the output tensor, H is the height of the input feature map, W is the width of the input feature map, H′ is the height of the output feature map, H′=H or H′=H / s, where s is the stride, and W′ is the width of the output feature map, W′=W or W′=W / s; Based on the image and video data, convolution is performed using the following input convolution calculation formula: The number of output channels is then adjusted using the following pointwise convolution method based on the depthwise convolution calculation results: Input Ydepth∈RH′×W′×Cin, convolution kernel Kpoint∈R1×1×Cin×Cout, output Y(h′,w′,c out )∈RH′×W′×Cout: Y(h′,w′,c) out ) as the result of convolution.

10. The apparatus according to claim 8, characterized in that, The image recognition unit is also used for: For the feature map Xi∈RH×W×C of the i-th layer of the backbone network, the generated partial intrinsic feature Y′=Xi*W′, where W′∈Rk×k×C×mk is the convolution kernel size, m is the number of output channels, and m<<C; Perform depthwise convolution on Y′ channel by channel to generate redundant features Y. ij : Y ij *Φ j (Y i ′),j=1,…,v v represents the number of Ghosts in each set of inherent features, and Φj() represents the linear transformation operation; Output Ghost feature map Yi = Concat([Y′,Y11,...,Ymv])∈RH×W×C′; Upsample the high-level features and add them element-wise to the low-level features: P i =Upsample(P i+1 )+Y i Downsampling of low-level features and fusing them with high-level features: Q i =Conv 1×1 (Y i )+Downsample(Q i-1 ) The features obtained from upsampling fusion and downsampling fusion are then bidirectionally aggregated: in, Indicates channel splicing, F i ∈RH×W×D; Heatmaps are generated from the bidirectional aggregated features using the following convolution: H i =σ(Conv 1×1 (F i ), where σ() is the Sigmoid activation function; Non-maximum suppression (NMS) and threshold filtering are applied to the heatmap to obtain the coordinates of the feature points.

11. The apparatus according to claim 8, characterized in that, The image recognition unit is also used for: The set of predicted bounding boxes output by the object detection model is B = {b1, b2, ..., bn}, where each box bi = (x, y, w, h, s), (x, y) represents the coordinates, and w, h, and s represent the width, height, and confidence score, respectively. For each predicted box bi, calculate the density of other boxes within its k-neighborhood, as follows: Where σ is the neighborhood radius hyperparameter; Normalize the density of the predicted bounding boxes: For the normalized prediction box density The basic IoU threshold τ base Set a threshold value, and use the following formula to calculate the dynamic threshold τ. i Adjustment: α is the adjustment coefficient, which is a set constant; Based on the predicted bounding box bi, the ground truth bounding box G={g1,...,gm}, and the dynamic threshold τi, the IoU is calculated as follows: If gj∈G such that IoU(bi,gj)≥τi, then retain bi; otherwise, it is an irrelevant target.

12. The apparatus according to claim 8, characterized in that, The audio feature extraction unit is also used for: Loudness difference cancellation is performed on the audio data x(t), and the RMS energy within the sliding window w is calculated: Among them, E RMS [n] represents the RMS energy value of the nth frame in the audio data; For E RMS [n] performs dynamic gain adjustment as follows: , where x norm (t) represents the normalized output audio data, Ltarget represents the target loudness level, and ∈ represents the small constant to prevent zeroing; For x norm (t) Parallel multi-branch convolution captures different temporal features. The output of the k-th branch is as follows: Among them, L k w is the kernel length of the k-th branch. k (τ) represents the convolution kernel weight of the k-th branch, b k Y(t) is the bias term for the k-th branch, and Y(t) is the output of the multi-scale feature concatenation, where Y(t) = [y1(t); y2(t); ...; yK(t)]. The output Y(t) of the multi-scale feature concatenation is subjected to adversarial training noise separation, as follows: M(t) is the output noise mask. This is the estimated value of the separated audio.

13. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the steps of the edge computing-based proactive management method for high-risk work sites as described in any one of claims 1 to 7.

Images