Multi-device collaborative analysis method and system for explosion-proof video algorithm analysis brain-like intelligent boxes
By using a brain-like intelligent box to analyze explosion-proof video in explosion-proof environments, tasks are split and devices are grouped, and task allocation is dynamically adjusted. This solves the problem of uneven resource utilization in collaborative analysis between devices and improves recognition response time and system stability.
Patent Information
- Application Number
- CN202611585397.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-14
- Publication Date
- 2026-11-10
AI Technical Summary
In existing video analysis systems for explosion-proof sites, it is difficult to coordinate the needs for hazard identification at the monitored locations, the computational power consumption of identification tasks, the real-time load of equipment, and the data response capabilities between devices. This leads to problems such as insufficient computing resources on some devices, idle resources on others, and untimely response to hazard identification.
By acquiring the physical location, current computing load, and operational intensity of multiple explosion-proof video algorithms within the explosion-proof site, the brain-like intelligent box performs task splitting and device grouping, retrieves video footage across devices, and dynamically adjusts task allocation relationships based on computing power consumption and response time to achieve collaborative analysis between devices.
It improved the efficiency of computing resources utilization, shortened the response time for hazard identification, improved the timeliness of identification and the stability of system operation, and reduced the phenomenon of uneven equipment load.
Smart Images

Figure CN123071806A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent video analysis and edge computing technology, specifically to a multi-device collaborative analysis method and system for a brain-like intelligent box with explosion-proof video algorithm analysis. Background Technology
[0002] In explosion-proof environments such as petrochemical plants, oil and gas storage and transportation facilities, and dust-related operations, a large number of video surveillance devices are typically installed to continuously monitor storage tanks, pipelines, loading and unloading areas, work passages, and maintenance areas. These devices are combined with video analysis equipment to identify safety hazards such as electrostatic sparks, abnormal discharges, and the wearing status of personal protective equipment. Due to the different physical locations of various monitoring devices, the monitoring targets and the types of hazards that need to be identified also differ.
[0003] Existing video analytics systems typically employ either deploying complete recognition algorithms on each field analysis device or centrally transmitting video feeds from various sources to a central node for unified analysis. The former approach often results in different field devices running algorithms unrelated to their actual monitoring scenarios, leading to excessive computational resource consumption. The latter approach requires continuous transmission of large amounts of video data to the central node, placing a high demand on communication links and central computing resources. As the number of monitoring points increases or the field operation status changes, data transmission pressure and processing load can easily rise. Furthermore, the computational resources required for different hazard identification tasks vary, and the computing power and real-time load of different field analysis devices also differ. The intensity of field operations also dynamically changes with operation time, number of personnel, and equipment operating status; for example, during loading / unloading, maintenance, or shift handover. During peak periods of operation, some areas may simultaneously generate multiple recognition needs, causing a rapid increase in the computing load of the corresponding analysis equipment, while analysis equipment in other areas still has a lot of idle computing power, resulting in an imbalance between busy and idle equipment. When using a fixed method to allocate recognition tasks, it is difficult to adapt to this dynamic change in a timely manner. At the same time, when multiple devices are processing video analysis tasks collaboratively, the response latency caused by the retrieval of video data between devices must also be considered. If task allocation is based solely on the remaining computing power of the devices, ignoring device location, monitoring scenarios, and cross-device data retrieval efficiency, recognition tasks may be assigned to devices that are far from the target video or have slow communication responses, thereby increasing processing latency. On the other hand, configuring tasks solely based on monitoring location may cause some devices to be in a high-load state for a long time, failing to fully utilize the idle computing power of other devices. Summary of the Invention
[0004] The purpose of this invention is to provide a multi-device collaborative analysis method and system for explosion-proof video algorithm analysis using a brain-like intelligent box. This addresses the existing problems in video analysis of explosion-proof sites, such as the need for hazard identification at monitoring locations, the computational power consumption of identification tasks, the real-time load on devices, and the difficulty in coordinating data response capabilities between devices. These problems can easily lead to some devices experiencing resource shortages, others being idle, and untimely responses to hazard identification.
[0005] To achieve the above objectives, the present invention provides the following technical solution: a multi-device collaborative analysis method for explosion-proof video algorithm analysis brain-like intelligent boxes, comprising:
[0006] S1. Obtain the physical locations of at least two explosion-proof video algorithm analysis brain-like intelligent boxes in the explosion-proof site and the first video images collected by the monitoring equipment corresponding to the explosion-proof video algorithm analysis brain-like intelligent boxes, and determine the type of hidden danger corresponding to the first video image based on the physical location;
[0007] S2. The pre-established identification task is split into at least one task branch corresponding to the type of hidden danger, and the basic computing power consumption corresponding to the task branch is obtained.
[0008] S3. If the current computing power load of the explosion-proof video algorithm analysis brain-like intelligent box does not match the work intensity corresponding to the first video screen, it is determined that the explosion-proof video algorithm analysis brain-like intelligent box has a busy-idle misalignment.
[0009] S4. For explosion-proof video algorithm analysis brain-like intelligent boxes with busy / idle misalignment, cluster the physical location and current computing load to obtain at least one device group.
[0010] S5. Select two explosion-proof video algorithm analysis brain-like intelligent boxes from the device group whose current computing power load difference is closest to the basic computing power consumption, and call the same first video screen across the terminal respectively. Use the time difference between the two calls as the response time.
[0011] S6. If the response time is greater than the preset time threshold, the computational logic contained in the task branch is blocked in turn, and the corresponding computational logic is extracted based on the change in basic computing power consumption before and after blocking to obtain the task characteristics.
[0012] S7. Classify and match the task characteristics, basic computing power consumption, and current computing power load of each explosion-proof video algorithm analysis brain-like intelligent box in the device group to determine the target allocation relationship;
[0013] S8. Based on the target allocation relationship, the task branch is distributed to the corresponding explosion-proof video algorithm analysis brain-like intelligent box in the equipment group to obtain the hazard identification results.
[0014] Preferably, step S1 includes acquiring the physical locations of at least two explosion-proof video algorithm analysis brain-like intelligent boxes within the explosion-proof site and the first video images captured by the monitoring equipment corresponding to the explosion-proof video algorithm analysis brain-like intelligent boxes; determining the three-dimensional spatial coordinates of the explosion-proof video algorithm analysis brain-like intelligent boxes based on their physical locations; extracting features from the first video images to obtain multi-dimensional image feature vectors; if the three-dimensional spatial coordinates are within a preset risk area, matching the multi-dimensional image feature vectors with the hazard classification model corresponding to the risk area to obtain the hazard type corresponding to the first video image.
[0015] Preferably, step S2 includes obtaining the feature dimensions corresponding to the type of hidden danger, parsing the pre-established identification task according to the feature dimensions to obtain a task parsing tree; if the computational complexity of the task parsing tree is greater than a preset complexity threshold, then the task parsing tree is split into nodes to obtain task branches; mapping is performed according to the task branches to obtain a resource scheduling sequence; and historical computing power consumption records are read according to the resource scheduling sequence to determine the basic computing power consumption corresponding to the task branch.
[0016] Preferably, step S3 includes obtaining the underlying hardware operation logs of the explosion-proof video algorithm analysis brain-like intelligent box to determine the current computing power load, and obtaining the first video frame captured by the explosion-proof video algorithm analysis brain-like intelligent box to obtain the corresponding workload; searching for the corresponding standard computing power load range in a pre-established computing power matching mapping table according to the workload; determining whether the current computing power load is within the standard computing power load range; if the current computing power load is not within the standard computing power load range, determining that the current computing power load and workload do not match, and determining that the explosion-proof video algorithm analysis brain-like intelligent box has a busy / idle misalignment.
[0017] Preferably, step S4 includes obtaining the physical location and current computing load of the explosion-proof video algorithm analysis brain-like intelligent box; obtaining spatial distribution coordinates and load feature vectors based on the physical location and current computing load; calculating the Euclidean distance matrix based on the spatial distribution coordinates and load feature vectors; processing the Euclidean distance matrix using a hierarchical clustering algorithm to generate a clustering dendrogram; and pruning the clustering dendrogram to obtain at least one device group.
[0018] Preferably, step S5 includes generating a load difference matrix based on the current computing power load value of the brain-like intelligent box analyzed by the explosion-proof video algorithm in the device group; determining the first target intelligent box and the second target intelligent box based on the load difference matrix and the basic computing power consumption; sending a cross-end retrieval command to the first target intelligent box and the second target intelligent box; receiving the first video frame and the first arrival timestamp returned by the first target intelligent box; receiving the second video frame and the second arrival timestamp returned by the second target intelligent box; if the frame feature vectors of the first video frame and the second video frame are consistent, then calculating the time difference between the second arrival timestamp and the first arrival timestamp to obtain the response time.
[0019] Preferably, step S6 includes obtaining the response time of the target service; if the response time is greater than a preset time threshold, then extracting the initial set of computational logic and the initial basic computing power consumption contained in the task branch corresponding to the target service; sequentially performing a masking operation on the initial set of computational logic to obtain a subset of the masked computational logic, and obtaining the basic computing power consumption corresponding to the subset of the masked computational logic; performing a difference operation based on the initial basic computing power consumption and the basic computing power consumption after masking to obtain the change in computing power consumption; extracting the corresponding target computational logic from the initial set of computational logic based on the change in computing power consumption, and performing feature mapping processing on the target computational logic to obtain task features.
[0020] Preferably, step S7 includes acquiring the task characteristics and basic computing power consumption corresponding to the explosion-proof video algorithm, and constructing a computing power feature vector; acquiring the current computing power load of each explosion-proof video algorithm analysis brain-like intelligent box in the device group based on the computing power feature vector, and generating a load state matrix; performing classification matching processing on the load state matrix and computing power feature vector through a support vector machine to obtain a classification matching probability value; if the classification matching probability value is greater than a preset probability threshold, generating an allocation mapping table based on the classification matching probability value, and determining the target allocation relationship based on the allocation mapping table.
[0021] Preferably, step S8 includes generating a routing distribution instruction based on the target allocation relationship, the routing distribution instruction being used to distribute the task branch to the corresponding explosion-proof video algorithm analysis brain-like intelligent box within the device group; performing frame extraction processing on the explosion-proof video data of the brain-like intelligent box to obtain a video frame sequence; extracting a hazard feature map based on the video frame sequence; and if the response value of the hazard feature map is greater than a preset threshold, parsing the hazard feature map to obtain the hazard identification result output by the explosion-proof video algorithm analysis brain-like intelligent box.
[0022] A multi-device collaborative analysis system for an explosion-proof video algorithm analysis brain-inspired intelligent box, used to implement the steps of the multi-device collaborative analysis method for the explosion-proof video algorithm analysis brain-inspired intelligent box, the system comprising:
[0023] The hazard type determination module acquires the physical locations of at least two explosion-proof video algorithm analysis brain-like intelligent boxes within the explosion-proof site and the first video images collected by the monitoring equipment corresponding to the explosion-proof video algorithm analysis brain-like intelligent boxes, and determines the hazard type corresponding to the first video image based on the physical location;
[0024] The task splitting module splits the pre-established identification task according to the type of hazard, obtains at least one task branch corresponding to the type of hazard, and obtains the basic computing power consumption corresponding to the task branch.
[0025] The busy / idle misalignment judgment module obtains the current computing power load of the explosion-proof video algorithm analysis brain-like intelligent box and the work intensity corresponding to the first video frame. If the current computing power load and work intensity do not match, it is determined that there is a busy / idle misalignment in the explosion-proof video algorithm analysis brain-like intelligent box.
[0026] The device grouping module, for the explosion-proof video algorithm analysis brain-like intelligent box with busy / idle time mismatch, uses hierarchical clustering algorithm to cluster and divide the physical location and current computing power load to obtain at least one device group;
[0027] The response time determination module selects two explosion-proof video algorithm analysis brain-like intelligent boxes from the device group whose current computing power load difference is closest to the basic computing power consumption, and retrieves the same first video frame across the device, and uses the time difference between the two retrievals as the response time.
[0028] If the response time of the task feature extraction module is greater than the preset time threshold, the computational logic contained in the task branch will be blocked in turn. The corresponding computational logic will be extracted based on the change in basic computing power consumption before and after blocking to obtain the task features.
[0029] The allocation relationship determination module uses support vector machines to classify and match task characteristics, basic computing power consumption, and the current computing power load of each explosion-proof video algorithm analysis brain-like intelligent box in the device group to determine the target allocation relationship.
[0030] The collaborative analysis module distributes task branches to the corresponding explosion-proof video algorithm analysis brain-like intelligent boxes within the equipment group according to the target allocation relationship, and obtains the hazard identification results output by the explosion-proof video algorithm analysis brain-like intelligent boxes.
[0031] As can be seen from the above technical solution, the present invention has the following beneficial effects:
[0032] This invention relates to a multi-device collaborative analysis method and system for explosion-proof video algorithm analysis brain-like intelligent boxes. By acquiring the physical location, current computing load, and operational intensity of multiple explosion-proof video algorithm analysis brain-like intelligent boxes within an explosion-proof environment, the system determines the corresponding hazard type based on the physical location and breaks down the identification task into task branches matching the hazard type. This allows explosion-proof video algorithm analysis brain-like intelligent boxes in different areas to perform corresponding identification tasks for the actual monitoring scenario, reducing the consumption of computing resources unrelated to the monitoring content. Simultaneously, by determining whether the current computing load matches the operational intensity, the system can identify the busy / idle mismatch status between different devices. Combining physical location and current computing load, devices with busy / idle mismatches can be grouped, which is beneficial for optimizing device computing resources while considering the correlation of the monitoring area. Coordinated utilization; furthermore, by cross-terminal retrieval of the same first video frame to obtain the response time between devices, and when the response time exceeds a preset time threshold, task features are extracted based on the impact of different calculation logics in the task branch on the basic computing power consumption. Then, the target allocation relationship is determined by combining the task features, the basic computing power consumption, and the current computing power load of each explosion-proof video algorithm analysis brain-like intelligent box in the device group. This allows the task branch to be matched with the actual processing capacity of the corresponding device, thereby reducing the problems of excessive load on some devices, idle computing power on some devices, and low efficiency of cross-device calls caused by fixed task deployment. This improves the utilization efficiency of computing power resources and the rationality of task allocation in the multi-device video analysis process in explosion-proof sites, and helps to shorten the response time of hazard identification, improve the timeliness of hazard identification, and enhance the stability of system operation. Attached Figure Description
[0033] Figure 1 This is a flowchart of the overall method of the present invention;
[0034] Figure 2 This is a line graph comparing the average computing power utilization rate during different monitoring periods in the experiment of this invention;
[0035] Figure 3 This is a scatter plot showing the computing power load distribution of each device in the experiment of this invention;
[0036] Figure 4 This is a line graph showing the changes in the number of busy / idle devices during different monitoring periods in the experiment of this invention;
[0037] Figure 5 This is a bar chart comparing the average identification response time for each type of hazard in the experiment of this invention. Detailed Implementation
[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0039] Example 1:
[0040] like Figure 1 As shown, this invention provides a technical solution: a multi-device collaborative analysis method for explosion-proof video algorithm analysis brain-like intelligent boxes, comprising:
[0041] S1. Obtain the physical locations of at least two explosion-proof video algorithm analysis brain-like intelligent boxes in the explosion-proof site and the first video images collected by the monitoring equipment corresponding to the explosion-proof video algorithm analysis brain-like intelligent boxes, and determine the type of hidden danger corresponding to the first video image based on the physical location;
[0042] S2. Based on the type of hidden danger, the pre-established identification task is split to obtain at least one task branch corresponding to the type of hidden danger, and the basic computing power consumption corresponding to the task branch is obtained.
[0043] S3. Obtain the current computing power load of the explosion-proof video algorithm analysis brain-like intelligent box and the work intensity corresponding to the first video frame. If the current computing power load and work intensity do not match, it is determined that the explosion-proof video algorithm analysis brain-like intelligent box has a busy-idle misalignment.
[0044] S4. For explosion-proof video algorithm analysis brain-like intelligent boxes with busy / idle misalignment, hierarchical clustering algorithm is used to cluster and divide the physical location and current computing power load to obtain at least one device group.
[0045] S5. Select two explosion-proof video algorithm analysis brain-like intelligent boxes with the closest difference in current computing power load from the device group, and call up the same first video screen across the device respectively. Use the time difference between the two calls as the response time.
[0046] S6. If the response time is greater than the preset time threshold, the computational logic contained in the task branch is blocked in turn, and the corresponding computational logic is extracted based on the change in basic computing power consumption before and after blocking to obtain the task characteristics.
[0047] S7. Using support vector machines, classify and match the task characteristics, basic computing power consumption, and current computing power load of each explosion-proof video algorithm analysis brain-like intelligent box in the device group to determine the target allocation relationship.
[0048] S8. Based on the target allocation relationship, the task branch is sent to the corresponding explosion-proof video algorithm analysis brain-like intelligent box in the equipment group to obtain the hidden danger identification results output by the explosion-proof video algorithm analysis brain-like intelligent box.
[0049] In this embodiment, each explosion-proof video algorithm analysis brain-like intelligent box in the explosion-proof site is first registered as a device. Each explosion-proof video algorithm analysis brain-like intelligent box is configured with a unique device identifier, installation location, and corresponding monitoring device identifier. The installation location is represented by at least one of the following: pre-established location coordinates, area number, production unit number, or monitoring point number.
[0050] Simultaneously, a correspondence table between physical locations and hazard types is pre-established, with each physical location corresponding to at least one hazard type, and this correspondence is stored in the device management module. When the monitoring device acquires the first video frame, the physical location of the brain-like intelligent box is analyzed according to the explosion-proof video algorithm corresponding to the monitoring device to query the correspondence table, thereby determining the hazard type corresponding to the first video frame. When two or more hazard types are pre-configured for the same physical location, all hazard types registered in the correspondence table are taken as the hazard types to be identified for the first video frame, and they are respectively entered into the subsequent identification task processing flow.
[0051] In addition, a database of identification tasks is pre-established, with different types of hazards being associated with corresponding identification tasks.
[0052] Specifically, each identification task consists of several computational logics with a defined execution order or data dependencies. Based on the hazard type corresponding to the current first video frame, the corresponding identification task is called from the identification task library, and the identification task is split according to the data input and output relationships between the computational logics. A group of computational logics with continuous data dependencies and capable of being executed as independent scheduling units constitutes a task branch. When one hazard type corresponds to one task branch, the task branch is directly used for subsequent computing power analysis. When one hazard type corresponds to two or more task branches, computing power analysis and equipment matching are performed on each task branch separately.
[0053] Furthermore, the basic computing power is pre-calibrated for each task branch.
[0054] When calibrating the basic computing power, the corresponding task branch is executed at least once using predefined video resolution, video frame rate, video duration, and device operating conditions. The actual resource usage of the processor, graphics processor, and neural network processing unit is recorded during the execution. For an explosion-proof video algorithm analysis brain-inspired intelligent box with one computing unit, the resource usage result of that computing unit is directly used as the basis for computing power consumption. For an explosion-proof video algorithm analysis brain-inspired intelligent box with two or more computing units, the resource usage results of each computing unit are merged according to a pre-set fixed weight to obtain the basic computing power consumption corresponding to the task branch. The basic computing power consumption of each task branch is determined according to the same resource statistics rules, so that different task branches can be compared according to a unified standard.
[0055] Furthermore, during the operation of the explosion-proof video algorithm analysis brain-like intelligent box, the current computing resource occupancy status is collected according to a preset sampling period.
[0056] For an explosion-proof video algorithm analysis brain-like intelligent box that simultaneously includes a processor, a graphics processor, and a neural network processing unit, the resource occupancy value of each computing unit in the current sampling period is obtained, the resource occupancy value is converted to a unified numerical range, and then a weighted sum is performed according to a fixed weight pre-written by the system. The result is used as the current computing power load of the explosion-proof video algorithm analysis brain-like intelligent box. The fixed weight remains unchanged in the same analysis period, so that the current computing power load obtained in adjacent sampling periods has the same computing benchmark.
[0057] In addition, within the same sampling period for acquiring the current computing load, the workload of the first video frame is calculated.
[0058] First, the number of valid targets identified by the target detection algorithm in the first video frame is counted. Second, the number of targets whose positional changes reach a preset displacement threshold is counted. Third, the number of targets that meet the preset hazard triggering conditions is counted. Fixed baseline count values are set for the above three types of target quantities, and the actual count is divided by the corresponding baseline count value to convert the three statistical results into a unified numerical range. When the actual count reaches or exceeds the corresponding baseline count value, the normalized result is treated as the maximum value. Subsequently, the three normalized results are multiplied by preset fixed weights, and the three calculation results are added together to obtain the work intensity corresponding to the first video frame. The sum of the three weights used to calculate the work intensity remains a fixed total value and remains unchanged within the same analysis period.
[0059] In this embodiment, the correspondence between the workload and the baseline computing power requirement is established in advance through equipment calibration.
[0060] During equipment calibration, the same recognition task is run under different operating intensities. The actual computing power load generated when the explosion-proof video algorithm analysis brain-like intelligent box stably completes the corresponding recognition task is recorded, and the operating intensities and corresponding computing power loads are written into the calibration table. During actual operation, the corresponding baseline computing power requirement is obtained from the calibration table based on the operating intensities of the current first video frame. If the current operating intensities are between two adjacent calibration values, the corresponding baseline computing power requirement is determined according to the linear ratio between the two adjacent calibration values, thereby avoiding the use of manual judgment to determine the baseline computing power requirement.
[0061] Furthermore, the current computing load of the explosion-proof video algorithm analysis brain-like intelligent box is compared with the baseline computing demand corresponding to the current first video frame, and the absolute difference between the two is calculated; a busy / idle misalignment judgment threshold is preset.
[0062] When the absolute difference is greater than the busy / idle misalignment judgment threshold, it is determined that the explosion-proof video algorithm analysis brain-like intelligent box has a busy / idle misalignment; when the absolute difference is less than or equal to the busy / idle misalignment judgment threshold, it is determined that the explosion-proof video algorithm analysis brain-like intelligent box does not have a busy / idle misalignment; the busy / idle misalignment judgment threshold is written into the system before the start of the current analysis cycle and is not modified during the current analysis cycle.
[0063] Furthermore, all explosion-proof video algorithm analysis brain-like intelligent boxes that were determined to have busy / idle misalignment were used as hierarchical clustering objects.
[0064] First, the physical distance between any two explosion-proof video algorithm-analyzed brain-like intelligent boxes is calculated. When the physical location is represented by spatial coordinates, the actual spatial distance is calculated based on the positional difference between the two devices in each coordinate axis direction. When the physical location is represented by a region number, the physical distance between the two devices is obtained based on the pre-established region adjacency relationship and the distance table between regions. Then, the physical distance is divided by the system's preset baseline distance to convert the physical distance to a uniform numerical range. When the converted physical distance exceeds the maximum value of this numerical range, it is processed according to the maximum value.
[0065] In addition, the difference in current computing power load between any two explosion-proof video algorithm analysis brain-like intelligent boxes is calculated simultaneously. Specifically, the current computing power load of the two devices is subtracted and the absolute value is taken.
[0066] Subsequently, the normalized physical distance is multiplied by the preset location weight, the current computing power load difference is multiplied by the preset load weight, and the two results are added together to obtain the clustering distance between the two explosion-proof video algorithm analysis brain-like intelligent boxes; the location weight and load weight are determined before the system runs and remain unchanged during the current clustering process.
[0067] Hierarchical clustering employs a step-by-step merging approach.
[0068] Specifically, in the initial state, each explosion-proof video algorithm analysis brain-like intelligent box with busy / idle misalignment is treated as an independent device group. The system sequentially calculates the average clustering distance between any two device groups. The average clustering distance between one device group and another device group is obtained by calculating the clustering distance of all cross-group device combinations within the two device groups and then taking the arithmetic mean of all clustering distances. In each round of clustering, the two device groups with the smallest average clustering distance are selected for merging. After one merging is completed, the average clustering distance between the newly formed device group and the remaining device groups is recalculated, and the device group merging is performed again. When the minimum average clustering distance between all current device groups is greater than the preset clustering cutoff threshold, clustering is stopped, and the device groups formed at the time of stopping are used as the device groups for subsequent task allocation.
[0069] In addition, for any device group, read the current computing load of all explosion-proof video algorithm analysis brain-like intelligent boxes in the group, and calculate the difference in current computing load for any two explosion-proof video algorithm analysis brain-like intelligent boxes in the device group.
[0070] For the current task branch to be processed, the current computing power load difference of each device combination is compared with the basic computing power consumption of the task branch. Specifically, the absolute difference between the current computing power load difference and the basic computing power consumption is calculated. After traversing all two device combinations in the device group, the device combination with the smallest absolute difference value is selected as the cross-end call test device. If two or more device combinations produce the same minimum absolute difference value, one of the device combinations is determined according to the pre-defined sorting rules of the device identifier, so that each selection yields a unique result.
[0071] Furthermore, the two selected explosion-proof video algorithm analysis brain-like smart boxes respectively retrieved the same first video frame across different terminals.
[0072] To ensure complete consistency of the data retrieved by the two devices, the system assigns a unique screen identifier to each first video frame. This screen identifier is generated by the monitoring device identifier, the frame acquisition time, and the video frame sequence number. Both explosion-proof video algorithm analysis brain-computer interfaces initiate video data retrieval requests based on the same screen identifier. The system records the completion time of the first explosion-proof video algorithm analysis brain-computer interface in retrieving the first video frame and the completion time of the second explosion-proof video algorithm analysis brain-computer interface in retrieving the same first video frame. The two completion times are then subtracted, and the absolute value is taken as the response time.
[0073] In this embodiment, a time threshold is preset, and the calculated response time is compared with the time threshold.
[0074] When the response time is less than or equal to the time threshold, the computational logic masking process is not executed, and subsequent task matching continues according to the basic computing power consumption of the current task branch; when the response time is greater than the time threshold, the item-by-item masking process of the computational logic inside the task branch is initiated to further extract task features that can reflect the internal computing power composition of the task.
[0075] For task branches that require execution of computational logic masking, first assign a fixed sequence number to each computational logic according to the actual execution order of the computational logic within the task branch.
[0076] Before the blocking begins, a complete task branch is run once with all computing logic enabled, and the basic computing power consumption corresponding to the complete task branch is recorded. Then, only the first computing logic is blocked, keeping the first input video frame, the execution device, and the remaining computing logic unchanged. The task branch is run again, and the computing power consumption after blocking is recorded. The basic computing power consumption of the complete task branch is subtracted from the computing power consumption after blocking, and the difference is taken as the change in computing power consumption corresponding to that computing logic.
[0077] After completing the masking test of a computational logic, restore the normal execution state of that computational logic, and then perform the same independent masking test on the next sequence of computational logic.
[0078] Specifically, each shielding test shuts down only one computing logic, not two computing logics at the same time, thus ensuring that each change in computing power consumption corresponds to a specific computing logic. After completing the shielding test of all computing logics within the task branch in the above manner, the logic identifier, execution sequence number, and corresponding change in computing power consumption of each computing logic are combined according to the execution order to form the task characteristics of that task branch.
[0079] Support vector machines are trained before actual task matching.
[0080] Specifically, the training samples consist of task characteristics recorded during historical operation, basic computing power consumption of task branches, current computing power load of the explosion-proof video algorithm analysis brain-like intelligent box, and actual task execution results. For a given training sample, if the corresponding task branch can complete the prescribed hazard identification process after being sent to the corresponding explosion-proof video algorithm analysis brain-like intelligent box, and the actual response time is less than or equal to a preset time threshold, the training sample is marked as a matching sample. If the task execution result does not meet the above conditions, the training sample is marked as a non-matching sample. The support vector machine establishes the classification boundary between the two types of samples based on the matching and non-matching samples.
[0081] In the process of online task matching, for the task branch to be assigned, the task characteristics, basic computing power consumption, and current computing power load of a certain explosion-proof video algorithm analysis brain-like intelligent box in the device group are combined into a set of classification input data and input into the trained support vector machine. The support vector machine outputs the classification result between the task branch and the current explosion-proof video algorithm analysis brain-like intelligent box according to the pre-formed classification boundary.
[0082] Perform the above classification process once on each of the explosion-proof video algorithm analysis brain-like intelligent boxes in the device group to obtain the matching results between the current task branch and each device in the device group.
[0083] When only one explosion-proof video algorithm analysis brain-like intelligent box is classified as a matching device in the device group, that device is directly identified as the target device of the current task branch.
[0084] When there are two or more explosion-proof video algorithm analysis brain-like intelligent boxes in the device group that are classified as matching devices, the classification distance value output by the support vector machine for each matching device is read, and the device with the largest classification distance value is selected as the target device; the classification distance value is used to represent the numerical distance between the current input data and the classification boundary of the support vector machine; when the classification distance values corresponding to two or more devices are the same, the device with the device identifier at the beginning is selected according to the fixed order of device identifier from smallest to largest, thereby obtaining a unique target device, and establishing the target allocation relationship between the current task branch and the target device.
[0085] When a single identification task contains two or more task branches, they are first arranged in order of their basic computing power consumption from the largest to the smallest, and the target devices for each task branch are determined in the order of arrangement. When the basic computing power consumption values of two task branches are the same, the processing order is determined according to the fixed numbering order of the task branches. After the device allocation for each task branch is completed, the basic computing power consumption of that task branch is added to the current computing power load of the target device to form the updated current computing power load.
[0086] When subsequent task branches perform support vector machine classification and matching, they all use the updated current computing load as input data, so that subsequent task allocation takes into account the impact of already allocated tasks on the device's computing power status.
[0087] Furthermore, based on the final target allocation relationship, each task branch is distributed to the corresponding explosion-proof video algorithm analysis brain-like intelligent box within the equipment group; upon receiving the task branch, the explosion-proof video algorithm analysis brain-like intelligent box processes the corresponding first video frame according to the pre-determined calculation logic execution order in the task branch and generates a hazard identification result; each hazard identification result is written with a unique screen identifier, task branch identifier, and hazard type identifier for the corresponding first video frame.
[0088] When the same first video frame corresponds to two or more task branches, the hidden danger identification results output by the brain-like intelligent box of each explosion-proof video algorithm analysis are associated according to the unique frame identifier. Then, the identification task corresponding to each identification result is determined according to the task branch identifier, and the results are classified according to the hidden danger type identifier, so as to obtain the hidden danger identification result corresponding to the first video frame.
[0089] Example 2:
[0090] S1 includes acquiring the physical locations of at least two explosion-proof video algorithm analysis brain-like intelligent boxes within the explosion-proof site and the first video images captured by the monitoring equipment corresponding to the explosion-proof video algorithm analysis brain-like intelligent boxes; determining the three-dimensional spatial coordinates of the explosion-proof video algorithm analysis brain-like intelligent boxes based on their physical locations; extracting features from the first video images to obtain multi-dimensional image feature vectors; if the three-dimensional spatial coordinates are within a preset risk area, matching the multi-dimensional image feature vectors with the hazard classification model corresponding to the risk area to obtain the hazard type corresponding to the first video image.
[0091] In this embodiment, after the equipment deployment is completed in the explosion-proof site, the fixed building reference point in the site is used as the origin of the spatial coordinate system, and the mutually perpendicular length direction, width direction and height direction are used as three coordinate directions respectively, and a three-dimensional spatial coordinate system is established with meters as the unified distance unit.
[0092] After each explosion-proof video algorithm analysis brain-like intelligent box is installed, the distances of the installation reference point of the intelligent box relative to the origin of the spatial coordinates are measured in the length, width, and height directions. The distance values in the three directions are used as the three-dimensional spatial coordinates of the explosion-proof video algorithm analysis brain-like intelligent box. A fixed correspondence is established between the three-dimensional spatial coordinates, the device identifier of the explosion-proof video algorithm analysis brain-like intelligent box, and the identifier of the monitoring device connected to it. When the device installation position is adjusted, the original three-dimensional spatial coordinates are remeasured and overwritten. When the position is not adjusted, the registered three-dimensional spatial coordinates are directly read.
[0093] The monitoring equipment continuously collects video data from the explosion-proof site according to the preset video acquisition frame rate.
[0094] After receiving video data from the corresponding monitoring device, the explosion-proof video algorithm analysis brain-like intelligent box extracts the current video frame to be analyzed as the first video frame according to the video frame sequence number. At the same time, it writes the monitoring device identifier, the frame acquisition time, and the video frame sequence number into the data record of the first video frame. The monitoring device identifier is used to determine the source of the first video frame, and the acquisition time and video frame sequence number are used to distinguish video frames generated by the same monitoring device at different times. This data record is then used in the process of cross-terminal retrieval of the same first video frame as described in Embodiment 1, thereby ensuring that the processing objects before and after are consistent.
[0095] In addition, before the first video frame enters the feature extraction process, the frame size is uniformly adjusted to 640 pixels by 640 pixels, and the three color channels of red, green and blue are retained. For each color channel, the pixel value is divided by 255 to convert the pixel value to a fixed range of 0 to 1; the frame after size adjustment and pixel normalization is used as the input of the feature extraction network.
[0096] In this embodiment, the feature extraction network is configured with four levels of convolutional feature extraction structures.
[0097] Specifically, the first stage uses 32 convolutional kernels with a size of 3 pixels by 3 pixels and a stride of 2. The convolutional results are then subjected to batch normalization and linear rectified activation processing. The second stage uses 64 convolutional kernels with a size of 3 pixels by 3 pixels and a stride of 2. Batch normalization and linear rectified activation processing are performed in the same order as the first stage. The third stage uses 128 convolutional kernels with a size of 3 pixels by 3 pixels and a stride of 2. The fourth stage uses 256 convolutional kernels with a size of 3 pixels by 3 pixels and a stride of 2. The convolutional results of the third and fourth stages are also subjected to batch normalization and linear rectified activation processing. The feature maps output from the previous stage are directly used as the input to the next stage, so that edge changes, local textures, target contours, and spatial relationships between targets in the image are gradually converted into deep image features.
[0098] After the fourth level of convolution is completed, the arithmetic mean of the values at all positions within each feature channel is calculated.
[0099] Specifically, the values corresponding to all positions within the same feature channel are summed and then divided by the total number of positions contained in that feature channel, resulting in an average feature value for each feature channel. Since the fourth level has 256 feature channels, a multi-dimensional image feature vector containing 256 feature values is finally formed according to the channel order. The first video frame uses the same image processing parameters and the same network parameters for feature extraction, ensuring that the feature dimension and feature arrangement order output during the training phase are consistent with those during the actual operation phase.
[0100] In addition, the feature extraction network was trained before formal deployment.
[0101] Specifically, the training samples consist of video footage collected by monitoring equipment in explosion-proof locations, with each training frame labeled with its actual hazard type. All training frames are divided into training data and verification data in an 8:2 ratio, and frames belonging to the same continuous time period of the original video are only included in one data set to prevent adjacent video frames from being included in both training data and verification data at the same time.
[0102] During training, a global averaging layer and a temporary classification layer are connected sequentially after the fourth-level convolutional structure. The number of output nodes of the temporary classification layer is consistent with the total number of all hazard types registered in the training data.
[0103] Specifically, 32 training images are input in each batch. The training images are then subjected to size adjustment, pixel normalization, 4-level convolutional feature extraction, and global averaging. The resulting 256 feature values are then input into a temporary classification layer. The temporary classification layer outputs one value for each type of hazard. Each output value is then subjected to exponential operation. Each exponential operation result is divided by the sum of all exponential operation results to obtain the normalized classification result corresponding to each type of hazard.
[0104] The training loss is determined using cross-entropy loss.
[0105] Specifically, the normalized classification result corresponding to the actual hazard type in the training footage is read, and the natural logarithm and its negative are taken as the loss value for that training footage. The loss values of 32 footage within the same training batch are added together and divided by 32 to obtain the training loss for the current batch. The adaptive moment estimation method is used to update the convolutional kernel parameters and the temporary classification layer parameters, with an initial learning rate set to 0.001. After processing all training data in each round, the validation loss is calculated on the validation data. The maximum number of training rounds is set to 100 rounds. Training is terminated when the validation loss obtained in 10 consecutive rounds is not lower than the previously obtained minimum validation loss, and the network parameters are restored to the round corresponding to the minimum validation loss. After training, the temporary classification layer is removed, and the network structure and parameters from the input of the first video footage to the output of the 256-dimensional feature vector are retained as the feature extraction network for the actual operation stage.
[0106] Furthermore, the risk areas of explosion-proof sites must be registered before the equipment is put into operation.
[0107] Each risk area records the start and end coordinates in the length direction, the start and end coordinates in the width direction, and the start and end coordinates in the height direction. The closed three-dimensional space is defined by six boundary coordinates, and a unique risk area identifier is assigned to each risk area. The boundary coordinates of the risk area use the same three-dimensional spatial coordinate system as the aforementioned explosion-proof video algorithm analysis brain-like intelligent box, so that the device position and the risk area can be directly compared in coordinates.
[0108] In addition, when determining whether the explosion-proof video algorithm analyzes whether the brain-like intelligent box is in a certain risk area, it compares its coordinates in three directions in turn.
[0109] If the device's coordinates in the length direction simultaneously satisfy the following conditions: not less than the starting coordinate and not greater than the ending coordinate in the length direction of the risk area; not less than the starting coordinate and not greater than the ending coordinate in the width direction; and not less than the starting coordinate and not greater than the ending coordinate in the width direction, and not greater than the starting coordinate and not greater than the ending coordinate in the height direction, then the explosion-proof video algorithm analysis brain-like intelligent box is determined to be within the risk area. If the device's coordinates are equal to the boundary coordinates of any risk area, it is treated as being within the risk area.
[0110] A corresponding hazard classification model is established for each risk area.
[0111] Specifically, the hazard classification model consists of an input layer, one hidden layer, and an output layer connected sequentially. The input layer has 256 input nodes, each corresponding to one of the 256 feature values in the multi-dimensional image feature vector. The hidden layer has 128 nodes, with each of the 256 nodes in the input layer fully connected to the 128 nodes in the hidden layer. The hidden layer uses linear rectified activation. The number of nodes in the output layer is the same as the number of hazard types pre-registered for the risk area, and each of the 128 nodes in the hidden layer fully connected to all output nodes. Thus, the hazard classification model for a risk area only processes the first video image corresponding to the explosion-proof video algorithm analysis brain-like intelligent box entering the risk area, without adding hazard categories from other risk areas to the current model's output range.
[0112] The training data for the hazard classification model was obtained from labeled video footage of the corresponding risk areas.
[0113] First, the training images are input into the feature extraction network that has been trained and has fixed parameters. Each training image is converted into a 256-dimensional image feature vector. Then, this feature vector is combined with the manually confirmed hazard type label to form a classification training sample. The classification training data and classification verification data are divided into classification training data in an 8:2 ratio, with 32 classification training samples input in each batch. After receiving 256 feature values, the hazard classification model completes the feature combination through 128 hidden nodes, and then the output layer generates a classification output value that is consistent with the number of hazard types in the risk area.
[0114] Furthermore, exponential operations are performed on the output values of each category in the output layer, and then each exponential operation result is divided by the sum of all exponential operation results, so that the sum of the classification results corresponding to each hidden danger type is equal to 1.
[0115] During training, the classification results corresponding to the actual hazard types are read. The natural logarithm of the classification result is taken and its negative value is used as the loss for a single sample. The batch loss is obtained by summing the losses of all samples in the same batch and dividing by 32. The adaptive moment estimation method is used to adjust the connection parameters from the input layer to the hidden layer and from the hidden layer to the output layer. The initial learning rate is set to 0.001 and the maximum number of training rounds is set to 100 rounds. After each round of training, the validation loss of the classification validation data is calculated. Training ends when the validation loss of 10 consecutive rounds is not lower than the previous minimum validation loss, and the model parameters of the round corresponding to the minimum validation loss are retained. After training, the hazard classification model parameters, risk area identifiers, and hazard type identifiers corresponding to the output nodes of the model are stored together.
[0116] In actual operation, when the three-dimensional spatial coordinates of the explosion-proof video algorithm analysis brain-like intelligent box meet the three sets of boundary conditions of a certain risk area, the hazard classification model bound to the risk area is read according to the risk area identifier, and the 256-dimensional image feature vector obtained by the feature extraction network of the current first video image is sequentially input into the 256 input nodes of the hazard classification model; after the hazard classification model completes the hidden layer feature combination and output layer classification calculation, it obtains the classification results corresponding to each hazard type in the risk area.
[0117] Furthermore, all classification results are compared numerically, and the hazard type corresponding to the classification result with the largest numerical value is taken as the hazard type of the first video image.
[0118] When two or more hazard types have the same maximum classification result, they are compared in ascending order according to the hazard type numbers pre-registered in the system. The hazard type with the smallest number is determined as the output result to ensure that a unique hazard type is generated under the same input conditions.
[0119] Therefore, the physical location of the explosion-proof video algorithm analysis brain-like intelligent box is first converted into three-dimensional spatial coordinates. The three-dimensional spatial coordinates are used to determine the risk area to which the current device belongs. The risk area further determines the hazard classification model that should be called. At the same time, the first video image is converted into a 256-dimensional image feature vector through a fixed-structure feature extraction network. This feature vector serves as the direct input of the hazard classification model. The hazard classification model outputs a classification result that corresponds one-to-one with the registered hazard category of the current risk area. Finally, the hazard type of the first video image is determined by the classification result.
[0120] Example 3:
[0121] S2 includes obtaining the feature dimensions corresponding to the hazard type, parsing the pre-established identification task based on the feature dimensions to obtain a task parsing tree; if the computational complexity of the task parsing tree is greater than a preset complexity threshold, then the task parsing tree is split into nodes to obtain task branches; mapping is performed based on the task branches to obtain a resource scheduling sequence; historical computing power consumption records are read based on the resource scheduling sequence to determine the basic computing power consumption corresponding to the task branch.
[0122] In this embodiment, after determining the type of hazard corresponding to the first video frame, a pre-established feature configuration table is read according to the type of hazard.
[0123] The feature configuration table is generated simultaneously with the identification task version, which records the hazard type identifier, the feature items involved in the identification, the arrangement position of each feature item in the task input data, and the number of feature items. The feature items are derived from the screen information actually called by the corresponding identification task, including target contour data, texture data, position data, motion data, and spatial relationship data between targets. After reading all the feature items corresponding to the current hazard type, the number of feature items actually involved in the identification calculation is used as the feature dimension corresponding to the hazard type. After the same identification task version is put into operation, the correspondence between hazard type and feature dimension remains unchanged.
[0124] Furthermore, the pre-established identification tasks are saved as a set of nodes according to the actual execution process.
[0125] Specifically, each computational process with independent input and output corresponds to one node. Each node records its node number, computational logic type, input data structure, output data structure, input feature dimension, and output feature dimension. When the output of a previous node is called by a subsequent node, a data dependency relationship from the previous node to the subsequent node is recorded between the two nodes. When parsing and identifying the task, starting from the starting node that receives the feature dimension data, the next node with a data dependency relationship with the current node is read sequentially, and the input data structure registered by the next node is checked to see if it is consistent with the output data structure of the current node. After verification, the reading continues until the end node that generates the hazard identification result is read. When one node outputs data to two or more nodes simultaneously, corresponding child node relationships are established for each node. When the outputs of two or more nodes are used as inputs for subsequent nodes, the subsequent node is recorded as a merging node. The task parsing tree is formed by each node and its data dependencies.
[0126] Furthermore, after the task parse tree is formed, the computational complexity of each computation node is statistically analyzed; the number of multiplications and additions for each convolution computation node is statistically analyzed according to the actual convolution process.
[0127] Specifically, the kernel length is first multiplied by the kernel width, and then multiplied by the number of input channels. The resulting value is the number of multiplications required to generate one output feature value. The number of additions required to generate one output feature value is one less than the number of multiplications. Subsequently, the number of multiplications and additions required to generate one output feature value are multiplied by the total number of positions in the output feature map, and then multiplied by the number of output channels to obtain the total number of multiplications and total number of additions for that convolution node.
[0128] The number of computations for a fully connected computing node is determined by the number of input features and the number of output nodes.
[0129] Specifically, the total number of multiplications is obtained by multiplying the number of input features by the number of output nodes; each output node accumulates all products, and the number of additions is the number of input features minus 1. This number of additions is then multiplied by the number of output nodes to obtain the total number of additions; the number of calculations for pooling nodes is determined by multiplying the number of comparisons or additions actually performed in each pooling window by the number of output positions; the number of activation operations for activation nodes is determined by the number of feature values actually processed; the number of normalization nodes is counted item by item based on the number of additions, multiplications, square roots, and divisions actually performed; and the total number of operations obtained for each node is summed to obtain the computational complexity of that node.
[0130] Furthermore, all nodes are read along the task parse tree from the starting node to the ending node, and the computational complexity of each node is added up item by item. The sum of the number of calculations is used as the computational complexity of the task parse tree. When there are parallel paths in the task parse tree, the computational complexity of each node in the parallel path is included, and the number of calculations of any node is not deleted due to parallel execution.
[0131] The complexity threshold was determined through benchmark testing when deploying the brain-like intelligent box for recognition tasks in the explosion-proof video algorithm analysis.
[0132] In this embodiment, the scheduling time slice for a single task branch is set to 50 milliseconds. A baseline computing task is established using the same video resolution, video frame rate, and input data format as the basic computing power calibration in Embodiment 1. The computational load of the baseline computing task is increased step by step at fixed computation intervals. Each level of baseline computing task is executed 10 times consecutively. The first two times are used as the initialization process of the computing unit and are not included in the time statistics. The actual execution time of the last 8 times is added together and divided by 8 to obtain the average execution time of the baseline computing task at that level.
[0133] Furthermore, benchmark tests were conducted in ascending order of computational complexity until the average execution time of a certain level of benchmark computation task exceeded 50 milliseconds.
[0134] The number of calculations corresponding to the last benchmark calculation task with an average execution time of less than or equal to 50 milliseconds is determined as the complexity threshold. When there are explosion-proof video algorithm analysis brain-like intelligent boxes with different hardware specifications in the device group, benchmark tests are completed according to the above method, and the minimum value among the complexity thresholds obtained by each device is taken as the complexity threshold used for the current recognition task, so as to ensure that the task branch formed according to the threshold can be executed on each explosion-proof video algorithm analysis brain-like intelligent box in the device group.
[0135] The computational complexity of the task parse tree is compared with the aforementioned complexity threshold.
[0136] When the computational complexity of the task parse tree is no greater than the complexity threshold, the recognition task corresponding to the complete task parse tree is determined as one task branch; when the computational complexity of the task parse tree is greater than the complexity threshold, the nodes are split according to the data dependency relationship between the nodes.
[0137] When performing node segmentation, starting from the initial node of the task parse tree, the computational complexity of each node is accumulated sequentially according to the actual execution order. Before adding the current node to be processed, the accumulated computational complexity is added to the computational complexity of the current node to be processed. If the sum is not greater than the complexity threshold, the current node is merged into the forming task branch, and the next node is read. If the sum is greater than the complexity threshold, the consecutive nodes accumulated before the current node are determined as one task branch, and the current node is used as the starting node of the next task branch to restart the complexity accumulation.
[0138] When the task parse tree has a branching relationship, the branching nodes are used as common input nodes, and the complexity accumulation is performed separately for each branch path. Nodes belonging to different parallel paths are not interleaved into the same continuous task branch.
[0139] When parallel paths reconnect at the merging node, the merging node is assigned to a subsequent task branch only after all upstream task branches on which the merging node depends have generated output data. If the computational complexity of a node exceeds the complexity threshold, the node is assigned to a separate task branch without further decomposing the already determined computational logic within the node. After all nodes are divided, each task branch saves its node number, node execution order, input data structure, output data structure, and data dependencies with the preceding and following task branches.
[0140] After obtaining the task branch, resource mapping is performed according to the node type in the task branch; the explosion-proof video algorithm analysis brain-like intelligent box establishes a fixed mapping table between operators and computing resources during the software deployment phase.
[0141] Specifically, convolution, matrix multiplication, and neural network inference nodes first read the operator support list of the neural network processing unit. If the operator corresponding to the current node is included in the support list, the node is mapped to the neural network processing unit. If it is not included in the support list, the graphics processor operator support list is read. If the graphics processor supports the operator, the node is mapped to the graphics processor. If neither the neural network processing unit nor the graphics processor supports the operator, the node is mapped to the processor. Video decoding, data format conversion, task control, and recognition result encapsulation nodes are directly mapped to the processor.
[0142] Furthermore, the resource mapping results are read sequentially according to the execution order of the nodes in the task branch.
[0143] Specifically, when consecutive nodes are mapped to the same type of computing resource, the consecutive nodes are recorded as the same resource execution segment; when the computing resource corresponding to the current node is different from that of the previous node, the previous resource execution segment ends and a new resource execution segment is established; the starting node, ending node, computing resource type and execution order corresponding to each resource execution segment are recorded in sequence, and arranged according to the actual execution order to form a resource scheduling sequence; when adjacent resource execution segments use different computing resources, the data transfer relationship between the two resource execution segments is recorded at the same time, so that historical computing power records can be retrieved in the order of computing resource usage consistent with the current task branch.
[0144] In addition, the explosion-proof video algorithm analysis brain-like intelligent box generates a corresponding historical computing power consumption record for each task branch execution completed.
[0145] Historical computing power consumption records include task branch number, resource scheduling sequence, video input resolution, number of frames processed, task execution time, average usage of each computing resource, and execution completion status. The average usage of computing resources is obtained according to a fixed sampling period during the actual execution of the task branch. All resource usage sampling values obtained during the execution period are added together and then divided by the number of samplings.
[0146] When determining the basic computing power consumption of the current task branch, historical computing power consumption records are first retrieved according to the current resource scheduling sequence, and only records whose node range, resource type, and resource execution order are consistent with the current task branch are retained; then records whose video resolution, video frame count, and basic computing power calibration conditions in Example 1 are consistent and whose execution completion status is normal are retained; historical records of missing node execution, interrupted task execution, or changed resource execution order are not included in the determination of basic computing power consumption.
[0147] In addition, each historical computing power consumption record that meets the conditions is processed according to the computing power statistics method determined in Implementation Example 1.
[0148] Specifically, when the device contains only one type of computing unit, the average resource usage of that computing unit is used as the computing power consumption value of that record; when the device contains two or more types of computing units, including processors, graphics processors, and neural network processing units, the average resource usage of each computing unit is converted to a uniform range of 0 to 1, and then multiplied by the resource weight determined in the device configuration file. Finally, the results are summed to obtain the uniform computing power consumption value corresponding to that historical record; the sum of the resource weights is 1, and it is consistent with the resource weights used in Example 1 when determining the current computing power load.
[0149] When one valid historical record is retrieved, the unified computing power consumption value of that record is directly determined as the basic computing power consumption of the task branch; when two or more valid historical records are retrieved, the 20 valid records whose time sequence is closest to the current moment are read; when there are fewer than 20 valid records, all valid records are read; the unified computing power consumption values read are added together and then divided by the actual number of records read, and the resulting arithmetic mean is used as the basic computing power consumption of the task branch.
[0150] In addition, if there are no historical computing power consumption records that meet the conditions, perform one basic computing power calibration according to the resource scheduling sequence of the current task branch.
[0151] The calibration uses the same video resolution, video frame rate and video duration as in Example 1. The average occupancy of each computing resource during task execution is recorded, and a unified computing power consumption value is calculated according to the aforementioned resource weights. This unified computing power consumption value is used as the basic computing power consumption of the current task branch and is also written into the historical computing power consumption record.
[0152] Example 4:
[0153] S3 includes obtaining the underlying hardware operation logs of the explosion-proof video algorithm analysis brain-like intelligent box to determine the current computing power load, and obtaining the first video frame captured by the explosion-proof video algorithm analysis brain-like intelligent box to obtain the corresponding task intensity; searching for the corresponding standard computing power load range in a pre-established computing power matching mapping table according to the task intensity; determining whether the current computing power load is within the standard computing power load range; if the current computing power load is not within the standard computing power load range, determining that the current computing power load and task intensity are mismatched, and determining that the explosion-proof video algorithm analysis brain-like intelligent box has a busy-idle misalignment.
[0154] In this embodiment, after the explosion-proof video algorithm analysis brain-like intelligent box runs, the underlying hardware monitoring process reads the running logs of the processor, graphics processor, and neural network processing unit according to a 1-second sampling period. Each sampling obtains the occupancy rate of each computing unit in the current 1-second period and converts the occupancy rate into a value between 0 and 1. The occupancy rate is recorded as 0 when the computing unit does not perform a computing task and as 1 when the upper limit of the effective computing capacity of the computing unit obtained by the device calibration is reached.
[0155] The computing power weights corresponding to the processor, graphics processor, and neural network processing unit are calibrated through a benchmark task before the equipment is put into use.
[0156] Specifically, the three computing units are each independently executed to perform a benchmark matrix calculation task with the same input size, and the number of effective operations completed by each computing unit within 10 seconds is recorded. The number of effective operations completed by the three computing units is added together, and then the number of effective operations completed by each computing unit is divided by the total number of operations mentioned above. The result is used as the computing power weight of the corresponding computing unit. When the device hardware configuration does not change, the computing power weight remains unchanged.
[0157] Specifically, when determining the current computing load, the processor's occupancy rate at the same sampling moment is multiplied by the processor's computing power weight, the graphics processor's occupancy rate is multiplied by the graphics processor's computing power weight, and the neural network processing unit's occupancy rate is multiplied by the neural network processing unit's computing power weight. The three results are then added together to obtain the computing load at that sampling moment. After reading five consecutive sampling periods, the five computing load values are added together and divided by 5. The result is taken as the current computing load.
[0158] The current computing load obtained thus adopts a numerical range of 0 to 1, which is consistent with the statistical caliber of the basic computing power consumption in Example 1 and Example 3.
[0159] Within the same 5-second time period of obtaining the current computing power load, read the first video frame collected by the corresponding monitoring device, and determine the work intensity based on the video content within that time period; the work intensity is determined by three data items: the number of effective targets, the number of targets that have undergone effective displacement, and the number of targets that have entered the hazard identification process.
[0160] The number of effective targets is obtained from the target detection results.
[0161] Specifically, the target detection model outputs recognition results and recognition confidence scores for targets in the first video frame. When the recognition confidence score reaches a predetermined detection threshold and the same target exists consecutively in two frames of the first video frame, the target is counted as a valid target. The detection threshold is determined using validation data before the target detection model is deployed. It starts from 0.50 and increases by 0.05 each time. The number of false detections and false negatives under each candidate threshold are counted. The candidate threshold corresponding to the minimum sum of the number of false detections and false negatives is determined as the detection threshold and is saved along with the target detection model.
[0162] The number of targets that have undergone effective displacement is determined based on the change in the center position of the target in adjacent first video frames.
[0163] Specifically, after the monitoring equipment is installed, 200 frames of video are continuously captured under the condition that there are no moving targets in the monitoring area. Fixed background feature points are extracted, and the pixel displacement distance of each background feature point between two adjacent frames is recorded. The maximum pixel displacement distance that occurs during the 200-frame detection process is determined as the static jitter threshold. In actual processing, if the center position distance of the same target in two adjacent frames is greater than the static jitter threshold, the target is counted in the number of targets with effective displacement; if the center position distance is less than or equal to the static jitter threshold, it is not counted.
[0164] The target number of hazard identification processes is determined based on the hazard types obtained in Example 2.
[0165] Specifically, for each valid target within a 5-second time period, the corresponding hazard type identification task is performed. When a target first meets the input conditions of the hazard type identification task, the target is counted once. If the same target continuously meets the same input conditions in consecutive video frames, it is not counted repeatedly until the target leaves the first video frame or the corresponding target tracking marker becomes invalid.
[0166] In this embodiment, the three benchmark values used for calculating the workload intensity are determined before the monitored area is put into operation.
[0167] First, determine the effective target baseline value based on the effective monitoring range of the monitoring equipment and the number of work objects that can exist simultaneously on site; the effective displacement target baseline value is the same as the effective target baseline value; the hazard identification target baseline value is determined based on the maximum number of targets that can enter the hazard identification process simultaneously in the area within one 5-second cycle; the above baseline values are written into the area configuration file of the corresponding monitoring equipment.
[0168] In addition, when calculating the work intensity, the first intensity value is obtained by dividing the number of effective targets within a 5-second time period by the effective target baseline value; the second intensity value is obtained by dividing the number of targets that have undergone effective displacement by the effective displacement target baseline value; and the third intensity value is obtained by dividing the number of targets that have entered the hazard identification process by the hazard identification target baseline value. If any calculation result is greater than 1, it is treated as 1.
[0169] In this embodiment, the first intensity value is multiplied by 0.3, the second intensity value is multiplied by 0.3, the third intensity value is multiplied by 0.4, and the three calculation results are added together. The resulting value is used as the operation intensity corresponding to the first video frame. The sum of the above three weights is 1, and remains fixed after being put into operation in the same monitoring area.
[0170] Therefore, the workload intensity is limited to between 0 and 1, and can be matched with the current computing power load using a unified numerical range.
[0171] The computing power matching mapping table is calibrated before the explosion-proof video algorithm analysis brain-like intelligent box officially executes the recognition task.
[0172] Specifically, the workload intensity between 0 and 1 is divided into 10 consecutive intervals at fixed intervals of 0.1, and actual operation videos are prepared as calibration videos for each workload intensity interval. The calibration videos use the same resolution, frame rate, hazard identification algorithm and task branch execution method as the actual operation.
[0173] In addition, for each work intensity range, the explosion-proof video algorithm analysis brain-like intelligent box continuously executes 30 calibration tasks, with each calibration taking 5 seconds. The first 5 calibrations are used to complete the algorithm model loading, cache establishment, and stable operation of the computing unit, and do not participate in the determination of the standard computing power load range. For the last 25 calibrations, a current computing power load value is calculated according to the aforementioned underlying hardware log processing method.
[0174] Furthermore, the last 25 current computing power load values are arranged in ascending order, and the minimum value is taken as the lower limit of the standard computing power load corresponding to the job intensity range, and the maximum value is taken as the upper limit of the standard computing power load. The job intensity range, the lower limit of the standard computing power load, and the upper limit of the standard computing power load are written into the same mapping record. After the calibration of 10 job intensity ranges is completed in sequence, a pre-established computing power matching mapping table is formed.
[0175] When the processor, graphics processor, or neural network processing unit of the explosion-proof video algorithm analysis brain-like intelligent box is replaced, or the version of the hidden danger identification model described in Embodiment 2 is changed, or the task branch structure and resource scheduling sequence described in Embodiment 3 are changed, the original computing power matching mapping table is no longer used, and a new computing power matching mapping table is re-established according to the above 30 calibration processes; thereby ensuring that the standard computing power load range in the mapping table corresponds to the current device hardware, identification algorithm, and task execution method.
[0176] In actual operation, the corresponding task intensity range is first determined based on the task intensity obtained in the current 5-second time period, and then the standard computing power load lower limit and standard computing power load upper limit corresponding to the range are read from the computing power matching mapping table.
[0177] When the current computing load is greater than or equal to the lower limit of the standard computing load and less than or equal to the upper limit of the standard computing load, it is determined that the current computing load is within the standard computing load range and the current computing load matches the workload.
[0178] When the current computing load is less than the lower limit of the standard computing load, it is determined that the current computing load has not reached the standard computing load range corresponding to the current workload. When the current computing load is greater than the upper limit of the standard computing load, it is determined that the current computing load exceeds the standard computing load range corresponding to the current workload. In any of the above situations, it is determined that the current computing load is not matched with the workload, and the corresponding explosion-proof video algorithm analysis brain-like intelligent box is marked as having a busy-idle mismatch.
[0179] Example 5:
[0180] S4 includes obtaining the physical location and current computing load of the explosion-proof video algorithm analysis brain-like intelligent box; obtaining spatial distribution coordinates and load feature vectors based on the physical location and current computing load; calculating the Euclidean distance matrix based on the spatial distribution coordinates and load feature vectors; processing the Euclidean distance matrix using a hierarchical clustering algorithm to generate a clustering dendrogram; and pruning the clustering dendrogram to obtain at least one device group.
[0181] In this embodiment, after determining the busy / idle misalignment, the physical location and current computing load of each explosion-proof video algorithm analysis brain-like intelligent box that is determined to have a busy / idle misalignment are read, and the data is used as input for hierarchical clustering processing.
[0182] The physical location uses the three-dimensional spatial coordinates determined in Example 2, and the current computing load uses the load value in the range of 0 to 1 calculated based on the underlying hardware operation log in Example 4, so as to ensure that the device location and load status used in this step are consistent with the aforementioned steps.
[0183] Furthermore, the three-dimensional spatial coordinates are normalized before being used for distance calculation.
[0184] Specifically, when establishing a three-dimensional spatial coordinate system in an explosion-proof location, the effective dimensions in the length, width, and height directions are pre-recorded. For any explosion-proof video algorithm analysis brain-like intelligent box, its length direction coordinates are divided by the effective length direction dimension of the explosion-proof location to obtain the normalized length direction coordinates; the width direction coordinates are divided by the effective width direction dimension to obtain the normalized width direction coordinates; and the height direction coordinates are divided by the effective height direction dimension to obtain the normalized height direction coordinates. All three normalized coordinates are within the range of 0 to 1 and are arranged in a fixed order of length, width, and height to form the spatial distribution coordinates of the device.
[0185] The current computing load has been converted to the range of 0 to 1 according to Example 4, so there is no need to perform a secondary scaling transformation. The current computing load is directly used as the load feature vector of the device.
[0186] Therefore, each explosion-proof video algorithm analysis brain-like intelligent box forms 3 spatial coordinate components and 1 computing load component. The 4 components use the same numerical range to enter the distance calculation. After adopting this processing method, the difference in the actual installation position between devices and the difference in current computing resource occupation jointly participate in the determination of the degree of device association, and will not directly change the calculation result due to the difference in the units between metric coordinates and normalized load.
[0187] When establishing the Euclidean distance matrix, any two explosion-proof video algorithm analysis brain-like intelligent boxes are selected sequentially according to the device number. For the two currently selected devices, the normalized coordinates of the length direction of the two devices are subtracted and the difference is squared. Then, the same processing is performed on the normalized coordinates of the width direction, the normalized coordinates of the height direction, and the current computing load. The four squared results are added together, and the square root operation is performed on the sum to obtain the Euclidean distance between the two devices.
[0188] For example, when two explosion-proof video algorithm analysis brain-like intelligent boxes are installed in close proximity and have similar current computing power loads, the differences between the four corresponding components are all close to zero. After squaring, summing, and square root processing, the Euclidean distance obtained decreases accordingly. When the differences in equipment installation locations or current computing power loads increase, the squared differences of the corresponding components increase, and the resulting Euclidean distance increases synchronously. Therefore, this Euclidean distance directly characterizes the comprehensive differences between the two devices in the explosion-proof environment in terms of both spatial distribution and current computing power status.
[0189] Furthermore, following the above calculation process, all combinations of two devices are traversed, and a matrix is established according to the device number.
[0190] Specifically, the matrix rows and columns correspond to the brain-like intelligent boxes for the explosion-proof video algorithm analysis participating in clustering. The distance between the same device and itself is written as 0, and the matrix positions corresponding to any two different devices are written with the Euclidean distance calculated above. Since the Euclidean distance between two devices does not change with the calculation direction, the corresponding positions with the main diagonal of the matrix as the boundary are filled with the same value, and finally the Euclidean distance matrix is formed.
[0191] This embodiment uses an agglomerative hierarchical clustering algorithm to process the Euclidean distance matrix.
[0192] The clustering hierarchy of the algorithm is generated entirely from the Euclidean distance matrix input and the determined clustering connection rules; at the start of clustering, each of the explosion-proof video algorithm analysis brain-like intelligent boxes is used as a leaf node and an independent clustering unit.
[0193] In the first-level clustering process, the distances between all different devices in the Euclidean distance matrix are read, the distance with the smallest value is selected, and the two device nodes corresponding to this distance are connected to the same parent cluster node to form a cluster unit containing two devices. When there are more than two identical minimum distances, the device numbers in the corresponding device combination are compared, and the device combination with the smaller minimum device number value is selected first. If they are still the same, the other device number in the device combination is compared, and the merging objects are determined according to the order of the device numbers from smallest to largest, so that the same input data corresponds to a unique clustering order.
[0194] Furthermore, after forming new cluster units, the average connectivity method is used to determine the distance between the new cluster unit and the other cluster units.
[0195] In the specific processing, each device in the first cluster unit is read, and the recorded Euclidean distance between the device and each device in the second cluster unit is calculated. The Euclidean distances corresponding to all cross-cluster device combinations are added together and then divided by the actual number of cross-cluster device combinations. The resulting arithmetic mean is used as the new cluster distance between the two cluster units.
[0196] Then, among all the current cluster units, find the two cluster units with the smallest cluster distance and establish a new parent cluster node; after each cluster merging, the distance between the new cluster unit and other cluster units is re-determined according to the same average connectivity calculation method; the above process continues until all explosion-proof video algorithm analysis brain-like intelligent boxes are connected to the same highest-level cluster node.
[0197] The clustering dendrogram is generated according to the above merging relationships.
[0198] Specifically, the bottom-level leaf nodes correspond to the brain-like intelligent boxes that participate in the clustering of explosion-proof video algorithm analysis; each intermediate node corresponds to one cluster merging; the lower-level nodes connected by the intermediate nodes correspond to the two cluster units being merged this time; the clustering distance used in each merging is simultaneously written to the intermediate node as the merging distance of the node in the tree diagram; by reading each intermediate node sequentially from the bottom-level device node upwards, the complete hierarchical relationship of devices merging from independent states to the same cluster unit can be obtained.
[0199] After the clustering dendrogram is formed, the pruning threshold is determined based on the merge distance obtained from this clustering.
[0200] Specifically, when the number of devices participating in clustering is no less than 3, all the merge distances formed during the clustering process are arranged in ascending order of value. Starting from the second merge distance, the current merge distance is subtracted from the previous merge distance to obtain the distance increment between two adjacent merge levels.
[0201] Furthermore, all distance increments are compared, and the distance increment with the largest value is selected. Then, the first and second merged distances that form the largest distance increment are read, the two merged distances are added together and divided by 2, and the resulting value is determined as the pruning threshold of the current clustering dendrogram. This pruning threshold is directly derived from the clustering results formed by the spatial location of the device and the current computing load, and there is no need to input an empirical threshold separately.
[0202] In addition, when two or more identical maximum distance increments occur, the pruning threshold is determined by selecting the first maximum distance increment that occurs in the order of merging. When all adjacent merged distances are the same, the identical merged distance is directly determined as the pruning threshold. When the number of devices participating in clustering is 2, adjacent merged distance calculation is not performed, and the two devices are directly kept in the same device group to meet the processing requirement of selecting two devices from the device group to perform cross-end retrieval in the subsequent implementation of Example 1.
[0203] During pruning, each merge node is read downwards from the highest-level node in the clustering tree diagram, and the merge distance recorded by that node is compared with the pruning threshold. If the merge distance is greater than the pruning threshold, the two lower-level clustering branches connected to that merge node are disconnected. If the merge distance is less than or equal to the pruning threshold, the original connection of that node is retained. After all merged nodes have completed the above processing, the device nodes that are still connected through the tree structure form a device group, and different disconnected branches form different device groups.
[0204] When a branch containing only one explosion-proof video algorithm analysis brain-like intelligent box appears after pruning, the average distance between this single device and the other device groups is further calculated.
[0205] Specifically, the Euclidean distance between the single device and each device in the target device group is read, and the sum of all Euclidean distances is divided by the number of devices in the target device group to obtain the average distance between the single device and the target device group. After completing the calculation between the single device and all other device groups in sequence, the single device is merged into the device group with the smallest average distance value. If there are the same minimum average distance, the minimum value of the device number in the corresponding device group is compared, and the single device is merged into the device group with the smaller minimum device number value.
[0206] Example 6:
[0207] S5 includes generating a load difference matrix based on the current computing power load value of the brain-like intelligent box according to the explosion-proof video algorithm analysis in the device group; determining the first target intelligent box and the second target intelligent box according to the load difference matrix and the basic computing power consumption; sending a cross-end retrieval command to the first target intelligent box and the second target intelligent box; receiving the first video frame and the first arrival timestamp returned by the first target intelligent box; receiving the second video frame and the second arrival timestamp returned by the second target intelligent box; if the frame feature vectors of the first video frame and the second video frame are consistent, calculating the time difference between the second arrival timestamp and the first arrival timestamp to obtain the response time.
[0208] In this embodiment, the device group obtained in Embodiment 5 is read, and the current computing load of each explosion-proof video algorithm analysis brain-like intelligent box in the device group is obtained according to Embodiment 4. The explosion-proof video algorithm analysis brain-like intelligent boxes are arranged in ascending order of device identifier, and this arrangement order is used as the row order and column order of the load difference matrix, so that each row and column intersection position in the matrix corresponds to two determined explosion-proof video algorithm analysis brain-like intelligent boxes.
[0209] When generating the load difference matrix, the current computing load of any two different explosion-proof video algorithm analysis brain-like intelligent boxes is read sequentially. The current computing load of one box is subtracted from the current computing load of the other box. The absolute value of the difference is taken and written into the matrix position corresponding to the two devices. The row and column intersection positions corresponding to the same device are written as 0. The absolute difference values obtained by calculating any two devices in reverse order are the same. Therefore, the corresponding values on both sides of the main diagonal in the load difference matrix are consistent. After completing the calculation of all two-device combinations in the device group, the load difference matrix corresponding to the current device group is formed.
[0210] In addition, when determining the first target smart box and the second target smart box, the basic computing power consumption of the current task branch obtained in Example 3 is read, and all load differences formed by different devices in the load difference matrix are traversed; for each load difference, the basic computing power consumption of the current task branch is subtracted from the load difference, and the absolute value of the calculation result is taken as the matching difference corresponding to the device combination; after all device combinations are calculated, the device combination with the smallest matching difference value is selected.
[0211] If two or more device combinations have the same minimum matching difference, the device identifier with the smallest value in each device combination is compared first, and the device combination with the smallest device identifier value is selected. If there are still the same results, the other device identifier in the device combination is compared, and the unique device combination is determined according to the order of the device identifier values from smallest to largest.
[0212] In the determined combination of devices, the device with the smaller current computing power load value is designated as the first target smart box, and the device with the larger current computing power load value is designated as the second target smart box; when the current computing power load of two devices is the same, the device with the smaller device identifier value is designated as the first target smart box.
[0213] Furthermore, after the first target smart box and the second target smart box are determined, the task scheduling terminal generates a cross-end retrieval task and sends cross-end retrieval instructions with the same content to the two target smart boxes respectively; the cross-end retrieval instructions are written with task identifier, monitoring device identifier, video acquisition time and video frame sequence number.
[0214] Among them, the monitoring equipment identifier, video acquisition time, and video frame sequence number are consistent with the video data record formed when processing the first video screen in Example 2. The three data items together define the specific video frame that needs to be retrieved this time; the task identifier is used to classify the returned data of the two target smart boxes into the same cross-end retrieval process.
[0215] Furthermore, after receiving the cross-end retrieval command, the first target smart box first locates the corresponding video cache according to the monitoring device identifier, then searches for the corresponding video segment according to the video capture time, and reads the specified video frame according to the video frame sequence number. After reading, the video frame is used as the first video screen, and encapsulated together with the task identifier, monitoring device identifier, video capture time, and video frame sequence number before being returned to the task scheduling terminal. The second target smart box performs the same positioning and reading process and returns the second video screen. Both target smart boxes use the same set of video positioning data as the retrieval condition, limiting the first video screen and the second video screen to point to the same original video frame from the data source.
[0216] In addition, when the task scheduler receives the first video frame returned by the first target smart box in its entirety, it immediately reads the current time of the local monotonic clock and records it as the first arrival timestamp in milliseconds; when it receives the second video frame returned by the second target smart box in its entirety, it records the second arrival timestamp using the same monotonic clock; both timestamps are generated by the same task scheduler and the same clock source, avoiding the local clock deviation between different explosion-proof video algorithm analysis brain-like smart boxes from being included in subsequent time difference calculations.
[0217] Before calculating the response time, a frame consistency check is performed on the first video frame and the second video frame.
[0218] First, compare the monitoring device identifier, video capture time, and video frame sequence number carried in the two returned data. If all three items are the same, continue to compare the frame feature vectors. If any one item is inconsistent, terminate the current response time calculation and resend the cross-end retrieval command for the corresponding video frame.
[0219] When extracting frame feature vectors, the first and second video frames are adjusted to 32 pixels by 32 pixels respectively, and each pixel is converted to a grayscale value. The grayscale conversion is performed with a fixed coefficient: the red channel pixel value is multiplied by 0.299, the green channel pixel value is multiplied by 0.587, and the blue channel pixel value is multiplied by 0.114. The three results are then added together, and the nearest integer is taken to form the grayscale value corresponding to the pixel.
[0220] The 32-pixel by 32-pixel grayscale image is then evenly divided into 64 image blocks, each containing 4 pixels by 4 pixels. For each image block, the grayscale values of 16 pixels are added together, then divided by 16, and the nearest integer is taken to obtain the block feature value of the image block. The 64 block feature values are arranged in a fixed order from left to right and from top to bottom to form a 64-dimensional frame feature vector of the corresponding video frame.
[0221] When comparing the frame feature vectors of the first video frame and the second video frame, 64 block feature values are compared item by item in the same dimension order.
[0222] Specifically, when all 64 corresponding block feature values are the same, the frame feature vectors of the first video frame and the second video frame are determined to be consistent; when any one corresponding block feature value is different, the frame feature vectors are determined to be inconsistent. This implementation uses complete consistency in each dimension as the judgment condition and does not set a similarity threshold. Therefore, the judgment result of frame feature consistency is directly determined by the actual feature values.
[0223] Furthermore, after the frame feature vectors of the first video frame and the second video frame are consistent, the first arrival timestamp and the second arrival timestamp are read.
[0224] The time difference is calculated by subtracting the time represented by the first arrival timestamp from the time represented by the second arrival timestamp, and taking the absolute value of the resulting time difference. This absolute value is determined as the response time, and the unit of recording is milliseconds. The response time obtained thus represents the time difference between the return data of the first target smart box and the second target smart box reaching the task scheduler after performing cross-end retrieval for the same video frame.
[0225] The obtained response time is used as the input of S6 in Example 1 and compared with the preset time threshold of S6. When the response time is greater than the time threshold, the task branch calculation logic is entered into the process of item-by-item masking and task feature extraction. When the response time is less than or equal to the time threshold, the calculation logic masking process is not started. Thus, the load difference matrix is responsible for limiting the two devices participating in cross-end invocation, the basic computing power consumption is used to determine the device combination corresponding to the current task branch, the same frame verification ensures that the two returned data correspond to the same video content, and the two arrival timestamps finally form the response time on which the subsequent task feature extraction is based.
[0226] Example 7:
[0227] S6 includes obtaining the response time of the target service. If the response time is greater than a preset time threshold, the initial set of computational logic and the initial basic computing power consumption of the task branch corresponding to the target service are extracted. The initial set of computational logic is then masked to obtain a subset of the computational logic after masking, and the basic computing power consumption after masking is obtained. The difference between the initial basic computing power consumption and the basic computing power consumption after masking is calculated to obtain the change in computing power consumption. The target computational logic is then extracted from the initial set of computational logic based on the change in computing power consumption, and the target computational logic is processed by feature mapping to obtain task features.
[0228] In this embodiment, the response time obtained in Embodiment 6 is taken as the response time of the target service, and the task branch that generates the response time is taken as the task branch corresponding to the target service. The task branch identifier is saved in the cross-end retrieval task. Therefore, after the response time is generated, the task branch record formed in Embodiment 3 can be read according to the task branch identifier, and the specific calculation logic that needs to be analyzed later can be determined.
[0229] Furthermore, the time threshold is obtained through a baseline retrieval before the equipment group is put into operation.
[0230] Specifically, two explosion-proof video algorithm analysis brain-like intelligent boxes within the device group whose current computing power load is within the standard computing power load range described in Example 4 are selected. The video source, video frame data volume, transmission path, and cross-end retrieval method are fixed, and the same frame retrieval described in Example 6 is executed continuously 30 times. The first 5 times are used to complete the communication connection and data cache initialization and do not participate in the threshold determination. The response times obtained in the last 25 times are added together and divided by 25 to obtain the baseline response time. Then, the absolute difference between the 25 response times and the baseline response time is calculated. The largest absolute difference is taken and added to the baseline response time. The resulting time is used as the time threshold corresponding to the current device group. If the device group members, video transmission link, or video encoding configuration changes, the above calibration process is re-executed.
[0231] In addition, when the response time of the target service is less than or equal to the time threshold, the computation logic shielding is not activated; when the response time is greater than the time threshold, all computation nodes contained in the task branch are read according to the task branch identifier, and the computation logic corresponding to each computation node is extracted according to the data dependency order determined in the task parsing tree of Example 3; all computation logic is arranged according to the actual execution order, and the resulting set is the initial computation logic set.
[0232] Furthermore, the initial basic computing power consumption adopts the basic computing power consumption determined in Example 3.
[0233] When entering this shielding analysis, the task branch is checked once again according to the same video resolution, video frame rate, execution device, and resource scheduling sequence as the basic computing power calibration. If the absolute difference between the unified computing power consumption value obtained in this execution and the already saved basic computing power consumption does not exceed the maximum fluctuation value of computing power load within the 5-second load statistics window in Example 4, then the already saved basic computing power consumption is directly used as the initial basic computing power consumption. If it exceeds the maximum fluctuation value, the calibration is re-executed according to the basic computing power calibration method in Example 3, and the updated value is used as the initial basic computing power consumption.
[0234] To ensure that the shielding operation only reflects the computing power consumption generated by a single computing logic, the current task branch is executed once before the shielding test begins, and the intermediate output data generated by each computing logic is saved.
[0235] Intermediate output data is stored according to the task branch identifier, calculation logic sequence number and the frame identifier of the first video frame, so that even if any calculation logic is blocked, its downstream calculation logic can still obtain the same data input as when it is fully executed.
[0236] The masking operation is performed item by item in the execution order of the initial computation logic set.
[0237] Specifically, when testing the first computational logic, only the actual computation process of the first computational logic is stopped, and the pre-saved intermediate output data of the computational logic is directly sent to its next computational logic. The remaining computational logics remain in their original execution state, thus forming the first subset of computational logics after masking. After the test is completed, the first computational logic is restored, and the second computational logic is masked in the same way. This process is repeated until all computational logics in the initial set of computational logics have been masked independently once. Only one computational logic is masked in each test, and no more than two computational logics are masked at the same time.
[0238] For each subset of computational logic formed after shielding, a computational power test is conducted using the same first video frame, video resolution, number of processing frames, execution device, and resource scheduling conditions as the initial basic computational power consumption. The shielded task branch is executed five times consecutively. The first execution is used to load intermediate cache and restore the state of computational resources and is not included in the statistics. The average resource usage of the processor, graphics processor, and neural network processing unit during the last four executions is read and converted into a unified computational power consumption value according to the resource weights determined in Example 3. The four unified computational power consumption values are added together and divided by 4. The result is used as the basic computational power consumption of the shielded computational logic subset.
[0239] Furthermore, for the currently blocked computing logic, the initial basic computing power consumption is subtracted from the basic computing power consumption after blocking, and the resulting value is used as the change in computing power consumption for that computing logic.
[0240] The change in computing power consumption corresponding to each computing logic in the initial computing logic set is calculated sequentially using the same method, thereby obtaining a one-to-one correspondence between computing logic and the change in computing power consumption.
[0241] The extraction of target computation logic is determined using a computing power change threshold; the computing power change threshold is determined based on the operational fluctuations of the complete task branch itself before the start of this shielding process.
[0242] Specifically, without shielding any computational logic, five complete task branches are executed consecutively. The result of the first execution is discarded, and the unified computing power consumption values of the next four executions are read. The maximum and minimum values are found, and the difference between the maximum and minimum values is used as the computing power change threshold for this task branch. This threshold uses the same input data, equipment, and resource scheduling conditions as the subsequent shielding test.
[0243] Furthermore, the change in computing power consumption for each computational logic is compared with the computing power change threshold.
[0244] Specifically, when the change in computing power consumption is greater than the computing power change threshold, the corresponding computing logic is extracted as the target computing logic; when the change in computing power consumption is less than or equal to the computing power change threshold, the corresponding computing logic is not extracted as the target computing logic; if the change in computing power consumption of all computing logics does not exceed the computing power change threshold, the computing logic with the largest change in computing power consumption is selected as the target computing logic; when the same maximum value occurs, the computing logic that is earlier in the execution order is selected.
[0245] Furthermore, after the target computation logic is determined, feature mapping is performed according to the pre-established computation logic feature mapping table. The feature mapping table is established simultaneously with the task node and resource mapping relationship in Embodiment 3, and at least records the computation logic type number, execution resource type number, and feature arrangement position; for each target computation logic, its computation logic type number, execution sequence number in the task branch, input feature dimension, output feature dimension, change in computing power consumption, and corresponding computing resource type are read.
[0246] Among them, the execution location feature is obtained by dividing the execution sequence number of the target computing logic by the total number of computing logics contained in the current task branch; the computing power contribution feature is obtained by dividing the change in computing power consumption of the target computing logic by the initial basic computing power consumption; the input feature dimension and the output feature dimension are respectively the actual dimension values already recorded in the task node; and then arranged in a fixed order according to the computing logic type number, execution location feature, computing power contribution feature, input feature dimension, output feature dimension and computing resource type number to form the feature data corresponding to the target computing logic.
[0247] When there are more than two target computation logics, the corresponding feature data are arranged in the order of execution of each target computation logic in the task branch.
[0248] When the task library is established, the number of computational logics contained in all task branches is read, and the largest number of computational logics is used as the feature capacity benchmark. When the number of target computational logics extracted from the current task branch does not reach the capacity benchmark, zeros are sequentially added to the end of the existing feature data until a fixed length is reached, thereby forming task features for classification and matching in Example 1 S7.
[0249] Example 8:
[0250] S7 includes acquiring the task characteristics and basic computing power consumption corresponding to the explosion-proof video algorithm, and constructing a computing power feature vector; acquiring the current computing power load of each explosion-proof video algorithm analysis brain-like intelligent box in the device group based on the computing power feature vector, and generating a load state matrix; performing classification matching processing on the load state matrix and computing power feature vector through support vector machine to obtain a classification matching probability value; if the classification matching probability value is greater than a preset probability threshold, generating an allocation mapping table based on the classification matching probability value, and determining the target allocation relationship based on the allocation mapping table.
[0251] In this embodiment, the task features obtained in Embodiment 7 and the basic computing power consumption determined in Embodiment 3 are read first. The task features are arranged in a predetermined order of computing logic type, execution location, computing power contribution, input feature dimension, output feature dimension and computing resource type, and the fixed data length determined in Embodiment 7 is maintained. The basic computing power consumption adopts a normalized value in the range of 0 to 1 determined in Embodiment 3 and is appended to the task features, thereby forming the computing power feature vector corresponding to the current task branch.
[0252] Furthermore, for the input feature dimension and output feature dimension in the task features, the maximum input feature dimension and the maximum output feature dimension appearing in all task branches are counted when the task recognition database is established.
[0253] When constructing the actual computing power feature vector, the input feature dimension of the current target computing logic is divided by the maximum input feature dimension, and the current output feature dimension is divided by the maximum output feature dimension, so that both data are converted to the range of 0 to 1; the computing logic type and computing resource type adopt the fixed number pre-assigned in the task library, and the execution position feature and computing power contribution feature directly adopt the normalization result obtained in Example 7; through the above processing, different task branches adopt the same data arrangement and the same data scale when entering the support vector machine.
[0254] Then, the device group obtained in Example 5 is read, and the current computing power load of each explosion-proof video algorithm analysis brain-like intelligent box in the group is obtained in ascending order of device identifiers according to Example 4. For each explosion-proof video algorithm analysis brain-like intelligent box, the current computing power load is read first, and then the current computing power load is subtracted from 1 to obtain the computing power value that is not currently occupied by the device. Then, the current computing power load is added to the basic computing power consumption of the task branch to be assigned to obtain the predicted computing power load after the current task branch is assigned to the device.
[0255] The predicted computing load is not truncated to its maximum value.
[0256] For example, if the sum of the current computing load and the basic computing power consumption is greater than 1, the actual sum is retained to represent the state that the computing power demand of the device exceeds the rated computing capacity after the task branch is issued; a set of device load status data is formed in the order of current computing load, unused computing power value and predicted computing load; each explosion-proof video algorithm analysis brain-like intelligent box in the device group corresponds to a set of data, and they are arranged row by row according to the device identification order, thus forming a load status matrix.
[0257] In addition, the support vector machine is trained before performing online classification matching.
[0258] Specifically, the training samples are obtained from the historical task allocation records that have been completed. Each training sample corresponds to the actual execution combination between a task branch and a brain-like intelligent box for explosion-proof video algorithm analysis. First, the computing power feature vector of the task branch is formed according to the same processing method as the online operation stage. Then, the current computing power load of the device before the task starts execution is read and the corresponding load status data is formed. The two parts of data are connected in a fixed order and used as the training input of a support vector machine.
[0259] In this embodiment, the category of the training samples is determined based on the actual execution results.
[0260] Specifically, after the task branch is issued to the corresponding explosion-proof video algorithm analysis brain-like intelligent box, if all computational logic is completed within the specified task execution cycle, the hazard identification result is generated normally, and the current computing power load obtained according to Example 4 is always within the standard computing power load range of the corresponding work intensity during the task execution, the sample is marked as a matching sample; if the task branch is not completed, the hazard identification result is not generated, or the current computing power load exceeds the standard computing power load range at least once during the task execution, the sample is marked as a non-matching sample.
[0261] Furthermore, historical samples are sorted from earliest to latest according to task completion time. The first 70% of the samples are used as support vector machine training samples, the middle 15% of the samples are used for classification probability calibration, and the remaining 15% of the samples are used for probability threshold determination. Data generated from the same task execution is only assigned to one dataset to avoid data from the same task execution participating in model training and threshold determination at the same time.
[0262] This embodiment uses a radial basis function kernel support vector machine to process the classification relationship between task features and device load status.
[0263] During training, the penalty parameter was set to 0.1, 1, 10 and 100 in sequence, and the radial basis kernel parameter was set to 0.01, 0.1, 1 and 10 in sequence. Five-fold cross-validation was performed on all parameter combinations. After each parameter group completed five validations, the number of samples misclassified in the five validations was added together as the number of classification errors for that parameter combination.
[0264] Furthermore, the number of classification errors corresponding to all parameter combinations is compared, and the parameter group with the smallest number of errors is determined as the formal running parameters. When there are more than two groups with the same minimum number of errors, the parameter group with the smaller penalty parameter value is selected first. When the penalty parameters are the same, the parameter group with the smaller radial basis function kernel parameter value is selected. After the parameters are determined, the support vector machine is retrained using all 70% of the training samples, and the training support vectors, classification boundary parameters and kernel parameters are saved.
[0265] In addition, during the actual classification and matching process, starting from the first row of the load state matrix, the load state data of one explosion-proof video algorithm analysis brain-like intelligent box is read sequentially and concatenated to the computing power feature vector of the current task branch to form the classification input data corresponding to the current task branch and the current device. This classification input data is then fed into the trained support vector machine to obtain the classification decision value between the current task branch and the current device. All explosion-proof video algorithm analysis brain-like intelligent boxes in the device group are traversed in the same way, so that each device obtains a corresponding classification decision value.
[0266] Furthermore, the classification decision values output by the support vector machine are further converted into classification matching probability values; this conversion relationship is established using the aforementioned 15% probability calibration samples.
[0267] Specifically, the probability calibration samples are input one by one into the already trained support vector machine, and the classification decision value and the actual matching category of each sample are recorded. Then, the monotonic sigmoid probability mapping method is used to adjust the correspondence between the decision value and the probability results from 0 to 1, so that the output corresponding to the matching sample is close to 1 and the output corresponding to the non-matching sample is close to 0.
[0268] The probability mapping parameters are solved iteratively using cross-entropy loss.
[0269] Specifically, after each iteration, the cross-entropy loss between the predicted probability and the actual class of all calibration samples is calculated, and the sum of the losses of all samples is divided by the number of calibration samples to obtain the current average loss; the upper limit of the number of iterations is set to 100; the iteration stops when the difference between the average loss of two adjacent iterations is less than 0.000001, and the current probability mapping parameters are saved; when running online, each classification decision value is input into the probability mapping relationship to obtain a classification matching probability value in the range of 0 to 1.
[0270] The probability threshold was determined using the remaining 15% of historical samples that were not involved in model training and probability calibration.
[0271] Specifically, the values between 0.50 and 0.95, arranged in 0.05 intervals, are used as candidate probability thresholds. For each candidate probability threshold, samples with a classification matching probability value greater than the candidate value are judged as matches, and samples with a classification matching probability value less than or equal to the candidate value are judged as non-matches.
[0272] Furthermore, for each candidate probability threshold, the number of samples that were actually mismatched but were judged as matches is counted, and the number of samples that were actually matches but were judged as mismatches is also counted. The number of the two types of erroneous samples is then added together. The total number of erroneous samples corresponding to all candidate probability thresholds is compared, and the candidate value with the smallest total number of erroneous samples is determined as the probability threshold. When two or more candidate values have the same minimum total number of erroneous samples, the candidate value with the largest value is selected as the probability threshold. After the support vector machine training data or model parameters are regenerated, the above probability calibration and probability threshold determination process is repeated.
[0273] After online matching is completed, the classification matching probability values of the current task branch and each explosion-proof video algorithm analysis brain-like intelligent box in the device group are compared.
[0274] When the classification matching probability value is greater than the probability threshold, the task branch and the corresponding device are written into the allocation mapping table; when the classification matching probability value is less than or equal to the probability threshold, no mapping record is established between the current task branch and the device.
[0275] In addition, each allocation mapping record must store at least the task branch identifier, the explosion-proof video algorithm analysis brain-like intelligent box device identifier, the classification matching probability value, the current computing power load, and the predicted computing power load. For the same task branch, all valid mapping records are arranged in descending order of classification matching probability value, and the record with the highest classification matching probability value is determined as the target record. The explosion-proof video algorithm analysis brain-like intelligent box in the target record is the target device of the current task branch.
[0276] When two or more devices have the same maximum classification matching probability value, compare the prediction computing load of each device and select the device with the smaller prediction computing load value; if the prediction computing load is still the same, then select the device with the smallest device identifier value in ascending order of device identifier.
[0277] Finally, the current task branch identifier is associated with the selected device identifier to form the target allocation relationship described in Example 1.
[0278] When the same identification task contains more than two task branches, the classification matching is performed in descending order of the basic computing power consumption determined in Example 3. When the basic computing power consumption is the same, the task branch identifiers are performed in ascending order. After the target device of each task branch is determined, the basic computing power consumption of the task branch is added to the current computing power load of the target device. The load state matrix is regenerated with the updated computing power load, and then the next task branch is processed, so that the subsequent classification matching uses the device load state after the task allocation has been completed.
[0279] Example 9:
[0280] S8 includes generating a routing distribution instruction based on the target allocation relationship, wherein the routing distribution instruction is used to distribute the task branch to the corresponding explosion-proof video algorithm analysis brain-like intelligent box within the device group; performing frame extraction processing on the explosion-proof video data of the brain-like intelligent box to obtain a video frame sequence; extracting a hazard feature map based on the video frame sequence; and if the response value of the hazard feature map is greater than a preset threshold, parsing the hazard feature map to obtain the hazard identification result output by the explosion-proof video algorithm analysis brain-like intelligent box.
[0281] In this embodiment, the target allocation relationship formed in Embodiment 8 is read first; each target allocation relationship includes at least a task branch identifier and a device identifier of the target explosion-proof video algorithm analysis brain-like intelligent box; the task scheduler queries the pre-saved device routing table according to the device identifier, obtains the communication address and task receiving port of the corresponding explosion-proof video algorithm analysis brain-like intelligent box, and writes the task branch identifier, target device identifier, monitoring device identifier, video start time, video end time, result return address and the unique task number of this task into the same data packet to form a routing distribution instruction.
[0282] Furthermore, after the routing distribution command is sent to the target explosion-proof video algorithm analysis brain-like intelligent box, the target device first compares the target device identifier in the command with the local device identifier.
[0283] When both are consistent, the task branch and its resource scheduling sequence formed in Example 3 are read according to the task branch identifier, and the video data source bound to the monitoring device is located according to the monitoring device identifier; the video start time and video end time are used to limit the range of video data to be processed in this task, so that the allocation relationship between the task branch and the target device determined in Example 8 is converted into the actual task execution relationship.
[0284] After obtaining the explosion-proof video data, frame extraction processing is performed.
[0285] This embodiment uses 200 milliseconds as a fixed frame extraction interval. The video start time is taken as the first sampling moment, and a new sampling moment is formed every 200 milliseconds thereafter, until the next sampling moment exceeds the video end time. For each sampling moment, the video frame with the smallest absolute time difference between the timestamp and the sampling moment is found in the video data, and this video frame is written into the video frame sequence. Video frames already selected in the previous sampling moment are not written again. When the duration of a video segment is less than 200 milliseconds, the first frame of the video segment is directly selected. After all frames are extracted, they are arranged in the order of their original timestamps from front to back to obtain the video frame sequence.
[0286] Each video frame in the video frame sequence is processed according to the input processing method determined in Example 2.
[0287] Specifically, the video frames are adjusted to 640 pixels by 640 pixels, while retaining the three color channels: red, green, and blue. Then, the pixel values of each channel are divided by 255 to convert the pixel values to the range of 0 to 1. The processed image is input into the 4-level convolutional feature extraction network that has been trained in Example 2.
[0288] Since the convolution stride of each of the four convolutional structures is 2, a 640-pixel by 640-pixel video frame is halved in size four times to obtain a 40-pixel by 40-pixel spatial feature map. The fourth convolutional structure contains 256 feature channels, so each video frame obtains convolutional feature data with 40 rows, 40 columns, and 256 channels. The spatial positional relationship of the 40 rows by 40 columns is preserved here to form a hazard feature map.
[0289] Furthermore, for each spatial location in the 40 rows by 40 columns, 256 feature values corresponding to that location are read in a fixed order according to the 256 channels, and the 256-dimensional local feature data is input into the hazard classification model corresponding to the current risk area in Example 2. The hazard classification model first sends the 256 input values to 128 hidden nodes. Each hidden node performs a weighted summation of the 256 input values according to the connection parameters saved after the model training is completed, and then performs linear rectification processing. The resulting 128 hidden layer outputs are then input into the classification output layer, which generates the classification results corresponding to each hazard type in the current risk area.
[0290] The classification output layer performs exponential operations on the output values corresponding to each hazard type, and then divides each exponential operation result by the sum of all exponential operation results to convert the results corresponding to each hazard type to the range of 0 to 1; reads the classification result of the hazard type corresponding to the current task branch, and uses this value as the hazard response value of the current location.
[0291] The 1600 spatial locations in the 40 rows by 40 columns are processed sequentially according to the above process. Then, the 1600 hazard response values are arranged according to the original row and column positions to obtain the single-frame hazard feature map corresponding to the current video frame.
[0292] When using this processing method, the model parameters, hidden layer structure, and output relationship of hazard classification model are all directly adopted from the model already trained in Example 2, without training a new map extraction model. The difference is that Example 2 uses global averaging of 256 feature channels to complete the hazard type determination of the entire image, while this step inputs the 256-dimensional features corresponding to each spatial location into the same hazard classification model to obtain the hazard response distribution that retains the spatial location relationship.
[0293] Generate single-frame hazard feature maps for all video frames in the video frame sequence using the same method.
[0294] Since all video frames come from the same fixed monitoring equipment and are all adjusted to the same 640-pixel by 640-pixel size, the same row and column positions in each single frame of the hazard feature map have a consistent screen coordinate reference.
[0295] When generating the hazard feature map corresponding to the video frame sequence, starting from the first row and first column, all single-frame hazard feature maps at that position are read, and the one with the largest value is selected and written into the corresponding position of the final hazard feature map. At the same time, the timestamp of the video frame that generated the largest response value is recorded. Then, the same process is performed position by position until all 1600 positions of 40 rows by 40 columns have been compared. Finally, a hazard feature map of 40 rows by 40 columns is formed, and the video frame time information corresponding to each position is obtained at the same time.
[0296] In this embodiment, the response value of the hazard feature map is obtained by traversing 1600 hazard response values in the map.
[0297] Compare all the hazard response values in turn, and determine the one with the largest value as the response value of the current hazard feature map. Record the row position, column position and corresponding video frame timestamp of the largest response value.
[0298] The preset threshold used to determine whether to analyze the hazard feature map is determined through independent verification data before the hazard classification model is put into operation.
[0299] The verification data consists of video clips corresponding to the current risk area. Each video clip has been confirmed to contain the current hazard type. The response value of the hazard feature map is calculated according to the same 200-millisecond frame extraction, 4-level convolutional feature extraction, spatial location classification, and multi-frame feature merging process as in the actual operation phase.
[0300] Furthermore, 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90 and 0.95 were selected as candidate thresholds in sequence.
[0301] Specifically, for each candidate threshold, verification videos with response values greater than the candidate threshold are judged to have corresponding hidden dangers, while verification videos with response values less than or equal to the candidate threshold are judged to not have corresponding hidden dangers. The number of samples that are judged to have hidden dangers but do not actually have hidden dangers, and the number of samples that are judged to not have hidden dangers but do actually have hidden dangers, are counted separately, and then the number of the two types of errors are added together.
[0302] Furthermore, the total number of errors corresponding to all candidate thresholds is compared, and the candidate value with the smallest total number of errors is determined as the preset threshold corresponding to the current hidden danger type.
[0303] When two or more candidate values have the same minimum number of errors, the candidate value with the largest value is selected as the preset threshold. If the parameters of the hazard classification model, the configuration of the risk area, or the hazard type changes, the above threshold determination process is repeated using the corresponding verification data.
[0304] When actually performing the task, the response value of the hazard feature map is compared with the preset threshold.
[0305] When the response value is less than or equal to the preset threshold, the hazard region analysis is not performed on the hazard feature map, and the identification status of the current task branch is recorded as no corresponding hazard detected; when the response value is greater than the preset threshold, the hazard feature map analysis process is entered.
[0306] In the process of analyzing the hazard feature map, positions in the 40-row by 40-column map with response values greater than a preset threshold are marked as valid response positions, and the remaining positions are marked as non-valid response positions. The valid response positions are divided into connected regions according to the 8-neighborhood connection relationship, that is, a valid response position and its adjacent valid response positions in the above, below, left, right and four diagonal directions are included in the same connected region. After traversing all valid response positions, at least one hazard response region is obtained.
[0307] Furthermore, for each hazard response area, the minimum row number, maximum row number, minimum column number, and maximum column number of the area are read to determine the boundary of the hazard area in the 40-row by 40-column feature map. The minimum column number is divided by 40 and then multiplied by 640 to obtain the left boundary of the hazard area in the 640-pixel image. The maximum column number is added by 1, divided by 40, and then multiplied by 640 to obtain the right boundary. The minimum and maximum row numbers are converted into the upper and lower boundaries using the same processing method.
[0308] Specifically, when the width and height of the original explosion-proof video image are not 640 pixels, the horizontal boundary of the aforementioned 640-pixel image is divided by 640 and multiplied by the original video width, and the vertical boundary is divided by 640 and multiplied by the original video height, thereby restoring the hidden danger area to the actual coordinate range of the original explosion-proof video image.
[0309] Simultaneously, read the maximum hazard response value within each hazard response area and read the timestamp of the video frame that generated the value; combine the hazard type corresponding to the current task branch, the coordinates of the hazard area in the original video frame, the hazard response value, the corresponding video frame timestamp, the monitoring device identifier, and the task branch identifier into one hazard identification record; when there are more than two hazard response areas, arrange the hazard identification records in descending order of the maximum hazard response value of each area.
[0310] Furthermore, after the target explosion-proof video algorithm analysis brain-like intelligent box completes the current task branch, it encapsulates the generated hazard identification record, equipment identifier, and task number into a hazard identification result, and sends it to the task scheduler according to the result return address recorded in the routing distribution instruction.
[0311] Example 10:
[0312] A multi-device collaborative analysis system for an explosion-proof video algorithm analysis brain-inspired intelligent box, used to implement the steps of the multi-device collaborative analysis method for the explosion-proof video algorithm analysis brain-inspired intelligent box, the system comprising:
[0313] The hazard type determination module acquires the physical locations of at least two explosion-proof video algorithm analysis brain-like intelligent boxes within the explosion-proof site and the first video images collected by the monitoring equipment corresponding to the explosion-proof video algorithm analysis brain-like intelligent boxes, and determines the hazard type corresponding to the first video image based on the physical location;
[0314] The task splitting module splits the pre-established identification task according to the type of hazard, obtains at least one task branch corresponding to the type of hazard, and obtains the basic computing power consumption corresponding to the task branch.
[0315] The busy / idle misalignment judgment module obtains the current computing power load of the explosion-proof video algorithm analysis brain-like intelligent box and the work intensity corresponding to the first video frame. If the current computing power load and work intensity do not match, it is determined that there is a busy / idle misalignment in the explosion-proof video algorithm analysis brain-like intelligent box.
[0316] The device grouping module, for the explosion-proof video algorithm analysis brain-like intelligent box with busy / idle time mismatch, uses hierarchical clustering algorithm to cluster and divide the physical location and current computing power load to obtain at least one device group;
[0317] The response time determination module selects two explosion-proof video algorithm analysis brain-like intelligent boxes from the device group whose current computing power load difference is closest to the basic computing power consumption, and retrieves the same first video frame across the device, and uses the time difference between the two retrievals as the response time.
[0318] If the response time of the task feature extraction module is greater than the preset time threshold, the computational logic contained in the task branch will be blocked in turn. The corresponding computational logic will be extracted based on the change in basic computing power consumption before and after blocking to obtain the task features.
[0319] The allocation relationship determination module uses support vector machines to classify and match task characteristics, basic computing power consumption, and the current computing power load of each explosion-proof video algorithm analysis brain-like intelligent box in the device group to determine the target allocation relationship.
[0320] The collaborative analysis module distributes task branches to the corresponding explosion-proof video algorithm analysis brain-like intelligent boxes within the equipment group according to the target allocation relationship, and obtains the hazard identification results output by the explosion-proof video algorithm analysis brain-like intelligent boxes.
[0321] In this embodiment, the hazard type determination module first obtains the physical location of each explosion-proof video algorithm analysis brain-like intelligent box in the explosion-proof site and the first video image collected by the corresponding monitoring equipment, and determines the hazard type corresponding to the first video image according to the processing method of three-dimensional spatial coordinates, risk area and hazard classification model described in Embodiment 2.
[0322] The task splitting module reads the corresponding identification task according to the type of hidden danger, and forms task branches according to the task parsing tree, computational complexity and node splitting process described in Example 3. At the same time, it determines the basic computing power consumption of each task branch based on the resource scheduling sequence and historical computing power consumption records.
[0323] The busy / idle misalignment judgment module reads the underlying hardware operation log to determine the current computing power load and calculates the job intensity based on the first video frame; then it reads the corresponding standard computing power load range according to the computing power matching mapping table established in Example 4; when the current computing power load is outside the range, the corresponding explosion-proof video algorithm analysis brain-like intelligent box is judged to have a busy / idle misalignment.
[0324] The device grouping module reads the physical location and current computing load of devices with busy / idle misalignment, forms an Euclidean distance matrix according to the method described in Example 5, and obtains device groups after hierarchical clustering and cluster tree pruning.
[0325] The response time determination module reads the current computing power load of each device in the device group, forms a load difference matrix, and selects two target smart boxes based on the basic computing power consumption of the task branch. The two target smart boxes retrieve the first video screen across the end according to the same video frame identifier. After confirming that the returned screen corresponds to the same video frame, the response time is obtained based on the time difference of the two arrival timestamps.
[0326] The task feature extraction module compares the response time with a pre-defined time threshold. When the response time is greater than the time threshold, the computational logic in the task branch is sequentially blocked in the manner described in Example 7. The basic computing power consumption before and after blocking is compared. The target computational logic is extracted based on the change in computing power consumption and converted into task features.
[0327] The allocation relationship determination module combines task characteristics with basic computing power consumption to form computing power characteristic data, and combines the current computing power load of the brain-like intelligent box with the explosion-proof video algorithm analysis of each device group to perform support vector machine classification matching; after the classification matching probability value reaches the probability threshold condition determined in Example 8, the target device corresponding to the task branch is determined according to the allocation mapping record to form the target allocation relationship.
[0328] The collaborative analysis module generates routing and distribution instructions based on the target allocation relationship, and sends the task branch to the corresponding explosion-proof video algorithm analysis brain-like intelligent box; the target device performs frame extraction and hazard feature map extraction on the explosion-proof video data in accordance with the method described in Example 9, and after the map response value reaches the corresponding judgment condition, it analyzes the hazard area, hazard type and video frame time information, and finally outputs the hazard identification result.
[0329] Example 11:
[0330] A computer-readable storage medium storing computer instructions thereon, which, when executed, performs the multi-device collaborative analysis method of the explosion-proof video algorithm analysis brain-like intelligent box.
[0331] In this embodiment, a computer-readable storage medium stores computer instructions for executing the multi-device collaborative analysis method; the computer-readable storage medium includes non-volatile memory, solid-state memory, flash memory, or other non-transitory storage medium capable of storing computer instructions.
[0332] When the explosion-proof video algorithm analysis brain-like intelligent box, task scheduling device, or connected computing device starts the corresponding program, the processor reads computer instructions from the computer-readable storage medium and loads them into the running storage area. According to the execution order specified by the computer instructions, it completes the following tasks: physical location and video image acquisition, hazard type determination, identification task splitting, basic computing power consumption determination, busy / idle misalignment judgment, device group division, cross-terminal video retrieval, response time determination, task feature extraction, support vector machine classification matching, task branching, and hazard identification result output.
[0333] When the computer instruction executes the defined processing procedure, it calls the three-dimensional spatial coordinate processing, task parsing tree construction, computing load calculation, hierarchical clustering, load difference calculation, computing logic shielding, support vector machine classification, and hidden danger feature map parsing program that have been determined in the aforementioned implementation, so that the computer instruction completes the same data processing procedure as the corresponding method after execution, and outputs the corresponding hidden danger identification results.
[0334] Example 12:
[0335] like Figures 2 to 5 As shown, an experiment was designed for this application, and a video analysis system for explosion-proof locations composed of multiple brain-like intelligent boxes was built to verify the effectiveness of this application.
[0336] Experimental environment and conditions:
[0337] Table 1 Hardware Environment
[0338]
[0339] Table 2 Software Environment
[0340]
[0341] Experimental scenario: Six brain-like intelligent boxes were set up in a simulated explosion-proof environment, covering the loading and unloading area, storage tank area, central control room, passageway, vehicle entrance and exit, and other monitoring areas, with a total of six video feeds connected; during the experiment, test videos containing potential hazards such as open flames, smoke, personnel intrusion, and vehicle violations were continuously played to simulate changes in the intensity of real operations.
[0342] The experiment was set up with two control groups:
[0343] Control group (fixed deployment): The recognition tasks were fixedly deployed on each brain-like intelligent box according to the region, without task splitting or dynamic allocation;
[0344] Experimental group (this method): The process is executed according to this method, including hazard type determination, task branch splitting, busy / idle mismatch judgment, equipment group division, response time detection, task feature extraction and support vector machine classification matching, and finally task branches are distributed according to the target allocation relationship.
[0345] The experiment lasted for 8 hours and was divided into 4 monitoring periods (2 hours each). The average computing power utilization rate, the number of devices with busy / idle schedules, and the average identification response time for each type of hidden danger were statistically analyzed for each period.
[0346] Table 3 shows the average computing power utilization of the two schemes under different monitoring periods.
[0347] Table 3 Comparison of average computing power utilization rates during different monitoring periods
[0348]
[0349] From Table 3 and Figure 2 It can be seen that the average computing power utilization rate of this method is significantly higher than that of the fixed deployment method in each monitoring period. The overall average utilization rate increased from 64.3% to 81.3%, an increase of about 17 percentage points. This is because this method splits the identification task according to the type of hidden danger and dynamically allocates the task branches to devices with idle computing power, reducing the occupation of computing resources unrelated to the monitoring content and making fuller use of computing power resources.
[0350] Table 4 shows the average computing load of the six devices during the experiment.
[0351] Table 4 Comparison of average computing power load for each device
[0352]
[0353] From Table 4 and Figure 3It can be seen that the load on devices varies greatly under the fixed deployment method, with some devices having a load as high as 92% and others only 35%, resulting in a load range of 57 percentage points and a significant mismatch between busy and idle periods. This method uses busy-idle mismatch judgment and device grouping to distribute task branches to devices with appropriate loads according to the target allocation relationship, so that the load of each device is concentrated between 68% and 75%, and the load range is reduced to 7 percentage points, effectively improving the problem of uneven computing power load among devices.
[0354] Table 5 shows the number of devices that experienced a mismatch between busy and idle periods during each monitoring time period.
[0355] Table 5 Comparison of the number of devices with staggered busy and idle periods in each monitoring time period
[0356]
[0357] From Table 5 and Figure 4 It can be seen that, under the fixed deployment method, an average of 3.8 devices have busy-idle misalignment in each time period, while the average of this method is only 0.8, which reduces the number of devices with busy-idle misalignment by about 79%. This shows that this method can promptly detect and adjust the busy-idle misalignment state by judging whether the current computing power load and the work intensity are matched in real time, so that the computing power of the device is matched with the monitoring work intensity, and the system runs more stably.
[0358] Table 6 shows the average identification and response time for each type of hazard under the two schemes.
[0359] Table 6 Comparison of average identification and response times for each type of hazard
[0360]
[0361] From Table 6 and Figure 5 As can be seen, the average response time for identifying all four types of hazards is significantly shortened under this method, with the overall average response time decreasing from 715 milliseconds to 445 milliseconds, a reduction of approximately 37.8%. This is because this method detects the response time of the same video feed across different devices, extracts task features when the response time exceeds the limit, and determines the target allocation relationship by combining task features, basic computing power consumption, and the current computing power load of the device. This ensures that the task branches match the actual processing capacity of the device, avoiding overload queuing of some devices due to fixed task deployment, thereby shortening the hazard identification response time and improving the timeliness of hazard identification.
[0362] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A multi-device collaborative analysis method for explosion-proof video algorithm analysis brain-like intelligent boxes, characterized in that, include: S1. Obtain the physical locations of at least two explosion-proof video algorithm analysis brain-like intelligent boxes in the explosion-proof site and the first video images collected by the monitoring equipment corresponding to the explosion-proof video algorithm analysis brain-like intelligent boxes, and determine the type of hidden danger corresponding to the first video image based on the physical location; S2. The pre-established identification task is split into at least one task branch corresponding to the type of hidden danger, and the basic computing power consumption corresponding to the task branch is obtained. S3. If the current computing power load of the explosion-proof video algorithm analysis brain-like intelligent box does not match the work intensity corresponding to the first video screen, it is determined that the explosion-proof video algorithm analysis brain-like intelligent box has a busy-idle misalignment. S4. For explosion-proof video algorithm analysis brain-like intelligent boxes with busy / idle misalignment, cluster the physical location and current computing load to obtain at least one device group. S5. Select two explosion-proof video algorithm analysis brain-like intelligent boxes from the device group whose current computing power load difference is closest to the basic computing power consumption, and call the same first video screen across the terminal respectively. Use the time difference between the two calls as the response time. S6. If the response time is greater than the preset time threshold, the computational logic contained in the task branch is blocked in turn, and the corresponding computational logic is extracted based on the change in basic computing power consumption before and after blocking to obtain the task characteristics. S7. Classify and match the task characteristics, basic computing power consumption, and current computing power load of each explosion-proof video algorithm analysis brain-like intelligent box in the device group to determine the target allocation relationship; S8. Based on the target allocation relationship, the task branch is distributed to the corresponding explosion-proof video algorithm analysis brain-like intelligent box in the equipment group to obtain the hazard identification results.
2. The multi-device collaborative analysis method for explosion-proof video algorithm analysis of brain-like intelligent boxes according to claim 1, characterized in that: S1 includes: Acquire the physical locations of at least two explosion-proof video algorithm analysis brain-like intelligent boxes within the explosion-proof site and the first video images captured by the monitoring equipment corresponding to the explosion-proof video algorithm analysis brain-like intelligent boxes; The three-dimensional spatial coordinates of the brain-like intelligent box are determined based on the physical location and video algorithm analysis. Feature extraction is performed on the first video frame to obtain a multi-dimensional image feature vector; If the three-dimensional spatial coordinates are within the preset risk area, the multi-dimensional image feature vector is matched with the hazard classification model corresponding to the risk area to obtain the hazard type corresponding to the first video image.
3. The multi-device collaborative analysis method for explosion-proof video algorithm analysis of brain-like intelligent boxes according to claim 1, characterized in that: S2 includes: Obtain the feature dimensions corresponding to the hazard type, and parse the pre-established identification task based on the feature dimensions to obtain the task parse tree; If the computational complexity of the task parse tree is greater than the preset complexity threshold, then the task parse tree is split into nodes to obtain task branches. The resource scheduling sequence is obtained by mapping the task branches. Based on the resource scheduling sequence, read the historical computing power consumption records to determine the basic computing power consumption corresponding to the task branch.
4. The multi-device collaborative analysis method for explosion-proof video algorithm analysis brain-like intelligent box according to claim 1, characterized in that: S3 includes: Obtain the underlying hardware operation logs of the explosion-proof video algorithm analysis brain-like intelligent box to determine the current computing load, and obtain the first video frame captured by the explosion-proof video algorithm analysis brain-like intelligent box to obtain the corresponding workload. The corresponding standard computing load range is found in the pre-established computing power matching mapping table based on the workload intensity. Determine whether the current computing load is within the standard computing load range; If the current computing load is not within the standard computing load range, it is determined that the current computing load is not matched with the workload, and the explosion-proof video algorithm analysis brain-like intelligent box is found to have a busy-idle misalignment.
5. The multi-device collaborative analysis method for explosion-proof video algorithm analysis of brain-like intelligent boxes according to claim 1, characterized in that: S4 includes: Obtain the physical location and current computing load of the explosion-proof video analysis brain-like intelligent box; The spatial distribution coordinates and load feature vector are obtained based on the physical location and the current computing load. The Euclidean distance matrix is obtained by calculating the spatial distribution coordinates and load feature vectors. A hierarchical clustering algorithm is used to process the Euclidean distance matrix to generate a clustering dendrogram; Pruning the clustering dendrogram yields at least one device group.
6. The multi-device collaborative analysis method for explosion-proof video algorithm analysis brain-like intelligent box according to claim 1, characterized in that: S5 includes: Based on the explosion-proof video algorithm analysis of the brain-like intelligent box in the device group, a load difference matrix is generated. The first and second target smart boxes are determined based on the load difference matrix and the basic computing power consumption. Send cross-end retrieval commands to the first target smart box and the second target smart box; Receive the first video image and the first arrival timestamp returned by the first target smart box; Receive the second video feed and the second arrival timestamp returned by the second target smart box; If the frame feature vectors of the first video frame and the second video frame are the same, then the time difference between the second arrival timestamp and the first arrival timestamp is calculated to obtain the response time.
7. The multi-device collaborative analysis method for explosion-proof video algorithm analysis brain-like intelligent box according to claim 1, characterized in that: S6 includes: Obtain the response time of the target business. If the response time is greater than the preset time threshold, extract the initial set of computing logic and the initial basic computing power consumption of the task branch corresponding to the target business. The initial computation logic set is masked sequentially to obtain a masked subset of computation logic, and the basic computing power consumption corresponding to the masked subset of computation logic is obtained. The difference between the initial basic computing power consumption and the basic computing power consumption after shielding is calculated to obtain the change in computing power consumption. Based on the change in computing power consumption, the corresponding target computing logic is extracted from the initial set of computing logic, and the target computing logic is subjected to feature mapping processing to obtain task features.
8. The multi-device collaborative analysis method for explosion-proof video algorithm analysis brain-like intelligent box according to claim 1, characterized in that: S7 includes: Obtain the task characteristics and basic computing power consumption corresponding to the explosion-proof video algorithm, and construct a computing power feature vector; The current computing load of each explosion-proof video algorithm analysis brain-like intelligent box in the device group is obtained based on the computing power feature vector, and a load status matrix is generated. The load state matrix and computing power feature vector are classified and matched using a support vector machine to obtain the classification and matching probability value. If the classification matching probability value is greater than the preset probability threshold, an allocation mapping table is generated based on the classification matching probability value, and the target allocation relationship is determined based on the allocation mapping table.
9. The multi-device collaborative analysis method for explosion-proof video algorithm analysis brain-like intelligent box according to claim 1, characterized in that: S8 includes: A routing distribution instruction is generated based on the target allocation relationship. The routing distribution instruction is used to distribute the task branch to the corresponding explosion-proof video algorithm analysis brain-like intelligent box in the device group. The explosion-proof video data of the brain-like intelligent box is processed by frame extraction to obtain a video frame sequence; Extract hazard feature maps from video frame sequences; If the response value of the hazard feature map is greater than the preset threshold, the hazard feature map is analyzed to obtain the hazard identification result output by the explosion-proof video algorithm analysis brain-like intelligent box.
10. A multi-device collaborative analysis system for an explosion-proof video algorithm analysis brain-inspired intelligent box, used to implement the steps of the multi-device collaborative analysis method for the explosion-proof video algorithm analysis brain-inspired intelligent box according to any one of claims 1-9, characterized in that, The system includes: The hazard type determination module acquires the physical locations of at least two explosion-proof video algorithm analysis brain-like intelligent boxes within the explosion-proof site and the first video images collected by the monitoring equipment corresponding to the explosion-proof video algorithm analysis brain-like intelligent boxes, and determines the hazard type corresponding to the first video image based on the physical location; The task splitting module splits the pre-established identification task according to the type of hazard, obtains at least one task branch corresponding to the type of hazard, and obtains the basic computing power consumption corresponding to the task branch. The busy / idle misalignment judgment module obtains the current computing power load of the explosion-proof video algorithm analysis brain-like intelligent box and the work intensity corresponding to the first video frame. If the current computing power load and work intensity do not match, it is determined that there is a busy / idle misalignment in the explosion-proof video algorithm analysis brain-like intelligent box. The device grouping module, for the explosion-proof video algorithm analysis brain-like intelligent box with busy / idle time mismatch, uses hierarchical clustering algorithm to cluster and divide the physical location and current computing power load to obtain at least one device group; The response time determination module selects two explosion-proof video algorithm analysis brain-like intelligent boxes from the device group whose current computing power load difference is closest to the basic computing power consumption, and retrieves the same first video frame across the device, and uses the time difference between the two retrievals as the response time. If the response time of the task feature extraction module is greater than the preset time threshold, the computational logic contained in the task branch will be blocked in turn. The corresponding computational logic will be extracted based on the change in basic computing power consumption before and after blocking to obtain the task features. The allocation relationship determination module uses support vector machines to classify and match task characteristics, basic computing power consumption, and the current computing power load of each explosion-proof video algorithm analysis brain-like intelligent box in the device group to determine the target allocation relationship. The collaborative analysis module distributes task branches to the corresponding explosion-proof video algorithm analysis brain-like intelligent boxes within the equipment group according to the target allocation relationship, and obtains the hazard identification results output by the explosion-proof video algorithm analysis brain-like intelligent boxes.
