Excavator operation early warning infrared image processing method based on edge computing

Abstract

The application relates to the technical field of edge computing, and particularly discloses an infrared image processing method for excavator operation early warning based on edge computing, which comprises the following steps: dividing an infrared image into regions according to the temperature values of pixel points in the infrared image to obtain an active region and a background region; distributing the active region to edge nodes for preprocessing according to a network condition score F of the edge nodes; after all the active regions received by an edge node j are preprocessed, the active regions are redistributed to the edge node for preprocessing; after preprocessing of all the active regions is completed, the active regions are combined with the background regions which have not been preprocessed into a target image, and the target image is used for excavator operation early warning; and the real-time degree of the excavator operation early warning is improved.

Description

Technical Field

[0001] This invention relates to the field of edge computing technology, and more specifically to an infrared image processing method for early warning of excavator operations based on edge computing. Background Technology

[0002] Excavators often face collision risks during construction due to complex environments, and traditional methods relying on manual observation are often insufficient to detect potential hazards in a timely manner. To address this, infrared imaging is introduced as a sensing method. By capturing the thermal signals of surrounding targets, it can maintain stable monitoring under various lighting and weather conditions. When an obstacle is detected within the work area, a warning message is issued to alert the operators. This allows operators to take evasive action earlier, without relying on limited visibility, thus effectively improving operational safety.

[0003] In excavator operation early warning systems, while infrared images can provide target information in low-light environments, their inherent limitations are particularly prominent in collision detection applications. Infrared imaging is often accompanied by strong noise, resulting in blurred image edges and lost details, directly affecting the accuracy of obstacle recognition. To obtain usable detection results, additional complex processing steps such as image enhancement, denoising, and segmentation are usually required. However, these steps all take time to complete, and the insufficient computing power of edge nodes further prolongs the processing time. For scenarios like excavators that require real-time operation early warning, any significant delay can lead to untimely response, thereby weakening the safety assurance effect. Summary of the Invention

[0004] The purpose of this invention is to provide an infrared image processing method for excavator operation early warning based on edge computing, thereby solving the above-mentioned technical problems.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] An infrared image processing method for excavator operation early warning based on edge computing includes the following steps:

[0007] Infrared images are collected by infrared devices pre-deployed around the excavator. Each pixel in the infrared image carries a corresponding temperature value. The infrared image is divided into regions based on the temperature values ​​of the pixels in the infrared image to obtain active regions and background regions.

[0008] Obtain the network status score F of the edge nodes within the excavator's working area, calculate the network weight Ki = Fi / F' of edge node i, where F' represents the sum of the network status scores and Fi represents the network status score of edge node i. Extract all active areas from the infrared image and... Each active region is sent to edge node i for preprocessing, which includes correction, denoising, and enhancement.

[0009] After edge node j finishes preprocessing all the received active regions, it obtains the target number of the remaining edge nodes. The target number represents the number of active regions that have not finished preprocessing. The target numbers are sorted in descending order of size to obtain the number sort.

[0010] Obtain the target quantity P1 that ranks first in the quantity ranking and the corresponding network status score FF1 of the edge node. Obtain the network status score FF2 of the edge node j at this time. Send the N active regions from the target quantity to the edge node j for preprocessing. ;

[0011] After preprocessing, all active areas are merged with the unprocessed background areas to form the target image, which is then used for excavator operation early warning.

[0012] As a further aspect of the present invention: the division of regions includes:

[0013] Calculate the standard deviation between the temperature value of pixel j and the temperature values ​​of its neighboring pixels in the infrared image, and use this as the local temperature fluctuation of pixel j.

[0014] The infrared image is divided into multiple non-overlapping rectangular sub-blocks. The size of the rectangular sub-blocks is preset. The average value of the local temperature fluctuation of all pixels in the rectangular sub-block is calculated as the regional activity index of the rectangular sub-block.

[0015] Rectangular sub-blocks whose regional activity index exceeds a dynamic threshold are marked as active regions, and the remaining rectangular sub-blocks are marked as background regions. The dynamic threshold is adaptively determined based on the overall distribution of regional activity indices of all rectangular sub-blocks in the infrared image.

[0016] As a further aspect of the present invention: the division of regions includes:

[0017] Calculate the standard deviation between the temperature value of pixel j and the temperature values ​​of its neighboring pixels in the infrared image, and use this as the local temperature fluctuation of pixel j.

[0018] The infrared image is divided into multiple non-overlapping rectangular sub-blocks. The size of the rectangular sub-blocks is preset. The average value of the local temperature fluctuation of all pixels in the rectangular sub-block is calculated as the regional activity index of the rectangular sub-block.

[0019] Rectangular sub-blocks whose regional activity index exceeds a dynamic threshold are marked as active regions, and the remaining rectangular sub-blocks are marked as background regions. The dynamic threshold is adaptively determined based on the overall distribution of regional activity indices of all rectangular sub-blocks in the infrared image.

[0020] As a further aspect of the present invention: if If it is not an integer, then get If the decimal part is less than 0.5, then... Each active region is sent to edge node i for noise reduction. If the decimal part is greater than or equal to 0.5, then... Each active region is sent to edge node i for noise reduction.

[0021] As a further aspect of the present invention: obtaining the network condition score includes:

[0022] Collect network condition parameters, including latency, jitter, and packet loss rate;

[0023] Network condition scores are calculated based on network condition parameters and the distance between superior and inferior solutions.

[0024] As a further aspect of the present invention: Early warning of excavator operation via target image includes:

[0025] The active regions in the target image are obtained. If two target regions share a common edge, they are merged into a new target region.

[0026] The area of ​​the target region is obtained. If the area exceeds a preset area threshold, a prompt message is sent to the cockpit display terminal, including the location of the target region.

[0027] As a further aspect of the present invention: if a network condition score is less than a preset score threshold, the corresponding edge node is marked as an abnormal node, and the abnormal node does not participate in subsequent steps.

[0028] The beneficial effects of this invention compared to the prior art are as follows:

[0029] 1) This invention adaptively partitions infrared images, performs correction, denoising, and enhancement only on active parts, and performs adjacent target merging and area discrimination within the target image. This can enhance the continuity of target boundaries and structures, suppress background interference and noise artifacts, highlight target areas related to the operation, reduce false triggering and omissions, and output more stable and interpretable early warning results.

[0030] 2) This invention allocates tasks according to network conditions, schedules and redistributes unprocessed tasks in descending order after a node completes its task, removes abnormal nodes and processes only active areas, reduces invalid transmission and computation, avoids congestion and queuing, improves the continuity and coordination efficiency of the processing link, shortens the response path from data collection to early warning, and outputs prompt information to the cockpit more promptly.

[0031] 3) For complex working conditions, this invention can adaptively adjust the partitioning and processing range according to changes in scene temperature difference and noise; improve the stability of the system under communication fluctuations by weighting based on network conditions and isolating abnormal nodes; trigger early warning and provide location with the target image as a unified exit, which makes it easier for drivers to understand the on-site risks and enhances the reliability and safety of the entire operation process. Attached Figure Description

[0032] The invention will now be further described with reference to the accompanying drawings.

[0033] Figure 1 This is a flowchart illustrating the infrared image processing method for excavator operation early warning based on edge computing according to the present invention. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Please see Figure 1 As shown, this invention is an infrared image processing method for excavator operation early warning based on edge computing, comprising the following steps:

[0036] Infrared images are collected by infrared devices pre-deployed around the excavator. Each pixel in the infrared image carries a corresponding temperature value. The infrared image is divided into regions based on the temperature values ​​of the pixels in the infrared image to obtain active regions and background regions.

[0037] It should be noted that the angle between the optical axis of the infrared device and the vertical plane is greater than 30 degrees to ensure that infrared images from a slightly distant location can be acquired. The specific angle value can be set based on experience.

[0038] In a preferred embodiment of the present invention, the region division includes:

[0039] Calculate the standard deviation between the temperature value of pixel j and the temperature values ​​of its neighboring pixels in the infrared image, and use this as the local temperature fluctuation of pixel j.

[0040] The infrared image is divided into multiple non-overlapping rectangular sub-blocks. The size of the rectangular sub-blocks is preset. The average value of the local temperature fluctuation of all pixels in the rectangular sub-block is calculated as the regional activity index of the rectangular sub-block.

[0041] Rectangular sub-blocks whose regional activity index exceeds a dynamic threshold are marked as active regions, and the remaining rectangular sub-blocks are marked as background regions. The dynamic threshold is adaptively determined based on the overall distribution of regional activity indices of all rectangular sub-blocks in the infrared image.

[0042] Understandably, objects in infrared images with temperature differences from their surroundings will experience more dramatic temperature changes in space, with more pronounced fluctuations in temperature values ​​between adjacent pixels. This intensity of local variation can be reflected by measuring the neighborhood standard deviation. By averaging the pixel-level local temperature fluctuations within a rectangular sub-block of a preset size, the judgment scale is elevated from a single pixel to the sub-block level, making the distinction based on the characteristic of "whether the region as a whole exhibits significant temperature fluctuations." Then, based on the overall distribution of the activity index of each sub-block in the entire image, a threshold is adaptively determined, marking sub-blocks with high activity as active regions and sub-blocks with low activity as background regions. The judgment is based on relative distribution rather than a fixed constant, thus assigning areas with significant temperature differences from the surroundings and concentrated spatial variations to active regions, while areas with relatively uniform temperature distribution are assigned to background regions.

[0043] First, divide the image into active and background regions based on local temperature fluctuations. Separate the areas that show obvious temperature fluctuations in space and are more relevant to operational risks from the overall image. This allows subsequent correction, denoising, and enhancement to be applied only to active areas, avoiding ineffective processing of the background with uniform temperature distribution, thereby reducing the burden of data transmission and computation.

[0044] In a preferred embodiment, the process of determining the dynamic threshold includes:

[0045] Calculate the arithmetic mean and standard deviation of the region activity index for all rectangular sub-blocks;

[0046] Obtain the maximum and minimum values ​​of all regional activity metrics;

[0047] The weighted sum of the arithmetic mean and standard deviation is used as the base value for the dynamic threshold.

[0048] The base value is compared with the maximum and minimum values. If the base value is greater than or equal to the maximum value, the maximum value is taken as the dynamic threshold. If the base value is less than or equal to the minimum value, the minimum value is taken as the dynamic threshold. If the base value is greater than the minimum value and less than the maximum value, the base value is taken as the dynamic threshold.

[0049] It is worth noting that the arithmetic mean of the activity indices of all sub-blocks is used to characterize the overall thermal fluctuation level of the current frame, and the standard deviation is used to characterize the dispersion of activity among sub-blocks. The two are weighted and summed to obtain the base value, which reflects the comprehensive criterion of "overall level + fluctuation amplitude". Then, the maximum and minimum values ​​are used as the upper and lower bounds, and the base value is compared with them and the value is taken accordingly. This allows the threshold to adapt to the distribution of the current frame without exceeding the actual observation range, so that the threshold is determined based on the relative distribution of activity in the whole map.

[0050] The threshold is adjusted synchronously with the overall thermal fluctuations and dispersion of the current scene to avoid the division deviation caused by the mismatch between the fixed threshold and the data distribution under different working conditions. The discrimination threshold is set based on the statistical distribution of the current frame to reduce the confusion between activity and background caused by excessively high or low thresholds. This results in a partitioning standard that is compatible with the current image distribution, providing a more suitable pre-partitioning basis for subsequent processing and early warning based on active areas.

[0051] Obtain the network status score F of the edge nodes within the excavator's working area, calculate the network weight Ki = Fi / F' of edge node i, where F' represents the sum of the network status scores and Fi represents the network status score of edge node i. Extract all active areas from the infrared image and... Each active region is sent to edge node i for preprocessing, which includes correction, denoising, and enhancement.

[0052] In another preferred embodiment of the present invention, if If it is not an integer, then get If the decimal part is less than 0.5, then... Each active region is sent to edge node i for noise reduction. If the decimal part is greater than or equal to 0.5, then... Each active region is sent to edge node i for noise reduction.

[0053] It should be noted that if a network condition score is lower than a preset score threshold, the corresponding edge node will be marked as an abnormal node, and the abnormal node will not participate in subsequent steps.

[0054] The weights obtained from the network condition scores are used to proportionally allocate active regions, ensuring that the processing load received by each edge node matches its current network capacity. Non-integer assignments are rounded down at the midpoint to limit the discrete errors caused by proportional calculations, preventing tasks from being over-split or deviating from the weight relationship, thus maintaining consistency between allocation and network conditions. As a result, active regions are prioritized for nodes with better network conditions, allowing them to enter the preprocessing process (including denoising steps) faster, reducing waiting and congestion caused by poor links, and shortening the path from capture to preprocessing completion. The aim is to improve the availability speed of active regions and provide more timely input for subsequent target image-based job early warnings.

[0055] Correction refers to the process of correcting deviations in infrared images caused by sensor imaging, acquisition angle, or environmental interference, such as inaccurate temperature value mapping or pixel misalignment. Through correction, an image that is closer to the actual situation can be obtained.

[0056] Denoising refers to the process of eliminating random noise or irregular stripes in an image caused by thermal noise, sensor jitter, or external interference, while maintaining the main contours of temperature changes and weakening the interference from irrelevant noise, so that subsequent processing can focus on the real active areas.

[0057] Enhancement refers to further strengthening contrast and boundary features after denoising, making the difference between temperature fluctuation areas and the background more obvious, highlighting useful structural information, and facilitating subsequent partitioning and target extraction.

[0058] In another preferred embodiment of the present invention, obtaining the network condition score includes:

[0059] Collect network condition parameters, including latency, jitter, and packet loss rate;

[0060] Network condition scores are calculated based on network condition parameters and the distance between superior and inferior solutions.

[0061] Obtain the target quantity P1 that ranks first in the quantity ranking and the corresponding network status score FF1 of the edge node. Obtain the network status score FF2 of the edge node j at this time. Send the N active regions from the target quantity to the edge node j for preprocessing. ;

[0062] It is important to note that for the node with the highest number of incomplete tasks, its network status score is compared with that of the current node to determine the number of active areas that need to be transferred and handed over to the current node for preprocessing. This aims to alleviate congestion from the most congested location, allowing waiting active areas to enter the processing flow as quickly as possible. Dynamic redistribution based on differences in network conditions can alleviate single-point queuing and transmission delays, reduce idle time and backlog caused by imbalances, and maintain the continuity of the processing link. By promptly digesting backlogged tasks, the overall processing rhythm is more in line with the current network conditions, and the progress of preprocessing is more stable, thus providing direct support for the timeliness and reliability of job early warning.

[0063] After preprocessing, all active areas are merged with the unprocessed background areas to form the target image, which is then used for excavator operation early warning.

[0064] The method for acquiring the active region of the target image is the same as that for infrared images, and will not be described in detail here;

[0065] A preferred embodiment of the present invention provides an early warning system for excavator operations based on target images, comprising:

[0066] The active regions in the target image are obtained. If two target regions share a common edge, they are merged into a new target region.

[0067] The area of ​​the target region is obtained. If the area exceeds a preset area threshold, a prompt message is sent to the cockpit display terminal, including the location of the target region.

[0068] It is important to note that after preprocessing, the image is synthesized with the background to form the target image. On the same carrier, active blocks that "share a common edge" are merged into a complete target based on geometric connectivity. The area is used as the trigger threshold because the area directly originates from the geometric properties of the target region and can objectively reflect the actual area occupied by the active region in the image, matching the continuous active region caused by temperature difference. Even if there are temperature fluctuations, the area of ​​scattered noise or fragments is usually limited. The area threshold can be used to distinguish them from the actual target with continuous external connections. When the area is small, it generally does not cause collision risk. The specific area threshold can be set based on experience.

[0069] By combining connectivity merging with area threshold constraints, the warning is triggered based on the criteria of "whether a continuous target is formed" and "whether there is a sufficient spatial range". This reduces the inclusion of scattered small blocks in the warning and highlights more practically significant active areas, thus providing clear and executable terminal criteria for the implementation of excavator operation warnings.

[0070] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the present invention should still fall within the scope of the present invention.

Claims

1. An infrared image processing method for excavator operation early warning based on edge computing, characterized in that, Includes the following steps: Infrared images are collected by infrared devices pre-deployed around the excavator. Each pixel in the infrared image carries a corresponding temperature value. The infrared image is divided into regions based on the temperature values ​​of the pixels in the infrared image to obtain active regions and background regions. Obtain the network status score F of the edge nodes within the excavator's working area, calculate the network weight Ki = Fi / F' of edge node i, where F' represents the sum of the network status scores and Fi represents the network status score of edge node i. Extract all active areas from the infrared image and... Each active region is sent to edge node i for preprocessing, which includes correction, denoising, and enhancement. After edge node j finishes preprocessing all the received active regions, it obtains the target number of the remaining edge nodes. The target number represents the number of active regions that have not finished preprocessing. The target numbers are sorted in descending order of size to obtain the number sort. Obtain the target quantity P1 that ranks first in the quantity ranking and the corresponding network status score FF1 of the edge node. Obtain the network status score FF2 of the edge node j at this time. Send the N active regions from the target quantity to the edge node j for preprocessing. ; After preprocessing, all active areas are merged with the unprocessed background areas to form the target image, which is then used for excavator operation early warning. The division of regions includes: Calculate the standard deviation between the temperature value of pixel j and the temperature values ​​of its neighboring pixels in the infrared image, and use this as the local temperature fluctuation of pixel j. The infrared image is divided into multiple non-overlapping rectangular sub-blocks. The size of the rectangular sub-blocks is preset. The average value of the local temperature fluctuation of all pixels within the rectangular sub-block is calculated as the regional activity index of the rectangular sub-block. Rectangular sub-blocks whose regional activity index exceeds a dynamic threshold are marked as active regions, and the remaining rectangular sub-blocks are marked as background regions. The dynamic threshold is adaptively determined based on the overall distribution of regional activity indices of all rectangular sub-blocks in the infrared image. The process of determining the dynamic threshold includes: Calculate the arithmetic mean and standard deviation of the region activity index for all rectangular sub-blocks; Obtain the maximum and minimum values ​​of all regional activity metrics; The weighted sum of the arithmetic mean and standard deviation is used as the base value for the dynamic threshold. The base value is compared with the maximum and minimum values. If the base value is greater than or equal to the maximum value, the maximum value is taken as the dynamic threshold. If the base value is less than or equal to the minimum value, the minimum value is taken as the dynamic threshold. If the base value is greater than the minimum value and less than the maximum value, the base value is taken as the dynamic threshold.

2. The infrared image processing method for excavator operation early warning based on edge computing according to claim 1, characterized in that, like If it is not an integer, then get If the decimal part is less than 0.5, then... Each active region is sent to edge node i for noise reduction. If the decimal part is greater than or equal to 0.5, then... Each active region is sent to edge node i for noise reduction.

3. The infrared image processing method for excavator operation early warning based on edge computing according to claim 1, characterized in that, The network condition score is obtained by: Collect network condition parameters, including latency, jitter, and packet loss rate; Network condition scores are calculated based on network condition parameters and the distance between superior and inferior solutions.

4. The infrared image processing method for excavator operation early warning based on edge computing according to claim 1, characterized in that, Early warning of excavator operations based on target images includes: The active regions in the target image are obtained. If two target regions share a common edge, they are merged into a new target region. The area of ​​the target region is obtained. If the area exceeds a preset area threshold, a prompt message is sent to the cockpit display terminal, including the location of the target region.

5. The infrared image processing method for excavator operation early warning based on edge computing according to claim 1, characterized in that, If a network condition score is lower than a preset score threshold, the corresponding edge node will be marked as an abnormal node, and the abnormal node will not participate in subsequent steps.

Images