Safety specification detection method and device for electric power operation personnel

Abstract

The invention discloses a safety specification detection method and device for an electric power worker, and relates to the field of electric power safety detection, and the method comprises the steps: obtaining initial dressing images of the worker at a plurality of different collection angles; performing dressing detection according to the initial dressing images of the operator at a plurality of different acquisition angles to obtain a dressing detection result of the operator; under the condition that the dressing detection result is that dressing meets the requirement, obtaining point cloud data of a plurality of different acquisition angles of an operator at each acquisition moment; performing trajectory analysis on the point cloud data of the plurality of different acquisition angles at each acquisition moment to obtain a limb moving trajectory; and inputting the limb moving trajectory into the behavior detection large model, and generating a behavior detection result of the operator. Through the implementation of the method and the device, the blind area limitation existing in single-view-angle detection is avoided, and the problems that in the prior art, dressing and behavior standard judgment is inaccurate due to dependence on a single view angle, and illegal operation risks cannot be found in time are solved.

Description

Technical Field

[0001] The present invention relates to the field of electric power safety detection, and in particular to a method and device for detecting safety regulations of electric power workers. Background Art

[0002] In the field of power operations, ensuring worker safety and compliance is crucial. Traditional safety compliance monitoring methods rely on single-angle cameras to monitor worker safety behavior. However, when workers are in complex working scenarios or face obstructions, single-angle monitoring data is limited, leading to inaccurate judgments about dress and behavior, and failing to promptly identify potential violations. Summary of the Invention

[0003] The embodiments of the present invention provide a method and device for detecting safety standards for power workers, avoiding the blind spot limitations of single-view detection, thereby solving the problem in the prior art of relying on a single viewpoint, resulting in inaccurate judgment of dress and behavioral standards and failure to timely detect risks of illegal operations.

[0004] An embodiment of the present invention provides a method for detecting safety standards of power workers, comprising:

[0005] Obtaining initial clothing images of the operator at several different acquisition angles;

[0006] Perform clothing detection based on the initial clothing images of the operator at several different acquisition angles to obtain the clothing detection results of the operator;

[0007] When the clothing inspection result shows that the clothing meets the requirements, obtain point cloud data of the operator at several different collection angles at each collection moment;

[0008] Perform trajectory analysis on point cloud data from several different acquisition angles at each acquisition moment to obtain the limb movement trajectory;

[0009] The limb movement trajectory is input into the behavior detection model so that the behavior detection model generates the operator's behavior detection results based on the limb movement trajectory.

[0010] Furthermore, clothing detection is performed based on the initial clothing images of the operator at several different acquisition angles to obtain the clothing detection results of the operator, including:

[0011] Perform image fusion on the initial clothing images of the operator at several different acquisition angles to obtain a fused clothing image;

[0012] The fused clothing image is input into the trained clothing detection model so that the clothing detection model generates the clothing detection results of the operator based on the fused clothing image.

[0013] Furthermore, the clothing detection model is determined by the following method:

[0014] Obtain the location of the operator;

[0015] Determine the power operation area where the operator is located based on the operator's location;

[0016] From the preset power operation large model library, the power operation large model corresponding to the power operation area where the operator is located is selected as the clothing detection large model.

[0017] Furthermore, the initial clothing images of the operator at several different acquisition angles are fused to obtain a fused clothing image, including:

[0018] Taking any initial clothing image as the first target image, and taking all the initial clothing images except the first target image as the second target image; taking the acquisition angle of the first target image as the first acquisition angle, and taking the acquisition angle of the second target image as the second acquisition angle;

[0019] Calculating an angle difference between the first acquisition angle and the second acquisition angle to obtain a first angle difference;

[0020] determining a mapping relationship between the first target image and the second target image according to the first angle difference, the first image acquisition range at the first acquisition angle, and the second image acquisition range at the second acquisition angle;

[0021] determining, according to the mapping relationship, overlapping image regions and non-overlapping image regions between the first target image and the second target image;

[0022] The first target image and the second target image are fused according to the overlapping image area and the non-overlapping image area between the first target image and the second target image to obtain a fused clothing image.

[0023] Further, the non-overlapping image region includes: a first non-overlapping image region located in the first target image and a second non-overlapping image region located in the second target image;

[0024] The first target image and the second target image are fused according to the overlapping image area and the non-overlapping image area between the first target image and the second target image to obtain a fused clothing image, including:

[0025] Clustering each pixel point in the first non-overlapping image area and sorting them according to a preset sorting rule to obtain a plurality of first cluster categories;

[0026] For each first cluster category, calculating the distance between the first cluster category and a central pixel point in the overlapping image region in the first target image to obtain a first pixel distance; calculating the distance between the first cluster category and the nearest image edge in the overlapping image region in the first target image to obtain a first edge distance;

[0027] Clustering each pixel point in the second non-overlapping image area and sorting them according to a preset sorting rule to obtain a plurality of second cluster categories;

[0028] For each second cluster category, calculating the distance between the second cluster category and a central pixel point in the overlapping image area in the second target image to obtain a second pixel distance; calculating the distance between the second cluster category and the nearest image edge in the overlapping image area in the second target image to obtain a second edge distance;

[0029] For each first cluster category, adjusting the pixel value of the first cluster category according to the first pixel distance and the first edge distance of the first cluster category, and the second pixel distance and the second edge distance of the second cluster category with the same ranking as the first cluster category, to obtain an adjusted pixel value of the first cluster category;

[0030] Fusing the adjusted pixel values of the first cluster category with the pixel values of the second cluster category to obtain a fused non-overlapping image area;

[0031] splicing the overlapping image area of the first target image and the overlapping image area of the second target image to obtain a spliced overlapping image area;

[0032] The stitched overlapping image areas and the fused non-overlapping image areas are combined to obtain a fused clothing image.

[0033] Furthermore, determining a mapping relationship between the first target image and the second target image according to the first angle difference, the first image acquisition range at the first acquisition angle, and the second image acquisition range at the second acquisition angle includes:

[0034] Determining an angle ratio relationship according to the first image acquisition range and the second image acquisition range;

[0035] Determining a correlation coefficient based on the first angle difference and the angle ratio relationship;

[0036] Dividing the first image acquisition range into a plurality of first angle intervals, and determining a second angle interval corresponding to the second target image in each first angle interval according to the correlation coefficient;

[0037] For each first angle interval, determining a pixel offset between the first target image and the second target image according to the first angle interval and the corresponding second angle interval;

[0038] A mapping relationship between the first target image and the second target image is determined according to the pixel offsets corresponding to all first angle intervals.

[0039] Furthermore, trajectory analysis is performed on the point cloud data of several different acquisition angles at each acquisition moment to obtain the limb movement trajectory, including:

[0040] At each acquisition moment, among the point cloud data acquired at several different acquisition angles, any point cloud data is used as the first target point cloud, and the other point cloud data except the first target point cloud is used as the second target point cloud; the acquisition angle of the first target point cloud is used as the third acquisition angle, and the acquisition angle of the second target point cloud is used as the fourth acquisition angle;

[0041] Calculating an angle difference between the third acquisition angle and the fourth acquisition angle to obtain a second angle difference;

[0042] Reading the reflection intensity information from the first target point cloud as the first point cloud reflection intensity; reading the reflection intensity information from the second target point cloud as the second point cloud reflection intensity;

[0043] Performing point cloud alignment on the first target point cloud and the second target point cloud according to the second angle difference, the reflection intensity of the first point cloud and the reflection intensity of the second point cloud to obtain a first aligned point cloud and a second aligned point cloud;

[0044] Perform point cloud fusion on the first aligned point cloud and the second aligned point cloud to obtain the target fused point cloud;

[0045] Perform posture estimation based on the fused point cloud to obtain the operator's limb posture;

[0046] The limb postures at the continuous acquisition moments are analyzed to obtain the limb movement trajectory.

[0047] Further, the first target point cloud and the second target point cloud are aligned according to the second angle difference, the reflection intensity of the first point cloud, and the reflection intensity of the second point cloud to obtain a first aligned point cloud and a second aligned point cloud, including:

[0048] determining a reflection intensity ratio according to the reflection intensity of the first point cloud and the reflection intensity of the second point cloud;

[0049] Determine a first reflection intensity gradient in a three-dimensional direction based on the first point cloud reflection intensity of each coordinate in the first target point cloud, and determine a second reflection intensity gradient in the three-dimensional direction based on the second point cloud reflection intensity of each coordinate in the second target point cloud;

[0050] Transforming each coordinate in the first target point cloud according to the reflection intensity ratio and the first reflection intensity gradient to obtain a first transformed coordinate, and transforming each coordinate in the second target point cloud according to the reflection intensity ratio and the second reflection intensity gradient to obtain a second transformed coordinate;

[0051] Determine the rotation angle parameter of the point cloud in the three-dimensional direction according to the second angle difference;

[0052] According to the rotation angle parameter of the point cloud in the three-dimensional direction, angle rotation compensation is performed on the first transformed coordinates and the second transformed coordinates to obtain a first aligned point cloud and a second aligned point cloud.

[0053] Furthermore, performing point cloud fusion on the first aligned point cloud and the second aligned point cloud to obtain a target fused point cloud includes:

[0054] Calculate the point cloud curvature of the first aligned point cloud and the second aligned point cloud;

[0055] Performing curvature feature grading on each point cloud in the first aligned point cloud and the second aligned point cloud according to the curvature of the point clouds, to obtain a first curvature feature grade of each point cloud in the first aligned point cloud and a second curvature feature grade of each point cloud in the second aligned point cloud;

[0056] Determine two point clouds with the same first curvature feature level and the same second curvature feature level as a curvature matching point cloud pair;

[0057] The point clouds other than the curvature matching point cloud pairs in the first aligned point cloud are used as the first non-matching point cloud, and the point clouds other than the curvature matching point cloud pairs in the second aligned point cloud are used as the second non-matching point cloud;

[0058] According to the curvature feature level of the curvature matching point cloud pair and the spatial distance between the two point clouds in the curvature matching point cloud pair, the curvature matching point cloud pair is fused to obtain the initial fused point cloud;

[0059] Determining an interpolation ratio according to an average distance between the first non-matching point cloud and the initial fused point cloud, and an average distance between the second non-matching point cloud and the initial fused point cloud;

[0060] According to the interpolation ratio, the first non-matching point cloud and the second non-matching point cloud are inserted into the corresponding positions in the initial fused point cloud to obtain the target fused point cloud.

[0061] Based on the above method embodiment, the present invention provides a corresponding device embodiment, including: a clothing image acquisition module, a clothing detection module, a point cloud data acquisition module, a trajectory analysis module, and a behavior detection module;

[0062] A clothing image acquisition module is used to acquire initial clothing images of the operator at several different acquisition angles;

[0063] The clothing detection module is used to perform clothing detection based on the initial clothing images of the operator at several different acquisition angles to obtain the clothing detection results of the operator;

[0064] The point cloud data acquisition module is used to obtain point cloud data of the operator at several different acquisition angles at each acquisition moment when the clothing detection result shows that the clothing meets the requirements;

[0065] The trajectory analysis module is used to perform trajectory analysis on the point cloud data of several different acquisition angles at each acquisition moment to obtain the limb movement trajectory;

[0066] The behavior detection module is used to input the limb movement trajectory into the behavior detection model so that the behavior detection model can generate the operator's behavior detection results based on the limb movement trajectory.

[0067] Compared with the prior art, the beneficial effects of the embodiment of this solution are:

[0068] The present invention obtains the initial clothing images of the operator at several different acquisition angles, performs clothing detection based on the initial clothing images of the operator at several different acquisition angles, and obtains the clothing detection results of the operator. Compared with traditional single-view recognition, multiple acquisition angles can obtain more comprehensive clothing detail information. When the clothing detection result shows that the clothing meets the requirements, the point cloud data of the operator at several different acquisition angles at each acquisition moment is obtained, and the trajectory analysis of the point cloud data at several different acquisition angles at each acquisition moment is performed to restore the operator's limb movement trajectory. The three-dimensional restoration of the limb movement trajectory can more accurately capture illegal actions and avoid missed detections caused by movement occlusion in a single perspective. Finally, the limb movement trajectory is input into the behavior detection model to generate the operator's behavior detection results.

[0069] In summary, the present invention effectively breaks through the blind spot limitations of single-view detection by collecting the initial clothing images and point cloud data of the workers from multiple angles, thereby solving the problem in the existing technology that the single-view detection data is limited, resulting in inaccurate judgment of clothing and behavioral norms, and failure to timely discover the risks of illegal operations, and achieving the accuracy of power operation safety standard detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 This is a flow chart of a safety compliance detection method for electric power workers provided by one embodiment of the present invention;

[0071] Figure 2It is a structural diagram of a safety standard detection device for electric power workers provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0072] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0073] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features.

[0074] like Figure 1 As shown, in order to solve the problem in the prior art that reliance on a single perspective leads to inaccurate judgment of dress and behavioral standards and inability to timely detect risks of illegal operations, an embodiment of the present invention provides a method for detecting safety standards for power workers, the method comprising at least the following steps:

[0075] Step S1: Acquire the initial clothing images of the operator at several different acquisition angles;

[0076] In step S1, multiple visual devices (e.g., cameras) are deployed within the power work environment, with the heads of the visual devices distributed at different positions and angles to ensure that images of the worker's clothing can be captured from multiple perspectives. When the worker enters the power work environment, each visual device captures an initial image of the worker's clothing at the same or similar time, thereby obtaining comprehensive visual information about the worker's clothing.

[0077] In this example, three cameras are installed in the high-voltage distribution room of the substation, located in the upper left corner, upper right corner, and front of the room. When worker B enters the room, the three cameras simultaneously capture images of B's initial clothing. These multi-angle images capture the worker's clothing from multiple perspectives, including the front, side, and back, effectively avoiding obstructions that may occur from a single angle.

[0078] Step S2: performing clothing detection based on the initial clothing images of the operator at several different acquisition angles to obtain the clothing detection results of the operator;

[0079] For step S2, these multi-perspective clothing images are analyzed using an image recognition algorithm. By identifying the wearing conditions, color identification, integrity and other features of key safety equipment such as safety helmets, insulating clothing, and safety shoes, cross-validation and comprehensive judgment are made to determine whether the workers' clothing complies with safety regulations. Finally, the workers' clothing detection results are obtained.

[0080] In this embodiment, the clothing detection result includes whether the clothing meets the requirements or whether the clothing does not meet the requirements.

[0081] In a preferred embodiment, performing clothing detection based on initial clothing images of the operator at several different acquisition angles to obtain clothing detection results of the operator includes the following sub-steps:

[0082] Step S21: fusing the initial clothing images of the operator at several different acquisition angles to obtain a fused clothing image;

[0083] For step S21, the initial clothing images of the operator at several different acquisition angles are fused. By collecting and fusing images with multiple cameras at different angles, the clothing details from multiple perspectives such as the back, side, and front can be integrated. Compared with a single angle, the clothing status of the operator can be presented more completely and comprehensively, avoiding the omission of some clothing information due to the limitation of the viewing angle. In complex working scenarios such as the high-voltage distribution room of a substation where there is equipment obstruction and special spatial layout, multi-perspective information fusion can reduce misjudgments and missed judgments.

[0084] Continuing with the above example, three cameras are installed in the high-voltage distribution room of a substation, located in the upper left corner, upper right corner, and front of the room. When worker B enters the room, the three cameras simultaneously capture an initial image of B's clothing. The three cameras fuse the details of B's back captured by the upper-left camera, the side view captured by the upper-right camera, and the front view captured by the front camera to generate a fused clothing image.

[0085] In a preferred embodiment, the image fusion of the initial clothing images of the operator at several different acquisition angles to obtain a fused clothing image includes the following sub-steps:

[0086] Step S211: taking any initial clothing image as a first target image, and taking all the initial clothing images except the first target image as second target images; setting the acquisition angle of the first target image as a first acquisition angle, and setting the acquisition angle of the second target image as a second acquisition angle;

[0087] In step S211, one of the initial clothing images is randomly selected or selected according to a preset selection rule (e.g., acquisition time order) as the first target image, and the remaining initial clothing images are marked as second target images. Simultaneously, the acquisition angle corresponding to the first target image is recorded as the first acquisition angle, and the acquisition angle corresponding to the second target image is recorded as the second acquisition angle.

[0088] Continuing with the above example, in the high-voltage distribution room of a substation, three cameras capture three initial images of worker B in their clothing, labeled Image M, Image N, and Image O. Image M is selected as the first target image, while Images N and O are the second target images, in chronological order. The camera capture angle for Image M is 30° horizontally to the left (the first capture angle), the camera capture angle for Image N is 45° horizontally to the right (the second capture angle), and the camera capture angle for Image O is 15° vertically downward (the second capture angle).

[0089] Step S212: Calculate the angle difference between the first acquisition angle and the second acquisition angle to obtain a first angle difference;

[0090] For step S212, the first angle difference between the second acquisition angle and the first acquisition angle is calculated to obtain the first angle difference. The size of the angle difference can reflect the complementarity of the two perspectives. A large difference means that more clothing details can be supplemented and the information after fusion is more complete; conversely, a small difference means that local details can be enhanced.

[0091] Step S213: determining a mapping relationship between the first target image and the second target image according to the first angle difference, the first image acquisition range at the first acquisition angle, and the second image acquisition range at the second acquisition angle;

[0092] In step S213, a first image acquisition range at the first acquisition angle and a second image acquisition range at the second acquisition angle are obtained. The acquisition range can be determined by parameters such as the camera's field of view and the distance to the target. Next, a mapping relationship between the first target image and the second target image is determined based on the first angle difference, the first image acquisition range at the first acquisition angle, and the second image acquisition range at the second acquisition angle, as described in steps 2131 to 2135.

[0093] Step S214: determining an overlapping image area and a non-overlapping image area between the first target image and the second target image according to the mapping relationship;

[0094] For step S214, based on the mapping relationship, the homography matrix mapping between the first target image and the second target image is determined, and each pixel in the second target image is mapped to the coordinate system of the first target image through the homography matrix. All pixels of the second target image are traversed to determine whether the mapped pixel is within the range of the first target image. If it is within the range, the area corresponding to the pixel is the overlapping image area; if it is not within the range, the area corresponding to the pixel is the non-overlapping image area. For the first target image, the area of its own pixel after being mapped to the second target image is similarly determined to determine the overlapping image area and the non-overlapping image area.

[0095] Continuing with the above example, based on the homography matrix between image N and image M, each pixel of image N is mapped to the coordinate system of image M. After traversal and determination, it is found that the upper left corner of image N is mapped to the middle area of image M; this portion is the overlapping image area. The right and lower areas of image N exceed the range of image M after mapping; this portion is the non-overlapping image area. Furthermore, some areas of image M are outside the mapping range of image N and are also determined to be non-overlapping image areas.

[0096] Step S215: fusing the first target image and the second target image according to the overlapping image area and the non-overlapping image area between the first target image and the second target image to obtain a fused clothing image.

[0097] For step S215, the overlapping parts of images from different angles often contain key common features of the workers' clothing, and the non-overlapping parts can supplement information that cannot be covered by a single perspective. Therefore, the first target image and the second target image are fused based on the overlapping image area and the non-overlapping image area to obtain a fused clothing image. The fused image contains both high-confidence overlapping features and retained differentiated non-overlapping features, as described in steps 2151 to 2158.

[0098] Based on multi-perspective geometric transformation and multi-resolution analysis, the present invention realizes the effective fusion of the initial clothing images of workers at multiple different acquisition angles in the power operation environment, so that the finally generated fused clothing image integrates the clothing details of the workers at multiple acquisition angles, and can present the clothing status of the workers more comprehensively and accurately, providing high-quality data input for subsequent large-scale model clothing recognition, and effectively improving the accuracy and reliability of clothing detection.

[0099] In a preferred embodiment, determining a mapping relationship between the first target image and the second target image according to the first angle difference, the first image acquisition range at the first acquisition angle, and the second image acquisition range at the second acquisition angle includes the following sub-steps:

[0100] Step S2131: determining an angle ratio relationship according to the first image acquisition range and the second image acquisition range;

[0101] For step S2131, the acquisition range in the embodiment of the present invention can be determined by the horizontal field angle θ of the camera. h1 ,θ h2 and vertical field angle θ v1 ,θ v2 To quantify, for example, in the form of angle intervals, therefore, the horizontal angle ratio and the vertical angle ratio are calculated respectively, where the horizontal angle ratio Vertical angle ratio θ h1 represents the horizontal field of view angle corresponding to the first target image, θ h2 Indicates the horizontal field of view angle corresponding to the second target image, θ v1 Indicates the vertical field angle corresponding to the first target image, θ v2 Indicates the vertical field of view angle corresponding to the second target image.

[0102] Step S2132: determining a correlation coefficient based on the first angle difference and the angle ratio relationship;

[0103] In step S2132, according to the first angle difference, the horizontal angle ratio and the vertical angle ratio are combined, and the formula The correlation coefficient C is calculated, where θ represents the first angle difference.

[0104] Continuing with the above embodiment, in the high-voltage distribution room of the substation, the horizontal field angle θ of the camera corresponding to the first target image (image M) is h1 =60°, vertical field of view angle θ v1 =45°; the horizontal field of view angle θ of the camera corresponding to the second target image (image N) h2 =70°, vertical field of view angle θ v2 =50°. The first angle difference θ = 15°. Calculate the horizontal angle ratio Vertical angle ratio Then calculate the correlation coefficient according to the formula:

[0105]

[0106] Step S2133: Divide the first image acquisition range into a plurality of first angle intervals, and determine the second angle interval corresponding to the second target image in each first angle interval according to the correlation coefficient;

[0107] For step S2133, the first image acquisition range R1 is divided into n equal first angle intervals in the horizontal direction and vertical direction. For example, the first angle intervals divided in the horizontal direction are The first angle interval divided in the vertical direction is

[0108] For each first angle interval in the horizontal direction and the vertical direction, a second angle interval corresponding to the second target image in each first angle interval is determined according to the correlation coefficient C.

[0109] Taking the horizontal direction as an example, for example, the i-th first angle interval is Then the corresponding second angle interval i The calculation method is: in, and is the starting angle of the i-th angle interval of the first target image and the second target image in the horizontal direction, and the vertical direction is calculated similarly.

[0110] Continuing with the above embodiment, the image M is divided into five first angle intervals within the horizontal acquisition range of 60°, that is, each first angle interval is 12°. Therefore, the five first angle intervals are [0°, 12°], [12°, 24°], [24°, 36°], [36°, 48°], and [48°, 60°]. Based on the correlation coefficient C = 20.8, the second angle interval corresponding to the image N is calculated. Taking the first first angle interval [0°, 12°] as an example, Therefore, the first second angle interval is [0°, 12.48°], and so on, other second angle intervals of image N are calculated according to the first angle interval in image M.

[0111] Step S2134: For each first angle interval, determine the pixel offset between the first target image and the second target image according to the first angle interval and the corresponding second angle interval;

[0112] In step S2134, for each corresponding horizontal angle interval and vertical angle interval, the horizontal angle difference is calculated. Taking the horizontal direction as an example, in the i-th corresponding angle interval, the horizontal angle difference is calculated by the following formula:

[0113] According to the camera imaging principle, the angle difference is converted into pixel offset. For example, if the horizontal pixel resolution of the camera is P h , the horizontal field of view angle is θ h , then the horizontal pixel offset The same is true for the vertical direction, by calculating the vertical angle difference Combined with the vertical pixel resolution P v and vertical field angle θ v , get the vertical pixel offset

[0114] In one embodiment of the present invention, the horizontal angle difference is calculated in the first horizontal angle interval [0°, 12°] corresponding to the image M and the first horizontal angle interval [0°, 33.6°] corresponding to the image N. Given an image M, the camera's horizontal pixel resolution P h =1920, horizontal field of view angle θ h =60°, horizontal pixel offset Pixels. Similarly, calculate the vertical pixel offset.

[0115] Step S2135: Determine a mapping relationship between the first target image and the second target image according to the pixel offsets corresponding to all first angle intervals.

[0116] For step S2135, the mapping relationship between the first target image and the second target image is determined based on the pixel offsets corresponding to all first angle intervals. The specific mapping method is as follows: for any pixel (x1, y1) in the first target image, the pixel offset corresponding to the interval is obtained based on the angle interval in which it is located. Then the coordinates of the corresponding pixel point (x2, y2) in the second target image are calculated as follows: By traversing all the pixels of the first target image, the pixel mapping relationship between the two images is determined. In one embodiment, in image M, there is a pixel with coordinates (300, 200), which is located in the first horizontal angle interval [0°, 12°] and the first vertical angle interval. According to the previous calculation, the horizontal pixel offset of this interval is Pixels, vertical pixel offset Pixel. Then the corresponding pixel coordinates in image N are (300+15.36,200+17)=(315.36,217).

[0117] The present invention uses the angle ratio relationship and the correlation coefficient to determine the angle interval correspondence, and combines the camera imaging parameters to convert the angle difference into the pixel offset, and finally realizes the pixel-level precise mapping. Therefore, the comprehensive influence of the image acquisition range and the angle difference is taken into account, and the geometric relationship between images of different perspectives can be processed more accurately, so that in the subsequent image fusion process, the image content can be seamlessly spliced and aligned based on this mapping relationship, and the problems of image dislocation and deformation caused by different perspectives can be effectively avoided, which provides a reliable guarantee for generating high-quality fused clothing images, thereby improving the accuracy and stability of large-scale model recognition of clothing images, thereby improving the accuracy and reliability of clothing detection.

[0118] In a preferred embodiment, the non-overlapping image regions include: a first non-overlapping image region located in the first target image and a second non-overlapping image region located in the second target image;

[0119] The first target image and the second target image are fused according to the overlapping image area and the non-overlapping image area between the first target image and the second target image to obtain a fused clothing image, including the following sub-steps:

[0120] Step S2151: clustering each pixel point in the first non-overlapping image area and sorting them according to a preset sorting rule to obtain a plurality of first cluster categories;

[0121] For step S2151, a density-based clustering algorithm (such as the DBSCAN algorithm) is used to cluster the pixels in the first non-overlapping image area, wherein the DBSCAN algorithm divides closely connected pixels into a cluster category based on the distance and density between the pixels. A neighborhood radius ∈ and a minimum number of samples MinPts are set, each pixel in the first non-overlapping image area is traversed, and the number of pixels in its neighborhood is calculated. If the number of pixels in the neighborhood is greater than or equal to the neighborhood radius ∈, the point is taken as the core point, and all points in its neighborhood are classified into a cluster category. This process continues until all pixels are traversed and sorted according to a preset sorting rule to obtain a plurality of first cluster categories.

[0122] Step S2152: For each first cluster category, calculate the distance between the first cluster category and the center pixel point in the overlapping image area in the first target image to obtain a first pixel distance; calculate the distance between the first cluster category and the nearest image edge in the overlapping image area in the first target image to obtain a first edge distance;

[0123] For step S2152, for each first cluster category in the first non-overlapping image area, calculate the distance from each pixel point in the category to the central pixel point of the first overlapping image area of the first target image, and take the average value as the first pixel distance; calculate the distance from each pixel point in the category to the nearest edge of the first overlapping image area, and take the minimum value as the first edge distance.

[0124] Step S2153: clustering each pixel point in the second non-overlapping image area, and sorting them according to a preset sorting rule to obtain a plurality of second cluster categories;

[0125] Regarding step S2153 , similarly, a density-based clustering algorithm (such as the DBSCAN algorithm) is used to cluster the pixels in the second non-overlapping image region, and the pixels are sorted according to a preset sorting rule to obtain a plurality of second cluster categories.

[0126] Continuing with the above example, in the high-voltage distribution room of a substation, the first target image (image M) and the second target image (image N) are divided into regions to obtain their respective non-overlapping image regions. The first non-overlapping image region of image M is clustered using the DBSCAN algorithm, with a neighborhood radius ∈ = 5 pixels and a minimum sample number MinPts = 10. After calculation, the pixels in this region are divided into three first cluster categories: category M1, category M2, and category M3. The same operation is performed on the second non-overlapping image region of image N, resulting in three second cluster categories: category N1, category N2, and category N3.

[0127] Step S2154: For each second cluster category, calculate the distance between the second cluster category and the center pixel point in the overlapping image area in the second target image to obtain a second pixel distance; calculate the distance between the second cluster category and the nearest image edge in the overlapping image area in the second target image to obtain a second edge distance;

[0128] In step S2154 , for each second cluster category in the second non-overlapping image region, a second pixel distance to a central pixel of a second overlapping image region of the second target image and a second edge distance to a nearest edge are calculated.

[0129] Continuing with the above embodiment, taking the first cluster category M1 of image M as an example, the distances from all pixels in the first cluster category M1 to the center pixel of the first overlapping image area are calculated, and these distances are added and divided by the total number of pixels to obtain the first pixel distance d M1；pixel = 20 pixels; calculate the distance from each pixel in category M1 to the nearest edge of the first overlapping image area, take the minimum value, and obtain the first edge distance d M1；edge=5 pixels. Perform the same calculation on the second cluster category N1 of image N to obtain the second pixel distance d N1；pixel = 22 pixels, the second edge distance d N1；edge = 6 pixels. The same applies to other cluster categories.

[0130] Step S2155: For each first cluster category, adjust the pixel value of the first cluster category according to the first pixel distance and the first edge distance of the first cluster category, and the second pixel distance and the second edge distance of the second cluster category with the same ranking as the first cluster category, to obtain an adjusted pixel value of the first cluster category;

[0131] For step S2155, a pixel value adjustment algorithm based on distance attenuation is used. For example, if the pixel value of the first target category is P M1 , the pixel value of the second target category is P N1 , the first pixel distance is d M1；pixel , the first edge distance is d M1；edge , the second pixel distance is d N1；pixel , the second edge distance is d N1；edge , through the formula The pixel value of the first target category is adjusted to obtain the adjusted pixel value P of the first target category. M1；adjusted , α is the adjustment coefficient.

[0132] Similarly, the same operation is performed on other first cluster categories and corresponding second cluster categories to complete the pixel value adjustment of all first target categories. Continuing with the above embodiment, for the first cluster category M1 and the second cluster category N1, it is known that P M1 =120, P N1 =130,d M1；pixel , d M1；edge =5,d N1；pixel =22,d N1；edge =6, adjustment coefficient α = 0.5. Calculated according to the formula The adjusted pixel value of category M1 is 123.

[0133] Step S2156: fusing the adjusted pixel values of the first cluster category with the pixel values of the second cluster category to obtain a fused non-overlapping image area;

[0134] For step S2156, for the non-overlapping image area, the adjusted pixel values of the first target category are replaced and fused with the pixel values of the second target category, that is, according to the correspondence between the pixel points, the adjusted pixel values of the first target category are replaced to the corresponding positions in the fused image to complete the fusion of the non-overlapping image area.

[0135] Step S2157: stitching the overlapping image area of the first target image and the overlapping image area of the second target image to obtain a stitched overlapping image area;

[0136] In step S2157 , the first overlapping image area of the first target image and the second overlapping image area of the second target image are directly spliced together, while keeping the pixel values of their respective pixels unchanged, to obtain a spliced overlapping image area.

[0137] Step S2158: combining the spliced overlapping image area and the fused non-overlapping image area to obtain a fused clothing image.

[0138] In step S2158, the spliced overlapping image area and the fused non-overlapping image area are combined to obtain a fused clothing image.

[0139] Continuing with the above example, the overlapping image regions of images M and N are first directly concatenated to produce an image of the overlapping portions. Next, the adjusted pixel values of the first cluster category (e.g., 123 for category M1) are replaced and fused with the corresponding pixel values of the second cluster category (e.g., 130 for category N1), and the pixel values of category M1 are updated to the corresponding positions in the fused image. After all non-overlapping image regions are fused, they are combined with the overlapping image regions to produce the final fused clothing image.

[0140] The present invention classifies the pixels in the non-overlapping areas and adjusts the pixel values based on their distance relationship with the overlapping areas. This allows the non-overlapping areas to be reasonably adjusted in pixel value during fusion based on their relative position to the overlapping areas and the situation of the corresponding areas in the other image. While maintaining the pixel values in the overlapping areas unchanged, the adjusted pixel values in the non-overlapping areas are fused, effectively reducing the visual differences and splicing traces during image fusion. This allows for more detailed processing of the relationships between different image areas. The resulting fused clothing image retains the original image information, providing high-quality image data for accurate recognition of subsequent large models, thereby improving the accuracy and reliability of clothing detection.

[0141] Step S22: Input the fused clothing image into the trained clothing detection model, so that the clothing detection model generates the clothing detection result of the operator based on the fused clothing image.

[0142] In step S22, the fused clothing image is preprocessed, such as adjusting the image size and format conversion, and the preprocessed and fused clothing image is input into the trained clothing detection model. The clothing detection model uses deep learning architectures such as convolutional neural networks (CNN) to extract and analyze image features such as color, texture, and shape, and then compares them with the sample features in the training data. Based on the comparison results, the clothing recognition result is output, that is, whether the operator's clothing meets the requirements of the power operation environment area. If the model determines that the clothing in the image matches the clothing features that meet the requirements in the training data, the output clothing recognition result is "the clothing meets the requirements"; otherwise, the output clothing recognition result is "the clothing does not meet the requirements."

[0143] Continuing with the above example, the generated fused clothing image of worker B is resized to the model's required 224*224 pixels, converted to RGB format, and input into the clothing detection model. The clothing detection model analyzes the image and finds that worker B is wearing the required high-voltage insulating clothing, insulating shoes, hard hat, and insulating gloves, which are consistent with the required attire for high-voltage distribution rooms in the training data. The model outputs the clothing recognition result as "Attire meets requirements."

[0144] In a preferred embodiment, the clothing detection model is determined by:

[0145] Obtain the location of the operator;

[0146] Determine the power operation area where the operator is located based on the operator's location;

[0147] From the preset power operation large model library, the power operation large model corresponding to the power operation area where the operator is located is selected as the clothing detection large model.

[0148] In one embodiment of the present invention, due to the significant differences in protective clothing levels and marking requirements for different power operation environments, such as substations, transmission lines, and distribution stations, the present invention enables the clothing detection model to accurately match the environmental requirements of power operations. Specifically, the current location of the operator is obtained based on the positioning device worn by the operator, where the location includes latitude, longitude, and coordinate information.

[0149] Next, the operator's current location is matched with the pre-stored geographic boundary information of various power operation environment areas. For example, the coordinate ranges of various areas of the substation (such as high-voltage distribution rooms, transformer areas, etc.) and different sections along the transmission lines are stored. Through spatial position calculation and area judgment algorithms, the power operation environment area where the operator is currently located is determined.

[0150] After the power operation area where the operator is located is clarified, since the preset power operation big model library has trained and stored corresponding power operation big models for different areas such as substations, transmission lines, and distribution substations, the power operation big model corresponding to the power operation area where the operator is located is selected from the preset power operation big model library and used as the clothing detection big model in the current scenario. It will be used for subsequent detection of whether the operator's clothing is compliant, ensuring that the clothing detection can meet the safety operation requirements of the area.

[0151] It should be noted that the preset power operation large model library contains multiple power operation large models, and each power operation large model is trained in the following way:

[0152] For each power operation environment, we collected images of clothing that met and did not meet safety regulations for that area from multiple angles as training images. We then fused these multi-angle training images to generate fused sample clothing images.

[0153] Acquire a number of training samples; each training sample includes: a fused sample clothing image of the power operation environment area and its clothing label; wherein the clothing label is a result of manually labeling the fused sample clothing image, including whether the clothing meets the requirements or does not meet the requirements;

[0154] Several training samples are input into the large electric power operation model to be trained. The large electric power operation model takes the fused sample clothing images as input and the predicted clothing results as output for training. During the training process, the loss function is calculated based on the predicted clothing results and the corresponding clothing labels, and the network parameters of the large electric power operation model are adjusted according to the loss function until the loss function converges to obtain a trained large electric power operation model.

[0155] In this example, a large substation consists of three power operation environments: a high-voltage distribution room, a transformer area, and a low-voltage distribution cabinet area. Worker A, wearing a UWB positioning tag, enters the substation. After receiving location data from worker A's positioning device, the system compares the coordinates of each area in the database and finds that worker A's location is within the coordinate range of the high-voltage distribution room. This confirms that worker A is in the high-voltage distribution room, and then generates a corresponding large power operation model from the model library.

[0156] Step S3: If the clothing inspection result shows that the clothing meets the requirements, obtain point cloud data of the operator at several different collection angles at each collection time;

[0157] In step S3, when the clothing detection result shows that the clothing meets the requirements, point cloud data of the operator at several different collection angles at each collection moment collected by the visual device are obtained.

[0158] In a preferred embodiment, when the clothing detection result shows that the clothing does not meet the requirements, an audible and visual alarm is issued. In this embodiment, the audible and visual alarm uses high-frequency flashing and sharp beeps to warn the operating personnel and surrounding personnel, and at the same time, the location of the violator and the details of the missing clothing are marked in real time on the electronic signboard in the working area.

[0159] Step S4: performing trajectory analysis on the point cloud data of several different acquisition angles at each acquisition moment to obtain the limb movement trajectory;

[0160] In step S4, trajectory analysis is performed based on the point cloud data of multiple different acquisition angles at each acquisition moment to obtain the limb movement trajectory of the operator during the power operation, as specifically described in steps S41 to S44.

[0161] In a preferred embodiment, trajectory analysis is performed on point cloud data of several different acquisition angles at each acquisition moment to obtain the limb movement trajectory, including the following sub-steps:

[0162] Step S41: For each acquisition moment, among the point cloud data acquired at different acquisition angles, any point cloud data is used as a first target point cloud, and the other point cloud data except the first target point cloud is used as a second target point cloud; the acquisition angle of the first target point cloud is used as a third acquisition angle, and the acquisition angle of the second target point cloud is used as a fourth acquisition angle;

[0163] In step S41, at each acquisition moment, among the point cloud data collected at different acquisition angles, one point cloud data is randomly selected or selected according to a preset selection rule (such as the device number sequence) as the first target point cloud, and the remaining point cloud data is marked as the second target point cloud. At the same time, the acquisition angle of the acquisition device corresponding to the first target point cloud is recorded as the third acquisition angle, and the acquisition angle of the acquisition device corresponding to the second target point cloud is recorded as the fourth acquisition angle.

[0164] Continuing with the above example, in a substation high-voltage distribution room operating scenario, at a certain acquisition moment, three LiDARs collected three sets of point cloud data of operator C, recorded as point cloud X, point cloud Y, and point cloud Z. According to the device number sequence, point cloud X is selected as the first target point cloud, and point clouds Y and Z are the second target point clouds. The LiDAR acquisition angle corresponding to point cloud X is 25° horizontally to the left (the third acquisition angle), the LiDAR acquisition angle corresponding to point cloud Y is 35° horizontally to the right (the fourth acquisition angle), and the LiDAR acquisition angle corresponding to point cloud Z is 20° vertically downward (the fourth acquisition angle).

[0165] Step S42: calculating the angle difference between the third acquisition angle and the fourth acquisition angle to obtain a second angle difference;

[0166] In step S42, based on the third acquisition angle and the fourth acquisition angle, the angle difference between the third acquisition angle and the fourth acquisition angle is calculated to obtain a second angle difference. By calculating the second angle difference, the spatial rotation relationship between the two perspectives can be quantified.

[0167] Step S43: reading reflection intensity information from the first target point cloud as the first point cloud reflection intensity; reading reflection intensity information from the second target point cloud as the second point cloud reflection intensity;

[0168] In step S43, the point cloud processing algorithm extracts numerical information about the reflection intensity of each point or the point cloud as a whole from the first target point cloud as the first point cloud reflection intensity. Similarly, numerical information about the reflection intensity of each point or the point cloud as a whole is extracted from the second target point cloud as the second point cloud reflection intensity. This reflection intensity information reflects the object surface's ability to reflect the detection signal. Object surfaces of different materials and in different states often have different reflection intensities. For example, workers' helmets, work clothes, work equipment, and surrounding power facilities all exhibit different reflection intensities due to their own properties. By comparing the reflection intensities of the first point cloud and the second point cloud, it can assist in determining whether the items worn by the workers meet the safety requirements.

[0169] Step S44: performing point cloud alignment on the first target point cloud and the second target point cloud according to the second angle difference, the reflection intensity of the first point cloud, and the reflection intensity of the second point cloud to obtain a first aligned point cloud and a second aligned point cloud;

[0170] For step S44, the first target point cloud and the second target point cloud are aligned according to the second angle difference, the reflection intensity of the first point cloud and the reflection intensity of the second point cloud to obtain the first aligned point cloud and the second aligned point cloud, respectively, as described in steps S441 to S444.

[0171] In a preferred embodiment, performing point cloud alignment on the first target point cloud and the second target point cloud based on the second angle difference, the reflection intensity of the first point cloud and the reflection intensity of the second point cloud to obtain a first aligned point cloud and a second aligned point cloud includes the following sub-steps:

[0172] Step S441: determining a reflection intensity ratio according to the reflection intensity of the first point cloud and the reflection intensity of the second point cloud;

[0173] For step S441, the first point cloud reflection intensity I1 and the second point cloud reflection intensity I2 are obtained by accumulating the reflection intensity of each point. Calculate the reflection intensity ratio r. Continuing with the above embodiment, in the high-voltage distribution room of the substation, the first point cloud reflection intensity I1 of the first target point cloud (point cloud X) is 120, and the second point cloud reflection intensity I2 of the second target point cloud (point cloud Y) is 100. The reflection intensity ratio is calculated according to the formula

[0174] Step S442: determining a first reflection intensity gradient in a three-dimensional direction based on the first point cloud reflection intensity of each coordinate in the first target point cloud, and determining a second reflection intensity gradient in the three-dimensional direction based on the second point cloud reflection intensity of each coordinate in the second target point cloud;

[0175] In step S442, for each coordinate (x1, y1, z1) in the first target point cloud, the reflection intensity gradient is calculated in the x, y, and z directions respectively. Taking the x direction as an example, the central difference method is used, and the formula is Among them, I(x,y,z) is the reflection intensity of point (x,y,z), Δx is the small displacement (which can be set according to the point cloud density). Similarly, the gradient G in the y and z directions is calculated. y1 , G z1 , get the first reflection intensity gradient vector Furthermore, for each coordinate (x2, y2, z2) in the second target point cloud, the reflection intensity gradient in the x, y, and z directions is calculated using the same method to obtain the second reflection intensity gradient vector

[0176] Continuing with the above example, in point cloud X, for the point with coordinates (10, 10, 10), Δx = Δy = Δz = 1, calculate the reflection intensity gradient in the x direction (For example, I(11,10,10)=15, I(9,10,10)=13). Then G x1 =1; Similarly, calculate the reflection intensity gradient in the y and z directions, for example, 0.5 and 0.8, thus obtaining the first reflection intensity gradient vector Perform the same calculation on the corresponding coordinate point in the second target point cloud Y, for example, to obtain the second reflection intensity gradient vector

[0177] Step S443: transforming each coordinate in the first target point cloud according to the reflection intensity ratio and the first reflection intensity gradient to obtain a first transformed coordinate, and transforming each coordinate in the second target point cloud according to the reflection intensity ratio and the second reflection intensity gradient to obtain a second transformed coordinate;

[0178] In step S443, for each coordinate (x1, y1, z1) in the first target point cloud, according to the reflection intensity ratio r and the first reflection intensity gradient vector Perform coordinate transformation. The formula is:

[0179] x 1；new =x1+α1·r·G x1 .

[0180] y 1；new =y1+α1·r·G y1 .

[0181] z 1；new =z1+α1·r·G z1 .

[0182] Where α1 is the first preset scaling factor, and the first transformed coordinates are obtained through the above calculation:

[0183] (x 1；new ,y 1；new ,z 1；new ).

[0184] For each coordinate (x2, y2, z2) in the second target point cloud, according to the second reflection intensity gradient vector To perform coordinate transformation, the formula is:

[0185] x 2；new =x2+α2·G x2 .

[0186] y 2；new =y2+α2·G y2 .

[0187] z 2；new =z2+α2·G z2 .

[0188] Wherein, α2 is the second preset scaling factor, and the second transformed coordinates are obtained by the above calculation:

[0189] (x 2；neu ,y 2；neu ,z 2；new ).

[0190] Continuing with the above example, set α1 = 0.5, α2 = 0.3. For the point with coordinates (10, 10, 10) in point cloud X, calculate the coordinates after the first transformation:

[0191] x 1；neu =10+0.5×1.2×1=10.6.

[0192] y 1；new =10+0.5×1.2×0.5=10.3.

[0193] z 1；new=10+0.5×1.2×0.8=10.48.

[0194] For the corresponding coordinate point in point cloud Y, calculate the coordinate after the second transformation:

[0195] x 2；new =10+0.3×0.8=10.24.

[0196] y 2；new =10+0.3×0.6=10.18.

[0197] z 2；new =10+0.3×0.7=10.21.

[0198] Step S444: determining a rotation angle parameter of the point cloud in a three-dimensional direction according to the second angle difference;

[0199] In step S444, according to the second angle difference β, the rotation angle parameter θ of the point cloud in the x, y, and z directions is calculated using the trigonometric function relationship. x ,θ y ,θ z For example, the rotations of the point cloud around the three axes are independent of each other. θ z =β·cos(ω), where ω is the direction parameter set according to the actual acquisition equipment layout.

[0200] Step S445: performing angle rotation compensation on the first transformed coordinates and the second transformed coordinates according to the rotation angle parameter of the point cloud in the three-dimensional direction, to obtain a first aligned point cloud and a second aligned point cloud.

[0201] In step S445, for the first transformed coordinate (x 1；new ,y 1；ncw ,z 1；new ), the rotation matrix is:

[0202]

[0203] After matrix multiplication operation [x 1；final ,y 1；final ,z 1；final ,1] T =R1·[x 1；new ,y 1；new ,z 1；new ,1] T , get the coordinates of the first aligned point cloud.

[0204] The second transformed coordinate (x 2；new ,y 2；new ,z 2；new) Perform the same rotation operation to obtain the coordinates of the second aligned point cloud.

[0205] Continuing with the above embodiment, the second angle difference β = 15°, setting ω=30°. Calculate the rotation angle parameters:

[0206] θ x =15·cos(45°)·sin(30°)≈5.3°.

[0207] θ y =15·sin(45°)·sin(30°)≈5.3°.

[0208] θ z =15·cos(30°)≈13°.

[0209] For the transformed x coordinates of the point cloud (10.6, 10.3, 10.48), we construct the rotation matrix R1 and perform matrix operations to obtain the coordinates of the first aligned point cloud. We perform the same operation on the transformed y coordinates of the second target point cloud to obtain the coordinates of the second aligned point cloud.

[0210] The present invention uses the reflection intensity ratio and gradient to transform the coordinates, and can adjust the spatial structure according to the reflection characteristics of the point cloud, so that the point cloud is more matched at the feature level, and performs rotation compensation based on the angle difference, accurately correcting the spatial angle of the point cloud. Therefore, it comprehensively considers the reflection intensity characteristics and spatial angle information, effectively reducing the dislocation and deviation of the point cloud caused by differences in acquisition angle and reflection characteristics, providing an accurate data basis for subsequent point cloud fusion and posture estimation, and can more realistically restore the position and form of the operator's limbs in space, thereby improving the accuracy and reliability of behavior detection.

[0211] Step S45: performing point cloud fusion on the first aligned point cloud and the second aligned point cloud to obtain a target fused point cloud;

[0212] For step S45, the point cloud curvature of the first aligned point cloud and the second aligned point cloud is calculated, and the first aligned point cloud and the second aligned point cloud are fused according to the point cloud curvature to obtain the target fused point cloud, as described in steps S451 to S457.

[0213] In a preferred embodiment, performing point cloud fusion on the first aligned point cloud and the second aligned point cloud to obtain a target fused point cloud includes the following sub-steps:

[0214] Step S451: calculating the point cloud curvature of the first aligned point cloud and the second aligned point cloud;

[0215] For step S451, the point cloud curvature is calculated using the local surface fitting method. For each point p in the point cloud, a quadratic surface is fitted in its neighborhood by the least squares method. The curvature of the point is the rate of change of the normal vector of the fitted surface at that point. Let the neighborhood point set of point p be N p , fitting the quadratic surface ax by the least squares method 2 +bxy+cy 2 +dx+ey+f=0, calculate the point cloud curvature k of point p based on the fitting parameters.

[0216] Step S452: performing curvature feature grading on each point cloud in the first aligned point cloud and the second aligned point cloud according to the point cloud curvature, to obtain a first curvature feature grade of each point cloud in the first aligned point cloud and a second curvature feature grade of each point cloud in the second aligned point cloud;

[0217] For step S452, the point cloud curvature is graded according to a certain threshold range, and n curvature feature levels are set. For example, the curvature range [0, t1) is divided into level 1, [t1, t2) is divided into level 2, and so on. n；1 ,+∞) is divided into level n, and the first curvature feature level of each point cloud in the first aligned point cloud is obtained. For each point cloud in the second aligned point cloud, the above point cloud curvature classification operation is repeated to obtain the second curvature feature level of each point cloud in the second aligned point cloud.

[0218] Continuing with the above embodiment, in the operation scene of the high-voltage distribution room of the substation, the point cloud in the first aligned point cloud (point cloud X′) is processed. For a point p in the point cloud X′ X′ , select the 10 nearest neighbor points in its neighborhood, fit the quadratic surface by the least squares method, and calculate the curvature k pX′ = 0.05. Set 5 curvature feature levels, with thresholds of t1 = 0.02, t2 = 0.04, t3 = 0.06, and t4 = 0.08. Since 0.04 < 0.05 < 0.06, the point p X′ The first curvature feature level of is 3. Similarly, the point clouds in the second aligned point cloud (point cloud Y′) are calculated and graded to obtain the second curvature feature level of each point cloud.

[0219] Step S453: determining two point clouds having the same first curvature feature level and the same second curvature feature level as a curvature matching point cloud pair;

[0220] In step S453, each point cloud in the first aligned point cloud and its corresponding first curvature feature level are traversed, and each point cloud in the second aligned point cloud and its corresponding second curvature feature level are traversed. If it is determined that the first curvature feature level of a point cloud in the first aligned point cloud is the same as the second curvature feature level of a point cloud in the second aligned point cloud, the two point clouds are determined to be a set of curvature matching point cloud pairs.

[0221] Continuing with the above embodiment, when traversing the curvature feature levels of point cloud X′ and point cloud Y′, it is found that point p in point cloud X′ X′1 The first curvature feature level is 3, and the point p in the point cloud Y′ Y′1 The second curvature characteristic level is also 3, so point p X′1 and point p Y′1 The result is a set of curvature matching point cloud pairs. The process continues to traverse and find all other point cloud pairs with the same curvature feature level to form multiple sets of curvature matching point cloud pairs.

[0222] Step S454: the point clouds other than the curvature matching point cloud pairs in the first aligned point cloud are used as first non-matching point clouds, and the point clouds other than the curvature matching point cloud pairs in the second aligned point cloud are used as second non-matching point clouds;

[0223] In step S454, the points in the first aligned point cloud, excluding the curvature-matched point cloud pairs, are treated as the first non-matching point cloud, and the points in the second aligned point cloud, excluding the curvature-matched point cloud pairs, are treated as the second non-matching point cloud. These non-matching point clouds contain differentiated areas or areas that are not matched due to differences in viewing angles.

[0224] Step S455: fusing the curvature matching point cloud pair according to the curvature feature level of the curvature matching point cloud pair and the spatial distance between the two point clouds in the curvature matching point cloud pair to obtain an initial fused point cloud;

[0225] In step S455, for each set of curvature matching point cloud pairs, the spatial distance d between the two point clouds is calculated. Among them, (x1, y1, z1) and (x2, y2, z2) are the center coordinates of the two point clouds respectively.

[0226] The fusion strategy is determined based on the curvature feature level and spatial distance. For point cloud pairs with a low curvature feature level (indicating a relatively flat surface), if the spatial distance is small, the average value of the two point clouds is directly taken as the fused point cloud; for point cloud pairs with a high curvature feature level (indicating a relatively complex surface), a weighted fusion method is used, where the weight is determined by the inverse of the spatial distance, as shown in the formula: Among them, p1 and p2 are two point clouds, d1 and d2 are the distances between the two point clouds and the fusion center point. Through the above steps, all curvature matching point cloud pairs are fused to obtain the initial fused point cloud.

[0227] Match point cloud pairs with a set of curvatures (points p X′1 and point p Y′1 ) as an example, point p X′1 The center coordinates of are (10,10,10), point p Y′1 The center coordinates of are (10.2, 10.2, 10.2), calculate the spatial distance between them Since their curvature feature level is 3, which belongs to the medium curvature feature level, the weight-based fusion method is used. X′1 The distance to the fusion center is d1 = 0.2, point p Y′1 The distance to the fusion center is d2 = 0.15, then the fused point cloud After calculation, the coordinates of the fused point cloud are obtained, and similar operations are performed on all curvature matching point cloud pairs to obtain the initial fused point cloud.

[0228] Step S456: determining an interpolation ratio based on an average distance between the first non-matching point cloud and the initial fused point cloud, and an average distance between the second non-matching point cloud and the initial fused point cloud;

[0229] For step S456, the average distance D1 between the first non-matching point cloud and the initial fused point cloud is calculated. unmatch1 , calculate the Euclidean distance d from it to all point clouds in the initial fused point cloud i1 ,i=1,2,…,m, where m is the number of point clouds after initial fusion), and then the formula Calculate the average distance, where n1 is the number of points in the first non-matching point cloud. Similarly, calculate the average distance D2 between the second non-matching point cloud and the initial fused point cloud using the formula: Where n2 is the number of point clouds of the second non-matching point cloud, d i2 is the distance from each point cloud in the second non-matching point cloud to each point cloud in the initial fused point cloud.

[0230] Determine the interpolation scale based on the average distance

[0231] Continuing with the above example, there are 5 first non-matching point clouds in the first aligned point cloud (point cloud X′), and there are 3 second non-matching point clouds in the second aligned point cloud (point cloud Y′). The initial fused point cloud contains 10 point clouds. Calculate the distances from it to the 10 point clouds in the initial fusion point cloud as d 11,d 12 ,…,d 110 , then the average distance between the first non-matching point cloud and the initial fused point cloud is For example, D1 = 0.5. Similarly, calculate the average distance between the second non-matching point cloud and the initial fused point cloud For example, D2 = 0.3. Calculate the interpolation ratio based on D1 = 0.5, D2 = 0.3 and the formula

[0232] Step S457: inserting the first non-matching point cloud and the second non-matching point cloud into corresponding positions in the initial fused point cloud according to the interpolation ratio to obtain the target fused point cloud.

[0233] In step S457, the first non-matching point cloud and the second non-matching point cloud are interpolated according to the interpolation ratio λ. unmatch1 , find its corresponding insertion position in the initial fused point cloud (which can be determined according to the spatial position relationship), adjust its coordinates according to the interpolation ratio λ and then insert it. For each point cloud p in the second non-matching point cloud unmatch2, Adjust its coordinates according to the ratio 1-λ and insert them into the corresponding position in the initial fused point cloud. After completing the insertion operation of all non-matching point clouds, the target fused point cloud is obtained.

[0234] Continuing with the above embodiment, it is known that the interpolation ratio λ = 0.625, for the point in the first non-matching point cloud Its original coordinates are (11,11,11). After finding the corresponding insertion position in the initial fused point cloud, the adjusted coordinates are (11×0.625,11×0.625,11×0.625)=(6.875,6.875,6.875). Insert it into the corresponding position of the initial fused point cloud. The original coordinates are (12, 12, 12), and the adjusted coordinates are (12 × (1-0.625), 12 × (1-0.625), 12 × (1-0.625)) = (4.5, 4.5, 4.5). These coordinates are inserted into the corresponding positions of the initial fused point cloud. After inserting all non-matching point clouds, the target fused point cloud is obtained.

[0235] The present invention utilizes curvature feature grading and matching to reasonably combine and fuse point clouds with similar geometric characteristics, reducing the unnatural fusion problem caused by geometric differences in point clouds. The fusion strategy based on curvature feature level and spatial distance enables the fused point cloud to reasonably handle the spatial position relationship while maintaining geometric consistency. The interpolation ratio is determined by calculating the average distance between the non-matching point cloud and the initial fused point cloud, and the interpolation insertion operation is performed. It can completely integrate all point cloud data and fill spatial gaps, thereby fully considering the curvature characteristics and spatial distribution characteristics of the point cloud. The generated target fused point cloud can more accurately and completely describe the limb form of the operator, provide a high-quality data basis for subsequent posture estimation and limb motion trajectory analysis, and improve the accuracy and reliability of behavior detection.

[0236] Step S46: performing posture estimation based on the fused point cloud to obtain the operator's limb posture;

[0237] For step S46, a posture estimation algorithm based on point cloud features, such as the PFH point feature histogram algorithm, is used to extract the PFH features of the fused point cloud, which are matched with the features in the pre-trained posture template library to find the most matching posture template and determine the limb posture of the operator at the time of collection.

[0238] Step S47: Analyze the limb postures at the continuous acquisition moments to obtain the limb movement trajectory.

[0239] For step S47, the limb postures of the consecutive acquisition moments are arranged in chronological order, and a trajectory tracking algorithm based on Kalman filtering is used. t is the limb posture state vector at time t (including position, rotation angle and other information), z t For the observation value at time t (i.e. the limb posture obtained by posture estimation), establish the state equation x t =Ex t；1 +Fu t +w t and the observation equation z t =Hx t +v t , where E is the state transfer matrix, F is the control matrix, u t is the control input, w t and v t are process noise and observation noise respectively. By continuously updating the state vector x t , predict and correct limb postures and obtain limb motion trajectories.

[0240] In one embodiment, the limb postures of operator C are acquired at 10 consecutive acquisition moments and are sequentially input into the trajectory tracking algorithm based on the Kalman filter in chronological order. At the first acquisition moment, the initial state vector x1 is determined based on the initial posture estimate. At each subsequent moment, the state vector x is continuously updated through the state equation and the observation equation. t ,After calculation, the continuous limb motion trajectory of operator C at these 10 acquisition moments is obtained, such as the complete trajectory of the arm lifting from the initial position and moving to the ,device operation button to operate.

[0241] The present invention utilizes point cloud alignment combined with reflection intensity constraints to effectively improve the accuracy and stability of point cloud alignment. Based on point cloud curvature fusion, it can reasonably integrate point cloud data from different angles, fully present the limb form of the operator, and convert discrete posture information into continuous and reliable limb motion trajectory, thereby fully considering the various characteristics of point cloud data. In a complex power operation environment, it can more accurately capture the details of the operator's limb movement, provide precise motion trajectory data support for the detection of power operation behavior norms, and improve the accuracy and reliability of behavior detection.

[0242] Step S5: Input the limb movement trajectory into the behavior detection model, so that the behavior detection model generates the operator's behavior detection results based on the limb movement trajectory.

[0243] For step S5, the limb movement trajectory is formatted according to the model input requirements so that it can be recognized by the power operation large model, and the processed limb movement trajectory is input into the behavior detection large model. The behavior detection large model analyzes the time series information, movement pattern, etc. of the limb movement trajectory through deep learning models such as recurrent neural networks (RNN) and long short-term memory networks (LSTM), and compares them with the sample behavior characteristics in the training data. Based on the comparison results, the power operation environment detection results are output, that is, whether the behavior of the operator meets the standard requirements of the power operation environment area. If the power operation large model determines that the limb movement trajectory matches the trajectory characteristics of the standard behavior in the training data, the output behavior detection result is "standard behavior"; otherwise, the output behavior detection result is "non-standard behavior".

[0244] Continuing with the above example, the limb movement trajectory of operator B in the high-voltage distribution room is formatted and input into the corresponding behavior detection model. The behavior detection model analyzes and finds that when operating the high-voltage switch, B strictly follows the prescribed process and action specifications. The characteristics of his arm movement trajectory and body posture changes are consistent with the trajectory characteristics of the standard operation in the training data. The output behavior detection result is "behavior standard."

[0245] In a preferred embodiment, when the behavior detection result is that the behavior is irregular, an audible and visual alarm is issued. In this embodiment, the audible and visual alarm uses high-frequency flashing and sharp beeps to warn the operating personnel and surrounding personnel. If the irregular behavior continues, the distribution box in the dangerous area will automatically cut off the power.

[0246] The present invention matches the corresponding large-scale power operation model through different power operation environment areas. Since the large-scale power operation model is obtained based on samples that meet the requirements of different power operation environment areas, it can dynamically and deeply adapt to the detection needs in different operation environments to ensure that the judgment of clothing and behavioral norms meets the requirements of specific environments. In terms of clothing detection, the initial clothing image is collected from multiple angles by visual equipment and fused, which can obtain more comprehensive clothing detail information, and then combined with the large-scale power operation model to perform clothing recognition, thereby improving the accuracy and reliability of clothing detection. In the behavior detection link, trajectory analysis is performed based on multi-angle point cloud data to completely and accurately restore the limb movement trajectory of the operator, and then combined with the large-scale power operation model to perform trajectory analysis, thereby improving the accuracy and reliability of behavior detection. Therefore, the accuracy and reliability of power operation environment detection are improved, and the safety of the operator is guaranteed to the greatest extent.

[0247] like Figure 2 As shown, based on the above method embodiment, a corresponding device embodiment is provided;

[0248] An embodiment of the present invention provides a safety standard detection device for electric power workers, comprising: a clothing image acquisition module, a clothing detection module, a point cloud data acquisition module, a trajectory analysis module, and a behavior detection module;

[0249] A clothing image acquisition module is used to acquire initial clothing images of the operator at several different acquisition angles;

[0250] The clothing detection module is used to perform clothing detection based on the initial clothing images of the operator at several different acquisition angles to obtain the clothing detection results of the operator;

[0251] The point cloud data acquisition module is used to obtain point cloud data of the operator at several different acquisition angles at each acquisition moment when the clothing detection result shows that the clothing meets the requirements;

[0252] The trajectory analysis module is used to perform trajectory analysis on the point cloud data of several different acquisition angles at each acquisition moment to obtain the limb movement trajectory;

[0253] The behavior detection module is used to input the limb movement trajectory into the behavior detection model so that the behavior detection model can generate the operator's behavior detection results based on the limb movement trajectory.

[0254] It can be understood that the above-mentioned device embodiment corresponds to the method embodiment of the present invention, which can implement the safety standard detection method for power workers provided by any of the above-mentioned method embodiments of the present invention.

[0255] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Furthermore, in the drawings of the device embodiments provided by the present invention, the connection relationship between modules indicates that they have a communication connection, which may be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement the present invention without inventive effort.

[0256] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for detecting safety standards of electric power workers, characterized in that: include: Obtaining initial clothing images of the operator at several different acquisition angles; Perform clothing detection based on the initial clothing images of the operator at several different acquisition angles to obtain the clothing detection results of the operator; When the clothing inspection result shows that the clothing meets the requirements, point cloud data of the operator at several different collection angles at each collection moment are obtained; Perform trajectory analysis on point cloud data from several different acquisition angles at each acquisition moment to obtain the limb movement trajectory; The limb movement trajectory is input into the behavior detection model, so that the behavior detection model generates the operator's behavior detection result according to the limb movement trajectory.

2. The method for detecting safety standards of electric power workers according to claim 1, characterized in that: Based on the initial clothing images of the operator at several different acquisition angles, clothing detection is performed to obtain the operator's clothing detection results, including: Perform image fusion on the initial clothing images of the operator at several different acquisition angles to obtain a fused clothing image; The fused clothing image is input into a trained clothing detection model, so that the clothing detection model generates a clothing detection result of the operator based on the fused clothing image.

3. The method for detecting safety standards of electric power workers according to claim 2, characterized in that: The clothing detection model is determined by the following method: Obtain the location of the operator; Determine the power operation area where the operator is located based on the operator's location; From a preset power operation large model library, the power operation large model corresponding to the power operation area where the operator is located is selected as the clothing detection large model.

4. The method for detecting safety standards of electric power workers according to claim 2, wherein: The initial clothing images of the operator at several different acquisition angles are fused to obtain a fused clothing image, including: Taking any initial clothing image as the first target image, and taking all initial clothing images other than the first target image as the second target image; taking the acquisition angle of the first target image as the first acquisition angle, and taking the acquisition angle of the second target image as the second acquisition angle; Calculating an angle difference between the first acquisition angle and the second acquisition angle to obtain a first angle difference; determining a mapping relationship between a first target image and a second target image according to the first angle difference, a first image acquisition range at the first acquisition angle, and a second image acquisition range at the second acquisition angle; determining, according to the mapping relationship, overlapping image regions and non-overlapping image regions between the first target image and the second target image; The first target image and the second target image are fused according to the overlapping image area and the non-overlapping image area between the first target image and the second target image to obtain the fused clothing image.

5. The method for detecting safety standards of electric power workers according to claim 4, characterized in that: The non-overlapping image areas include: a first non-overlapping image area located in the first target image and a second non-overlapping image area located in the second target image; The fusing the first target image and the second target image according to the overlapping image area and the non-overlapping image area between the first target image and the second target image to obtain the fused clothing image includes: Clustering each pixel point in the first non-overlapping image area and sorting them according to a preset sorting rule to obtain a plurality of first cluster categories; For each first cluster category, calculating the distance between the first cluster category and a central pixel point in the overlapping image region in the first target image to obtain a first pixel distance; calculating the distance between the first cluster category and the nearest image edge in the overlapping image region in the first target image to obtain a first edge distance; Clustering each pixel in the second non-overlapping image area and sorting them according to a preset sorting rule to obtain a plurality of second cluster categories; For each second cluster category, calculating the distance between the second cluster category and a central pixel point in the overlapping image area in the second target image to obtain a second pixel distance; calculating the distance between the second cluster category and the nearest image edge in the overlapping image area in the second target image to obtain a second edge distance; For each first cluster category, adjusting the pixel value of the first cluster category according to the first pixel distance and the first edge distance of the first cluster category, and the second pixel distance and the second edge distance of the second cluster category with the same ranking as the first cluster category, to obtain an adjusted pixel value of the first cluster category; Fusing the adjusted pixel values of the first cluster category with the pixel values of the second cluster category to obtain a fused non-overlapping image area; splicing the overlapping image area of the first target image and the overlapping image area of the second target image to obtain a spliced overlapping image area; The spliced overlapping image area and the fused non-overlapping image area are combined to obtain the fused clothing image.

6. The method for detecting safety standards of electric power workers according to claim 5, characterized in that: Determining a mapping relationship between a first target image and a second target image according to the first angle difference, a first image acquisition range at the first acquisition angle, and a second image acquisition range at the second acquisition angle includes: determining an angle ratio relationship according to the first image acquisition range and the second image acquisition range; determining a correlation coefficient according to the first angle difference and the angle ratio relationship; Dividing the first image acquisition range into a plurality of first angle intervals, and determining a second angle interval corresponding to the second target image in each first angle interval according to a correlation coefficient; For each first angle interval, determining a pixel offset between the first target image and the second target image according to the first angle interval and the corresponding second angle interval; A mapping relationship between the first target image and the second target image is determined according to the pixel offsets corresponding to all first angle intervals.

7. The method for detecting safety standards of electric power workers according to claim 1, characterized in that: Perform trajectory analysis on point cloud data from several different acquisition angles at each acquisition moment to obtain the limb movement trajectory, including: At each acquisition moment, among the point cloud data acquired at several different acquisition angles, any point cloud data is used as the first target point cloud, and the other point cloud data except the first target point cloud is used as the second target point cloud; the acquisition angle of the first target point cloud is used as the third acquisition angle, and the acquisition angle of the second target point cloud is used as the fourth acquisition angle; Calculating an angle difference between the third acquisition angle and the fourth acquisition angle to obtain a second angle difference; Reading the reflection intensity information from the first target point cloud as the first point cloud reflection intensity; reading the reflection intensity information from the second target point cloud as the second point cloud reflection intensity; performing point cloud alignment on the first target point cloud and the second target point cloud according to the second angle difference, the first point cloud reflection intensity, and the second point cloud reflection intensity to obtain a first aligned point cloud and a second aligned point cloud; Perform point cloud fusion on the first aligned point cloud and the second aligned point cloud to obtain the target fused point cloud; Performing posture estimation based on the fused point cloud to obtain the operator's limb posture; The limb postures at the continuous acquisition moments are analyzed to obtain the limb movement trajectory.

8. The method for detecting safety standards of electric power workers according to claim 7, characterized in that: Performing point cloud alignment on the first target point cloud and the second target point cloud according to the second angle difference, the first point cloud reflection intensity, and the second point cloud reflection intensity to obtain a first aligned point cloud and a second aligned point cloud, including: determining a reflection intensity ratio according to the reflection intensity of the first point cloud and the reflection intensity of the second point cloud; Determine a first reflection intensity gradient in a three-dimensional direction based on the first point cloud reflection intensity of each coordinate in the first target point cloud, and determine a second reflection intensity gradient in the three-dimensional direction based on the second point cloud reflection intensity of each coordinate in the second target point cloud; transforming each coordinate in the first target point cloud according to the reflection intensity ratio and the first reflection intensity gradient to obtain a first transformed coordinate, and transforming each coordinate in the second target point cloud according to the reflection intensity ratio and the second reflection intensity gradient to obtain a second transformed coordinate; Determining a rotation angle parameter of the point cloud in a three-dimensional direction according to the second angle difference; According to the rotation angle parameter of the point cloud in the three-dimensional direction, angle rotation compensation is performed on the first transformed coordinates and the second transformed coordinates to obtain a first aligned point cloud and a second aligned point cloud.

9. The method for detecting safety standards of electric power workers according to claim 7, characterized in that: Performing point cloud fusion on the first aligned point cloud and the second aligned point cloud to obtain the target fused point cloud, including: Calculate the point cloud curvature of the first aligned point cloud and the second aligned point cloud; Performing curvature feature grading on each point cloud in the first aligned point cloud and the second aligned point cloud according to the curvature of the point clouds, to obtain a first curvature feature grade of each point cloud in the first aligned point cloud and a second curvature feature grade of each point cloud in the second aligned point cloud; Determine two point clouds with the same first curvature feature level and the same second curvature feature level as a curvature matching point cloud pair; The point clouds other than the curvature matching point cloud pair in the first aligned point cloud are used as first non-matching point clouds, and the point clouds other than the curvature matching point cloud pair in the second aligned point cloud are used as second non-matching point clouds; According to the curvature feature level of the curvature matching point cloud pair and the spatial distance between the two point clouds in the curvature matching point cloud pair, the curvature matching point cloud pair is fused to obtain the initial fused point cloud; Determining an interpolation ratio according to an average distance between the first non-matching point cloud and the initial fused point cloud, and an average distance between the second non-matching point cloud and the initial fused point cloud; According to the interpolation ratio, the first non-matching point cloud and the second non-matching point cloud are inserted into the corresponding positions in the initial fused point cloud to obtain the target fused point cloud.

10. A safety standard detection device for electric power workers, characterized in that: include: Clothing image acquisition module, clothing detection module, point cloud data acquisition module, trajectory analysis module and behavior detection module; The clothing image acquisition module is used to acquire initial clothing images of the operator at several different acquisition angles; The clothing detection module is used to perform clothing detection based on the initial clothing images of the operator at several different acquisition angles to obtain the clothing detection results of the operator; The point cloud data acquisition module is used to acquire point cloud data of the operator at several different acquisition angles at each acquisition moment when the clothing detection result shows that the clothing meets the requirements; The trajectory analysis module is used to perform trajectory analysis on the point cloud data of several different acquisition angles at each acquisition moment to obtain the limb movement trajectory; The behavior detection module is used to input the limb movement trajectory into the behavior detection model, so that the behavior detection model generates the operator's behavior detection results based on the limb movement trajectory.

Images