Non-inductive receiving management method and system for safety tools of power supply station

Abstract

The invention relates to a non-inductive receiving management method and system for safety tools of a power supply station, and the method comprises the steps: shooting an image sequence when a receiving person enters and leaves a receiving region in the receiving region, then extracting an image matched with a posture feature, and extracting a plurality of regions which may carry the safety tools from the image; then, through comparing the image features of the plurality of areas during entering and leaving, whether each area has the possibility of carrying a safety tool or not is quickly judged, if the image features are matched, the safety tool is not carried, and if the image features are not matched, interference information removal is carried out on the image area during leaving based on the image features during entering; and keeping the distinguishing image information, and inputting the distinguishing image information into the recognition model to obtain a receiving recognition result. Through computer vision, pure radio frequency identification information can be verified and supplemented, and the situation that the radio frequency identification tag cannot be automatically updated due to loss or damage is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of inventory management, and in particular to a method and system for seamlessly managing the use of safety tools in a power supply station. Background Art

[0002] Safety tools at power stations are tools and equipment used to protect the personal safety of power workers during the operation, maintenance, and repair of electrical equipment, as well as to ensure the safe and stable operation of the power grid. These tools typically have protective, insulating, and detection functions, and primarily include insulating tools, testers, grounding wires, safety protective equipment, and signage.

[0003] Existing safety equipment management relies on paper ledgers and management personnel, requiring manual verification for both collection and return, a cumbersome process. Consequently, many warehouses are beginning to implement seamless collection using technologies like identity recognition and radio frequency identification, automatically updating ledgers without the need for manual registration. However, for safety equipment used outdoors at power stations, radio frequency tags are prone to falling off or becoming damaged, rendering the automatic update mechanism ineffective. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method and system for seamless management of safety tools in power supply stations to solve the above technical problems.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for managing the seamless use of safety tools in a power supply station according to the present invention comprises the following steps:

[0007] Obtain a first image sequence of the person picking up materials when entering the material picking location and a second image sequence of the person leaving the material picking location; and obtain radio frequency identification information of tools in the tool cabinet at multiple time points;

[0008] Screening out the first image and the second image whose posture features match from the first image sequence and the second image sequence, and extracting dynamic contour features and human body key points from the first image and the second image;

[0009] Extracting an initial assessment area from the first image based on the human body key points, and extracting a receiving assessment area from the second image based on the human body key points, wherein the initial assessment area and the receiving assessment area include a head area, a hand area, a waist area, and a foot area;

[0010] Extracting image features of the pickup assessment area and the initial assessment area respectively, matching the image features of the pickup assessment area with the image features of the initial assessment area, and when the image features of the pickup assessment area and the image features of the initial assessment area do not match, removing interference information of the pickup assessment area based on the initial assessment area to obtain an input image; inputting the input image into a pre-built recognition model to obtain a pickup recognition result;

[0011] The radio frequency identification information of the tool at the corresponding time point is verified or supplemented based on the receipt and identification result.

[0012] In one embodiment of the present application, selecting the first image and the second image having matching posture features from the first image sequence and the second image sequence includes:

[0013] Extracting human body key points from a plurality of images in the first image sequence and human body key points from a plurality of images in the second image sequence, respectively, wherein the human body key points include reference key points;

[0014] Connecting the key points of the human body to obtain human body posture features;

[0015] Calculating a minimum bounding rectangle of a human body posture feature, selecting any one image in the first image sequence and any one image in the second image sequence as a matching combination, aligning the images in the matching combination based on reference key points, and scaling the images so that diagonals of the minimum bounding rectangles of the images in the matching combination overlap;

[0016] Calculate the distance S(P n ,P n ′), where P n ,P n ′ are the nth human key points of the two images in the matching combination;

[0017] Calculate all distances S(P i ,P i ′), and the images in the matching combination that satisfy: the average value is less than the preset threshold and the average value is the smallest are used as the first image and the second image for posture feature matching.

[0018] In one embodiment of the present application, extracting an initial evaluation area from the first image based on the human body key points, and extracting a collection evaluation area from the second image based on the human body key points, include:

[0019] Extracting foreground areas of the first image and the second image, wherein the human body key points are located in the foreground areas;

[0020] The center of the pre-constructed extraction frame is overlapped with the target key point, and an initial evaluation area is extracted from the first image based on the overlapped extraction frame, or an evaluation area is extracted from the second image based on the overlapped extraction frame, wherein the target key point is one of the head key point, the hand key point, the waist key point and the foot key point.

[0021] In one embodiment of the present application, extracting image features of the receiving assessment area and the initial assessment area respectively includes:

[0022] Performing high-pass filtering and grayscale processing on the acceptance evaluation area and the initial evaluation area respectively to obtain a first grayscale image Gray1 and a second grayscale image Gray2;

[0023] Brightness normalization is performed on the first grayscale image Gray1 and the second grayscale image Gray2 to obtain a normalized first grayscale image nor_Gray1 and a normalized second grayscale image nor_Gray2, respectively. The mathematical expression of the normalization process is:

[0024]

[0025] Where Normalized(i,j) is the gray value of pixel (i,j) after normalization, gray(i,j) is the gray value of pixel (i,j), gray min is the minimum grayscale value of the first grayscale image or the second grayscale image, gray max is the maximum grayscale value of the first grayscale image or the second grayscale image;

[0026] Extract the normalized first grayscale image nor_Gray1 and the normalized first contour quantity OL1, the first average coordinate O1 of the contour pixel point, the first average grayscale gray1 and the first grayscale variance var1, and extract the second contour quantity OL2, the second average coordinate O2 of the contour pixel point, the second average grayscale gray2 and the second grayscale variance var2 of the second grayscale image nor_Gray2.

[0027] In one embodiment of the present application, matching the image features of the receipt assessment area with the image features of the initial assessment area includes:

[0028] Calculate the difference OL between the first contour number OL1 and the second contour number OL2 D ; Calculate the distance S between the first average coordinate O1 and the second average coordinate O2 O1,O2 ; Calculate the overall grayscale deviation rate gray of the first average grayscale gray1 and the second average grayscale gray2de ; and calculate the uniformity deviation rate var of the first grayscale variance var1 and the second grayscale variance var2 de ;

[0029] The difference OL D Compare the distance S with the preset tolerance range. O1,O2 Compare the total grayscale deviation rate gray with the preset distance tolerance range. de Compare with the preset overall grayscale deviation tolerance range, and convert the uniformity deviation rate var de Compare with the preset uniformity deviation tolerance range; and D Falling into the preset difference tolerance range, the distance S O1,O2 Falling into the preset distance tolerance range, the uniformity deviation rate var de Falling into the preset uniformity deviation tolerance range and the uniformity deviation rate var de When the uniformity deviation falls within a preset tolerance range, it is determined that the image features of the receiving evaluation area match the image features of the initial evaluation area; otherwise, it is determined that the image features of the receiving evaluation area do not match the image features of the initial evaluation area.

[0030] In one embodiment of the present application, removing interference information from the receiving evaluation area based on the initial evaluation area to obtain an input image includes:

[0031] Extracting contour features of the foreground portion of the initial evaluation area, and performing area screening and morphological operations on the contour features to obtain a target closed contour bin whose area is greater than a set value;

[0032] Extract the center point bin within the target closed contour bin o , the grayscale average value of all pixels in the target closed contour bin A and gray value standard deviation gray σ , construct the grayscale value reference range range within the target closed contour bin gray =(gray A -n×gray σ , gray A +n×gray σ ), and based on the center point bin of multiple target closed contours o And the gray value reference range range gray Constructing interference reference information of the initial assessment area, wherein n is a scale parameter;

[0033] Scan each pixel point in the evaluation area and find the pixel points that meet the distance from the center point bin o The distance is less than the preset reference radius, and the grayscale value falls into the grayscale value reference range range gray The pixel points are marked as interference information;

[0034] The grayscale values ​​of the background portion and interference information in the initial evaluation area are replaced with target values ​​to obtain an input image.

[0035] In one embodiment of the present application, the process of constructing the recognition model includes:

[0036] Obtain sample images of safety tools in power supply stations;

[0037] Extracting contour features from the sample image, and performing data enhancement and annotation on the contour features to obtain a training data set;

[0038] The artificial neural network is trained based on the training data set in combination with the gradient descent method to obtain a recognition model.

[0039] In one embodiment of the present application, verifying or supplementing the radio frequency identification information of the tool at the corresponding time point based on the receipt identification result includes:

[0040] Obtaining the change information of the radio frequency identification information of the tool at the time corresponding to the identification result;

[0041] When the receipt identification result is consistent with the change information, the change information is verified; when the change information is empty, the receipt record is supplemented based on the receipt identification result.

[0042] In one embodiment of the present application, it further includes:

[0043] When the collection record is supplemented based on the collection identification result, the supplementary information is sent to the target object.

[0044] This application also provides a non-sensing management system for the use of safety tools in power supply stations, including:

[0045] An acquisition module is used to acquire a first image sequence of the person picking up materials when entering the material picking location and a second image sequence of the person leaving the material picking location; and to acquire radio frequency identification information of tools in the tool cabinet at multiple time points;

[0046] a screening module, configured to screen out the first image and the second image having matching posture features from the first image sequence and the second image sequence, and extract dynamic contour features and human body key points from the first image and the second image;

[0047] a region extraction module, configured to extract an initial assessment region from the first image based on the human body key points, and extract a pickup assessment region from the first image based on the human body key points, wherein the initial assessment region and the pickup assessment region include a head region, a hand region, a waist region, and a foot region;

[0048] a feature matching and recognition module, configured to extract image features of the pickup assessment area and the initial assessment area, respectively, and match the image features of the pickup assessment area with the image features of the initial assessment area; if the image features of the pickup assessment area and the initial assessment area do not match, remove interference information from the pickup assessment area based on the initial assessment area to obtain an input image; and input the input image into a pre-built recognition model to obtain a pickup recognition result;

[0049] The management module is used to verify or supplement the radio frequency identification information of the tool at the corresponding time point based on the receipt and identification result.

[0050] The beneficial effects of the present invention are as follows: the present invention provides a method and system for seamless collection and management of safety tools in a power supply station, which captures a sequence of images of the collection personnel when they enter and leave the area, and then extracts images with matching posture features, and extracts multiple areas that may carry safety tools from the images. Then, by comparing the image features of multiple areas at the time of entry and exit, it is quickly determined whether there is a possibility of carrying safety tools in each area. If the image features match, it means that no safety tools are carried. If the image features do not match, the interference information of the image area at the time of exit is removed based on the image features at the time of entry, so as to retain the distinguishing image information, and input it into the recognition model to obtain the collection and recognition result. Through computer vision, simple RFID information can be verified and supplemented to avoid the situation where the RFID tag cannot be automatically updated due to loss or damage. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0052] Figure 1 This is a diagram of an application scenario of a method for seamlessly managing the use of safety tools in a power supply station in one embodiment of the present application;

[0053] Figure 2 This is a flow chart of a method for seamlessly managing the use of safety tools in a power supply station, as shown in one embodiment of the present application;

[0054] Figure 3 is a schematic diagram of human body posture features in one embodiment of the present application;

[0055] Figure 4A schematic diagram of image alignment and scaling in one embodiment of the present application;

[0056] Figure 5 Schematic diagram of extraction of the evaluation area in one embodiment of the present application;

[0057] Figure 6 This is a structural diagram of a non-sensing management system for the use of safety tools in power supply stations, as shown in one embodiment of the present application;

[0058] Figure 7 A schematic diagram of the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0059] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0060] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention. Therefore, the drawings only show the layers related to the present invention and are not drawn according to the number, shape and size ratio of the layers in actual implementation. In actual implementation, the type and number of each layer can be changed at will, and the layer layout may also be more complicated.

[0061] In the following description, numerous details are set forth to provide a more thorough explanation of the embodiments of the present invention; however, it is apparent to one skilled in the art that the embodiments of the present invention may be practiced without these specific details.

[0062] Figure 1 This is an application scenario diagram of a method for managing the non-sensing use of safety tools in a power supply station in one embodiment of the present application, such as Figure 1 As shown, in this application, a first camera 110 facing the entrance direction and a second camera 120 facing the exit direction are set at the door of the storage place of safety tools. The first camera 110 is used to shoot an image sequence (video) of the person picking up the materials when entering the storage place, and the second camera 120 is used to shoot an image sequence (video) of the person picking up the materials when leaving the storage place. The first camera 110 and the second camera 120 are connected to the analysis terminal 130, which is used to analyze and process the images and identify the items picked up by the person picking up the materials when leaving based on machine vision.

[0063] Furthermore, an RFID base station is installed within the storage area to determine whether safety tools have been removed from the tool cabinet by reading the RSSI (Reference Signal Strength) of the RFID tags on the tools. This information is automatically transmitted to the analysis terminal 130 and compared with the items retrieved through machine vision analysis, enabling seamless management of redundantly verified safety tool collection.

[0064] Figure 2 This is a flow chart of a method for managing the use of safety tools in a power supply station according to an embodiment of the present application. Figure 2 As shown in the figure: a method for managing the use of safety tools of a power supply station in this embodiment may include steps S210 to S250:

[0065] S210, obtaining a first image sequence of the material collector when entering the material collection location and a second image sequence of the material collector when leaving the material collection location; and obtaining radio frequency identification information of tools in the tool cabinet at multiple time points;

[0066] The first image sequence and the second image sequence are both collected by a surveillance camera, and a video segment with dynamic features is extracted by a dynamic frame difference method, and an image sequence is obtained by frame extraction.

[0067] In addition, there may be multiple first image sequences and second image sequences of recipients, so it is necessary to call the existing face recognition model to perform face recognition on the first image sequence and second image sequence of each recipient, and combine the first image sequence and the second image sequence through the face recognition results to wait for subsequent image processing and matching.

[0068] S220, screening out first and second images with matching posture features from the first and second image sequences, and extracting dynamic contour features and human body key points from the first and second images;

[0069] Since it is necessary to compare the differences between people entering and leaving an area, in order to reduce interference information in the image, this application extracts posture features and then uses images with similar posture features as comparison images to reduce information interference caused by different features. The specific process includes:

[0070] S221, extracting human body key points from a plurality of images in the first image sequence and human body key points from a plurality of images in the second image sequence, respectively, wherein the human body key points include reference key points;

[0071] Human keypoints (also known as human pose estimation or human skeleton) refer to identifying and locating the main joints and important parts of the human body in images or videos. These keypoints typically include: keypoints of the head / face, such as the eyes, nose, and ears; keypoints of the torso, such as the neck, shoulders, and waist; and keypoints of the limbs, such as the elbows, wrists, knees, and ankles.

[0072] In this application, an open source human feature point estimation model, such as OPENPOSE, is used to extract human key points in image sequences.

[0073] S222, connecting the key points of the human body to obtain human body posture features;

[0074] Figure 3 This is a schematic diagram of the human body posture features in one embodiment of the present application. The human body posture features obtained by connecting the lines are as follows: Figure 3 shown.

[0075] S223: Calculate a minimum bounding rectangle of a human body posture feature, select any one image in the first image sequence and any one image in the second image sequence as a matching combination, align the images in the matching combination based on reference key points, and scale them so that diagonals of the minimum bounding rectangles of the images in the matching combination overlap;

[0076] Figure 4 This is a schematic diagram of image alignment and scaling in one embodiment of the present application. Figure 4 As shown, in order to make the human body proportions of the first image and the second image consistent, the present application aligns the key points of the neck and scales them so that the diagonals of the minimum circumscribed rectangles coincide, thereby aligning the human body proportions.

[0077] S224, calculate the distance S (P) of each corresponding human key point in the matching combination. n ,P n ′), where P n ,P n ′ are the nth human key points of the two images in the matching combination;

[0078] After alignment and scaling, the distance between each key point of the human body can be calculated one by one, and the distance can be used to represent the similarity between posture features.

[0079] S225, calculate all distances S(P i ,P i ′), and the images in the matching combination that satisfy: the average value is less than the preset threshold and the average value is the smallest are used as the first image and the second image for posture feature matching.

[0080] If the human body posture features are similar, then all distances S(P i ,P i Therefore, the matching combination with an average value less than the preset threshold is used as the target combination, and then the combination with the smallest average value is selected from the target combination.

[0081] If there is no matching combination with an average value less than the preset threshold, the subsequent process will be abandoned and only RFID radio frequency identification information will be used to manage the collection information of tools.

[0082] S230, extracting an initial assessment area from the first image based on the human body key points, and extracting a receiving assessment area from the second image based on the human body key points, wherein the initial assessment area and the receiving assessment area include a head area, a hand area, a waist area, and a foot area;

[0083] The evaluation area is the area where people wear or carry work tools, such as holding tools in their hands, wearing a safety helmet on their heads, wearing gloves on their hands, wearing work boots on their feet, wearing a tool storage belt on their waists, etc. Therefore, the evaluation area in this application includes the head area, hand area, waist area and foot area.

[0084] Specifically, extracting an initial evaluation area from the first image based on the human body key points, and extracting a receiving evaluation area from the second image based on the human body key points, comprising:

[0085] S231, extracting foreground areas of the first image and the second image, wherein the human body key points are located in the foreground areas;

[0086] In this application, background modeling technology (such as frame difference method, mixed Gaussian model, etc.) is used to distinguish the foreground and background.

[0087] S232, overlapping the center of the pre-constructed extraction frame with the target key point, and extracting the initial evaluation area from the first image based on the overlapped extraction frame, or extracting the evaluation area from the second image based on the overlapped extraction frame, wherein the target key point is one of the head key point, the hand key point, the waist key point and the foot key point.

[0088] First, select a keypoint as a reference point. For example, if you're interested in hand movements, you might choose the wrist or palm keypoint as the target keypoint. Then, construct a bounding box of appropriate size for the application scenario. The box should be large enough to cover the feature area of ​​interest, but not so large that it includes too much irrelevant information. The center of the bounding box is precisely placed on the selected keypoint. This means that the position of the bounding box will dynamically adjust as the keypoint's position changes.

[0089] Finally, based on the aligned extraction box coordinates, the corresponding sub-image is cropped from the original image as the initial or extraction evaluation area. Ensure that the cropping operation takes into account the boundary conditions. That is, if the extraction box partially exceeds the image boundary, take appropriate measures (such as filling or extracting only the valid part within the box).

[0090] Figure 5 This is a schematic diagram of extracting the evaluation area in one embodiment of the present application. The evaluation area extracted based on the extraction frame is as follows: Figure 5 shown.

[0091] S240, extracting image features of the pickup assessment area and the initial assessment area respectively, and matching the image features of the pickup assessment area with the image features of the initial assessment area. If the image features of the pickup assessment area do not match the image features of the initial assessment area, removing interference information from the pickup assessment area based on the initial assessment area to obtain an input image; inputting the input image into a pre-built recognition model to obtain a pickup recognition result;

[0092] After quickly extracting the assessment area, each collection assessment area must be quickly evaluated to see if it has changed significantly compared to the initial assessment area. If no significant changes have occurred, the subsequent recognition process will not proceed. If significant changes have occurred, the image information within the initial assessment area can be used to remove interference from the collection assessment area before inputting it into the recognition model. This can increase recognition speed, reduce recognition computational complexity, and improve recognition accuracy.

[0093] The process of extracting the image features of the receiving assessment area and the initial assessment area respectively includes:

[0094] S2401, performing high-pass filtering and grayscale processing on the acceptance evaluation area and the initial evaluation area respectively to obtain a first grayscale image Gray1 and a second grayscale image Gray2;

[0095] High-pass filters are primarily used to enhance edge information in images and suppress low-frequency components (such as smooth backgrounds), thereby highlighting details and changes. Applying high-pass filtering to the acceptance and initial evaluation areas can emphasize local changes and facilitate subsequent feature extraction. A common method for implementing high-pass filtering is to use edge detection techniques such as the Laplace operator or the Sobel operator. Another method is to convert the image to the frequency domain via Fourier transform, design and apply a high-pass filter in the frequency domain, and then perform an inverse transform back to the spatial domain.

[0096] The process of converting a color image to a grayscale image usually uses a weighted average method to calculate the grayscale value of each pixel. The formula is as follows:

[0097] Gray=0.299R+0.587G+0.114B

[0098] Where R, G, and B represent the intensity values ​​of the red, green, and blue channels, respectively. After the above processing, a first grayscale image and a second grayscale image are obtained.

[0099] S2402: Normalize the brightness of the first grayscale image Gray1 and the second grayscale image Gray2 to obtain a normalized first grayscale image nor_Gray1 and a normalized second grayscale image nor_Gray2. The mathematical expression of the normalization process is:

[0100]

[0101] Where Normalized(i,j) is the gray value of pixel (i,j) after normalization, gray(i,j) is the gray value of pixel (i,j), gray min is the minimum grayscale value of the first grayscale image or the second grayscale image, gray max is the maximum grayscale value of the first grayscale image or the second grayscale image;

[0102] Brightness normalization is used to adjust the contrast of images so that the grayscale value ranges between different images are consistent, which facilitates comparison and analysis.

[0103] S2403, extract the normalized first grayscale image nor_Gray1 and the normalized first contour number OL1, the first average coordinate O1 of the contour pixel points, the first average grayscale gray1 and the first grayscale variance var1, and extract the second contour number OL2, the second average coordinate O2 of the contour pixel points, the second average grayscale gray2 and the second grayscale variance var2 of the second grayscale image nor_Gray2.

[0104] After brightness normalization, a series of statistical features are extracted from the normalized grayscale image, including:

[0105] (1) Contour Count: Use edge detection algorithms (such as Canny edge detection) to find contours in the image and count the number of contours. The number of contours can reflect the number or complexity of objects in the image.

[0106] (2) Average coordinates of contour pixels: For each contour, calculate the average value of the coordinates of all its pixels as the center position of the contour.

[0107] (3) Average grayscale: Calculate the average grayscale value of all pixels in the image to reflect the overall brightness level.

[0108] (4) Grayscale variance: It measures the degree of dispersion of the grayscale value distribution in the image. It is calculated as the average of the squares of the differences between the grayscale values ​​of each pixel and the average grayscale.

[0109] For the normalized first and second grayscale images, the aforementioned statistical features are extracted: the number of first contours, the first average coordinates of contour pixels, the first average grayscale, and the first grayscale variance; and the number of second contours, the second average coordinates of contour pixels, the second average grayscale, and the second grayscale variance. These features can be used for further image comparison to quickly determine whether there are significant differences between the assessment area and the initial assessment area.

[0110] Matching the image features of the pickup assessment area with the image features of the initial assessment area includes:

[0111] S2411, calculating the difference OL between the first contour number OL1 and the second contour number OL2 D ;

[0112] OL D =|OL1-OL2|

[0113] Calculate the distance S between the first average coordinate O1 and the second average coordinate O2 O1,O2 ;

[0114]

[0115] Wherein, (i1, j1) is the coordinate value of the first average coordinate O1, and (i2, j2) is the coordinate value of the second average coordinate O2.

[0116] Calculate the overall grayscale deviation rate gray of the first average grayscale gray1 and the second average grayscale gray2 de ;

[0117]

[0118] And calculate the uniformity deviation rate var of the first grayscale variance var1 and the second grayscale variance var2 de ;

[0119]

[0120] S2412, the difference OL D Compare the distance S with the preset tolerance range. O1,O2 Compare the total grayscale deviation rate gray with the preset distance tolerance range. de Compare with the preset overall grayscale deviation tolerance range, and convert the uniformity deviation rate var deCompare with the preset uniformity deviation tolerance range; and D Falling into the preset difference tolerance range, the distance S O1,O2 Falling into the preset distance tolerance range, the uniformity deviation rate var de Falling into the preset uniformity deviation tolerance range and the uniformity deviation rate var de When the uniformity deviation falls within a preset tolerance range, it is determined that the image features of the receiving evaluation area match the image features of the initial evaluation area; otherwise, it is determined that the image features of the receiving evaluation area do not match the image features of the initial evaluation area.

[0121] If all of the above conditions are met—that is, the difference, distance, overall grayscale deviation rate, and uniformity deviation rate all fall within their respective tolerances—then the image features of the incoming evaluation area are considered to match those of the initial evaluation area. If any of these conditions are not met, the two are considered mismatched. This method provides a fast and accurate means of quantifying and comparing the similarities or differences between two image regions.

[0122] If the above process determines that the image features of the collection assessment area and the initial assessment area do not match, it means that there may be safety tools in the collection assessment area that the person did not bring with him when entering the area, and therefore the subsequent AI recognition process is entered.

[0123] Before inputting the cropped image of the assessment area into the AI ​​model, this application uses the image with posture feature matching to remove interference information in order to improve recognition accuracy, and tries to retain only the image features of the safety tools, specifically including:

[0124] S2421, extracting contour features of the foreground portion of the initial evaluation area, and performing area screening and morphological operations on the contour features to obtain target closed contour bins whose areas are larger than a set value;

[0125] First, you need to extract the foreground from the initial evaluation area (usually obtained by background subtraction or segmentation algorithm). Then use edge detection algorithms (such as Canny) or contour finding algorithms (such as findContours function in OpenCV) to identify contours in the image.

[0126] All extracted contours are filtered by area, retaining only those with an area greater than a set threshold. This step can remove small, noisy contours. Applying morphological operations (such as dilation and erosion) can help clean up contour boundaries, fill small holes, or separate stuck objects. The final result is a closed contour of the target with an area greater than the set threshold.

[0127] S2422, extract the center point bin within the target closed contour bin o , the grayscale average value of all pixels in the target closed contour bin A and gray value standard deviation gray σ , construct the grayscale value reference range range within the target closed contour bin gray =(gray A -n×gray σ , gray A +n×gray σ ), and based on the center point bin of multiple target closed contours o And the gray value reference range range gray Constructing interference reference information of the initial assessment area, wherein n is a scale parameter;

[0128] Based on the center point positions of multiple closed target contours and their corresponding grayscale value reference ranges, a set of interference reference information can be constructed. This information will be used in subsequent steps to identify potential interference in the assessment area.

[0129] Specifically, the value of the scale parameter n is generally 3, that is, a grayscale value reference range that satisfies 3 times the standard deviation is constructed.

[0130] S2423, scan each pixel point in the evaluation area and find the pixel point that meets the distance from the center point bin o The distance is less than the preset reference radius, and the grayscale value falls into the grayscale value reference range range gray The pixel points are marked as interference information;

[0131] Scan each pixel in the assessment area. For each pixel, check whether it meets two conditions: (1) the distance from the center point of a target closed contour is less than the preset reference radius; (2) the grayscale value falls within the grayscale value reference range of the target closed contour. If both conditions are met, the pixel is marked as interference information.

[0132] S2424: Replace the grayscale values ​​of the background portion and interference information in the initial evaluation area with target values ​​to obtain an input image.

[0133] Finally, by replacing the grayscale values ​​of all pixels corresponding to the background and interference information to 255, we can obtain an input image that only retains the suspected information of the tool.

[0134] After receiving the input image, the image's contour features are extracted and fed into the recognition model to produce a recognition result. If the recognition result indicates a safety tool with a probability of at least 70%, it can be used for subsequent information verification and supplementation. Otherwise, the recognition fails and the image is sent to a human for manual recognition and confirmation.

[0135] The recognition model that the above process relies on can be constructed through the following process:

[0136] (1) Obtain sample images of safety tools in power supply stations;

[0137] First, we need to collect a large number of images of power station safety tools as a training basis. These images should cover as many possible situations as possible, including but not limited to different viewing angles, lighting conditions, and background environments. To ensure the generalization ability of the model, we should ensure that the dataset contains a variety of safety tools and has a sufficient number of samples for each tool.

[0138] (2) extracting contour features from the sample image, and performing data enhancement and annotation on the contour features to obtain a training data set;

[0139] To increase the diversity and size of the training dataset, new training samples can be generated by performing a series of transformations on the original images. Common data augmentation techniques include rotation, scaling, translation, cropping, flipping, color adjustment, and adding noise. Each processed image is then accurately labeled. This means specifying the specific type of safety tool present in the image. High-quality annotation is crucial for training an efficient recognition model.

[0140] (3) Based on the training data set and combined with the gradient descent method, the artificial neural network is trained to obtain a recognition model.

[0141] Select the appropriate artificial neural network architecture based on the task requirements. For image classification tasks, convolutional neural networks (CNNs) are one of the most effective choices. The training process includes: (3-1) Randomly initialize the network weights. (3-2) Pass the input image through the network to calculate the output result. (3-3) Compare the difference between the network output and the true label. The cross-entropy loss function is usually used to quantify this difference. (3-4) Using the gradient descent method and its variants (such as the Adam optimizer), the network weights are adjusted according to the calculated loss to minimize the loss function. (3-5) Repeat the above steps until a certain stopping condition is met, such as reaching a predetermined number of iterations or the loss value converges. Finally, an identification model that can accurately identify safety tools in power supply stations is obtained.

[0142] S250: Verify or supplement the radio frequency identification information of the tool at the corresponding time point based on the receipt and identification result.

[0143] The specific process of verification and supplementation includes:

[0144] S251, obtaining change information of the radio frequency identification information of the tool at the time point corresponding to the identification result;

[0145] For example, if the record at 10:30 shows a sign being removed, and the computer vision recognition result also shows that the sign was removed, then the verification passes. If the verification fails, the manual verification mechanism is triggered and the monitoring information is sent to the management staff for verification.

[0146] S252, when the receipt identification result is consistent with the change information, complete the verification of the change information; when the change information is empty, supplement the receipt record based on the receipt identification result.

[0147] In addition, in some scenarios where RFID tags fall off or are damaged, the RFID information will not be automatically updated, which will also lead to inconsistencies between the RFID management information and the identification information. In this case, the manual verification mechanism will be triggered and the monitoring information will be sent to the management personnel for verification.

[0148] In addition, when the receipt record is supplemented based on the receipt identification result, the supplementary information is also sent to the administrator for backup. If there is any identification error in the subsequent process, it can be corrected through the backup material.

[0149] The present invention provides a method for seamlessly managing the use of safety tools in power supply stations. The method captures a sequence of images of the users entering and leaving the area, extracts images with matching posture features, and extracts multiple areas that may carry safety tools from the images. By comparing the image features of multiple areas at the time of entry and exit, it is quickly determined whether there is a possibility of carrying safety tools in each area. If the image features match, it means that no safety tools are carried. If the image features do not match, the interference information of the image area at the time of exit is removed based on the image features at the time of entry to retain the distinguishing image information, and input it into the recognition model to obtain the recognition result. Through computer vision, simple radio frequency identification information can be verified and supplemented to avoid the situation where the radio frequency identification tag cannot be automatically updated due to loss or damage.

[0150] like Figure 6 As shown, the present application also provides a non-sensing management system for the use of safety tools in power supply stations, including:

[0151] An acquisition module is used to acquire a first image sequence of the person picking up materials when entering the material picking location and a second image sequence of the person leaving the material picking location; and to acquire radio frequency identification information of tools in the tool cabinet at multiple time points;

[0152] a screening module, configured to screen out the first image and the second image having matching posture features from the first image sequence and the second image sequence, and extract dynamic contour features and human body key points from the first image and the second image;

[0153] a region extraction module, configured to extract an initial assessment region from the first image based on the human body key points, and extract a pickup assessment region from the first image based on the human body key points, wherein the initial assessment region and the pickup assessment region include a head region, a hand region, a waist region, and a foot region;

[0154] a feature matching and recognition module, configured to extract image features of the pickup assessment area and the initial assessment area, respectively, and match the image features of the pickup assessment area with the image features of the initial assessment area; if the image features of the pickup assessment area and the initial assessment area do not match, remove interference information from the pickup assessment area based on the initial assessment area to obtain an input image; and input the input image into a pre-built recognition model to obtain a pickup recognition result;

[0155] The management module is used to verify or supplement the radio frequency identification information of the tool at the corresponding time point based on the receipt and identification result.

[0156] The present invention provides a non-sensing collection and management system for safety tools of power supply stations. The system captures a sequence of images of the users when they enter and leave the collection area, extracts images with matching posture features, and extracts multiple areas that may carry safety tools from the images. By comparing the image features of multiple areas at the time of entry and exit, it is quickly determined whether there is a possibility of carrying safety tools in each area. If the image features match, it means that no safety tools are carried. If the image features do not match, the interference information of the image area at the time of exit is removed based on the image features at the time of entry to retain the distinguishing image information, and input it into the recognition model to obtain the collection and recognition result. Through computer vision, simple radio frequency identification information can be verified and supplemented to avoid the situation where the radio frequency identification tag cannot be automatically updated due to loss or damage.

[0157] Figure 7 The following is a schematic diagram showing the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application. Figure 7 The computer system of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0158] like Figure 7 As shown, the computer system includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 702 or the program loaded from the storage part 708 into the random access memory (RAM) 703, such as executing the method in the above embodiment. Various programs and data required for system operation are also stored in the RAM 703. The CPU 701, ROM 702 and RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0159] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, and the like; an output section 707 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. Removable media 711, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 710 as needed, so that computer programs read therefrom can be installed into the storage section 708 as needed.

[0160] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from a removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the various functions defined in the system of the present application are executed.

[0161] It should be noted that the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable computer program. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0162] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. Each box in the flowchart or block diagram can represent a module, program segment or part of the code, and the above-mentioned module, program segment or part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two boxes shown in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart and the combination of boxes in the block diagram or flowchart can be implemented using a dedicated hardware-based system that performs the specified functions or operations, or can be implemented using a combination of dedicated hardware and computer instructions.

[0163] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. In some cases, the names of these units do not constitute limitations on the units themselves.

[0164] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a computer processor, the computer executes the aforementioned method. The computer-readable storage medium may be included in the electronic device described in the above embodiments, or may exist independently and not be incorporated into the electronic device.

[0165] Another aspect of the present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the above embodiments.

[0166] The above embodiments are only preferred embodiments for fully illustrating the present application, and the protection scope of the present application is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art based on the present application are within the protection scope of the present application.

Claims

1. A method for managing the use of safety tools in power supply stations, characterized in that: Including steps: Obtain a first image sequence of the person picking up materials when entering the material picking location and a second image sequence of the person leaving the material picking location; and obtain radio frequency identification information of tools in the tool cabinet at multiple time points; Screening out the first image and the second image whose posture features match from the first image sequence and the second image sequence, and extracting dynamic contour features and human body key points from the first image and the second image; Extracting an initial assessment area from the first image based on the human body key points, and extracting a receiving assessment area from the second image based on the human body key points, wherein the initial assessment area and the receiving assessment area include a head area, a hand area, a waist area, and a foot area; Extracting image features of the pickup assessment area and the initial assessment area respectively, matching the image features of the pickup assessment area with the image features of the initial assessment area, and when the image features of the pickup assessment area and the image features of the initial assessment area do not match, removing interference information of the pickup assessment area based on the initial assessment area to obtain an input image; inputting the input image into a pre-built recognition model to obtain a pickup recognition result; The radio frequency identification information of the tool at the corresponding time point is verified or supplemented based on the receipt and identification result.

2. The method for managing the use of safety tools in power supply stations according to claim 1, characterized in that: Filtering out the first image and the second image having matching posture features from the first image sequence and the second image sequence includes: Extracting human body key points from a plurality of images in the first image sequence and human body key points from a plurality of images in the second image sequence, respectively, wherein the human body key points include reference key points; Connecting the key points of the human body to obtain human body posture features; Calculating a minimum bounding rectangle of a human body posture feature, selecting any one image in the first image sequence and any one image in the second image sequence as a matching combination, aligning the images in the matching combination based on reference key points, and scaling the images so that diagonals of the minimum bounding rectangles of the images in the matching combination overlap; Calculate the distance S(P n ,P n ′), where P n ,P n ′ are the nth human key points of the two images in the matching combination; Calculate all distances S(P i ,P i ′), and the images in the matching combination that satisfy: the average value is less than the preset threshold and the average value is the smallest are used as the first image and the second image for posture feature matching.

3. The method for managing the use of safety tools in power supply stations according to claim 1, characterized in that: Extracting an initial evaluation area from the first image based on the human body key points, and extracting a collection evaluation area from the second image based on the human body key points, comprising: Extracting foreground areas of the first image and the second image, wherein the human body key points are located in the foreground areas; The center of the pre-constructed extraction frame is overlapped with the target key point, and an initial evaluation area is extracted from the first image based on the overlapped extraction frame, or an evaluation area is extracted from the second image based on the overlapped extraction frame, wherein the target key point is one of the head key point, the hand key point, the waist key point and the foot key point.

4. The method for managing the use of safety tools in power supply stations according to claim 1, characterized in that: Extracting image features of the collection assessment area and the initial assessment area respectively includes: Performing high-pass filtering and grayscale processing on the acceptance evaluation area and the initial evaluation area respectively to obtain a first grayscale image Gray1 and a second grayscale image Gray2; Brightness normalization is performed on the first grayscale image Gray1 and the second grayscale image Gray2 to obtain a normalized first grayscale image nor_Gray1 and a normalized second grayscale image nor_Gray2, respectively. The mathematical expression of the normalization process is: Where Normalized(i,j) is the gray value of pixel (i,j) after normalization, gray(i,j) is the gray value of pixel (i,j), gray min is the minimum grayscale value of the first grayscale image or the second grayscale image, gray max is the maximum grayscale value of the first grayscale image or the second grayscale image; Extract the normalized first grayscale image nor_Gray1 and the normalized first contour quantity OL1, the first average coordinate O1 of the contour pixel point, the first average grayscale gray1 and the first grayscale variance var1, and extract the second contour quantity OL2, the second average coordinate O2 of the contour pixel point, the second average grayscale gray2 and the second grayscale variance var2 of the second grayscale image nor_Gray2.

5. The method for managing the use of safety tools in power supply stations according to claim 4 is characterized in that: Matching the image features of the pickup assessment area with the image features of the initial assessment area includes: Calculate the difference OL between the first contour number OL1 and the second contour number OL2 D ; Calculate the distance S between the first average coordinate O1 and the second average coordinate O2 O1,O2 ; Calculate the overall grayscale deviation rate gray of the first average grayscale gray1 and the second average grayscale gray2 de ; and calculate the uniformity deviation rate var of the first grayscale variance var1 and the second grayscale variance var2 de ; The difference OL D Compare the distance S with the preset tolerance range. O1,O2 Compare the total grayscale deviation rate gray with the preset distance tolerance range. de Compare with the preset overall grayscale deviation tolerance range, and convert the uniformity deviation rate var de Compare with the preset uniformity deviation tolerance range; and D Falling into the preset difference tolerance range, the distance S O1,O2 Falling into the preset distance tolerance range, the uniformity deviation rate var de Falling into the preset uniformity deviation tolerance range and the uniformity deviation rate var de When the uniformity deviation falls within a preset tolerance range, it is determined that the image features of the receiving evaluation area match the image features of the initial evaluation area; otherwise, it is determined that the image features of the receiving evaluation area do not match the image features of the initial evaluation area.

6. The method for managing the use of safety tools in power supply stations according to claim 1, characterized in that: Removing interference information from the receiving evaluation area based on the initial evaluation area to obtain an input image includes: Extracting contour features of the foreground portion of the initial evaluation area, and performing area screening and morphological operations on the contour features to obtain a target closed contour bin whose area is greater than a set value; Extract the center point bin within the target closed contour bin o , the grayscale average value of all pixels in the target closed contour bin A and gray value standard deviation gray σ , construct the grayscale value reference range range within the target closed contour bin gray =(gray A -n×gray σ , gray A +n×gray σ ), and based on the center point bin of multiple target closed contours o And the gray value reference range range gray Constructing interference reference information of the initial assessment area, wherein n is a scale parameter; Scan each pixel point in the evaluation area and find the pixel points that meet the distance from the center point bin o The distance is less than the preset reference radius, and the grayscale value falls into the grayscale value reference range range gray The pixel points are marked as interference information; The grayscale values ​​of the background portion and interference information in the initial evaluation area are replaced with target values ​​to obtain an input image.

7. The method for managing the use of safety tools in power supply stations according to claim 1, characterized in that: The process of constructing the recognition model includes: Obtain sample images of safety tools in power supply stations; Extracting contour features from the sample image, and performing data enhancement and annotation on the contour features to obtain a training data set; The artificial neural network is trained based on the training data set in combination with the gradient descent method to obtain a recognition model.

8. The method for managing the use of safety tools in power supply stations according to claim 1, characterized in that: Verifying or supplementing the radio frequency identification information of the tool at the corresponding time point based on the receipt and identification result includes: Obtaining the change information of the radio frequency identification information of the tool at the time corresponding to the identification result; When the receipt identification result is consistent with the change information, the change information is verified; when the change information is empty, the receipt record is supplemented based on the receipt identification result.

9. A method for managing the use of safety tools in power supply stations according to claim 8, characterized in that: Also includes: When the collection record is supplemented based on the collection identification result, the supplementary information is sent to the target object.

10. A non-sensing management system for the use of safety tools in power supply stations, characterized in that: include: An acquisition module is used to acquire a first image sequence of the material recipient when entering the material receiving location and a second image sequence of the material recipient when leaving the material receiving location; and obtain radio frequency identification information of tools in the tool cabinet at multiple points in time; a screening module, configured to screen out the first image and the second image having matching posture features from the first image sequence and the second image sequence, and extract dynamic contour features and human body key points from the first image and the second image; a region extraction module, configured to extract an initial assessment region from the first image based on the human body key points, and extract a pickup assessment region from the first image based on the human body key points, wherein the initial assessment region and the pickup assessment region include a head region, a hand region, a waist region, and a foot region; a feature matching and recognition module, configured to extract image features of the pickup assessment area and the initial assessment area, respectively, and match the image features of the pickup assessment area with the image features of the initial assessment area; if the image features of the pickup assessment area and the initial assessment area do not match, remove interference information from the pickup assessment area based on the initial assessment area to obtain an input image; and input the input image into a pre-built recognition model to obtain a pickup recognition result; The management module is used to verify or supplement the radio frequency identification information of the tool at the corresponding time point based on the receipt and identification result.

Images