Substation human body weight identification method and device, medium and equipment

Abstract

The invention provides a transformer substation human body weight identification method and device, a medium and equipment, and relates to the technical field of transformer substation human body identification. The method comprises the steps of firstly obtaining a to-be-recognized character image, performing feature extraction on the to-be-recognized character image by using a human body weight recognition model to obtain an appearance feature map and a joint feature map, and secondly generating a to-be-recognized character short video by using a plurality of frame images adjacent to the to-be-recognized character image, performing time feature extraction on the to-be-recognized character short video by using the human body weight recognition model to obtain a character feature vector containing a time sequence feature; performing feature fusion processing on the appearance feature map, the joint feature map and the character feature vector based on a human body weight recognition model to generate a global feature vector and a local feature vector; and finally, acquiring a figure image library, and matching the global feature vector and the local feature vector with the figure image library based on a human body weight recognition model to obtain a target figure. According to the embodiment, the accuracy of human body weight identification is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of human body recognition in a substation, and in particular to a method, device, medium and equipment for human body recognition in a substation. Background Art

[0002] In industrial environments such as substations, ensuring the safety of workers is crucial, so an effective monitoring system is needed to identify and manage the postures and movements of workers. However, current monitoring technology faces some challenges, especially in complex environments or crowded places, where people are easily blocked, resulting in the monitoring system being unable to accurately identify the movements and postures of workers, which may cause false alarms or missed alarms. In addition, the high similarity in workers' clothing and movements further affects the accuracy of the system in security monitoring and tracking and locating personnel.

[0003] To solve the above problems, human weight recognition technology has been introduced in related technologies to improve the accuracy and reliability of monitoring systems. Human weight recognition technology matches workers through unobstructed perspectives to restore their complete posture information. However, human weight recognition technology faces multiple challenges, including dealing with differences in human recognition under different perspectives and postures, solving the difficulty of feature extraction caused by occlusion problems, and improving the distinction among highly similar groups of workers. Summary of the invention

[0004] The disclosed embodiments at least provide a method, device, medium and equipment for identifying human weight in a substation, which adds extraction of human features including time series features on the basis of extracting human image features, and uses the two features to match human beings in the total human image library, thereby improving the accuracy of human weight identification.

[0005] The present disclosure provides a method for identifying human weight in a substation, including:

[0006] Acquire an image of a person to be identified, and extract features of the image of the person to be identified using a trained human weight recognition model to obtain an appearance feature map and a joint feature map, wherein the image of the person to be identified is any image of a plurality of images of a person detected in a video clip of a substation collected from at least two viewing angles;

[0007] Using a plurality of frame images adjacent to the image of the person to be identified, generating a short video of the person to be identified corresponding to the image of the person to be identified, and using the trained human weight recognition model to extract time features from the short video of the person to be identified to obtain a person feature vector containing time series features;

[0008] Based on the trained human weight recognition model, feature fusion processing is performed on the appearance feature map, the joint feature map and the character feature vector to generate a global feature vector and a local feature vector;

[0009] Acquire a person image library, match the global feature vector and the local feature vector with the person image library based on the trained person weight recognition model, and obtain the person image to be identified and the corresponding target person in the person image library, wherein the person image library is constructed based on video clips of the substation captured from at least two perspectives.

[0010] The present disclosure provides a device for identifying human weight in a substation, comprising:

[0011] An image acquisition module is used to acquire an image of a person to be identified, and to extract features of the image of the person to be identified using a trained human weight recognition model to obtain an appearance feature map and a joint feature map, wherein the image of the person to be identified is any image of a plurality of images of a person detected in a video clip of a substation collected from at least two viewing angles;

[0012] A feature extraction module, used to generate a short video of the person to be identified corresponding to the image of the person to be identified by using a plurality of frame images adjacent to the image of the person to be identified, and to extract time features from the short video of the person to be identified by using the trained human weight recognition model to obtain a person feature vector containing time series features;

[0013] A feature processing module, used for performing feature fusion processing on the appearance feature map, the joint feature map and the character feature vector based on the trained human weight recognition model to generate a global feature vector and a local feature vector;

[0014] A character matching module is used to obtain a character image library, and based on the trained human weight recognition model, the global feature vector and the local feature vector are matched with the character image library to obtain the image of the person to be identified and the corresponding target person in the character image library, wherein the character image library is constructed based on video clips of the substation captured from at least two perspectives.

[0015] An embodiment of the present disclosure provides a computer device, including: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the method for identifying human weight in a substation as described in any possible implementation manner described above is performed.

[0016] An embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for identifying human weight in a substation as described in any possible implementation manner described above is implemented.

[0017] The method, device, medium and equipment for identifying human weight in a substation provided in the embodiments of the present disclosure use a trained human weight identification model to extract the appearance features and joint features of the image of the person to be identified, and on this basis, extract the time features of the short video corresponding to the image of the person to be identified, obtain the person features containing the time series features, and then fuse the two features, and match the fused results with the person images of other perspectives in the person image library to accurately identify the target person. In this way, the accuracy of human weight identification is significantly improved, and it is suitable for environments such as substations where the similarity of staff is high and efficient and safe monitoring is required.

[0018] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required to be cited in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A flow chart of a method for identifying human weight in a substation provided by an embodiment of the present disclosure is shown;

[0021] Figure 2 A flow chart of a time feature extraction method in a method for identifying human weight in a substation provided by an embodiment of the present disclosure is shown;

[0022] Figure 3 A flow chart of an image feature matching method in a method for identifying human weight in a substation provided by an embodiment of the present disclosure is shown;

[0023] Figure 4 A flow chart showing a method for training a human weight recognition model in a method for recognizing human weight in a substation provided by an embodiment of the present disclosure;

[0024] Figure 5A schematic diagram showing a training method of a human weight recognition model in a human weight recognition method for a substation provided by an embodiment of the present disclosure is shown;

[0025] Figure 6 A schematic diagram of the structure of a human weight recognition device for a substation provided by an embodiment of the present disclosure is shown;

[0026] Figure 7 A schematic diagram showing the structure of another device for identifying human weight in a substation provided by an embodiment of the present disclosure is shown;

[0027] Figure 8 A schematic diagram of the structure of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0029] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0030] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0031] In industrial environments such as substations, ensuring the safety of workers is crucial, so an effective monitoring system is needed to identify and manage the postures and movements of workers. However, current monitoring technology faces some challenges, especially in complex environments or crowded places, where people are easily blocked, resulting in the monitoring system being unable to accurately identify the movements and postures of workers, which may cause false alarms or missed alarms. In addition, the high similarity of workers' clothing and movements further affects the accuracy of the system in distinguishing different individuals.

[0032] In order to solve these problems, human body weight recognition technology is introduced in related technologies to improve the accuracy and reliability of the monitoring system. Human body weight recognition technology matches the staff by utilizing the unobstructed perspective to restore their complete posture information.

[0033] However, research has found that this technology also faces many technical challenges. First, when dealing with differences in human recognition under different perspectives and postures, it is necessary to ensure that the system can accurately identify and track the dynamics of workers at various angles; second, it is necessary to solve the difficulty of feature extraction caused by occlusion, which requires the system to be able to effectively analyze some visible human features and information about the surrounding environment; finally, it is necessary to improve the differentiation among highly similar groups of workers, ensuring that the system can accurately identify the differences between different individuals and avoid misidentification.

[0034] Based on the above research, a method, device, medium and equipment for human weight recognition in a substation are provided in the embodiments of the present disclosure: first, an image of a person to be recognized is obtained, and a human weight recognition model is used to perform feature extraction on the image of the person to be recognized to obtain an appearance feature map and a joint feature map; secondly, a plurality of frame images adjacent to the image of the person to be recognized are used to generate a short video of the person to be recognized, and a human weight recognition model is used to perform time feature extraction on the short video of the person to be recognized to obtain a person feature vector containing time series features; then, based on the human weight recognition model, feature fusion processing is performed on the appearance feature map, the joint feature map and the person feature vector to generate a global feature vector and a local feature vector; finally, a person image library is obtained, and the global feature vector and the local feature vector are matched with the person image library based on the human weight recognition model to obtain the target person.

[0035] The method, device, medium and equipment for identifying human weight in a substation provided in the embodiments of the present disclosure use a trained human weight identification model to extract the appearance features and joint features of the image of the person to be identified, and on this basis, extract the time features of the short video corresponding to the image of the person to be identified, obtain the person features containing the time series features, and then fuse the two features, and match the fused results with the person images of other perspectives in the person image library to accurately identify the target person. In this way, the accuracy of human weight identification is significantly improved, and it is suitable for environments such as substations where the similarity of staff is high and efficient and safe monitoring is required.

[0036] To facilitate understanding of this embodiment, the execution subject of the substation human body weight recognition method provided by the embodiment of the present disclosure is first introduced in detail. The execution subject of the substation human body weight recognition method provided by the embodiment of the present disclosure is a computer device. The computer device can be a server. Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, big data and artificial intelligence platforms.

[0037] The following is a detailed description of the method for identifying human weight in a substation provided by the embodiment of the present application in conjunction with the accompanying drawings. Figure 1 FIG. 1 is a flow chart of a method for identifying human weight in a substation provided by an embodiment of the present disclosure, and the method includes the following S101 to S105:

[0038] S101, obtaining an image of a person to be identified, and using a trained body weight recognition model to perform feature extraction on the image of the person to be identified to obtain an appearance feature map and a joint feature map.

[0039] Here, in the substation working scene, it is necessary to monitor the behavior of the staff to prevent illegal actions. However, if only one monitoring device is set up in the substation, the posture of the obscured person may not be well identified due to the limitation of the viewing angle. Therefore, at least two monitoring devices are usually set up in the substation working scene to capture the behavior of the staff, and the perspective of one of the monitoring devices is selected as the main perspective. If the target is obscured in the main perspective, the target is found from the auxiliary perspective (the perspective corresponding to other monitoring devices) and the complete object is framed, so as to perform the next step of posture estimation. Therefore, the person image to be identified in the present disclosure is any person image among the multiple person images detected in the video clip of the substation collected from at least two perspectives.

[0040] Specifically, before obtaining the image of the person to be identified, the following (a) to (b) may be included:

[0041] (a) obtaining video clips captured from the at least two viewing angles in the substation during the same time period, performing person image detection, person capture and person image cropping on the video clips based on a video target detection algorithm, and obtaining the plurality of person images;

[0042] (b) respectively labeling the image information of the plurality of character images; the image information includes a character number and a viewpoint number.

[0043] Among them, the video target detection algorithm can include the YOLO series algorithm, the SSD target detection algorithm or the traditional computer vision method, etc. The purpose is to use the video target detection algorithm to detect people images from video clips and intercept and crop the detected people images. Then, the captured multiple people images are labeled, and the person number and view number corresponding to each person image are marked in the image information of the corresponding person image.

[0044] In the disclosed embodiment, the substation is provided with two monitoring devices, one of which has a higher frame rate and clarity, and is selected as the main perspective, and the other monitoring device is selected as the auxiliary perspective. A video stream of about five minutes is obtained from each of the two perspectives, the videos of the two perspectives are aligned on the timeline, and pictures are captured at a frame rate of 15fps. The yolov8 target detection algorithm is used to frame all the characters in the video and crop them into pictures respectively, and then they are labeled, and the naming rule of each picture is set to "character number_c perspective number_f.jpg", and pictures with poor detection effects are manually and roughly screened out. In addition, in order to ensure the consistency of the movements of the staff in the two perspectives, the character image of one of the perspectives can be flipped horizontally.

[0045] In some other embodiments, when annotating a person image, the frame number corresponding to the person image may also be annotated in the image information of the corresponding person image.

[0046] It can be understood that after obtaining the image of the person to be identified, the trained human weight recognition model is used to extract features from the image of the person to be identified, and an appearance feature map and a joint feature map are obtained. Specifically, for the image of the person to be identified, the CNN model and the key point model (High-Resolution Network, HRNet) are used to extract features, and their appearance feature maps and joint feature maps are obtained respectively. Among them, the human weight recognition model includes a time feature extraction sub-model, a similarity sorting sub-model and a feature fusion module; the time feature extraction sub-model includes a feature encoding module, a feature merging module, a feature matching module and a feature aggregation module, which are used to store typical and recurring time patterns, and are used to extract time features from the short video of the person to be identified, and by calculating the corresponding time memory set, the time attention value is obtained, and the character representation feature map is output, so that it contains temporal feature information. The similarity sorting sub-model includes an adaptive directional graph convolution module, a cross-graph embedding alignment module and a similarity calculation module, which are used to calculate and sort the similarity between images to obtain the target person who matches the image of the person to be identified in other perspectives. The training method of the human weight recognition model will be described in detail later.

[0047] S102, using multiple frame images adjacent to the image of the person to be identified, generating a short video of the person to be identified corresponding to the image of the person to be identified, and using the trained person weight recognition model to extract time features from the short video of the person to be identified to obtain a person feature vector containing time series features.

[0048] Specifically, in the substation work scene, the uniforms of the staff are very similar in style and color. It is impossible to accurately distinguish different individuals simply by relying on clothing or color features. This feature makes recognition based on visual features more complex and challenging. In addition, the movements and postures of each staff member are also very similar. Whether standing, squatting or running, these common movements show high consistency between different individuals. If the analysis is not combined with time series information, only relying on skeleton posture information, objects with similar movements at different times will often be mistakenly matched together, resulting in errors in the target re-identification process. Therefore, when processing and analyzing the problem of human re-identification in substations, not only static visual features should be considered, but also dynamic time series information should be combined to improve the accuracy and robustness of recognition. Therefore, multiple frame images adjacent to the image of the person to be identified are extracted from the video clip, and then smoothly connected through image processing technology (such as interpolation, compensation, etc.) to generate a continuous and smooth short video to show the movements and postures of the person at different time points.

[0049] In the disclosed embodiment, the adjacent 6 frames containing the image of the person to be identified are merged into a short video as the input of the temporal feature extraction sub-model, so as to obtain its timing information. In some other embodiments, the adjacent 10 or 12 frames containing the image of the person to be identified can also be merged into a short video for processing, which is not specifically limited here.

[0050] After obtaining the short video of the person to be identified, the time feature extraction sub-model is used to extract the time features of the short video of the person to be identified. Specifically, refer to Figure 2 FIG. 1 is a flow chart of a time feature extraction method provided by an embodiment of the present disclosure, and the method includes the following S1021 to S1024:

[0051] S1021, based on the feature encoding module, respectively encode the multiple frame images constituting the short video of the person to be identified to obtain a time feature vector and a time person feature vector corresponding to each of the frame images; and based on the feature encoding module, encode the short video of the person to be identified to obtain a time memory set.

[0052] It can be understood that, based on the feature encoding module, the multiple frame images constituting the short video of the person to be identified are first encoded frame by frame to generate a time feature vector and a time character feature vector corresponding to each frame image. Then, the feature encoding module is also used to encode the entire short video of the person to be identified to obtain a time memory set containing time memory. In the time memory set, each time memory has a key-value structure, the key encodes the prototype of the time pattern, and the value memorizes the optimized time attention.

[0053] S1022, inputting the time feature vector corresponding to each of the frame images into the feature merging module for feature merging processing to obtain a global time feature vector.

[0054] For example, after obtaining the time feature vector corresponding to each frame image, the feature merging module is used to integrate the time feature vector corresponding to each frame image into a global time feature vector through global average pooling and LSTM module, which contains the key information of the entire video sequence. By comprehensively considering the features of each frame, a more comprehensive and accurate description can be provided, which is helpful for subsequent character recognition and behavior analysis tasks. In this way, it not only helps to simplify and optimize the feature processing process, but also improves the performance and effect of the model in complex video scenes.

[0055] S1023, according to the time memory set, based on the feature matching module, perform feature matching probability calculation on the global time feature vector, and determine the time attention value corresponding to the short video of the person to be identified based on the calculation result.

[0056] Here, according to the time memory set, the feature matching module compares and matches the global time feature vector with the time feature vector stored in the time memory set, and determines the matching probability of the global time feature vector and each time feature vector in the time memory set by calculating the feature matching probability calculation formula. The probability value reflects the similarity or importance between the global time feature vector in the short video and each time feature vector in the time memory set. The feature matching probability calculation formula is expressed as:

[0057]

[0058] Where N represents the total number of time feature vectors; is the nth time feature vector The matching probability of q t is the global temporal eigenvector; is the nth time feature vector The key value of the matching probability; is the n'th time feature vector The key value of the matching probability.

[0059] After obtaining the matching probability of each time feature vector, the time attention value corresponding to the short video of the person to be identified is calculated according to the time attention calculation formula, and the formula is:

[0060]

[0061] Among them, t Indicates time attention.

[0062] S1024, according to the temporal attention value, based on the feature aggregation module, weighted average calculation is performed on the temporal character feature vector corresponding to each of the frame images, and the calculation result is input into the feature aggregation module for feature aggregation processing to obtain the character feature vector containing the temporal feature.

[0063] It can be understood that after obtaining the temporal attention value, the temporal character feature vector corresponding to each frame image is weighted averaged to more effectively capture the important information in the time series. The weighted average calculation formula is:

[0064]

[0065] Among them, f i s Represents the character feature vector of the frame image; f t Represents the processed character feature vector; Represents the softmax function applied to the temporal attention value.

[0066] Then, the temporal character feature vector corresponding to each frame image after weighted averaging is input into the feature aggregation module for feature aggregation processing to obtain a character feature vector containing temporal features.

[0067] S103, performing feature fusion processing on the appearance feature map, the joint feature map and the character feature vector based on the trained human body weight recognition model to generate a global feature vector and a local feature vector.

[0068] Here, after obtaining the appearance feature map, joint feature map and character feature vector, a feature fusion module is used to fuse them. Feature fusion processing refers to combining feature information obtained from different feature extraction methods to obtain a more comprehensive and accurate information representation. Specific methods may include weighted average fusion method, serial fusion method and multiplication fusion method.

[0069] Specifically, the feature fusion module is used to perform feature fusion processing on the character feature vector and the appearance feature map to obtain a global feature vector. In this way, the integrity and accuracy of the character features are comprehensively considered, including not only the details of the appearance features, but also the basic feature information of the character itself. Then the global feature vector and the joint feature map are input into the feature fusion module for feature fusion processing to obtain a local feature vector. In this way, by fusing the joint feature map and the global feature vector, a more accurate characterization and analysis of the character's motion state and posture is achieved. The generation of local feature vectors enables the model to more deeply understand and describe the characteristics of the character in dynamic or static situations, further improving the accuracy and information richness of image feature processing.

[0070] S104, obtaining a person image library, matching the global feature vector and the local feature vector with the person image library based on the trained person weight recognition model, and obtaining the person image to be recognized and the corresponding target person in the person image library.

[0071] Here, the character image library is constructed based on video clips of the substation collected from at least two viewing angles. For a specific construction method, please refer to step S101, which will not be described in detail here.

[0072] Exemplarily, the character image library includes multiple character images, specifically including at least one character image under different viewing angle numbers corresponding to each character number, that is, the character image library may include multiple character images, and there may be multiple character numbers corresponding to the character images. Correspondingly, there may be multiple viewing angle numbers corresponding to the same character number (for example, character number 1 may correspond to two viewing angles, and the image information of its character image can be expressed as [character number_1 viewing angle_1], [character number_1 viewing angle_2]), but it is necessary to ensure that each character in the character image library has at least one character image from different viewing angles.

[0073] It is understandable that when matching the global feature vector and the local feature vector with the person image library based on the trained person weight recognition model, Figure 3 As shown, the following S1041 to S1044 may be included:

[0074] S1041, determining a character image subset in the character image library based on the viewing angle sequence number of the character image to be identified, wherein the viewing angle sequence number of the character images in the character image subset is different from the viewing angle sequence number of the character image to be identified.

[0075] Here, since human body recognition requires finding a target person that matches the image of the person to be identified in another perspective, it is necessary to first construct a person image set in the person image library whose perspective number is different from the perspective number of the image of the person to be identified, that is, to construct the person images in the person image library with different perspective numbers from the image of the person to be identified into a person image subset.

[0076] S1042: For each person image in the person image subset, determine local relationship information between the person image and the person image to be identified based on the local feature vector and the adaptive directional graph convolution module.

[0077] Specifically, in order to deal with the problem of information loss caused by occlusion, this paper proposes to use a graph convolutional network (GCN) to model high-order relational information. The local features of the image are regarded as nodes of the graph. By passing messages in the graph, not only node features but also edge features can be considered. However, the features of the occluded area are usually meaningless or even noisy, so an adaptive directional graph convolution module is proposed. For each person image in the person image subset, the adaptive directional graph convolution module is used to dynamically learn the direction and degree of message passing, thereby automatically suppressing the transmission of meaningless features and promoting the transmission of semantic features, and then analyzing and comparing its local relational information with the person image to be identified.

[0078] S1043 , for each person image in the person image subset, determine the global topological information of the person image and the person image to be identified based on the cross graph embedding alignment module according to the global feature vector.

[0079] It is understandable that the first-order matching strategy cannot handle the outlier problem, especially in severe occlusion conditions, so the present disclosure proposes a cross-graph embedding alignment module, which not only utilizes the character topology information learned by the graph matching algorithm, but also avoids sensitive one-to-one alignment. The cross-graph embedding alignment module takes two character images as input and outputs embedded features (i.e., global topology information).

[0080] S1044, for each person image in the person image subset, calculate the similarity corresponding to the person image based on the similarity calculation module according to the local relationship information and the global topology information; and sort each person image based on the similarity calculation result corresponding to each person image in the person image subset to determine the person image to be identified and the target person corresponding to the person image subset.

[0081] Specifically, after obtaining the local relationship information and global topology information of each person image in the person image subset relative to the person image to be identified, the similarity corresponding to the person image is calculated based on the similarity calculation module; the similarity value takes into account the local relationship information and the global topology information to ensure accurate matching. Then, the person images in the person image subset are sorted according to the calculation results to find the target person in the person image subset that is most similar to the person image to be identified.

[0082] The method, device, medium and equipment for identifying human weight in a substation provided in the embodiments of the present disclosure use a trained human weight identification model to extract the appearance features and joint features of the image of the person to be identified, and on this basis, extract the time features of the short video corresponding to the image of the person to be identified, obtain the person features containing the time series features, and then fuse the two features, and match the fused results with the person images of other perspectives in the person image library to accurately identify the target person. In this way, the accuracy of human weight identification is significantly improved, and it is suitable for environments such as substations where the similarity of staff is high and efficient and safe monitoring is required.

[0083] The following is a detailed description of the training method of the above-mentioned human weight recognition model. When training the human weight recognition model, the following (1) to (2) may be included:

[0084] (1) obtaining a training sample set and a sample image library, wherein the training sample set includes a plurality of training sample images and a sample short video corresponding to each of the training sample images;

[0085] (2) Obtaining a human weight recognition model to be trained, and performing supervised training on the human weight recognition model to be trained based on the training sample set and the sample image library to obtain the trained human weight recognition model.

[0086] In the embodiment of the present disclosure, the obtained training sample library is obtained by cropping the input image to 256x128, performing random horizontal flipping, padding 10 pixels, random cropping, random erasing and other data enhancement operations. During the model training process, the epoch is set to 120, and each round is iterated 200 times. The model selects the Adam optimizer, the initial learning rate is set to 3.5e-4, and decays to 0.1 at the 30th and 70th epochs. In the present disclosure, one-sixth of the training sample images selected from the training sample set are used as the sample image library, including at least one sample character image under different perspective numbers corresponding to the sample character numbers in the training sample set. .

[0087] After obtaining the training sample set and the training sample library, supervised training is performed on the human weight recognition model to be trained based on the training sample set and the sample image library. Figure 4 FIG. 4 is a flowchart of a method for training a human body weight recognition model provided by an embodiment of the present disclosure, and the method includes the following S401 to S406:

[0088] S401, for each training sample image, extract features of the training sample image based on the human weight recognition model to be trained to obtain an appearance sample feature map and a joint sample feature map.

[0089] Here, the specific scheme for extracting features from the training sample image has been described in detail in the above step S101. The method principle is the same, but the objects of action are different. Please refer to the above step S101 for details, which will not be repeated here.

[0090] S402, extracting time features from a sample short video corresponding to the training sample image based on a time feature extraction sub-model to obtain a sample character feature vector containing time series features.

[0091] Here, the specific scheme for extracting time features from the training sample images has been described in detail in the above step S102. The method principles are the same, but the objects of action are different. Please refer to the above step S102 for details, which will not be repeated here.

[0092] S403, based on the feature fusion module, perform feature fusion processing on the appearance sample feature map and the sample character feature vector to obtain a global sample feature vector; and based on the feature fusion module, perform feature fusion processing on the global sample feature vector and the joint sample feature map to obtain a local sample feature vector.

[0093] Here, the specific scheme for feature fusion processing of the appearance sample feature map and the sample character feature vector has been described in detail in the above step S103. The method principle is the same, but the objects of action are different. Please refer to the above step S103 for details, which will not be repeated here.

[0094] S404, determining a sample image subset in the sample image library based on the viewing angle sequence number of the training sample image; and for each sample person image in the sample image subset, determining local relationship sample information between the sample person image and the training sample image based on an adaptive directional graph convolution module; determining global topological sample information between the sample person image and the training sample image based on a cross graph embedding alignment module; and calculating the similarity corresponding to the sample person image based on the local relationship sample information and the global topological sample information based on a similarity calculation module, and sorting the sample person images according to the calculation results to determine the target sample person corresponding to the training sample image in the sample image subset;

[0095] Here, if the specific scheme for determining the target sample person corresponding to the training sample image in the sample image subset has been described in detail in the above step S104, the method principle is the same, but the object of action is different. Please refer to the above step S104 for details, which will not be repeated here.

[0096] S405, calculating the loss data of the human weight recognition model to be trained based on a preset loss function, and adjusting the model parameters of the human weight recognition model to be trained based on the loss data; the preset loss function includes a classification loss function, a triple loss function, a verification loss function and a time memory extension loss function.

[0097] Among them, the present disclosure introduces a memory extension loss function in the model loss calculation, the purpose of which is to prevent the model from accessing only specific memory items during the training process and ignoring other memory items. This can effectively avoid the model's excessive reliance on certain specific memory items, thereby improving the generalization ability and robustness of the model. The loss function minimizes the difference between the minimum and maximum number of times each memory item is accessed during the training process, thereby ensuring that all memory items can be fairly accessed and utilized during the training process. In this way, not only can the access frequency of each memory item be balanced, but also the information loss caused by too few accesses to certain memory items can be prevented, which helps to improve the overall performance of the model when processing complex tasks and ensure its adaptability and effectiveness on diversified data.

[0098] In some other embodiments, the model may be evaluated by calculating the accuracy based on the person images that match the training sample images and the training sample images.

[0099] Here, the preset loss function is expressed as:

[0100] L=L+λ R L R +λ T L T +L ST ;

[0101]

[0102] Where, L represents the loss data; L S , L R Contains classification loss and triple loss, L T is the validation loss, L SY extended loss of temporal memory; represents the matching probability of temporal memory, and α is a predefined boundary that ensures that the difference between the minimum and maximum values ​​is at least α.

[0103] After the loss data is calculated, the model parameters of the human weight recognition model to be trained can be adjusted according to the loss data.

[0104] S406, repeating the above steps S401-S405 until the training result meets the preset requirements, thereby obtaining the trained human weight recognition model.

[0105] In this way, applying the trained human weight recognition model to practice can achieve the same goal of finding the adjacent moments from one perspective to another perspective, solving the occlusion caused by the limitation of the perspective.

[0106] Combine the following Figure 5, and the entire model training process is described in detail. First, a fixed image of the same size to be identified is obtained from the training set as the input of the model, and the six adjacent frame images of the image to be identified are processed and merged into a sample short video. Secondly, the CNN algorithm is used to extract the appearance sample feature map of the image to be identified, and the joint feature extraction algorithm is used to extract the joint sample feature map; at the same time, the time feature extraction sub-model is used to extract the time feature of the sample short video. Then, the feature fusion module is used to perform feature fusion processing on the appearance sample feature map and the sample character feature vector to obtain the global sample feature vector; and based on the feature fusion module, the global sample feature vector and the joint sample feature map are subjected to feature fusion processing to obtain the local sample feature vector. After obtaining the global sample feature vector and the local sample feature vector, the adaptive directional graph convolution module (ADGC) is used to determine the local relationship sample information of the sample person image and the training sample image, and the cross graph embedding alignment module (CGEA) is used to determine the global topological sample information of the sample person image and the training sample image, and the similarity corresponding to the sample person image is calculated based on the similarity calculation module. According to the calculation results, each sample person image is sorted to determine the target sample person corresponding to the training sample image in the sample image subset. Finally, the model loss is calculated, and then it is determined whether the iteration cycle of the model reaches 120 times. If it does not reach it, the model training continues. If it reaches it, the trained human weight recognition model can be obtained. At the same time, after completing the matching of the target object using the trained human weight recognition model, the accuracy can also be calculated based on the human image matching the training sample image and the training sample image to evaluate the model.

[0107] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0108] Based on the same inventive concept, the embodiment of the present disclosure also provides a substation human weight identification device corresponding to the substation human weight identification method. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned substation human weight identification method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0109] Reference Figure 6 FIG. 6 is a schematic diagram of a substation human weight recognition device 600 provided by an embodiment of the present disclosure, wherein the device comprises:

[0110] The image acquisition module 601 is used to acquire an image of a person to be identified, and extract features of the image of the person to be identified using a trained human weight recognition model to obtain an appearance feature map and a joint feature map. The image of the person to be identified is any image of a plurality of images of a person detected in a video clip of a substation collected from at least two viewing angles.

[0111] The feature extraction module 602 is used to generate a short video of the person to be identified corresponding to the image of the person to be identified by using a plurality of frame images adjacent to the image of the person to be identified, and to extract time features from the short video of the person to be identified by using the trained human weight recognition model to obtain a person feature vector containing time series features;

[0112] A feature processing module 603 is used to perform feature fusion processing on the appearance feature map, the joint feature map and the character feature vector based on the trained human weight recognition model to generate a global feature vector and a local feature vector;

[0113] The character matching module 604 is used to obtain a character image library, and match the global feature vector and the local feature vector with the character image library based on the trained human body weight recognition model to obtain the image of the person to be identified and the corresponding target person in the character image library, wherein the character image library is constructed based on video clips of the substation captured from at least two perspectives.

[0114] In some possible embodiments, the image acquisition module 601 is further used for:

[0115] Acquire video clips collected from the at least two viewing angles in the substation during the same time period, perform person image detection on the video clips based on a video target detection algorithm, and perform person capture and person image cropping to obtain the plurality of person images;

[0116] Marking the image information of the plurality of character images respectively; the image information includes a character serial number and a viewing angle serial number;

[0117] Accordingly, the image acquisition module 601 is also used for:

[0118] The character image library is determined based on the multiple character images, and the character image library includes at least one character image under a different viewing angle number corresponding to each character number in the multiple character images.

[0119] In some possible embodiments, the human body weight recognition model includes a time feature extraction sub-model; the time feature extraction sub-model includes a feature encoding module, a feature merging module, a feature matching module and a feature aggregation module; the feature extraction module 602 is specifically used for:

[0120] Based on the feature encoding module, multiple frame images constituting the short video of the person to be identified are respectively encoded to obtain a time feature vector and a time character feature vector corresponding to each frame image; and based on the feature encoding module, the short video of the person to be identified is encoded to obtain a time memory set;

[0121] Inputting the time feature vector corresponding to each frame image into the feature merging module for feature merging processing to obtain a global time feature vector;

[0122] According to the time memory set, based on the feature matching module, the feature matching probability of the global time feature vector is calculated, and based on the calculation result, the time attention value corresponding to the short video of the person to be identified is determined;

[0123] According to the temporal attention value, based on the feature aggregation module, a weighted average calculation is performed on the temporal character feature vector corresponding to each frame image, and the calculation result is input into the feature aggregation module for feature aggregation processing to obtain the character feature vector containing the temporal feature.

[0124] In some possible embodiments, the human body weight recognition model further includes a feature fusion module, and the feature processing module 603 is specifically used to:

[0125] Based on the feature fusion module, feature fusion processing is performed on the character feature vector and the appearance feature map to obtain the global feature vector;

[0126] The global feature vector and the joint feature map are input into the feature fusion module for feature fusion processing to obtain the local feature vector.

[0127] In some possible embodiments, the human weight recognition model further includes a similarity ranking sub-model; the similarity ranking sub-model includes an adaptive directional graph convolution module, a cross graph embedding alignment module and a similarity calculation module; the person matching module 604 is specifically used for:

[0128] Determine a person image subset in the person image library based on the viewpoint sequence number of the person image to be identified, wherein the viewpoint sequence number of the person images in the person image subset is different from the viewpoint sequence number of the person image to be identified;

[0129] For each person image in the person image subset, determining local relationship information between the person image and the person image to be identified based on the local feature vector and the adaptive directional graph convolution module;

[0130] For each person image in the person image subset, determining global topological information of the person image and the person image to be identified based on the cross graph embedding alignment module according to the global feature vector;

[0131] For each person image in the person image subset, the similarity corresponding to the person image is calculated based on the similarity calculation module according to the local relationship information and the global topology information; and the person images in the person image subset are sorted based on the similarity calculation results corresponding to each person image in the person image subset to determine the person image to be identified and the target person corresponding to the person image subset.

[0132] In some possible embodiments, reference Figure 7 As shown, the device 600 also includes:

[0133] The sample acquisition module 605 is used to acquire a training sample set and a sample image library, wherein the training sample set includes a plurality of training sample images and a sample short video corresponding to each of the training sample images; the sample image library includes at least one sample person image under different viewing angle numbers corresponding to each sample person number in the training sample set;

[0134] The model training module 606 is used to obtain a human weight recognition model to be trained, and perform supervised training on the human weight recognition model to be trained based on the training sample set and the sample image library to obtain the trained human weight recognition model.

[0135] In some possible embodiments, the model training module 606 is specifically used to:

[0136] Step 1: for each training sample image, extract features of the training sample image based on the human weight recognition model to be trained to obtain an appearance sample feature map and a joint sample feature map;

[0137] Step 2: extracting temporal features from the sample short video corresponding to the training sample image based on the temporal feature extraction sub-model to obtain a sample character feature vector containing temporal features;

[0138] Step 3: Based on the feature fusion module, the appearance sample feature map and the sample character feature vector are subjected to feature fusion processing to obtain a global sample feature vector; and based on the feature fusion module, the global sample feature vector and the joint sample feature map are subjected to feature fusion processing to obtain a local sample feature vector;

[0139] Step 4: Determine a sample image subset in the sample image library based on the viewing angle sequence number of the training sample image; and for each sample person image in the sample image subset, determine the local relationship sample information between the sample person image and the training sample image based on the adaptive directional graph convolution module; determine the global topological sample information between the sample person image and the training sample image based on the cross graph embedding alignment module; and calculate the similarity corresponding to the sample person image based on the local relationship sample information and the global topological sample information based on the similarity calculation module, and sort the sample person images according to the calculation results to determine the target sample person corresponding to the training sample image in the sample image subset;

[0140] Step 5: Calculate the loss data of the human weight recognition model to be trained based on a preset loss function, and adjust the model parameters of the human weight recognition model to be trained based on the loss data; the preset loss function includes a classification loss function, a triple loss function, a verification loss function, and a time memory extension loss function;

[0141] Step 6: Repeat the above steps 1-5 until the training result meets the preset requirements, and obtain the trained human weight recognition model.

[0142] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Figure 8 , which is a schematic diagram of the structure of a computer device 800 provided in an embodiment of the present disclosure, including a processor 801, a memory 802, and a bus 803. The memory 802 is used to store execution instructions, including a memory 8021 and an external memory 8022; the memory 8021 is also called an internal memory, which is used to temporarily store the operation data in the processor 801 and the data exchanged with the external memory 8022 such as a hard disk. The processor 801 exchanges data with the external memory 8022 through the memory 8021.

[0143] In the embodiment of the present application, the memory 802 is specifically used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 801. That is, when the computer device 800 is running, the processor 801 communicates with the memory 802 through the bus 803, so that the processor 801 executes the application code stored in the memory 802, and then executes the method described in any of the above embodiments.

[0144] Among them, the memory 802 can be, but is not limited to, random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0145] Processor 801 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0146] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the computer device 800. In other embodiments of the present application, the computer device 800 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0147] The embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for identifying human weight in a substation described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0148] The embodiments of the present disclosure also provide a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the substation human weight identification method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0149] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0150] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0151] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0152] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0153] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0154] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A method for identifying human weight in a substation, characterized in that: include: Acquire an image of a person to be identified, and extract features of the image of the person to be identified using a trained human weight recognition model to obtain an appearance feature map and a joint feature map, wherein the image of the person to be identified is any image of a plurality of images of a person detected in a video clip of a substation collected from at least two viewing angles; Using a plurality of frame images adjacent to the image of the person to be identified, generating a short video of the person to be identified corresponding to the image of the person to be identified, and using the trained human weight recognition model to extract time features from the short video of the person to be identified to obtain a person feature vector containing time series features; Based on the trained human weight recognition model, feature fusion processing is performed on the appearance feature map, the joint feature map and the character feature vector to generate a global feature vector and a local feature vector; Acquire a person image library, match the global feature vector and the local feature vector with the person image library based on the trained person weight recognition model, and obtain the person image to be identified and the corresponding target person in the person image library, wherein the person image library is constructed based on video clips of the substation captured from at least two perspectives.

2. The method according to claim 1, characterized in that: Before obtaining the image of the person to be identified, the method includes: Acquire video clips collected from the at least two viewing angles in the substation during the same time period, perform person image detection on the video clips based on a video target detection algorithm, and perform person capture and person image cropping to obtain the plurality of person images; Marking the image information of the plurality of character images respectively; the image information includes a character serial number and a viewing angle serial number; Accordingly, before obtaining the character image library, the following steps are included: The character image library is determined based on the multiple character images, and the character image library includes at least one character image under a different viewing angle number corresponding to each character number in the multiple character images.

3. The method according to claim 2, characterized in that The human weight recognition model includes a time feature extraction sub-model; the time feature extraction sub-model includes a feature encoding module, a feature merging module, a feature matching module and a feature aggregation module; the method of extracting time features from the short video of the person to be recognized by using the trained human weight recognition model includes: Based on the feature encoding module, multiple frame images constituting the short video of the person to be identified are respectively encoded to obtain a time feature vector and a time character feature vector corresponding to each frame image; and based on the feature encoding module, the short video of the person to be identified is encoded to obtain a time memory set; Inputting the time feature vector corresponding to each frame image into the feature merging module for feature merging processing to obtain a global time feature vector; According to the time memory set, based on the feature matching module, the feature matching probability of the global time feature vector is calculated, and based on the calculation result, the time attention value corresponding to the short video of the person to be identified is determined; According to the temporal attention value, based on the feature aggregation module, a weighted average calculation is performed on the temporal character feature vector corresponding to each frame image, and the calculation result is input into the feature aggregation module for feature aggregation processing to obtain the character feature vector containing the temporal feature.

4. The method according to claim 3, characterized in that The human body weight recognition model further includes a feature fusion module, which performs feature fusion processing on the appearance feature map, the joint feature map and the character feature vector based on the trained human body weight recognition model, including: Based on the feature fusion module, feature fusion processing is performed on the character feature vector and the appearance feature map to obtain the global feature vector; The global feature vector and the joint feature map are input into the feature fusion module for feature fusion processing to obtain the local feature vector.

5. The method according to claim 4, characterized in that The human weight recognition model also includes a similarity sorting sub-model; the similarity sorting sub-model includes an adaptive directional graph convolution module, a cross graph embedding alignment module and a similarity calculation module; the human weight recognition model based on the trained human weight recognition model matches the global feature vector and the local feature vector with the person image library to obtain the person image to be recognized and the corresponding target person in the person image library, including: Determine a person image subset in the person image library based on the viewpoint sequence number of the person image to be identified, wherein the viewpoint sequence number of the person images in the person image subset is different from the viewpoint sequence number of the person image to be identified; For each person image in the person image subset, determining local relationship information between the person image and the person image to be identified based on the local feature vector and the adaptive directional graph convolution module; For each person image in the person image subset, determining global topological information of the person image and the person image to be identified based on the cross graph embedding alignment module according to the global feature vector; For each person image in the person image subset, the similarity corresponding to the person image is calculated based on the similarity calculation module according to the local relationship information and the global topology information; and the person images in the person image subset are sorted based on the similarity calculation results corresponding to each person image in the person image subset to determine the person image to be identified and the target person corresponding to the person image subset.

6. The method according to any one of claims 1 to 5, characterized in that: The human weight recognition model is trained by the following steps: Acquire a training sample set and a sample image library, wherein the training sample set includes a plurality of training sample images and a sample short video corresponding to each of the training sample images; the sample image library includes at least one sample person image under different viewing angle numbers corresponding to each sample person number in the training sample set; A human weight recognition model to be trained is obtained, and supervised training is performed on the human weight recognition model to be trained based on the training sample set and the sample image library to obtain the trained human weight recognition model.

7. The method according to claim 6, characterized in that The supervised training of the human weight recognition model to be trained based on the training sample set and the sample image library includes: Step 1: for each training sample image, extract features of the training sample image based on the human weight recognition model to be trained to obtain an appearance sample feature map and a joint sample feature map; Step 2: extracting temporal features from the sample short video corresponding to the training sample image based on the temporal feature extraction sub-model to obtain a sample character feature vector containing temporal features; Step 3: Based on the feature fusion module, the appearance sample feature map and the sample character feature vector are subjected to feature fusion processing to obtain a global sample feature vector; and based on the feature fusion module, the global sample feature vector and the joint sample feature map are subjected to feature fusion processing to obtain a local sample feature vector; Step 4: Determine a sample image subset in the sample image library based on the viewing angle sequence number of the training sample image; and for each sample person image in the sample image subset, determine the local relationship sample information between the sample person image and the training sample image based on the adaptive directional graph convolution module; determine the global topological sample information between the sample person image and the training sample image based on the cross graph embedding alignment module; and calculate the similarity corresponding to the sample person image based on the local relationship sample information and the global topological sample information based on the similarity calculation module, and sort the sample person images according to the calculation results to determine the target sample person corresponding to the training sample image in the sample image subset; Step 5: Calculate the loss data of the human weight recognition model to be trained based on a preset loss function, and adjust the model parameters of the human weight recognition model to be trained based on the loss data; the preset loss function includes a classification loss function, a triple loss function, a verification loss function, and a time memory extension loss function; Step 6: Repeat the above steps 1-5 until the training result meets the preset requirements, and obtain the trained human weight recognition model.

8. A human weight recognition device for a substation, characterized in that: include: An image acquisition module is used to acquire an image of a person to be identified, and to extract features of the image of the person to be identified using a trained human weight recognition model to obtain an appearance feature map and a joint feature map, wherein the image of the person to be identified is any image of a plurality of images of a person detected in a video clip of a substation collected from at least two viewing angles; A feature extraction module, used to generate a short video of the person to be identified corresponding to the image of the person to be identified by using a plurality of frame images adjacent to the image of the person to be identified, and to extract time features from the short video of the person to be identified by using the trained human weight recognition model to obtain a person feature vector containing time series features; A feature processing module, used for performing feature fusion processing on the appearance feature map, the joint feature map and the character feature vector based on the trained human weight recognition model to generate a global feature vector and a local feature vector; A character matching module is used to obtain a character image library, and based on the trained human weight recognition model, the global feature vector and the local feature vector are matched with the character image library to obtain the image of the person to be identified and the corresponding target person in the character image library, wherein the character image library is constructed based on video clips of the substation captured from at least two perspectives.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images