Cable channel personnel identification method and device, server and storage medium
By using a face recognition network in the cable channel combined with visible light and infrared camera data to identify staff in real time, the problem of insufficient real-time and responsiveness of traditional cable tunnel platforms is solved, and the recognition efficiency and accuracy are improved.
Patent Information
- Application Number
- CN202411215159.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-30
AI Technical Summary
The personnel identification of traditional cable tunnel platforms relies on manual video surveillance, resulting in poor real-time identification and insufficient response capabilities for abnormal incidents.
The camera unit is used to read the real-time video stream of the cable channel, and the trained face recognition network uses the preset number of frames to read the image with faces for recognition, determine whether it is a staff member, and record identity information or output alarms. The network combines visible light and infrared camera data to extract and reconstruct features through backbone and branch networks to improve identification accuracy.
Real-time personnel identification and improved abnormal incident response capabilities, and enhanced the identification and management efficiency of non-staff members.
Smart Images

Figure CN119091355A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of face recognition technology, and in particular to a method, device, server and storage medium for identifying people in a cable channel. Background Art
[0002] Cable tunnels are corridors or tunnel structures used to accommodate a large number of cables laid on cable racks. In addition to better protecting cables, cable tunnels also make it easier for people to inspect and repair cables, and the demand for safety management and monitoring is becoming increasingly prominent.
[0003] In the prior art, personnel identification on traditional cable tunnel platforms is mainly achieved by manually discovering areas with personnel problems through video recording, and then conducting human patrols to check the areas with personnel problems, so as to further identify the personnel in the problem areas and ensure that there are no outsiders.
[0004] However, in the prior art, personnel identification on traditional cable tunnel platforms can only be done manually, which is prone to problems such as poor real-time performance of personnel identification and poor response capabilities to abnormal personnel events. Summary of the invention
[0005] The present application provides a method, device, server and storage medium for personnel identification in a cable channel to solve the problem that personnel identification in a traditional cable tunnel platform can only be performed manually, which is prone to poor real-time performance of personnel identification and poor response capability to abnormal personnel events.
[0006] In a first aspect, the present application provides a method for identifying personnel in a cable channel, which is applied to a server and includes:
[0007] Reading a real-time video stream captured by a camera unit on the cable channel;
[0008] In the real-time video stream, a frame of image with a human face is read every preset number of frames;
[0009] Inputting the image with the human face into a trained face recognition network for face recognition to determine whether the human face is a staff member;
[0010] If it is a staff member, the corresponding identity information is recorded; if it is a non-staff member, an alarm is output.
[0011] In one possible design, the camera unit includes a visible light camera and an infrared camera, and the face recognition network includes a backbone network and a branch network, wherein the backbone network processes visible light face images taken by the visible light camera and infrared face images taken by the infrared camera, and the branch network processes the visible light face images; accordingly, the training process of the face recognition network includes: collecting all face images of all staff members historically taken by the camera unit, and associating and annotating each face image with the corresponding staff member to obtain a data set of all annotated face images, wherein all face images include the visible light face images; face image and the infrared face image; input the data set into the backbone network for processing to obtain the face image corresponding to each staff member; determine the first recognition effect of the face of each staff member according to the face image and the corresponding associated annotation; input the visible light face image into the branch network for processing to obtain the reconstructed face image; determine the second recognition effect of the face of each staff member according to the reconstructed face image; measure the final recognition effect of the face recognition network according to the first recognition effect and the second recognition effect; if the final recognition effect meets the preset recognition requirements, it is determined that the face recognition network training is completed.
[0012] In a possible design, the data set is input into the backbone network for processing to obtain the facial images corresponding to each staff member, including: inputting the data set into the convolution layer in the backbone network for feature extraction to obtain the corresponding preliminary feature map; weighting the preliminary feature map according to the self-attention mechanism to obtain a first self-attention feature map; inputting the preliminary feature map into the first convolution layer for feature extraction to obtain an extracted feature map; weighting the extracted feature map according to the self-attention mechanism to obtain a second self-attention feature map; inputting the extracted feature map into the second convolution layer for feature extraction to obtain an initial intermediate layer feature map; fusing the first self-attention feature map, the second self-attention feature map and the initial intermediate feature map to obtain an intermediate feature map; inputting the intermediate feature map into the third convolution layer for feature extraction to obtain a final intermediate layer feature map; inputting the final intermediate layer feature map into the fully connected layer for nonlinear expression capability enhancement to obtain the facial images corresponding to each staff member.
[0013] In one possible design, the preliminary feature map is weighted according to a self-attention mechanism to obtain a first self-attention feature map, including: inputting the preliminary feature map into a preset convolutional layer to adjust the number of channels to obtain an adjusted preliminary feature map, and generating a key feature map matrix, a query feature map matrix, and a value feature map matrix according to the adjusted preliminary feature map; multiplying the query feature map matrix and the key feature map matrix to obtain an attention weight; and multiplying the attention weight with the value feature map matrix to obtain the first self-attention feature map.
[0014] In one possible design, the first convolutional layer includes multiple convolutions, rectified linear units, and pooling layers; accordingly, inputting the preliminary feature map into the first convolutional layer for feature extraction to obtain an extracted feature map includes: inputting the preliminary feature map into the first convolutional layer, and performing feature extraction through multiple convolutions, rectified linear units, and pooling layers in the first convolutional layer to obtain an extracted feature map.
[0015] In a possible design, the visible light facial image is input into the branch network for processing to obtain a reconstructed facial image, including: inputting the facial image taken by the visible light camera into the convolution module in the branch network for data extraction to obtain corresponding illumination parameters and viewpoint matrix; extracting the corresponding visible light intermediate feature map from the intermediate feature map, and extracting the standard albedo map and the standard depth map from the visible light intermediate feature map; inputting the visible light intermediate feature map into the encoder network layer and the decoder network layer for identity extraction respectively to obtain the depth map and albedo map related to the face of each staff member; calculating a normal map according to the depth map, and deriving a shadow map corresponding to the face of each staff member according to the normal map and the illumination parameters; calculating the standard facial image corresponding to each staff member according to the albedo map, illumination parameters and shadow map; reconstructing the standard facial image according to the viewpoint matrix and the depth map to obtain a reconstructed facial image.
[0016] In a possible design, the standard face image is reconstructed according to the view matrix and the depth map to obtain a reconstructed face image, and the calculation formula is:
[0017]
[0018] In the formula, is the reconstructed face image; a represents the albedo map; d represents the depth map; l represents the illumination parameter; w represents the viewpoint matrix; Λ represents the illumination parameter; Π represents the remapping function.
[0019] In a possible design, the calculation formula for determining the first recognition effect of the face of each staff member according to the face image and the corresponding associated annotation is:
[0020]
[0021] In the formula, F L is the classification loss function corresponding to the first recognition effect; N is the training batch of face image samples; x i is the face image input in this batch; y i is the i-th target; is the rank of the trainable weights of the i-th target in the face recognition network; f(x i ) is the d-dimensional feature vector before the fully connected layer; is the trainable bias of the i-th target in the face recognition network; C is the number of categories; W j T is the rank of the trainable weights of the jth category in the face recognition network; b j is the jth class trainable bias in the face recognition network;
[0022] Accordingly, the calculation formula for determining the second recognition effect of the face of each staff member according to the reconstructed face image is:
[0023]
[0024] In the formula, F R is the loss function corresponding to the second recognition effect; N represents the training batch of face samples, is the reconstructed face image, I is the recognized original face image;
[0025] Accordingly, the calculation formula for measuring the final recognition effect of the face recognition network according to the first recognition effect and the second recognition effect is:
[0026] F=ω1F L Visible light +ω2F L Infrared + ω3F R
[0027] In the formula, F is the joint loss function corresponding to the final recognition effect; ω1 is the proportion of the classification loss function corresponding to the visible light face image in the first recognition effect; ω2 is the proportion of the classification loss function corresponding to the infrared face image in the first recognition effect; ω3 is the proportion of the loss function corresponding to the second recognition effect.
[0028] In a second aspect, the present application provides a personnel identification device for a cable channel, which is applied to a server, comprising:
[0029] A first reading module, used for reading a real-time video stream shot by a camera unit on the cable channel;
[0030] A second reading module is used to read a frame of image with a human face every preset number of frames in the real-time video stream;
[0031] A first input module is used to input the image with a human face into a trained face recognition network for face recognition to determine whether the image is a staff member;
[0032] The output module is used to record the corresponding identity information if the person is a staff member; if the person is not a staff member, an alarm is output.
[0033] In a third aspect, the present application provides a server, comprising: at least one processor and a memory;
[0034] The memory stores computer-executable instructions;
[0035] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method for identifying personnel in a cable channel as described in the first aspect and various possible designs of the first aspect.
[0036] In a fourth aspect, the present application provides a computer storage medium storing computer execution instructions. When a processor executes the computer execution instructions, the method for identifying personnel in a cable channel as described in the first aspect and various possible designs of the first aspect is implemented.
[0037] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the method for identifying personnel in a cable channel as described in the first aspect and various possible designs of the first aspect.
[0038] The personnel identification method, device, server and storage medium for a cable channel provided by the present application read the real-time video stream shot by a camera unit on the cable channel; in the real-time video stream, read a frame of image with a face every preset number of frames; input the image with the face into a trained face recognition network for face recognition to determine whether the person is a staff member; if the person is a staff member, record the corresponding identity information; if the person is not a staff member, output an alarm, so that personnel identification is real-time and the response capability to abnormal personnel events is enhanced. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0040] Figure 1 A schematic diagram of an application scenario of the method for identifying personnel in a cable channel provided in an embodiment of the present application;
[0041] Figure 2 Schematic diagram of the process of the method for identifying people in a cable channel provided in an embodiment of the present application Figure 1 ;
[0042] Figure 3 Schematic diagram of the process of the method for identifying people in a cable channel provided in an embodiment of the present application Figure 2 ;
[0043] Figure 4 A schematic diagram of the structure of a personnel identification device for a cable channel provided in an embodiment of the present application;
[0044] Figure 5 A schematic diagram of the hardware structure of the server provided in an embodiment of the present application. DETAILED DESCRIPTION
[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0046] With the development of society, the use of electricity is indispensable to our lives, and cable tunnels are inextricably linked to electricity. A cable tunnel refers to a tunnel used for laying cables, a corridor or tunnel-like structure used to accommodate a large number of cables laid on cable racks. In addition to allowing the tunnel to better protect the cables, the cable tunnel also makes it convenient for people to inspect and repair the cables, and its safety management and monitoring needs are becoming increasingly significant. In the prior art, the identification of personnel on traditional cable tunnel platforms is mainly through video recording to manually discover areas with personnel problems, and human patrols to investigate the personnel problem areas to further identify the personnel in the problem areas to ensure that there are no outsiders. However, in the prior art, the identification of personnel on traditional cable tunnel platforms can only be done manually, which is prone to problems such as poor real-time identification and poor response to abnormal personnel events.
[0047] In order to solve the above-mentioned technical problems, the embodiments of the present application propose the following technical concept: Taking into account the poor real-time and responsiveness of personnel identification in the prior art, the inventors read the real-time video stream captured by the camera unit in the cable channel, and select images with faces from the real-time video stream. The images with faces are identified using a trained face recognition network to determine whether they are staff members, thereby making personnel identification real-time and enhancing the ability to respond to abnormal personnel events.
[0048] Figure 1 It is a schematic diagram of an application scenario of the method for identifying personnel in a cable channel provided in an embodiment of the present application.
[0049] like Figure 1 As shown, the scene includes: a camera unit 101, a server 102 and a display terminal 103.
[0050] The camera unit 101 includes a visible light camera and an infrared camera.
[0051] The server 102 may be an independent server or a cluster consisting of multiple servers.
[0052] The display terminal 103 may be a display screen or a terminal such as a personal computer.
[0053] The server 102 reads the real-time video stream shot by the camera unit 101 on the cable channel, and reads a frame of image with a face in the real-time video stream every preset number of frames, and performs face recognition on the image with a face through a trained face recognition network. If it is a staff member, the corresponding identity information is recorded; if it is a non-staff member, the alarm is transmitted to the display terminal 103 through the wireless network. The display terminal 103 outputs the alarm. The following is a detailed description using a detailed embodiment.
[0054] Figure 2 Schematic diagram of the process of the method for identifying people in a cable channel provided in an embodiment of the present application Figure 1 , the execution subject of this embodiment can be Figure 1 The server in the embodiment shown is not particularly limited in this embodiment. Figure 2 As shown, the method includes:
[0055] S201: Read the real-time video stream captured by the camera unit on the cable channel.
[0056] Specifically, the real-time video stream captured by the camera unit on the cable channel is read through the Real-Time Streaming Protocol (RTSP).
[0057] In this embodiment, the camera unit includes one or more cameras.
[0058] S202: In the real-time video stream, read a frame of image with a human face every preset number of frames.
[0059] In this embodiment, the preset frame number may be any one of 20, 25 or 30 frames, or other frame numbers.
[0060] S203: Input the image with the face into the trained face recognition network for face recognition to determine whether the person is a staff member.
[0061] In this embodiment, the trained face recognition network can be trained through a back propagation algorithm to continuously optimize the weights and biases of the network.
[0062] S204: If the person is a staff member, the corresponding identity information is recorded; if the person is not a staff member, an alarm is output.
[0063] In this embodiment, the alarm may be a voice alarm, a text alarm or other alarm methods on the display terminal.
[0064] In addition, if the person is not a staff member, the corresponding facial image is recorded.
[0065] In summary, the personnel identification method for the cable channel provided in this embodiment reads the real-time video stream shot by the camera unit on the cable channel; in the real-time video stream, reads a frame of image with a face every preset number of frames; inputs the image with the face into a trained face recognition network for face recognition to determine whether the person is a staff member; if the person is a staff member, the corresponding identity information is recorded; if the person is not a staff member, an alarm is output, so that personnel identification is real-time and the response capability to abnormal personnel events is enhanced.
[0066] Figure 3 Schematic diagram of the process of the method for identifying people in a cable channel provided in an embodiment of the present application Figure 2 In the embodiment of the present application, Figure 2 Based on the embodiment provided, the camera unit includes a visible light camera and an infrared camera, and the face recognition network includes a backbone network and a branch network, wherein the backbone network processes visible light face images taken by the visible light camera and infrared face images taken by the infrared camera, and the branch network processes visible light face images; the specific training method of the trained face recognition network in step S203 is described in detail. Figure 3 As shown, the method includes:
[0067] S301: Collect all facial images of all staff members historically photographed by the camera unit, and associate and annotate each facial image with the corresponding staff member to obtain a data set of all annotated facial images, wherein all facial images include visible light facial images and infrared facial images.
[0068] In this embodiment, the data set may also be divided into a training set and a test set according to a preset ratio.
[0069] The preset ratio may be any ratio of 7:3, 8:2 or 9:1, or other ratios.
[0070] Among them, the training set is used to train the face recognition network; the test set is used to test the subsequently trained face recognition network.
[0071] S302: Input the data set into the backbone network for processing to obtain the facial images corresponding to each staff member.
[0072] Specifically, step S302 is as follows:
[0073] S3021: Input the data set into the convolutional layer in the backbone network for feature extraction to obtain the corresponding preliminary feature map.
[0074] S3022: Perform weighted processing on the preliminary feature map according to the self-attention mechanism to obtain a first self-attention feature map.
[0075] Specifically, step S3022 is as follows:
[0076] S30221: Input the preliminary feature map into a preset convolutional layer to adjust the number of channels to obtain an adjusted preliminary feature map, and generate a key feature map matrix, a query feature map matrix, and a value feature map matrix according to the adjusted preliminary feature map.
[0077] In this embodiment, the preset convolution layer may be a 1×1 convolution layer or other convolution layers.
[0078] In this embodiment, the number of channels can be adjusted to any number of 3, 5 or 7 channels.
[0079] Exemplarily, the preliminary feature map is input into a 1×1 convolutional layer to adjust the number of channels to obtain a preliminary feature map with 3 channels, and a key (K) feature map matrix, a query (Q) feature map matrix, and a value (V) feature map matrix are generated respectively according to the preliminary feature map with 3 channels.
[0080] S30222: Multiply the query feature map matrix and the key feature map matrix to obtain the attention weight.
[0081] Exemplarily, the query (Q) feature map matrix and the key (K) feature map matrix are multiplied to obtain the attention weights.
[0082] In addition, the obtained attention weights need to be scaled and softmaxed to ensure the numerical stability and normalization of the weights.
[0083] S30223: Multiply the attention weight by the value feature map matrix to obtain the first self-attention feature map.
[0084] Exemplarily, the attention weight is multiplied by the value (V) feature map matrix to obtain a self-attention feature map.
[0085] S3023: Input the preliminary feature map into the first convolutional layer for feature extraction to obtain an extracted feature map.
[0086] In this embodiment, the first convolutional layer includes multiple convolutions, rectified linear units and pooling layers; accordingly, step S3023 is specifically as follows:
[0087] The preliminary feature map is input into the first convolutional layer, and feature extraction is performed through multiple convolutions, rectified linear units, and pooling layers in the first convolutional layer to obtain the extracted feature map.
[0088] Exemplarily, the convolution layer is VGG16, the convolution is conv, the rectified linear unit is relu, and the pooling is pooling.
[0089] S3024: Perform weighted processing on the extracted feature map according to the self-attention mechanism to obtain a second self-attention feature map.
[0090] S3025: Input the extracted feature map into the second convolutional layer for feature extraction to obtain an initial intermediate layer feature map.
[0091] Specifically, the extracted feature map is input into the second convolutional layer, and feature extraction is performed through multiple convolutions, rectified linear units and pooling layers in the second convolutional layer to obtain an initial intermediate layer feature map.
[0092] S3026: Fusing the first self-attention feature map, the second self-attention feature map, and the initial intermediate feature map to obtain an intermediate feature map.
[0093] In this embodiment, the fusion is weighted fusion.
[0094] S3027: Input the intermediate feature map into the third convolutional layer for feature extraction to obtain the final intermediate layer feature map.
[0095] Specifically, the intermediate feature map is input into the third convolutional layer, and feature extraction is performed through multiple convolutions, rectified linear units, and pooling layers in the third convolutional layer to obtain the final intermediate layer feature map.
[0096] S3028: Input the final intermediate layer feature map into the fully connected layer to improve the nonlinear expression capability to obtain the facial images corresponding to each staff member.
[0097] S303: Determine a first recognition effect of each staff member's face according to the face image and the corresponding associated annotation.
[0098] In this embodiment, the calculation formula for determining the first recognition effect of each staff member's face according to the face image and the corresponding associated annotation is:
[0099]
[0100] In the formula, F L is the classification loss function corresponding to the first recognition effect; N is the training batch of face image samples; x i is the face image input in this batch; y i is the i-th target; is the rank of the trainable weights of the i-th target in the face recognition network; f(x i ) is the d-dimensional feature vector before the fully connected layer; is the trainable bias of the i-th target in the face recognition network; C is the number of categories; W j T is the rank of the trainable weights of the jth category in the face recognition network; b j is the j-th class trainable bias in the face recognition network.
[0101] Exemplarily, the first recognition effect includes visible light face recognition effect and infrared face recognition effect, and the corresponding classification loss functions are F L Visible light and F L Infrared.
[0102] In addition, in F L A regularization term is added to the loss function to penalize the size of the model feature vector, that is, the f(x i )First, normalize it, then magnify the normalized value by a factor of α, and then multiply it by the corresponding weight layer.
[0103] S304: Input the visible light face image into the branch network for processing to obtain a reconstructed face image.
[0104] Specifically, step S304 is as follows:
[0105] S3041: Input the face image taken by the visible light camera into the convolution module in the branch network for data extraction to obtain the corresponding lighting parameters and viewpoint matrix.
[0106] In this embodiment, the convolution module has a total of 10 layers of conv-relu×5, and each convolution module has an output head conv-tanh for generating corresponding illumination parameters and viewpoint matrices.
[0107] S3042: Extracting a corresponding visible light intermediate feature map from the intermediate feature map, and extracting a canonical albedo map and a canonical depth map from the visible light intermediate feature map.
[0108] S3043: Inputting the visible light intermediate feature map into the encoder network layer and the decoder network layer respectively for identity extraction, so as to obtain a depth map and an albedo map related to the face of each staff member;
[0109] In this embodiment, the encoder network consists of multiple convolutional layers, group normalization layers, LeakyReLU and ReLU activation functions.
[0110] In this embodiment, the decoder network layer reconstructs a three-dimensional object representation with obvious symmetry and deformation characteristics through operations such as deconvolution and normalization.
[0111] S3044: Calculate a normal map based on the depth map, and derive a shadow map corresponding to the face of each staff member based on the normal map and the lighting parameters.
[0112] Specifically, the normal map is calculated according to the depth map gradient.
[0113] S3045: Calculate the standard face image corresponding to each staff member according to the albedo map, lighting parameters and shadow map.
[0114] In this embodiment, the standard face image is a standard frontal and side face image.
[0115] S3046: Reconstruct the standard face image according to the view matrix and the depth map to obtain a reconstructed face image.
[0116] In this embodiment, the standard face image is reconstructed according to the view matrix and the depth map to obtain a reconstructed face image. The calculation formula is:
[0117]
[0118] In the formula, is the reconstructed face image; a represents the albedo map; d represents the depth map; l represents the illumination parameter; w represents the viewpoint matrix; Λ represents the illumination parameter; Π represents the remapping function.
[0119] S305: Determine a second recognition effect of the face of each staff member according to the reconstructed face image.
[0120] In this embodiment, the second recognition effect of each staff member's face is determined according to the reconstructed face image, and the calculation formula is:
[0121]
[0122] In the formula, F R is the loss function corresponding to the second recognition effect; N represents the training batch of face samples, is the reconstructed face image, and I is the recognized original face image.
[0123] S306: Measure the final recognition effect of the face recognition network according to the first recognition effect and the second recognition effect.
[0124] In this embodiment, the final recognition effect of the face recognition network is measured according to the first recognition effect and the second recognition effect, and the calculation formula is:
[0125] F=ω1F L Visible light +ω2F L Infrared + ω3F R
[0126] Where F is the joint loss function corresponding to the final recognition effect; ω1 is the proportion of the classification loss function corresponding to the visible light face image in the first recognition effect; ω2 is the proportion of the classification loss function corresponding to the infrared face image in the first recognition effect; ω3 is the proportion of the loss function corresponding to the second recognition effect.
[0127] S307: If the final recognition result meets the preset recognition requirements, it is determined that the face recognition network training is completed.
[0128] Specifically, if the final recognition effect satisfies that the corresponding joint loss function converges after the gradient update, it is determined that the face recognition network training is completed.
[0129] In summary, the personnel identification method for the cable channel provided in the present embodiment collects all facial images of all staff members historically taken by the camera unit, wherein all facial images include visible light facial images and infrared facial images; the data set is input into the backbone network for processing to obtain the facial image corresponding to each staff member; the first recognition effect of the face of each staff member is determined according to the facial image and the corresponding associated annotation; the visible light facial image is input into the branch network for processing to obtain the reconstructed facial image to determine the second recognition effect of the face of each staff member; the final recognition effect of the facial recognition network is measured according to the first recognition effect and the second recognition effect; if the final recognition effect meets the preset recognition requirements, it is determined that the facial recognition network training is completed, and the accuracy and robustness of personnel recognition are improved through the joint action of infrared facial recognition and visible light facial recognition.
[0130] In addition, the infrared face image in this embodiment has a low requirement for illumination, and can make up for the texture, contour and other information of the visible light face image in a dark or strong light environment.
[0131] In addition, this embodiment adopts a self-attention mechanism, which enables the model to learn and focus more intensively on key feature areas in facial images.
[0132] Figure 4 This is a schematic diagram of the structure of a personnel identification device for a cable channel provided in an embodiment of the present application. Figure 4 As shown, the personnel identification device of the cable channel includes: a first reading module 401 , a second reading module 402 , a first input module 403 and an output module 404 .
[0133] A first reading module 401 is used to read a real-time video stream captured by a camera unit on the cable channel;
[0134] A second reading module 402 is used to read a frame of image with a face every preset number of frames in the real-time video stream;
[0135] The first input module 403 is used to input the image with a human face into a trained face recognition network for face recognition to determine whether the human face is a staff member;
[0136] The output module 404 is used to record the corresponding identity information if the person is a staff member; if the person is not a staff member, output an alarm.
[0137] In a possible implementation, the camera unit includes a visible light camera and an infrared camera, the face recognition network includes a trunk network and a branch network, wherein the trunk network processes visible light face images taken by the visible light camera and infrared face images taken by the infrared camera, and the branch network processes the visible light face images; accordingly, the device further includes:
[0138] A collection module 405 is used to collect all facial images of all staff members historically photographed by the camera unit, and associate and annotate each facial image with a corresponding staff member to obtain a data set of all annotated facial images, wherein all facial images include the visible light facial images and the infrared facial images;
[0139] A second input module 406 is used to input the data set into the backbone network for processing to obtain a face image corresponding to each staff member;
[0140] A first determination module 407, configured to determine a first recognition effect of the face of each staff member according to the face image and the corresponding associated annotation;
[0141] A third input module 408, configured to input the visible light face image into the branch network for processing to obtain a reconstructed face image;
[0142] A second determination module 409, configured to determine a second recognition effect of the face of each staff member according to the reconstructed face image;
[0143] A measuring module 410, configured to measure a final recognition effect of the face recognition network according to the first recognition effect and the second recognition effect;
[0144] The third determination module 411 is used to determine that the face recognition network training is completed if the final recognition result meets the preset recognition requirement.
[0145] In a possible implementation, the second input module 406 specifically includes:
[0146] The first input unit 4061 is used to input the data set into the convolution layer in the backbone network for feature extraction to obtain a corresponding preliminary feature map;
[0147] A first processing unit 4062 is used to perform weighted processing on the preliminary feature map according to the self-attention mechanism to obtain a first self-attention feature map;
[0148] A first extraction unit 4063, configured to input the preliminary feature map into a first convolutional layer for feature extraction to obtain an extracted feature map;
[0149] A second processing unit 4064 is used to perform weighted processing on the extracted feature map according to the self-attention mechanism to obtain a second self-attention feature map;
[0150] A second extraction unit 4065 is used to input the extracted feature map into the second convolutional layer for feature extraction to obtain an initial intermediate layer feature map;
[0151] A fusion unit 4066, configured to fuse the first self-attention feature map, the second self-attention feature map and the initial intermediate feature map to obtain an intermediate feature map;
[0152] A third extraction unit 4067 is used to input the intermediate feature map into a third convolutional layer for feature extraction to obtain a final intermediate layer feature map;
[0153] The improving unit 4068 is used to input the final intermediate layer feature map into the fully connected layer to improve the nonlinear expression ability, so as to obtain the face image corresponding to each staff member.
[0154] In a possible implementation, the first processing unit 4062 specifically includes:
[0155] An input unit 40621 is used to input the preliminary feature map into a preset convolutional layer to adjust the number of channels to obtain an adjusted preliminary feature map, and generate a key feature map matrix, a query feature map matrix and a value feature map matrix according to the adjusted preliminary feature map;
[0156] A first calculation unit 40622, configured to multiply the query feature map matrix and the key feature map matrix to obtain an attention weight;
[0157] The second calculation unit 40623 is used to multiply the attention weight with the value feature map matrix to obtain the first self-attention feature map.
[0158] In a possible implementation, the first convolutional layer includes multiple convolutions, rectified linear units and pooling layers; accordingly, the second extraction unit 4063 is specifically used to: input the preliminary feature map into the first convolutional layer, and perform feature extraction through the multiple convolutions, rectified linear units and pooling layers in the first convolutional layer to obtain the extracted feature map.
[0159] In a possible implementation, the third input module 408 specifically includes:
[0160] The first input unit 4081 is used to input the face image taken by the visible light camera into the convolution module in the branch network for data extraction to obtain corresponding illumination parameters and viewpoint matrix;
[0161] An extraction unit 4082 is used to extract a corresponding visible light intermediate feature map from the intermediate feature map, and to extract a canonical albedo map and a canonical depth map from the visible light intermediate feature map;
[0162] The second input unit 4083 is used to input the visible light intermediate feature map into the encoder network layer and the decoder network layer for identity extraction, so as to obtain a depth map and an albedo map related to the face of each staff member;
[0163] A first calculation unit 4084 is used to calculate a normal map according to the depth map, and derive a shadow map corresponding to the face of each staff member according to the normal map and the illumination parameter;
[0164] A second calculation unit 4085 is used to calculate the standard face image corresponding to each staff member according to the albedo map, the illumination parameters and the shadow map;
[0165] The reconstruction unit 4086 is used to reconstruct the standard face image according to the view matrix and the depth map to obtain a reconstructed face image.
[0166] In a possible implementation, the standard face image is reconstructed according to the view matrix and the depth map to obtain a reconstructed face image, and the calculation formula is:
[0167]
[0168] In the formula, is the reconstructed face image; a represents the albedo map; d represents the depth map; l represents the illumination parameter; w represents the viewpoint matrix; Λ represents the illumination parameter; Π represents the remapping function.
[0169] In a possible implementation, the calculation formula for determining the first recognition effect of the face of each staff member according to the face image and the corresponding associated annotation is:
[0170]
[0171] In the formula, F L is the classification loss function corresponding to the first recognition effect; N is the training batch of face image samples; x i is the face image input in this batch; y i is the i-th target; is the rank of the trainable weights of the i-th target in the face recognition network; f(x i ) is the d-dimensional feature vector before the fully connected layer; is the trainable bias of the i-th target in the face recognition network; C is the number of categories; Wj T is the rank of the trainable weights of the jth category in the face recognition network; b j is the jth class trainable bias in the face recognition network;
[0172] Accordingly, the calculation formula for determining the second recognition effect of the face of each staff member according to the reconstructed face image is:
[0173]
[0174] In the formula, F R is the loss function corresponding to the second recognition effect; N represents the training batch of face samples, is the reconstructed face image, I is the recognized original face image;
[0175] Accordingly, the calculation formula for measuring the final recognition effect of the face recognition network according to the first recognition effect and the second recognition effect is:
[0176] F=ω1F L Visible light +ω2F L Infrared + ω3F R
[0177] In the formula, F is the joint loss function corresponding to the final recognition effect; ω1 is the proportion of the classification loss function corresponding to the visible light face image in the first recognition effect; ω2 is the proportion of the classification loss function corresponding to the infrared face image in the first recognition effect; ω3 is the proportion of the loss function corresponding to the second recognition effect.
[0178] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.
[0179] Figure 5 The hardware structure diagram of the server provided in the embodiment of the present application is shown in FIG. Figure 5 As shown, the server of this embodiment includes: a processor 501 and a memory 502; the memory stores computer execution instructions; at least one processor executes the computer execution instructions stored in the memory, so that at least one processor executes the above-mentioned cable channel personnel identification method.
[0180] Optionally, the memory 502 may be independent or integrated with the processor 501 .
[0181] When the memory 502 is independently provided, the server further includes a bus 503 for connecting the memory 502 and the processor 501 .
[0182] The embodiment of the present application further provides a computer storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the personnel identification method for the cable channel as described above is implemented.
[0183] An embodiment of the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the personnel identification method for the cable channel as described above is implemented.
[0184] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.
[0185] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.
[0186] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The above-mentioned module-composed unit can be implemented in the form of hardware or in the form of hardware plus software functional units.
[0187] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present application.
[0188] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.
[0189] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.
[0190] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.
[0191] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special-purpose computer.
[0192] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a main control device.
[0193] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.
[0194] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for identifying personnel in a cable channel, characterized in that: Applicable to servers, including: Reading a real-time video stream captured by a camera unit on the cable channel; In the real-time video stream, a frame of image with a human face is read every preset number of frames; Inputting the image with the human face into a trained face recognition network for face recognition to determine whether the human face is a staff member; If it is a staff member, the corresponding identity information is recorded; if it is a non-staff member, an alarm is output.
2. The method according to claim 1, characterized in that The camera unit includes a visible light camera and an infrared camera, and the face recognition network includes a trunk network and a branch network, wherein the trunk network processes visible light face images taken by the visible light camera and infrared face images taken by the infrared camera, and the branch network processes the visible light face images; Accordingly, the training process of the face recognition network includes: Collect all facial images of all staff members historically photographed by the camera unit, and associate and annotate each facial image with a corresponding staff member to obtain a data set of all annotated facial images, wherein all facial images include the visible light facial images and the infrared facial images; Inputting the data set into the backbone network for processing to obtain facial images corresponding to each staff member; Determining a first recognition effect of the face of each staff member according to the face image and the corresponding associated annotation; Inputting the visible light face image into the branch network for processing to obtain a reconstructed face image; Determining a second recognition effect of the face of each staff member according to the reconstructed face image; Measuring a final recognition effect of the face recognition network according to the first recognition effect and the second recognition effect; If the final recognition effect meets the preset recognition requirements, it is determined that the face recognition network training is completed.
3. The method according to claim 2, characterized in that The step of inputting the data set into the backbone network for processing to obtain the facial images corresponding to each staff member includes: Inputting the data set into the convolutional layer in the backbone network for feature extraction to obtain a corresponding preliminary feature map; The preliminary feature map is weighted according to the self-attention mechanism to obtain a first self-attention feature map; Inputting the preliminary feature map into the first convolutional layer for feature extraction to obtain an extracted feature map; The extracted feature map is weighted according to the self-attention mechanism to obtain a second self-attention feature map; Inputting the extracted feature map into the second convolutional layer for feature extraction to obtain an initial intermediate layer feature map; Fusing the first self-attention feature map, the second self-attention feature map, and the initial intermediate feature map to obtain an intermediate feature map; Inputting the intermediate feature map into the third convolutional layer for feature extraction to obtain a final intermediate layer feature map; The final intermediate layer feature map is input into the fully connected layer to improve the nonlinear expression capability, so as to obtain the facial images corresponding to each staff member.
4. The method according to claim 3, characterized in that The step of weighting the preliminary feature map according to the self-attention mechanism to obtain a first self-attention feature map includes: Input the preliminary feature map into a preset convolutional layer to adjust the number of channels to obtain an adjusted preliminary feature map, and generate a key feature map matrix, a query feature map matrix, and a value feature map matrix according to the adjusted preliminary feature map; Multiplying the query feature map matrix and the key feature map matrix to obtain an attention weight; The attention weight is multiplied by the value feature map matrix to obtain the first self-attention feature map.
5. The method according to claim 3, characterized in that: wherein the first convolutional layer includes a plurality of convolutional, rectified linear units and pooling layers; Accordingly, the step of inputting the preliminary feature map into the first convolutional layer for feature extraction to obtain an extracted feature map includes: The preliminary feature map is input into a first convolutional layer, and feature extraction is performed through a plurality of convolutions, rectified linear units, and pooling layers in the first convolutional layer to obtain an extracted feature map.
6. The method according to claim 3, characterized in that The step of inputting the visible light face image into the branch network for processing to obtain a reconstructed face image includes: Inputting the face image captured by the visible light camera into the convolution module in the branch network for data extraction to obtain corresponding illumination parameters and viewpoint matrix; Extracting a corresponding visible light intermediate feature map from the intermediate feature map, and extracting a canonical albedo map and a canonical depth map from the visible light intermediate feature map; Inputting the visible light intermediate feature map into the encoder network layer and the decoder network layer for identity extraction, so as to obtain a depth map and an albedo map related to the face of each staff member; Calculating a normal map according to the depth map, and deriving a shadow map corresponding to the face of each staff member according to the normal map and the illumination parameters; Calculate the standard face image corresponding to each staff member according to the albedo map, the lighting parameters and the shadow map; The standard face image is reconstructed according to the viewpoint matrix and the depth map to obtain a reconstructed face image.
7. The method according to claim 6, characterized in that The calculation formula for reconstructing the standard face image according to the viewpoint matrix and the depth map to obtain a reconstructed face image is: In the formula, is the reconstructed face image; a represents the albedo map; d represents the depth map; l represents the illumination parameter; w represents the viewpoint matrix; Λ represents the illumination parameter; Π represents the remapping function.
8. The method according to any one of claims 2 to 7, characterized in that: The calculation formula for determining the first recognition effect of the face of each staff member according to the face image and the corresponding associated annotation is: In the formula, F L is the classification loss function corresponding to the first recognition effect; N is the training batch of face image samples; x i is the face image input in this batch; y i is the i-th target; is the rank of the trainable weights of the i-th target in the face recognition network; f(x i ) is the d-dimensional feature vector before the fully connected layer; is the i-th target trainable bias in the face recognition network; C is the number of categories; is the rank of the trainable weights of the jth category in the face recognition network; b j is the jth class trainable bias in the face recognition network; Accordingly, the calculation formula for determining the second recognition effect of the face of each staff member according to the reconstructed face image is: In the formula, F R is the loss function corresponding to the second recognition effect; N represents the training batch of face samples, is the reconstructed face image, I is the recognized original face image; Accordingly, the calculation formula for measuring the final recognition effect of the face recognition network according to the first recognition effect and the second recognition effect is: F=ω1F L Visible light +ω2F L Infrared + ω3F R In the formula, F is the joint loss function corresponding to the final recognition effect; ω1 is the proportion of the classification loss function corresponding to the visible light face image in the first recognition effect; ω2 is the proportion of the classification loss function corresponding to the infrared face image in the first recognition effect; ω3 is the proportion of the loss function corresponding to the second recognition effect.
9. A personnel identification device for a cable channel, characterized in that: Applicable to servers, including: A first reading module, used for reading a real-time video stream shot by a camera unit on the cable channel; A second reading module is used to read a frame of image with a human face every preset number of frames in the real-time video stream; A first input module is used to input the image with a human face into a trained face recognition network for face recognition to determine whether the image is a staff member; The output module is used to record the corresponding identity information if the person is a staff member; if the person is not a staff member, an alarm is output.
10. A server, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method for identifying personnel in a cable channel according to any one of claims 1 to 8.
11. A computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, and when the processor executes the computer executable instructions, the method for identifying personnel in a cable channel as described in any one of claims 1 to 8 is implemented.
12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for identifying personnel in a cable channel according to any one of claims 1 to 8 is implemented.
Citation Information
Patent Citations
Image forming method and device
CN108241855A
Cross-modal pedestrian re-identification method based on image generation and shared learning network
CN114241517A
Cross-modal face recognition method, apparatus and device, and storage medium
WO2022127111A1
