Fluorescent clothes detection method based on image pedestrian attributes

Abstract

The invention relates to the technical field of target detection, in particular to a fluorescent clothes detection method based on image pedestrian attributes. The method comprises the following steps: S1, capturing a real-time video stream by using a camera, decoding the video stream by combining an H264 decoder with GPU hardware acceleration processing, and detecting pedestrians from the video stream by using an SSD human shape detection algorithm; s2, performing attribute detection on the detected pedestrian picture to obtain all attribute data of the pedestrian; s3, searching fluorescent clothes attributes in the attribute data, and calculating the confidence coefficient of the fluorescent clothes attributes; s4, judging whether the worker has the fluorescent clothes attribute or not according to the fluorescent clothes attribute confidence; and S5, performing result integration through a GstPlugin interface, loading the pedestrian attribute model subjected to TensorRT acceleration optimization through an nvinfer interface of DeepStream, displaying a detection frame and a detection area in combination with an OSD (On Screen Display), and finally performing desktop display through EGLSink.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and more specifically, to a method for detecting fluorescent clothing based on pedestrian attributes in images. Background Technology

[0002] Fluorescent clothing detection methods based on pedestrian attributes in images typically involve computer vision and machine learning techniques, particularly the application of deep learning in image processing and analysis. This approach aims to automatically identify and detect pedestrians wearing fluorescent clothing from images for effective monitoring and analysis in various scenarios.

[0003] Existing methods for detecting fluorescent clothing based on pedestrian attributes in images face challenges in crowded scenes. Pedestrians may be obscured by other pedestrians or objects, affecting detection accuracy. Different shooting angles and pedestrian postures can alter the appearance of the fluorescent clothing, making detection more difficult. Complex backgrounds may contain colors or patterns similar to the fluorescent clothing, increasing the risk of false detections. Furthermore, the color of the fluorescent clothing may change under different lighting conditions, potentially reducing the accuracy of the detection algorithm. Therefore, this paper proposes a method for detecting fluorescent clothing based on pedestrian attributes in images. Summary of the Invention

[0004] The purpose of this invention is to provide a fluorescent clothing detection method based on pedestrian attributes in images, addressing the issues raised in the aforementioned background techniques. In crowded scenes, pedestrians may be obscured by other pedestrians or objects, affecting the detection accuracy of fluorescent clothing. Different shooting angles and pedestrian postures can alter the appearance of the fluorescent clothing, making detection more difficult. Complex backgrounds may contain colors or patterns similar to the fluorescent clothing, increasing the risk of false detection. Furthermore, the color of the fluorescent clothing may change under different lighting conditions, potentially reducing the accuracy of the detection algorithm.

[0005] To achieve the above objectives, the present invention aims to provide a method for detecting fluorescent clothing based on pedestrian attributes in images, comprising the following steps:

[0006] S1. Use a camera to capture real-time video stream, use an H264 decoder combined with GPU hardware acceleration to decode the video stream, and use an SSD human detection algorithm to detect pedestrians from the video stream.

[0007] S2. Perform attribute detection on the detected pedestrian images to obtain all attribute data of the pedestrians;

[0008] S3. Find the fluorescent clothing attribute in the attribute data and calculate the confidence level of the fluorescent clothing attribute;

[0009] Among them, the attributes of fluorescent clothing include whether fluorescent strips are worn and whether fluorescent clothing is worn;

[0010] S4. Determine whether a worker has fluorescent clothing based on the confidence level of the fluorescent clothing attribute;

[0011] S5. The results are integrated through the GstPlugin interface, and the pedestrian attribute model accelerated and optimized by TensorRT is loaded through the nvinfer interface of DeepStream. The detection boxes and detection areas are displayed in conjunction with OSD, and finally displayed on the desktop through EGLISink.

[0012] As a further improvement to this technical solution, in step S1, the specific steps for decoding the video stream using an H.264 decoder combined with GPU hardware acceleration processing are as follows:

[0013] S2.1 First, you need to create an H264 decoder instance and specify the format of the input data for it;

[0014] The S2.2 H264 decoder will read compressed H264 data from the network and local files;

[0015] The S2.3 and H264 decoders will parse the NAL units in the input data and extract the encoded data of each image.

[0016] S2.4. The encoded data is obtained and decoded into the original image using GPU hardware acceleration technology;

[0017] GPU hardware acceleration technologies include:

[0018] Transform coding:

[0019]

[0020] Where X[u][v] represents the frequency domain coefficients obtained after DCT exchange; α(u)α(v) represents the normalization factor; x[x][y] represents the spatial domain element value of the original image; N represents the size of the image patch; and cos represents the cosine function.

[0021] Quantification:

[0022]

[0023] Where Q[u][v] represents the quantized frequency domain coefficients; round represents the rounding function; Q factor Indicates the quantification factor;

[0024] Inverse transform coding:

[0025]

[0026] Where, x[x][y] 逆Y[u][v] represents the frequency domain coefficients obtained after inverse DCT transformation; Y[u][v] represents the frequency domain coefficients after quantization.

[0027] The S2.5 and H264 decoders will output the decoded image.

[0028] As a further improvement to this technical solution, in S1, the SSD human detection algorithm includes:

[0029] Bounding box prediction:

[0030]

[0031] Among them, b x b represents the x-coordinate of the center point of the predicted bounding box; y b represents the y-coordinate of the center point of the predicted bounding box; w Indicates the width of the predicted bounding box; b h Indicates the height of the predicted bounding box; (x c y c () represents the coordinates of the anchor frame center relative to its parent grid cell; w represents the width of the anchor frame; h represents the height of the anchor frame; (t) represents the coordinates of the anchor frame center relative to its parent grid cell; x , t y , t w , t h ) represents the predicted offset; e represents the exponential function;

[0032] Prediction of classification scores:

[0033] s i =softmax(W i ·f(x));

[0034] Among them, s i W represents the predicted class probability; i The weight parameter is represented by ; softmax() represents the activation function, which transforms a set of real numbers into a probability distribution; f(x) represents the feature map, which is the feature representation of the input x after a series of transformations.

[0035] As a further improvement to this technical solution, the specific steps of attribute detection in S2 are as follows:

[0036] The gender classifier outputs p gendnr ∈[0,1], where p gendnr A value greater than 0.5 indicates male; otherwise, it indicates female.

[0037] Age classifier output p age ∈[0,1], where p age A value greater than 0.5 indicates an adult; otherwise, it indicates a child.

[0038] The clothing color classifier outputs two probabilities p. bright p dark ∈[0,1],p bright >p dark Indicates a light color; otherwise, it is a dark color.

[0039] Does the hat classifier output p? hat ∈[0,1], where p hat A value greater than 0.5 indicates that a hat is present; otherwise, no hat is present.

[0040] As a further improvement to this technical solution, in step S3, the calculation of the confidence level of the fluorescent clothing attribute is specifically as follows:

[0041]

[0042] Where C represents the confidence level; X represents the category of fluorescent clothing; Y represents other clothing categories; P(X∩Y) represents the probability that X and Y occur simultaneously; and P(X) represents the probability that X occurs.

[0043] As a further improvement to this technical solution, in step S4, the detailed steps for determining whether a worker possesses fluorescent clothing attributes based on the confidence level of the fluorescent clothing attribute are as follows:

[0044] Let there be an amplitude threshold T. When the confidence level C of the fluorescent clothing attribute is greater than T, the worker has the fluorescent clothing attribute.

[0045]

[0046] Here, IsMovingPixel(C, T) represents a function that takes a confidence level C for the fluorescent clothing attribute and a threshold T as input, and outputs a Boolean value. If C > T, that is, the confidence level exceeds the set threshold, the function returns 1, indicating that the worker has the fluorescent clothing attribute; if C ≤ T, that is, the confidence level does not exceed or equal to the threshold, the function returns 0, indicating that the worker does not have the fluorescent clothing attribute.

[0047] As a further improvement to this technical solution, the TensorRT optimization in S5 includes:

[0048] Quantification:

[0049]

[0050] Where q represents the quantized integer value; round() represents the function that performs rounding; x represents the original floating-point number; S represents the scaling factor; and Z represents the zero point.

[0051] Inverse quantization:

[0052] x′=(qZ)×S;

[0053] Where x′ represents the floating-point number converted back from the quantized integer value;

[0054] Calculation of scaling factor and zero point:

[0055]

[0056] Among them, S act The scaling factor representing the activation value; max act The maximum value of the activation value; min act Z represents the minimum activation value. act Indicates the zero point of the activation value; N represents the quantization precision;

[0057] Batch normalized parameters are fused into convolutional weights:

[0058]

[0059] Among them, new weight Represents the new weight matrix; weight represents the weight matrix of the original convolutional layer; new bias γ represents the new bias term; bias represents the bias term of the original convolutional layer; γ represents the scaling factor of the batch normalization layer; β represents the translation factor of the batch normalization layer; ε represents a constant; mean represents the mean of the batch data calculated during training.

[0060] Computational cost after layer fusion:

[0061]

[0062] Among them, compute fused The computational cost after layer fusion is represented by M; the width of the input feature map is represented by M; and the height of the input feature map is represented by N. This indicates the size of the convolution kernel.

[0063] As a further improvement to this technical solution, in S5, EGLSink specifically refers to:

[0064]

[0065] Among them, Rendered Frame Output represents the rendered video frame. Frame This represents the video frame output from the inference module; This represents the detection results plotted on a video frame; This represents the final output after rendering using EGL Sink; eGL Sink This indicates that video frames will be rendered onto the display device.

[0066] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0067] 1. In this method for detecting fluorescent clothing based on image pedestrian attributes, the SSD human detection algorithm is used to detect pedestrians from the video stream. This method offers faster detection speed and higher accuracy. By predicting at multiple scales, SSD can detect targets of different sizes. The confidence level of the fluorescent clothing attribute is used to determine whether a worker is wearing fluorescent clothing, further improving detection accuracy and enabling more precise target detection.

[0068] 2. In this fluorescent clothing detection method based on pedestrian attributes in images, the results are integrated through the GstPlugin interface, and the segmentation model optimized by TensorRT is loaded through the nvinfer interface of DeepStream. The detection box and detection area are displayed in conjunction with OSD, and finally displayed on the desktop through EGLISink, which ensures good detection results, improves alarm efficiency, and avoids accidents. Attached Figure Description

[0069] Figure 1 This is a flowchart of the overall method of the present invention. Detailed Implementation

[0070] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] Example:

[0072] Please see Figure 1 As shown, this embodiment provides a fluorescent clothing detection method based on pedestrian attributes in images, including the following steps:

[0073] S1. Use a camera to capture real-time video stream, use an H264 decoder combined with GPU hardware acceleration to decode the video stream, and use an SSD human detection algorithm to detect pedestrians from the video stream.

[0074] In this example, the specific steps for decoding the video stream using an H.264 decoder combined with GPU hardware acceleration are as follows:

[0075] S2.1 First, you need to create an H264 decoder instance and specify the format of the input data for it;

[0076] The S2.2 H264 decoder will read compressed H264 data from the network and local files;

[0077] The S2.3 and H264 decoders will parse the NAL units in the input data and extract the encoded data of each image.

[0078] S2.4. The encoded data is obtained and decoded into the original image using GPU hardware acceleration technology;

[0079] GPU hardware acceleration technologies include:

[0080] Transform coding:

[0081]

[0082] Where X[u][v] represents the frequency domain coefficients obtained after DCT exchange; α(u)α(v) represents the normalization factor; x[x][y] represents the spatial domain element value of the original image; N represents the size of the image patch; and cos represents the cosine function.

[0083] Quantification:

[0084]

[0085] Where Q[u][v] represents the quantized frequency domain coefficients; round represents the rounding function; Qfacto r Indicates the quantification factor;

[0086] Inverse transform coding:

[0087]

[0088] Where, x[x][y] 逆 Y[u][v] represents the frequency domain coefficients obtained after inverse DCT transformation; Y[u][v] represents the frequency domain coefficients after quantization.

[0089] The S2.5 and H264 decoders will output the decoded image.

[0090] Specifically, an H.264 decoder is a software or hardware tool used to decode the H.264 video encoding format. H.264, also known as MPEG-4 Part 10 or AVC (Advanced Video Coding), is a widely used video compression standard that offers a high data compression ratio while maintaining good video quality. This makes H.264 the preferred video encoding format for applications such as digital television broadcasting, video conferencing, online video streaming, and multimedia messaging.

[0091] An H.264 decoder works by receiving H.264 encoded video data and converting it back into a visual video signal. The decoding process involves multiple steps, such as decompression, motion compensation, transform, and entropy coding, to reconstruct the original video content. While the H.264 standard defines the decoding method, it allows manufacturers the freedom to develop competitive decoders, provided they achieve the same results as those used in the standard.

[0092] In this example, the SSD human detection algorithm includes:

[0093] Bounding box prediction:

[0094]

[0095] Among them, b x b represents the x-coordinate of the center point of the predicted bounding box; y b represents the y-coordinate of the center point of the predicted bounding box; w Indicates the width of the predicted bounding box; b h Indicates the height of the predicted bounding box; (x c y c () represents the coordinates of the anchor frame center relative to its parent grid cell; w represents the width of the anchor frame; h represents the height of the anchor frame; (t) represents the coordinates of the anchor frame center relative to its parent grid cell; x , t y , t w , t h ) represents the predicted offset; e represents the exponential function;

[0096] Prediction of classification scores:

[0097] s i =softmax(W i ·f(x));

[0098] Among them, s i W represents the predicted class probability; i The weight parameter is represented by ; softmax() represents the activation function, which transforms a set of real numbers into a probability distribution; f(x) represents the feature map, which is the feature representation of the input x after a series of transformations.

[0099] Specifically, SSD (Single Shot MultiBox Detector) is a popular object detection algorithm that can perform object detection and classification simultaneously in a single forward propagation. The core idea of ​​the SSD algorithm is to use pre-defined anchor boxes on feature maps at different levels to predict the bounding box and confidence score of the target.

[0100] S2. Perform attribute detection on the detected pedestrian images to obtain all attribute data of the pedestrians;

[0101] In this example, the specific steps for attribute detection are as follows:

[0102] The gender classifier outputs p gendnr ∈[0,1], where p gendnr A value greater than 0.5 indicates male; otherwise, it indicates female.

[0103] Age classifier output p age ∈[0,1], where p age A value greater than 0.5 indicates an adult; otherwise, it indicates a child.

[0104] The clothing color classifier outputs two probabilities p. bright p dark ∈[0,1],p bright >p dark Indicates a light color; otherwise, it is a dark color.

[0105] Does the hat classifier output p? hat ∈[0,1], where p hat A value greater than 0.5 indicates that a hat is present; otherwise, no hat is present.

[0106] S3. Find the fluorescent clothing attribute in the attribute data and calculate the confidence level of the fluorescent clothing attribute;

[0107] Among them, the attributes of fluorescent clothing include whether fluorescent strips are worn and whether fluorescent clothing is worn;

[0108] In this example, the confidence level of the fluorescent clothing attribute is calculated as follows:

[0109]

[0110] Where C represents the confidence level; X represents the category of fluorescent clothing; Y represents other clothing categories; P(X∩Y) represents the probability that X and Y occur simultaneously; and P(X) represents the probability that X occurs.

[0111] S4. Determine whether a worker has fluorescent clothing based on the confidence level of the fluorescent clothing attribute;

[0112] In this example, the detailed steps for determining whether a worker wears fluorescent clothing based on the confidence level of the fluorescent clothing attribute are as follows:

[0113] Let there be an amplitude threshold T. When the confidence level C of the fluorescent clothing attribute is greater than T, the worker has the fluorescent clothing attribute.

[0114]

[0115] Here, IsMovingPixel(C, T) represents a function that takes a confidence level C for the fluorescent clothing attribute and a threshold T as input, and outputs a Boolean value. If C > T, that is, the confidence level exceeds the set threshold, the function returns 1, indicating that the worker has the fluorescent clothing attribute; if C ≤ T, that is, the confidence level does not exceed or equal to the threshold, the function returns 0, indicating that the worker does not have the fluorescent clothing attribute.

[0116] S5. The results are integrated through the GstPlugin interface, and the pedestrian attribute model accelerated and optimized by TensorRT is loaded through the nvinfer interface of DeepStream. The detection boxes and detection areas are displayed in conjunction with OSD, and finally displayed on the desktop through EGLISink.

[0117] In this example, TensorRT optimizations include:

[0118] Quantification:

[0119]

[0120] Where q represents the quantized integer value; round() represents the function that performs rounding; x represents the original floating-point number; S represents the scaling factor; and Z represents the zero point.

[0121] Inverse quantization:

[0122] x′=(qZ)×S;

[0123] Where x′ represents the floating-point number converted back from the quantized integer value;

[0124] Calculation of scaling factor and zero point:

[0125]

[0126] Among them, S act The scaling factor representing the activation value; max act The maximum value of the activation value; min act Z represents the minimum activation value. act Indicates the zero point of the activation value; N represents the quantization precision;

[0127] Batch normalized parameters are fused into convolutional weights:

[0128]

[0129] Among them, new weight Represents the new weight matrix; weight represents the weight matrix of the original convolutional layer; new biasγ represents the new bias term; bias represents the bias term of the original convolutional layer; γ represents the scaling factor of the batch normalization layer; β represents the translation factor of the batch normalization layer; ε represents a constant; mean represents the mean of the batch data calculated during training.

[0130] Computational cost after layer fusion:

[0131]

[0132] Among them, compute fused The computational cost after layer fusion is represented by M; the width of the input feature map is represented by M; and the height of the input feature map is represented by N. This indicates the size of the convolution kernel.

[0133] In this example, EGLSink is specifically:

[0134]

[0135] Among them, Rendered Frame Output represents the rendered video frame. Frame This represents the video frame output from the inference module; This represents the detection results plotted on a video frame; This represents the final output after rendering using EGL Sink; eGL Sink This indicates that video frames will be rendered onto the display device.

[0136] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting fluorescent clothing based on pedestrian attributes in images, characterized in that: Includes the following steps: S1. Use a camera to capture real-time video stream, use an H264 decoder combined with GPU hardware acceleration to decode the video stream, and use an SSD human detection algorithm to detect pedestrians from the video stream. S2. Perform attribute detection on the detected pedestrian images to obtain all attribute data of the pedestrians; S3. Find the fluorescent clothing attribute in the attribute data and calculate the confidence level of the fluorescent clothing attribute; Among them, the attributes of fluorescent clothing include whether fluorescent strips are worn and whether fluorescent clothing is worn; S4. Determine whether a worker has fluorescent clothing based on the confidence level of the fluorescent clothing attribute; S5. The results are integrated through the GstPlugin interface, and the pedestrian attribute model accelerated and optimized by TensorRT is loaded through the nvinfer interface of DeepStream. The detection boxes and detection areas are displayed in conjunction with OSD, and finally displayed on the desktop through EGLISink.

2. The fluorescent clothing detection method based on pedestrian attributes in an image according to claim 1, characterized in that: In step S1, the specific steps for decoding the video stream using an H.264 decoder combined with GPU hardware acceleration are as follows: S2.1 First, you need to create an H264 decoder instance and specify the format of the input data for it; The S2.2 H264 decoder will read compressed H264 data from the network and local files; The S2.3 and H264 decoders will parse the NAL units in the input data and extract the encoded data of each image. S2.

4. The encoded data is obtained and decoded into the original image using GPU hardware acceleration technology; The S2.5 and H264 decoders will output the decoded image.

3. The fluorescent clothing detection method based on pedestrian attributes in an image according to claim 1, characterized in that: In S1, the SSD human detection algorithm includes: Bounding box prediction: Among them, b x b represents the x-coordinate of the center point of the predicted bounding box; y b represents the y-coordinate of the center point of the predicted bounding box; w Indicates the width of the predicted bounding box; b h Indicates the height of the predicted bounding box; (x c y c () represents the coordinates of the anchor frame center relative to its parent grid cell; w represents the width of the anchor frame; h represents the height of the anchor frame; (t) represents the coordinates of the anchor frame center relative to its parent grid cell; x , t y , t w , t h ) represents the predicted offset; e represents the exponential function; Prediction of classification scores: s i =softmax(W i ·f(x)); Among them, s i W represents the predicted class probability. i The weight parameter is represented by ; softmax() represents the activation function, which transforms a set of real numbers into a probability distribution; f(x) represents the feature map, which is the feature representation of the input x after a series of transformations.

4. The fluorescent clothing detection method based on pedestrian attributes in an image according to claim 1, characterized in that: In step S2, the specific steps for attribute detection are as follows: The gender classifier outputs p gendnr ∈[0,1], where p gendnr A value greater than 0.5 indicates male; otherwise, it indicates female. The age classifier outputs page∈[0,1], where page>0.5 represents an adult, otherwise it represents a child; The clothing color classifier outputs two probabilities p. bright p dark ∈[0,1],p bright >p dark Indicates a light color; otherwise, it is a dark color. Does the hat classifier output pha? t ∈[0,1], where pha t A value greater than 0.5 indicates that a hat is present; otherwise, no hat is present.

5. The fluorescent clothing detection method based on pedestrian attributes in an image according to claim 1, characterized in that: In step S3, the confidence level of the fluorescent clothing attribute is calculated as follows: Where C represents the confidence level; X represents the category of fluorescent clothing; Y represents other clothing categories; P(X ∩ Y) represents the probability that X and Y occur simultaneously; and P(X) represents the probability that X occurs.

6. The fluorescent clothing detection method based on pedestrian attributes in an image according to claim 1, characterized in that: In step S4, the detailed steps for determining whether a worker possesses fluorescent clothing attributes based on the confidence level of the fluorescent clothing attribute are as follows: Let there be an amplitude threshold T. When the confidence level C of the fluorescent clothing attribute is greater than T, the worker has the fluorescent clothing attribute. Here, IsMovingPixel(C, T) represents a function that takes a confidence level C for the fluorescent clothing attribute and a threshold T as input, and outputs a Boolean value. If C > T, that is, the confidence level exceeds the set threshold, the function returns 1, indicating that the worker has the fluorescent clothing attribute; if C ≤ T, that is, the confidence level does not exceed or equal to the threshold, the function returns 0, indicating that the worker does not have the fluorescent clothing attribute.

7. The fluorescent clothing detection method based on pedestrian attributes in an image according to claim 1, characterized in that: In S5, TensorRT optimization includes: Quantification: Where q represents the quantized integer value; round() represents the function that performs rounding; x represents the original floating-point number; S represents the scaling factor; and Z represents the zero point. Inverse quantization: x′=(qZ)×S; Where x′ represents the floating-point number converted back from the quantized integer value; Calculation of scaling factor and zero point: Among them, S act The scaling factor representing the activation value; max act This represents the maximum value of the activation value; min act Z represents the minimum activation value. act Indicates the zero point of the activation value; N represents the quantization precision; Batch normalized parameters are fused into convolutional weights: Among them, new weight Represents the new weight matrix; weight represents the weight matrix of the original convolutional layer; new bias γ represents the new bias term; bias represents the bias term of the original convolutional layer; γ represents the scaling factor of the batch normalization layer; β represents the translation factor of the batch normalization layer; ε represents a constant; mean represents the mean of the batch data calculated during training. Computational cost after layer fusion: Among them, compute fused The computational cost after layer fusion is represented by M; the width of the input feature map is represented by M; and the height of the input feature map is represented by N. This indicates the size of the convolution kernel.

8. The fluorescent clothing detection method based on pedestrian attributes in an image according to claim 1, characterized in that: In S5, EGLSink specifically refers to: Among them, Rendered Frame Output represents the rendered video frame. Frame This represents the video frame output from the inference module; This represents the detection results plotted on a video frame; This represents the final output after rendering using EGL Sink; Sink This indicates that video frames will be rendered onto the display device.

Images