Clothing feature recognition method, training method and device of recognition model

Abstract

The application provides a dressing feature recognition method and a training method and device of a recognition model, and relates to the field of image processing.The method comprises the following steps: acquiring camera video stream data; performing data preprocessing on the camera video stream data to obtain preprocessed key frame image data; the data preprocessing comprises the following steps: extracting key frame image data of the camera video stream data, and performing contrast enhancement processing on the key frame image data; performing pedestrian detection and face recognition on the preprocessed key frame image data to obtain human body coordinate information of a target person; inputting the human body coordinate information of the target person into a dressing recognition model to output dressing features of the target person; and performing feature matching on the dressing features of the target person and dressing features in a database to obtain a matching result.In the application, dressing information of a specific employee is quickly and accurately detected and distinguished, and work area information of a specific person is effectively obtained, thereby improving work efficiency.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to a method for recognizing clothing features, a training method for a recognition model, and an apparatus. Background Technology

[0002] With the emergence of fintech and smart branches, video surveillance has been widely applied in business outlets to monitor and inspect the internal conditions of bank branches. However, the massive amount of video surveillance streams means that when important events or problems occur, a significant amount of manpower and resources are needed to search for effective information. One such application is the identification of bank employees' attire, which aims to ensure compliance. Since a large number of pedestrians enter and exit the branch, the ability to automatically and effectively identify branch employees from among the numerous pedestrians in the video footage would significantly improve the efficiency of video surveillance personnel. Furthermore, this research is of great significance for addressing issues such as specific pedestrian search, pedestrian behavior analysis, and video retrieval.

[0003] Traditional methods mainly rely on HOG (Histogram of Oriented Gradient) and SVM (Support Vector Machine) operators to achieve pedestrian detection and clothing classification, and achieve segmentation based on clothing color histograms by constructing appearance segmentation models. However, there are certain limitations. For example, the resolution of video images captured by video surveillance is not high, which makes it difficult to identify some fine-grained features in the algorithm. Also, the appearance features of the target will change greatly under complex scene conditions such as drastic changes in lighting, different viewing angles, and different cameras. Summary of the Invention

[0004] This invention provides a method and apparatus for clothing feature recognition, which addresses the challenges of sophisticated recognition in existing technologies and the significant changes in target appearance features under complex scene conditions.

[0005] To solve the above problems, the present invention is implemented as follows:

[0006] In a first aspect, the present invention provides a method for identifying clothing features, comprising:

[0007] Acquire camera video stream data;

[0008] The camera video stream data is preprocessed to obtain preprocessed keyframe image data; the data preprocessing includes: extracting keyframe image data from the camera video stream data and performing contrast enhancement processing on the keyframe image data;

[0009] Pedestrian detection and face recognition are performed on the preprocessed keyframe image data to obtain the human body coordinate information of the target person;

[0010] Input the human body coordinate information of the target person into the clothing recognition model, and output the clothing features of the target person;

[0011] The clothing characteristics of the target person are matched with the clothing characteristics in the database to obtain the matching result.

[0012] Optionally, the step of performing pedestrian detection and face recognition on the preprocessed keyframe image data to obtain the human coordinate information of the target person includes:

[0013] Pedestrian detection is performed on the preprocessed keyframe image data to obtain pedestrian coordinate information;

[0014] Face recognition is performed on the preprocessed keyframe image data to obtain the face coordinate information of at least one target person;

[0015] Based on the pedestrian coordinate information and the face coordinate information, the overlap is calculated. When the overlap meets a preset threshold, the human body coordinate information of the target person is output.

[0016] Optionally, the step of inputting the target person's body coordinate information into the clothing recognition model and outputting the target person's clothing features includes:

[0017] The human body coordinate information of the target person is converted from the current color space to at least one target color space to obtain the converted image data;

[0018] The converted image data is input into the first network to obtain high-frequency features and low-frequency features; the high-frequency features are those with color changes greater than a preset range; the low-frequency features are those with color changes less than or equal to a preset range.

[0019] The high-frequency features and the low-frequency features are input into the second network to perform feature combination, resulting in the combined high-frequency features and the combined low-frequency features.

[0020] The combined high-frequency features and the combined low-frequency features are integrated to obtain the clothing features of the target person.

[0021] Optionally, the color space includes at least one of the following: RGB color space, YUV color space, HSV color space, YCbCr color space, and YDbDr color space.

[0022] Secondly, the present invention provides a training method for a clothing feature recognition model, comprising:

[0023] Acquire camera video stream data samples; divide the camera video stream data samples into training set data and test set data according to a preset ratio;

[0024] The camera video stream data samples are preprocessed to obtain preprocessed keyframe image data; the data preprocessing includes: extracting keyframe image data from the camera video stream data samples and performing contrast enhancement processing on the keyframe image data;

[0025] Pedestrian detection and face recognition are performed on the preprocessed keyframe image data to obtain the human body coordinate information of the target person;

[0026] The keyframe image data corresponding to the training set data is input into the clothing recognition model for model training to obtain the training results;

[0027] The parameters of the clothing recognition model are adjusted based on the training results;

[0028] The keyframe image data corresponding to the test set data is input into the clothing recognition model to verify the output of the clothing recognition model, and the verification result is obtained.

[0029] Based on the verification results, the parameters of the clothing recognition model are adjusted again to obtain the trained clothing recognition model.

[0030] Optionally, the step of inputting the training set data into the clothing recognition model for model training to obtain the training result includes:

[0031] The human body coordinate information of the target person is converted from the current color space to at least one target color space to obtain the converted image data;

[0032] The converted image data is input into the first network to obtain high-frequency features and low-frequency features; the high-frequency features are those with color changes greater than a preset range; the low-frequency features are those with color changes less than or equal to a preset range.

[0033] The high-frequency features and the low-frequency features are input into the second network to perform feature combination, resulting in the combined high-frequency features and the combined low-frequency features.

[0034] The combined high-frequency features and the combined low-frequency features are integrated to obtain the clothing features of the target person.

[0035] Optionally, the color space includes at least one of the following: RGB color space, YUV color space, HSV color space, YCbCr color space, and YDbDr color space.

[0036] Thirdly, the present invention provides a clothing feature recognition device, comprising:

[0037] The first acquisition module is used to acquire camera video stream data;

[0038] The first data preprocessing module is used to preprocess the camera video stream data to obtain preprocessed keyframe image data; the data preprocessing includes: extracting keyframe image data from the camera video stream data and performing contrast enhancement processing on the keyframe image data;

[0039] The first processing module is used to perform pedestrian detection and face recognition on the preprocessed keyframe image data to obtain the human body coordinate information of the target person.

[0040] The first clothing recognition module is used to input the human body coordinate information of the target person into the clothing recognition model and output the clothing features of the target person.

[0041] The first matching module is used to perform feature matching between the clothing characteristics of the target person and the clothing characteristics in the database to obtain the matching result.

[0042] Optionally, the first processing module includes:

[0043] The pedestrian detection submodule is used to detect pedestrians from the preprocessed keyframe image data and obtain pedestrian coordinate information.

[0044] The face recognition submodule is used to perform face recognition on the preprocessed keyframe image data to obtain the face coordinate information of at least one target person.

[0045] The calculation submodule is used to calculate the overlap based on the pedestrian coordinate information and the face coordinate information. When the overlap meets a preset threshold, it outputs the human body coordinate information of the target person.

[0046] Optionally, the first clothing recognition module includes:

[0047] The conversion submodule is used to convert the human body coordinate information of the target person from the current color space to at least one target color space to obtain the converted image data;

[0048] The first processing submodule is used to input the converted image data into the first network to obtain high-frequency features and low-frequency features; the high-frequency features are features with color changes greater than a preset range; the low-frequency features are features with color changes less than or equal to a preset range.

[0049] The feature combination submodule is used to input the high-frequency features and the low-frequency features into the second network to perform feature combination, so as to obtain the combined high-frequency features and the combined low-frequency features;

[0050] The integration submodule is used to integrate the combined high-frequency features and the combined low-frequency features to obtain the clothing features of the target person.

[0051] Optionally, the color space includes at least one of the following: RGB color space, YUV color space, HSV color space, YCbCr color space, and YDbDr color space.

[0052] Fourthly, the present invention provides a training apparatus for a clothing feature recognition model, comprising:

[0053] The second processing module is used to acquire camera video stream data samples; and to divide the camera video stream data samples into training set data and test set data according to a preset ratio.

[0054] The second data preprocessing data is used to preprocess the camera video stream data samples to obtain preprocessed keyframe image data; the data preprocessing includes: extracting keyframe image data from the camera video stream data samples and performing contrast enhancement processing on the keyframe image data.

[0055] The third processing module is used to perform pedestrian detection and face recognition on the preprocessed keyframe image data to obtain the human body coordinate information of the target person.

[0056] The training module is used to input the keyframe image data corresponding to the training set into the clothing recognition model for model training and to obtain the training results.

[0057] The first parameter adjustment module is used to adjust the parameters of the clothing recognition model according to the training results;

[0058] The verification module is used to input the keyframe image data corresponding to the test set data into the clothing recognition model to verify the output result of the clothing recognition model and obtain the verification result.

[0059] The second parameter adjustment module is used to adjust the parameters of the clothing recognition model again based on the verification results to obtain the trained clothing recognition model.

[0060] Optionally, the step of inputting the training set data into the clothing recognition model for model training to obtain the training result includes:

[0061] The human body coordinate information of the target person is converted from the current color space to at least one target color space to obtain the converted image data;

[0062] The converted image data is input into the first network to obtain high-frequency features and low-frequency features; the high-frequency features are those with color changes greater than a preset range; the low-frequency features are those with color changes less than or equal to a preset range.

[0063] The high-frequency features and the low-frequency features are input into the second network to perform feature combination, resulting in the combined high-frequency features and the combined low-frequency features.

[0064] The combined high-frequency features and the combined low-frequency features are integrated to obtain the clothing features of the target person.

[0065] Optionally, the color space includes at least one of the following: RGB color space, YUV color space, HSV color space, YCbCr color space, and YDbDr color space.

[0066] Fifthly, the present invention provides a server comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, it implements the steps of the clothing feature recognition method as described in any one of the first aspects, or, when the program is executed by the processor, it implements the steps of the training method for the clothing feature recognition model as described in any one of the second aspects.

[0067] In a sixth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein when executed by a processor, the computer program implements the steps of the clothing feature recognition method as described in any one of the first aspects, or, when executed by the processor, the program implements the steps of the training method for the clothing feature recognition model as described in any one of the second aspects.

[0068] In this invention, data is collected from target individuals entering the camera's capture area. The pre-processed data images are then used for facial recognition and pedestrian recognition to confirm the identity of specific individuals. The collected pedestrian data images are then transformed in color space and transmitted to a model network for frequency domain feature extraction and output. Finally, the data is compared with clothing feature information in the database. This allows for the rapid and accurate detection and identification of specific employees' clothing information, timely and effective acquisition of their work area information, and provides effective data support for employee management, thereby improving work efficiency. Attached Figure Description

[0069] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0070] Figure 1 A flowchart illustrating a clothing feature recognition method provided in an embodiment of the present invention;

[0071] Figure 2 A schematic diagram of the overall process of a clothing feature recognition method provided in an embodiment of the present invention;

[0072] Figure 3 A schematic diagram of the overall process of another clothing feature recognition method provided in an embodiment of the present invention;

[0073] Figure 4 This is a schematic diagram of the clothing recognition model network structure of a clothing feature recognition method provided in an embodiment of the present invention;

[0074] Figure 5 This is a schematic diagram of the structure of the first network block in the clothing recognition model network of a clothing feature recognition method provided in an embodiment of the present invention;

[0075] Figure 6 This is a schematic diagram of the structure of the second network in the clothing recognition model network of a clothing feature recognition method provided in an embodiment of the present invention;

[0076] Figure 7 This is a schematic diagram of the overall device structure of a clothing feature recognition method provided in an embodiment of the present invention;

[0077] Figure 8 A schematic diagram of the overall device structure of another clothing feature recognition method provided in an embodiment of the present invention;

[0078] Figure 9 A flowchart illustrating a training method for a clothing feature recognition model provided in an embodiment of the present invention;

[0079] Figure 10 This is a schematic diagram of the structure of a clothing feature recognition device provided in an embodiment of the present invention;

[0080] Figure 11 A schematic diagram of the structure of a training device for a clothing feature recognition model provided in an embodiment of the present invention;

[0081] Figure 12 This is a schematic diagram of the structure of a server provided in an embodiment of the present invention. Detailed Implementation

[0082] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0083] Please refer to Figure 1 and Figure 2 This invention provides a method for identifying clothing features, including:

[0084] Step 11: Obtain camera video stream data;

[0085] Step 12: Perform data preprocessing on the camera video stream data to obtain preprocessed keyframe image data; the data preprocessing includes: extracting keyframe image data from the camera video stream data and performing contrast enhancement processing on the keyframe image data;

[0086] Step 13: Perform pedestrian detection and face recognition on the preprocessed keyframe image data to obtain the human body coordinate information of the target person;

[0087] Step 14: Input the human body coordinate information of the target person into the clothing recognition model and output the clothing features of the target person;

[0088] Step 15: Perform feature matching between the clothing characteristics of the target person and the clothing characteristics in the database to obtain the matching result.

[0089] In this embodiment of the invention, data is collected from target individuals entering the area captured by network cameras. The pre-processed data images are then used for facial recognition and pedestrian recognition to confirm the identity of specific individuals. The collected pedestrian data images are then transformed in color space and transmitted to a model network for frequency domain feature extraction and output. Finally, the data is compared with clothing feature information in the database. This allows for the rapid and accurate detection and identification of specific employees' clothing information, timely and effective acquisition of their work area information, and provides effective data support for employee management, thereby improving work efficiency.

[0090] In this embodiment of the invention, in step 11, camera video stream data is acquired; wherein, the video stream data acquired by the camera video stream mainly consists of pedestrian behavior data. After the camera configuration is completed, video can be played and acquired by specifying the RTSP (Real Time Streaming Protocol) address.

[0091] In this embodiment of the invention, in step 12, since the acquired video stream data cannot be directly input into the model, it needs to be processed by frame extraction first. The video is usually composed of I-frames, P-frames and B-frames. This embodiment adopts the extraction method of extracting key I-frames and performs contrast enhancement processing.

[0092] In this embodiment of the invention, step 13, which involves performing pedestrian detection and face recognition on the preprocessed keyframe image data to obtain the human body coordinate information of the target person, includes: performing pedestrian detection on the preprocessed keyframe image data to obtain pedestrian coordinate information; performing face recognition on the preprocessed keyframe image data to obtain the face coordinate information of at least one target person; calculating the overlap based on the pedestrian coordinate information and the face coordinate information; and outputting the human body coordinate information of the target person when the overlap meets a preset threshold. The pedestrian detection algorithm used on the preprocessed keyframe image is YOLO. The v3 network accurately locates pedestrians in the image to be detected. It locates each pedestrian in the image and obtains multiple pedestrian coordinate data within the frame. For the preprocessed keyframe image, a face recognition algorithm is applied. The FaceNet face recognition network is used to output the coordinate information of the face of a specific person in the image to be detected. Each face in the image is located and multiple face coordinate information within the frame is obtained. Then, combined with the coordinate information output by the pedestrian recognition, the IOU overlap between the face coordinate frame and the pedestrian coordinate frame is calculated. When the overlap meets a preset threshold, the human body coordinate information of the specific person is output.

[0093] In this embodiment of the invention, step 14, inputting the human body coordinate information of the target person into the clothing recognition model and outputting the clothing features of the target person, includes: converting the human body coordinate information of the target person from the current color space to at least one target color space to obtain converted image data; inputting the converted image data into a first network to obtain high-frequency features and low-frequency features; the high-frequency features are features with color changes greater than a preset range; the low-frequency features are features with color changes less than or equal to a preset range; inputting the high-frequency features and the low-frequency features into a second network for feature combination to obtain combined high-frequency features and combined low-frequency features; integrating the combined high-frequency features and combined low-frequency features to obtain the clothing features of the target person; wherein, the clothing recognition model mainly consists of input data, color space transformation, a first network Block 1, and β second network Blocks. 2. The system consists of a fully connected (FC) layer and a feature connection layer. Based on coordinate information, it processes the original image to obtain an image with precise localization of a specific person, and then performs deep learning modeling. Since the main source of data is pedestrians under the camera, and the purpose of modeling is to achieve discriminative analysis of the clothing of a specific pedestrian, the output of the model is related to the color space of the image. Furthermore, since different frequency domains of the image have different characteristics in expressing feature information, a deep learning neural network model is used to identify the clothing of a specific pedestrian.

[0094] In this embodiment of the invention, in step 15, the clothing features of the target person are used to obtain the feature vector of the corresponding input image. Then, the feature vector is compared with the registered users in the database. Euclidean distance is used for calculation, and the value with the smallest distance is selected. If it is less than the set threshold, the match is successful; otherwise, the match fails. The setting of this threshold mainly limits the range and provides a specific recommended value. The specific value can be set as needed in actual use, which improves the user experience.

[0095] In this embodiment of the invention, optionally, the color space includes at least one of the following: RGB color space, YUV color space, HSV color space, YCbCr color space, and YDbDr color space.

[0096] Please refer to Figure 2In this embodiment of the invention, firstly, a camera video stream is acquired; the video stream data acquired by the camera mainly consists of pedestrian behavior data; secondly, data preprocessing is performed; frame extraction is performed, where the video is usually composed of I-frames, P-frames, and B-frames. This embodiment uses an extraction method that extracts key I-frames and performs contrast enhancement processing; thirdly, face recognition and pedestrian recognition are performed on the preprocessed key frame image data to obtain specific human coordinate information, i.e., the human coordinate information of the target person; fourthly, the specific human coordinate information and the original image of the preprocessed key frame are input into a clothing recognition model to determine whether the person is wearing clothing; finally, the clothing features of the target person are output and stored in the database.

[0097] Please refer to Figure 3 In this embodiment of the invention, in S1, acquiring video stream data from cameras, the dataset mainly consists of data collected by cameras within the business outlet, from which video stream data from 10 cameras with an average duration of 10 minutes is obtained. This dataset can be changed according to different user needs. In S2, video stream data preprocessing, the camera video stream data is preprocessed using the H.264 encoding format. The video stream is decomposed into I-frames, P-frames, and B-frames. The keyframe I-frame is extracted, and gamma correction is used to adjust image contrast. In S3, acquiring specific pedestrian image data, the FaceNet face recognition algorithm and the YOLOv3 pedestrian detection algorithm are used to accurately locate the coordinates of the specified pedestrians. Based on the specific coordinates in the image, a screenshot of the specific pedestrian is obtained and transmitted to the clothing recognition model. The face recognition algorithm and pedestrian detection algorithm can be changed according to actual conditions. In the S4 deep learning-based network model construction, the clothing recognition model mainly consists of input data, color space transformation, the first network Block 1, β second network Block 2, FC fully connected layers, and feature connection layers. The input data is mainly image data of specific pedestrians. It is video stream data that has undergone the above steps, frame extraction and data augmentation, and then specific target localization is completed through face recognition and pedestrian detection. Based on coordinate information, the specific pedestrian image is extracted from a single frame image.

[0098] The clothing recognition model is constructed by combining existing technological advancements and improving and optimizing the model based on image data enhancement and frequency domain information extraction. It considers the influence of factors such as light intensity within the network points, transforming the image color gamut to reduce the impact of illumination on the image itself. This allows the model to learn more data feature information and improves its robustness. Simultaneously, improvements are made to the original feature extraction method by introducing frequency domain feature information extraction. This addresses the issue of low spatial resolution, where pedestrian data collected by high-resolution cameras within the network points typically constitutes a small proportion of the overall image. Furthermore, considering computational cost and memory constraints, a holistic optimization of the network model is proposed. Fine-grained detail coding is used to extract higher frequencies, while global structure coding is used to extract lower frequencies, simultaneously reducing the overall number of network parameters and accelerating network training and inference.

[0099] Please refer to Figures 4-6 For the specific network structure of the clothing recognition model in this embodiment of the invention, please refer to... Figure 4 First, Input Image takes the image data as input; then, Gamut Transformation is used to transform the color space of the image, converting the GRB image into RGB, YUV, HSV, YCbCr, and YDbDr images.

[0100] The calculation formulas for RGB to YUV conversion are: Y = 0.299R + 0.587G + 0.114B; U = -0.147R - 0.289G + 0.436B; V = 0.615R - 0.515G - 0.100B. The calculation formula for RGB to HSV conversion is: let max be the largest of r, g, and b; min be the smallest of r, g, and b; h be between 0 and 360°; s be between 0 and 100%; and v be between 0 and max. The formula for converting RGB to YCbCr is: The formula for converting RGB to YDbDr is: After obtaining images in different color spaces (Multi-color space), the corresponding images are processed using the first network block 1; please refer to the organization structure of the first network block 1. Figure 5The convolutional kernels are all 3x3 with a stride of 1, and the activation function used is leaky ReLU. After concatenating features from different color spaces, high-frequency features and low-frequency features are extracted separately. An experimental α value of 0.25 is set to indicate a greater focus on low-frequency information. The output features are then input into the second network, Block 2. Please refer to [reference needed]. Figure 6 The second network, Block 2, consists of β basic High-low Blocks with the same structure (β=3 in this embodiment). The outputs of the High and Low modules are combined, with the convolution kernel size being 3*3 and the activation function being leaky_ReLU, to obtain the combined high-frequency features and combined low-frequency features. The FC layer consists of two fully connected layers with 256 nodes and n_class nodes respectively. After the output of Block 2, the feature combination is completed. The clothing recognition model also includes inputting the combined results into the final fully connected layer, and finally the model outputs the final recognition result, completing the classification result for specific pedestrian clothing. The final recognition result includes the recognition result of the feature recognition probability.

[0101] To ensure the accuracy of the clothing recognition model, in step S5, specific pedestrian data images are acquired, and the model is trained using a deep neural network. Based on the NVIDIA v100 chip and using the Keras framework, the video stream data undergoes frame extraction and data augmentation. Then, a self-developed pedestrian detection algorithm is used to extract pedestrian data images, resulting in approximately 20,000 pedestrian data images. These pedestrian image data are divided into a training set, training set labels, a test set, and a test set labels. The ratio of the training set to the test set is set to 8:2, but can be freely adjusted according to actual conditions. The training set serves as the input to the model in step S4, enabling model training. The training and optimization process uses a test set to validate the overall model performance. During training, backpropagation and Adam optimization algorithms are employed. The weights of each layer are iteratively updated based on the forward propagation loss value. An early-stopping strategy is used; the training model is saved when the loss value reaches its minimum. In the S6 clothing feature output and comparison, the output clothing features are converted into clothing feature vectors and matched against registered clothing feature vectors in the database. Euclidean distance is used for calculation; if the minimum distance is less than a set threshold, the match is successful, and identity verification is passed.

[0102] Please refer to Figure 7This invention also provides a clothing recognition device, comprising: a camera activation module 01 for acquiring the user's initial input selection, indicating whether the camera function needs to be activated; a video stream acquisition module 02 for acquiring video stream data from the camera on a side device and uploading it to the cloud; a data preprocessing module 03 for preprocessing the acquired video stream data, including video frame extraction and contrast enhancement; an AI algorithm module 04 for combining face recognition and pedestrian detection algorithms to accurately locate specific pedestrians, then using a trained AI (Artificial Intelligence) network model to classify and identify the processed pedestrian image data and output corresponding results; a threshold modification module 05 for modifying the threshold according to user preferences, increasing or decreasing the strength of the judgment; and a result storage module 06 for storing the model's output results in a specified database.

[0103] Please refer to Figure 8 This invention also provides a clothing recognition device, including an input section, an output section, a communication bus, a CPU, a GPU, a storage section, and a driver. The input section includes a camera with video capture capabilities for collecting pedestrian data within a specified space. The output section can be a single piece of pedestrian clothing data from a database, used to query and display the clothing records of a specific pedestrian. The communication bus is mainly used for communication between modules, including an SPI communication bus and an I2C communication bus. The CPU is a central processing unit used for core algorithm program execution and logic processing, recognizing input user behavior data and user input operations. The GPU is a graphics processing unit used for performing related calculations according to program logic algorithms, completing standard functions for input user behavior data, storing the processed results in the storage module, and sending them to the output end for display via the communication bus. The storage unit stores the clothing characteristics of specific pedestrians and model output results from different time periods. The driver provides underlying driving functions to ensure the smooth operation of upper-layer applications.

[0104] In this embodiment of the invention, keyframe data is extracted by frame extraction processing of acquired video stream data. Combined with face recognition and pedestrian detection algorithms, the coordinate information of a specific pedestrian is calibrated and located. Clothing is analyzed and judged using a clothing recognition model. The method and steps for extracting clothing feature information and outputting the category determination are completed through frequency domain data information. By extracting video keyframes and transforming multiple color spaces, a method to reduce interference from external factors is achieved, enabling rapid and accurate detection and judgment of the clothing information of a specific employee. It also allows for timely and effective acquisition of the work area information of a specific person, providing effective data support for employee management and improving work efficiency.

[0105] Please refer to Figure 9 This invention provides a training method for a clothing feature recognition model, comprising:

[0106] Step 91: Obtain camera video stream data samples; divide the camera video stream data samples into training set data and test set data according to a preset ratio;

[0107] Step 92: Perform data preprocessing on the camera video stream data sample to obtain preprocessed keyframe image data; the data preprocessing includes: extracting keyframe image data from the camera video stream data sample and performing contrast enhancement processing on the keyframe image data;

[0108] Step 93: Perform pedestrian detection and face recognition on the preprocessed keyframe image data to obtain the human body coordinate information of the target person;

[0109] Step 94: Input the keyframe image data corresponding to the training set data into the clothing recognition model for model training to obtain the training results;

[0110] Step 95: Adjust the parameters of the clothing recognition model based on the training results;

[0111] Step 96: Input the keyframe image data corresponding to the test set data into the clothing recognition model to verify the output of the clothing recognition model and obtain the verification result;

[0112] Step 97: Adjust the parameters of the clothing recognition model again based on the verification results to obtain the trained clothing recognition model.

[0113] In this embodiment of the invention, to ensure the accuracy of the clothing recognition model, a deep neural network is trained on the model. Based on the NVIDIA v100 chip, the Keras framework is used to perform frame extraction and data augmentation on the video stream data. Then, a self-developed pedestrian detection algorithm is used to extract pedestrian data images, resulting in approximately 20,000 pedestrian data images. The pedestrian image data is divided into a training set, training set labels, a test set, and a test set labels, with the ratio of the training set to the test set set set to 8:2, which can be freely set according to actual conditions. The training set serves as the input to the model in step S4 above to optimize the model training, while the test set is used to verify the overall performance of the model. During the model training process, the backpropagation algorithm and the Adam optimization algorithm are used. The weights of each layer are iteratively updated based on the forward propagation loss value. An early-stopping strategy is adopted, and the training model is saved when the model's loss value reaches the minimum value. The running speed of the algorithm model is introduced as a new variable to simultaneously measure the quality of the algorithm.

[0114] In this embodiment of the invention, optionally, the step of inputting the training set data into the clothing recognition model for model training to obtain the training result includes:

[0115] The human body coordinate information of the target person is converted from the current color space to at least one target color space to obtain the converted image data;

[0116] The converted image data is input into the first network to obtain high-frequency features and low-frequency features; the high-frequency features are those with color changes greater than a preset range; the low-frequency features are those with color changes less than or equal to a preset range.

[0117] The high-frequency features and the low-frequency features are input into the second network to perform feature combination, resulting in the combined high-frequency features and the combined low-frequency features.

[0118] The combined high-frequency features and the combined low-frequency features are integrated to obtain the clothing features of the target person.

[0119] In this embodiment of the invention, optionally, the color space includes at least one of the following: RGB color space, YUV color space, HSV color space, YCbCr color space, and YDbDr color space.

[0120] In this embodiment of the invention, the video stream data is processed by frame extraction and data enhancement, and then the pedestrian data image is extracted by a self-developed pedestrian detection algorithm. This ensures both the accuracy and real-time performance of the algorithm. In addition, the running rate of the algorithm model is introduced as a new variable to measure the quality of the algorithm in a synchronous manner, based on the original loss function.

[0121] Please refer to Figure 10 The present invention provides a clothing feature recognition device, comprising:

[0122] The first acquisition module 101 is used to acquire camera video stream data;

[0123] The first data preprocessing module 102 is used to preprocess the camera video stream data to obtain preprocessed keyframe image data; the data preprocessing includes: extracting keyframe image data from the camera video stream data and performing contrast enhancement processing on the keyframe image data.

[0124] The first processing module 103 is used to perform pedestrian detection and face recognition on the preprocessed keyframe image data to obtain the human body coordinate information of the target person.

[0125] The first clothing recognition module 104 is used to input the human body coordinate information of the target person into the clothing recognition model and output the clothing features of the target person.

[0126] The first matching module 105 is used to perform feature matching between the clothing characteristics of the target person and the clothing characteristics in the database to obtain a matching result.

[0127] In this embodiment of the invention, optionally, the first processing module includes:

[0128] The pedestrian detection submodule is used to detect pedestrians from the preprocessed keyframe image data and obtain pedestrian coordinate information.

[0129] The face recognition submodule is used to perform face recognition on the preprocessed keyframe image data to obtain the face coordinate information of at least one target person.

[0130] The calculation submodule is used to calculate the overlap based on the pedestrian coordinate information and the face coordinate information. When the overlap meets a preset threshold, it outputs the human body coordinate information of the target person.

[0131] In this embodiment of the invention, optionally, the first clothing recognition module includes:

[0132] The conversion submodule is used to convert the human body coordinate information of the target person from the current color space to at least one target color space to obtain the converted image data;

[0133] The first processing submodule is used to input the converted image data into the first network to obtain high-frequency features and low-frequency features; the high-frequency features are features with color changes greater than a preset range; the low-frequency features are features with color changes less than or equal to a preset range.

[0134] The feature combination submodule is used to input the high-frequency features and the low-frequency features into the second network to perform feature combination, so as to obtain the combined high-frequency features and the combined low-frequency features;

[0135] The integration submodule is used to integrate the combined high-frequency features and the combined low-frequency features to obtain the clothing features of the target person.

[0136] In this embodiment of the invention, optionally, the color space includes at least one of the following: RGB color space, YUV color space, HSV color space, YCbCr color space, and YDbDr color space.

[0137] The network-side device provided in this embodiment of the invention can achieve Figure 1 The various processes implemented by the clothing feature recognition method in the method embodiment will not be described again here to avoid repetition.

[0138] Please refer to Figure 11The present invention provides a training device for a clothing feature recognition model, comprising:

[0139] The second processing module 111 is used to acquire camera video stream data samples; and to divide the camera video stream data samples into training set data and test set data according to a preset ratio.

[0140] The second data preprocessing data 112 is used to preprocess the camera video stream data sample to obtain preprocessed keyframe image data; the data preprocessing includes: extracting keyframe image data from the camera video stream data sample and performing contrast enhancement processing on the keyframe image data.

[0141] The third processing module 113 is used to perform pedestrian detection and face recognition on the preprocessed keyframe image data to obtain the human body coordinate information of the target person.

[0142] Training module 114 is used to input the keyframe image data corresponding to the training set into the clothing recognition model for model training and obtain training results;

[0143] The first parameter adjustment module 115 is used to adjust the parameters of the clothing recognition model according to the training results;

[0144] Verification module 116 is used to input the keyframe image data corresponding to the test set data into the clothing recognition model to verify the output result of the clothing recognition model and obtain the verification result;

[0145] The second parameter adjustment module 117 is used to adjust the parameters of the clothing recognition model again based on the verification results to obtain the trained clothing recognition model.

[0146] In this embodiment of the invention, optionally, the step of inputting the training set data into the clothing recognition model for model training to obtain the training result includes:

[0147] The human body coordinate information of the target person is converted from the current color space to at least one target color space to obtain the converted image data;

[0148] The converted image data is input into the first network to obtain high-frequency features and low-frequency features; the high-frequency features are those with color changes greater than a preset range; the low-frequency features are those with color changes less than or equal to a preset range.

[0149] The high-frequency features and the low-frequency features are input into the second network to perform feature combination, resulting in the combined high-frequency features and the combined low-frequency features.

[0150] The combined high-frequency features and the combined low-frequency features are integrated to obtain the clothing features of the target person.

[0151] In this embodiment of the invention, optionally, the color space includes at least one of the following: RGB color space, YUV color space, HSV color space, YCbCr color space, and YDbDr color space.

[0152] The network-side device provided in this embodiment of the invention can achieve Figure 9 The various processes of training the clothing feature recognition model in the method embodiment will not be described again here to avoid repetition.

[0153] Please refer to Figure 4 The present invention also provides a server 120, including a processor 121, a memory 122, and a computer program stored in the memory 122 and executable on the processor 121. When the computer program is executed by the processor 121, it implements the various processes of the above-described embodiments of the clothing feature recognition method and the training method of the clothing feature recognition model, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0154] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described embodiments of the clothing feature recognition method and the training method of the clothing feature recognition model, achieving the same technical effects. To avoid repetition, these will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0155] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0156] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or other terminal, etc.) to execute the methods described in the various embodiments of the present invention.

[0157] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A method for recognizing clothing features, characterized in that, include: Acquire camera video stream data; The camera video stream data is preprocessed to obtain preprocessed keyframe image data; The data preprocessing includes: extracting keyframe image data from the camera video stream data, and performing contrast enhancement processing on the keyframe image data; Pedestrian detection and face recognition are performed on the preprocessed keyframe image data to obtain the human body coordinate information of the target person; Input the human body coordinate information of the target person into the clothing recognition model, and output the clothing features of the target person; The clothing characteristics of the target person are matched with the clothing characteristics in the database to obtain the matching result; The step of inputting the target person's human body coordinate information into the clothing recognition model and outputting the target person's clothing features includes: The human body coordinate information of the target person is converted from the current color space to at least one target color space to obtain the converted image data; The converted image data is input into the first network to obtain high-frequency features and low-frequency features; the high-frequency features are those with color changes greater than a preset range; the low-frequency features are those with color changes less than or equal to a preset range. The high-frequency features and the low-frequency features are input into a second network for feature combination, resulting in combined high-frequency features and combined low-frequency features. Specifically, the second network processes the high-frequency features using convolutional kernels to obtain a first high-frequency feature, and then performs average pooling and convolutional kernel processing on the high-frequency features to obtain a second high-frequency feature. Simultaneously, the low-frequency features are processed using convolutional kernels to obtain a first low-frequency feature, and then upsampled on the first low-frequency feature to obtain a second low-frequency feature. The first high-frequency feature and the second low-frequency feature are combined to obtain the combined high-frequency features; the second high-frequency feature and the first low-frequency feature are combined to obtain the combined low-frequency features. The combined high-frequency features and the combined low-frequency features are integrated to obtain the clothing features of the target person.

2. The clothing feature recognition method according to claim 1, characterized in that, Pedestrian detection and face recognition are performed on the preprocessed keyframe image data to obtain the human coordinate information of the target person, including: Pedestrian detection is performed on the preprocessed keyframe image data to obtain pedestrian coordinate information; Face recognition is performed on the preprocessed keyframe image data to obtain the face coordinate information of at least one target person; Based on the pedestrian coordinate information and the face coordinate information, the overlap is calculated. When the overlap meets a preset threshold, the human body coordinate information of the target person is output.

3. The clothing feature recognition method according to claim 1, characterized in that, The color space includes at least one of the following: RGB color space, YUV color space, HSV color space, YCbCr color space, and YDbDr color space.

4. A training method for a clothing feature recognition model, characterized in that, include: Obtain samples of camera video stream data; The camera video stream data samples are divided into training set data and test set data according to a preset ratio; The camera video stream data samples are preprocessed to obtain preprocessed keyframe image data; The data preprocessing includes: extracting keyframe image data from the camera video stream data samples, and performing contrast enhancement processing on the keyframe image data; Pedestrian detection and face recognition are performed on the preprocessed keyframe image data to obtain the human body coordinate information of the target person; The keyframe image data corresponding to the training set data is input into the clothing recognition model for model training to obtain the training results; The parameters of the clothing recognition model are adjusted based on the training results; The keyframe image data corresponding to the test set data is input into the clothing recognition model to verify the output of the clothing recognition model, and the verification result is obtained. Based on the verification results, the parameters of the clothing recognition model are adjusted again to obtain the trained clothing recognition model; The step of inputting the training set data into the clothing recognition model for model training to obtain the training result includes: The human body coordinate information of the target person is converted from the current color space to at least one target color space to obtain the converted image data; The converted image data is input into the first network to obtain high-frequency features and low-frequency features; the high-frequency features are those with color changes greater than a preset range; the low-frequency features are those with color changes less than or equal to a preset range. The high-frequency features and the low-frequency features are input into a second network for feature combination, resulting in combined high-frequency features and combined low-frequency features. Specifically, the second network processes the high-frequency features using convolutional kernels to obtain a first high-frequency feature, and then performs average pooling and convolutional kernel processing on the high-frequency features to obtain a second high-frequency feature. Simultaneously, the low-frequency features are processed using convolutional kernels to obtain a first low-frequency feature, and then upsampled on the first low-frequency feature to obtain a second low-frequency feature. The first high-frequency feature and the second low-frequency feature are combined to obtain the combined high-frequency features; the second high-frequency feature and the first low-frequency feature are combined to obtain the combined low-frequency features. The combined high-frequency features and the combined low-frequency features are integrated to obtain the clothing features of the target person.

5. The training method for the clothing feature recognition model according to claim 4, characterized in that, The color space includes at least one of the following: RGB color space, YUV color space, HSV color space, YCbCr color space, and YDbDr color space.

6. A clothing feature recognition device, characterized in that, include: The first acquisition module is used to acquire camera video stream data; The first data preprocessing module is used to preprocess the camera video stream data to obtain preprocessed keyframe image data. The data preprocessing includes: extracting keyframe image data from the camera video stream data, and performing contrast enhancement processing on the keyframe image data; The first processing module is used to perform pedestrian detection and face recognition on the preprocessed keyframe image data to obtain the human body coordinate information of the target person. The first clothing recognition module is used to input the human body coordinate information of the target person into the clothing recognition model and output the clothing features of the target person. The first matching module is used to perform feature matching between the clothing characteristics of the target person and the clothing characteristics in the database to obtain a matching result; The step of inputting the target person's human body coordinate information into the clothing recognition model and outputting the target person's clothing features includes: The human body coordinate information of the target person is converted from the current color space to at least one target color space to obtain the converted image data; The converted image data is input into the first network to obtain high-frequency features and low-frequency features; the high-frequency features are those with color changes greater than a preset range; the low-frequency features are those with color changes less than or equal to a preset range. The high-frequency features and the low-frequency features are input into a second network for feature combination, resulting in combined high-frequency features and combined low-frequency features. Specifically, the second network processes the high-frequency features using convolutional kernels to obtain a first high-frequency feature, and then performs average pooling and convolutional kernel processing on the high-frequency features to obtain a second high-frequency feature. Simultaneously, the low-frequency features are processed using convolutional kernels to obtain a first low-frequency feature, and then upsampled on the first low-frequency feature to obtain a second low-frequency feature. The first high-frequency feature and the second low-frequency feature are combined to obtain the combined high-frequency features; the second high-frequency feature and the first low-frequency feature are combined to obtain the combined low-frequency features. The combined high-frequency features and the combined low-frequency features are integrated to obtain the clothing features of the target person.

7. A training device for a clothing feature recognition model, characterized in that, include: The second processing module is used to acquire camera video stream data samples; The camera video stream data samples are divided into training set data and test set data according to a preset ratio; The second data preprocessing data is used to preprocess the camera video stream data samples to obtain preprocessed keyframe image data. The data preprocessing includes: extracting keyframe image data from the camera video stream data samples, and performing contrast enhancement processing on the keyframe image data; The third processing module is used to perform pedestrian detection and face recognition on the preprocessed keyframe image data to obtain the human body coordinate information of the target person. The training module is used to input the keyframe image data corresponding to the training set into the clothing recognition model for model training and to obtain the training results. The first parameter adjustment module is used to adjust the parameters of the clothing recognition model according to the training results; The verification module is used to input the keyframe image data corresponding to the test set data into the clothing recognition model to verify the output result of the clothing recognition model and obtain the verification result. The second parameter adjustment module is used to adjust the parameters of the clothing recognition model again based on the verification results to obtain the trained clothing recognition model. The step of inputting the training set data into the clothing recognition model for model training to obtain the training result includes: The human body coordinate information of the target person is converted from the current color space to at least one target color space to obtain the converted image data; The converted image data is input into the first network to obtain high-frequency features and low-frequency features; the high-frequency features are those with color changes greater than a preset range; the low-frequency features are those with color changes less than or equal to a preset range. The high-frequency features and the low-frequency features are input into a second network for feature combination, resulting in combined high-frequency features and combined low-frequency features. Specifically, the second network processes the high-frequency features using convolutional kernels to obtain a first high-frequency feature, and then performs average pooling and convolutional kernel processing on the high-frequency features to obtain a second high-frequency feature. Simultaneously, the low-frequency features are processed using convolutional kernels to obtain a first low-frequency feature, and then upsampled on the first low-frequency feature to obtain a second low-frequency feature. The first high-frequency feature and the second low-frequency feature are combined to obtain the combined high-frequency features; the second high-frequency feature and the first low-frequency feature are combined to obtain the combined low-frequency features. The combined high-frequency features and the combined low-frequency features are integrated to obtain the clothing features of the target person.

8. A server, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, it implements the steps of the clothing feature recognition method as described in any one of claims 1-3, or, when the program is executed by the processor, it implements the steps of the training method for the clothing feature recognition model as described in any one of claims 4 or 5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the clothing feature recognition method as described in any one of claims 1-3, or, when executed by the processor, implements the steps of the training method for the clothing feature recognition model as described in any one of claims 4 or 5.

Images