Pedestrian target optimal screening method for edge device customer behavior analysis
By collecting and analyzing pedestrian behavior picture data in real time on edge devices, using YOLOv5 and Hungarian algorithms for detection and matching, combining clarity and exposure prediction, the shortcomings of pedestrian picture quality analysis and screening in the existing technology are solved, and efficient , accurate pedestrian target screening and customer behavior analysis.
Patent Information
- Application Number
- CN202410998767.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-11-01
AI Technical Summary
The existing technology fails to effectively perform pedestrian picture quality analysis and optimal screening on edge devices in customer behavior analysis, resulting in a large consumption of network bandwidth and back-end computing resources, neglecting data privacy protection, high computing complexity and insufficient Considering the quality of pedestrian pictures, it leads to reduced analysis accuracy and misleading interpretation of behavior patterns.
Actual image data is collected in real time through the camera, and the detection box is matched by YOLOv5 detection box and the Hungarian algorithm is used to track pedestrian targets, identify the completeness of the calculation body, predict clarity and exposure, build a pose picture queue, calculate the picture quality score, and follow the quality score according to the quality score. Filter the optimal pedestrian target.
Efficient screening of pedestrian goals on edge devices is achieved, improving the accuracy and versatility of posture recognition, ensuring data privacy and security, reducing network bandwidth usage, and improving the accuracy of customer behavior analysis.
Smart Images

Figure CN118887701A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to an optimal screening method for pedestrian targets for edge device customer behavior analysis. Background Art
[0002] In commercial retail application scenarios, customer behavior analysis is of great significance. Through in-depth analysis of customer behavior, retailers can better understand customer needs, improve customer satisfaction, optimize product layout, improve marketing efficiency and enhance market competitiveness. In the context of digital transformation, making full use of artificial intelligence technology for customer behavior analysis has become the key for retailers to win the market. It can handle more complex feature extraction, better understand the context, have better generalization ability, support multi-task learning, integrate multimodal data, and have better adaptability and scalability.
[0003] At present, there are some shortcomings in the existing technologies when applied to customer behavior analysis. First, a large amount of computing resources and storage space are usually required. The calculation process of pedestrian image quality analysis and optimal screening is not performed on the edge device, resulting in a large amount of network bandwidth and back-end computing resources being occupied. Secondly, the importance of data privacy protection is ignored. The calculated images need to be uploaded to the external network for analysis, which cannot guarantee data privacy security. In addition, the current screening calculation process is simple and has high computational complexity, which is not suitable for edge devices with limited computing power. At the same time, the current screening algorithm fails to fully consider the quality requirements for pedestrian images. High-quality customer images should have representative postures and good imaging conditions, but existing technologies often ignore these key factors. As a result, the images uploaded to the back-end for in-depth analysis may reduce the analysis accuracy due to unrepresentative postures or poor imaging quality, or even mislead the interpretation of behavior patterns. Summary of the invention
[0004] In view of the problems existing in the above-mentioned existing optimal screening method for pedestrian targets for customer behavior analysis of edge devices, the present invention is proposed.
[0005] Therefore, the problem to be solved by the present invention is that since a large amount of computing resources and storage space are usually required, the calculation process of pedestrian image quality analysis and optimal screening is not performed on the edge device, resulting in a large amount of network bandwidth and back-end computing resources being occupied. Secondly, the importance of data privacy protection is ignored, and the calculated images need to be uploaded to the external network for analysis, which cannot guarantee data privacy security. In addition, the current screening calculation process is simple and has high computational complexity, which is not suitable for edge devices with limited computing power. At the same time, the current screening algorithm fails to fully consider the quality requirements for pedestrian images. High-quality customer images should have representative postures and good imaging conditions, but the existing technology often ignores these key factors. As a result, the images uploaded to the back-end for in-depth analysis may reduce the analysis accuracy due to unrepresentative postures or poor imaging quality, or even mislead the interpretation of behavioral patterns.
[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: an optimal screening method for pedestrian targets for edge device customer behavior analysis, which comprises:
[0007] Collect customer behavior image data in real time through the camera, and perform detection frame detection on the collected behavior image data;
[0008] Based on the detection results of the detection box, the detection box is matched to a complete pedestrian target through the Hungarian algorithm;
[0009] Based on the pedestrian target as the tracking target, the pedestrian behavior is tracked;
[0010] During the continuous tracking process, the computational body integrity of the behavioral image data is identified, and based on the body integrity score, the clarity and exposure of the behavioral image data are predicted to build a posture image queue;
[0011] Based on the posture image queue, calculate the image quality score, and construct an image quality optimization queue by arranging them in descending order of quality scores;
[0012] The optimal queue is selected based on the image quality, and the optimal behavior image data is selected to complete the optimal pedestrian target screening.
[0013] As a preferred solution of the pedestrian target optimal screening method for edge device customer behavior analysis described in the present invention, wherein: the customer's behavior picture data is collected in real time by a camera, and the collected behavior picture data is subjected to detection frame detection, including:
[0014] The customer behavior image data is collected through the camera, and the pedestrian image detection frame detection is performed using the YOLOv5 algorithm, including head detection frame, head and shoulder detection frame, and human body detection frame detection;
[0015] The head area, head-shoulder area, and whole body area of the human body image in the behavior picture data are set as the detection targets. The images in the behavior picture data are divided into S×S grids using the YOLOv5 algorithm. The grids are used as units, and the geometric center of the human body image is defined as the center point coordinates. The coordinate values of the detection targets are calculated based on the center point coordinates. The image data is classified and located according to the coordinate values of the head area, head-shoulder area, and whole body area.
[0016] The head target, head-shoulder target and human target area are surrounded by horizontal rectangular frames to form a head detection frame, a head-shoulder detection frame and a human detection frame.
[0017] As a preferred solution of the pedestrian target optimal screening method for edge device customer behavior analysis of the present invention, wherein: the detection frame is matched to a complete pedestrian target by the Hungarian algorithm based on the detection result of the detection frame, including:
[0018] The first match is performed based on the head detection frame and the head and shoulder detection frame, and the first match IoU value is calculated, which is expressed as:
[0019]
[0020] Among them, A is the head detection frame, B is the head and shoulder detection frame, Area is the area of the detection frame, and IoU represents the degree of overlap between the head detection frame and the head and shoulder detection frame;
[0021] Based on the behavior image data of the same pedestrian, the IoU value of the head detection box and the head and shoulder detection box is calculated and used as the element C ij Construct the cost matrix, where C ij Represents the IoU value between the i-th head detection box and the j-th head and shoulder detection box;
[0022] Based on the cost matrix, the minimum cost of matching the head detection frame and the head and shoulder detection frame is calculated by the Hungarian algorithm, which is expressed as:
[0023] cost ij =1-C ij ;
[0024] Among them, cost ij Represents the cost of matching the i-th head detection frame with the j-th head and shoulder detection frame, and defines the minimum cost cost ij The head detection frame and head and shoulder detection frame are the best match;
[0025] Based on the IoU of the head detection box and the head and shoulder detection box matching in different environments, draw the IoU distribution map, and set the threshold T through the obvious valley points in the IoU distribution map;
[0026] The head detection frame and head-shoulder detection frame with the best matching IoU value lower than the threshold T are removed from the matching queue. The head detection frame and head-shoulder detection frame with the best matching IoU value greater than the threshold T are considered to be matched successfully. The successfully matched head detection frame and head-shoulder detection frame are used as the upper body human target, and the unsuccessfully matched head detection frame and head-shoulder detection frame are used as the head human target and the head-shoulder human target respectively.
[0027] Perform a second match between the upper body human target and the human detection frame and calculate the IoU value, and set the human detection frame with the maximum IoU value of the second match as the matching object of the upper body human target;
[0028] Perform a third match between the head and shoulders human target and the human detection frame and calculate the IoU value, and set the human detection frame with the maximum IoU value of the third match as the matching object of the head and shoulders human target;
[0029] Perform a fourth match between the human head and human body target and the human body detection frame and calculate the IoU value, and set the human body detection frame with the maximum IoU value of the fourth match as the matching object of the human head and human body target;
[0030] The matched head detection frame, head and shoulder detection frame, and human body detection frame are combined into a complete pedestrian target.
[0031] As a preferred solution of the pedestrian target optimal screening method for edge device customer behavior analysis of the present invention, wherein: based on the pedestrian target as the tracking target, the pedestrian behavior is tracked,
[0032] include,
[0033] Based on pedestrian targets, head detection and tracking are mainly carried out, followed by head and shoulder detection and tracking;
[0034] Define the minimum acquisition frame number of the camera device as R frame, set the behavior image data of R frame as the minimum head detection frame update period, and define the behavior image data of twice R frame as the minimum head and shoulder detection frame update period;
[0035] Continuously collect behavioral image data of pedestrians, and use the head detection frame as the main tracking object to perform a fourth match with the human body detection frame. If the head detection frame is not matched in consecutive R frames, it is determined that the pedestrian's head features are lost, and the head detection frame tracking is terminated;
[0036] Then, the head and shoulder detection frame is used as the tracking object and the third matching is performed with the human body detection frame. If the head and shoulder detection frame matching is not completed in two consecutive R frames, it is determined that the pedestrian's head and shoulder features are lost, and the head and shoulder detection frame tracking is terminated;
[0037] If both the head detection frame tracking and the head and shoulder detection frame tracking have been terminated, re-match the head detection frame as the main tracking object.
[0038] As a preferred solution of the pedestrian target optimal screening method for edge device customer behavior analysis of the present invention, wherein: the computational volume integrity of the identification behavior image data during the continuous tracking process includes:
[0039] The filtering area threshold is set to determine the area size of the human detection frame. During the continuous tracking process, the human detection frame of each behavior image data is matched to the corresponding pedestrian target. The selection is made by judging the area of the human detection frame and the filtering area threshold P. If it is smaller than the filtering area threshold P, the corresponding image is discarded.
[0040] Set the aspect ratio threshold Q of the human detection frame to determine the aspect ratio of the human detection frame. If it is greater than the filter area threshold P, continue to compare the aspect ratio of the human detection frame. If the aspect ratio data exceeds the aspect ratio threshold Q, the corresponding behavior image data is discarded.
[0041] If the aspect ratio of the human body detection frame is less than the aspect ratio threshold Q, the skeleton points of the behavior picture data are detected through the OpenPose skeleton point coordinate prediction model, and the human body integrity is calculated based on the skeleton points detected in the behavior picture data, which is expressed as:
[0042]
[0043] Among them, N v is the number of visible skeleton points, N is the total number of skeleton points, Score com The score for the degree of human body integrity.
[0044] As a preferred solution of the pedestrian target optimal screening method for edge device customer behavior analysis described in the present invention, wherein: based on the human body integrity score, the clarity and exposure of the behavior image data are predicted, and the posture image queue is constructed, including:
[0045] By human body integrity score Score com , define the geometric center of the skeleton point as the center position, subtract the coordinates of the center position of the skeleton point from the coordinates of each skeleton point, and move the center point position of the skeleton to the origin to normalize the coordinates of the skeleton point;
[0046] Flatten the normalized skeleton point coordinates into feature vectors, and use the SVM classification tool to classify the feature vectors into postures of the behavior image data;
[0047] Detection is performed based on classified behavior image data, and the clarity of the behavior image data is predicted based on the DeepIQA model;
[0048] Build a Deep Learning Models prediction model to predict image exposure based on behavioral image data;
[0049] The behavior picture data is allocated to the corresponding posture picture queue according to the posture classification of the behavior picture data.
[0050] As a preferred solution of the pedestrian target optimal screening method for edge device customer behavior analysis described in the present invention, wherein: the posture image queue is based on calculating the image quality score, and the image quality optimization queue is constructed according to the descending order of the quality score, including:
[0051] The image size is calculated based on the behavior image data, and the image quality score is calculated based on the image size, the human body integrity score, and the clarity of the behavior image data, expressed as:
[0052]
[0053] Score=Score size ·k size +Score e ·k e +Score s ·k s +Score com ·k com ;
[0054] Among them, Score w is the width of the image, Score h is the height of the picture, frame w is the width of the image frame, frame h is the height of the image frame, Score size is the image size, Score s Score is the clarity of the model prediction. e is the exposure predicted by the model, k size , k s , k com are the weight values of clarity, exposure and image size respectively, and Score is the image quality score;
[0055] By comparing the image quality scores in the posture optimization queue, an image quality optimization queue is constructed in descending order of quality scores;
[0056] The maximum number of pictures to allocate to the picture quality optimization queue based on the camera memory;
[0057] During the tracking and matching process of each pedestrian target, the human body detection frame of the pedestrian target is updated in real time, and the posture classification of the corresponding behavior image data of the human body detection frame is added to the corresponding image quality optimization queue;
[0058] If the number of image quality optimization queues reaches the maximum number, the quality scores of the two are determined, and the image corresponding to the human detection frame with the high quality score is retained;
[0059] If the number of picture quality priority queues does not reach the maximum number, directly add to the corresponding picture quality priority queue.
[0060] As a preferred solution of the pedestrian target optimal screening method for edge device customer behavior analysis of the present invention, wherein: the screening is performed based on the image quality optimization queue, and the optimal image is selected to complete the optimal screening, including:
[0061] When the pedestrian disappears from the monitoring screen, the first three pictures are selected from the quality optimization queue of the behavior picture data;
[0062] Compare the sizes of the first and second images. If the size of the first image is greater than 25% of the size of the second image, select the first image. Otherwise, continue to compare the quality scores of the first and second images. If the quality score of the first image is higher, select the first image. If the quality score of the second image is higher, discard the first image and compare the sizes of the second and third images.
[0063] If the size of the second picture is greater than 25% of the size of the third picture, select the second picture. Otherwise, continue to compare the quality scores of the second and third pictures. If the quality score of the second picture is larger, select the second picture. If the quality score of the third picture is larger, discard the second picture and select the third picture to complete the optimal screening of pedestrian targets.
[0064] A computer device comprises: a memory and a processor; the memory stores a computer program, and the processor implements the steps of the above-mentioned pedestrian target optimal screening method for edge device customer behavior analysis when executing the computer program.
[0065] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned pedestrian target optimal screening method for edge device customer behavior analysis.
[0066] The beneficial effects of the present invention are as follows: by collecting customer behavior picture data through a camera and matching detection frames through the Hungarian algorithm, different detection frames can be effectively associated with the same pedestrian target; through the detection frame update cycle, the real-time tracking and data processing efficiency are ensured according to the camera performance and collection requirements; through the human body integrity score, the influence of human body position information on posture recognition is eliminated, so that posture recognition pays more attention to the posture characteristics of the human body itself, and the accuracy and versatility of posture recognition are improved; through the prediction of image clarity and exposure, subsequent analysis is made more reliable; by calculating the image quality score and arranging it in descending order, high-quality behavior picture data can be quickly screened out, which helps to improve the efficiency of data selection; by screening out high-quality behavior picture data, the calculation process fully considers the pedestrian picture specifications that are most conducive to customer behavior analysis; the screening method of the present invention can improve the accuracy of customer behavior analysis tasks. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.
[0068] Figure 1 Schematic diagram of the flow of the optimal pedestrian target screening method for customer behavior analysis on edge devices.
[0069] Figure 2 Schematic diagram of the detection box matching process in the optimal pedestrian target screening method for customer behavior analysis on edge devices.
[0070] Figure 3 Schematic diagram of the pedestrian tracking process in the optimal pedestrian target screening method for customer behavior analysis on edge devices.
[0071] Figure 4 Schematic diagram of the process of establishing image quality optimization team in the optimal pedestrian target screening method for customer behavior analysis on edge devices.
[0072] Figure 5 Schematic diagram of the process of selecting the best images in the optimal pedestrian target screening method for customer behavior analysis on edge devices. DETAILED DESCRIPTION
[0073] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.
[0074] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0075] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.
[0076] Example 1
[0077] Reference Figure 1 to Figure 5 , which is the first embodiment of the present invention, and which provides an optimal screening method for pedestrian targets for edge device customer behavior analysis. The optimal screening method for pedestrian targets for edge device customer behavior analysis includes:
[0078] S1, collects customer behavior image data in real time through the camera, and performs detection frame detection on the collected behavior image data;
[0079] Preferably, the customer's behavior picture data is collected in real time by a camera, and a detection frame detection is performed on the collected behavior picture data, including:
[0080] The customer behavior image data is collected through the camera, and the pedestrian image detection frame detection is performed using the YOLOv5 algorithm, including head detection frame, head and shoulder detection frame, and human body detection frame detection;
[0081] The head area, head-shoulder area, and whole body area of the human body image in the behavior picture data are set as the detection targets. The images in the behavior picture data are divided into S×S grids using the YOLOv5 algorithm. The grids are used as units, and the geometric center of the human body image is defined as the center point coordinates. The coordinate values of the detection targets are calculated based on the center point coordinates. The image data is classified and located according to the coordinate values of the head area, head-shoulder area, and whole body area.
[0082] The head target, head-shoulder target and human target area are surrounded by horizontal rectangular frames to form a head detection frame, a head-shoulder detection frame and a human detection frame;
[0083] The head detection frame is the minimum enveloping rectangular frame that fits the edge of the head area. The head and shoulder detection frame is the minimum enveloping rectangular frame that is a combination of the head area and the chest area. The human body detection frame is a rotated rectangular frame that envelops the entire human body.
[0084] By collecting customer behavior image data through cameras, real-time monitoring of customer behavior is achieved, the efficiency of monitoring customer behavior is improved, abnormal behavior is discovered in a timely manner, and rich original data is provided for subsequent data analysis, which helps to optimize the layout of shopping malls and improve customer experience. Through the YOLOv5 algorithm, pedestrian images are detected by detection frames, and high-precision recognition of different parts of customers is achieved, which improves the recognition speed and accuracy and reduces the false detection rate. The detection of different parts can analyze customer behavior in more detail, provide accurate data support for subsequent behavior analysis, and provide merchants with more targeted marketing strategies.
[0085] S2, based on the detection result of the detection box, the detection box is matched to a complete pedestrian target through the Hungarian algorithm;
[0086] Preferably, based on the detection result of the detection frame, the detection frame is matched to a complete pedestrian target by the Hungarian algorithm, including:
[0087] The first match is performed based on the head detection frame and the head and shoulder detection frame, and the first match IoU value is calculated, which is expressed as:
[0088]
[0089] Where A is the head detection frame, B is the head and shoulder detection frame, Area is the area of the detection frame, Area(A∩B) is the intersection area of the head detection frame and the head and shoulder detection frame, Area(A∪B) is the union area of the head detection frame and the head and shoulder detection frame, and IoU represents the degree of overlap between the head detection frame and the head and shoulder detection frame.
[0090] Based on the behavior image data of the same pedestrian, the IoU value of the head detection box and the head and shoulder detection box is calculated and used as the element C ij Construct the cost matrix, where C ij Represents the IoU value between the i-th head detection box and the j-th head and shoulder detection box;
[0091] Based on the cost matrix, the minimum cost of matching the head detection frame and the head and shoulder detection frame is calculated by the Hungarian algorithm, which is expressed as:
[0092] cost ij =1-C ij ;
[0093] Among them, cost ij Represents the cost of matching the i-th head detection frame with the j-th head and shoulder detection frame, and defines the minimum cost cost ij The head detection frame and the head and shoulder detection frame are optimally matched (the optimal match between the head detection frame and the head and shoulder detection frame is obtained by calculating the minimum cost of matching the head detection frame and the head and shoulder detection frame)
[0094] Based on the IoU of the head detection box and the head and shoulder detection box matching in different environments, draw the IoU distribution map, and set the threshold T through the obvious valley points in the IoU distribution map;
[0095] The head detection frame and head-shoulder detection frame with the best matching IoU value lower than the threshold T are removed from the matching queue. The head detection frame and head-shoulder detection frame with the best matching IoU value greater than the threshold T are considered to be matched successfully. The successfully matched head detection frame and head-shoulder detection frame are used as the upper body human target, and the unsuccessfully matched head detection frame and head-shoulder detection frame are used as the head human target and the head-shoulder human target respectively.
[0096] Perform a second match between the upper body human target and the human detection frame and calculate the IoU value, and set the human detection frame with the maximum IoU value of the second match as the matching object of the upper body human target;
[0097] Perform a third match between the head and shoulders human target and the human detection frame and calculate the IoU value, and set the human detection frame with the maximum IoU value of the third match as the matching object of the head and shoulders human target;
[0098] Perform a fourth match between the human head and human body target and the human body detection frame and calculate the IoU value, and set the human body detection frame with the maximum IoU value of the fourth match as the matching object of the human head and human body target;
[0099] The matched head detection frame, head and shoulder detection frame, and human body detection frame are combined into a complete pedestrian target.
[0100] By matching detection frames using the Hungarian algorithm, different detection frames can be effectively associated with the same pedestrian target. This matching method takes into account the degree of overlap (IoU) between detection frames, ensures the accuracy of matching, and thus improves the accuracy of target tracking. By calculating the IoU value and constructing the cost matrix, the system can adapt to detection frame matching in different environments. By setting the IoU threshold T, the system can automatically remove detection frames with low matching degrees, thereby optimizing resource allocation and investing more computing resources in detection frames with high matching degrees, improving overall processing efficiency, and combining successfully matched detection frames into complete pedestrian targets, providing more refined information for subsequent data analysis. The multi-level matching process helps to more comprehensively understand the behavioral characteristics of pedestrians and provide rich data support for retail analysis application scenarios.
[0101] S3, based on the pedestrian target as the tracking target, the pedestrian's behavior is tracked;
[0102] Preferably, based on the pedestrian target as the tracking target, the pedestrian behavior tracking is performed, including:
[0103] Based on pedestrian targets, head detection and tracking are mainly carried out, followed by head and shoulder detection and tracking;
[0104] Define the minimum acquisition frame number of the camera device as R frame, set the behavior image data of R frame as the minimum head detection frame update period, and define the behavior image data of twice R frame as the minimum head and shoulder detection frame update period;
[0105] Continuously collect behavioral image data of pedestrians, and use the head detection frame as the main tracking object to perform a fourth match with the human body detection frame. If the head detection frame is not matched in consecutive R frames, it is determined that the pedestrian's head features are lost, and the head detection frame tracking is terminated;
[0106] Then, the head and shoulder detection frame is used as the tracking object and the third matching is performed with the human body detection frame. If the head and shoulder detection frame matching is not completed in two consecutive R frames, it is determined that the pedestrian's head and shoulder features are lost, and the head and shoulder detection frame tracking is terminated;
[0107] If both the head detection frame tracking and the head and shoulder detection frame tracking have been terminated, the head detection frame is re-matched as the main tracking object until a matching head detection frame appears for the pedestrian target.
[0108] The system uses head detection frame tracking as the main method and head and shoulder detection tracking as the secondary method. It prioritizes the accuracy of pedestrian head tracking and uses head and shoulder detection as an auxiliary method to enhance the stability and reliability of tracking. Through the detection frame update cycle, the real-time tracking and data processing efficiency are ensured according to the camera performance and acquisition requirements. It can adapt to different environments and occlusion situations. Through flexible tracking strategies, it enhances the system's adaptability in complex scenarios and provides reliable data support for pedestrian behavior analysis.
[0109] S4, during the continuous tracking process, identifies the calculated body integrity of the behavior image data, predicts the clarity and exposure of the behavior image data based on the body integrity score, and constructs a posture image queue;
[0110] Preferably, during the continuous tracking process, the computational integrity of the behavioral image data is identified, including:
[0111] The filtering area threshold P is set to determine the area size of the human detection frame. During the continuous tracking process, the human detection frame of each behavior image data is matched to the corresponding pedestrian target. The selection is made by judging the area of the human detection frame and the filtering area threshold P. If it is smaller than the filtering area threshold P, the corresponding image is discarded and the filtering area threshold is set to 512.
[0112] Set the aspect ratio threshold Q of the human detection frame to determine the aspect ratio of the human detection frame. If it is greater than the filter area threshold P, continue to compare the aspect ratio of the human detection frame. If the aspect ratio data exceeds the aspect ratio threshold Q, discard the corresponding behavior image data. Set the aspect ratio threshold Q of the human detection frame to be less than 0.166 or greater than 1.
[0113] If the aspect ratio of the human body detection frame is less than the aspect ratio threshold, the skeleton points of the behavior image data are detected through the OpenPose skeleton point coordinate prediction model, and the human body integrity is calculated based on the skeleton points detected in the behavior image data, which is expressed as:
[0114]
[0115] Among them, N v is the number of visible skeleton points, N is the total number of skeleton points, N is 17, Score com The score for the degree of human body integrity.
[0116] By setting the filtering area threshold and aspect ratio threshold, human detection frames with too small area and abnormal aspect ratio can be effectively filtered out. This filtering mechanism helps to improve the quality of behavioral image data and ensure that the data for subsequent analysis and processing is more accurate and reliable. Discarding behavioral image data that does not meet the threshold requirements helps to optimize the allocation of computing resources and improve overall processing efficiency and performance. By judging the filtering area threshold and aspect ratio threshold, the OpenPose skeleton point coordinate prediction model is used to detect skeleton points to more accurately analyze human behavior. The human body integrity score can reflect the degree of integrity of the human body in the behavioral image data, providing an important reference for subsequent behavioral analysis. By calculating the human body integrity, the behavioral image data can be refined.
[0117] Furthermore, based on the human body integrity score, the clarity and exposure of the behavior image data are predicted, and a posture image queue is constructed, including:
[0118] By human body integrity score Score com , define the geometric center of the skeleton point as the center position, subtract the coordinates of the center position of the skeleton point from the coordinates of each skeleton point, and move the center point position of the skeleton to the origin to normalize the coordinates of the skeleton point;
[0119] Flatten the normalized skeleton point coordinates into feature vectors, and use the SVM classification tool to classify the feature vectors into postures of the behavior image data;
[0120] Detection is performed based on classified behavior image data, and the clarity of the behavior image data is predicted based on the DeepIQA model;
[0121] Build a Deep Learning Models prediction model to predict image exposure based on behavioral image data;
[0122] The behavior picture data is allocated to the corresponding posture picture queue according to the posture classification of the behavior picture data.
[0123] The human body integrity score is used to eliminate the influence of human body position information on posture recognition, so that posture recognition focuses more on the posture characteristics of the human body itself, improving the accuracy and versatility of posture recognition. The normalized skeleton point coordinates are flattened into feature vectors, and the feature vectors are used through the SVM classification tool to perform posture classification of behavioral image data, thereby converting human body posture information into a form that can be processed by the model, improving the efficiency and accuracy of posture classification. The clarity of behavioral image data is predicted by the DeepIQA model, which achieves quality assessment of behavioral image data. The prediction of image clarity and exposure makes subsequent analysis more reliable. A posture image queue is built to optimize the data management process, facilitating more in-depth behavioral analysis and research.
[0124] S5, based on the posture image queue, calculate the image quality score, and construct an image quality optimization queue by arranging them in descending order according to the quality score;
[0125] Preferably, based on the posture picture queue, the picture quality score is calculated, and the picture quality optimization queue is constructed by arranging the pictures in descending order according to the quality score, including:
[0126] The image size is calculated based on the behavior image data, and the image quality score is calculated based on the image size, the human body integrity score, and the clarity of the behavior image data, expressed as:
[0127]
[0128] Score=Score size ·k size +Score e ·k e +Score s ·k s +Score com ·k com ;
[0129] Among them, Score w is the width of the image, Score h is the height of the picture, frame w is the width of the image frame, frame h is the height of the image frame, Score size is the image size, Score s Score is the clarity of the model prediction. e is the exposure predicted by the model, k size , k s , k com are the weight values of clarity, exposure and image size respectively, and Score is the image quality score;
[0130] By comparing the image quality scores in the posture optimization queue, an image quality optimization queue is constructed in descending order of quality scores;
[0131] The maximum number of pictures to allocate to the picture quality optimization queue based on the camera memory;
[0132] During the tracking and matching process of each pedestrian target, the human body detection frame of the pedestrian target is updated in real time, and the posture classification of the corresponding behavior image data of the human body detection frame is added to the corresponding image quality optimization queue;
[0133] If the number of image quality optimization queues reaches the maximum number, the quality scores of the two are determined, and the image corresponding to the human detection frame with the high quality score is retained;
[0134] If the number of picture quality priority queues does not reach the maximum number, directly add to the corresponding picture quality priority queue.
[0135] By calculating the image quality scores and arranging them in descending order, high-quality behavioral image data can be quickly screened out, which helps to improve the efficiency of data selection and ensure that the most valuable data is processed first under limited resources. The maximum number of images allocated to the image quality priority queue through the camera memory helps to reasonably allocate storage resources. This restriction mechanism ensures that high-quality image data can be efficiently stored and managed when storage space is limited. The human body detection frame of the pedestrian target is updated in real time, and the human body detection frame is added to the image quality priority queue according to the posture classification of the corresponding behavioral image data, which enhances the system's adaptability to dynamic changes. By retaining the human body detection frame with a high quality score, the accuracy of tracking and analysis can be ensured. High-quality image data helps to improve the stability of pedestrian tracking and the accuracy of behavior analysis.
[0136] S6, screening based on the image quality optimization queue, and selecting the best behavior image data to complete the optimal pedestrian target screening;
[0137] Preferably, the optimal queue is selected based on the image quality, and the optimal screening is completed by selecting the best image, including:
[0138] When the pedestrian disappears from the monitoring screen, the first three pictures are selected from the quality optimization queue of the behavior picture data;
[0139] Compare the sizes of the first and second images. If the size of the first image is greater than 25% of the size of the second image, select the first image. Otherwise, continue to compare the quality scores of the first and second images. If the quality score of the first image is higher, select the first image. If the quality score of the second image is higher, discard the first image and compare the sizes of the second and third images.
[0140] If the size of the second picture is greater than 25% of the size of the third picture, select the second picture. Otherwise, continue to compare the quality scores of the second and third pictures. If the quality score of the second picture is larger, select the second picture. If the quality score of the third picture is larger, discard the second picture and select the third picture to complete the optimal screening of pedestrian targets.
[0141] By selecting the first three pictures in the quality optimization queue of behavioral image data and comparing their sizes and quality scores, the system can quickly screen out the best pictures. This screening mechanism helps to improve the efficiency of data selection and obtain the most valuable data within a limited time. By comparing picture sizes and quality scores, the best picture data can be efficiently stored and managed, which helps to improve resource utilization efficiency. This real-time screening helps the system maintain efficiency and accuracy in different scenarios. By selecting the best pictures, the accuracy of tracking and analysis can be ensured, providing more comprehensive and in-depth analysis results. The best picture data and accurate human behavior analysis help to provide more personalized services.
[0142] Example 2
[0143] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.
[0144] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.
[0145] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.
[0146] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0147] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. An optimal screening method for pedestrian targets for edge device customer behavior analysis, characterized by: include, Collect customer behavior image data in real time through the camera, and perform detection frame detection on the collected behavior image data; Based on the detection results of the detection box, the detection box is matched to a complete pedestrian target through the Hungarian algorithm; Based on the pedestrian target as the tracking target, the pedestrian behavior is tracked; During the continuous tracking process, the computational body integrity of the behavioral image data is identified, and based on the body integrity score, the clarity and exposure of the behavioral image data are predicted to build a posture image queue; Based on the posture image queue, calculate the image quality score, and construct an image quality optimization queue by arranging them in descending order of quality scores; The optimal queue is selected based on the image quality, and the optimal behavior image data is selected to complete the optimal pedestrian target screening.
2. The optimal screening method for pedestrian targets for edge device customer behavior analysis according to claim 1, characterized in that: The method of collecting the customer's behavior picture data in real time through the camera and performing detection frame detection on the collected behavior picture data includes: The camera is used to collect customer behavior image data, and the YOLOv5 algorithm is used to perform detection frame detection on pedestrian images, including head detection frame, head and shoulder detection frame, and human body detection frame detection; The head area, head-shoulder area, and whole body area of the human body image in the behavior picture data are set as the detection targets. The images in the behavior picture data are divided into S×S grids using the YOLOv5 algorithm. The grids are used as units, and the geometric center of the human body image is defined as the center point coordinates. The coordinate values of the detection targets are calculated based on the center point coordinates. The image data is classified and located according to the coordinate values of the head area, head-shoulder area, and whole body area. The head target, head-shoulder target and human target area are surrounded by horizontal rectangular frames to form a head detection frame, a head-shoulder detection frame and a human detection frame.
3. The optimal screening method for pedestrian targets for edge device customer behavior analysis according to claim 2, characterized in that: Based on the detection result of the detection frame, the detection frame is matched to a complete pedestrian target through the Hungarian algorithm. include, The first match is performed based on the head detection frame and the head and shoulder detection frame, and the first match IoU value is calculated, which is expressed as: Among them, A is the head detection frame, B is the head and shoulder detection frame, Area is the area of the detection frame, and IoU represents the degree of overlap between the head detection frame and the head and shoulder detection frame; Based on the behavior image data of the same pedestrian, the IoU value of the head detection box and the head and shoulder detection box is calculated and used as the element C ij Construct the cost matrix, where C ij Represents the IoU value between the i-th head detection box and the j-th head and shoulder detection box; Based on the cost matrix, the minimum cost of matching the head detection frame and the head and shoulder detection frame is calculated by the Hungarian algorithm, which is expressed as: cost ij =1-C ij ; Among them, cost ij Represents the cost of matching the i-th head detection frame with the j-th head and shoulder detection frame, and defines the minimum cost cost ij The head detection frame and head and shoulder detection frame are the best match; Based on the IoU of the head detection box and the head and shoulder detection box matching in different environments, draw the IoU distribution map, and set the threshold T through the obvious valley points in the IoU distribution map; The head detection frame and head-shoulder detection frame with the best matching IoU value lower than the threshold T are removed from the matching queue. The head detection frame and head-shoulder detection frame with the best matching IoU value greater than the threshold T are considered to be matched successfully. The successfully matched head detection frame and head-shoulder detection frame are used as the upper body human target, and the unsuccessfully matched head detection frame and head-shoulder detection frame are used as the head human target and the head-shoulder human target respectively. Perform a second match between the upper body human target and the human detection frame and calculate the IoU value, and set the human detection frame with the maximum IoU value of the second match as the matching object of the upper body human target; Perform a third match between the head and shoulders human target and the human detection frame and calculate the IoU value, and set the human detection frame with the maximum IoU value of the third match as the matching object of the head and shoulders human target; Perform a fourth match between the human head and human body target and the human body detection frame and calculate the IoU value, and set the human body detection frame with the maximum IoU value of the fourth match as the matching object of the human head and human body target; The matched head detection frame, head and shoulder detection frame, and human body detection frame are combined into a complete pedestrian target.
4. The optimal screening method for pedestrian targets for edge device customer behavior analysis according to claim 3, characterized in that: The method of tracking the behavior of pedestrians based on the pedestrian target as the tracking target includes: Based on pedestrian targets, head detection and tracking are mainly carried out, followed by head and shoulder detection and tracking; Define the minimum acquisition frame number of the camera device as R frame, set the behavior image data of R frame as the minimum head detection frame update period, and define the behavior image data of twice R frame as the minimum head and shoulder detection frame update period; Continuously collect behavioral image data of pedestrians, and use the head detection frame as the main tracking object to perform a fourth match with the human body detection frame. If the head detection frame is not matched in consecutive R frames, it is determined that the pedestrian's head features are lost, and the head detection frame tracking is terminated; Then, the head and shoulder detection frame is used as the tracking object and the third matching is performed with the human body detection frame. If the head and shoulder detection frame matching is not completed in two consecutive R frames, it is determined that the pedestrian's head and shoulder features are lost, and the head and shoulder detection frame tracking is terminated; If both the head detection frame tracking and the head and shoulder detection frame tracking have been terminated, re-match the head detection frame as the main tracking object.
5. The optimal screening method for pedestrian targets for edge device customer behavior analysis according to claim 4, characterized in that: The computational integrity of the identification behavior image data during the continuous tracking process includes: The filtering area threshold is set to determine the area size of the human detection frame. During the continuous tracking process, the human detection frame of each behavior image data is matched to the corresponding pedestrian target. The selection is made by judging the area of the human detection frame and the filtering area threshold P. If it is smaller than the filtering area threshold P, the corresponding image is discarded. Set the aspect ratio threshold Q of the human detection frame to determine the aspect ratio of the human detection frame. If it is greater than the filter area threshold P, continue to compare the aspect ratio of the human detection frame. If the aspect ratio data exceeds the aspect ratio threshold Q, the corresponding behavior image data is discarded. If the aspect ratio of the human body detection frame is less than the aspect ratio threshold Q, the skeleton points of the behavior picture data are detected through the OpenPose skeleton point coordinate prediction model, and the human body integrity is calculated based on the skeleton points detected in the behavior picture data, which is expressed as: Among them, N v is the number of visible skeleton points, N is the total number of skeleton points, Score com The score for the degree of human body integrity.
6. The optimal screening method for pedestrian targets for edge device customer behavior analysis according to claim 5, characterized in that: The method predicts the clarity and exposure of the behavior image data based on the human body integrity score and constructs a posture image queue, including: By human body integrity score Score com , define the geometric center of the skeleton point as the center position, subtract the coordinates of the center position of the skeleton point from the coordinates of each skeleton point, and move the center point position of the skeleton to the origin to normalize the coordinates of the skeleton point; Flatten the normalized skeleton point coordinates into feature vectors, and use the SVM classification tool to classify the feature vectors into postures of the behavior image data; Detection is performed based on classified behavior image data, and the clarity of the behavior image data is predicted based on the DeepIQA model; Build a Deep Learning Models prediction model to predict image exposure based on behavioral image data; The behavior picture data is allocated to the corresponding posture picture queue according to the posture classification of the behavior picture data.
7. The optimal screening method for pedestrian targets for edge device customer behavior analysis according to claim 6, characterized in that: The method of calculating the image quality scores based on the posture image queue and constructing the image quality optimization queue according to the descending order of the quality scores includes: The image size is calculated based on the behavior image data, and the image quality score is calculated based on the image size, the human body integrity score, and the clarity of the behavior image data, expressed as: Score=Score size ·k size +Score e ·k e +Score s ·k s +Score com ·k com ; Among them, Score w is the width of the image, Score h is the height of the picture, frame w is the width of the image frame, frame h is the height of the image frame, Score size is the image size, Score s Score is the clarity of the model prediction. e is the exposure predicted by the model, k size , k s , k com are the weight values of clarity, exposure and image size respectively, and Score is the image quality score; By comparing the image quality scores in the posture optimization queue, an image quality optimization queue is constructed in descending order of quality scores; The maximum number of pictures to allocate to the picture quality optimization queue based on the camera memory; During the tracking and matching process of each pedestrian target, the human body detection frame of the pedestrian target is updated in real time, and the posture classification of the corresponding behavior image data of the human body detection frame is added to the corresponding image quality optimization queue; If the number of image quality optimization queues reaches the maximum number, the quality scores of the two are determined, and the image corresponding to the human detection frame with the high quality score is retained; If the number of picture quality priority queues does not reach the maximum number, directly add to the corresponding picture quality priority queue.
8. The optimal screening method for pedestrian targets for edge device customer behavior analysis according to claim 7, characterized in that: The screening based on the picture quality optimization queue and selecting the best picture to complete the optimal screening include: When the pedestrian disappears from the monitoring screen, the first three pictures are selected from the quality optimization queue of the behavior picture data; Compare the sizes of the first and second images. If the size of the first image is greater than 25% of the size of the second image, select the first image. Otherwise, continue to compare the quality scores of the first and second images. If the quality score of the first image is higher, select the first image. If the quality score of the second image is higher, discard the first image and compare the sizes of the second and third images. If the size of the second picture is greater than 25% of the size of the third picture, select the second picture. Otherwise, continue to compare the quality scores of the second and third pictures. If the quality score of the second picture is larger, select the second picture. If the quality score of the third picture is larger, discard the second picture and select the third picture to complete the optimal screening of pedestrian targets.
9. A computer device comprising: A memory and a processor; the memory stores a computer program, characterized in that: when the processor executes the computer program, the steps of the pedestrian target optimal screening method for edge device customer behavior analysis described in any one of claims 1 to 8 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the pedestrian target optimal screening method for edge device customer behavior analysis described in any one of claims 1 to 8 are implemented.
