Human body posture tracking method with random addition and subtraction of movable cameras
By initializing multiple movable cameras in the target area and dynamically adjusting the camera position and angle according to the movement trajectory of the target person, the problem that the prior art cannot adapt to complex and changing environments and dynamically changing goals is solved, and the accuracy of human posture is achieved track.
Patent Information
- Application Number
- CN202410510538.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-26
- Publication Date
- 2024-12-17
- Estimated Expiration
- 2044-04-26
AI Technical Summary
The existing human posture tracking technology cannot adapt to complex and changing environments and dynamic changes. Especially when the target person is moving, the fixed camera layout is difficult to capture complete posture information, resulting in unsatisfactory tracking effect.
Initialize multiple movable cameras in the target area, dynamically adjust the position and angle of the camera according to the movement trajectory and posture changes of the target person, and use the video streams of multiple cameras to obtain accurate posture tracking results.
It achieves the goal of accurately tracking human postures, adapting to complex environments and dynamic changes, and improving the tracking effect.
Smart Images
Figure CN118365677B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of human body posture tracking, and in particular to a human body posture tracking method in an environment where a movable camera can be randomly added or subtracted. Background Art
[0002] With the rapid development of science and technology and the increasing maturity of intelligent monitoring technology, human posture tracking technology is being used more and more widely in various fields. Whether it is intelligent security, sports training, human-computer interaction or virtual reality, accurate and real-time human posture tracking plays a vital role. However, in practical applications, human posture tracking methods often face many challenges.
[0003] Traditional human posture tracking methods mainly rely on fixed camera layouts to identify and track human postures through image processing technology. However, the limitation of this method is that it cannot adapt to complex and changing environments and dynamically changing targets. Especially when the target person is moving, the fixed camera layout often finds it difficult to capture complete posture information, resulting in unsatisfactory tracking results. Summary of the invention
[0004] The present application provides a method for tracking human posture in an environment where a movable camera can be randomly added or subtracted, which is used to solve the technical problem that the existing technology cannot adapt to complex and changing environments and dynamically changing targets.
[0005] In view of the above problems, the present application provides a method for tracking human body posture in an environment where a movable camera is randomly added or subtracted.
[0006] In a first aspect of the present application, a method for tracking human body posture in an environment where a movable camera can be randomly added or subtracted is provided, the method comprising:
[0007] Initialize multiple movable cameras in a target area, and obtain multiple initial information of the multiple movable cameras, wherein the initial information includes initial position information and state information, and the state information includes a rotation angle and a field of view adjustment range; determine a dynamic monitoring window based on the multiple initial information and in combination with the regional structure information of the target area; obtain a target route of a target person, and determine a number of selected cameras from the multiple movable cameras based on the target route; perform dynamic recognition on the several selected cameras in the dynamic monitoring window to obtain a number of video streams; perform gesture recognition on the several video streams based on a gesture recognition model to obtain a number of gesture recognition results, wherein the gesture recognition result includes a first abnormal gesture identifier; fuse the multiple gesture recognition results to obtain a gesture fusion result, perform abnormal gesture recognition on the gesture fusion result to obtain a second abnormal gesture; obtain a gesture tracking result of the target person based on the first abnormal gesture and the second abnormal gesture.
[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages:
[0009] The present application initializes multiple movable cameras in a target area, and obtains multiple initial information of the multiple movable cameras, wherein the initial information includes initial position information and state information, and the state information includes rotation angle and field of view adjustment range; based on multiple initial information, combined with regional structure information of the target area, a dynamic monitoring window is determined; the target route of the target person is obtained, and several selected cameras are determined from multiple movable cameras based on the target route; in the dynamic monitoring window, several selected cameras are dynamically identified to obtain several video streams; gesture recognition is performed on several video streams based on a gesture recognition model to obtain several gesture recognition results, wherein the gesture recognition results include a first abnormal gesture identifier; multiple gesture recognition results are fused to obtain a gesture fusion result, abnormal gesture recognition is performed on the gesture fusion result to obtain a second abnormal gesture; based on the first abnormal gesture and the second abnormal gesture, the gesture tracking result of the target person is obtained. The present invention solves the technical problem that the prior art cannot adapt to complex and changeable environments and dynamically changing targets, and achieves the technical effect of accurately tracking human body gestures by dynamically adjusting the position and angle of the camera according to the motion trajectory and gesture changes of the target person. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0011] Figure 1 A schematic diagram of a flow chart of a method for tracking human posture in an environment where a movable camera can be randomly added or subtracted provided in an embodiment of the present application;
[0012] Figure 2 A schematic diagram of the process of obtaining several posture recognition results by a human posture tracking method in an environment of randomly increasing or decreasing movable cameras provided in an embodiment of the present application. DETAILED DESCRIPTION
[0013] The present application provides a method for tracking human posture in an environment with randomly increasing or decreasing movable cameras, aiming to solve the technical problem that the existing technology cannot adapt to complex and changeable environments and dynamically changing targets. By dynamically adjusting the position and angle of the camera according to the motion trajectory and posture changes of the target person, the technical effect of accurately tracking the human posture is achieved.
[0014] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0015] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices.
[0016] Embodiment 1
[0017] like Figure 1 As shown, the present application provides a method for tracking human posture in an environment where a movable camera can be randomly added or subtracted, the method comprising:
[0018] Step S100: initializing a plurality of movable cameras in a target area, and obtaining a plurality of initial information of the plurality of movable cameras, wherein the initial information includes initial position information and state information, and the state information includes a rotation angle and a field of view adjustment range;
[0019] In an embodiment of the present application, there are multiple movable cameras in the target area, and each camera is mechanically adjusted to be moved to a preset initial position.
[0020] For each camera, record its initial position information and status information. The initial position information includes specific coordinates or relative position relative to a reference point. The status information includes rotation angle and field of view adjustment range. The rotation angle is the current horizontal rotation angle and vertical rotation angle of the camera. The field of view adjustment range is the maximum and minimum field of view that the camera can adjust to.
[0021] Through the above steps, the initialization of multiple movable cameras is completed, and their initial position information and status information are obtained.
[0022] Step S200: determining a dynamic monitoring window based on the multiple initial information and in combination with the regional structure information of the target area;
[0023] In the embodiment of the present application, the regional structure information of the target area includes the geographical layout of the target area, main roads and buildings, potential travel routes, and dense areas of human activities.
[0024] Based on the initial information of the camera and the regional structure information of the target area, the blind area is delineated. The blind area refers to the area not covered by any camera. The blind area is identified by comparing the camera's rotation angle range with the coverage area of adjacent cameras.
[0025] After determining the blind area, determine the displacement range of the camera. The displacement range refers to the maximum distance the camera can move to cover the blind area.
[0026] After determining the blind spots and displacement range, start optimizing the camera configuration, adjusting the camera position, rotation angle, and field of view adjustment range to ensure that the blind spots are effectively covered and the entire target area is fully monitored.
[0027] Finally, based on the analysis and optimization of the above steps, the dynamic monitoring window is determined. The dynamic monitoring window is determined based on the camera's displacement range, rotation angle range, and regional structure information of the target area. Define the key areas that the camera focuses on in the tracking task and ensure effective coverage of the entire target area.
[0028] Step S300: obtaining a target travel route of a target person, and determining a plurality of selected cameras from the plurality of movable cameras based on the target travel route;
[0029] In the embodiment of the present application, the current orientation of the target person is determined by a camera, and the orientation of the target person is used as the moving direction.
[0030] By comparing the coverage area of the camera with the direction of travel of the target person, the cameras that can cover the route of the target person are preliminarily screened out.
[0031] Since there is a great deal of uncertainty about the target person's direction of travel, some backup cameras are selected to deal with possible unexpected situations.
[0032] After confirming the selected camera, adjust the camera's position, rotation angle or field of view to ensure that they can best cover the target person's direction of travel.
[0033] Since the target person's direction of travel may change at any time, the configuration of the selected camera is updated and adjusted in real time by real-time monitoring of the target person's orientation and direction of travel, and the camera parameters are adjusted or the camera is reselected as needed.
[0034] After obtaining the target person's target route, compare and analyze it with the coverage areas of multiple movable cameras. Compare the target person's route with the coverage areas of each camera to find the camera that can cover the route.
[0035] Step S400: in the dynamic monitoring window, dynamically identifying the selected cameras to obtain a plurality of video streams;
[0036] In the embodiment of the present application, the selected camera is dynamically identified and tracked in the dynamic monitoring window. The video stream of each camera is analyzed in real time using image recognition technology to identify the target person. Once the target person is identified, the target tracking algorithm is used to detect and match the features of the target person in continuous video frames to track its position and motion trajectory in the video stream and determine its position and movement direction.
[0037] When the target person moves from the field of view of one camera to the field of view of another camera, cross-camera tracking is achieved by comparing the target person's features in the video streams of different cameras and utilizing the continuity in time and space.
[0038] During dynamic recognition and tracking, real-time video streams are obtained from the selected cameras through network connections or dedicated video transmission systems to ensure the real-time and stability of the video streams. Image enhancement is performed on the video streams to improve the visibility of the target person in complex backgrounds or low-light conditions. Key features of the target person, such as color, shape, texture, etc., are extracted from the video streams for more accurate recognition and tracking.
[0039] Through the above process, several selected cameras are dynamically identified in the dynamic monitoring window, and real-time video streams are obtained.
[0040] Step S500: performing posture recognition on the plurality of video streams based on a posture recognition model to obtain a plurality of posture recognition results, wherein the posture recognition results include a first abnormal posture identifier;
[0041] In the embodiment of the present application, before performing posture recognition, a posture recognition model is pre-trained. The posture recognition model is constructed using a convolutional neural network, and then the model is trained using a large amount of annotated data, which includes images of various human postures and corresponding posture labels.
[0042] For several video streams obtained from the selected cameras, a series of preprocessing operations are performed to input them into the gesture recognition model. The preprocessing operation includes extracting continuous image frames from the video stream as the input of the gesture recognition model. The extracted image frames are scaled to the input size required by the model, and necessary cropping is performed to remove irrelevant background. The images are normalized to eliminate the influence of external factors such as lighting and color on gesture recognition. The preprocessed image frames are input into the gesture recognition model, and the model will recognize the human gesture in each frame.
[0043] The posture recognition model extracts features related to human posture from images through structures such as convolutional layers. Using the extracted features, the positions of various joints of the human body are determined and the posture skeleton of the human body is constructed.
[0044] Based on the estimated posture skeleton, the model matches it with the predefined posture category and assigns a posture label to each frame. Through the processing of the posture recognition model, the posture recognition results of each frame in several video streams are obtained. The posture recognition results are expressed in the form of posture labels, such as standing, walking, running, etc.
[0045] The gesture recognition result includes a first abnormal gesture identifier. This identifier indicates that the target person shows an abnormal gesture that is significantly different from the normal gesture in the video stream. In order to identify this abnormal gesture, a specific abnormal gesture category is defined in the gesture recognition model, and the labeled data containing the abnormal gesture is used for training during the training process. When the model recognizes an abnormal gesture, the first abnormal gesture identifier is marked in the corresponding gesture recognition result.
[0046] Step S600: fusing the plurality of gesture recognition results to obtain a gesture fusion result, and performing abnormal gesture recognition on the gesture fusion result to obtain a second abnormal gesture;
[0047] In the embodiment of the present application, since multiple cameras shoot the target person from different angles and distances, the gesture recognition results they generate are different or redundant. In order to obtain more accurate and comprehensive gesture information, these results are fused.
[0048] When performing fusion, first ensure that all video streams and corresponding gesture recognition results are synchronized in time through timestamps. Then select a suitable fusion algorithm to fuse multiple gesture recognition results. Common fusion methods include weighted average, voting method, probability-based method, etc. By fusing multiple gesture recognition results, a more comprehensive and accurate gesture description is obtained, which provides a basis for subsequent abnormal gesture recognition.
[0049] After obtaining the fused posture result, abnormal posture recognition is performed on it to obtain a second abnormal posture. The abnormal posture is predefined, and here, falling quickly, standing still for a long time, or abnormal joint bending are regarded as abnormal postures.
[0050] Based on the fused posture results, an anomaly detection model is established to identify abnormal postures. This model is based on rule-based methods, such as setting some thresholds or conditions to determine whether the posture is abnormal.
[0051] The anomaly detection model is applied to perform anomaly detection on the fused posture results. Based on the output of the model, it is determined which postures are abnormal, and a second abnormal posture identifier is obtained. This identifier indicates that an abnormal posture that is significantly different from the normal posture is detected in the fused posture results.
[0052] Step S700: obtaining a posture tracking result of the target person based on the first abnormal posture and the second abnormal posture.
[0053] In the embodiment of the present application, the first abnormal posture and the second abnormal posture are deeply analyzed to determine the specific characteristics, duration, frequency of occurrence and possible causes of the abnormal posture. By comparative analysis, the posture changes of the target person at different times or in different situations are determined to infer the behavioral intention or health status.
[0054] After analyzing the abnormal posture, the posture changes of the target person are tracked by continuously analyzing the video stream or a series of image data to capture the dynamic changes of the target person's posture. During the tracking process, the transitions between abnormal postures and their relationship with other normal postures are determined.
[0055] In order to have a more comprehensive understanding of the target person's posture changes, a posture trajectory is constructed, and the identified different postures, including abnormal postures, are arranged in chronological order to form a complete posture change trajectory.
[0056] Finally, the above analysis results are integrated to form the posture tracking results of the target person.
[0057] Furthermore, step S200 in the method provided in the application embodiment also includes:
[0058] Acquiring the regional structure information of the target area;
[0059] Acquire adjacent first movable camera and second movable camera in the target area;
[0060] Based on the state information, respectively analyze the field of view coverage of the first movable camera and the second movable camera to obtain a first field of view coverage and a second field of view coverage;
[0061] Based on the regional structure information, performing blind spot analysis on the first visual field coverage range and the second visual field coverage range, and determining a first dynamic monitoring window and a second dynamic monitoring window according to the analysis result;
[0062] The first dynamic monitoring window and the second dynamic monitoring window are added to the dynamic monitoring window until the plurality of movable cameras are traversed.
[0063] In an embodiment of the present application, regional structure information is acquired through pre-map data, and the regional structure information includes the geographical layout of the target area, main roads and buildings, potential travel routes, and dense areas of human activities.
[0064] Next, the first movable camera and the second movable camera adjacent to each other are identified and located in the target area. The first movable camera and the second movable camera are randomly selected. The position and status information of the cameras are obtained through the communication protocol or the control interface.
[0065] After obtaining the status information of the cameras, their field of view coverage is analyzed. The lens parameters, current position and posture of the cameras are modeled and calculated to determine the areas they can cover. In this way, the first field of view coverage and the second field of view coverage are obtained. Next, combined with the structural information of the target area, the blind spot analysis of the determined field of view coverage is performed. Blind spots refer to areas that cannot be covered by any camera due to obstacles, camera position or viewing angle limitations. By analyzing the blind spots, it is determined which areas cannot be monitored by the camera.
[0066] Based on the results of the blind spot analysis, the first dynamic monitoring window and the second dynamic monitoring window are determined. These windows represent areas that require special attention or enhanced monitoring. They are potential intrusion paths, areas around important facilities, or other key areas. Once the dynamic monitoring windows are determined, the first dynamic monitoring window and the second dynamic monitoring window are added to the set of dynamic monitoring windows.
[0067] Traverse all movable cameras in the target area and repeat the above steps until the field of view coverage of all cameras and the corresponding dynamic monitoring windows are analyzed and determined.
[0068] Furthermore, step S300 in the method provided in the application embodiment also includes:
[0069] Performing grid division on the target area to obtain a plurality of divided areas;
[0070] Based on the target travel route, determining a plurality of travel areas of the target person in the plurality of divided areas;
[0071] Based on the dynamic monitoring window, determining a plurality of dynamic monitoring areas of the plurality of movable cameras within the plurality of divided areas;
[0072] The intersection of the plurality of traveling areas and the plurality of dynamic monitoring areas is calculated, and the plurality of movable cameras are inversely selected according to the intersection calculation result to determine the plurality of selected cameras.
[0073] In the embodiment of the present application, the target area is divided into a grid, which is divided into a plurality of smaller, regular divided areas, and the size of the grid is adjusted according to the actual situation.
[0074] Based on the target person's route, several travel areas are determined in the divided area. These travel areas are the areas that the target person passes through during the travel process. By analyzing the travel route, key paths and nodes are identified and marked as travel areas.
[0075] Next, based on the determined dynamic monitoring window, the dynamic monitoring areas of multiple movable cameras are determined in the divided area. The dynamic monitoring area is the area that the camera can cover and effectively monitor. According to the camera's field of view, angle, and mobility, the area that each camera can cover is determined and marked as a dynamic monitoring area.
[0076] Based on the target person's moving area and the camera's dynamic monitoring area, the intersection of these areas is calculated. Through the intersection calculation, it is determined which cameras can cover the target person's moving area, so as to achieve effective monitoring of the target person.
[0077] Based on the intersection calculation results, the camera is reversely selected. The reverse selection process is to select those cameras that can cover the target person's moving area from all movable cameras.
[0078] Through the reverse selection process, several selected cameras are determined. These cameras are responsible for monitoring the target person's travel area and providing real-time monitoring data.
[0079] Further, such as Figure 2 As shown, step S500 in the method provided in the application embodiment also includes:
[0080] Acquire a first video stream, perform multi-node image frame extraction on the first video stream, and obtain a captured image set;
[0081] Tracking and marking the collected image set based on the limb structure model to obtain a limb structure set;
[0082] Performing posture recognition on the limb structure set based on the posture recognition model to obtain a plurality of posture recognition results with posture deviation coefficients;
[0083] The posture recognition result in which the posture deviation coefficient is less than the preset posture deviation coefficient is identified as a first abnormal posture.
[0084] In an embodiment of the present application, one of several video streams is randomly selected as the first video stream. When performing multi-node image frame extraction on the first video stream, the frequency of frame extraction is first determined, and a frame is set to be extracted every certain time, such as every second or every two seconds, or a frame is extracted when a specific event, such as a motion change, is detected. Use video processing tools or libraries, such as traversing each frame of the video stream through OpenCV, and selectively extract key frames according to the set strategy. These key frames reflect the key information and changes in the video stream, that is, the behavior of the target person changes significantly. Finally, the extracted key frames are saved as image files to form a collection of captured images.
[0085] After obtaining the collection of collected images, the limb structure model is used to track and mark each image. The limb structure model is built based on human anatomy to identify and mark various joints and parts of the human body. Through this step, the limb structure information of the target person is extracted from the image to form a limb structure set.
[0086] When performing posture recognition on a set of limb structures based on a posture recognition model, the posture recognition model receives the set of limb structures as input, and performs operations such as feature extraction, nonlinear transformation, regression or classification on the input data through its internal network structure, and extracts feature representations related to the target posture from the input limb structure. Subsequently, the posture recognition model outputs the posture recognition result based on the learned feature representation. In order to quantify the accuracy of posture recognition, each posture recognition result is accompanied by a posture deviation coefficient. The posture deviation coefficient is a numerical indicator that measures the degree of deviation between the recognition result and the true posture. It is obtained by calculating the Euclidean distance, angle difference or other measurement methods between the recognition result and the true posture.
[0087] Finally, through this process, several posture recognition results with posture deviation coefficients are obtained.
[0088] The preset posture deviation coefficient is a pre-set threshold used to judge the accuracy of posture recognition results. The preset posture deviation coefficient is determined by domain experts based on specific application scenarios and historical experience, and represents the minimum requirement for posture recognition accuracy.
[0089] The posture deviation coefficient of each posture recognition result is compared with the preset posture deviation coefficient. If the posture deviation coefficient of a posture recognition result is less than the preset posture deviation coefficient, then the result is considered abnormal and is identified as the first abnormal posture.
[0090] Furthermore, to construct the limb structure model, the method further comprises:
[0091] Acquire a standard image of the target person by using an image acquisition device;
[0092] Performing human skeleton key point detection based on the standard image to obtain a skeleton key point set;
[0093] The skeleton key point set is used to model the limb structure to obtain a limb structure model.
[0094] In the embodiment of the present application, firstly, a standard image of a target person is captured by a camera, and the standard image is a full-body image of the target person.
[0095] Before detecting skeleton key points, the standard image is preprocessed, including image scaling, normalization, grayscale conversion and other operations, to remove redundant information, reduce the amount of calculation, and enhance the contrast and clarity of the image.
[0096] When performing key point detection, a pre-trained skeleton key point detection model is used to process a standard image. The skeleton key point detection model is built on deep learning technology, such as convolutional neural network. The skeleton key point detection model is trained on a large amount of labeled data to learn how to extract the features of key points from the image and predict the location of the key points based on these features. When processing a standard image, the skeleton key point detection model first extracts features from the image and converts the pixel information in the image into a high-level feature representation. Then, based on these features, it searches for possible key point locations in the image. Finally, the skeleton key point detection model obtains a set of skeleton key points. Skeleton key points include the main joints of the human body, such as shoulders, elbows, knees, etc.
[0097] When constructing a limb structure model, the detected set of skeletal key points is first preprocessed to remove noise points, correct false positives, and smooth the positions of key points. Next, the preprocessed key points are connected based on the knowledge of human anatomy and kinematics. The connection between key points follows the natural joint structure of the human body, such as connecting the key points of the shoulder, elbow, and wrist in sequence to form the upper arm and forearm. In this way, a preliminary skeletal framework is constructed. After connecting the key points, the constraints and motion rules between the joints are defined. These constraints and motion rules are based on the biomechanical characteristics and kinematic laws of the human body to ensure that the model is realistic and reasonable when simulating human motion. For example, define parameters such as the rotation range and motion speed of the joints to limit the range of motion of the model and simulate the real motion state.
[0098] Based on the above steps, a complete limb structure model is constructed.
[0099] Further, based on the posture recognition model, posture recognition is performed on the limb structure set to obtain a plurality of posture recognition results with posture deviation coefficients. The method further includes:
[0100] Based on the abnormal posture database, a set of abnormal posture images is obtained, and combined with the limb structure model, an abnormal limb structure set is obtained;
[0101] Performing feature extraction on the first abnormal limb structure through the first feature extraction channel of the posture recognition model to obtain a first abnormal feature extraction result;
[0102] Extract features of the first limb structure through a second feature extraction channel of the posture recognition model to obtain a first feature extraction result;
[0103] Wherein, the first feature extraction channel and the second feature extraction channel have the same structure and shared parameters, and are used for extracting features of limb structure;
[0104] Performing deviation analysis on the first abnormal feature extraction result and the first feature extraction result to obtain a first feature deviation coefficient;
[0105] The abnormal limb structure set is traversed to obtain multiple characteristic deviation coefficients, and the minimum value of the multiple characteristic deviation coefficients is obtained as the first posture deviation coefficient.
[0106] In an embodiment of the present application, an abnormal posture image set is obtained based on an abnormal posture database. The abnormal posture image set contains various human postures that do not meet the normal posture standards, such as twisting, excessive bending or abnormal stretching. Next, in combination with the previously constructed limb structure model, these abnormal posture images are further processed and analyzed. First, the main skeletal key points of the human body are extracted from the abnormal posture images using image processing techniques, such as image segmentation and key point positioning. Then, based on these extracted skeletal key points, the limb structure model is used to construct the abnormal limb structure. By connecting key points, establishing joint constraints and defining motion rules, an abnormal limb structure set corresponding to the abnormal posture image is generated.
[0107] Two feature extraction channels of the gesture recognition model are used for feature extraction. The first feature extraction channel and the second feature extraction channel have the same structure and share parameters.
[0108] A structure is randomly selected from the abnormal limb structure set as the first abnormal limb structure. The first abnormal limb structure is then input into the first feature extraction channel of the posture recognition model. The first feature extraction channel is designed for limb structure feature extraction and has a strong feature representation capability. It contains multiple convolutional layers, pooling layers, fully connected layers and other neural network structures, which are used to extract high-level feature representations from the input limb structure.
[0109] In the feature extraction process, the first feature extraction channel processes the input abnormal limb structure layer by layer, extracts local features through convolution operations, reduces the dimension of features through pooling operations, and maps features to a higher-level representation space through the fully connected layer. After being processed by the first feature extraction channel, the first abnormal feature extraction result is obtained. This result is a feature vector or feature map, which contains key information of the abnormal limb structure, such as shape, position, posture, etc.
[0110] Then, the normal first limb structure is input into the second feature extraction channel to obtain a first feature extraction result. This result represents the feature representation of the normal limb structure. The first limb structure is processed limb structure data that can accurately reflect the normal posture.
[0111] After obtaining these two feature extraction results, deviation analysis is performed. Deviation analysis is a method of comparing the differences between two feature sets to quantify the difference between abnormal limb structure and normal limb structure. By comparing the first abnormal feature extraction result and the first feature extraction result, the first feature deviation coefficient is obtained. The first feature deviation coefficient reflects the distance or difference between the abnormal limb structure and the normal limb structure in the feature space.
[0112] In order to obtain more comprehensive results, the entire set of abnormal limb structures is traversed, and the above-mentioned feature extraction and deviation analysis steps are performed on each abnormal limb structure to obtain multiple feature deviation coefficients. Finally, the minimum value is selected from these coefficients as the first posture deviation coefficient.
[0113] Furthermore, after obtaining the first characteristic deviation coefficient, the method further includes:
[0114] Obtaining a preset feature deviation threshold, and determining whether the first feature deviation coefficient is less than or equal to the preset feature deviation threshold;
[0115] If so, generating an adding instruction, and adding the first characteristic deviation coefficient to the plurality of characteristic deviation coefficients according to the adding instruction;
[0116] If not, a skip instruction is generated, and the first characteristic deviation coefficient is skipped according to the skip instruction.
[0117] In the embodiment of the present application, a feature deviation threshold is preset. The feature deviation threshold is determined according to the needs of the actual application and domain knowledge. For example, if the sensitivity to abnormal posture is high, a lower threshold is set to capture more abnormal situations.
[0118] Next, the magnitude relationship between the first characteristic deviation coefficient and the preset characteristic deviation threshold is compared. If the first characteristic deviation coefficient is less than or equal to the preset characteristic deviation threshold, it means that the characteristic deviation of the abnormal limb structure is within the range of concern, so an add instruction is generated. According to this add instruction, the first characteristic deviation coefficient is added to the set of multiple characteristic deviation coefficients for further subsequent analysis and processing.
[0119] If the first characteristic deviation coefficient is greater than the preset characteristic deviation threshold, that is, exceeds the concerned range, a skip instruction is generated. According to the skip instruction, the first characteristic deviation coefficient is skipped and is no longer added to the set of characteristic deviation coefficients.
[0120] Furthermore, the method further comprises:
[0121] Acquire a first posture load coefficient of the first abnormal limb structure;
[0122] using the first posture load factor as a subsidiary feature of the first feature deviation coefficient;
[0123] When any a-th characteristic deviation coefficient is the minimum value among the plurality of characteristic deviation coefficients, the first posture deviation coefficient has an a-th posture load coefficient.
[0124] In the embodiment of the present application, the posture load coefficient is a quantitative index used to evaluate the load caused by a specific posture. To calculate the posture load coefficient, data and parameters related to the first abnormal limb structure are collected. Including geometric parameters and kinematic data of the limb structure. The geometric parameters of the limb structure include joint angles, limb lengths, muscle volume, etc. Kinematic data include joint range of motion, movement speed, acceleration, etc. These parameters are obtained through a camera.
[0125] According to the collected data and parameters, a suitable calculation method or model is selected to calculate the posture load coefficient. For example, based on domain knowledge and expert experience, a set of rules is formulated to evaluate the load degree of different postures. The collected data and parameters are input into the selected calculation method or model for calculation, and finally the first posture load coefficient of the first abnormal limb structure is obtained.
[0126] After obtaining the first posture load coefficient, the first posture load coefficient is integrated as a subsidiary feature of the first feature deviation coefficient by creating a data structure, such as a structure, a dictionary, or an object, which includes the feature deviation coefficient as a main feature and the posture load coefficient as a subsidiary feature.
[0127] When any a-th characteristic deviation coefficient is determined as the minimum value among multiple characteristic deviation coefficients, the characteristic difference between the abnormal posture and the normal posture represented by the specific a-th characteristic deviation coefficient is the smallest. In this case, the first posture deviation coefficient corresponding to the a-th characteristic deviation coefficient has the a-th posture load coefficient.
[0128] In the embodiments of the present application, in summary, the embodiments of the present application have at least the following technical effects:
[0129] The present application initializes multiple movable cameras in a target area, and obtains multiple initial information of the multiple movable cameras, wherein the initial information includes initial position information and state information, and the state information includes rotation angle and field of view adjustment range; based on multiple initial information, combined with regional structure information of the target area, a dynamic monitoring window is determined; the target route of the target person is obtained, and several selected cameras are determined from multiple movable cameras based on the target route; in the dynamic monitoring window, several selected cameras are dynamically identified to obtain several video streams; gesture recognition is performed on several video streams based on a gesture recognition model to obtain several gesture recognition results, wherein the gesture recognition results include a first abnormal gesture identifier; multiple gesture recognition results are fused to obtain a gesture fusion result, abnormal gesture recognition is performed on the gesture fusion result to obtain a second abnormal gesture; based on the first abnormal gesture and the second abnormal gesture, the gesture tracking result of the target person is obtained. The present invention solves the technical problem that the prior art cannot adapt to complex and changeable environments and dynamically changing targets, and achieves the technical effect of accurately tracking human body gestures by dynamically adjusting the position and angle of the camera according to the motion trajectory and gesture changes of the target person.
[0130] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. Other embodiments are within the scope of the attached claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0131] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
[0132] This specification and drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.
Claims
1. A method for tracking human posture in an environment with random increase or decrease of movable cameras, characterized in that: The method comprises: Initializing multiple movable cameras in a target area, and obtaining multiple initial information of the multiple movable cameras, wherein the initial information includes initial position information and state information, and the state information includes a rotation angle and a field of view adjustment range; Determining a dynamic monitoring window based on the multiple initial information and in combination with the regional structure information of the target area; Acquire a target travel route of a target person, and determine a number of selected cameras from the plurality of movable cameras based on the target travel route; In the dynamic monitoring window, dynamically identifying the selected cameras to obtain a plurality of video streams; Performing posture recognition on the plurality of video streams based on a posture recognition model to obtain a plurality of posture recognition results, wherein the posture recognition results include a first abnormal posture; Fusing the plurality of posture recognition results to obtain a posture fusion result, and performing abnormal posture recognition on the posture fusion result to obtain a second abnormal posture; Based on the first abnormal posture and the second abnormal posture, obtaining a posture tracking result of the target person; Acquiring a target route of a target person, and determining a plurality of selected cameras from the plurality of movable cameras based on the target route, comprising: Performing grid division on the target area to obtain a plurality of divided areas; Based on the target travel route, determining a plurality of travel areas of the target person in the plurality of divided areas; Based on the dynamic monitoring window, determining a plurality of dynamic monitoring areas of the plurality of movable cameras within the plurality of divided areas; Calculating intersections of the plurality of traveling areas and the plurality of dynamic monitoring areas, and deselecting the plurality of movable cameras according to the intersection calculation result to determine the plurality of selected cameras; Performing gesture recognition on the plurality of video streams based on the gesture recognition model to obtain a plurality of gesture recognition results, including: Acquire a first video stream, perform multi-node image frame extraction on the first video stream, and obtain a captured image set; Tracking and marking the collected image set based on the limb structure model to obtain a limb structure set; Performing posture recognition on the limb structure set based on the posture recognition model to obtain a plurality of posture recognition results with posture deviation coefficients; Identify the posture recognition result whose posture deviation coefficient is less than the preset posture deviation coefficient as a first abnormal posture; Constructing the limb structure model includes: Acquire a standard image of the target person by using an image acquisition device; Performing human skeleton key point detection based on the standard image to obtain a skeleton key point set; Using the skeleton key point set to model the limb structure to obtain a limb structure model; Performing posture recognition on the limb structure set based on the posture recognition model to obtain a plurality of posture recognition results with posture deviation coefficients, including: Based on the abnormal posture database, a set of abnormal posture images is obtained, and combined with the limb structure model, an abnormal limb structure set is obtained; Performing feature extraction on the first abnormal limb structure through the first feature extraction channel of the posture recognition model to obtain a first abnormal feature extraction result; Extract features of the first limb structure through a second feature extraction channel of the posture recognition model to obtain a first feature extraction result; Wherein, the first feature extraction channel and the second feature extraction channel have the same structure and shared parameters, and are used for extracting features of limb structure; Performing deviation analysis on the first abnormal feature extraction result and the first feature extraction result to obtain a first feature deviation coefficient; The abnormal limb structure set is traversed to obtain multiple characteristic deviation coefficients, and the minimum value of the multiple characteristic deviation coefficients is obtained as the first posture deviation coefficient.
2. The method according to claim 1, characterized in that Determining a dynamic monitoring window based on the multiple initial information and in combination with the regional structure information of the target area includes: Acquiring the regional structure information of the target area; Acquire adjacent first movable camera and second movable camera in the target area; Based on the state information, respectively analyze the field of view coverage of the first movable camera and the second movable camera to obtain a first field of view coverage and a second field of view coverage; Based on the regional structure information, performing blind spot analysis on the first visual field coverage range and the second visual field coverage range, and determining a first dynamic monitoring window and a second dynamic monitoring window according to the analysis result; The first dynamic monitoring window and the second dynamic monitoring window are added to the dynamic monitoring window until the plurality of movable cameras are traversed.
3. The method according to claim 1, characterized in that After obtaining the first characteristic deviation coefficient, it also includes: Obtaining a preset feature deviation threshold, and determining whether the first feature deviation coefficient is less than or equal to the preset feature deviation threshold; If so, generating an adding instruction, and adding the first characteristic deviation coefficient to the plurality of characteristic deviation coefficients according to the adding instruction; If not, a skip instruction is generated, and the first characteristic deviation coefficient is skipped according to the skip instruction.
4. The method according to claim 1, characterized in that Also includes: Acquire a first posture load coefficient of the first abnormal limb structure; using the first posture load factor as a subsidiary feature of the first feature deviation coefficient; When any a-th characteristic deviation coefficient is the minimum value among the plurality of characteristic deviation coefficients, the first posture deviation coefficient has an a-th posture load coefficient.