A method and system for recognizing people in complex scenes based on deep learning

Abstract

This invention discloses a method and system for recognizing people in complex scenes based on deep learning, belonging to the field of data recognition technology. The method includes: acquiring video data within a complex scene to be monitored using a video acquisition device; parsing and recognizing the video data using multiple data models to obtain recognition data; generating a detection model based on the recognition data and corresponding video data; acquiring target video data within the target complex scene using the same video acquisition device; inputting the target video data into the detection model; and detecting the target video data based on the detection model to obtain person recognition data within the target complex scene. This invention can be used to recognize people in complex scenes.

Description

Technical Field

[0001] This invention relates to the field of data recognition technology, and more specifically, to a method and system for recognizing people in complex scenes based on deep learning. Background Technology

[0002] With the advancement of social and economic development, the demand for video surveillance has surged, ranging from indoor scenarios such as office areas, commercial activities, supermarkets, and school classrooms to outdoor scenarios such as parks, roads, and security checkpoints. This explosive growth in video data is disastrous for manual interpretation, as humans cannot quickly extract high-value information from massive amounts of video data. Video data structured analysis systems based on deep learning models can effectively solve these problems. Current inventions, which integrate multiple systems in specific access scenarios, combining access control systems and facial recognition gate management platforms, suffer from a lack of versatility in their core facial recognition systems, resulting in limited application scenarios. Furthermore, the core detection module uses a single model, leading to limited analytical accuracy and poor reusability.

[0003] A comprehensive analysis of various application scenarios reveals that they all revolve around the detection of human-related activities. To better analyze video surveillance data in various scenarios, this invention proposes a method and system for structuring complex scene human videos based on deep learning. This invention adds a scene-human separation module and a human structure decomposition module to further refine the human video structure, provides a general calling interface, and matches other system modules in specific scenarios. It can be widely applied to complex scenarios involving the monitoring of human activities. Summary of the Invention

[0004] To address the above problems, this invention proposes a method for recognizing people in complex scenes based on deep learning, comprising:

[0005] Based on video acquisition equipment in the complex scene to be monitored, video data in the complex scene to be monitored is collected, and the video data is parsed and identified based on multiple data models to obtain the identification data of the video data.

[0006] A detection model is generated based on the identification data and the corresponding video data.

[0007] Based on the video acquisition device within the target complex scene, target video data is acquired within the target complex scene;

[0008] The target video data is input into the detection model, and the target video data is detected based on the detection model to obtain the human recognition data in the target complex scene.

[0009] Optionally, based on the identification data and the corresponding video data, a detection model is generated, including:

[0010] The identification data and the corresponding video data are used as sample data. The sample data are divided according to a preset ratio to obtain a training dataset and a validation dataset.

[0011] The training dataset is input into a deep learning-based model to learn from the training dataset and obtain a preliminary detection model for recognizing people in complex scenes. The validation dataset is input into the preliminary detection model to validate the preliminary detection model. Based on the validation data, the parameters of the preliminary detection model are adjusted to generate a detection model.

[0012] Optional, multiple data models, including: video data decoding model, video data preprocessing model, person / target separation model, person / target structure decomposition model, attribute information detection model, and behavior information detection model;

[0013] The video data decoding model is used to decode video data to obtain decoded video data.

[0014] The video data preprocessing model is used to perform color and distortion correction, normalization, and image enhancement on the decoded data to obtain preprocessed data of the decoded data.

[0015] The character target separation model is used to separate character targets from scenes in preprocessed data to obtain character target data.

[0016] The character target structure decomposition model is used to decompose the structure of most character target data to obtain the character structure decomposition data.

[0017] The attribute information detection model is used to extract features from the structural decomposition data of the person to obtain the person's feature data.

[0018] The behavioral information detection model is used to determine a person's behavioral characteristics based on their feature data, so as to obtain the person's behavioral feature data.

[0019] Optional, the identification data includes: decoded data of video data, preprocessed data of decoded data, target data of people, structural decomposition data of people, feature data of people, and behavioral feature data of people.

[0020] Optionally, the method further includes: generating structured information from the identification data, and uploading the structured information to an external system through a generic calling interface.

[0021] Optional external systems include: access control systems, surveillance systems, or public systems.

[0022] Furthermore, this invention also proposes a system for recognizing people in complex scenes based on deep learning, comprising:

[0023] The parsing unit is used to collect video data in the complex scene to be monitored based on the video acquisition device in the complex scene to be monitored, and to parse and identify the video data based on multiple data models to obtain the identification data of the video data.

[0024] The model generation unit is used to generate a detection model based on the recognition data and the corresponding video data.

[0025] The acquisition unit acquires target video data within the target complex scene based on the video acquisition device within the target complex scene;

[0026] The recognition unit is used to input the target video data into the detection model, and to detect the target video data based on the monitoring model in order to obtain the human recognition data in the target complex scene.

[0027] Optionally, based on the identification data and the corresponding video data, a detection model is generated, including:

[0028] The identification data and the corresponding video data are used as sample data. The sample data are divided according to a preset ratio to obtain a training dataset and a validation dataset.

[0029] The training dataset is input into a deep learning-based model to learn from the training dataset and obtain a preliminary detection model for recognizing people in complex scenes. The validation dataset is input into the preliminary detection model to validate the preliminary detection model. Based on the validation data, the parameters of the preliminary detection model are adjusted to generate a detection model.

[0030] Optional, multiple data models, including: video data decoding model, video data preprocessing model, person / target separation model, person / target structure decomposition model, attribute information detection model, and behavior information detection model;

[0031] The video data decoding model is used to decode video data to obtain decoded video data.

[0032] The video data preprocessing model is used to perform color and distortion correction, normalization, and image enhancement on the decoded data to obtain preprocessed data of the decoded data.

[0033] The character target separation model is used to separate character targets from scenes in preprocessed data to obtain character target data.

[0034] The character target structure decomposition model is used to decompose the structure of most character target data to obtain the character structure decomposition data.

[0035] The attribute information detection model is used to extract features from the structural decomposition data of the person to obtain the person's feature data.

[0036] The behavioral information detection model is used to determine a person's behavioral characteristics based on their feature data, so as to obtain the person's behavioral feature data.

[0037] Optional, the identification data includes: decoded data of video data, preprocessed data of decoded data, target data of people, structural decomposition data of people, feature data of people, and behavioral feature data of people.

[0038] Optionally, the identification unit is also used to generate structured information from the identification data and upload the structured information to an external system through a generalized calling interface. The external system includes an access control system, a monitoring system, or a public system.

[0039] Optional external systems include: access control systems, surveillance systems, or public systems.

[0040] In another aspect, the present invention also provides a computing device, comprising: one or more processors;

[0041] A processor is used to execute one or more programs;

[0042] When the one or more programs are executed by the one or more processors, the method described above is implemented.

[0043] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the method described above.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0045] This invention provides a method for identifying people in complex scenes based on deep learning, comprising: acquiring video data within the complex scene to be monitored using a video acquisition device; parsing and identifying the video data using multiple data models to obtain identification data; generating a detection model based on the identification data and corresponding video data; acquiring target video data within the target complex scene using the same video acquisition device; inputting the target video data into the detection model; and detecting the target video data based on the detection model to obtain person identification data within the target complex scene. This invention can be used to identify people in complex scenes. Attached Figure Description

[0046] Figure 1 This is a flowchart of a method for recognizing people in complex scenes based on deep learning, according to the present invention.

[0047] Figure 2 This is a structural diagram of a system for recognizing people in complex scenes based on deep learning, according to the present invention. Detailed Implementation

[0048] Exemplary embodiments of the invention will now be described with reference to the accompanying drawings. However, the invention may be embodied in many different forms and is not limited to the embodiments described herein. These embodiments are provided to fully and completely disclose the invention and to fully convey its scope to those skilled in the art. The terminology used in the exemplary embodiments illustrated in the drawings is not intended to limit the invention. In the drawings, the same units / elements are referred to by the same reference numerals.

[0049] Unless otherwise stated, the terms used herein (including technical terms) have their common meaning as understood by one of ordinary skill in the art. Furthermore, it is understood that terms defined in commonly used dictionaries should be understood to have a meaning consistent with the context of their relevant field, and not to be interpreted as having an idealized or overly formal meaning.

[0050] Example 1:

[0051] This invention proposes a method for recognizing people in complex scenes based on deep learning, such as... Figure 1 As shown, it includes:

[0052] Based on video acquisition equipment in the complex scene to be monitored, video data in the complex scene to be monitored is collected, and the video data is parsed and identified based on multiple data models to obtain the identification data of the video data.

[0053] A detection model is generated based on the identification data and the corresponding video data.

[0054] Based on the video acquisition device within the target complex scene, target video data is acquired within the target complex scene;

[0055] The target video data is input into the detection model, and the target video data is detected based on the detection model to obtain the human recognition data in the target complex scene.

[0056] The detection model is generated based on the identification data and the corresponding video data, including:

[0057] The identification data and the corresponding video data are used as sample data. The sample data are divided according to a preset ratio to obtain a training dataset and a validation dataset.

[0058] The training dataset is input into a deep learning-based model to learn from the training dataset and obtain a preliminary detection model for recognizing people in complex scenes. The validation dataset is input into the preliminary detection model to validate the preliminary detection model. Based on the validation data, the parameters of the preliminary detection model are adjusted to generate a detection model.

[0059] Among them are various data models, including: video data decoding model, video data preprocessing model, person / target separation model, person / target structure decomposition model, attribute information detection model, and behavior information detection model;

[0060] The video data decoding model is used to decode video data to obtain decoded video data.

[0061] The video data preprocessing model is used to perform color and distortion correction, normalization, and image enhancement on the decoded data to obtain preprocessed data of the decoded data.

[0062] The character target separation model is used to separate character targets from scenes in preprocessed data to obtain character target data.

[0063] The character target structure decomposition model is used to decompose the structure of most character target data to obtain the character structure decomposition data.

[0064] The attribute information detection model is used to extract features from the structural decomposition data of the person to obtain the person's feature data.

[0065] The behavioral information detection model is used to determine a person's behavioral characteristics based on their feature data, so as to obtain the person's behavioral feature data.

[0066] The identification data includes: video data decoding data, preprocessed data of the decoding data, target data of the person, structural decomposition data of the person, feature data of the person, and behavioral feature data of the person.

[0067] The method further includes: generating structured information from the identification data, and uploading the structured information to an external system through a generalized calling interface.

[0068] External systems include: access control systems, monitoring systems, or public systems.

[0069] The present invention will be further described below with reference to specific embodiments:

[0070] Collect video data of the area to be monitored, including various video acquisition devices and instruments such as surveillance cameras and cameras.

[0071] It decodes video streams acquired from external video capture devices and saves the raw data. The data saving time can be freely configured. The video encoding and decoding formats include common formats such as MPEG-1, MPEG-2, MPEG-4, H.264, H.265, WMV, DV, RM, WMV9, MJPEG, DIRAC, DIVX, PV8, and PV9, and can be further expanded for connected video capture devices.

[0072] The decoded video data undergoes preprocessing, including color adjustment, distortion correction, normalization, image enhancement, and other processes. In addition, relevant segments featuring people in the video are pre-cropped, and time segments are located.

[0073] This deep learning-based object detection model is used to segment the human target region and various complex backgrounds in an image, roughly remove single invalid background regions, and highlight the human-related regions as input for subsequent modules. The core model of this module is RetinaNet. In the process of segmenting human targets and backgrounds, there is a serious class imbalance problem between human target regions and background regions. During the segmentation process, a large number of candidate regions are generated, but only a small portion are real human targets. This problem often causes the loss function of the algorithm model to be dominated by negative samples, which is the root cause of large segmentation errors. To address the poor segmentation effect caused by the class imbalance between human targets and background regions, the RetinaNet model adjusts the loss function to eliminate the extreme imbalance between foreground human targets and background categories and mine hard-to-separate samples. Its original loss function formula is shown in (1):

[0074]

[0075] Where y∈{±1} is the true label of the category, 1 is the target label of the foreground person, -1 is the background label, and p∈[0,1] is the probability that the model predicts the label as 1. The modified formula is shown in (2):

[0076] FL(p t )=-α t (1-p t ) γ log(p t (2)

[0077] in:

[0078]

[0079] (1-p t ) γ As an adjustment factor, γ can adjust the loss of the easily classified targets in the model. When the value is larger, the loss value of the easily classified part is smaller and approaches 0. In this invention, γ is set to 0.65, while the loss value of the difficult-to-classify part is still large, which effectively solves the problem of loss function failure caused by class imbalance. In iterative training, the difficult-to-classify samples contribute more loss value.

[0080] This deep learning-based semantic segmentation model decomposes human targets into regions such as head, hands, neck, chest, legs, and feet, which are then used as input for subsequent detection of human attribute and behavioral information. This module takes the foreground human target output from the human target separation module as new input, uses the RetinaNet model, and adds category labels for head, hands, neck, legs, and feet.

[0081] The detection of the attributes of the target person in the video includes facial feature extraction, gender detection, glasses detection, mask detection, hat detection, hairstyle detection, bag detection, backpack detection, upper garment color detection, lower garment color detection, and environmental information of the target person area. The above-mentioned detection information is collected and input into the next layer. During the human-computer interaction process, the detection information that is judged to be correct by humans enters the sample generation module to generate new training samples. Using the YOLO model, the detection of objects around the target person area is realized, and the object position information and classification probability information are transformed into a regression problem. The model generates candidate boxes and predictive boxes in each grid and compares them. By iteratively training, the loss value is reduced, and the target person surrounding object detection model is obtained. Its loss function is shown in (3):

[0082]

[0083] in, This indicates that the i-th grid and the j-th box contain objects. This indicates that the i-th grid and the j-th box do not contain any objects. The first two lines of the formula represent the position loss of the boxes, and x and y are the horizontal and vertical coordinates of the box positions. Let p be the true x and y coordinates of the i-th box. i (c) represents the category confidence score for the i-th target, C i ω represents the confidence level of the box location. i and h i These are the offset and height of the target offset box, respectively.

[0084] The system detects behavioral information of people in the video, including movement direction detection, number of people gathered together detection, bicycle riding detection, smoking detection, dangerous behavior detection, missing items detection, and driving behavior detection. This behavioral detection information is then collected and input into the next layer. During human-computer interaction, detection information deemed correct by the operator enters the sample generation module to generate new training samples.

[0085] The behavioral and attribute information that is correctly detected after receiving human-computer interaction data is combined with video data to create new training samples. The generated training samples are then fed into the deep learning model update server to train a new structured detection model for human videos.

[0086] All models used in the face video structuring system are periodically iterated and updated. Retrained models can be manually controlled or periodically automated to update the face video structuring system. The detection accuracy of the current target detection model is determined using formula (4), including:

[0087]

[0088] Where accuracy is the detection precision; TP represents the number of ground truth values ​​labeled as human targets and the model correctly predicts them as human targets; FP represents the number of ground truth values ​​labeled as background targets and the model predicts them as human targets; FN represents the number of ground truth values ​​labeled as human targets and the model predicts them as background targets; and TN represents the number of ground truth values ​​labeled as background targets and the model predicts them as background targets.

[0089] It collects the results of people video detection and outputs them in a unified manner, providing a set of universal interfaces for connecting to systems such as park access control, public area monitoring, and traffic roads. It provides these systems with complete structured information on people videos, improving system compatibility.

[0090] This invention improves detection accuracy and generalization capabilities across different scenarios, while introducing a universal calling interface to further enhance the system's compatibility with other video structuring systems. It also incorporates a person attribute detection module and a person behavior detection module in person video structuring, achieving the separation of person video structured attributes and behaviors, enriching and refining the structured information of person videos, and continuously improving its detection accuracy.

[0091] Example 2:

[0092] This invention also proposes a system 200 for recognizing people in complex scenes based on deep learning, such as... Figure 2 As shown, it includes:

[0093] The parsing unit 201 is used to collect video data in the complex scene to be monitored based on the video acquisition device in the complex scene to be monitored, and to parse and identify the video data based on multiple data models to obtain the identification data of the video data.

[0094] The model generation unit 202 is used to generate a detection model based on the recognition data and the corresponding video data.

[0095] Acquisition unit 203 acquires target video data within the target complex scene based on the video acquisition device within the target complex scene;

[0096] The recognition unit 204 is used to input the target video data into the detection model, and to detect the target video data based on the monitoring model in order to obtain the human recognition data in the target complex scene.

[0097] The detection model is generated based on the identification data and the corresponding video data, including:

[0098] The identification data and the corresponding video data are used as sample data. The sample data are divided according to a preset ratio to obtain a training dataset and a validation dataset.

[0099] The training dataset is input into a deep learning-based model to learn from the training dataset and obtain a preliminary detection model for recognizing people in complex scenes. The validation dataset is input into the preliminary detection model to validate the preliminary detection model. Based on the validation data, the parameters of the preliminary detection model are adjusted to generate a detection model.

[0100] Among them are various data models, including: video data decoding model, video data preprocessing model, person / target separation model, person / target structure decomposition model, attribute information detection model, and behavior information detection model;

[0101] The video data decoding model is used to decode video data to obtain decoded video data.

[0102] The video data preprocessing model is used to perform color and distortion correction, normalization, and image enhancement on the decoded data to obtain preprocessed data of the decoded data.

[0103] The character target separation model is used to separate character targets from scenes in preprocessed data to obtain character target data.

[0104] The character target structure decomposition model is used to decompose the structure of most character target data to obtain the character structure decomposition data.

[0105] The attribute information detection model is used to extract features from the structural decomposition data of the person to obtain the person's feature data.

[0106] The behavioral information detection model is used to determine a person's behavioral characteristics based on their feature data, so as to obtain the person's behavioral feature data.

[0107] The identification data includes: video data decoding data, preprocessed data of the decoding data, target data of the person, structural decomposition data of the person, feature data of the person, and behavioral feature data of the person.

[0108] The identification unit is further configured to generate structured information from the identification data and upload the structured information to an external system via a generalized calling interface. The external system includes an access control system, a monitoring system, or a public system.

[0109] External systems include: access control systems, monitoring systems, or public systems.

[0110] Example 3:

[0111] Based on the same inventive concept, this invention also provides a computer device, which includes a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to implement corresponding method flows or corresponding functions, thereby implementing the steps of the methods in the above embodiments.

[0112] Example 4:

[0113] Based on the same inventive concept, this invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the steps of the method in the above embodiments.

[0114] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0115] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0116] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0117] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0118] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0119] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for recognizing people in complex scenes based on deep learning, characterized in that, The method includes: Based on video acquisition equipment in the complex scene to be monitored, video data in the complex scene to be monitored is collected, and the video data is parsed and identified based on multiple data models to obtain the identification data of the video data. A detection model is generated based on the identification data and the corresponding video data. Based on video acquisition devices within the target complex scene, target video data within the target complex scene is acquired; The target video data is input into the detection model, and the target video data is detected based on the detection model to obtain the human recognition data in the target complex scene; Among them, the detection model is used to segment the target area of ​​the person in the image from various complex backgrounds, roughly remove single invalid background areas, and highlight the related areas of the person as input for subsequent modules. The core model of this module is RetinaNet. The RetinaNet model addresses the problem of poor segmentation caused by the imbalance between the target area of ​​the person and the background. It adjusts the loss function to eliminate the extreme imbalance between the target area of ​​the foreground person and the background and to mine hard-to-segment samples. Its original loss function formula is shown in (1): Where y∈{±1} is the true label of the category, 1 is the target label of the foreground person, -1 is the background label, and p∈[0,1] is the probability that the model predicts the label as 1. The modified formula is shown in (2): FL(p t )=-a t (1-p t ) γ log(p t ) (2) in: (1-p t ) γ As an adjustment factor, γ can adjust the loss of the easily classified targets in the model. When the value is larger, the loss value of the easily classified part is smaller and approaches 0. γ is set to 0.65, while the loss value of the difficult-to-classify part is still large, which effectively solves the problem of loss function failure caused by class imbalance. In iterative training, the difficult-to-classify samples contribute more loss value. The deep learning-based semantic segmentation model decomposes the various structures of a person into head, hand, neck, chest, leg, and foot regions, which are used as inputs for subsequent detection of person attribute information and behavior information. This module takes the foreground person target output by the person target separation module as new input, uses the RetinaNet model, and adds category labels for head, hand, neck, leg, and foot. The detection of the attribute information of the target person in the video includes: facial feature extraction, gender detection, glasses detection, mask detection, hat detection, hairstyle detection, bag detection, backpack detection, upper garment color detection, lower garment color detection, and environmental information of the target person area. The above-mentioned detection information is collected and input into the next layer. During the human-computer interaction process, the detection information that is judged to be correct enters the sample generation module to generate new training samples. Using the YOLO model, the detection of objects around the target person area is realized, and the object position information and classification probability information are transformed into a regression problem. The model generates candidate boxes and predictive boxes in each grid and compares them. By iteratively training, the loss value is reduced, and the target person surrounding object detection model is obtained. Its loss function is shown in (3): in, This indicates that the i-th grid and the j-th box contain objects. This indicates that the i-th grid and the j-th box do not contain any objects. The first two lines of the formula represent the position loss of the boxes, and x and y are the horizontal and vertical coordinates of the box positions. Let p be the true x and y coordinates of the i-th box. i (c) represents the category confidence score for the i-th target, C i ω represents the confidence level of the box location. i and h i These are the offset and height of the target offset box, respectively; The system detects behavioral information of people in the video, including: detection of people's movement direction, detection of the number of people gathered together, monitoring of people riding bicycles, detection of people smoking, detection of people's dangerous behavior, detection of people leaving behind items, and detection of people driving vehicles. All of these behavioral detection information are collected and input into the next layer. During the human-computer interaction process, the detection information that is judged as correct by the human enters the sample generation module to generate new training samples. The behavior and attribute information that are correctly detected after receiving human-computer interaction are combined with video data to create new training samples. The generated training samples are sent to the deep learning model update server to train a new structured detection model for human videos. All models used in the face video structuring system are periodically iterated and updated. The retrained models can be manually controlled or periodically and automatically updated into the face video structuring system. The detection accuracy of the current target detection model is determined using formula (4), including: Wherein, accuracy is the detection precision; TP represents the number of correctly predicted targets when the ground truth is a person; FP represents the number of correctly predicted targets when the ground truth is background; FN represents the number of correctly predicted targets when the ground truth is a person; TN represents the number of correctly predicted targets when the ground truth is background; the step of generating a detection model based on the recognition data and the corresponding video data includes: The identification data and the corresponding video data are used as sample data. The sample data are divided according to a preset ratio to obtain a training dataset and a validation dataset. The training dataset is input into a deep learning-based model to learn from the training dataset and obtain a preliminary detection model for recognizing people in complex scenes. The validation dataset is input into the preliminary detection model to validate the preliminary detection model. Based on the validation data, the parameters of the preliminary detection model are adjusted to generate a detection model. The various data models include: video data decoding model, video data preprocessing model, person / target separation model, person / target structure decomposition model, attribute information detection model, and behavior information detection model. The video data decoding model is used to decode video data to obtain decoded video data. The video data preprocessing model is used to perform color and distortion correction, normalization, and image enhancement on the decoded data to obtain preprocessed data of the decoded data. The character target separation model is used to separate character targets from scenes in preprocessed data to obtain character target data. The character target structure decomposition model is used to decompose the structure of most character target data to obtain the character structure decomposition data. The attribute information detection model is used to extract features from the structural decomposition data of the person to obtain the person's feature data. The behavior information detection model is used to determine a person's behavior characteristics based on their feature data, so as to obtain the person's behavior feature data.

2. The method according to claim 1, characterized in that, The identification data includes: video data decoding data, decoded data preprocessing data, person target data, person structural decomposition data, person feature data, and person behavioral feature data.

3. The method according to claim 1, characterized in that, The method further includes: generating structured information from the identification data, and uploading the structured information to an external system through a generalized calling interface.

4. The method according to claim 3, characterized in that, The external systems include: access control systems, monitoring systems, or public systems.

5. A system for recognizing people in complex scenes based on deep learning, characterized in that, The system includes: The parsing unit is used to collect video data in the complex scene to be monitored based on the video acquisition device in the complex scene to be monitored, and to parse and identify the video data based on multiple data models to obtain the identification data of the video data. The model generation unit is used to generate a detection model based on the recognition data and the corresponding video data. The acquisition unit, based on a video acquisition device within the target complex scene, acquires target video data within the target complex scene; The recognition unit is used to input the target video data into the detection model, and to detect the target video data based on the detection model in order to obtain the human recognition data in the target complex scene; Among them, the detection model is used to segment the target area of ​​the person in the image from various complex backgrounds, roughly remove single invalid background areas, and highlight the related areas of the person as input for subsequent modules. The core model of this module is RetinaNet. The RetinaNet model addresses the problem of poor segmentation caused by the imbalance between the target area of ​​the person and the background. It adjusts the loss function to eliminate the extreme imbalance between the target area of ​​the foreground person and the background and to mine hard-to-segment samples. Its original loss function formula is shown in (1): Where y∈{±1} is the true label of the category, 1 is the target label of the foreground person, -1 is the background label, and p∈[0,1] is the probability that the model predicts the label as 1. The modified formula is shown in (2): FL(p t )=-a t (1-p t ) γ log(p t ) (2) in: (1-p t ) γ As an adjustment factor, γ can adjust the loss of the easily classified targets in the model. When the value is larger, the loss value of the easily classified part is smaller and approaches 0. γ is set to 0.65, while the loss value of the difficult-to-classify part is still large, which effectively solves the problem of loss function failure caused by class imbalance. In iterative training, the difficult-to-classify samples contribute more loss value. The deep learning-based semantic segmentation model decomposes the various structures of a person into head, hand, neck, chest, leg, and foot regions, which are used as inputs for subsequent detection of person attribute information and behavior information. This module takes the foreground person target output by the person target separation module as new input, uses the RetinaNet model, and adds category labels for head, hand, neck, leg, and foot. The detection of the attribute information of the target person in the video includes: facial feature extraction, gender detection, glasses detection, mask detection, hat detection, hairstyle detection, bag detection, backpack detection, upper garment color detection, lower garment color detection, and environmental information of the target person area. The above-mentioned detection information is collected and input into the next layer. During the human-computer interaction process, the detection information that is judged to be correct enters the sample generation module to generate new training samples. Using the YOLO model, the detection of objects around the target person area is realized, and the object position information and classification probability information are transformed into a regression problem. The model generates candidate boxes and predictive boxes in each grid and compares them. By iteratively training, the loss value is reduced, and the target person surrounding object detection model is obtained. Its loss function is shown in (3): in, This indicates that the i-th grid and the j-th box contain objects. This indicates that the i-th grid and the j-th box do not contain any objects. The first two lines of the formula represent the position loss of the boxes, and x and y are the horizontal and vertical coordinates of the box positions. Let p be the true x and y coordinates of the i-th box. i (c) represents the category confidence score for the i-th target, C i ω represents the confidence level of the box location. i and h i These are the offset and height of the target offset box, respectively; The system detects behavioral information of people in the video, including: detection of people's movement direction, detection of the number of people gathered together, monitoring of people riding bicycles, detection of people smoking, detection of people's dangerous behavior, detection of people leaving behind items, and detection of people driving vehicles. All of these behavioral detection information are collected and input into the next layer. During the human-computer interaction process, the detection information that is judged as correct by the human enters the sample generation module to generate new training samples. The behavior and attribute information that are correctly detected after receiving human-computer interaction are combined with video data to create new training samples. The generated training samples are sent to the deep learning model update server to train a new structured detection model for human videos. All models used in the face video structuring system are periodically iterated and updated. The retrained models can be manually controlled or periodically and automatically updated into the face video structuring system. The detection accuracy of the current target detection model is determined using formula (4), including: Where accuracy is the detection precision; TP represents the number of ground truth values ​​labeled as human targets and the model correctly predicts them as human targets; FP represents the number of ground truth values ​​labeled as background and the model predicts them as human targets; FN represents the number of ground truth values ​​labeled as human targets and the model predicts them as background targets; TN represents the number of ground truth values ​​labeled as background and the model predicts them as background targets. The step of generating a detection model based on the identification data and the corresponding video data includes: The identification data and the corresponding video data are used as sample data. The sample data are divided according to a preset ratio to obtain a training dataset and a validation dataset. The training dataset is input into a deep learning-based model to learn from the training dataset and obtain a preliminary detection model for recognizing people in complex scenes. The validation dataset is input into the preliminary detection model to validate the preliminary detection model. Based on the validation data, the parameters of the preliminary detection model are adjusted to generate a detection model. The various data models include: video data decoding model, video data preprocessing model, person / target separation model, person / target structure decomposition model, attribute information detection model, and behavior information detection model. The video data decoding model is used to decode video data to obtain decoded video data. The video data preprocessing model is used to perform color and distortion correction, normalization, and image enhancement on the decoded data to obtain preprocessed data of the decoded data. The character target separation model is used to separate character targets from scenes in preprocessed data to obtain character target data. The character target structure decomposition model is used to decompose the structure of most character target data to obtain the character structure decomposition data. The attribute information detection model is used to extract features from the structural decomposition data of the person to obtain the person's feature data. The behavior information detection model is used to determine a person's behavior characteristics based on their feature data, so as to obtain the person's behavior feature data.

6. The system according to claim 5, characterized in that, The identification data includes: video data decoding data, decoded data preprocessing data, person target data, person structural decomposition data, person feature data, and person behavioral feature data.

7. The system according to claim 5, characterized in that, The identification unit is also used to generate structured information from the identification data and upload the structured information to an external system through a generalized calling interface. The external system includes an access control system, a monitoring system, or a public system.

8. The system according to claim 5, characterized in that, External systems include: access control systems, surveillance systems, or public systems.

9. A computer device, characterized in that, include: One or more processors; A processor is used to execute one or more programs; When the one or more programs are executed by the one or more processors, the method described in any one of claims 1-4 is implemented.

10. A computer-readable storage medium, characterized in that, It contains a computer program, which, when executed, implements the method as described in any one of claims 1-4.

Images