A method, device, equipment and storage medium for identifying and assisting disabled persons

Abstract

The present invention discloses a method, device, equipment and storage medium for identifying disabled people and calling for help, and relates to the field of video surveillance technology. The method is to first locate each pedestrian in the pedestrian crossing waiting area according to the on-site video data collected by the camera in real time, obtain the pedestrian positioning result, then determine the key pedestrians through the walker recognition result, and then obtain the posture estimation results of each key pedestrian through posture estimation processing, and finally for each key pedestrian, if the corresponding posture estimation result is found to be the walker use posture, then determine that the corresponding pedestrian is a disabled person with limited mobility, and trigger the corresponding walking assistance call action, so as to achieve the purpose of automatically identifying disabled people with limited mobility and triggering the walking assistance call action, and then remind other pedestrians to provide walking assistance to disabled people with limited mobility, and reduce the risk of disabled people crossing the street.

Description

Technical Field

[0001] The present invention belongs to the field of video surveillance technology, and specifically relates to a method, device, equipment and storage medium for identifying and assisting disabled persons. Background Art

[0002] Video surveillance has a long history and is traditionally widely used in the security field. It is an important means to assist public security departments in combating crime and maintaining social stability. With the popularization of broadband, the development of computer technology and the improvement of image processing technology, video surveillance is increasingly penetrating into various other fields such as education, government, entertainment, medical care, hotels and sports.

[0003] In social life, there are often special scenes where disabled people with mobility impairments (such as blind people or people with mobility impairments) travel alone and cross the street alone. Since disabled people generally do not want to trouble others and rarely ask for help, and even if someone notices a disabled person crossing the street, they rarely offer a hand or other assistance because the disabled person does not ask for help, which leads to certain risks for disabled people crossing the street. Therefore, how to improve the existing video surveillance technology to automatically identify such disabled people with walking difficulties and trigger a call for assistance is a topic that technicians in this field urgently need to study. Summary of the invention

[0004] The purpose of the present invention is to provide a method, device, computer equipment and computer-readable storage medium for identifying disabled people and calling for walking assistance, so as to solve the problem that existing video surveillance solutions cannot automatically detect and identify disabled people with walking difficulties and trigger a walking assistance call action.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] First, a method for identifying and assisting disabled persons is provided, including:

[0007] Obtain live video data collected by cameras in real time in the pedestrian crossing waiting area;

[0008] According to the on-site video data, each pedestrian appearing in the pedestrian crossing waiting area is located to obtain a pedestrian positioning result, wherein the pedestrian positioning result includes a human body marking frame of each pedestrian in a video frame image of the on-site video data;

[0009] For each pedestrian, extract a corresponding pedestrian video frame image from the on-site video data according to a corresponding human body marker frame;

[0010] For each of the pedestrians, the corresponding pedestrian video frame image is imported into a pre-trained walker recognition model based on a target detection algorithm to obtain a corresponding walker recognition result;

[0011] For each of the pedestrians, if the corresponding walking aid recognition result indicates the presence of a walking aid, the corresponding pedestrian is regarded as a key pedestrian;

[0012] For each focused pedestrian, perform corresponding posture estimation processing according to the corresponding pedestrian video frame image to obtain the corresponding posture estimation result;

[0013] For each of the key pedestrians, if the corresponding posture estimation result is found to be a walking aid use posture, the corresponding pedestrian is determined to be a disabled person with limited mobility, and a corresponding walking aid call action is triggered.

[0014] Based on the above invention content, a new video surveillance solution is provided, which can automatically identify disabled people with walking difficulties and trigger a call for walking assistance. That is, first, based on the on-site video data collected by the camera in real time, each pedestrian appearing in the pedestrian crossing waiting area is located to obtain the pedestrian positioning result, and then the key pedestrians are determined by the walker recognition result, and then the posture estimation results of each key pedestrian are obtained through posture estimation processing. Finally, for each key pedestrian, if the corresponding posture estimation result is found to be a walker use posture, the corresponding pedestrian is determined to be a disabled person with walking difficulties, and the corresponding call for walking assistance is triggered. In this way, the purpose of automatically identifying disabled people with walking difficulties and triggering a call for walking assistance can be achieved, and then other pedestrians can be reminded to provide walking assistance to disabled people with walking difficulties, reduce the risk of disabled people crossing the street, and facilitate practical application and promotion.

[0015] In a possible design, based on the on-site video data, each pedestrian appearing in the pedestrian crossing waiting area is located to obtain a pedestrian positioning result, including:

[0016] Using a multi-scale sliding window scanning method to locate each human body appearing in the video frame image of the live video data, and obtaining a positioning window of each human body, wherein the positioning window refers to a sliding window used to locate and determine the corresponding human body;

[0017] For each human body, the corresponding human body is regarded as a pedestrian appearing in the pedestrian crossing waiting area, and a human body marking frame of the corresponding pedestrian in the video frame image of the on-site video data is obtained according to the corresponding positioning window.

[0018] In a possible design, for a particular focused pedestrian, corresponding posture estimation processing is performed according to the corresponding pedestrian video frame image to obtain the corresponding posture estimation result, including:

[0019] According to the pedestrian video frame image of the pedestrian of particular interest, a human body instance mask of the pedestrian of particular interest is obtained using a segmentation network;

[0020] Passing the human instance mask to a pose interpreter network, and outputting 6-DoF pose estimation data of the particular pedestrian of interest;

[0021] The 6-DoF posture estimation data is imported into a posture estimation model based on a neural network and has been pre-trained, and a posture estimation result of the pedestrian of interest is output, wherein the posture estimation result includes a result estimated as a walker-using posture or a walker-carrying posture.

[0022] In a possible design, for a particular focused pedestrian, corresponding posture estimation processing is performed according to the corresponding pedestrian video frame image to obtain the corresponding posture estimation result, including:

[0023] Extracting continuous video frame feature data of the certain focused pedestrian from the pedestrian video frame image of the certain focused pedestrian using a histogram of oriented gradients descriptor and a normalized aspect ratio;

[0024] The continuous video frame feature data of the key pedestrian is imported into a continuous viewpoint posture estimation model based on a support vector machine and which has completed iterative optimization training in advance, and a continuous viewpoint posture estimation result of the key pedestrian is output, wherein the continuous viewpoint posture estimation result includes a result estimated as a walker usage posture or a walker carrying posture.

[0025] In a possible design, for a pedestrian of particular interest whose posture estimation result is a walking aid use posture, a corresponding walking aid call action is triggered, including:

[0026] Acquire speed measurement data collected concurrently with the on-site video data, wherein the speed measurement data is collected by a speed measurement radar from each pedestrian in the pedestrian crossing waiting area, and the speed measurement data contains the radar-recognized walking speed of each pedestrian;

[0027] Determine the radar-recognized walking speed of the pedestrian of particular interest by using radar-visual fusion technology based on the speed measurement data and the on-site video data;

[0028] Calculate the time required for the pedestrian to cross the street based on the radar-recognized walking speed of the pedestrian and the known length of the pedestrian crossing passage corresponding to the pedestrian crossing waiting area;

[0029] Determine whether the time required for the pedestrian of particular concern to cross the street is greater than the known green light time of the pedestrian crossing passage;

[0030] If so, a corresponding pedestrian assistance call action is triggered for the particular pedestrian of focus.

[0031] In a possible design, for a pedestrian of particular interest whose posture estimation result is a walking aid use posture, a corresponding walking aid call action is triggered, including:

[0032] Extracting character portrait information of the certain focused pedestrian from the pedestrian video data of the certain focused pedestrian, wherein the character portrait information includes facial type and / or body height;

[0033] A walking assistance call text message containing the portrait information of the character is broadcasted by the flying base station to remind other pedestrians who are holding text message receiving devices and appearing in the pedestrian crossing waiting area to provide walking assistance to the key pedestrian, wherein the signal coverage range of the flying base station includes the pedestrian crossing waiting area.

[0034] In a possible design, when the field of view of the camera also includes a pedestrian crossing passage corresponding to the pedestrian crossing waiting area, after triggering a corresponding walking aid call action for a certain focused pedestrian whose posture estimation result is a walking aid use posture, the method further includes:

[0035] According to the on-site video data, a multi-target tracking algorithm is used to track each pedestrian appearing in the pedestrian crossing passage to obtain a pedestrian tracking result;

[0036] According to the pedestrian tracking result, it is determined whether there is a pedestrian whose trajectory is synchronized with the pedestrian of particular interest, and if so, a pedestrian video frame image of the pedestrian is extracted from the on-site video data;

[0037] According to the pedestrian video frame image of the pedestrian, using a face recognition algorithm to obtain the identity information of the pedestrian;

[0038] Generate a helpfulness record event containing the identity information of the pedestrian, and upload the helpfulness record event to a personal credit management platform to improve the personal credit of the pedestrian.

[0039] In a second aspect, a device for identifying and assisting pedestrians is provided, comprising a data acquisition module, a pedestrian positioning module, an image extraction module, a walking aid identification module, a key pedestrian determination module, a posture estimation module and a disabled person confirmation module which are sequentially connected in communication;

[0040] The data acquisition module is used to acquire the on-site video data collected in real time by the camera in the pedestrian crossing waiting area;

[0041] The pedestrian positioning module is used to locate each pedestrian appearing in the pedestrian crossing waiting area according to the on-site video data to obtain a pedestrian positioning result, wherein the pedestrian positioning result includes a human body marking frame of each pedestrian in the video frame image of the on-site video data;

[0042] The image extraction module is used to extract the corresponding pedestrian video frame image from the on-site video data according to the corresponding human body mark frame for each pedestrian;

[0043] The walker recognition module is used for importing the corresponding pedestrian video frame image into a walker recognition model that has been pre-trained based on a target detection algorithm for each pedestrian, to obtain a corresponding walker recognition result;

[0044] The key pedestrian determination module is used for, for each of the pedestrians, if the corresponding pedestrian aid recognition result indicates the presence of a pedestrian aid, to identify the corresponding pedestrian as a key pedestrian;

[0045] The posture estimation module is used to perform corresponding posture estimation processing on each focused pedestrian according to the corresponding pedestrian video frame image to obtain a corresponding posture estimation result;

[0046] The disabled person confirmation module is used to determine that the corresponding pedestrian is a disabled person with limited mobility and trigger a corresponding walking aid call action for each of the key pedestrians if the corresponding posture estimation result is found to be a walking aid use posture.

[0047] In a third aspect, the present invention provides a computer device comprising a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the method for identifying and assisting disabled persons as described in the first aspect or any possible design of the first aspect.

[0048] In a fourth aspect, the present invention provides a computer-readable storage medium having instructions stored thereon. When the instructions are executed on a computer, the method for identifying and assisting disabled persons as described in the first aspect or any possible design of the first aspect is executed.

[0049] In a fifth aspect, the present invention provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the method for identifying and assisting disabled persons as described in the first aspect or any possible design of the first aspect.

[0050] Beneficial effects of the above scheme:

[0051] (1) The present invention creatively provides a new video surveillance solution that can automatically identify disabled people with walking difficulties and trigger a walking assistance call action, that is, first, based on the on-site video data collected by the camera in real time, each pedestrian appearing in the pedestrian crossing waiting area is located to obtain the pedestrian positioning result, and then the pedestrian of focus is determined by the walking aid recognition result, and then the posture estimation results of each key pedestrian are obtained through posture estimation processing. Finally, for each key pedestrian, if the corresponding posture estimation result is found to be a walking aid use posture, the corresponding pedestrian is determined to be a disabled person with walking difficulties, and the corresponding walking assistance call action is triggered. In this way, the purpose of automatically identifying disabled people with walking difficulties and triggering a walking assistance call action can be achieved, and other pedestrians can be reminded to provide walking assistance to disabled people with walking difficulties, thereby reducing the risk of disabled people crossing the street;

[0052] (2) The accuracy of posture estimation of pedestrians can be improved by using continuous viewpoint posture estimation technology, thereby further improving the accuracy of automatic identification of people with disabilities who have difficulty walking.

[0053] (3) The walking speed and the time required to cross the street of a disabled person with walking difficulties can also be obtained through the radar and vision fusion technology. Only when the time required to cross the street is longer than the known green light duration of the pedestrian crossing, the call for help will be triggered. In addition, other pedestrians can be reminded to provide walking assistance only when the disabled person has difficulty crossing the street, thereby achieving the purpose of efficiently utilizing the assistance resources of others;

[0054] (4) Multi-target tracking technology can also be used to identify the helpers and recognize their identity information. Finally, by uploading the record of helping others containing the helpers' identity information to the personal credit management platform, it can help improve the personal credit of the helpers, thereby achieving the purpose of encouraging and rewarding the helpers, which is convenient for practical application and promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 A flowchart of the method for identifying and assisting disabled persons provided in an embodiment of the present application.

[0057] Figure 2 This is a schematic diagram of the structure of the device for identifying and assisting walking provided in an embodiment of the present application.

[0058] Figure 3A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0060] It should be understood that although the terms first and second, etc. may be used herein to describe various objects, these objects should not be limited by these terms. These terms are only used to distinguish one object from another object. For example, a first object can be referred to as a second object, and similarly, a second object can be referred to as a first object without departing from the scope of the exemplary embodiments of the present invention.

[0061] It should be understood that the term "and / or" that may appear in this document is merely a description of the association relationship between associated objects, indicating that there may be three relationships. For example, A and / or B can indicate three situations: A exists alone, B exists alone, or A and B exist at the same time. For another example, A, B and / or C can indicate the existence of any one of A, B and C or any combination of them. The term " / and" that may appear in this document describes another type of association object relationship, indicating that there may be two relationships. For example, A / and B can indicate two situations: A exists alone or A and B exist at the same time. In addition, the character " / " that may appear in this document generally indicates that the previous and next associated objects are in an "or" relationship.

[0062] Example:

[0063] like Figure 1As shown, the method for identifying disabled people and calling for walking assistance provided in the first aspect of this embodiment can be, but is not limited to, executed by a computer device with certain computing resources and which is respectively connected to the camera, speed radar and flying base station in communication, such as an intelligent roadside unit (RSU, a key basic equipment for networking and intelligentization of road infrastructure in a vehicle-road cooperative system), a platform server, a personal computer (PC, a multi-purpose computer with a size, price and performance suitable for personal use; desktops, laptops, small laptops, tablets and ultrabooks are all personal computers), smart phones, personal digital assistants (PDA) or wearable devices and other electronic devices. Figure 1 As shown, the method for identifying disabled persons and calling for walking assistance may include, but is not limited to, the following steps S1 to S7.

[0064] S1. Obtain live video data collected by a camera in real time in a pedestrian crossing waiting area.

[0065] In step S1, the camera is a commonly used video surveillance camera, which is mainly used to remotely monitor the pedestrian crossing waiting area (which is connected to the end area of ​​the pedestrian crossing passage) and retain video data. The camera's lens field of view covers the pedestrian crossing waiting area, and is used to collect video frame images of the pedestrian crossing waiting area in real time to obtain on-site video data containing a number of continuous video frame images. In addition, the camera can transmit the collected data to a local device in a conventional manner.

[0066] S2. Based on the on-site video data, locate each pedestrian appearing in the pedestrian crossing waiting area to obtain a pedestrian positioning result, wherein the pedestrian positioning result includes but is not limited to a human body marking box of each pedestrian in the video frame image of the on-site video data.

[0067] In the step S2, specifically, the following steps S21 to S22 are included but are not limited to: S21. Use a multi-scale sliding window scanning method to locate each human body appearing in the video frame image of the live video data, and obtain the positioning window of each human body, wherein the positioning window refers to a sliding window used to locate and determine the corresponding human body; S22. For each human body, the corresponding human body is regarded as a pedestrian appearing in the pedestrian crossing waiting area, and the human body marking frame of the corresponding pedestrian in the video frame image of the live video data is obtained according to the corresponding positioning window. In the aforementioned step S21, the multi-scale sliding window scanning method is a commonly used human body detection technology, and its technical idea is: first scale the image to different scales, then use a sliding window of a fixed size to slide on the entire image with an equidistant step length, and perform human body detection on each sliding window, thereby obtaining the positioning window of each human body, and the human body marking frame can be obtained according to the position of the positioning window in the video frame image of the live video data.

[0068] S3. For each pedestrian, extract a corresponding pedestrian video frame image from the on-site video data according to the corresponding human body marker frame.

[0069] In step S3, the specific image extraction process is: for each pedestrian, the image located in the corresponding human body mark frame is cut out from the video frame image of the on-site video data, and the image is used as the corresponding pedestrian video frame image. In addition, for multiple video frame images that are continuous in time sequence, multiple pedestrian video frame images that are still continuous in time sequence can be extracted.

[0070] S4. For each of the pedestrians, the corresponding pedestrian video frame image is imported into a walker recognition model that has been pre-trained and based on a target detection algorithm to obtain a corresponding walker recognition result.

[0071] In step S4, the walking aid refers to an auxiliary tool for assisting the blind or disabled people with mobility impairments to walk, specifically including but not limited to a guide stick, a cane, an elbow stick and / or a walking frame. The target detection algorithm is an existing artificial intelligence recognition algorithm for identifying objects in a picture and marking the position of the objects. Specifically, it can be but not limited to Faster R-CNN (Faster Regions with Convolutional Neural Networks features, a target detection algorithm proposed by He Kaiming et al. in 2015, which won multiple first places in the 2015 ILSVRV and COCO competitions) target detection algorithm, SSD (Single Shot MultiBox Detector, a target detection algorithm proposed by Wei Liu at ECCV 2016, which is one of the currently popular main detection frameworks) target detection algorithm or YOLO (You only look once, currently the latest version has been developed to V4, and is also widely used in the industry. Its basic principle is: first, the input image is divided into 7x7 grids, and 2 borders are predicted for each grid. Then, according to a threshold, target windows with relatively low probability are removed, and finally, redundant windows are removed by border merging to obtain a detection result) target detection algorithm, etc. Therefore, the walker recognition process can be performed based on the target detection algorithm to obtain a walker recognition result. The target detection algorithm preferably adopts the YOLO v4 target detection algorithm, wherein the specific model structure of the YOLO v4 target detection algorithm consists of three parts, namely the backbone network backbone, the neck network neck and the head network head. The backbone network Backbone can adopt the CSPDarknet53 (CSP stands for Cross Stage Partial) network for feature extraction. The neck network neck is composed of the SPP (Spatial Pyramid Pooling block) block and the PANet (Path Aggregation Network) network, the former is used to increase the receptive field and separate the most important features, and the latter is used to ensure that semantic features are received from the high-level layer and fine-grained features are received from the low-level layer of the horizontal backbone network at the same time. The head network head performs detection based on the anchor frame and detects three different sizes of feature maps: a feature map of size 13×13, a feature map of size 26×26, and a feature map of size 52×52, which are used to detect targets from large to small respectively (here, the feature map of large size contains richer information, so the feature map of size 52×52 is used to detect small targets, and vice versa).

[0072] S5. For each of the pedestrians, if it is found that the corresponding walking aid recognition result indicates the presence of a walking aid, the corresponding pedestrian is regarded as a key pedestrian.

[0073] In step S5, since the presence of a walker is identified in the pedestrian video frame image, the corresponding pedestrian may be a carrier of the walker, or a blind person or a disabled person with mobility impairment who uses a walker, and thus the pedestrian needs to be the focus of attention.

[0074] S6. For each focused pedestrian, perform corresponding posture estimation processing according to the corresponding pedestrian video frame image to obtain a corresponding posture estimation result.

[0075] In the step S6, specifically, for a certain focused pedestrian, corresponding posture estimation processing is performed according to the corresponding pedestrian video frame image to obtain a corresponding posture estimation result, including but not limited to the following steps S61 to S63.

[0076] S61. According to the pedestrian video frame image of the certain focused pedestrian, a human body instance mask of the certain focused pedestrian is obtained using a segmentation network.

[0077] In the step S61, the segmentation network preferably uses a dilated residual network (DRN) trained for semantic segmentation so as to serve as the first component in the end-to-end system. The segmentation network is used to receive real RGB images and output segmentation labels, which are converted into binary instance masks with associated object categories and fed into the subsequent posture interpreter network. Although the dilated residual network also uses downsampling to increase the receptive field size compared to the conventional residual network, it is different at the cost of spatial accuracy. The dilated residual network uses dilated convolution to maintain spatial resolution while maintaining a high receptive field. Therefore, these networks are particularly suitable for dense prediction tasks such as semantic segmentation. Compared with other semantic segmentation architectures such as SegNet or DeepLab, the present embodiment observes that the DRN trained on the RGB image dataset of the present embodiment generates higher fidelity segmentation results and a lower false positive rate. The present embodiment assumes that there is at most one instance for each object category when using the semantic segmentation model for instance segmentation. However, the system of the present embodiment can adapt to processing multiple instances by simply replacing the semantic segmentation component with an instance segmentation model such as Mask R-CNN. In addition, the segmentation network needs to be conventionally trained on real RGB images before use in order to achieve the purpose of accurately obtaining the human instance mask.

[0078] S62. Pass the human body instance mask to the posture interpreter network, and output the 6-DoF posture estimation data of the pedestrian of particular interest.

[0079] In step S62, 6-DoF refers to the six degrees of freedom of an object in three-dimensional space, namely three degrees of freedom of translation and three degrees of freedom of rotation. In the field of computer vision, 6-DoF is usually used to describe the pose (position and orientation) of an object. In this embodiment, the position p = (x, y, z) and the orientation q = (q0, q x ,q y ,q z ) are used to represent the pose of the object, which are translations and rotations relative to the camera coordinate system. Any given rotation can have multiple equivalent forms, and this embodiment finds that it is crucial to force only one unique form. For example, the axis-angle rotation (ω, θ) is equivalent to (-ω, -θ), that is, a rotation of -θ around the axis -ω. These two forms are resolved as unique forms in the unit quaternion. Rotation (ω, θ) is also equivalent to (-ω, 2π-θ), which is resolved as -q in the unit quaternion. This embodiment solves the double coverage between q and -q by requiring the real part q0 of the quaternion to be non-negative, which is equivalent to limiting the rotation angle θ to the range (-π, π). The posture interpreter network is used to operate on a single object instance of a known object category and is trained entirely with synthetic data. The network adopts a simple CNN architecture, including a ResNet-18 feature extractor and a multi-layer perceptron. This embodiment removes the global average pooling layer from the feature extractor to retain the spatial information in the feature map. The multilayer perceptron consists of a fully connected layer with 256 nodes, followed by two parallel branches, corresponding to position and orientation, respectively. Each branch includes another separate fully connected layer, and each object category has a separate output set. This embodiment uses five object categories to train the posture interpreter network of this embodiment, so the position branch has 15 outputs and the orientation branch has 20 outputs. The quaternion orientation output is normalized to unit size. This embodiment finds that this normalization is very critical because it is difficult to directly regress the unit quaternion value. By normalizing the output, the network of this embodiment can predict the relative weights of the four quaternion components.

[0080] In step S62, this embodiment can use point cloud L1 loss for object pose prediction. This embodiment compares the proposed loss with several baseline loss functions. The simplest baseline is the L1 loss of the target and output pose: The weighted constant α is used to balance the position and direction terms. This embodiment also considers a modified version of L1, in which this embodiment uses the direction loss of PoseCNN, which approximates the minimum distance between the target and the predicted direction, that is: Next, this embodiment proposes a new loss function, called L3, which operates entirely in three-dimensional space, rather than quaternion space. Using the three-dimensional models of the objects in the synthetic dataset of this embodiment, this embodiment generates a point cloud representing each object. Given a target pose and an output pose of an object, this embodiment transforms the point cloud of the object using the target and output poses, and compares the two transformed point clouds. This embodiment calculates the L3 loss between corresponding point pairs as follows: Among them, each x (i) is one of the m points in the object point cloud, and the function H converts the object pose (p, q) into an equivalent transformation matrix. This loss function directly compares the point clouds in 3D space and does not require adjustment of additional hyperparameters α, as required in L1 and L2, to balance the position and orientation loss components. This embodiment requires that the first component q0 of the unit quaternion direction is non-negative. In practice, this embodiment finds that adding an additional term to penalize predictions with negative q0 helps convergence. Therefore, this embodiment prefers to use the following L4 loss, which is a variant of the L3 loss proposed in this embodiment, that is,:

[0081] S63. Import the 6-DoF posture estimation data into a posture estimation model based on a neural network and that has been pre-trained, and output a posture estimation result of the pedestrian of interest, wherein the posture estimation result includes a result estimated as a walker-using posture or a walker-carrying posture.

[0082] S7. For each of the key pedestrians, if the corresponding posture estimation result is found to be a walking aid use posture, the corresponding pedestrian is determined to be a disabled person with limited mobility, and a corresponding walking aid call action is triggered.

[0083] In step S7, the specific implementation method of the walking assistance call action may be, but is not limited to, a conventional offline call method such as voice broadcast. In addition, considering that disabled people may mind such highly public offline call methods, other more obscure call methods may also be provided, that is, preferably, for a certain focused pedestrian whose posture estimation result is a walking aid use posture, a corresponding walking assistance call action is triggered, including but not limited to the following steps S71 to S72.

[0084] S71. Extracting character portrait information of the certain key pedestrian from the pedestrian video data of the certain key pedestrian, wherein the character portrait information includes but is not limited to facial type and / or body height, etc.

[0085] In the step S71, the specific method of extracting the character portrait information is an existing conventional method, such as calling the face recognition method provided in the OpenCv library (there are three types, namely, the LBPH method, the EigenFishfaces method and the Fisherfaces method) to identify the face type. In detail, this embodiment adopts the LBPH (Local BinaryPatterns Histogram) method, that is, in the OpenCv library, first use the function cv2.face.LBPHFaceRecognizer_create() to generate an LBPH recognizer instance model, then use the cv2.face_FaceRecognizer.train() function to complete the training of the LBPH recognizer instance model, and finally use the cv2.face_FaceRecognizer.predict() function to complete the face recognition processing based on the pedestrian video data of the certain focused pedestrian and the LBPH recognizer instance model to obtain the face type recognition result of the certain focused pedestrian.

[0086] S72. Send a walking assistance call text message containing but not limited to the portrait information of the person through the flying base station broadcast, so as to remind other pedestrians who are holding text message receiving devices and appearing in the pedestrian crossing waiting area to provide walking assistance to the key pedestrian, wherein the signal coverage range of the flying base station includes the pedestrian crossing waiting area.

[0087] In step S72, the Femto Site is the smallest of the four base stations and is often used as a home base station and is accessed by home broadband. Therefore, it can also be applied to pedestrian crossing scenarios, so that only the SMS receiving device in the pedestrian crossing waiting area can receive the walking assistance call SMS, thereby achieving the purpose of reminding other pedestrians to provide walking assistance to the key pedestrian.

[0088] Therefore, based on the method for identifying disabled people and calling for walking assistance described in the aforementioned steps S1 to S7, a new video surveillance solution is provided that can automatically identify disabled people with walking difficulties and trigger a calling for walking assistance action, that is, first, based on the on-site video data collected by the camera in real time, each pedestrian appearing in the pedestrian crossing waiting area is located to obtain the pedestrian positioning result, and then the key pedestrians are determined through the walking aid recognition result, and then the posture estimation results of each key pedestrian are obtained through posture estimation processing. Finally, for each key pedestrian, if the corresponding posture estimation result is found to be a walking aid use posture, the corresponding pedestrian is determined to be a disabled person with walking difficulties, and the corresponding calling for walking assistance action is triggered. In this way, the purpose of automatically identifying disabled people with walking difficulties and triggering a calling for walking assistance action can be achieved, and then other pedestrians can be reminded to provide walking assistance to disabled people with walking difficulties, reduce the risk of disabled people crossing the street, and facilitate practical application and promotion.

[0089] Based on the technical solution of the first aspect mentioned above, this embodiment also provides another possible design for how to perform posture estimation on a key pedestrian, that is, for a key pedestrian, corresponding posture estimation processing is performed according to the corresponding pedestrian video frame image to obtain a corresponding posture estimation result, including but not limited to the following steps S601 to S602.

[0090] S601. Using a histogram of oriented gradients descriptor and a normalized aspect ratio, extracting continuous video frame feature data of the certain focused pedestrian from the pedestrian video frame image of the certain focused pedestrian.

[0091] In the above-mentioned step S601, the Histogram of Oriented Gradient (HOG) descriptor is a feature descriptor used for object detection in computer vision and image processing, and the normalized aspect ratio is specifically the ratio of the height to the width of the pedestrian video frame image. Therefore, for multiple pedestrian video frame images that are continuous in time sequence, the continuous video frame feature data can be obtained by conventional extraction.

[0092] S602. Import the continuous video frame feature data of the key pedestrian into a continuous viewpoint posture estimation model based on a support vector machine that has pre-completed iterative optimization training, and output a continuous viewpoint posture estimation result of the key pedestrian, wherein the continuous viewpoint posture estimation result includes a result estimated as a walker usage posture or a walker carrying posture.

[0093] In step S602, the support vector machine (SVM) is a generalized linear classifier that performs binary classification on data in a supervised learning manner, and its decision boundary is a maximum-margin hyperplane solved for learning samples, so a conventional iterative optimization algorithm can be used to optimize the model parameters and labels of the continuous viewpoint posture estimation model, and the continuous viewpoint posture estimation model can be used to estimate the posture of the pedestrian of interest. The specific construction and training measures of the continuous viewpoint posture estimation model in this embodiment include but are not limited to the following points (A) to (D).

[0094] (A) In this embodiment, given an example object, the model for each category (i.e., walker use posture or walker carrying posture) returns a confidence score indicating the likelihood that the object belongs to that category, as well as a discrete viewpoint label associated with the canonical pose for that category. In many cases, these poses have semantic meaning, such as the front / side view of a car. This embodiment designs each model as a mixture of HOG-based templates corresponding to multiple canonical poses for that category. The score formula for the example object x is as follows:

[0095]

[0096] In the formula, w={ω1,ω2,...,ω V} is the learned template mixture, v = {1, 2, ..., V}, V is the number of canonical viewpoints in the model, ψ v (x) is the feature representation of x under viewpoint label v. Since the dimensions of the template may be different, ψ v (x) is designed to be v In addition, the predicted viewpoint label of x is: Here, the subscript d represents a discrete label.

[0097] (B) This example prefers HOG-based features because they encode the spatial layout of object shapes and can handle intra-class and intra-view variations well. This example uses an implementation of "Histograms of oriented gradients for human detection" for feature construction and normalization.

[0098] (C) This embodiment adopts a standard multi-scale window scanning framework to localize objects in an image: windows with scores above a learned threshold are selected as candidate detections, and non-maximum suppression is applied as post-processing to eliminate redundant window detections.

[0099] (D) This embodiment extends the training algorithm of "Histograms of oriented gradients for human detection" to cope with viewpoint classification. For each category, from {x1, x2, ..., x P} and {z1, z2, ..., z N} to learn a mixture of V component templates w = {w1, w2, ..., w V}. The learning framework of this embodiment attempts to match each positive example with at least one of these templates and each negative example with none of these templates. Mathematically, the large margin optimization formula of this scheme is:

[0100]

[0101] In the formula, λ is a binary label, l(s) = max(0, 1-s) represents the hinge-loss loss, where s is the score of the classifier and C Pos and C Neg is the regularization parameter.

[0102] (E) Training process: Each template w v Initialized by a set of positive examples initially labeled as viewpoint v. In each iteration, all templates are updated simultaneously by mining hard negative examples and updating the viewpoint labels λ of the positive examples. In this embodiment, the true viewpoint labels of the positive examples are represented as {v d (x1), v d (x2), ..., v d (x P )}. In the following content, this embodiment considers three cases, where these labels are completely known, partially known, and unknown. This embodiment calls them supervised, semi-supervised, and unsupervised cases respectively.

[0103] Under supervision, each are fixed. The model is initialized by dividing the positive examples into groups based on the viewpoint label and learns a viewpoint template from each group. In the model update step, the optimization is simplified to a linear SVM formulation. Although this embodiment does not change the component labels during training, this is different from training each component independently because the training process uses a single regularization constraint and enforces the boundaries of all clusters simultaneously. This has been shown to be crucial for learning a hybrid model to achieve balanced and accurate viewpoint prediction.

[0104] In the semi-supervised case, the present embodiment first builds a multi-view classifier using positive examples with known viewpoint labels. In practice, the present embodiment uses the libsvm multi-class classification toolbox to classify HOG features. Once the remaining positive examples are classified, the present embodiment initializes the component templates based on known or estimated labels. In the model update step, the present embodiment fixes the labels of those known viewpoint labels and allows other labels to change.

[0105] In the unsupervised case, model initialization is crucial for accurate viewpoint classification because no explicit constraints are imposed on viewpoint labels in the later stages of optimization. This example uses a clustering scheme based on normalized cuts for initialization. This example defines two positive examples x i and x j The apparent distance between is:

[0106] d(x i , x j )=α·χ 2 (ψ0(x i ), ψ0(x j ))+(1-α)·||Asp(x i )-Asp(x j )||2

[0107] In the formula, ψ0(x i ) is x i HOG descriptor under standard template size, and Asp(x i ) is x i Normalized aspect ratio of the bounding box. Next, this embodiment converts the distance to an affinity measure using an exponential function and obtains the component groups by applying the normalized cut algorithm on the resulting affinity matrix. This provides a relatively uniform partitioning of the positive examples, which is very important for good unsupervised performance.

[0108] In the model update step, since the aforementioned large margin optimization formula describes a non-convex problem based on integers, a feasible solution is to optimize w with fixed labels λ and optimize λ with fixed template weights w. The former is SVM, while the optimization step of the latter is very simple: In addition, the continuous viewpoint attitude estimation model can also output attitude information such as rotation matrix and translation vector.

[0109] Therefore, based on the above-mentioned possible design one, the accuracy of posture estimation of key pedestrians can be improved by adopting continuous viewpoint posture estimation technology, thereby further improving the accuracy of automatic identification of disabled people with walking difficulties.

[0110] Based on the technical solution of the aforementioned first aspect or possible design one, this embodiment further provides a possible design two for how to identify disabled people who have difficulty crossing the street, that is, for a key pedestrian whose posture estimation result is a walking aid usage posture, triggering a corresponding walking aid call action, including but not limited to the following steps S701 to S705.

[0111] S701. Obtain speed measurement data collected concurrently with the on-site video data, wherein the speed measurement data is collected by a speed measurement radar on each pedestrian appearing in the pedestrian crossing waiting area, and the speed measurement data contains the radar-recognized walking speed of each pedestrian.

[0112] In step S701, the specific method of acquiring the speed measurement data is the prior art content. Specifically, the speed measurement radar is preferably a millimeter wave radar.

[0113] S702. Determine the radar-recognized walking speed of the pedestrian of particular interest through radar-vision fusion technology based on the speed measurement data and the on-site video data.

[0114] In step S702, the specific process of the radar-visual fusion technology is the content of the prior art. Preferably, firstly, according to the on-site video data, the video-recognized walking speed of each pedestrian is obtained by a conventional video speed measurement method; then, according to the speed from large to small or from small to small, the radar-recognized walking speed and the video-recognized walking speed of each pedestrian are sorted respectively, and finally, according to the two sorting results, the radar-recognized walking speed of a certain focused pedestrian is determined by the same order bit matching method.

[0115] S703. Calculate the time required for the key pedestrian to cross the street based on the radar-recognized walking speed of the key pedestrian and the known length of the pedestrian crossing passage corresponding to the pedestrian crossing waiting area.

[0116] In step S703, specifically, the time required for crossing the street is equal to the known length divided by the walking speed recognized by the radar. In addition, when the field of view of the camera also includes a pedestrian crossing passage corresponding to the pedestrian crossing waiting area, the length of the pedestrian crossing passage can also be obtained on-site based on the on-site video data through an existing visual calibration algorithm.

[0117] S704: Determine whether the time required for the pedestrian of particular concern to cross the street is greater than the known green light time of the pedestrian crossing passage.

[0118] S705. If yes, trigger a corresponding pedestrian assistance call action for the particular pedestrian of focus.

[0119] Based on the above-mentioned possible design 2, the walking speed and the time required to cross the street of a disabled person with walking difficulties can be obtained through the radar and vision fusion technology, and the call for assistance will be triggered only when the time required to cross the street is longer than the known green light duration of the pedestrian crossing. Therefore, other pedestrians can be reminded to provide walking assistance only when the disabled have difficulty crossing the street, thereby achieving the purpose of efficiently utilizing the assistance resources of others.

[0120] Based on the technical solutions of the aforementioned first aspect, possible design one or possible design two, this embodiment further provides a possible design three for how to encourage and reward the assisting personnel, that is, when the field of view of the camera also includes the pedestrian crossing passage corresponding to the pedestrian crossing waiting area, after triggering a corresponding walking assistance call action for a key pedestrian whose posture estimation result is a walking aid use posture, the method also includes but is not limited to the following steps S81 to S84.

[0121] S81. Based on the on-site video data, a multi-target tracking algorithm is used to track each pedestrian appearing in the pedestrian crossing passage to obtain a pedestrian tracking result.

[0122] In step S81, the multi-target tracking algorithm is used to detect pedestrian images located in the video frame image, and perform video tracking on the detected pedestrians (during the tracking process, each detected pedestrian can be assigned a unique pedestrian number, such as a digital number such as 1, 2, 3 or 4). The multi-target tracking algorithm preferably adopts a deepsort target tracking algorithm, wherein the specific process of the deepsort target tracking algorithm is as follows: first, a target bounding box bbox is detected by a target detector, and detection target information detections (which is used to save all targets detected in the current frame image) is generated according to the target bounding box bbox, and then the Kalman filter method is used to predict the position of the tracking target information tracks (which is used to save all targets tracked in the previous frame image) in the previous frame image, and then the Mahalanobis distance based on the appearance feature is used to calculate the cost matrix of the tracking target and the detection target, and then the tracking target and the detection target are cascade matched and IOU (Intersection over Union) matched successively, and finally all matching pairs, unmatched tracking targets and unmatched detection targets of the current frame image are obtained, and for each successfully matched tracking target, the corresponding detection target is used to update the position, and the unmatched tracking targets and detection targets are processed. In addition, in the deepsort target tracking algorithm, the target detector, the Kalman filter method, the appearance feature, the Mahalanobis distance, the cosine distance, the cost matrix, the cascade matching, and the IOU matching are all existing terms or technical features, and those skilled in the art can routinely obtain the specific process details of the deepsort target tracking algorithm.

[0123] This embodiment considers that the unmatched tracking target may include the occluded target, but it is not detected in the current frame image. Therefore, in the deepsort target tracking algorithm, it is preferably included: for the unmatched tracking target, judging whether it leaves the pedestrian crossing passage according to the corresponding current predicted position (that is, judging whether it still appears in the frame image according to the positional relationship between the current predicted position and the frame image boundary, if it does, it is judged that it has not left the pedestrian crossing passage, otherwise it is judged that it has left the pedestrian crossing passage), if so, the tracking target is abandoned in the next position prediction, otherwise the Kalman filter method is still used to predict the corresponding position in the next position prediction. In this way, when it is found that the tracking target is occluded (that is, when the tracking target does not match the detection target and the current predicted position still appears in the current frame image), the tracking target can be marked as masked, and then the Kalman filter method is still used to predict the position in the next frame image until the detection target is matched or the tracking target is found to be beyond the range of the image, thereby avoiding the premature termination of video tracking due to occlusion and ensuring the long-term continuity of tracking compared to the general discarding processing method.

[0124] S82. Based on the pedestrian tracking result, determine whether there is a pedestrian whose trajectory is synchronized with the focused pedestrian; if so, extract a pedestrian video frame image of the pedestrian from the on-site video data.

[0125] In step S82, the trajectory synchronization means that at any moment, the ratio of the center distance between the human body marking frames of two pedestrians (i.e., the certain key pedestrian and the certain pedestrian) to the average diagonal length of the human body marking frames corresponding to the two pedestrians is always less than a preset threshold (for example, specifically half of the average diagonal length), so as to determine whether the certain pedestrian has helped the certain key pedestrian to cross the street. If so, the certain pedestrian is determined to be a helping person.

[0126] S83. Based on the pedestrian video frame image of the pedestrian, use a face recognition algorithm to obtain the identity information of the pedestrian.

[0127] In the step S83, the face recognition algorithm may specifically adopt the LBPHFaceRecognizer face recognition algorithm provided in the OpenCv library.

[0128] S84. Generate a helpfulness record event containing the identity information of the pedestrian, and upload the helpfulness record event to a personal credit management platform to improve the personal credit of the pedestrian.

[0129] In step S84, for example, for the same helping person, the corresponding personal credit may be positively correlated with the total number of corresponding helping events until a certain value is reached. In addition, the helping event may also record the on-site video data collected during the street crossing process and including the pedestrian video frame image of the pedestrian.

[0130] Based on the aforementioned possible design three, multi-target tracking technology can also be used to determine the helpers and identify their identity information. Finally, by uploading the record of helping others containing the helpers' identity information to the personal credit management platform, it can help improve the personal credit of the helpers, and thus achieve the purpose of encouraging and rewarding the helpers.

[0131] like Figure 2 As shown, the second aspect of this embodiment provides a virtual device for implementing the method for identifying disabled persons and calling for walking assistance as described in the first aspect or any possible design of the first aspect, comprising a data acquisition module, a pedestrian positioning module, an image extraction module, a walking aid identification module, a key pedestrian determination module, a posture estimation module and a disabled person confirmation module which are sequentially connected in communication;

[0132] The data acquisition module is used to acquire the on-site video data collected in real time by the camera in the pedestrian crossing waiting area;

[0133] The pedestrian positioning module is used to locate each pedestrian appearing in the pedestrian crossing waiting area according to the on-site video data to obtain a pedestrian positioning result, wherein the pedestrian positioning result includes a human body marking frame of each pedestrian in the video frame image of the on-site video data;

[0134] The image extraction module is used to extract the corresponding pedestrian video frame image from the on-site video data according to the corresponding human body mark frame for each pedestrian;

[0135] The walker recognition module is used for importing the corresponding pedestrian video frame image into a walker recognition model that has been pre-trained based on a target detection algorithm for each pedestrian, to obtain a corresponding walker recognition result;

[0136] The key pedestrian determination module is used for, for each of the pedestrians, if the corresponding pedestrian aid recognition result indicates the presence of a pedestrian aid, to identify the corresponding pedestrian as a key pedestrian;

[0137] The posture estimation module is used to perform corresponding posture estimation processing on each focused pedestrian according to the corresponding pedestrian video frame image to obtain a corresponding posture estimation result;

[0138] The disabled person confirmation module is used to determine that the corresponding pedestrian is a disabled person with limited mobility and trigger a corresponding walking aid call action for each of the key pedestrians if the corresponding posture estimation result is found to be a walking aid use posture.

[0139] The working process, working details and technical effects of the aforementioned device provided in the second aspect of this embodiment can be referred to the disabled person identification and walking assistance calling method described in the first aspect or any possible design of the first aspect, and will not be repeated here.

[0140] like Figure 3As shown, the third aspect of this embodiment provides a computer device for executing the method for identifying and assisting the disabled as described in the first aspect or any possible design in the first aspect, including a memory, a processor and a transceiver that are sequentially connected in communication, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the method for identifying and assisting the disabled as described in the first aspect or any possible design in the first aspect. For example, the memory may include, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a flash memory, a first-in-first-out memory (FIFO) and / or a first-in-last-out memory (FILO), etc.; the processor may include, but is not limited to, a microprocessor of the STM32F105 series. In addition, the computer device may also include, but is not limited to, a power module, a display screen and other necessary components.

[0141] The working process, working details and technical effects of the aforementioned computer device provided in the third aspect of this embodiment can be referred to the disabled person identification and walking assistance calling method described in the first aspect or any possible design of the first aspect, and will not be repeated here.

[0142] In a fourth aspect of this embodiment, there is provided a computer-readable storage medium storing instructions including the method for identifying and assisting walking as described in the first aspect or any possible design of the first aspect, that is, the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the method for identifying and assisting walking as described in the first aspect or any possible design of the first aspect is executed. The computer-readable storage medium refers to a carrier for storing data, which may include but is not limited to computer-readable storage media such as floppy disks, optical disks, hard disks, flash memories, USB flash drives, and / or memory sticks, and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0143] The working process, working details and technical effects of the aforementioned computer-readable storage medium provided in the fourth aspect of this embodiment can be referred to the method for identifying disabled persons and calling for assistance as described in the first aspect or any possible design in the first aspect, and will not be repeated here.

[0144] In a fifth aspect, the present embodiment provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the method for identifying and assisting disabled persons as described in the first aspect or any possible design of the first aspect. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0145] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for identifying and assisting disabled persons, characterized in that: include: Obtain live video data collected by cameras in real time in the pedestrian crossing waiting area; According to the on-site video data, each pedestrian appearing in the pedestrian crossing waiting area is located to obtain a pedestrian positioning result, wherein the pedestrian positioning result includes a human body marking frame of each pedestrian in a video frame image of the on-site video data; For each pedestrian, extract a corresponding pedestrian video frame image from the on-site video data according to a corresponding human body marker frame; For each of the pedestrians, the corresponding pedestrian video frame image is imported into a pre-trained walker recognition model based on a target detection algorithm to obtain a corresponding walker recognition result; For each of the pedestrians, if the corresponding walking aid recognition result indicates the presence of a walking aid, the corresponding pedestrian is regarded as a key pedestrian; For each focused pedestrian, perform corresponding posture estimation processing according to the corresponding pedestrian video frame image to obtain the corresponding posture estimation result; For each of the key pedestrians, if it is found that the corresponding posture estimation result is a walking aid using posture, the corresponding pedestrian is determined to be a disabled person with limited mobility, and a corresponding walking assistance call action is triggered. Specifically, for a key pedestrian whose posture estimation result is a walking aid using posture, the corresponding walking assistance call action is triggered, including: extracting character portrait information of the key pedestrian from the pedestrian video data of the key pedestrian, wherein the character portrait information includes facial type and / or body height; broadcasting a walking assistance call text message containing the character portrait information through the flying base station, so as to remind other pedestrians who hold text message receiving devices and appear in the pedestrian crossing waiting area to provide walking assistance to the key pedestrian, wherein the signal coverage range of the flying base station includes the pedestrian crossing waiting area.

2. The method for identifying and assisting disabled persons according to claim 1, characterized in that: According to the on-site video data, each pedestrian in the pedestrian crossing waiting area is located to obtain a pedestrian positioning result, including: Using a multi-scale sliding window scanning method to locate each human body appearing in the video frame image of the live video data, and obtaining a positioning window of each human body, wherein the positioning window refers to a sliding window used to locate and determine the corresponding human body; For each human body, the corresponding human body is regarded as a pedestrian appearing in the pedestrian crossing waiting area, and a human body marking frame of the corresponding pedestrian in the video frame image of the on-site video data is obtained according to the corresponding positioning window.

3. The method for identifying and assisting disabled persons according to claim 1, characterized in that: For a particular focused pedestrian, the corresponding posture estimation process is performed according to the corresponding pedestrian video frame image to obtain the corresponding posture estimation result, including: According to the pedestrian video frame image of the pedestrian of particular interest, a human body instance mask of the pedestrian of particular interest is obtained using a segmentation network; Passing the human instance mask to a pose interpreter network, and outputting 6-DoF pose estimation data of the pedestrian of interest; The 6-DoF posture estimation data is imported into a posture estimation model based on a neural network and has been pre-trained, and a posture estimation result of the pedestrian of interest is output, wherein the posture estimation result includes a result estimated as a walker-using posture or a walker-carrying posture.

4. The method for identifying and assisting disabled persons according to claim 1, characterized in that: For a particular focused pedestrian, the corresponding posture estimation process is performed according to the corresponding pedestrian video frame image to obtain the corresponding posture estimation result, including: Extracting continuous video frame feature data of the certain focused pedestrian from the pedestrian video frame image of the certain focused pedestrian using a histogram of oriented gradients descriptor and a normalized aspect ratio; The continuous video frame feature data of the key pedestrian is imported into a continuous viewpoint posture estimation model based on a support vector machine and which has completed iterative optimization training in advance, and a continuous viewpoint posture estimation result of the key pedestrian is output, wherein the continuous viewpoint posture estimation result includes a result estimated as a walker usage posture or a walker carrying posture.

5. The method for identifying and assisting disabled persons according to claim 1, characterized in that: For a pedestrian of interest whose posture estimation result is a walking aid use posture, a corresponding walking aid call action is triggered, including: Acquire speed measurement data collected concurrently with the on-site video data, wherein the speed measurement data is collected by a speed measurement radar from each pedestrian in the pedestrian crossing waiting area, and the speed measurement data contains the radar-recognized walking speed of each pedestrian; Determine the radar-recognized walking speed of the pedestrian of particular interest by using radar-visual fusion technology according to the speed measurement data and the on-site video data; Calculate the time required for the pedestrian to cross the street based on the radar-recognized walking speed of the pedestrian and the known length of the pedestrian crossing passage corresponding to the pedestrian crossing waiting area; Determine whether the time required for the pedestrian of particular concern to cross the street is greater than the known green light time of the pedestrian crossing passage; If so, a corresponding pedestrian assistance call action is triggered for the particular pedestrian of focus.

6. The method for identifying and assisting disabled persons according to claim 1, characterized in that: When the camera's field of view also includes a pedestrian crossing passage corresponding to the pedestrian crossing waiting area, after triggering a corresponding walking aid call action for a pedestrian of particular interest whose posture estimation result is a walking aid use posture, the method further includes: According to the on-site video data, a multi-target tracking algorithm is used to track each pedestrian appearing in the pedestrian crossing passage to obtain a pedestrian tracking result; According to the pedestrian tracking result, it is determined whether there is a pedestrian whose trajectory is synchronized with the pedestrian of interest, and if so, a pedestrian video frame image of the pedestrian is extracted from the on-site video data; According to the pedestrian video frame image of the pedestrian, using a face recognition algorithm to obtain the identity information of the pedestrian; Generate a helpfulness record event containing the identity information of the pedestrian, and upload the helpfulness record event to a personal credit management platform to improve the personal credit of the pedestrian.

7. A device for identifying and assisting disabled persons, characterized in that: It includes a data acquisition module, a pedestrian positioning module, an image extraction module, a walking aid recognition module, a key pedestrian determination module, a posture estimation module and a disabled person identification module which are sequentially connected in communication; The data acquisition module is used to acquire the on-site video data collected in real time by the camera in the pedestrian crossing waiting area; The pedestrian positioning module is used to locate each pedestrian appearing in the pedestrian crossing waiting area according to the on-site video data to obtain a pedestrian positioning result, wherein the pedestrian positioning result includes a human body marking frame of each pedestrian in the video frame image of the on-site video data; The image extraction module is used to extract the corresponding pedestrian video frame image from the on-site video data according to the corresponding human body mark frame for each pedestrian; The walker recognition module is used for importing the corresponding pedestrian video frame image into a walker recognition model that has been pre-trained based on a target detection algorithm for each pedestrian, to obtain a corresponding walker recognition result; The key pedestrian determination module is used for, for each of the pedestrians, if the corresponding pedestrian aid recognition result indicates the presence of a pedestrian aid, to identify the corresponding pedestrian as a key pedestrian; The posture estimation module is used to perform corresponding posture estimation processing on each focused pedestrian according to the corresponding pedestrian video frame image to obtain a corresponding posture estimation result; The disabled person confirmation module is used for determining, for each of the key pedestrians, that the corresponding pedestrian is a disabled person with limited mobility if the corresponding posture estimation result is found to be a walking aid using posture, and triggering a corresponding walking assistance call action. Specifically, for a key pedestrian whose posture estimation result is a walking aid using posture, triggering a corresponding walking assistance call action includes: extracting character portrait information of the key pedestrian from the pedestrian video data of the key pedestrian, wherein the character portrait information includes facial type and / or body height; and broadcasting a walking assistance call text message containing the character portrait information through the flying base station, so as to remind other pedestrians who hold text message receiving devices and appear in the pedestrian crossing waiting area to provide walking assistance to the key pedestrian, wherein the signal coverage range of the flying base station includes the pedestrian crossing waiting area.

8. A computer device, characterized in that: It includes a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the method for identifying disabled persons and calling for assistance as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on the computer, the method for identifying and assisting disabled people as described in any one of claims 1 to 6 is executed.

Images