Casualty Identification Method and System Based on Deep Learning in Injury Point Scenario

Abstract

The present invention relates to a method and system for identifying injured persons based on a deep learning neural network, wherein the method comprises the following steps: S10, using a depth camera to collect at least one image of an injured person in an environment where an injury is collected, and compiling the images into a live dataset of injured person images; S20, generating additional images of injured persons of smaller size for original images of injured persons taken at close range in the live dataset using a data augmentation method, and associating the additional images with the original images of injured persons and storing them in the live dataset; S30, inputting the live images taken by the depth camera into a deep learning-based neural network to calculate and output the number of injured persons in the live images, wherein the neural network is trained using a pre-training dataset and the live dataset. The system comprises a depth camera, a memory, and a processor, wherein the processor implements the method when executing instructions stored in the memory.

Description

Technical Field

[0001] The present invention relates to a deep learning-based casualty identification method and system in a casualty cluster scenario, belonging to the technical field of computer software, especially robot vision recognition. The technical solution of the present invention is particularly suitable for emergency rescue applications. Background Art

[0002] In large-scale casualties caused by collapse environments such as earthquakes, fires, and accidents, rescue teams generally place the injured in a certain area with the following characteristics: open and far away from the disaster site; flat ground or grass; and transfer features to facilitate the subsequent evacuation of the injured. Such areas where the injured are concentrated are generally called gathering points. In order to quickly classify the injuries of the injured in the gathering point scenario, it is necessary to design a system that can carry navigation sensors and medical sensors to the injured for autonomous injury assessment. Figure 1 For example, in the scenario of concentrated damage points after an earthquake, a four-legged robot dog equipped with a robotic arm, a depth camera, a radar, and various medical sensors acts as a concentrated damage point ( Figure 1 Therefore, a system needs to be designed whose primary task is to quickly identify the location of the casualty, the location of the collection point, and whether there are any casualties in the collection point environment. Summary of the Invention

[0003] The present invention provides a method and system for casualty identification based on deep learning in a clustered injury point scenario, aiming to solve at least one of the technical problems existing in the prior art.

[0004] The technical solution of the present invention is, on the one hand, a method for identifying injured persons based on a deep learning neural network, the method comprising the following steps:

[0005] S10. Collecting at least one image of a wounded person in the wound collection point environment using a depth camera, and compiling the images into a field dataset of the wounded person images;

[0006] S20, generating additional injured person images of smaller size for the original injured person images taken at close range in the on-site dataset in a data augmentation manner, and associating the additional injured person images with the original injured person images and storing them in the on-site dataset;

[0007] S30. Inputting the scene picture taken by the depth camera into a deep learning-based neural network to calculate and output the number of wounded in the scene picture, wherein the neural network is trained by a pre-training dataset and the scene dataset.

[0008] Furthermore, the step S10 includes:

[0009] When the mobile device moves toward the injury collection point environment, image acquisition is performed from a farther position using a depth camera carried by the mobile device, wherein the mobile device includes a quadruped robot dog, a mobile robot, a mobile smart car, or a flying drone.

[0010] Furthermore, step S20 includes:

[0011] S21. Images of injured persons taken at close range are identified as images with a low proportion of small objects;

[0012] S22. Reduce the image size of the picture with a low proportion of small objects to one quarter of the original size, and splice the four reduced images into an image with the same size as the original image.

[0013] Furthermore, the step S21 includes:

[0014] If the area of ​​the human body outline in the captured image is smaller than a preset pixel product threshold, the image is determined to be an image containing a small object.

[0015] Further, for step S30, the neural network includes: a basic network layer of the VGG16 network architecture; an auxiliary convolution layer for extracting feature maps of different scales; and a prediction convolution layer, wherein the prediction convolution layer includes a position prediction convolution layer and a category prediction convolution layer.

[0016] Furthermore, the neural network is configured to: send the feature maps output by the conv4_3 layer, conv7 layer, conv8_2 layer, conv9_2 layer, conv10_2 layer, and conv11_2 layer to the prediction convolution layer to obtain the position information and classification information of the injured person in the picture; perform feature fusion on the feature maps output by the conv4_3 layer, conv7 layer, and conv8_2 layer to form a feature map of the same size as the feature map output by the conv4_3 layer, and replace the feature map of the conv4_3 layer before sending it to the prediction convolution layer.

[0017] Furthermore, the neural network is configured to:

[0018] The H1xW1xC1 feature map of the conv4_3 layer passes through a 3x3 convolution layer, then connects to the L2norm layer, and then connects to the ReLu layer, thereby outputting the first feature map of the same size;

[0019] The H2xW2xC2 feature map of the Conv7 layer is passed through the deconvolution layer, connected to the L2norm layer, and then connected to the ReLu layer, thereby outputting a second feature map of size H1xW1xC1 / 2;

[0020] The feature map of the Conv8_2 layer with a size of H3xW3xC3 is passed through the deconvolution layer, connected to the L2norm layer, and then connected to the ReLu layer, thereby outputting a third feature map with a size of H1xW1xC1 / 2;

[0021] The output first feature map, second feature map and third feature map are concatenated to generate a feature map of size H1xW1xC1*2, where H, W and C represent the size dimensions of the feature map respectively.

[0022] Furthermore, the training of the neural network comprises the following steps:

[0023] S40, calculating the loss value of each point of the feature map output by the prediction convolution layer by a loss function, and updating the model parameters of the neural network during the training process of the neural network until the sum of the loss values ​​of all points of the trained feature map is less than a preset threshold, then stopping the training, wherein the calculation of the loss function includes:

[0024] Calculate the overlap between the predicted box and the true box for each point in the feature map; if the overlap is greater than the set threshold, the predicted box is the same class marked by the true box and is set to the positive class; if the overlap is less than the set threshold, the class marked by the predicted box is considered to be the background and is set to the negative class;

[0025] The loss function between the predicted box and the true box is equal to the sum of the position loss and the classification loss of the predicted box, where

[0026] The position loss of the prediction box is calculated as

[0027]

[0028] The classification loss of the prediction box is calculated as

[0029]

[0030] Among them, L loc Position loss function, L class is the classification loss function, N p is the number of positive prediction boxes, N n is the number of negative prediction boxes, Box i_pred It is the coordinate information of the prediction box, Box i_real It is the coordinate information of the corresponding real frame, Distance() represents the Euclidean distance calculation function between coordinates, and CE_loss represents the cross entropy loss function.

[0031] Furthermore, the training of the neural network may further include the following steps:

[0032] S51. Capture multiple images of injured people from the Internet and add them to the pre-training dataset;

[0033] S52. Using data augmentation, the image size of the original close-up images of the injured person in the pre-data set is reduced to one-fourth of the original size, and the four reduced images are spliced ​​into an image with the same size as the original image. The spliced ​​injured person image is associated with the original injured person image and then stored in the pre-data set.

[0034] Another aspect of the technical solution of the present invention relates to a wounded person identification system, comprising:

[0035] at least one depth camera carried by a mobile device;

[0036] A computer device connected to the depth camera includes a computer-readable storage medium on which program instructions are stored. When the program instructions are executed by a processor, the above method is implemented.

[0037] The beneficial effects of the present invention are as follows.

[0038] The identification method and system of the present invention can use neural networks to self-learn on-site, adapting to the process of traveling from a remote location to a collection point to quickly identify the location of the injured, the location of the collection point, and whether there are injured people. This can significantly reduce the time the injured spend at the collection point, buying valuable time for their rapid transfer to the appropriate treatment center. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a schematic diagram of the hardware system for autonomous injury triage of injured persons in an injury cluster scenario.

[0040] Figure 2 FIG. 4 is a basic flow chart of a method for identifying a wounded person according to an embodiment of the present invention.

[0041] Figure 3 4 is a flow chart of adding a small-sized picture of a wounded person by means of picture augmentation according to an embodiment of the method of the present invention.

[0042] Figure 4 FIG. 4 is a schematic diagram of a basic network of a wounded person identification network in an embodiment of the method according to the present invention.

[0043] Figure 5 2 is a schematic diagram of the auxiliary convolutional layer of the casualty identification network in an embodiment of the method according to the present invention.

[0044] Figure 6 2 is a schematic diagram of a convolutional layer of a casualty identification network prediction method according to an embodiment of the present invention.

[0045] Figure 7 2 is a schematic diagram of a convolutional layer of a casualty recognition network prediction after feature fusion in an embodiment of the method according to the present invention.

[0046] Figure 8 3 is a schematic diagram of feature map fusion details in an embodiment of the method according to the present invention.

[0047] Figure 9 FIG. 4 is a flow chart of training a casualty identification network in an embodiment of the method according to the present invention.

[0048] Figure 10 Schematic diagram of pictures of injured persons in a training data set in an embodiment of the method according to the present invention.

[0049] Figure 11 This is a rendering of a casualty identification experiment in a casualty collection point scenario according to the technical solution of the present invention, in which a simulated casualty is located on a paved road.

[0050] Figure 12 This is a rendering of a wounded person identification experiment in a wounded point scenario according to the technical solution of the present invention, wherein the simulated wounded person is located on a flat grassland.

[0051] It should be understood that the textual content presented in the above drawings is already part of the textual content of this specification, and can be combined with any embodiment in the specific implementation part of this specification as long as it does not violate the technical solution principle of the present invention. DETAILED DESCRIPTION

[0052] The following will provide a clear and complete description of the concept, specific structure and technical effects of the present invention in conjunction with the embodiments and drawings to fully understand the purpose, scheme and effects of the present invention.

[0053] It should be noted that, unless otherwise specified, when a feature is referred to as being "fixed" or "connected" to another feature, it may be directly fixed or connected to the other feature, or it may be indirectly fixed or connected to the other feature. The singular forms "a", "said" and "the" used herein are also intended to include the plural forms, unless the context clearly indicates otherwise. In addition, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art. The terms used in this specification are only for describing specific embodiments and are not intended to limit the invention. The term "and / or" used herein includes any combination of one or more related listed items.

[0054] Should be understood that, although the present disclosure may adopt the term first, second, third etc. to describe various elements, these elements should not be limited to these terms.These terms are only used to distinguish the elements of the same type from each other.For example, without departing from the scope of the present disclosure, the first element may also be referred to as the second element, and similarly, the second element may also be referred to as the first element.The use of any and all examples or exemplary language ("for example", "such as" etc.) provided herein is only intended to better illustrate embodiments of the present invention, and unless otherwise required, will not impose limitations on the scope of the present invention.

[0055] Reference Figure 1 In some embodiments, the wounded identification system according to the present invention is generally used to identify the location of the wounded collection point in the wounded collection point scenario, identify whether there are wounded or the number of wounded at the wounded collection point, etc. The wounded identification system may include at least one depth camera carried by a mobile device and a computer device connected to the depth camera. The mobile device includes a four-legged robot dog, a mobile robot, a mobile smart car, or a flying drone. Figure 1 Taking a quadruped robot dog as an example, the robot dog 100 carries a depth camera 110 (e.g., an Intel Realsense D455 depth camera) to collect on-site image data at the accident scene and then generate images for computer equipment to perform calculations and analysis. Figure 1 As shown, in the working task of the robot dog 100, it is necessary to start from a relatively far distance to search and advance to each injury collection point (one of the injury collection points is as follows Figure 1 As shown in the dotted area in the figure, there may also be multiple injury collection points) to detect the wounded, and gradually approach each wounded from far to near according to the detection results. In this process, the application running in the computer device implements the wounded identification method according to the present invention, by real-time analysis of the image (or video) transmitted by the depth camera 110 to intercept pictures containing the wounded, analyze the number of wounded classifications, and then use the shooting angle of the wounded picture to reversely calculate the direction of the wounded relative to the robot dog 100. Next, combined with Figures 2 to 12 , the wounded person identification method according to the present invention implemented in the computer device is described in detail with more embodiments.

[0056] Reference Figure 2 In some embodiments, the method for identifying a wounded person according to the present invention comprises at least the following steps:

[0057] S10. Collecting at least one image of a wounded person in the wound collection point environment using a depth camera, and compiling the images into a field dataset of the wounded person images;

[0058] S20, generating additional injured person images of smaller size for the original injured person images taken at close range in the on-site dataset in a data augmentation manner, and associating the additional injured person images with the original injured person images and storing them in the on-site dataset;

[0059] S30: Input the scene image captured by the depth camera into a deep learning-based neural network to calculate and output the number of injured persons in the scene image. If the injured person image used for detection is a resized, smaller-sized additional injured person image, the identified number of injured persons can be associated with the original injured person image at its original size.

[0060] In addition, the neural network is trained by the pre-training data set and the field data set. Therefore, the method according to the present invention may further include the step of training the neural network:

[0061] S40. Calculate the loss value of each point of the feature map output by the prediction convolution layer through a loss function, and use it to update the model parameters of the neural network during the training process of the neural network until the sum of the loss values ​​of all points in the feature map after training is less than a preset threshold, then stop training.

[0062] Specific implementation of step S10

[0063] As the mobile device moves toward the injury collection point, it uses a depth camera mounted on the device (such as a quadruped robot dog) to capture images from a distance. During this process, an application running on a computer analyzes the images (or videos) transmitted by the depth camera in real time, extracting images containing injured individuals to analyze and categorize the number of injured individuals. The application then uses the angle at which the images of the injured individuals are taken to inversely calculate their orientation relative to the mobile device.

[0064] In addition, the images of injured persons that have been identified and marked (for example, confirmed by system review, confirmation by the upper system, or confirmation by other auxiliary sensors) collected by the onboard depth camera can also be used to create a dataset of injured person images for continuous updating and training of neural networks to learn and adapt to the on-site injured person scenes.

[0065] Specific implementation of step S20

[0066] S21, determining the image of the injured person taken at close range as an image with a low proportion of small targets, wherein if the area of ​​the image occupied by the human body outline in the captured image is less than a preset pixel product threshold (e.g., 32 pixels x 32 pixels), then the image is determined to contain a small target;

[0067] S22. Reduce the image size of the picture with a low proportion of small objects to one quarter of the original size, and splice the four reduced images into an image with the same size as the original image.

[0068] Specifically, the depth camera carried by a mobile device (such as a four-legged robot dog) starts detecting the injured person from a relatively far distance and gradually approaches the injured person from far to near based on the detection results. At the same time, because the field of view of the depth camera is very wide, the injured person detection during the process of the four-legged robot moving to the side of the injured person is mostly small target detection, that is, the injured person is a target smaller than 32 pixels x 32 pixels within the camera's field of view.

[0069] However, the size of the wounded in the corresponding images in the initial version of the dataset does not conform to this feature. In order to increase the number of small target wounded samples in the wounded dataset, this section proposes a data enhancement method, which aims to train the target detector in a data-balanced manner, construct and expand small target samples using an oversampling method, and reduce the size of images with a low proportion of small targets to 1 / 4 of the original size. The four reduced images are then spliced ​​into an image of the same size as the original image for feeding into the network. The spliced ​​image inevitably contains smaller target objects, thereby increasing the weight of the small target data, such as Figure 3 shown.

[0070] Continue to refer to Figure 3 Preferably, if the size of the injured in the injured picture is smaller than a fixed value, it is considered to be a picture containing small-sized injured persons and can be directly input into the injured person detection network. Otherwise, it needs to go through a pre-processing module to reduce it to 1 / 4, and the four pictures are merged into a picture of the original size, so that the picture can contain small-sized target injured persons and can be input into the injured person detection network. Of course, when synthesizing four pictures, these four pictures can be different, which can greatly increase the injured person data set and increase the number of small-sized injured targets. It should be noted here that because the picture obtained by integrating multiple pictures through the pre-processing module is equal to the number of injured persons appearing multiple times (for example, 4 times), the detection structure of the number of injured persons obtained by the detection network to identify this integrated picture needs to be divided by the original integration multiple (for example, 4) to obtain the actual number of injured persons.

[0071] Specific implementation of step S30

[0072] The neural network structure for casualty identification includes a basic network layer, an auxiliary convolution layer, and a prediction convolution layer. The basic network layer can be based on the VGG16 network architecture, while the auxiliary convolution layer is used to extract feature maps of different scales. The use of feature maps of multiple different depth layers to detect and locate casualties is to achieve detection of casualties of different sizes. In this casualty identification network, the auxiliary convolution layer outputs 4 feature maps of different scales. The prediction convolution layer is divided into position prediction and classification prediction convolution layers, where position prediction has four position information and classification prediction has two information. The basic network, auxiliary convolution layer, and prediction convolution layer are respectively as follows: Figure 4 、 Figure 5 、 Figure 6 shown.

[0073] In the case of a casualty cluster, a mobile device equipped with a depth camera starts identifying casualties from a distance and then gradually approaches them based on their location. Because the camera has a wide field of view, the mobile device spends most of its time identifying small casualties during this process. In some preferred embodiments, considering small casualty targets, the casualty identification network can be optimized in the following ways.

[0074] Preferably, refer to Figure 7 According to the basic network and auxiliary convolutional layer of the casualty recognition network, the network used in the recognition method according to the present invention is to obtain the position information and classification information of the casualty in the picture by outputting six feature maps of the conv4_3 layer, conv7 layer, conv8_2 layer, conv9_2 layer, conv10_2 layer and conv11_2 layer to the prediction convolutional layer. In order to better obtain contextual information as additional information to assist in the detection of small-sized casualties, the three feature maps output by the conv4_3 layer, conv7 layer and conv8_2 layer are fused into feature maps of the same size as the feature maps output by the conv4_3 layer, and are input to the prediction convolutional layer instead of the conv4_3 layer feature maps. The reason why the conv4_3 layer, conv7 layer and conv8_2 layer are selected for feature fusion instead of the last three layers is because the pixels of small-sized casualties are limited and the resolution is low, so feature maps of larger sizes are required for fusion. The schematic diagram of the prediction convolutional layer of the casualty recognition network after feature fusion is shown as follows. Figure 7 shown.

[0075] Reference Figure 8 , the specific feature fusion details are as follows:

[0076] (1) The conv4_3 layer feature map H1xW1xC1 passes through a convolution layer (where the convolution size is 3x3, padding is 1, and stride is 1), connected to the L2norm layer, and then connected to the ReLu layer, thereby outputting a feature map of the same size;

[0077] (2) The Conv7 layer feature map H2xW2xC2 passes through the deconvolution layer, connects to the L2norm layer, and then connects to the ReLu layer, thereby outputting a feature map of size H1xW1xC1 / 2;

[0078] (3) The feature map H3xW3xC3 of the Conv8_2 layer passes through the deconvolution layer, connects to the L2norm layer, and then connects to the ReLu layer, thereby outputting a feature map of size H1xW1xC1 / 2;

[0079] (4) The three feature maps outputted at the end are concatenated to generate a feature map of size H1xW1xC1*2.

[0080] Among them, H1xW1xC1 is 38x38x512, H2xW2xC2 is 19x19x1024, and H3xW3xC3 is 10x10x512.

[0081] Specific implementation of step S40

[0082] The overall flow chart of network training is as follows Figure 9 As shown, it is necessary to input batches of image data (such as pre-training data sets, field data sets, and initial data sets) into the neural network model for training, and then adjust the model parameters through the calculation of the loss function. The key to training the network is to determine the calculation of the loss function. The prediction convolution layer predicts the position information and classification information of each point in the feature map, and the loss value of the point is equal to the sum of the loss of the position and the loss of the classification. Here, the IoU (overlap) of the priori box and the true box of each point in the feature map is calculated. If the IoU is greater than the set fixed threshold, the class marked by the priori box and the true box is the same, which is called the positive class; if it is less than the set threshold, the class marked by the priori box is considered to be the background, which is called the negative class. The loss function of the predicted box and the true box is equal to the sum of the loss of the predicted box position and the loss of the classification, which can be expressed as

[0083] Loss = L class +L loc (1.1)

[0084] Where Loss is the overall loss function, L class is the classification loss function, L loc Position loss function. L class and L loc The calculation method is shown in Equations 1.2 and 1.3 below:

[0085]

[0086]

[0087] where N pis the number of positive prediction boxes, N n is the number of negative class prediction boxes, Box i_pred Refers to the coordinate information of the prediction box, Box i_real Refers to the coordinate information of the corresponding true value frame, Distance() means finding the Euclidean distance between coordinates, and CE_loss means finding the cross entropy loss function.

[0088] In addition, for the initial training data set of the network model, the current public data sets are mostly target detection data sets and pedestrian detection data sets, and there is no specific wounded person detection data set. Due to the specific lying posture of the wounded, if there is no specific wounded person data set, the accuracy of the wounded person detection results will not be satisfactory. By using the depth camera carried by the quadruped robot dog to simulate the collection of wounded person pictures in the injury point scene, and searching for wounded person pictures on the Internet, thousands of wounded person pictures can be sorted out to form the initial pre-training data set. Then, the data augmentation method can be used to reduce the image size of the original wounded person pictures taken at close range in the pre-data set to one-fourth of the original size, and splice the four reduced pictures into a picture with the same size as the original image, and associate the spliced ​​wounded person pictures with the original wounded person pictures and store them in the pre-data set, so that it contains thousands of wounded targets. These pictures containing wounded people constitute the initial version of the wounded person image data set (such as Figure 10 As shown) for network model training.

[0089] Experimental verification of the wounded identification method and system according to the present invention

[0090] In order to better simulate the scene of the injured point, the experimental scene was first determined to be a relatively open cement or grassy area. At the same time, four injured models were arranged to lie neatly on the ground. Other pedestrians imitated rescue workers and shuttled between the injured models. The four-legged robot dog equipped with a depth camera identified the injured from far to near. Figure 10 As shown in FIG, the LYING PERSON identifier represents the detected injured person.

[0091] It should be appreciated that the method steps in the embodiments of the present invention can be implemented or executed by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method can use standard programming techniques. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if desired, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. In addition, for this purpose, the program can be run on a programmed application-specific integrated circuit.

[0092] Furthermore, the operations of the processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by the context. The processes described herein (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or a combination thereof. The computer program includes a plurality of instructions that can be executed by one or more processors.

[0093] Further, the methods can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Various aspects of the present invention can be implemented as machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, RAM, ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the processes described herein. In addition, the machine-readable code, or portions thereof, can be transmitted over a wired or wireless network. When such media includes instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor, the invention described herein includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention can also include the computer itself.

[0094] The computer program can be applied to input data to perform the functions described herein, thereby converting the input data to generate output data that is stored in a non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents a physical and tangible object, including a specific visual depiction of the physical and tangible object produced on the display.

[0095] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the aforementioned embodiments. As long as the technical effects of the present invention are achieved by the same means, any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention. Within the scope of protection of the present invention, various modifications and variations of the technical solutions and / or implementation methods are possible.

Claims

1. A method for identifying casualties based on deep learning neural networks, characterized in that: The method comprises the following steps: S10. Capturing at least one image of a casualty in an injury collection point environment using a depth camera, and compiling the images into an on-site dataset of casualty images; wherein, while the mobile device is moving toward the injury collection point environment, the depth camera onboard the mobile device is used to capture images from a relatively distant position, wherein the mobile device includes a quadruped robot dog, a mobile robot, a mobile smart car, or a flying drone; S20, generating additional injured person images of smaller size for the original injured person images taken at close range in the on-site dataset in a data augmentation manner, and associating the additional injured person images with the original injured person images and storing them in the on-site dataset; S30, inputting the scene picture taken by the depth camera into a deep learning-based neural network to calculate and output the number of wounded in the scene picture, wherein the neural network is trained by a pre-training dataset and the scene dataset; Wherein, the neural network includes: The base network layer of the VGG16 network architecture; Auxiliary convolutional layers for extracting feature maps of different scales; A prediction convolution layer, comprising a position prediction convolution layer and a category prediction convolution layer; the neural network is configured to: send feature maps output by the conv4_3 layer, the conv7 layer, the conv8_2 layer, the conv9_2 layer, the conv10_2 layer, and the conv11_2 layer to the prediction convolution layer to obtain position information and classification information of the injured person in the image; perform feature fusion on the feature maps output by the conv4_3 layer, the conv7 layer, and the conv8_2 layer to form a feature map of the same size as the feature map output by the conv4_3 layer, and replace the feature map of the conv4_3 layer before sending it to the prediction convolution layer; Among them, the H1xW1xC1 size feature map of the conv4_3 layer is passed through a 3x3 size convolution layer, then connected to the L2norm layer, and then connected to the ReLu layer, thereby outputting a first feature map of the same size; the H2xW2xC2 size feature map of the Conv7 layer is passed through a deconvolution layer, connected to the L2norm layer, and then connected to the ReLu layer, thereby outputting a second feature map of size H1xW1xC1 / 2; the H3xW3xC3 size feature map of the Conv8_2 layer is passed through a deconvolution layer, connected to the L2norm layer, and then connected to the ReLu layer, thereby outputting a third feature map of size H1xW1xC1 / 2; the output first feature map, second feature map and third feature map are spliced ​​to generate a feature map of size H1xW1xC1*2, where H, W and C represent the size dimensions of the feature map respectively.

2. The method according to claim 1, characterized in that The step S20 includes: S21. Images of injured persons taken at close range are identified as images with a low proportion of small objects; S22. Reduce the image size of the picture with a low proportion of small objects to one quarter of the original size, and splice the four reduced images into an image with the same size as the original image.

3. The method according to claim 2, characterized in that The step S21 includes: If the area of ​​the human body outline in the captured image is smaller than a preset pixel product threshold, the image is determined to be an image containing a small object.

4. The method according to claim 3, wherein: The training of the neural network comprises the following steps: S40, calculating the loss value of each point of the feature map output by the prediction convolution layer by a loss function, and updating the model parameters of the neural network during the training process of the neural network until the sum of the loss values ​​of all points of the trained feature map is less than a preset threshold, then stopping the training, wherein the calculation of the loss function includes: Calculate the overlap between the predicted box and the true box for each point in the feature map; if the overlap is greater than the set threshold, the predicted box is the same class marked by the true box and is set to the positive class; if the overlap is less than the set threshold, the class marked by the predicted box is considered to be the background and is set to the negative class; The loss function between the predicted box and the true box is equal to the sum of the position loss and the classification loss of the predicted box, where The position loss of the prediction box is calculated as The classification loss of the prediction box is calculated as Among them, L loc Position loss function, L class is the classification loss function, N p is the number of positive prediction boxes, N n is the number of negative prediction boxes, Box i_pred It is the coordinate information of the prediction box, Box i_real It is the coordinate information of the corresponding real frame, Distance() represents the Euclidean distance calculation function between coordinates, and CE_loss represents the cross entropy loss function.

5. The method according to any one of claims 1 to 4, wherein: The training of the neural network comprises the following steps: S51. Capture multiple images of injured people from the Internet and add them to the pre-training dataset; S52. Using data augmentation, the image size of the original close-up images of the injured person in the pre-data set is reduced to one-fourth of the original size, and the four reduced images are spliced ​​into an image with the same size as the original image. The spliced ​​injured person image is associated with the original injured person image and then stored in the pre-data set.

6. A wounded person identification system, characterized in that: include: at least one depth camera carried by a mobile device; A computer device connected to the depth camera, the computer device comprising a computer-readable storage medium on which program instructions are stored, wherein the program instructions implement the method according to any one of claims 1 to 5 when executed by a processor.

Images