Infrared human body detection method, device and robot

Abstract

The application provides an infrared human body detection method and device and a robot, and the method comprises the following steps: S1, acquiring an infrared image collected by the robot; S2, converting the infrared image into an RGB image by using an improved DCGAN, wherein a generator convolutional neural network of the DCGAN is replaced by YOLOP, and a discriminator convolutional neural network of the DCGAN is replaced by EfficientNET to obtain the improved DCGAN; and S3, inputting the RGB image into YOLOV5 for human body detection to obtain a human body detection result. The infrared image is converted into an RGB color image, so that the accuracy and robustness of human body detection are greatly improved and progressed. The original 3-path anchors of YOLOV5 are improved into 4-path anchors, and the feature map decrement multiple is modified, so that the small target detection precision is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to an infrared human body detection method and device and a robot. BACKGROUND

[0002] Infrared human body detection is needed in scenarios such as night factories and night parks. Many factories, parks and other places have security guards on patrol after work at night, with the purpose of discovering suspicious personnel who should not be present at this time point.

[0003] The existing infrared human body detection technical solutions mainly include the following:

[0004] (1) Using a traditional non-deep learning algorithm to detect infrared human bodies: This solution first obtains an infrared image from an infrared camera, then uses a block algorithm such as a multi-scale sliding window algorithm to divide the infrared image into blocks, and sends each small block to a traditional non-deep learning algorithm to calculate features such as HOG (Histogram of Oriented Gradient) features, and finally uses an SVM (Support Vector Machine) to classify the calculated features to determine whether there is a human body in the image block.

[0005] (2) Using a deep learning target detection network to detect infrared human bodies: This solution first obtains an infrared image from an infrared camera, then sends the infrared image to a deep learning target detection network such as YOLOV5 for inference, and decodes the position of the human body in the infrared image from the inference result, finally outputting N (N is an integer greater than or equal to 0) rectangular frames (represented by the upper left corner and lower right corner coordinates) and the probability value of the target being a human body in the frame.

[0006] (3) Using a deep learning human key point detection network from bottom to top to detect infrared human bodies: This solution first obtains an infrared image from an infrared camera, then sends the infrared image to a deep learning human key point detection network from bottom to top such as HigherHRNet for inference, and decodes N (N is an integer greater than or equal to 0) sets of human key points from the inference result, each set representing a human body.

[0007] In scenarios such as night factories and night parks, since the infrared image captured by the infrared camera loses a large amount of texture features compared to the RGB color image captured by the ordinary camera, it is difficult to perform high-precision infrared human body detection using the above three technical solutions.

[0008] The background description provided herein is for the purpose of generally presenting the context of the disclosure. The subject matter of the section is not prior art to the present application and is not admitted to be prior art by inclusion in this section. SUMMARY

[0009] To solve the above technical problems in the related art, the present application provides an infrared human body detection method, comprising the following steps:

[0010] S1, acquiring an infrared image collected by a robot;

[0011] S2, converting the infrared image into an RGB image using an improved DCGAN, wherein a generator convolutional neural network of the DCGAN is replaced with YOLOP, and a discriminator convolutional neural network of the DCGAN is replaced with EfficientNET to obtain the improved DCGAN;

[0012] S3, inputting the RGB image into YOLOV5 for human body detection to obtain a human body detection result.

[0013] Specifically, the step S3 further comprises: inputting the RGB image into an improved YOLOV5 for human body detection to obtain a human body detection result, wherein the improved YOLOV5 is to replace original 3-path Anchors of YOLOV5 with 4-path Anchors.

[0014] Specifically, the step S3 further comprises: inputting the RGB image into an improved YOLOV5 for human body detection to obtain a human body detection result, wherein the improved YOLOV5 is to change a feature map decrement multiple of YOLOV5 from [8, 16, 32] to [4, 8, 16, 32].

[0015] Specifically, the method further comprises a step S5 of alarming when a human body is detected.

[0016] In a second aspect, another embodiment of the present application discloses an infrared human body detection device, comprising the following units:

[0017] An infrared image acquisition unit is configured to acquire an infrared image collected by a robot;

[0018] An RGB image conversion unit is configured to convert the infrared image into an RGB image using an improved DCGAN, wherein a generator convolutional neural network of the DCGAN is replaced with YOLOP, and a discriminator convolutional neural network of the DCGAN is replaced with EfficientNET to obtain the improved DCGAN;

[0019] A human body detection unit is configured to input the RGB image into YOLOV5 to perform human body detection to obtain a human body detection result.

[0020] Specifically, the human body detection unit further includes: inputting the RGB image into improved YOLOV5 to perform human body detection to obtain a human body detection result, wherein the improved YOLOV5 is to replace the original 3-path Anchors of YOLOV5 with 4-path Anchors.

[0021] Specifically, the human body detection unit further includes: inputting the RGB image into improved YOLOV5 to perform human body detection to obtain a human body detection result, wherein the improved YOLOV5 is to replace the original 3-path Anchors of YOLOV5 with 4-path Anchors.

[0022] Specifically, the alarm unit is configured to alarm when a human body is detected.

[0023] In a third aspect, another embodiment of the present application discloses a robot, which includes: a processing module, a chassis, a storage module, and a laser radar and an infrared camera, the storage module stores instructions, and the instructions are executed to implement the above-mentioned infrared human body detection method.

[0024] In a fourth aspect, another embodiment of the present application discloses a non-volatile memory, which stores instructions, and the instructions are executed by a processor to implement the above-mentioned infrared human body detection method.

[0025] The infrared human body detection method of the present application improves the DCGAN to convert the infrared image into an RGB color image with more abundant texture features, so that the accuracy and robustness of human body detection are greatly improved compared with direct detection from the infrared image. Further, the present application improves the original 3-path Anchors of YOLOV5 to 4-path Anchors, and changes the feature map decrement multiple from [8, 16, 32] to [4, 8, 16, 32], thereby improving the detection accuracy of small targets. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0027] Figure 1is a flow chart of an infrared human body detection method provided by an embodiment of the present application;

[0028] Figure 2 is an infrared image before conversion provided by an embodiment of the present application;

[0029] Figure 3 is an RGB image after conversion provided by an embodiment of the present application;

[0030] Figure 4 is a human body detection schematic diagram before conversion provided by an embodiment of the present application;

[0031] Figure 5 is a human body detection schematic diagram after conversion provided by an embodiment of the present application;

[0032] Figure 6 is a schematic diagram of an infrared detection device provided by an embodiment of the present application;

[0033] Figure 7 is a schematic diagram of an infrared detection device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art belong to the scope of protection of the present application. Embodiment one

[0035] Reference Figure 1 The present embodiment discloses an infrared human body detection method, which comprises the following steps:

[0036] S1, acquiring an infrared image collected by a robot;

[0037] The robot of the present embodiment is provided with an infrared camera at its head, and the infrared camera is used to shoot infrared pictures. The infrared camera of the present embodiment can rotate 360 degrees. By constantly patrolling fixed routes in a factory, a park and the like, infrared images of the factory, the park and the like are collected in real time.

[0038] The robot of the present embodiment comprises a laser radar, which is used to acquire laser point cloud data for navigation and positioning. The laser radars are all arranged at the head of the robot.

[0039] The robot generally needs to establish a map of the target area in advance when navigating using a laser radar. In this embodiment, the detection of an infrared human body is taken as an example for a park, and thus the target area of this embodiment is a park. Therefore, it is necessary to establish a map, such as a 3D map, for the park. Specifically, the mapping method can use a SLAM or the like. The SLAM mapping method used in this embodiment is as follows: the robot is manually controlled to travel along the road of the target scene, and the point cloud data collected in each frame is uploaded to the background and aligned through a certain matching algorithm to establish a 3D point cloud map for the target scene.

[0040] The robot of this embodiment can transmit the captured scene image of the park or the like to the background server through a wireless communication method such as 5G communication for detection of an infrared human body.

[0041] In another embodiment, the robot of this embodiment can perform detection of an infrared human body locally. Specifically, the robot can be configured with a processing unit, such as a central processing unit, a graphics processing unit or the like, to perform detection of an infrared human body.

[0042] S2, converting the infrared image into an RGB image using an improved DCGAN, wherein a generator convolutional neural network of the DCGAN is replaced with YOLOP, and a discriminator convolutional neural network of the DCGAN is replaced with EfficientNET to obtain the improved DCGAN;

[0043] This embodiment uses DCGAN in a deep learning generative adversarial network to convert an infrared image captured by an infrared camera of an inspection robot into an RGB color image. In order to improve the ability and speed of the generator to generate an RGB color image, the generator convolutional neural network of the DCGAN is replaced with YOLOP (only one image segmentation branch is retained for generating an image) in this embodiment. At the same time, in order to improve the discrimination ability of the discriminator, i.e., the ability to classify true and false pictures, the discriminator convolutional neural network of the DCGAN is replaced with the classification network EfficientNet in this embodiment. The training method still uses the DCGAN training method.

[0044] Reference Figures 2-3 , Figure 2 is an infrared image obtained, Figure 3 is converted into an RGB image using an improved DCGAN, Figure 3 wherein the infrared image is converted into an RGB image using an improved DCGAN, Figure 3 is a color image.

[0045] S3, sending the RGB image into YOLOV5 for human body detection to obtain a human body detection result;

[0046] The embodiment uses YOLOV5 in deep learning to detect the human body in the image converted by DCGAN, and outputs the left upper corner coordinates (x1, y1) and the right lower corner coordinates (x2, y2) of the human body frame.

[0047] In actual application scenarios, the detected target is only one category, that is, the human body, and the human body is usually far away from the camera, that is, there may be many small target human bodies. In order to increase the detection accuracy, the original 3-path Anchors of YOLOV5 is replaced by 4-path Anchors, and the feature map decrement multiple is changed from [8, 16, 32] to [4, 8, 16, 32].

[0048] The backbone network of YOLOV5 is kept unchanged for training. The data set used for training the model is all network open source RGB color picture data set, including COCO data set, ExDark data set, Crowd data set and Wider data set. Because the picture size taken by the camera carried by the inspection robot is 1920*1080, in order to improve the model inference speed and keep the detection accuracy, the size of the training and inference picture is uniformly adjusted to 832*480. Figures 4-5 The effect diagram obtained by using the YOLOV5 modified by the embodiment to detect before and after the conversion of DCGAN can be seen. It can be seen that two human bodies are detected after the conversion, and the confidence of the same human body detected after the conversion is also improved.

[0049] Further, the embodiment further includes a step S5 of alarming when the human body is detected.

[0050] Further, the alarm of the embodiment is to push the alarm information to the management terminal, and at the same time, the image of the detected human body is sent to the management terminal, so that the management personnel can take corresponding action according to the alarm information. In another embodiment, a voice such as "you have entered the control area, please leave quickly" can also be played.

[0051] The infrared human body detection method of the embodiment improves the DCGAN to convert the infrared image into an RGB color image with more rich texture features, so that the accuracy and robustness of detecting the human body are greatly improved and progressed compared with directly detecting from the infrared image. Further, the original 3-way Anchors of YOLOV5 is improved to 4-way Anchors in the embodiment, and the feature map decrement multiple is changed from [8, 16, 32] to [4, 8, 16, 32], which improves the detection accuracy of small targets. The human body detection method of the embodiment can be used for robots, and the robots carry a single ordinary and low-cost infrared camera that can rotate 360 degrees on the top of the head to patrol the night factory, park and other scenes according to a fixed route. Embodiment two

[0052] Reference Figure 6 The embodiment discloses an infrared human body detection device, which comprises the following units:

[0053] An infrared image acquisition unit is configured to acquire an infrared image collected by a robot.

[0054] The robot of the embodiment is provided with an infrared camera on the head, and the infrared camera is used to shoot infrared pictures. The infrared camera of the embodiment can rotate 360 degrees. By constantly patrolling the fixed route in the factory and park, the infrared images of the factory, park and other scenes are collected in real time.

[0055] The robot of the embodiment comprises a laser radar, which is used to acquire laser point cloud data for navigation and positioning. The laser radar is arranged on the head of the robot.

[0056] When the robot uses the laser radar for navigation, it is usually necessary to establish a map of the target area in advance. The embodiment takes a park as an example to detect the infrared human body, so the target area of the embodiment is the park. Therefore, it is necessary to establish a map, such as a 3D map, of the park. Specifically, the mapping method can use SLAM and the like. The SLAM mapping method used in the embodiment is as follows: the robot is manually controlled to drive along the road of the target scene, and the point cloud data collected in each frame is uploaded to the background and aligned through a certain matching algorithm to establish a 3D point cloud map of the target scene.

[0057] The robot of the embodiment can transmit the photographed images of the park and other scenes to the background server through wireless communication such as 5G communication for detection of the infrared human body.

[0058] In another embodiment, the robot of the embodiment can detect the infrared human body locally. Specifically, the robot can be configured with a processing unit, such as a central processing unit, a graphics processing unit and the like, to detect the infrared human body.

[0059] an RGB image conversion unit configured to convert the infrared image into an RGB image using an improved DCGAN, wherein a generator convolutional neural network of the DCGAN is replaced with YOLOP, and a discriminator convolutional neural network of the DCGAN is replaced with EfficientNET to obtain the improved DCGAN;

[0060] The embodiment uses DCGAN in a deep learning generative adversarial network to convert an infrared image taken by an infrared camera of an inspection robot into an RGB color image. In order to improve the ability and speed of the generator to generate an RGB color image, the embodiment replaces the generator convolutional neural network of the DCGAN with YOLOP (only one image segmentation branch is reserved for generating an image). Meanwhile, in order to improve the discrimination ability of the discriminator, i.e., the ability to classify true and false images, the embodiment replaces the discriminator convolutional neural network of the DCGAN with the classification network EfficientNet. The training method still uses the DCGAN training method.

[0061] Reference Figures 2-3 , Figure 2 is an infrared image obtained, Figure 3 is converted into an RGB image using an improved DCGAN, wherein Figure 3 is an infrared image obtained, Figure 3 is an RGB color image.

[0062] a human body detection unit configured to input the RGB image into YOLOV5 for human body detection to obtain a human body detection result;

[0063] The embodiment uses YOLOV5 in deep learning to perform human body detection on the image converted by the DCGAN and outputs the left upper corner coordinates (x1, y1) and the right lower corner coordinates (x2, y2) of the human body frame.

[0064] Since the target to be detected in the actual application scenario is only one category, i.e., human body, and the human body is usually far away from the camera, i.e., there can be many small target human bodies, in order to increase the detection accuracy, the embodiment replaces the original 3-path Anchors of YOLOV5, i.e., [[10, 13, 16, 30, 33, 23], [30, 61, 62, 45, 59, 119], [116, 90, 156, 198, 373, 326]], with 4-path Anchors, i.e., [[3, 9, 5, 11, 20, 4], [10, 13, 16, 30, 33, 23], [30, 61, 62, 45, 59, 119], [116, 90, 156, 198, 373, 326]], and the feature map decrement multiple is changed from [8, 16, 32] to [4, 8, 16, 32].

[0065] The embodiment keeps the backbone network of YOLOV5 unchanged for training. The data set for training the model is all network open source RGB color picture data set, including COCO data set, ExDark data set, Crowd data set and Wider data set. Because the picture size taken by the camera carried by the inspection robot is 1920*1080, in order to improve the model inference speed and keep the detection accuracy, the size of the training and inference picture is uniformly adjusted to 832*480. Figures 4-5 For DCGAN before and after conversion, the effect diagram obtained by using the YOLOV5 detection modified by the embodiment can be seen that two human bodies are detected after conversion, and the confidence of the same human body detected after conversion is also improved.

[0066] Further, the embodiment also includes an alarm unit which alarms when a human body is detected.

[0067] Further, the alarm unit of the embodiment can push alarm information to the management terminal, and at the same time send the image of the detected human body to the management terminal, so that the management personnel can take corresponding action according to the alarm information. In another embodiment, a voice such as "you have entered the control area, please leave quickly" can also be played.

[0068] The infrared human body detection method of the embodiment improves the DCGAN to convert the infrared image into an RGB color image with more rich texture features, so that the accuracy and robustness of detecting human bodies are greatly improved and progressed compared with directly detecting from the infrared image. Further, the embodiment improves the original 3-way Anchors of YOLOV5 to 4-way Anchors, and changes the feature map decrement multiple from [8, 16, 32] to [4, 8, 16, 32], which improves the detection accuracy of small targets. The human body detection method of the embodiment can be used for a robot, which carries a single ordinary and low-cost infrared camera that can rotate 360 degrees on the top of the robot to patrol the night factory, park and other scenes according to a fixed route. Embodiment three

[0069] The embodiment discloses a robot, which comprises a processing module, a chassis, a storage module, a laser radar and an infrared camera. The storage module stores instructions which, when executed, implement the infrared human body detection method of the embodiment one.

[0070] In another embodiment, a robot comprises a processing module, a chassis, a storage module, a laser radar and an infrared camera, and further comprises the infrared human body detection device of the embodiment two. Embodiment four

[0071] Reference Figure 7 , Figure 7 is a structural schematic diagram of an infrared human body detection device according to an embodiment. The infrared human body detection device 20 according to the embodiment includes a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. The processor 21 implements the steps in the above method embodiments when executing the computer program. Alternatively, the processor 21 implements the functions of each module / unit in the above device embodiments when executing the computer program.

[0072] For example, the computer program can be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the infrared human body detection device 20. For example, the computer program can be divided into the modules in Embodiment Two, and the specific functions of each module are described in the working process of the device in the above embodiments, which will not be repeated here.

[0073] The infrared human body detection device 20 can include, but is not limited to, the processor 21 and the memory 22. Those skilled in the art can understand that the schematic diagram is only an example of the infrared human body detection device 20 and does not limit the infrared human body detection device 20, which can include more or fewer components than the diagram, or combine certain components, or different components, for example, the infrared human body detection device 20 can also include an input / output device, a network access device, a bus, etc.

[0074] The processor 21 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor and the like. The processor 21 is the control center of the infrared human body detection device 20, which connects all parts of the infrared human body detection device 20 through various interfaces and lines.

[0075] The memory 22 can be used to store the computer programs and / or modules, and the processor 21 realizes various functions of the infrared human body detection device 20 by running or executing the computer programs and / or modules stored in the memory 22, and calling the data stored in the memory 22. The memory 22 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required for a function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), and the like. In addition, the memory 22 can include a high-speed random access memory, and can also include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0076] The modules / units of the infrared human body detection device 20 integrated therein, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor 21 executes the computer program, the steps of the above-mentioned various method embodiments can be realized. The computer program includes computer program code, which can be in the form of source code, object code, an executable file, or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0077] It should be noted that the apparatus embodiments described above are merely illustrative, and the units described as separate units can or can not be physically separate, and the units shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. In addition, the connection relationship between the modules in the apparatus embodiment provided by the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement it without creative labor.

[0078] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An infrared human body detection method, comprising the following steps; S1, acquire the infrared image collected by the robot; S2, the infrared image is converted into an RGB image using an improved DCGAN, wherein the generator convolutional neural network of DCGAN is replaced with YOLOP and the discriminator convolutional neural network of DCGAN is replaced with EfficientNET to obtain the improved DCGAN. S3, the RGB image is sent to the improved YOLOV5 for human detection to obtain human detection results; the improved YOLOV5 is to replace the original 3-channel anchors of YOLOV5 with 4-channel anchors.

2. The method according to claim 1, wherein step S3 further comprises: The RGB image is fed into the improved YOLOV5 for human detection to obtain human detection results. The improved YOLOV5 is that the feature map reduction factor of YOLOV5 is changed from [8,16,32] to [4,8,16,32].

3. The method according to claim 2 further includes step S5, which involves triggering an alarm when a human body is detected.

4. An infrared human body detection device, comprising the following units; Infrared image acquisition unit, used to acquire infrared images collected by the robot; An RGB image conversion unit is used to convert the infrared image into an RGB image using an improved DCGAN, wherein the generator convolutional neural network of DCGAN is replaced with YOLOP and the discriminator convolutional neural network of DCGAN is replaced with EfficientNET to obtain the improved DCGAN. The human detection unit is used to send the RGB image into the improved YOLOv5 to perform human detection and obtain human detection results; the improved YOLOv5 is to replace the original 3-channel anchors of YOLOv5 with 4-channel anchors.

5. The apparatus according to claim 4, wherein the human body detection unit further comprises: The RGB image is fed into the improved YOLOV5 for human detection to obtain human detection results. The improved YOLOV5 is that the feature map reduction factor of YOLOV5 is changed from [8,16,32] to [4,8,16,32].

6. The apparatus according to claim 5 further includes an alarm unit for triggering an alarm when a human body is detected.

7. A robot, the robot comprising: The system includes a processing module, a chassis, a storage module, a lidar, and an infrared camera. The storage module stores instructions, which, when executed, are used to implement the infrared human body detection method as described in any one of claims 1-3.

8. A non-volatile memory storing instructions that, when executed by a processor, implement the infrared human body detection method as described in any one of claims 1-3.

Images