A robot personnel detection method based on real-time video stream

Abstract

This application discloses a robot personnel detection method based on real-time video stream. After extracting the predicted person area from the collected real-time shooting picture, on the one hand, face recognition is performed on the predicted person area to prevent unregistered construction workers from entering the construction site; on the other hand, the relationship between the target area and the face area is used to determine whether the person is wearing a safety helmet or safety clothing. This application has the advantages of high efficiency and real-time performance, which helps to improve the safety of the workplace, and take various alarm measures in time to protect the safety of personnel and improve the personnel's ability to perceive dangerous situations.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a robot personnel detection method based on real-time video stream. Background Art

[0002] In the industrial and security fields, the safety and accurate detection of personnel is of vital importance. Traditional personnel detection methods often rely on manual inspections, which have problems such as low efficiency, easy fatigue, and high subjectivity. In order to solve these problems, the use of computer vision and machine learning technology for safety detection has become more and more common.

[0003] However, the traditional solution is to identify and judge the safety of recorded on-site videos, which has some limitations. For example, it is impossible to control on-site staff in real time, it is impossible to determine whether a person is in danger in real time, and it is impossible to take timely alarm actions when it is determined that a person is in danger. Summary of the invention

[0004] The purpose of this application is to provide a robot personnel detection method based on real-time video stream, which can improve the above problems.

[0005] The embodiment of the present application is implemented as follows:

[0006] In a first aspect, the present application provides a robot personnel detection method based on real-time video stream, which comprises:

[0007] S1: Obtain a real-time shooting picture captured by a camera, and obtain a person prediction area in the real-time shooting picture through a YOLO model;

[0008] S2: extracting a face area within the person prediction area to perform face recognition;

[0009] S3: extracting a clustered area of ​​target color pixels in the person prediction area as a target area;

[0010] S4: judging whether the person in the person prediction area is in a dangerous state according to the relationship between the target area and the face area;

[0011] S5: Execute an alarm operation when face recognition fails or when it is determined that the person in the person prediction area is in a dangerous state.

[0012] Among them, S1, S2, etc. are only step identifiers, and the execution order of the method is not necessarily in the order of numbers from small to large. For example, step S2 may be executed first and then step S1. This application does not impose any limitation on this.

[0013] It can be understood that the present application discloses a robot personnel detection method based on real-time video stream. After extracting the predicted person area from the collected real-time shooting picture, on the one hand, face recognition is performed on the predicted person area to prevent unregistered construction personnel from entering the construction site; on the other hand, the relationship between the target area and the face area is used to determine whether the person is wearing a safety helmet or safety clothing. The present application has the advantages of high efficiency and real-time performance, which helps to improve the safety of the workplace, and take various alarm measures in time to protect the safety of personnel and improve the personnel's ability to perceive dangerous situations.

[0014] In an optional embodiment of the present application, the alarm operation includes at least one of the following: sending an alarm signal to a speaker to drive the speaker to emit an alarm audio; sending an alarm signal to a display to drive the display to display alarm information; sending an alarm signal to an indicator light to drive the indicator light to light up the alarm signal; sending alarm information to a monitoring terminal through a communication device.

[0015] It can be understood that the alarm operations of the present application are divided into two categories. One is to alarm through existing alarm equipment, such as speakers, displays, indicator lights, etc.; the other is to send alarm information to the monitoring terminal through communication equipment, that is, the monitoring personnel can view the real-time shooting screen and alarm information from the monitoring terminal, and the monitoring personnel can then control the robot through the communication equipment to take corresponding safety measures.

[0016] In an optional embodiment of the present application, step S3 includes:

[0017] S31: removing the known target area in the person prediction area to obtain an area to be processed;

[0018] S32: randomly selecting pixels of the target color in the area to be processed as target pixels, and assigning the target pixels to a newly created cluster area;

[0019] S33: Also classifying pixels of the target color among the pixels adjacent to the target pixel into the newly created cluster area;

[0020] S34: Continue to analyze the adjacent pixels adjacent to each pixel in the newly created cluster area, and continue to classify the pixels of the target color among the adjacent pixels into the newly created cluster area until the number of pixels in the newly created cluster area stops increasing;

[0021] S35: taking the newly created cluster area as the target area.

[0022] Wherein, the number of the target colors includes at least one.

[0023] It can be understood that the purpose of obtaining the clustered area of ​​the target color pixels is to identify the helmet area and the safety clothing element area in the person prediction area. Therefore, the target color is the color used by the helmet and the safety clothing, which is generally yellow. This method counts the clustered area of ​​the target color pixels, that is, preliminarily determines the candidate areas of the helmet area and the safety clothing element area. Further confirmation needs to be determined based on the positional relationship and proportional relationship between the clustered area of ​​the target color pixels and the face area.

[0024] In an optional embodiment of the present application, step S4 includes at least one of the following:

[0025] S41: if the positional relationship and the proportional relationship between the target area and the face area meet a first preset condition, determining that the person in the person prediction area is wearing a helmet;

[0026] S42: Under the second preset condition of the position relationship and the ratio relationship between the target area and the face area, determining whether the person in the person prediction area is wearing safety clothing;

[0027] S43: When it is determined that the person in the person prediction area is not wearing a helmet or safety clothing, determine whether the person is in a dangerous state.

[0028] It can be understood that the present application does not adopt the traditional image recognition model to identify the helmets and safety clothing, but determines the helmet and safety clothing element areas in the character prediction area through statistics of the target color pixel clustering areas and the positional relationship and proportional relationship between these clustering areas and the face area. This greatly reduces the amount of calculation and model training and improves the recognition efficiency.

[0029] In an optional embodiment of the present application, the first preset condition includes: the ratio of the total number of pixels in the target area to the total number of pixels in the face area is within a first ratio range, and there are N adjacent pixels in the target area that are adjacent to the pixels in the face area, and N is greater than a first threshold.

[0030] In an optional embodiment of the present application, the second preset condition includes: the ratio of the total number of pixels in the target area to the total number of pixels in the face area is within a second ratio range, and the minimum distance between the target area and the center point of the face area is less than a second threshold.

[0031] In an optional embodiment of the present application, the above-mentioned robot personnel detection method based on real-time video stream also includes:

[0032] S6: If the face recognition is successful, performing gesture recognition on the real-time captured image;

[0033] S7: When the target gesture is recognized, the lighting device is turned on.

[0034] It can be understood that after the robot recognizes the target person as a registered construction worker through facial recognition, it can turn on the lighting equipment for the construction worker through gestures to provide support for his night work.

[0035] In a second aspect, the present application provides a detection robot, including a robot chassis, a driving power supply, a sensor, a communication device, an alarm device, a lighting device, and an industrial control host.

[0036] The industrial control host includes a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute any method as described in the first aspect.

[0037] Among them, the processor of the industrial control host can be an Intel i7 processor, and the industrial control host uses the mainstream Linux operating system to control the robot and exchange data.

[0038] The robot chassis includes a tracked differential wire-controlled chassis, which has extremely strong terrain adaptability and can adapt to different terrains in various construction periods of engineering construction scenes. It has excellent climbing ability and is competent for various complex environments.

[0039] The communication device is used to communicate with the monitoring terminal. The detection robot is equipped with a multi-network card that supports weak network aggregation to support stable communication in weak signal areas. The monitoring personnel open the client program through the monitoring terminal to directly view the monitoring interface and control the robot.

[0040] The sensors include cameras and obstacle avoidance laser radars, wherein the camera is the main sensing device of the detection robot. In addition, the detection robot can also be equipped with a blind spot camera for the lateral field of view and a single-line laser radar with obstacle avoidance capability, which can provide more security for the remote control of the monitoring personnel.

[0041] The alarm device includes at least one of a speaker, a display, and an indicator light.

[0042] The driving power supply is used to power the sensor, the communication device, the alarm device, the lighting device and the processor. The driving power supply can use a large-capacity lead-acid battery to provide energy for the main control, sensor, and communication device, and can support 5-6 hours of continuous work to meet most inspection needs.

[0043] In a third aspect, the present application provides a computer-readable storage medium storing a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method as described in any one of the first aspects.

[0044] Beneficial effects:

[0045] This application discloses a robot personnel detection method based on real-time video stream. After extracting the predicted person area from the collected real-time shooting picture, on the one hand, face recognition is performed on the predicted person area to prevent unregistered construction workers from entering the construction site; on the other hand, the relationship between the target area and the face area is used to determine whether the person is wearing a safety helmet or safety clothing. This application has the advantages of high efficiency and real-time performance, which helps to improve the safety of the workplace, and take various alarm measures in time to protect the safety of personnel and improve the personnel's ability to perceive dangerous situations.

[0046] This application does not use traditional image recognition models to identify safety helmets and safety clothing. Instead, it determines the safety helmet and safety clothing element areas in the person prediction area through statistics of target color pixel clustering areas and the positional and proportional relationships between these clustering areas and the face area. This greatly reduces the amount of calculation and model training and improves recognition efficiency.

[0047] In order to make the above-mentioned objects, features and advantages of the present application more obvious and understandable, optional embodiments are specifically listed below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 This application provides a flowchart of a robot personnel detection method based on real-time video stream;

[0050] Figure 2 This is a schematic diagram of obtaining a predicted area of ​​a person in a real-time shooting picture by using a YOLO model provided by the present application;

[0051] Figure 3 is a schematic diagram of a face area and various target areas in a person prediction area provided by the present application;

[0052] Figure 4It is a schematic diagram of the principle of the target area confirmation algorithm provided in this application. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0054] First, as Figure 1 As shown, the present application provides a robot personnel detection method based on real-time video stream, which includes:

[0055] S1: Obtain the real-time shooting picture captured by the camera, and obtain the person prediction area in the real-time shooting picture through the YOLO model.

[0056] YOLO (You Only Look Once) is a deep learning model for object detection that can quickly and accurately identify and locate multiple objects in an image. Compared with traditional object detection methods, YOLO has higher real-time performance because it converts the object detection task into a single neural network prediction problem.

[0057] The YOLO model can include an input layer, a composite hidden layer, a prediction layer, and an output layer. After the input layer obtains the input image, multiple convolution-pooling modules in the composite hidden layer repeatedly perform convolution and pooling operations on the input image to extract the feature image of the input image; the two fully connected layers in the prediction layer predict the prediction box position information, prediction box credibility score, and category probability table of each element in the input image based on the feature image. The output layer outputs the prediction result based on the prediction box position information, prediction box credibility score, and category probability table. Figure 2 In the real-time shooting picture 100 shown, the person prediction area 10 can be identified through the YOLO model.

[0058] S2: Extract the face area in the person prediction area for face recognition.

[0059] The face recognition model is a technology that uses the biometric features of face images to identify people. The principle is to extract key feature points from face images and identify the differences between different faces by analyzing and comparing these feature points. The face recognition model is mainly divided into two stages: feature extraction and feature matching. In the feature extraction stage, the model extracts feature points from face images through algorithms and converts them into digital vectors. In the feature matching stage, the model compares the digital vectors with the digital vectors of known face images stored in the database to identify the identity of the face. At the same time, the face recognition model can also optimize its recognition capabilities through continuous learning to improve the accuracy and speed of recognition.

[0060] like Figure 3 As shown, the face area 11 can be extracted from the person prediction area 10 through the face recognition model.

[0061] In the example of this application, the facial image information of all registered construction workers has been stored in the database. The facial recognition model verifies the face in the face area based on the registered facial images in the database. If the verification fails, it means that the person is not a registered construction worker and has illegally entered the construction site.

[0062] S3: Extract the clustered area of ​​target color pixels in the person prediction area as the target area.

[0063] Among them, the number of target colors includes at least one. It can be understood that the purpose of obtaining the clustered area of ​​the target color pixels is to identify the helmet area and the safety clothing element area in the person prediction area. Therefore, the target color is the color used for the helmet and safety clothing, which is generally dark yellow, light yellow, etc. This method counts the clustered area of ​​the target color pixels, that is, preliminarily determines the candidate areas of the helmet area and the safety clothing element area. Further confirmation needs to be determined based on the positional relationship and proportional relationship between the clustered area of ​​the target color pixels and the face area.

[0064] like Figure 3 As shown, in addition to the face area 11, the character prediction area 10 can also determine the first target area 12 and the second target area 13 based on the color pixels, wherein the first target area 12 is the clustering area of ​​the helmet pixels, and the second target area 13 is the clustering area of ​​the safety suit pixels.

[0065] S4: judging whether the person in the person prediction area is in a dangerous state according to the relationship between the target area and the face area.

[0066] In an optional embodiment of the present application, step S4 includes at least one of the following:

[0067] S41: When the position relationship and the proportion relationship between the target area and the face area meet the first preset condition, it is determined that the person in the person prediction area is wearing a helmet.

[0068] In an optional embodiment of the present application, the first preset condition includes: the ratio of the total number of pixels in the target area to the total number of pixels in the face area is within a first ratio range, and there are N adjacent pixels in the target area that are adjacent to the pixels in the face area, and N is greater than a first threshold.

[0069] S42: Under the second preset condition of the positional relationship and the proportional relationship between the target area and the face area, it is determined whether the person in the person prediction area is wearing safety clothing.

[0070] In an optional embodiment of the present application, the second preset condition includes: the ratio of the total number of pixels in the target area to the total number of pixels in the face area is within a second ratio range, and the minimum distance between the target area and the center point of the face area is less than a second threshold.

[0071] S43: When it is determined that the person in the person prediction area is not wearing a helmet or safety clothing, determine whether the person is in a dangerous state.

[0072] It can be understood that the present application does not adopt the traditional image recognition model to identify the helmets and safety clothing, but determines the helmet and safety clothing element areas in the character prediction area through statistics of the target color pixel clustering areas and the positional relationship and proportional relationship between these clustering areas and the face area. This greatly reduces the amount of calculation and model training and improves the recognition efficiency.

[0073] S5: Execute an alarm operation when face recognition fails or when it is determined that a person in the person prediction area is in a dangerous state.

[0074] Among them, S1, S2, etc. are only step identifiers, and the execution order of the method is not necessarily in the order of numbers from small to large. For example, step S2 may be executed first and then step S1. This application does not impose any limitation on this.

[0075] It can be understood that the present application discloses a robot personnel detection method based on real-time video stream. After extracting the predicted person area from the collected real-time shooting picture, on the one hand, face recognition is performed on the predicted person area to prevent unregistered construction personnel from entering the construction site; on the other hand, the relationship between the target area and the face area is used to determine whether the person is wearing a safety helmet or safety clothing. The present application has the advantages of high efficiency and real-time performance, which helps to improve the safety of the workplace, and take various alarm measures in time to protect the safety of personnel and improve the personnel's ability to perceive dangerous situations.

[0076] In an optional embodiment of the present application, the alarm operation includes at least one of the following: sending an alarm signal to a speaker to drive the speaker to emit an alarm audio; sending an alarm signal to a display to drive the display to display alarm information; sending an alarm signal to an indicator light to drive the indicator light to light up the alarm signal; sending an alarm information to a monitoring terminal through a communication device.

[0077] It can be understood that the alarm operations of the present application are divided into two categories. One is to alarm through existing alarm equipment, such as speakers, displays, indicator lights, etc.; the other is to send alarm information to the monitoring terminal through communication equipment, that is, the monitoring personnel can view the real-time shooting screen and alarm information from the monitoring terminal, and the monitoring personnel can then control the robot through the communication equipment to take corresponding safety measures.

[0078] In an optional embodiment of the present application, step S3 includes:

[0079] S31: Remove the known target area in the person prediction area to obtain the area to be processed.

[0080] Steps S31 to S35 are a cyclic process, and a target area can be obtained in each cycle. The main purpose of step S31 is to remove the target area determined last time and obtain the area to be processed in this cycle.

[0081] S32: randomly selecting pixels of the target color in the area to be processed as target pixels, and assigning the target pixels to the newly created cluster area.

[0082] S33: pixels of the target color among the pixels adjacent to the target pixel are also classified into the newly created clustering area.

[0083] S34: Continue to analyze the adjacent pixels adjacent to each pixel in the newly created cluster area, and continue to classify the pixels of the target color among the adjacent pixels into the newly created cluster area until the number of pixels in the newly created cluster area stops increasing.

[0084] Take the gathering area of ​​helmet pixels, i.e., the first target area 12, as an example for explanation. Figure 4 As shown, a part of the first target area 12 is selected for enlarged description, such as Figure 4Pixels 1 to 85 are all in the area to be processed, pixel 50 is randomly selected as the target pixel, and the target pixel 50 is classified into the newly created cluster area; with the target pixel 50 as the center, the pixels 35, 36, 37, 51, 65, 64, 63, and 49 adjacent to the target pixel 50 are analyzed in turn to see if they belong to the target color. If all of these 8 pixels belong to the target color, these 8 pixels are classified into the newly created cluster area. At this time, the newly created cluster area includes 9 pixels: pixel 35, 36, 37, 51, 65, 64, 63, 49, and 50; continue to analyze whether the adjacent pixels belong to the target color with these 9 pixels as the center. If they do belong to the target color, continue to classify the adjacent pixels into the newly created cluster area until the number of pixels in the newly created cluster area no longer increases. Figure 4 As shown, with pixel 67 belonging to the newly created cluster area as the center, its adjacent pixels 52, 53, and 66 belong to the target color, while its adjacent pixels 54, 68, 82, 81, and 80 do not belong to the target color.

[0085] S35: The newly created cluster area is used as the target area.

[0086] After executing steps S32 to S34 on the area to be processed, the entire newly created cluster area will be obtained, and the newly created cluster area will be used as a target area. Figure 3 As shown, in addition to the face area 11, the character prediction area 10 can also determine the first target area 12 and the second target area 13 based on the color pixels, wherein the first target area 12 is the clustering area of ​​the helmet pixels, and the second target area 13 is the clustering area of ​​the safety suit pixels.

[0087] In an optional embodiment of the present application, the above-mentioned robot personnel detection method based on real-time video stream also includes:

[0088] S6: When the face recognition is successful, gesture recognition is performed on the real-time shooting image.

[0089] S7: When the target gesture is recognized, the lighting device is turned on.

[0090] It can be understood that after the robot recognizes the target person as a registered construction worker through facial recognition, it can turn on the lighting equipment for the construction worker through gestures to provide support for his night work.

[0091] In a second aspect, the present application provides a detection robot, including a robot chassis, a driving power supply, a sensor, a communication device, an alarm device, a lighting device, and an industrial control host.

[0092] The industrial control host includes a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute any method of the first aspect.

[0093] Among them, the processor of the industrial control host can be an Intel i7 processor, and the industrial control host uses the mainstream Linux operating system to control the robot and interact with data. It should be understood that in the embodiment of the present invention, the processor can also be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0094] The robot chassis includes a tracked differential wire-controlled chassis, which has strong terrain adaptability and can adapt to the different terrains in various construction periods of engineering construction scenarios. It has excellent climbing ability and is competent for various complex environments.

[0095] The communication device is used to communicate with the monitoring terminal. The detection robot is equipped with a multi-network card that supports weak network aggregation to support stable communication in weak signal areas. The monitoring personnel open the client program through the monitoring terminal to directly view the monitoring interface and control the robot.

[0096] The sensors include cameras and obstacle avoidance laser radars. The camera is the main sensing device of the detection robot. In addition, the detection robot can also be equipped with a blind spot camera for the side view and a single-line laser radar with obstacle avoidance capability, which can provide more security for the remote control of the monitoring personnel.

[0097] The alarm device includes at least one of a speaker, a display, and an indicator light.

[0098] The driving power supply is used to power sensors, communication equipment, alarm equipment, lighting equipment and processors. The driving power supply can use a large-capacity lead-acid battery to provide energy for the main control, sensors, and communication equipment, which can support 5-6 hours of continuous work and meet most inspection needs.

[0099] In a third aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the steps of any method of the first aspect are implemented.

[0100] The computer-readable storage medium may be an internal storage unit of the terminal device of any of the aforementioned embodiments, such as a hard disk or memory of the terminal device. The computer-readable storage medium may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the terminal device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the terminal device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0101] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0102] In the several embodiments provided in the present application, it should be understood that the disclosed terminal device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0103] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0104] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0105] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0106] The expressions "first", "second", "the first" or "the second" used in various embodiments of the present disclosure may modify various components regardless of order and / or importance, but these expressions do not limit the corresponding components. The above expressions are only configured for the purpose of distinguishing an element from other elements. For example, a first user device and a second user device represent different user devices, although both are user devices. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the present disclosure.

[0107] When one element (e.g., a first element) is referred to as being "(operably or communicatively) coupled" or "(operably or communicatively) coupled to" or "connected to" another element (e.g., a second element), it is understood that the one element is directly connected to the other element or the one element is indirectly connected to the other element via yet another element (e.g., a third element). Conversely, it is understood that when an element (e.g., a first element) is referred to as being "directly connected" or "directly coupled" to another element (the second element), no element (e.g., a third element) is interposed between the two.

[0108] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0109] The above description is only an optional embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other.

[0110] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0111] The above description is only an optional embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other.

[0112] The above description is only an optional embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A robot personnel detection method based on real-time video stream, characterized in that: include: S1: Obtain a real-time shooting picture captured by a camera, and obtain a person prediction area in the real-time shooting picture through a YOLO model; S2: extracting a face area within the person prediction area to perform face recognition; S3: extracting a clustered area of ​​target color pixels in the person prediction area as a target area; S4: judging whether the person in the person prediction area is in a dangerous state according to the relationship between the target area and the face area; S5: Executing an alarm operation when face recognition fails or when it is determined that the person in the person prediction area is in a dangerous state; The step S3 comprises: S31: removing the known target area in the person prediction area to obtain an area to be processed; S32: randomly selecting pixels of the target color in the area to be processed as target pixels, and assigning the target pixels to a newly created cluster area; S33: Also classifying pixels of the target color among the pixels adjacent to the target pixel into the newly created cluster area; S34: Continue to analyze the adjacent pixels adjacent to each pixel in the newly created cluster area, and continue to classify the pixels of the target color among the adjacent pixels into the newly created cluster area until the number of pixels in the newly created cluster area stops increasing; S35: taking the newly created cluster area as the target area; The target pixel is a safety helmet pixel or a safety clothing pixel.

2. The robot personnel detection method based on real-time video stream according to claim 1 is characterized in that: The alarm operation includes at least one of the following: Sending an alarm signal to a speaker, driving the speaker to emit an alarm audio; Sending an alarm signal to a display to drive the display to display alarm information; Sending an alarm signal to the indicator light, driving the indicator light to light up the alarm signal; Send alarm information to the monitoring terminal through communication equipment.

3. The robot personnel detection method based on real-time video stream according to claim 1 is characterized in that: The number of the target colors includes at least one.

4. The robot personnel detection method based on real-time video stream according to claim 1 is characterized in that: The step S4 includes at least one of the following: S41: if the positional relationship and the proportional relationship between the target area and the face area meet a first preset condition, determining that the person in the person prediction area is wearing a helmet; S42: Under the second preset condition of the position relationship and the ratio relationship between the target area and the face area, determining whether the person in the person prediction area is wearing safety clothing; S43: When it is determined that the person in the person prediction area is not wearing a helmet or safety clothing, determine whether the person is in a dangerous state.

5. The robot personnel detection method based on real-time video stream according to claim 4 is characterized in that: The first preset condition includes: The ratio of the total number of pixels in the target area to the total number of pixels in the face area is within a first ratio range, and There are N adjacent pixels in the target area that are adjacent to the pixels in the face area, and N is greater than a first threshold.

6. The robot personnel detection method based on real-time video stream according to claim 4 is characterized in that: The second preset condition includes: The ratio of the total number of pixels in the target area to the total number of pixels in the face area is within a second ratio range, and The minimum distance between the target area and the center point of the face area is less than a second threshold.

7. The robot personnel detection method based on real-time video stream according to claim 1 is characterized in that: Also includes: S6: If the face recognition is successful, performing gesture recognition on the real-time captured image; S7: When the target gesture is recognized, the lighting device is turned on.

8. A detection robot, characterized in that: Including robot chassis, drive power supply, sensors, communication equipment, alarm equipment, lighting equipment, industrial control host, The industrial control host comprises a processor and a memory, the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 7.

9. The detection robot according to claim 8, characterized in that: The robot chassis comprises a tracked differential wire-controlled chassis; The sensors include cameras and obstacle avoidance laser radars; The communication device is used to communicate with the monitoring terminal; The alarm device includes at least one of a speaker, a display, and an indicator light; The driving power supply is used to supply power to the sensor, the communication device, the alarm device, the lighting device and the processor.

Images