Robot-based body temperature monitoring method, device, equipment and storage medium

Abstract

The application discloses a kind of based on robot body temperature monitoring method, device, equipment and storage medium, and the application belongs to robot control technical field.The method comprises: obtaining color image and infrared image including monitoring object by color camera and infrared camera;By preset face recognition algorithm, the face coordinates of monitoring object in color image are identified;According to the resolution ratio conversion relationship of infrared temperature measurement camera and color camera, the face coordinates of monitoring object in infrared image are determined, and feature area is determined according to face coordinates;The average temperature of feature area is identified as the body temperature monitoring result of monitoring object.This technical solution can measure human temperature flexibly;And after recognizing face, feature area is identified again, and the average temperature of feature area is taken as the way of body temperature monitoring result, which can make measurement result more accurate, avoid the influence of high-temperature object, and reduce the probability of unnecessary manual intervention due to calculation error.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of robot control, and particularly relates to a body temperature monitoring method and device based on a robot, equipment and a storage medium. BACKGROUND

[0002] Nowadays, the body temperature monitoring equipment in indoor places is mostly a body temperature detection door. The body temperature detection door adopts an infrared radiation detection principle. Since all objects in nature with a temperature higher than absolute zero (-273.15 DEG C) will radiate infrared rays, through the combination of electronic technology and infrared technology, an infrared thermal image can be presented, and the heat of the surface of the radiation source can be converted through a heat radiation algorithm to realize the conversion between the thermal image and the temperature. When a person passes through the body temperature detection door, a display screen will display the temperature of the passing person in real time, and whether the passing person is safe is determined according to the temperature of the human body. If the temperature of the passing person exceeds a set value, the temperature detection security door will alarm through a real person voice to remind the staff that the person needs to be observed medically.

[0003] However, the body temperature detection door is too large in size and inconvenient to move in actual application. The temperature detection door has a too large measurement range and does not perform face recognition. As long as the user carries a high-temperature object, the alarm will be triggered. Meanwhile, the physical radiation degree is affected by many factors such as dust and water vapor. Since infrared radiation has obvious absorbability, the sensing ability of the infrared temperature measuring machine equipment to the radiation source will also be directly affected, thereby causing the difference between the measured temperature and the actual temperature of the human body. Therefore, how to only identify the temperature of the human body and reduce the difference between the measured temperature of the human body and the actual temperature of the human body is a problem to be solved in the field. SUMMARY

[0004] The embodiments of the application provide a body temperature monitoring method, device, equipment and storage medium based on a robot, which aims to solve the problem that the body temperature monitoring equipment in the prior art is inconvenient to move, face recognition is not performed when the temperature is monitored, and the inspection range is too large, thereby causing the user to be regarded as a human body temperature when carrying a high-temperature object and thus false alarms are generated. Meanwhile, the abnormal body temperature cannot be counted, which also increases the management pressure of the relevant places. The present scheme can flexibly measure the temperature of the human body. By taking the way of identifying the feature region after identifying the face and taking the average temperature of the feature region as the body temperature monitoring result, the measurement result can be more accurate, the influence of the high-temperature object is avoided, and the probability of unnecessary manual intervention due to calculation errors is reduced.

[0005] In a first aspect, the embodiments of the application provide a body temperature monitoring method based on a robot. The method is performed by a robot, the robot includes a photographing device, the photographing device includes a color camera and an infrared temperature measuring camera, and the robot interacts with a background dispatching system, the background dispatching system is used to control the robot. The method comprises the following steps.

[0006] acquiring a color image including the monitoring object by a color camera, and acquiring an infrared image including the monitoring object by an infrared temperature measurement camera;

[0007] identifying a face coordinate of the monitoring object in the color image by a preset face recognition algorithm;

[0008] determining a face coordinate of the monitoring object in the infrared image according to a resolution ratio conversion relationship between the infrared temperature measurement camera and the color camera;

[0009] determining a feature region according to the face coordinate of the monitoring object in the infrared image;

[0010] identifying a temperature average value of the feature region as a body temperature monitoring result of the monitoring object.

[0011] Further, before identifying the face coordinate of the monitoring object in the color image by the preset face recognition algorithm, the method further comprises:

[0012] compressing the color image by a preset compression algorithm to obtain a compressed image; wherein a compression range of the preset compression algorithm is greater than or equal to 1 / 12 and less than 1 / 1;

[0013] Correspondingly, identifying the face coordinate of the monitoring object in the color image by the preset face recognition algorithm comprises:

[0014] identifying the face coordinate of the monitoring object in the compressed image of the color image by the preset face recognition algorithm.

[0015] Further, determining the feature region according to the face coordinate of the monitoring object in the infrared image comprises:

[0016] determining face angle information of the monitoring object by the preset face recognition algorithm according to the face coordinate of the monitoring object in the infrared image;

[0017] determining a feature region in the face coordinate according to the face angle information.

[0018] Further, the feature region includes a circular region with a diameter of 1 centimeter at a center position of a forehead of the monitoring object.

[0019] Further, identifying the temperature average value of the feature region as the body temperature monitoring result of the monitoring object comprises:

[0020] If the temperature average value is within a preset normal temperature range, identifying a low temperature normal range corresponding to the temperature average value or a high temperature warning range corresponding to the temperature average value.

[0021] Further, the average temperature of the feature region is identified as a body temperature monitoring result of the monitoring object, including:

[0022] If the average temperature is not within a preset normal temperature range, a filtering instruction is generated to perform filtering operation on the current color image and the infrared image;

[0023] A next set of color image and infrared image acquired by the color camera and the infrared temperature measurement camera are acquired to perform body temperature monitoring.

[0024] Further, after identifying whether the average temperature corresponds to a low temperature normal range or a high temperature early warning range if the average temperature is within the preset normal temperature range, the method further includes:

[0025] If the average temperature corresponds to the high temperature early warning range, it is determined as an abnormal body temperature;

[0026] Real-time coordinates of the robot are acquired as a real-time position of the abnormal body temperature;

[0027] The real-time position is reported to a background dispatching system for archiving processing of the abnormal body temperature by the background dispatching system.

[0028] In a second aspect, the embodiments of the present application provide a body temperature monitoring device based on a robot, the device is configured in a robot, the robot includes a photographing device, the photographing device includes a color camera and an infrared temperature measurement camera; and interacts with a background dispatching system, the background dispatching system is used to control the robot; the device includes:

[0029] An acquisition module is configured to acquire a color image including a monitoring object by a color camera, and acquire an infrared image including a monitoring object by an infrared temperature measurement camera;

[0030] An identification module is configured to identify a face coordinate of a monitoring object in the color image by a preset face recognition algorithm;

[0031] A coordinate determination module is configured to determine a face coordinate of a monitoring object in the infrared image according to a resolution ratio conversion relationship between the infrared temperature measurement camera and the color camera;

[0032] A feature region determination module is configured to determine a feature region according to the face coordinate of the monitoring object in the infrared image;

[0033] A monitoring module is configured to identify an average temperature of the feature region as a body temperature monitoring result of the monitoring object.

[0034] In a third aspect, an electronic device is provided, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, and the program or instructions, when executed by the processor, implement the steps of the method according to the first aspect.

[0035] In a fourth aspect, a readable storage medium is provided, which stores a program or instructions, and the program or instructions, when executed by a processor, implement the steps of the method according to the first aspect.

[0036] In a fifth aspect, a chip is provided, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is configured to execute a program or instructions to implement the method according to the first aspect.

[0037] In the embodiments of the present application, a color image including a monitoring object is acquired by a color camera, and an infrared image including the monitoring object is acquired by an infrared temperature measurement camera; a face coordinate of the monitoring object in the color image is identified by a preset face recognition algorithm; the face coordinate of the monitoring object in the infrared image is determined according to a resolution ratio conversion relationship between the infrared temperature measurement camera and the color camera; a feature region is determined according to the face coordinate of the monitoring object in the infrared image; and an average temperature of the feature region is identified as a body temperature monitoring result of the monitoring object. Through the above-mentioned robot-based body temperature monitoring method, the human body temperature can be flexibly measured; and by identifying the face and then identifying the feature region, and taking the average temperature of the feature region as the body temperature monitoring result, the measurement result can be more accurate, the influence of high-temperature objects can be avoided, and the probability of unnecessary manual intervention due to calculation errors can be reduced. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 FIG. 1 is a flowchart of a robot-based body temperature monitoring method provided by an embodiment of the present application;

[0039] Figure 2 FIG. 2 is a flowchart of a robot-based body temperature monitoring method provided by another embodiment of the present application;

[0040] Figure 3 FIG. 3 is a structural diagram of a robot-based body temperature monitoring device provided by another embodiment of the present application;

[0041] Figure 4 FIG. 4 is a structural diagram of an electronic device provided by another embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purposes, technical solutions and advantages of the present application clearer, the following further describes the specific embodiments of the present application with reference to the drawings. It should be understood that the specific embodiments described herein are merely intended to explain the present application, but not to limit the present application. In addition, it should be noted that, for the purpose of description, only the parts related to the present application are shown in the drawings, but not all. Before discussing the example embodiments in more detail, it should be mentioned that some example embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The processes can be terminated when the operations are completed, but can also have additional steps not included in the drawings. The processes can correspond to methods, functions, procedures, subroutines, etc.

[0043] The technical solutions in the embodiments of the present application will be described clearly below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art belong to the scope of protection of the present application.

[0044] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of a kind, and are not limited to the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / ", generally means that the objects before and after are in an "or" relationship.

[0045] The robot-based body temperature monitoring method, device, equipment and storage medium provided by the embodiments of the present application will be described in detail below with reference to the drawings and specific embodiments and application scenarios.

[0046] Embodiment one

[0047] Figure 1 is a flowchart of the robot-based body temperature monitoring method provided by the first embodiment of the present application.

[0048] As shown in Figure 1 , the specific steps include the following steps:

[0049] S101, acquires a color image of the monitored object using a color camera, and acquires an infrared image of the monitored object using an infrared thermometer camera.

[0050] Firstly, this solution can be used in scenarios where robots monitor human body temperature and interact with the back-end scheduling system, reporting abnormal human body temperatures to the back-end scheduling system for staff to manage accordingly.

[0051] Based on the above usage scenarios, it is understandable that the executing entity of this application can be the robot, and no further restrictions are imposed here.

[0052] In this solution, the method is executed by a robot, which includes a photography device comprising a color camera and an infrared temperature measurement camera; and interacts with a background scheduling system, which is used to control the robot.

[0053] A robot can be an intelligent machine capable of semi-autonomous or fully autonomous operation, able to perform tasks such as work or movement through programming and automatic control. It possesses basic characteristics such as perception, decision-making, and execution, and can assist or even replace humans in completing dangerous, heavy, and complex tasks, improving work efficiency and quality, serving human life, and expanding or extending the scope of human activities and capabilities. In this solution, the robot can be a disinfection and epidemic prevention robot. By installing a disinfection system inside the robot to generate disinfectant gas, the robot's pneumatic system rapidly diffuses the disinfectant gas throughout the indoor space, increasing the coverage and uniformity of disinfection. This effectively and thoroughly kills pathogenic microorganisms in the air. The disinfection robot can automatically, efficiently, and accurately disinfect and prevent epidemics indoors according to a pre-set route.

[0054] The color camera can be the same camera used to take color photographs. In this solution, the color camera is used to capture images of the user whose temperature needs to be measured, so that the robot can perform body temperature monitoring based on these images.

[0055] An infrared temperature measurement camera is a camera that uses infrared light to create an image. Besides using ordinary light, it can also form an image based on the heat of infrared rays. In this solution, the infrared temperature measurement camera is used to capture an image containing the user whose temperature needs to be measured and the user's body temperature. This image is then used by the robot to perform body temperature monitoring based on the robot's input and images captured by a color camera.

[0056] The background dispatching system can be an internal system of the robot, and the system can be applied to terminal devices within a certain range. For example, when the robot is used in a hospital, the system can be applied to intelligent terminal devices (such as desktop computers, notebook computers, mobile phones, and the like) in the hospital. The staff can receive relevant information (such as abnormal body temperature and abnormal body temperature related positions) transmitted by the robot through the system, and manage according to the information. At the same time, the background dispatching system can control the robot to move to different positions, so that the robot can perform corresponding temperature measurement work subsequently.

[0057] In the present scheme, the monitoring object can be a user whose temperature is to be collected. For example, when the robot is applied in a hospital, the monitoring object can be a patient whose body temperature is to be measured daily. The monitoring object can also be a user entering the hospital. Since the hospital needs to perform daily epidemic prevention work, the body temperature of all users entering the hospital needs to be measured, and only users with normal body temperature can enter the hospital.

[0058] The color image can be an image containing a user to be measured by a color camera. The infrared image can be an image containing a user to be measured and the body temperature of the user by an infrared temperature measurement camera.

[0059] The obtaining can be a process of capturing a color image and an infrared image of the monitoring object by a color camera and an infrared temperature measurement camera. The acquisition frequency can be determined according to the camera frame rate. For example, if the camera frame rate is 25 frames, the camera will take 25 pictures per second. When the background dispatching system controls the robot to move to different positions, the robot will continuously capture pictures by using the color camera and the infrared temperature measurement camera during the movement, and the body temperature monitoring is realized based on the pictures.

[0060] In S102, a face coordinate of the monitoring object in the color image is recognized by using a preset face recognition algorithm.

[0061] The face recognition algorithm can refer to, after detecting a face and positioning facial key feature points, a main face region can be cropped out, preprocessed, and fed into a back-end recognition algorithm. In the present scheme, a convolutional neural network algorithm can be used as the face recognition algorithm. The convolutional neural network is a kind of feedforward neural network containing convolution calculation and having a deep structure, and is one of the representative algorithms of deep learning. The convolutional neural network has a representation learning ability and can perform translation-invariant classification on input information according to its hierarchical structure. The convolutional neural network is constructed by imitating the visual perception mechanism of organisms and can perform supervised learning and unsupervised learning. The convolution kernel parameters in the hidden layer are shared and the sparsity of the interlayer connection, so that the convolutional neural network can learn the grid features such as pixels and audio with a small amount of calculation, has stable effect and does not require additional feature engineering on data. The implementation process of face recognition based on the convolutional neural network is to obtain a face by using opencv (a cross-platform computer vision library), collect face data, load the collected face data into memory, build a convolutional neural network of its own, train the network with face data, save the trained network as a model, and finally use opencv to obtain real-time face and use the previously trained model to recognize the face. The opencv is a cross-platform computer vision and machine learning software library that implements many general algorithms in image processing and computer vision.

[0062] The face coordinates can be the coordinates of the rectangle of the face. After the face recognition algorithm is used to detect and mark the region where the face is located, the coordinates of the four vertices of the rectangular region where the face is located can be further determined, wherein the determination of the four vertex coordinates is determined according to the pixel width and height of the image. For example, the pixel width of a face image is 600px, and the pixel height is 400px. The upper left corner coordinates of the rectangular region are set to (0, 0), the upper right corner coordinates are (600, 0), the lower right corner coordinates are (600, 400), and the lower left corner coordinates are (0, 400).

[0063] The recognition can be a process of determining face coordinates by using a preset face recognition algorithm, i.e., a process of determining face coordinates by using a convolutional neural network algorithm, and the convolutional neural network algorithm is built based on opencv, so the face rectangle frame coordinates are recognized by opencv. Specifically, the Harr classifier (Haar-like feature + integral image method + AdaBoost (iterative algorithm) + cascade) of opencv is used to detect a face and output the rectangle coordinates of the face. The Harr classifier is actually an application of the Boosting algorithm, and the AdaBoost algorithm in the Boosting algorithm is used, only the strong classifier trained by the AdaBoost algorithm is cascaded, and the efficient rectangle feature and integral image method are adopted in the bottom feature extraction. The core idea of the Boosting algorithm is to improve a weak learning method into a strong learning algorithm. Weak learning means that the recognition rate of a learning algorithm for a set of concepts is only a little better than random recognition; strong learning means that the recognition rate of a learning algorithm for a set of concepts is very high. The core idea of AdaBoost is to train different classifiers (weak classifiers) for the same training set, and then combine these weak classifiers to form a stronger final classifier (strong classifier).

[0064] The determination of the image coordinates is based on the world coordinate system, the camera coordinate system plane coordinate system and the pixel coordinate system. The world coordinate system, also known as the measurement coordinate system, is a three-dimensional rectangular coordinate system (xw, yw, zw). The spatial positions of the camera and the object to be measured can be described in the world coordinate system. The position of the world coordinate system is determined according to the actual situation. The camera coordinate system is also a three-dimensional rectangular coordinate system (xc, yc, zc). The origin of the camera coordinate system is the optical center of the lens, the x and y axes are parallel to the two sides of the plane, and the z axis is the optical axis of the lens, which is perpendicular to the image plane. The transformation from the world coordinate system to the camera coordinate system is a rigid body transformation, that is, only the spatial position (translation) and orientation (rotation) of the object are changed, and the shape of the object is not changed. The rotation matrix, translation vector and camera external parameters can be used to obtain the conversion matrix from the world coordinate system to the camera coordinate system. The camera external parameters are parameters in the world coordinate system, such as the position of the camera and the direction of rotation, etc. From the camera coordinate system to the image coordinate system, it belongs to the perspective projection relationship, which converts from 3D to 2D. The plane coordinate system represents the position of the pixel in physical units, with a unit of mm. The coordinate origin is the intersection position of the camera optical axis and the image coordinate system, and the conversion matrix from the camera coordinate system to the ideal image coordinate system is obtained according to the similar triangle principle and homogeneous coordinates. The pixel coordinate system uses pixels as the unit, and the coordinate origin is at the top left corner. There is no rotation from the image coordinate system to the pixel coordinate system, only the coordinate origin and the unit are different, and the conversion matrix from the image coordinate system to the pixel coordinate system is represented by homogeneous coordinates. Finally, the conversion matrix from the world coordinate system to the pixel coordinate system can be determined jointly according to the above three conversion matrices, and the face coordinates, i.e. the rectangular coordinates, can be further determined.

[0065] S103, determining the face coordinates of the monitoring object in the infrared image according to the resolution ratio conversion relationship between the infrared temperature measurement camera and the color camera.

[0066] The resolution ratio conversion relationship can be a coordinate system conversion matrix determined by the infrared temperature measurement camera coordinate system and the color camera coordinate system according to different resolutions. Since the resolutions of the infrared temperature measurement camera and the color camera are different, the coordinate systems are different, and thus the coordinates of the same object in the two cameras are different. Since the rectangular coordinates of the face in the color camera are obtained in advance, the resolution ratio conversion relationship between the infrared temperature measurement camera and the color camera can be determined to determine the coordinates of the face in the infrared temperature measurement camera.

[0067] The determination can be a process of calculating the face coordinates of the monitoring object in the infrared image according to the resolution ratio conversion relationship and the rectangular coordinates of the face in the color camera. For example, the resolution of the color camera is 1024*768, the top-left corner coordinates of the face rectangular coordinates are (187.5, 375), and the resolution of the infrared temperature measurement camera is 1920*1080. The coordinate conversion process in the infrared image is:

[0068] X = 187.5 * 1024 ÷ 1920 = 100

[0069] Y = 375 * 1024 ÷ 1920 = 200

[0070] Then the top-left coordinate of the face rectangle is represented as (100, 200) in the infrared image. Further, assuming that the face coordinate of the color camera is (XA, YA), the resolution of the color camera is a*b, and the resolution of the infrared temperature camera is c*d, then the conversion formula (i.e., the resolution ratio conversion relationship) of the (XB, YB) coordinate of the infrared temperature camera is:

[0071] XB = XA * a ÷ c

[0072] YB = YA * b ÷ d

[0073] When determining other coordinates in the face rectangle, the pre-set resolution ratio conversion relationship can be called for calculation respectively, and finally the four coordinates are combined to determine the face coordinate of the monitoring object in the infrared image.

[0074] S104, determining a feature region according to the face coordinate of the monitoring object in the infrared image.

[0075] The feature region can be a region selected when monitoring the body temperature. In the present scheme, the feature region can be the forehead. Since most of the temporal superficial arteries of the human body need to pass through the central part of the forehead to ensure the blood oxygen supply of the intracranial blood vessels, the forehead central part measured by the forehead thermometer is generally the most accurate.

[0076] The determination can be a process of identifying 468 3D face landmarks of the face using MediaPipe Face Mesh and determining the forehead position according to the face landmarks. MediaPipe Face Mesh is a face geometry solution that can estimate 468 3D face landmarks in real time even on mobile devices. It uses machine learning (ML) to infer 3D surface geometry, requiring only a single camera input without a dedicated depth sensor. In addition, the solution is bundled with the "Face Geometry" module, which bridges the gap between face landmark estimation and useful real-time augmented reality (AR) applications. It establishes a metric 3D space and uses face landmark screen positions to estimate face geometry within that space. The face geometry data consists of common 3D geometry primitives, including face pose transformation matrices and triangular face meshes. In the background, a lightweight statistical analysis method called "Statistical Analysis Method" is used to drive robust, high-performance, and portable logic. The analysis runs on the CPU and has minimal speed / memory footprint based on ML model inference.

[0077] On the basis of the above technical solutions, optionally, according to the face coordinates of the monitoring object in the infrared image, a feature region is determined, comprising:

[0078] According to the face coordinates of the monitoring object in the infrared image, and by the preset face recognition algorithm, the face angle information of the monitoring object is determined.

[0079] According to the face angle information, the feature region in the face coordinates is determined.

[0080] The face angle information can be the coordinates of the points of the forehead position edge positioned by the facial living body fixed-point measurement method after the MediaPipe Face Mesh recognizes 468 3D face landmarks of the face, and each angle is calculated by connecting the symmetric points, and the face angle information is obtained by summarizing each angle.

[0081] Determining the feature region can be that after the MediaPipe Face Mesh recognizes the coordinates of each point of the forehead position, the connection mode of each point is determined according to the angle information, each point is connected according to the coordinates and the connection mode, and the process of drawing the feature region, that is, the process of drawing the forehead region, is drawn.

[0082] The present scheme further determines the feature region by setting the face angle information. In the traditional face recognition temperature measurement method, the highest temperature in the face rectangular frame is taken as the broadcast temperature. In actual application, there are often measurement errors, for example, there is a high temperature of the mobile phone back cover caused by making a phone call, so that the measured temperature is too high. The present scheme can more accurately draw the feature region of the face according to the face angle and the coordinates of each point, so that the result of identifying the average value of the temperature of the feature region is more accurate, and the possibility of unnecessary manual intervention caused by calculation error can be further reduced.

[0083] On the basis of the above technical solutions, optionally, the feature region includes a circular region with a diameter of 1 centimeter at the center position of the forehead of the monitoring object.

[0084] In the scheme, although the face coordinates are rectangular coordinates, that is, only four points of the rectangular frame are selected as the face coordinates, when the face coordinates of the monitoring object in the color image are identified, the coordinates of all pixels in the rectangular frame are also determined after the rectangular frame is determined. When the face coordinates of the monitoring object in the infrared image are determined according to the resolution ratio conversion relationship between the infrared temperature measurement camera and the color camera, the coordinates of all pixels in the rectangular frame are also converted, so after the feature area is determined, the coordinates of all pixels in the circular area with a diameter of 1 cm at the center of the forehead can also be determined. When the average temperature of the feature area is identified, that is, the temperature values corresponding to all pixel points in the circular area with a diameter of 1 cm are first counted, and then the average temperature of this area is calculated according to the preset average temperature algorithm, and the calculated average temperature is taken as the body temperature monitoring result of the monitoring object.

[0085] In the scheme, by further refining the feature area into a circular area with a diameter of 1 cm at the center of the forehead, the measurement result can be closer to the real temperature of the human body, the influence of high-temperature objects can be avoided, and the possibility of unnecessary manual intervention due to calculation errors can be reduced.

[0086] S105, identifying the average temperature of the feature area as the body temperature monitoring result of the monitoring object.

[0087] The average temperature can be the average temperature of the face feature area, that is, the average temperature of the forehead. Since the infrared image can reflect the temperature value of each pixel in the image, when the forehead position is determined, the average temperature can be calculated according to the temperature values corresponding to all pixel points at the forehead position in the infrared image. The body temperature monitoring result can be the final determined body temperature of the monitoring object, that is, the body temperature of the monitoring object.

[0088] The identification can be a process of calculating the average temperature according to the temperatures corresponding to all pixel points at the forehead position in the infrared image captured by the infrared temperature measurement camera, and the calculation process can be represented as:

[0089] Average temperature = (pixel point 1 temperature + pixel point 2 temperature +... + pixel point n temperature) ÷ pixel point number

[0090] On the basis of the above technical scheme, optionally, identifying the average temperature of the feature area as the body temperature monitoring result of the monitoring object, comprising:

[0091] If the average temperature is within the preset normal temperature range, the average temperature corresponding to the low-temperature normal range or the high-temperature early warning range is identified.

[0092] In the scheme, the preset normal temperature range can be the normal human body temperature range (35℃-42℃).

[0093] The low-temperature normal range can be a temperature range in which a human body does not have fever, and can be 36℃-38℃.

[0094] The high-temperature early warning range can be a temperature range in which a human body has fever, and can be 38℃-42℃.

[0095] The identification can be a process of determining whether the body temperature monitoring result of the monitoring object is normal according to comparison of the temperature average value of the feature region in the infrared image with the low-temperature normal range and the high-temperature early warning range. If the temperature average value is not in the low-temperature normal range and the high-temperature early warning range, that is, the temperature average value is between 35℃ and 36℃, it is regarded as invalid temperature, which can be caused by low ambient temperature or cold weather and less clothing. If it is invalid temperature, the temperature average value of the feature region in the next color image and the corresponding infrared image is automatically identified.

[0096] In the scheme, by setting the low-temperature normal range and the high-temperature early warning range, it can be further determined whether the human body temperature is abnormal, that is, whether there is fever, based on the body temperature monitoring result. This method saves manpower and further improves the work efficiency of the staff, and facilitates the staff to manage personnel based on the body temperature monitoring result.

[0097] On the basis of the above technical scheme, optionally, after the temperature average value corresponding to the low-temperature normal range or the high-temperature early warning range is identified, the method further comprises:

[0098] If the temperature average value corresponds to the high-temperature early warning range, it is determined as abnormal body temperature.

[0099] The real-time coordinates of the robot are acquired as a real-time position of the abnormal body temperature;

[0100] The real-time position is reported to a background dispatching system for archiving processing of the abnormal body temperature by the background dispatching system.

[0101] In the scheme, the abnormal body temperature can be the temperature average value corresponding to the high-temperature early warning range, and the determination can be a process of identifying the temperature average value by the robot and comparing it with the high-temperature early warning range to further identify whether it is abnormal body temperature. For example, the temperature average value is 41℃, and the robot identifies that it is in the high-temperature early warning range, and further regards this temperature as abnormal body temperature.

[0102] The real-time coordinates can be coordinates of the robot on a world map or coordinates of the robot on an internal map of a closed space. Specifically, if the robot moves in an open area, the coordinates can be coordinates on a world map, which can be in the form of (latitude, longitude); if the robot moves in a closed space, for example, moves in a hospital, since the world map coordinates cannot show the specific floor, only a point coordinate can be located, if the robot is at the same position on two floors, the coordinates displayed are also the same, so a specific internal map of the hospital is needed to display the coordinate position of the robot. The coordinates on the internal map of the closed space can be in the form of (floor, latitude, longitude). The obtained real-time coordinates are real-time positions of the monitored abnormal body temperature.

[0103] The obtaining can be through a GPS (Navigation Satellite Timing And Ranging Global Position System) positioning tracker inside the robot, through GPS (Navigation Satellite Timing And Ranging Global Position System) and GPRS (General packet radio service) positioning technologies, to accurately know the specific position of the positioning object in a short time. Since the GPS positioning tracker obtains the position information through the GPRS in the Internet card, the Internet card transmits the real-time position of the robot to the server through the network, and the server obtains the real-time coordinates of the robot after data conversion, i.e., the real-time position of the monitored abnormal body temperature. GPS is a satellite-based positioning system used to obtain geographic position information and accurate universal coordinated time. GPRS is a wireless packet switching technology that provides end-to-end and wide-area wireless IP (Internet Protocol) connections.

[0104] The reporting can be that the server obtains the real-time coordinates of the robot and reports them to the background dispatching system in the form of a map, for example, the real-time position of the robot can be displayed on the intelligent mobile device of the staff in three forms of electronic map, satellite image map and topographic map. When the robot moves in an open space, the staff can use the electronic map to obtain the real-time position and specific coordinates of the robot on the map, and the electronic map will have the real-time position of the staff, when the staff moves, the position on the electronic map will also move, which is convenient for the staff to check the distance from the robot; when the robot moves in a closed space, the staff can use the topographic map to obtain the real-time position and specific coordinates of the robot on the map, and the topographic map will have the real-time position of the staff, when the staff moves, the position on the topographic map will also move, which is convenient for the staff to check the distance from the robot.

[0105] The archiving process can be a process in which the background dispatching system creates a database table and stores the abnormal temperature and the corresponding coordinates in the database table. For example, when the robot temperature measurement is used for hospital rounds and the like, a field of patient names can also be added in the database table, which is used for the daily temperature management of different patients and can be expressed in the form of patient name-abnormal temperature-corresponding coordinates.

[0106] In the present scheme, by setting the way of reporting the real-time position of the abnormal temperature to the background dispatching system, the daily management of the staff can be assisted, and the work efficiency of the staff can be improved; it can also be used for the regular temperature measurement and monitoring of the patient's physical condition, and can replace part of the medical staff to perform the rounds, thereby reducing the work burden of the medical staff.

[0107] On the basis of the above technical scheme, optionally, the average temperature of the feature area is identified as the body temperature monitoring result of the monitoring object, comprising:

[0108] If the average temperature is not within the preset normal temperature range, a filtering instruction is generated to perform a filtering operation on the current color image and the infrared image;

[0109] A next set of color images and infrared images obtained by the color camera and the infrared temperature measurement camera are acquired to perform body temperature monitoring.

[0110] In the present scheme, the filtering instruction can be an instruction automatically generated after the robot identifies that the average temperature is not within the preset normal temperature range, i.e., below 35℃ or above 42℃, to filter this temperature and identify the average temperature according to the next color image and infrared image.

[0111] The filtering operation can be considered as a deletion operation, i.e., an operation of automatically deleting the data when the average temperature is not within the preset normal temperature range. The condition for generating the filtering instruction can be that the robot detects that the average temperature is not within the preset normal temperature range, and this condition can also be considered as a triggering mechanism for generating the filtering instruction.

[0112] The acquisition can be a process in which the robot calls the images taken by the color camera and the infrared temperature measurement camera to perform body temperature monitoring. Since the shooting interval is determined according to the camera frame rate, if the camera frame rate is 25 frames, i.e., the camera will take 25 pictures per second, the robot will perform body temperature monitoring according to the order of the taken pictures. If the first set of color images and infrared images can detect the body temperature, other pictures will not be used for monitoring; if the first set of color images and infrared images cannot detect the body temperature, the second set of color images and infrared images are used to detect the body temperature, and so on.

[0113] In the scheme, by generating the filtering instruction when the temperature average value is not in the preset normal temperature range, and automatically monitoring the body temperature according to the next set of color images and infrared images, the temperature beyond the normal human body range can be filtered, and the probability of unnecessary manual intervention due to calculation errors can be further reduced.

[0114] In the embodiment of the present application, a color image including a monitoring object is acquired by a color camera, and an infrared image including the monitoring object is acquired by an infrared temperature measurement camera. The face coordinates of the monitoring object in the color image are identified by a preset face recognition algorithm. The face coordinates of the monitoring object in the infrared image are determined according to the resolution ratio conversion relationship between the infrared temperature measurement camera and the color camera. The feature region of the monitoring object in the infrared image is determined according to the face coordinates of the monitoring object in the infrared image. The average temperature of the feature region is identified as the body temperature monitoring result of the monitoring object. Through the above-mentioned robot-based body temperature monitoring method, the human body temperature can be flexibly measured. Moreover, by identifying the face and then identifying the feature region, and taking the average temperature of the feature region as the body temperature monitoring result, the measurement result can be more accurate, the influence of high-temperature objects can be avoided, and the probability of unnecessary manual intervention due to calculation errors can be reduced.

[0115] Embodiment Two

[0116] Figure 2 is a flowchart of the robot-based body temperature monitoring method provided in Embodiment Two of the present application.

[0117] As shown in Figure 2 , the method specifically comprises the following steps:

[0118] S201, a color image including a monitoring object is acquired by a color camera, and an infrared image including the monitoring object is acquired by an infrared temperature measurement camera.

[0119] S202, a preset compression algorithm is used to compress the color image to obtain a compressed image; wherein the compression range of the preset compression algorithm is greater than or equal to 1 / 12 and less than 1 / 1.

[0120] The preset compression algorithm can be a lossy compression algorithm, that is, the secondary information data is compressed away, some quality is sacrificed to reduce the data volume, and the compression ratio is improved. Specifically, it can be a lossy compression mechanism using lossy transform coding. First, the image or sound is sampled, cut into small pieces, transformed into a new space, and quantized, and then the quantized values are encoded to complete image compression.

[0121] The compressed image can be a compressed color image obtained by compressing the color image using a preset compression algorithm. The compressed image occupies a memory size of at least 1 / 12 of the original image and at most the same memory size as the original image. The compression degree is determined by the size of the original image. If the original image occupies a large memory, the compression degree is large. If the original image occupies a small memory, the compression degree is small. The compression degree can be preset in the compression algorithm. For example, it can be specified that a picture with a size greater than 1MB (MByte) needs to be compressed to 1MB or less. A picture with a size of 1MB or less does not need to be compressed.

[0122] The compression can be a process of compressing the color image using a preset compression algorithm, and the compression range can be determined by the preset compression algorithm and the preset compression degree.

[0123] In S203, the face coordinates of the monitoring object in the compressed image of the color image are identified by a preset face recognition algorithm.

[0124] The identification can be a process of face recognition using a preset face algorithm based on the compressed color image (i.e. the compressed image).

[0125] In S204, the face coordinates of the monitoring object in the infrared image are determined according to the resolution ratio conversion relationship between the infrared temperature measurement camera and the color camera.

[0126] In S205, the feature region is determined according to the face coordinates of the monitoring object in the infrared image.

[0127] In S206, the average temperature of the feature region is identified as the body temperature monitoring result of the monitoring object.

[0128] In this embodiment, the color image is compressed by using a compression algorithm before face recognition. The traditional temperature detection method checks a too large range and does not perform face recognition. False positives will occur as long as the user carries a high-temperature object, and the method is greatly affected by the environment. The monitoring method of the infrared camera has a requirement that the number of people cannot be too large when applied, because the performance of the camera chip is poor, resulting in poor recognition speed and effect. The present scheme improves the efficiency of face recognition to some extent, and finally exhibits good body temperature synchronous monitoring effect.

[0129] Embodiment Three

[0130] Figure 3 is a structural schematic diagram of a body temperature monitoring device based on a robot provided by the third embodiment of the present application.

[0131] As shown in Figure 3 , it specifically includes the following:

[0132] The acquisition module 301 is configured to acquire a color image including the monitoring object by using a color camera, and acquire an infrared image including the monitoring object by using an infrared temperature measurement camera.

[0133] The recognition module 302 is configured to recognize the face coordinates of the monitoring object in the color image by using a preset face recognition algorithm.

[0134] The coordinate determination module 303 is configured to determine the face coordinates of the monitoring object in the infrared image according to a resolution ratio conversion relationship between the infrared temperature measurement camera and the color camera.

[0135] The feature region determination module 304 is configured to determine a feature region according to the face coordinates of the monitoring object in the infrared image.

[0136] The monitoring module 305 is configured to recognize a temperature average value of the feature region as a body temperature monitoring result of the monitoring object.

[0137] In the embodiment of the present application, the acquisition module is configured to acquire a color image including the monitoring object by using a color camera, and acquire an infrared image including the monitoring object by using an infrared temperature measurement camera; the recognition module is configured to recognize the face coordinates of the monitoring object in the color image by using a preset face recognition algorithm; the coordinate determination module is configured to determine the face coordinates of the monitoring object in the infrared image according to a resolution ratio conversion relationship between the infrared temperature measurement camera and the color camera; the feature region determination module is configured to determine a feature region according to the face coordinates of the monitoring object in the infrared image; and the recognition module is configured to recognize a temperature average value of the feature region as a body temperature monitoring result of the monitoring object. Through the above-mentioned robot-based body temperature monitoring device, the human body temperature can be flexibly measured; and by recognizing the face and then recognizing the feature region, and taking the average temperature of the feature region as the body temperature monitoring result, the measurement result can be more accurate, the influence of high-temperature objects can be avoided, and the probability of unnecessary manual intervention due to calculation errors can be reduced.

[0138] The robot-based body temperature monitoring device provided in the embodiment of the present application can realize Figures 1 to 2 The method embodiment realizes each process, to avoid repetition, here will not repeat.

[0139] Embodiment four

[0140] As Figure 4 shown, the embodiment of the present application also provides an electronic device 400, which includes a processor 401, a memory 402, a program or instruction stored in the memory 402 and executable on the processor 401, the program or instruction is executed by the processor 401 to realize each process of the above-mentioned robot-based body temperature monitoring method embodiment, and can achieve the same technical effect, to avoid repetition, here will not repeat.

[0141] It should be noted that the electronic device in the embodiments of the present application includes the mobile electronic device and the non-mobile electronic device described above.

[0142] Embodiment five

[0143] The embodiments of the present application also provide a readable storage medium, which stores a program or instructions, and the program or instructions are executed by a processor to realize the processes of the above-mentioned robot-based body temperature monitoring method embodiments and achieve the same technical effects. To avoid repetition, details are not described herein.

[0144] The processor is the processor in the electronic device described in the above-mentioned embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, etc.

[0145] Embodiment six

[0146] The embodiments of the present application also provide a chip, which includes a processor and a communication interface, the communication interface is coupled with the processor, and the processor is used to run a program or instructions to realize the processes of the above-mentioned robot-based body temperature monitoring method embodiments and achieve the same technical effects. To avoid repetition, details are not described herein.

[0147] It should be understood that the chip mentioned in the embodiments of the present application can also be referred to as a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0148] It should be noted that in this document, the term "comprising" or "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to the order of performing the functions shown or discussed, but can also include performing the functions in a substantially simultaneous manner or in a reverse order, for example, the described method can be performed in an order different from that described, and various steps can be added, omitted or combined. In addition, the features described with reference to certain examples can be combined in other examples.

[0149] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned example method can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product in essence or in the form of a computer software product that contributes to the prior art. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disc, optical disc), and includes a plurality of instructions for making a terminal (which can be a mobile phone, computer, server, or network device, etc.) execute the method described in each embodiment of the present application.

[0150] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, and the above-mentioned specific embodiments are only illustrative and not restrictive. Those skilled in the art can make many forms without departing from the scope of the present application and the protection scope of the claims under the inspiration of the present application, which all belong to the protection scope of the present application.

[0151] The above are only the preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions made by those skilled in the art without departing from the scope of the present application and the protection scope of the claims also belong to the protection scope of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments, and more other equivalent embodiments can be included without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A robot-based method for body temperature monitoring, characterized in that, The method is performed by a robot, which includes a photographic device, including a color camera and an infrared temperature measurement camera. The method includes interacting with a backend scheduling system, which is used to control the robot; the method includes: A color image of the monitored object is acquired using a color camera, and an infrared image of the monitored object is acquired using an infrared thermometer camera. The facial coordinates of the monitored object in the color image are identified using a preset facial recognition algorithm. Based on the resolution ratio conversion between the infrared temperature measurement camera and the color camera, the facial coordinates of the monitored object in the infrared image are determined. Based on the facial coordinates of the monitored object in the infrared image, a feature region is determined, which includes: determining the facial angle information of the monitored object based on the facial coordinates of the monitored object in the infrared image and through the preset facial recognition algorithm, and determining the feature region in the facial coordinates based on the facial angle information; The average temperature of the characteristic region is identified as the body temperature monitoring result of the monitored object.

2. The method according to claim 1, characterized in that, Before identifying the facial coordinates of the monitored object in the color image using a preset facial recognition algorithm, the method further includes: A color image is compressed using a preset compression algorithm to obtain a compressed image; wherein the compression range of the preset compression algorithm is greater than or equal to 1 / 12 and less than 1 / 1. Accordingly, the facial coordinates of the monitored object in the color image are identified using a preset facial recognition algorithm, including: The facial coordinates of the monitored object are identified in the compressed image of the color image using a preset facial recognition algorithm.

3. The method according to claim 1, characterized in that, The feature area includes a circular area with a diameter of 1 cm at the center of the forehead of the monitored object.

4. The method according to claim 1, characterized in that, Identifying the average temperature of the characteristic region as the body temperature monitoring result of the monitored object includes: If the average temperature is within the preset normal temperature range, then the average temperature is identified as either corresponding to the low temperature normal range or the high temperature warning range.

5. The method according to claim 1, characterized in that, Identifying the average temperature of the characteristic region as the body temperature monitoring result of the monitored object includes: If the average temperature is not within the preset normal temperature range, a filter instruction is generated to perform a filter operation on the current color image and the infrared image. The next set of color and infrared images acquired by the color camera and the infrared temperature measurement camera are used for body temperature monitoring.

6. The method according to claim 4, characterized in that, After identifying whether the average temperature falls within a preset normal temperature range, and whether the average temperature corresponds to a normal low-temperature range or a high-temperature warning range, the method further includes: If the average temperature corresponds to the high temperature warning range, it is determined to be an abnormal body temperature; Obtain the robot's real-time coordinates as the real-time location where abnormal body temperature was detected; The real-time location is reported to the back-end dispatch system so that the back-end dispatch system can record and process abnormal body temperatures.

7. A robot-based body temperature monitoring device, characterized in that, The device is configured on a robot, and the robot includes a photographic device, which includes a color camera and an infrared temperature measurement camera. It interacts with a background scheduling system, which is used to control the robot; the device includes: The acquisition module is used to acquire color images of the monitored object using a color camera, and to acquire infrared images of the monitored object using an infrared thermometer camera. The recognition module is used to identify the facial coordinates of the monitored object in the color image using a preset facial recognition algorithm; The coordinate determination module is used to determine the facial coordinates of the monitored object in the infrared image based on the resolution ratio conversion relationship between the infrared temperature measurement camera and the color camera. The feature region determination module is used to determine the feature region based on the face coordinates of the monitored object in the infrared image, wherein the module includes: determining the face angle information of the monitored object based on the face coordinates of the monitored object in the infrared image and through the preset face recognition algorithm, and determining the feature region in the face coordinates based on the face angle information; The monitoring module is used to identify the average temperature of the characteristic area as the body temperature monitoring result of the monitored object.

8. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the robot-based body temperature monitoring method as described in any one of claims 1-6.

9. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the robot-based body temperature monitoring method as described in any one of claims 1-6.

Images