An intelligent monitoring method for a rehabilitation accompanying robot

Abstract

The present invention relates to the field of image data processing technology, and in particular to an intelligent monitoring method for a rehabilitation accompanying robot, comprising: obtaining a grayscale image of an accompanying object and a target area of ​​the accompanying object in the grayscale image of the accompanying object. Calculating the target motion amplitude based on the area of ​​the accompanying object target area and the difference in pixel points on the boundary, and comparing the difference in the target motion amplitude of the accompanying object grayscale images of adjacent frames to obtain the target motion risk coefficient. Constructing a target surrounding area based on the distance between the center of mass of the accompanying object target area and the pixel points on its boundary, and calculating the target collision risk coefficient of each frame of the accompanying object grayscale image based on the corner points of the target surrounding area. Combining the target collision risk coefficient and the target motion risk coefficient, a target care factor is obtained, and then an enhanced image is generated and identified. The present invention improves the clarity of the accompanying object grayscale image by performing feature enhancement on the accompanying object grayscale image, thereby improving the accuracy of recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to an intelligent monitoring method for a rehabilitation accompanying robot. Background Art

[0002] Intelligent monitoring methods for rehabilitation accompanying robots can be achieved by combining artificial intelligence technology and human-computer interaction design. The robot can identify the user's emotions by analyzing facial expressions and body movements, thereby better understanding the user's needs and status. Therefore, it is necessary to use rehabilitation accompanying robots to obtain images of the accompanying elderly and identify the elderly person. However, due to the influence of lighting, the recognition results may be misjudged.

[0003] Existing problem: When the rehabilitation accompanying robot acquires images of the elderly person it is accompanying, there may be different changes in the lighting in the elderly person's room, causing the elderly person's part to be darker. As a result, when the elderly person's facial expressions, body movements, etc. are subsequently analyzed, there will be a low image contrast and unclear image due to the dim light, which may cause recognition errors, thereby causing the rehabilitation accompanying robot to misjudge the dangerous behavior of the accompanying person. Summary of the Invention

[0004] The present invention provides an intelligent monitoring method for a rehabilitation accompanying robot to solve the existing problems.

[0005] The intelligent monitoring method of a rehabilitation accompanying robot of the present invention adopts the following technical solutions:

[0006] One embodiment of the present invention provides an intelligent monitoring method for a rehabilitation accompanying robot, the method comprising the following steps:

[0007] Acquire a plurality of frames of grayscale images of the accompanying object and a target area of ​​the accompanying object in the grayscale images of the accompanying object;

[0008] Obtaining the target motion amplitude of each frame of the accompanying object grayscale image according to the area of ​​the accompanying object target region and the position difference of all pixel points on the boundary of each frame of the accompanying object grayscale image;

[0009] Obtaining a target action risk coefficient for each frame of the accompanying object grayscale image according to a difference in target action amplitude between each frame of the accompanying object grayscale image and the next frame of the accompanying object grayscale image;

[0010] Constructing a target surrounding area corresponding to each frame of the accompanying object grayscale image according to the distance between the centroid of the accompanying object target area of ​​each frame of the accompanying object grayscale image and the pixel points on the boundary of the accompanying object target area;

[0011] Obtaining a number of corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image; obtaining a target collision risk coefficient for each frame of the accompanying object grayscale image based on the difference in distances between all corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image and the center of mass of the accompanying object target area;

[0012] According to the target collision risk coefficient and target action risk coefficient of each frame of the accompanying object grayscale image, the target attention factor of each frame of the accompanying object grayscale image is obtained; using the target attention factor of each frame of the accompanying object grayscale image as the contrast gain value, the target surrounding area corresponding to each frame of the accompanying object grayscale image is linearly enhanced to obtain an enhanced accompanying object grayscale image; using the trained convolutional neural network to judge the dangerous behavior of the accompanying object in each frame of the enhanced accompanying object grayscale image, the dangerous behavior judgment result is obtained.

[0013] Furthermore, the target motion amplitude of each frame of the grayscale image of the accompanying object is obtained based on the area of ​​the target region of the accompanying object and the position difference of all pixels on the boundary of each frame of the grayscale image of the accompanying object, including the following specific steps:

[0014] Using contour detection technology to calculate the target area of ​​the accompanying object in each frame of the accompanying object grayscale image, to obtain the minimum circumscribed rectangle of the target area of ​​the accompanying object in each frame of the accompanying object grayscale image;

[0015] According to The minimum bounding rectangle area of ​​the accompanying object target area of ​​the frame accompanying object grayscale image and the position difference of all pixels on the boundary are obtained. The frame accompanies the target motion amplitude of the object grayscale image.

[0016] Furthermore, according to The minimum bounding rectangle area of ​​the accompanying object target area of ​​the frame accompanying object grayscale image and the position difference of all pixels on the boundary are obtained. The specific calculation formula for the target motion amplitude of the frame accompanying the object grayscale image is:

[0017]

[0018] in, Indicates the The target motion amplitude of the frame accompanying the grayscale image of the object, Indicates the The first frame of the accompanying object gray image is on the boundary of the accompanying object target area The distance from the pixel point to the centroid of the target area of ​​the accompanying object; Indicates the The first frame of the accompanying object gray image is on the boundary of the accompanying object target area The distance from the pixel point to the centroid of the target area of ​​the accompanying object; Indicates the The number of pixels on the boundary of the escort object target area of ​​the frame escort object grayscale image; The area of ​​the minimum circumscribed rectangle of the target area of ​​the accompanying object in the grayscale image of the accompanying object; Indicates the The area of ​​the target region of the escorted object in the frame escorted object grayscale image; represents the hyperbolic tangent function; express Belong to the interval All internal time The variance of .

[0019] Furthermore, the target action risk coefficient of each frame of the accompanying object grayscale image is obtained based on the difference between the target action amplitude of each frame of the accompanying object grayscale image and the next frame of the accompanying object grayscale image, including the following specific steps:

[0020] Obtaining a centroid interval distance between grayscale images of the accompanying object in each frame according to the position of the centroid of the accompanying object target area in the grayscale images of the accompanying object in adjacent frames;

[0021] According to The grayscale image of the accompanying object and the The difference in the target motion amplitude of the grayscale image of the frame accompanying the object and the The centroid distance of the grayscale image of the frame accompanying the object is obtained. The target action risk factor of the frame accompanying the grayscale image of the object.

[0022] Furthermore, the method of obtaining the centroid interval distance of each frame of the accompanying object grayscale image according to the centroid position of the accompanying object target area of ​​the accompanying object grayscale image of adjacent frames includes the following specific steps:

[0023] In each frame of the grayscale image of the accompanying object, a coordinate system is obtained by taking the vertex of the upper left corner of each frame of the grayscale image of the accompanying object as the origin, horizontally to the right as the positive direction of the horizontal axis, and vertically downward as the positive direction of the vertical axis;

[0024] In the coordinate system, The coordinates of the centroid of the target area of ​​the accompanying object in the frame accompanying object grayscale image are the same as the coordinates of the centroid of the target area of ​​the frame accompanying object grayscale image The Euclidean distance between the coordinates of the centroid of the target area of ​​the accompanying object in the grayscale image of the accompanying object is recorded as The distance between the centroids of the grayscale images of the frame accompanying the object.

[0025] Furthermore, according to The grayscale image of the accompanying object and the The difference in the target motion amplitude of the grayscale image of the frame accompanying the object and the The centroid distance of the grayscale image of the frame accompanying the object is obtained. The specific calculation formula for the target action risk coefficient of the frame escort object grayscale image is:

[0026]

[0027] in, No. Target action risk factor of the grayscale image of the frame escort object; Indicates the The centroid distance between the grayscale images of the frame accompanying the object; Indicates the The target motion amplitude of the frame chaperone object grayscale image; Indicates the The target motion amplitude of the frame chaperone object grayscale image; represents the hyperbolic tangent function.

[0028] Furthermore, the target surrounding area corresponding to each frame of the accompanying object grayscale image is constructed according to the distance between the centroid of the accompanying object target area of ​​each frame of the accompanying object grayscale image and the pixel points on the boundary of the accompanying object target area, including the following specific steps:

[0029] Will be The center of mass of the target area of ​​the accompanying object in the grayscale image of the accompanying object is the center of the circle, and the maximum value of the distance from all pixels on the boundary of the accompanying object target area to the center of the circle is added to The sum of the values ​​of the circle with radius is called The frame accompanies the target surrounding area corresponding to the grayscale image of the object; wherein, The preset threshold length.

[0030] Furthermore, obtaining the target collision risk coefficient of each frame of the accompanying object grayscale image according to the difference in distances between all corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image and the centroid of the accompanying object target area includes the following specific steps:

[0031] The first The first frame of the target surrounding area corresponding to the grayscale image of the object Corner point to The distance between the centroid of the target area of ​​the accompanying object in the frame accompanying the object grayscale image is recorded as The grayscale image of the frame accompanying the object The distance between the corner points;

[0032] According to The grayscale image of the accompanying object and the The difference in the target motion amplitude of the grayscale image of the frame accompanying the object and the The distance between all corner points of the grayscale image of the frame accompanying the object is obtained. Target collision risk coefficient of the frame accompanying the grayscale image of the object.

[0033] Furthermore, according to The grayscale image of the accompanying object and the The difference in the target motion amplitude of the grayscale image of the frame accompanying the object and the The distance between all corner points of the grayscale image of the frame accompanying the object is obtained. The specific calculation formula for the target collision risk coefficient of the grayscale image of the frame escort object is:

[0034]

[0035] in, Indicates the Target collision risk coefficient of the grayscale image of the frame escort object; Indicates the The number of corner points in the target surrounding area corresponding to the grayscale image of the frame accompanying the object; Indicates the The grayscale image of the frame accompanying the object The distance between the corner points; Represents an exponential function with a natural constant as its base.

[0036] Furthermore, the target care factor of each frame of the accompanying object grayscale image is obtained according to the target collision risk coefficient and the target action risk coefficient of each frame of the accompanying object grayscale image, including the following specific steps:

[0037] The first The product of the target collision risk coefficient and the target action risk coefficient of the grayscale image of the frame escort object is recorded as The target illumination factor of the grayscale image of the frame accompanying the object.

[0038] The beneficial effects of the technical solution of the present invention are:

[0039] Several frames of grayscale images of the accompanying person and the accompanying person's target area in the accompanying person grayscale images are obtained; based on the area of ​​the accompanying person's target area in each frame of the accompanying person grayscale image and the position differences of all pixels on the boundary, the target movement amplitude of each frame of the accompanying person grayscale image is obtained, thereby improving the accuracy of determining whether the accompanying person is in a dangerous situation. Based on the difference in the target movement amplitude between each frame of the accompanying person grayscale image and the next frame of the accompanying person grayscale image, the target movement risk factor of each frame of the accompanying person grayscale image is obtained, further improving the accuracy of determining whether the accompanying person is in a dangerous situation.

[0040] Based on the distance from the centroid of the target region of each grayscale image to the pixel points on the boundary of the target region, the target surrounding region corresponding to each grayscale image of the accompanying object is constructed. Several corner points within the target surrounding region corresponding to each grayscale image of the accompanying object are obtained. The target collision risk coefficient for each grayscale image of the accompanying object is determined based on the difference in distances between all corner points within the target surrounding region and the centroid of the target region, accurately reflecting the surrounding environment of the accompanying object. Based on the target collision risk coefficient and target action risk coefficient of each grayscale image of the accompanying object, the target attention factor of each grayscale image of the accompanying object is determined, improving the accuracy of the recognition results. Using the target attention factor of each grayscale image of the accompanying object as the contrast gain value, the target surrounding region corresponding to each grayscale image of the accompanying object is linearly enhanced to obtain an enhanced grayscale image of the accompanying object. The trained convolutional neural network is used to determine the dangerous behavior of the accompanying object in each enhanced grayscale image of the accompanying object, obtaining a dangerous behavior judgment result. The present invention enhances the features of the grayscale image of the accompanying object to improve the clarity of the grayscale image of the accompanying object, thereby improving the accuracy of recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 The present invention provides a flowchart of the steps of an intelligent monitoring method for a rehabilitation accompanying robot. DETAILED DESCRIPTION

[0043] To further illustrate the technical means and effectiveness of the present invention in achieving its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effectiveness of an intelligent monitoring method for a rehabilitation companion robot proposed in accordance with the present invention. In the following description, references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0044] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0045] The specific scheme of the intelligent monitoring method of a rehabilitation accompanying robot provided by the present invention is described in detail below with reference to the accompanying drawings.

[0046] See also Figure 1 , which shows a flowchart of a method for intelligent monitoring of a rehabilitation accompanying robot provided by one embodiment of the present invention, the method comprising the following steps:

[0047] Step S001: Acquire several frames of grayscale images of a person being cared for and target areas of the person being cared for in the grayscale images of the person being cared for.

[0048] First, several frames of accompanying object pictures are captured from the video images of human activities in a poorly lit room, and then each frame of the accompanying object picture is grayscale processed to obtain several frames of accompanying object grayscale images.

[0049] What needs to be explained is that: the rehabilitation accompanying robot is equipped with a target tracking system, which will track and collect the image information of the accompanying object in real time, and monitor and identify the sitting, lying and walking postures of the accompanying object through image data. The existing method is to realize human posture recognition by matching the changes of key nodes of human limbs in continuous frames with preset normal postures, and to obtain expression information through key points of the face, and combine with vital signs such as heart rate and blood pressure to relatively comprehensively evaluate the daily physical and psychological condition of the accompanying object. I will not explain too much about the existing technology; but all this needs to be based on reliable image data collection. Taking into account the actual room orientation, house type, weather, and time, some poor lighting will be caused to the collected image. This problem is common and greatly limits the practicality of the rehabilitation accompanying robot, and has a great impact on the monitoring results of the rehabilitation accompanying robot. Therefore, it is necessary to capture the picture of the accompanying object from the video image of the human activities in the room with poor lighting.

[0050] The embodiment of the present invention adopts a segmentation neural network to identify the target area and background area of ​​the accompanying object in each frame of the accompanying object grayscale image.

[0051] The relevant content of the segmentation neural network is as follows:

[0052] The segmentation neural network used in this example is the Mask R-CNN neural network, and the dataset used is a dataset of grayscale images of caregivers. Mask R-CNN is a well-known technique, and its specific method will not be described here. Mask R-CNN stands for "Mask Region-based Convolutional Neural Network" in Chinese and "Mask Region-based Convolutional Neural Network" in English.

[0053] The pixels that need to be segmented are divided into two categories, that is, the labeling process of the training set is: single-channel semantic label, the corresponding position pixel belongs to the background area is labeled as 0, and the corresponding position pixel belongs to the target area of ​​the accompanying object is labeled as 1.

[0054] The task of the network is classification, so the loss function used is the cross entropy loss function.

[0055] The target area and background area of ​​the escorted object in each frame of the escorted object grayscale image are obtained by segmentation neural network. This process is a well-known technology and the specific method will not be introduced here.

[0056] Through the above process, the target area of ​​the accompanying object in each frame of the accompanying object grayscale image is obtained.

[0057] Step S002: obtaining a target motion amplitude of each frame of the accompanying object grayscale image according to the area of ​​the accompanying object target region and the position difference of all pixels on the boundary of each frame of the accompanying object grayscale image;

[0058] It should be noted that: in order to determine whether the grayscale image of the current escort object needs to be enhanced, it is necessary to make a judgment based on the obtained target area of ​​the escort object. Since the target area of ​​the escort object obtained by the above operation may be only a rough connected domain due to insufficient lighting and other reasons, it is necessary to enhance the grayscale image of the escort object to facilitate the escort robot to analyze whether the escort object has dangerous behavior.

[0059] It is further necessary to explain that: when the accompanying object makes a larger movement, the possibility of him / her falling or other dangerous behaviors is greater; therefore, it is necessary to analyze the edge features of the obtained accompanying object target area, wherein when the accompanying object makes a larger movement, the arms and legs undergo large changes, so the acquired accompanying object target area has a large convex part; and compare the changes in the moving distance and movement amplitude of the accompanying object between adjacent image frames.

[0060] The contour detection technology is used to calculate the target area of ​​the accompanying object in each frame of the accompanying object grayscale image to obtain the minimum bounding rectangle of the accompanying object target area in each frame of the accompanying object grayscale image. Among them, the contour detection technology is a well-known technology and the specific method is not introduced here.

[0061] First Taking the grayscale image of the accompanying object as an example, the first The specific calculation formula for the target motion amplitude of the frame escort object grayscale image is:

[0062]

[0063] in, Indicates the The target motion amplitude of the frame accompanying the grayscale image of the object, Indicates the The first frame of the accompanying object gray image is on the boundary of the accompanying object target area The distance from the pixel point to the centroid of the target area of ​​the accompanying object; Indicates the The first frame of the accompanying object gray image is on the boundary of the accompanying object target area The distance from the pixel point to the centroid of the target area of ​​the accompanying object; Indicates the The number of pixels on the boundary of the escort object target area of ​​the frame escort object grayscale image; The area of ​​the minimum circumscribed rectangle of the target area of ​​the accompanying object in the grayscale image of the accompanying object; Indicates the The area of ​​the target region of the escorted object in the frame escorted object grayscale image; represents the hyperbolic tangent function; express Belong to the interval All internal time The variance of .

[0064] It should be noted that when the accompanying object makes a large movement, its limbs are stretched outward, so the variance of the distance between the pixel point on the boundary of the accompanying object target area of ​​the accompanying object grayscale image and the centroid of the accompanying object target area is The larger the value, the larger the convex part of the target area of ​​the escort object, so the larger the target movement amplitude is. There is a positive correlation; and the area difference between the minimum enclosing rectangle of the escort target area and the escort target area , the larger the value, the larger and more bulges there are in the target area of ​​the accompanying object in the grayscale image of the accompanying object, which indicates that the target movement amplitude is larger, and the There is a positive correlation.

[0065] Through the above process, we obtained The target motion amplitude of the frame chaperone object grayscale image;

[0066] Each frame of the accompanying object grayscale image is operated according to the above process to obtain the target motion amplitude of each frame of the accompanying object grayscale image.

[0067] Step S003: obtaining a target action risk coefficient of each frame of the accompanying object grayscale image according to a difference in target action amplitude between each frame of the accompanying object grayscale image and the next frame of the accompanying object grayscale image.

[0068] What needs to be explained is that for the target motion amplitude of each frame of the grayscale image of the accompanying object, when the accompanying object finds that it has fallen or other dangerous actions, the target motion amplitude will change greatly and its position will also change greatly. Therefore, it is necessary to analyze and compare the changes in the target motion amplitude of the previous and next frames and the changes in the position of the accompanying object.

[0069] In each frame of the grayscale image of the accompanying object, a coordinate system is obtained by taking the vertex at the upper left corner of each frame of the grayscale image of the accompanying object as the origin, horizontally to the right as the positive direction of the horizontal axis, and vertically downward as the positive direction of the vertical axis.

[0070] In the coordinate system, The coordinates of the centroid of the target area of ​​the accompanying object in the frame accompanying object grayscale image are the same as the coordinates of the centroid of the target area of ​​the frame accompanying object grayscale image The Euclidean distance between the coordinates of the centroid of the target area of ​​the accompanying object in the grayscale image of the accompanying object is recorded as The distance between the centroids of the grayscale images of the frame accompanying the object.

[0071] It should be noted that the distance between the centroids of the grayscale images of the last frame of the accompanying object is the same as the distance between the centroids of the grayscale images of the previous frame of the accompanying object.

[0072] Rule No. The specific calculation formula for the target action risk coefficient of the grayscale image of the frame escort object is:

[0073]

[0074] in, No. Target action risk factor of the grayscale image of the frame escort object; Indicates the The centroid distance between the grayscale images of the frame accompanying the object; Indicates the The target motion amplitude of the frame chaperone object grayscale image; Indicates the The target motion amplitude of the frame chaperone object grayscale image; represents the hyperbolic tangent function.

[0075] It should be noted that the target action risk coefficient of the last frame of the accompanying object grayscale image is the same as the target action risk coefficient of the previous frame of the accompanying object grayscale image.

[0076] It is further explained that: when the accompanying person performs some dangerous actions, such as falling, his body will change greatly in a short period of time and his position will move. The centroid distance between the grayscale images of the frame accompanying the object The larger the value, the greater the displacement of the escort object in a short period of time, indicating that the possibility of danger is higher, and the higher the risk coefficient of the action is. There is a positive correlation; and for The difference between the target motion amplitude of the grayscale image of the accompanying object in one frame and the grayscale image of the accompanying object in the next frame The larger the value, the greater the change in the current escort object's movement amplitude is, and the more likely it is to be dangerous. Therefore, the higher the risk coefficient of the action, the more likely it is to be dangerous. There is a positive correlation.

[0077] Through the above process, we obtained Target action risk factor of the grayscale image of the frame escort object;

[0078] Each frame of the accompanying object grayscale image is calculated according to the above process to obtain the target action risk coefficient of each frame of the accompanying object grayscale image.

[0079] Step S004: constructing the target surrounding area corresponding to each frame of the accompanying object grayscale image according to the distance between the centroid of the accompanying object target area in each frame of the accompanying object grayscale image and the pixel points on the boundary of the accompanying object target area.

[0080] It should be noted that: for the acquired grayscale image of the accompanying person, in addition to analyzing whether there is any dangerous action, it is also necessary to analyze whether there is a risk of collision in the environment where the accompanying person is located.

[0081] Still with the first For example, the grayscale image of the accompanying object is The center of mass of the target area of ​​the accompanying object in the grayscale image of the accompanying object is the center of the circle, and the maximum value of the distance from all pixels on the boundary of the accompanying object target area to the center of the circle is added to The sum of the values ​​of the circle with radius is called The frame accompanies the grayscale image of the target area. The preset threshold length is The value is 20, and this is used as an example for description.

[0082] Each frame of the accompanying object grayscale image is operated according to the above process to obtain the target surrounding area corresponding to each frame of the accompanying object grayscale image.

[0083] Step S005: obtaining a number of corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image; obtaining the target collision risk coefficient of each frame of the accompanying object grayscale image based on the difference in distances between all corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image and the centroid of the accompanying object target area;

[0084] What needs to be explained is that: for the acquired target surrounding area, it is necessary to analyze whether the accompanying person is at risk of collision in the target surrounding area. Collisions often occur because furniture such as tables and chairs have sharp corners. Therefore, it is necessary to perform corner point detection on the target surrounding area. If there are many corner points around the accompanying person, it indicates that the current accompanying person may be in an environment where collisions are more likely to occur.

[0085] A corner detection algorithm is used to calculate the target surrounding area corresponding to each frame of the grayscale image of the accompanying object, thereby obtaining a number of corner points within the target surrounding area corresponding to each frame of the grayscale image of the accompanying object. The corner detection algorithm is a well-known technique, and the specific method will not be described here.

[0086] The first The first frame of the target surrounding area corresponding to the grayscale image of the object Corner point to The distance between the centroid of the target area of ​​the accompanying object in the frame accompanying the object grayscale image is recorded as The grayscale image of the frame accompanying the object The distance between the corner points.

[0087] Rule No. The specific calculation formula for the target collision risk coefficient of the grayscale image of the frame escort object is:

[0088]

[0089] in, Indicates the Target collision risk coefficient of the grayscale image of the frame escort object; Indicates the The number of corner points in the target surrounding area corresponding to the grayscale image of the frame accompanying the object; Indicates the The grayscale image of the frame accompanying the object The distance between the corner points; Represents an exponential function with a natural constant as its base.

[0090] It should be noted that: since it is necessary to detect whether there is a risk of collision around the escorted object, the corner points of the target surrounding area corresponding to the grayscale image of the escorted object in the i-th frame are compared with the corner points of the grayscale image of the escorted object in the i-th frame. The average of the distance and the centroid of the target area of ​​the accompanying object in the frame accompanying object grayscale image is obtained The smaller the value, the closer the corner point of the target area is to the escort object, so the target collision risk coefficient is There is a negative correlation, and for The number of corner points in the target area corresponding to the grayscale image of the frame accompanying the object , The larger the value, the higher the risk of collision around the person being cared for. and There is a positive correlation.

[0091] Through the above process, the target collision risk coefficient of the grayscale image of the escort object in the i-th frame is obtained;

[0092] Each frame of the accompanying object grayscale image is calculated according to the above process to obtain the target collision risk coefficient of each frame of the accompanying object grayscale image.

[0093] Step S006: According to the target collision risk coefficient and the target action risk coefficient of each frame of the accompanying object grayscale image, the target care factor of each frame of the accompanying object grayscale image is obtained; the target care factor of each frame of the accompanying object grayscale image is used as the contrast gain value, and the target surrounding area corresponding to each frame of the accompanying object grayscale image is linearly enhanced to obtain an enhanced accompanying object grayscale image; the trained convolutional neural network is used to judge the dangerous behavior of the accompanying object in each frame of the enhanced accompanying object grayscale image to obtain a dangerous behavior judgment result.

[0094] It should be noted that: when the escort object makes dangerous movements or may collide, the escort robot needs to analyze the state of the escort object and take care of it, so the target care factor of the escort object's grayscale image is obtained.

[0095] The first The product of the target collision risk coefficient and the target action risk coefficient of the grayscale image of the frame escort object is recorded as The target illumination factor of the grayscale image of the frame accompanying the object.

[0096] Each frame of the accompanying object grayscale image is operated according to the above process to obtain the target care factor of each frame of the accompanying object grayscale image.

[0097] It should be noted that: in order for the accompanying robot to subsequently analyze the state of the accompanying object, each frame of the accompanying object grayscale image needs to be enhanced according to the target attention factor of each frame of the accompanying object grayscale image.

[0098] In each frame of the accompanying object grayscale image, obtain the gradient histogram of the target surrounding area of ​​each frame of the accompanying object grayscale image, multiply each gradient value on the gradient histogram by the target illumination factor of each frame of the accompanying object grayscale image plus The sum of the values ​​of is used to obtain the enhanced grayscale image of the accompanying object corresponding to each frame of the accompanying object grayscale image. The process of obtaining the gradient histogram of the target surrounding area of ​​each frame of the accompanying object grayscale image is a well-known technique, and the specific method will not be described here. is the preset threshold gradient. The threshold gradient preset in this embodiment is The value is 1, and this is used as an example for description.

[0099] It should be noted that the above-mentioned enhancement operation is a linear transformation of the gradient histogram of the image, which is a well-known technology. The target illumination factor of each frame of the grayscale image of the accompanying object reflects the contrast gain value in the linear transformation. When the target illumination factor is larger, it means that the image is more important. Therefore, a larger contrast gain value is assigned to improve the image contrast and make the image clear.

[0100] The trained convolutional neural network is used to judge the dangerous behavior of the accompanying object in each frame of the enhanced accompanying object grayscale image to obtain the dangerous behavior judgment result.

[0101] It should be noted that the input of the convolutional neural network in this embodiment is the enhanced grayscale image of the accompanying object in each frame, and the output is the dangerous behavior judgment result, that is, whether the accompanying object's behavior is normal or dangerous. The convolutional neural network model is ResNet500, which is used as an example for description. The dataset is composed of all frames of enhanced grayscale images of the accompanying object. Whether the accompanying object's behavior is normal in each frame of enhanced grayscale image of the accompanying object is used as the sample label, such as normal accompanying object behavior is marked as 1, and dangerous accompanying object behavior is marked as 2. The neural network is trained using the above-obtained dataset, and the loss function used is the cross-entropy loss function. The specific training process is well known and will not be described in detail in this embodiment.

[0102] So far, the present invention is completed.

[0103] In summary, in an embodiment of the present invention, a grayscale image of the accompanying object and a target area of ​​the accompanying object in the grayscale image of the accompanying object are obtained. The target motion amplitude is calculated based on the difference between the area of ​​the target area of ​​the accompanying object and the pixel points on the boundary, and the difference in the target motion amplitude of the grayscale images of the accompanying object in adjacent frames is compared to obtain the target motion risk coefficient. According to the distance between the center of mass of the target area of ​​the accompanying object and the pixel points on its boundary, a target surrounding area is constructed, and according to the corner points of the target surrounding area, the target collision risk coefficient of each frame of the grayscale image of the accompanying object is calculated. Combining the target collision risk coefficient and the target motion risk coefficient, the target attention factor is obtained, and then an enhanced image is generated and identified. The present invention improves the clarity of the grayscale image of the accompanying object by performing feature enhancement on the grayscale image of the accompanying object, thereby improving the accuracy of recognition.

[0104] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An intelligent monitoring method for a rehabilitation accompanying robot, characterized in that: The method comprises the following steps: Acquire a plurality of frames of grayscale images of the accompanying object and a target area of ​​the accompanying object in the grayscale images of the accompanying object; Obtaining the target motion amplitude of each frame of the accompanying object grayscale image according to the area of ​​the accompanying object target region and the position difference of all pixel points on the boundary of each frame of the accompanying object grayscale image; Obtaining a target action risk coefficient for each frame of the accompanying object grayscale image according to a difference in target action amplitude between each frame of the accompanying object grayscale image and the next frame of the accompanying object grayscale image; Constructing a target surrounding area corresponding to each frame of the accompanying object grayscale image according to the distance between the centroid of the accompanying object target area of ​​each frame of the accompanying object grayscale image and the pixel points on the boundary of the accompanying object target area; Obtaining a number of corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image; obtaining a target collision risk coefficient for each frame of the accompanying object grayscale image based on the difference in distances between all corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image and the center of mass of the accompanying object target area; According to the target collision risk coefficient and the target action risk coefficient of each frame of the accompanying object grayscale image, the target attention factor of each frame of the accompanying object grayscale image is obtained; using the target attention factor of each frame of the accompanying object grayscale image as the contrast gain value, the target surrounding area corresponding to each frame of the accompanying object grayscale image is linearly enhanced to obtain an enhanced accompanying object grayscale image; using the trained convolutional neural network to judge the dangerous behavior of the accompanying object in each frame of the enhanced accompanying object grayscale image, and obtain a dangerous behavior judgment result; The step of obtaining the target motion amplitude of each frame of the grayscale image of the accompanying object according to the area of ​​the target region of the accompanying object and the position difference of all pixels on the boundary of each frame of the grayscale image of the accompanying object includes the following specific steps: Using contour detection technology to calculate the target area of ​​the accompanying object in each frame of the accompanying object grayscale image, to obtain the minimum circumscribed rectangle of the target area of ​​the accompanying object in each frame of the accompanying object grayscale image; According to The minimum bounding rectangle area of ​​the accompanying object target area of ​​the frame accompanying object grayscale image and the position difference of all pixels on the boundary are obtained. The target motion amplitude of the frame chaperone object grayscale image; According to the The minimum bounding rectangle area of ​​the accompanying object target area of ​​the frame accompanying object grayscale image and the position difference of all pixels on the boundary are obtained. The specific calculation formula for the target motion amplitude of the frame accompanying the object grayscale image is: in, Indicates the The target motion amplitude of the frame accompanying the grayscale image of the object, Indicates the The first frame of the accompanying object gray image is on the boundary of the accompanying object target area The distance from the pixel point to the centroid of the target area of ​​the accompanying object; Indicates the The first frame of the accompanying object gray image is on the boundary of the accompanying object target area The distance from the pixel point to the centroid of the target area of ​​the accompanying object; Indicates the The number of pixels on the boundary of the escort object target area of ​​the frame escort object grayscale image; The area of ​​the minimum circumscribed rectangle of the target area of ​​the accompanying object in the grayscale image of the accompanying object; Indicates the The area of ​​the target region of the escorted object in the frame escorted object grayscale image; represents the hyperbolic tangent function; express Belong to the interval All internal time The variance of .

2. The intelligent monitoring method for a rehabilitation accompanying robot according to claim 1, characterized in that: The step of obtaining the target action risk coefficient of each frame of the accompanying object grayscale image according to the difference between the target action amplitude of each frame of the accompanying object grayscale image and the next frame of the accompanying object grayscale image includes the following specific steps: Obtaining a centroid interval distance between grayscale images of the accompanying object in each frame according to the position of the centroid of the accompanying object target area in the grayscale images of the accompanying object in adjacent frames; According to The grayscale image of the accompanying object and the The difference in the target motion amplitude of the grayscale image of the frame accompanying the object and the The centroid distance of the grayscale image of the frame accompanying the object is obtained. The target action risk factor of the frame accompanying the grayscale image of the object.

3. The intelligent monitoring method for a rehabilitation accompanying robot according to claim 2, characterized in that: The method of obtaining the centroid distance of each frame of the accompanying object grayscale image according to the centroid position of the accompanying object target area of ​​the accompanying object grayscale image of adjacent frames includes the following specific steps: In each frame of the grayscale image of the accompanying object, a coordinate system is obtained by taking the vertex of the upper left corner of each frame of the grayscale image of the accompanying object as the origin, horizontally to the right as the positive direction of the horizontal axis, and vertically downward as the positive direction of the vertical axis; In the coordinate system, The coordinates of the centroid of the target area of ​​the accompanying object in the frame accompanying object grayscale image are the same as the coordinates of the centroid of the target area of ​​the frame accompanying object grayscale image The Euclidean distance between the coordinates of the centroid of the target area of ​​the accompanying object in the grayscale image of the accompanying object is recorded as The distance between the centroids of the grayscale images of the frame accompanying the object.

4. The intelligent monitoring method for a rehabilitation accompanying robot according to claim 2, characterized in that: According to the The grayscale image of the accompanying object and the The difference in the target motion amplitude of the grayscale image of the frame accompanying the object and the The centroid distance of the grayscale image of the frame accompanying the object is obtained. The specific calculation formula for the target action risk coefficient of the frame escort object grayscale image is: in, No. Target action risk factor of the grayscale image of the frame escort object; Indicates the The centroid distance between the grayscale images of the frame accompanying the object; Indicates the The target motion amplitude of the frame chaperone object grayscale image; Indicates the The target motion amplitude of the frame chaperone object grayscale image; represents the hyperbolic tangent function.

5. The intelligent monitoring method for a rehabilitation accompanying robot according to claim 1, characterized in that: The method of constructing the target surrounding area corresponding to each frame of the accompanying object grayscale image according to the distance between the centroid of the accompanying object target area of ​​each frame of the accompanying object grayscale image and the pixel points on the boundary of the accompanying object target area includes the following specific steps: Will be The center of mass of the target area of ​​the accompanying object in the grayscale image of the accompanying object is the center of the circle, and the maximum value of the distance from all pixels on the boundary of the accompanying object target area to the center of the circle is added to The sum of the values ​​of the circle with radius is called The frame accompanies the target surrounding area corresponding to the grayscale image of the object; wherein, The preset threshold length.

6. The intelligent monitoring method for a rehabilitation accompanying robot according to claim 1, characterized in that: The method of obtaining the target collision risk coefficient of each frame of the accompanying object grayscale image according to the difference in distances between all corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image and the centroid of the accompanying object target area includes the following specific steps: The first The first frame of the target surrounding area corresponding to the grayscale image of the object Corner point to The distance between the centroid of the target area of ​​the accompanying object in the frame accompanying the object grayscale image is recorded as The grayscale image of the frame accompanying the object The distance between the corner points; According to The grayscale image of the accompanying object and the The difference in the target motion amplitude of the grayscale image of the frame accompanying the object and the The distance between all corner points of the grayscale image of the frame accompanying the object is obtained. Target collision risk coefficient of the frame accompanying the grayscale image of the object.

7. The intelligent monitoring method for a rehabilitation accompanying robot according to claim 6, characterized in that: According to the The grayscale image of the accompanying object and the The difference in the target motion amplitude of the grayscale image of the frame accompanying the object and the The distance between all corner points of the grayscale image of the frame accompanying the object is obtained. The specific calculation formula for the target collision risk coefficient of the grayscale image of the frame escort object is: in, Indicates the Target collision risk coefficient of the grayscale image of the frame escort object; Indicates the The number of corner points in the target surrounding area corresponding to the grayscale image of the frame accompanying the object; Indicates the The grayscale image of the frame accompanying the object The distance between the corner points; Represents an exponential function with a natural constant as its base.

8. The intelligent monitoring method for a rehabilitation accompanying robot according to claim 1, characterized in that: The target care factor of each frame of the grayscale image of the escorted object is obtained according to the target collision risk coefficient and the target action risk coefficient of each frame of the grayscale image of the escorted object, including the following specific steps: The first The product of the target collision risk coefficient and the target action risk coefficient of the grayscale image of the frame escort object is recorded as The target illumination factor of the grayscale image of the frame accompanying the object.

Images