Intelligent monitoring method for rehabilitation accompanying robot

Abstract

The invention relates to the technical field of image data processing, in particular to an intelligent monitoring method for a rehabilitation accompanying robot, and the method comprises the steps: obtaining an accompanying object gray image and an accompanying object target region in the accompanying object gray image; and calculating a target action amplitude according to the target region area of the accompanying object and the pixel point difference on the boundary, and comparing the difference between the target action amplitudes of the adjacent frames of accompanying object grayscale images to obtain a target action danger coefficient. According to the distance between the mass center of the target area of the accompanying object and pixel points on the boundary of the target area, a target surrounding area is constructed, and according to the angular points of the target surrounding area, the target collision risk coefficient of each frame of the accompanying object gray level image is calculated. And a target collision risk coefficient and a target action risk coefficient are combined to obtain a target light factor, and then an enhanced image is generated and identified. According to the method, feature enhancement is performed on the grayscale image of the accompanying object, so that the definition of the grayscale image of the accompanying object is improved, and the recognition accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to an intelligent monitoring method of a rehabilitation accompanying robot. Background Art

[0002] The intelligent monitoring method of rehabilitation accompanying robots can be realized by combining artificial intelligence technology and human-computer interaction design. The robot can identify the user's emotions by analyzing the user's facial expressions, body movements, etc., so as to better understand the user's needs and status; therefore, it is necessary to use rehabilitation accompanying robots to obtain images of the accompanying elderly and identify the elderly, obtain images of the elderly and analyze and identify the elderly's emotions and movements. However, due to the influence of light, there will be misjudgment in the recognition results.

[0003] Existing problem: When the rehabilitation accompanying robot acquires images of the accompanying elderly, there may be different changes in the lighting in the room where the elderly are being accompanied, causing the elderly part to be darker. As a result, when the facial expressions, body movements, etc. of the elderly are subsequently analyzed, there will be a low image contrast and unclear images due to the dim light, which may cause recognition errors, thereby causing the rehabilitation accompanying robot to misjudge the dangerous behavior of the accompanying person. Summary of the invention

[0004] The invention provides an intelligent monitoring method for a rehabilitation accompanying robot to solve the existing problems.

[0005] The intelligent monitoring method of a rehabilitation accompanying robot of the present invention adopts the following technical scheme: An embodiment of the present invention provides an intelligent monitoring method for a rehabilitation accompanying robot, the method comprising the following steps: Acquire several frames of grayscale images of the accompanying object and a target area of ​​the accompanying object in the grayscale images of the accompanying object; According to the area of ​​the target region of the accompanying object in each frame of the accompanying object grayscale image and the position difference of all pixel points on the boundary, the target motion amplitude of each frame of the accompanying object grayscale image is obtained; According to the difference between the target action amplitude of each frame of the grayscale image of the accompanying object and the target action amplitude of the grayscale image of the accompanying object in the next frame, the target action risk coefficient of each frame of the grayscale image of the accompanying object is obtained; Constructing a target surrounding area corresponding to each frame of the accompanying object grayscale image according to the distance from the centroid of the accompanying object target area of ​​each frame of the accompanying object grayscale image to the pixel point on the boundary of the accompanying object target area; Obtain a number of corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image; obtain the target collision risk coefficient of each frame of the accompanying object grayscale image according to the difference in distances between all corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image and the centroid of the accompanying object target area; According to the target collision risk coefficient and the target action danger coefficient of each frame of the accompanying object grayscale image, the target attention factor of each frame of the accompanying object grayscale image is obtained; the target attention factor of each frame of the accompanying object grayscale image is used as the contrast gain value, and the target surrounding area corresponding to each frame of the accompanying object grayscale image is linearly enhanced to obtain an enhanced accompanying object grayscale image; the trained convolutional neural network is used to judge the dangerous behavior of the accompanying object in each frame of the enhanced accompanying object grayscale image to obtain the dangerous behavior judgment result.

[0006] Furthermore, the target motion amplitude of each frame of the grayscale image of the escorted object is obtained according to the area of ​​the target region of the escorted object in each frame of the grayscale image of the escorted object and the position difference of all pixels on the boundary, and the specific steps include the following: Using contour detection technology to calculate the target area of ​​the escorted object in each frame of the grayscale image of the escorted object, and obtaining the minimum circumscribed rectangle of the target area of ​​the escorted object in each frame of the grayscale image of the escorted object; According to The minimum circumscribed rectangle area of ​​the escort object grayscale image of the escort object and the position difference of all pixels on the boundary are obtained. The frame accompanies the target motion amplitude of the object grayscale image.

[0007] Furthermore, according to The minimum circumscribed rectangle area of ​​the escort object grayscale image of the escort object and the position difference of all pixels on the boundary are obtained. The specific calculation formula for the target motion amplitude of the grayscale image of the frame escort object is: in, Indicates The target motion amplitude of the frame accompanying the grayscale image of the object, Indicates The first The distance from the pixel point to the centroid of the target area of ​​the escort object; Indicates The first The distance from the pixel point to the centroid of the target area of ​​the escort object; Indicates The number of pixels on the boundary of the escort object target area of ​​the frame escort object grayscale image; The area of ​​the minimum circumscribed rectangle of the target area of ​​the escort object in the grayscale image of the escort object; Indicates The area of ​​the target region of the escort object in the frame escort object grayscale image; represents the hyperbolic tangent function; express Belong to the interval All internal time The variance of .

[0008] Furthermore, the target action risk coefficient of each frame of the accompanying object grayscale image is obtained according to the difference between the target action amplitude of each frame of the accompanying object grayscale image and the next frame of the accompanying object grayscale image, and the specific steps include the following: According to the position of the centroid of the target area of ​​the escorted object in the grayscale image of the escorted object in adjacent frames, the centroid interval distance of the grayscale image of the escorted object in each frame is obtained; According to The grayscale image of the object being cared for is The difference in the target motion amplitude of the grayscale image of the frame accompanying the object and the The centroid distance of the grayscale image of the frame accompanying the object is obtained. The target action risk factor of the grayscale image of the frame escort object.

[0009] Furthermore, the centroid spacing distance of each frame of the accompanying object grayscale image is obtained according to the position of the centroid of the accompanying object target area of ​​the accompanying object grayscale image of adjacent frames, including the following specific steps: In each frame of the grayscale image of the accompanying object, a coordinate system is obtained by taking the vertex at the upper left corner of each frame of the grayscale image of the accompanying object as the origin, horizontally to the right as the positive direction of the horizontal axis, and vertically downward as the positive direction of the vertical axis; In the coordinate system, The coordinates of the centroid of the target area of ​​the accompanying object in the grayscale image of the accompanying object in the frame are The Euclidean distance between the coordinates of the centroid of the target area of ​​the accompanying object in the grayscale image of the accompanying object in the frame is recorded as The centroid distance between the grayscale images of the frame accompanying the object.

[0010] Furthermore, according to The grayscale image of the object being cared for is The difference in the target motion amplitude of the grayscale image of the frame accompanying the object and the The centroid distance of the grayscale image of the frame accompanying the object is obtained. The specific calculation formula for the target action risk coefficient of the grayscale image of the frame escort object is: in, No. The target action risk factor of the grayscale image of the frame escort object; Indicates The centroid distance between the grayscale images of the frame escort objects; Indicates The target motion amplitude of the frame chaperone object grayscale image; Indicates The target motion amplitude of the frame chaperone object grayscale image; represents the hyperbolic tangent function.

[0011] Furthermore, the target surrounding area corresponding to each frame of the accompanying object grayscale image is constructed according to the distance from the centroid of the accompanying object target area of ​​each frame of the accompanying object grayscale image to the pixel point on the boundary of the accompanying object target area, and the specific steps include the following: Will be The centroid of the target area of ​​the escort object grayscale image of the frame escort object is the center of the circle, and the maximum value of the distance from all pixels on the boundary of the target area of ​​the escort object to the center of the circle plus The sum of the values ​​of the circle with radius is denoted as The frame accompanies the target surrounding area corresponding to the grayscale image of the object; wherein, is the preset threshold length.

[0012] Furthermore, the target collision risk coefficient of each frame of the accompanying object grayscale image is obtained according to the difference in distances between all corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image and the centroid of the accompanying object target area, and the specific steps include the following: The first The grayscale image of the object in the frame corresponds to the surrounding area of ​​the target Corner point to The distance between the centroids of the target area of ​​the accompanying object in the grayscale image of the accompanying object in the frame is recorded as The grayscale image of the frame accompanying the object The distance between the corner points; According to The grayscale image of the object being cared for is The difference in the target motion amplitude of the grayscale image of the frame accompanying the object and the The distance between all corner points of the grayscale image of the frame accompanying the object is obtained. Target collision risk coefficient of the grayscale image of the frame escort object.

[0013] Furthermore, according to The grayscale image of the object being cared for is The difference in the target motion amplitude of the grayscale image of the frame accompanying the object and the The distance between all corner points of the grayscale image of the frame accompanying the object is obtained. The specific calculation formula of the target collision risk coefficient of the grayscale image of the frame escort object is: in, Indicates Target collision risk factor of the grayscale image of the frame escort object; Indicates The number of corner points in the target surrounding area corresponding to the grayscale image of the frame escort object; Indicates The grayscale image of the frame accompanying the object The distance between the corner points; Represents an exponential function with a natural constant as its base.

[0014] Furthermore, the target care factor of each frame of the grayscale image of the escorted object is obtained according to the target collision risk coefficient and the target action risk coefficient of each frame of the grayscale image of the escorted object, and the specific steps include the following: The first The product of the target collision risk coefficient and the target action risk coefficient of the grayscale image of the frame escort object is recorded as The target illumination factor of the grayscale image of the frame accompaniment object.

[0015] The beneficial effects of the technical solution of the present invention are: Several frames of grayscale images of the accompanying object and the target area of ​​the accompanying object in the grayscale images of the accompanying object are obtained; the target movement amplitude of each frame of the accompanying object grayscale image is obtained according to the area of ​​the target area of ​​the accompanying object in each frame of the accompanying object grayscale image and the position difference of all pixels on the boundary, thereby improving the accuracy of judging whether the accompanying object is in a dangerous situation. According to the difference in the target movement amplitude of each frame of the accompanying object grayscale image and the accompanying object grayscale image of the next frame, the target movement risk coefficient of each frame of the accompanying object grayscale image is obtained, thereby further improving the accuracy of judging whether the accompanying object is in a dangerous situation.

[0016] According to the distance from the centroid of the target area of ​​the accompanying object in each frame of the accompanying object grayscale image to the pixel point on the boundary of the target area of ​​the accompanying object, the target surrounding area corresponding to each frame of the accompanying object grayscale image is constructed; a number of corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image are obtained; according to the difference in the distance between all corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image and the centroid of the target area of ​​the accompanying object, the target collision risk coefficient of each frame of the accompanying object grayscale image is obtained, which accurately reflects the situation of the surrounding environment of the accompanying object. According to the target collision risk coefficient and the target action risk coefficient of each frame of the accompanying object grayscale image, the target attention factor of each frame of the accompanying object grayscale image is obtained, which improves the accuracy of the recognition result. With the target attention factor of each frame of the accompanying object grayscale image as the contrast gain value, the target surrounding area corresponding to each frame of the accompanying object grayscale image is linearly enhanced to obtain an enhanced accompanying object grayscale image; the trained convolutional neural network is used to judge the dangerous behavior of the accompanying object in each frame of the enhanced accompanying object grayscale image to obtain the dangerous behavior judgment result. The present invention enhances the features of the grayscale image of the escort object to improve the clarity of the grayscale image of the escort object, thereby improving the accuracy of recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 The present invention is a flowchart of the steps of an intelligent monitoring method of a rehabilitation accompanying robot. DETAILED DESCRIPTION

[0019] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the intelligent monitoring method of a rehabilitation accompanying robot proposed by the present invention, its specific implementation method, structure, characteristics and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0020] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0021] The specific scheme of the intelligent monitoring method of a rehabilitation accompanying robot provided by the present invention is described in detail below with reference to the accompanying drawings.

[0022] See also Figure 1 , which shows a flowchart of a method for intelligent monitoring of a rehabilitation accompanying robot provided by an embodiment of the present invention, the method comprising the following steps: Step S001: Acquire several frames of grayscale images of a care object and a target area of ​​the care object in the grayscale images of the care object.

[0023] First, several frames of pictures of the accompanying object are captured from the video images of the human activities in the room with poor light, and then each frame of the accompanying object picture is gray-scaled to obtain several frames of gray-scale images of the accompanying object.

[0024] What needs to be explained is that: for the rehabilitation accompanying robot, it is equipped with a target tracking system, which will track and collect the image information of the accompanying object in real time, and monitor and identify the sitting, lying and walking postures of the accompanying object through image data. The existing method realizes human posture recognition by matching the changes of key nodes of human limbs in continuous frames with preset normal postures, and obtains expression information through key points of the face, and combines with vital signs such as heart rate and blood pressure to relatively comprehensively evaluate the daily physical and psychological conditions of the accompanying object. I will not explain too much about the existing technology; but all this needs to be based on reliable image data collection. Taking into account the actual room orientation, house type, weather, and time, some poor light will be caused to the collected image. This problem is common and greatly limits the practicality of the rehabilitation accompanying robot, and has a great impact on the monitoring results of the rehabilitation accompanying robot. Therefore, it is necessary to capture the picture of the accompanying object from the video image of the human activities in the room with poor light.

[0025] The embodiment of the present invention adopts a segmentation neural network to identify the target area and the background area of ​​the escort object in each frame of the greyscale image of the escort object.

[0026] The relevant content of the segmentation neural network is as follows: The segmentation neural network used in this embodiment is the Mask R-CNN neural network; the data set used is the grayscale image data set of the accompanying object. Mask R-CNN is a well-known technology, and the specific method is not introduced here. The full name of Mask R-CNN in Chinese is "Mask Region-based Convolutional Neural Network", and the full name in English is "Mask Region-based Convolutional Neural Network".

[0027] The pixels that need to be segmented are divided into two categories, that is, the labeling process of the training set is: single-channel semantic label, the corresponding position pixel belongs to the background area is labeled as 0, and the corresponding position pixel belongs to the target area of ​​the escort object is labeled as 1.

[0028] The task of the network is classification, so the loss function used is the cross entropy loss function.

[0029] The target area and background area of ​​the escort object in each frame of the escort object grayscale image are obtained by segmenting the neural network. This process is a well-known technology and the specific method will not be introduced here.

[0030] Through the above process, the escort object target area of ​​each frame of the escort object grayscale image is obtained.

[0031] Step S002: obtaining a target motion amplitude of each frame of the grayscale image of the accompanying object according to the area of ​​the target region of the accompanying object in each frame of the grayscale image of the accompanying object and the position difference of all pixel points on the boundary; It should be noted that in order to determine whether the grayscale image of the current escort object needs to be enhanced, it is necessary to make a judgment based on the obtained target area of ​​the escort object. Since the target area of ​​the escort object obtained by the above operation may be only a rough connected domain due to insufficient lighting and other reasons, it is necessary to enhance the grayscale image of the escort object so as to facilitate the escort robot to analyze whether the escort object has any dangerous behavior.

[0032] It is further necessary to explain that: when the escort object makes a larger movement, the possibility of dangerous behavior such as falling is greater; therefore, it is necessary to analyze the edge features of the obtained escort object target area, wherein when the escort object makes a larger movement, the arms and legs undergo large changes, so the acquired escort object target area has a large convex part; and compare the changes in the movement distance and movement amplitude of the escort object between adjacent image frames.

[0033] The contour detection technology is used to calculate the target area of ​​the escorted object in each frame of the escorted object grayscale image to obtain the minimum circumscribed rectangle of the target area of ​​the escorted object in each frame of the escorted object grayscale image. Among them, the contour detection technology is a well-known technology, and the specific method is not introduced here.

[0034] First Taking the grayscale image of the frame escort object as an example, the The specific calculation formula for the target motion amplitude of the frame escort object grayscale image is: in, Indicates The target motion amplitude of the frame accompanying the grayscale image of the object, Indicates The first The distance from the pixel point to the centroid of the target area of ​​the escort object; Indicates The first The distance from the pixel point to the centroid of the target area of ​​the escort object; Indicates The number of pixels on the boundary of the escort object target area of ​​the frame escort object grayscale image; The area of ​​the minimum circumscribed rectangle of the target area of ​​the escort object in the grayscale image of the escort object; Indicates The area of ​​the target region of the escort object in the frame escort object grayscale image; represents the hyperbolic tangent function; express Belong to the interval All internal time The variance of .

[0035] It should be noted that when the escort object makes a large movement, its limbs are stretched outward, so the variance of the distance between the pixel point on the boundary of the escort object target area of ​​the escort object grayscale image and the centroid of the escort object target area is The larger the value, the larger the convex part of the target area of ​​the escort object. Therefore, the larger the target movement amplitude is, the more There is a positive correlation; and the area difference between the minimum enclosing rectangle of the escort target area and the escort target area , the larger the value, the larger and more bulges there are in the target area of ​​the accompanying object in the grayscale image of the accompanying object, which indicates that the larger the target movement amplitude is, the higher the value is. There is a positive correlation.

[0036] Through the above process, we obtain The target motion amplitude of the frame chaperone object grayscale image; Each frame of the grayscale image of the accompanying object is calculated according to the above process to obtain the target motion amplitude of each frame of the grayscale image of the accompanying object.

[0037] Step S003: obtaining the target action risk coefficient of each frame of the accompanying object grayscale image according to the difference between the target action amplitude of each frame of the accompanying object grayscale image and the next frame of the accompanying object grayscale image.

[0038] It should be noted that for the target motion amplitude of each frame of the grayscale image of the escort object, when the escort object finds that it has fallen or other dangerous movements, the target motion amplitude will change greatly and its position will also change greatly. Therefore, it is necessary to analyze and compare the changes in the target motion amplitude of the previous and next frames and the changes in the position of the escort object.

[0039] In each frame of the grayscale image of the accompanying object, a coordinate system is obtained by taking the vertex at the upper left corner of each frame of the grayscale image of the accompanying object as the origin, horizontally to the right as the positive direction of the horizontal axis, and vertically downward as the positive direction of the vertical axis.

[0040] In the coordinate system, The coordinates of the centroid of the target area of ​​the accompanying object in the grayscale image of the accompanying object in the frame are The Euclidean distance between the coordinates of the centroid of the target area of ​​the accompanying object in the grayscale image of the accompanying object in the frame is recorded as The centroid distance between the grayscale images of the frame accompanying the object.

[0041] It should be noted that the centroid interval distance of the last frame of the accompanying object grayscale image is the same as the centroid interval distance of the previous frame of the accompanying object grayscale image.

[0042] The first The specific calculation formula of the target action risk coefficient of the grayscale image of the frame escort object is: in, No. The target action risk factor of the grayscale image of the frame escort object; Indicates The centroid distance between the grayscale images of the frame escort objects; Indicates The target motion amplitude of the frame chaperone object grayscale image; Indicates The target motion amplitude of the frame chaperone object grayscale image; represents the hyperbolic tangent function.

[0043] It should be noted that the target action risk coefficient of the last frame of the grayscale image of the escorted object is the same as the target action risk coefficient of the previous frame of the grayscale image of the escorted object.

[0044] It is further necessary to explain that: when the accompanying person performs some dangerous actions, such as falling, his body will change greatly and move in a short period of time. The centroid distance between the grayscale images of the frame accompanying the object The larger the value, the greater the displacement of the escort object in a short period of time, indicating that the possibility of danger is higher, and the higher the risk coefficient of the action is. There is a positive correlation; and for The difference between the target motion amplitude of the grayscale image of the accompanying object in one frame and the grayscale image of the accompanying object in the next frame The larger the value, the greater the change in the current escort object's movement amplitude is, and the more likely it is to be dangerous. Therefore, the higher the action risk coefficient is, the more likely it is to be dangerous. There is a positive correlation.

[0045] Through the above process, we obtain The target action risk factor of the grayscale image of the frame escort object; Each frame of the grayscale image of the escort object is calculated according to the above process to obtain the target action risk coefficient of each frame of the grayscale image of the escort object.

[0046] Step S004: constructing the target surrounding area corresponding to each frame of the accompanying object grayscale image according to the distance from the centroid of the accompanying object target area of ​​each frame of the accompanying object grayscale image to the pixel points on the boundary of the accompanying object target area.

[0047] It should be noted that: for the acquired grayscale image of the escort object, in addition to analyzing whether there is any dangerous action, it is also necessary to analyze whether there is a risk of collision in the environment where the escort object is located.

[0048] Still For example, the grayscale image of the object being escorted by the frame The centroid of the target area of ​​the escort object grayscale image of the frame escort object is the center of the circle, and the maximum value of the distance from all pixels on the boundary of the target area of ​​the escort object to the center of the circle plus The sum of the values ​​of the circle with radius is denoted as The frame accompanies the target surrounding area corresponding to the grayscale image of the object. is the preset threshold length. The preset threshold length in this embodiment is The value is 20, and this is used as an example for description.

[0049] Each frame of the grayscale image of the escorted object is operated according to the above process to obtain the target surrounding area corresponding to each frame of the grayscale image of the escorted object.

[0050] Step S005: obtaining a number of corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image; obtaining the target collision risk coefficient of each frame of the accompanying object grayscale image according to the difference in distances between all corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image and the centroid of the accompanying object target area; It should be noted that: for the acquired target surrounding area, it is necessary to analyze whether the escort object is at risk of bumping in the target surrounding area. Bumping often occurs because of the edges and corners of furniture such as tables and chairs. Therefore, it is necessary to perform corner point detection on the target surrounding area. If there are many corner points around the escort object, it indicates that the current escort object may be in an environment where bumps are more likely to occur.

[0051] A corner point detection algorithm is used to calculate the target surrounding area corresponding to each frame of the accompanying object grayscale image to obtain a number of corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image. The corner point detection algorithm is a well-known technology and the specific method is not introduced here.

[0052] The first The grayscale image of the object in the frame corresponds to the surrounding area of ​​the target Corner point to The distance between the centroids of the target area of ​​the accompanying object in the grayscale image of the accompanying object in the frame is recorded as The grayscale image of the frame accompanying the object The distance between the corner points.

[0053] The first The specific calculation formula of the target collision risk coefficient of the grayscale image of the frame escort object is: in, Indicates Target collision risk factor of the grayscale image of the frame escort object; Indicates The number of corner points in the target surrounding area corresponding to the grayscale image of the frame escort object; Indicates The grayscale image of the frame accompanying the object The distance between the corner points; Represents an exponential function with a natural constant as its base.

[0054] It should be noted that: since it is necessary to detect whether there is a risk of collision around the escort object, the corner points of the target surrounding area corresponding to the grayscale image of the escort object in the i-th frame are compared with the corner points of the grayscale image of the escort object in the i-th frame. The average of the distance and the centroid of the target area of ​​the accompanying object in the frame accompanying object grayscale image is obtained The smaller the value, the closer the corner point of the target area is to the escort object. Therefore, the target collision risk coefficient is is negatively correlated, and for The number of corner points in the target area corresponding to the grayscale image of the frame escort object , The larger the value, the higher the risk of collision around the person being cared for. and There is a positive correlation.

[0055] Through the above process, the target collision risk coefficient of the grayscale image of the escort object in the i-th frame is obtained; Each frame of the grayscale image of the escort object is calculated according to the above process to obtain the target collision risk coefficient of each frame of the grayscale image of the escort object.

[0056] Step S006: according to the target collision risk coefficient and the target action risk coefficient of each frame of the grayscale image of the accompanying object, the target attention factor of each frame of the grayscale image of the accompanying object is obtained; the target attention factor of each frame of the grayscale image of the accompanying object is used as the contrast gain value, and the target surrounding area corresponding to each frame of the grayscale image of the accompanying object is linearly enhanced to obtain an enhanced grayscale image of the accompanying object; the trained convolutional neural network is used to judge the dangerous behavior of the accompanying object in each frame of the enhanced grayscale image of the accompanying object to obtain a dangerous behavior judgment result.

[0057] It should be noted that: when the escort object makes dangerous movements or may collide, the escort robot needs to analyze the state of the escort object and take care of it, so the target care factor of the grayscale image of the escort object is obtained.

[0058] The first The product of the target collision risk coefficient and the target action risk coefficient of the grayscale image of the frame escort object is recorded as The target illumination factor of the grayscale image of the frame accompaniment object.

[0059] Each frame of the grayscale image of the accompanying object is operated according to the above process to obtain the target attention factor of each frame of the grayscale image of the accompanying object.

[0060] It should be noted that: in order for the accompanying robot to analyze the state of the accompanying object subsequently, each frame of the accompanying object grayscale image needs to be enhanced according to the target attention factor of each frame of the accompanying object grayscale image.

[0061] In each frame of the grayscale image of the accompanying object, a gradient histogram of the target surrounding area of ​​each frame of the accompanying object grayscale image is obtained, and each gradient value on the gradient histogram is multiplied by the target illumination factor of each frame of the accompanying object grayscale image plus The sum of the values ​​of is used to obtain the enhanced grayscale image of the accompanying object corresponding to each frame of the accompanying object grayscale image. Wherein, obtaining the gradient histogram of the target surrounding area of ​​each frame of the accompanying object grayscale image is a well-known technology, and the specific method is not introduced here. is a preset threshold gradient. The threshold gradient preset in this embodiment is The value is 1, and this example is used for description.

[0062] It should be noted that the above-mentioned enhancement operation is to perform a linear transformation on the gradient histogram of the image, which is a well-known technology. The target illumination factor of each frame of the grayscale image of the accompanying object reflects the contrast gain value in the linear transformation. The larger the target illumination factor, the more important the image is. Therefore, a larger contrast gain value is assigned to improve the image contrast and make the image clear.

[0063] The trained convolutional neural network is used to judge the dangerous behavior of the accompanying object in each frame of the enhanced accompanying object grayscale image to obtain the dangerous behavior judgment result.

[0064] It should be noted that the input of the convolutional neural network in this embodiment is the enhanced grayscale image of the accompanying object in each frame, and the output is the result of the dangerous behavior judgment, that is, the accompanying object's behavior is normal and the accompanying object's behavior is dangerous. The convolutional neural network model is ResNet500, which is used as an example for description. The data set is a data set composed of all frame enhanced accompanying object grayscale images. Whether the accompanying object's behavior is normal in each frame enhanced accompanying object grayscale image is used as the label of the sample, such as the accompanying object's normal behavior is marked as 1, and the accompanying object's dangerous behavior is marked as 2. The neural network is trained using the above-obtained data set, and the loss function used is the cross entropy loss function. The specific training process is well known and will not be described in detail in this embodiment.

[0065] So far, the present invention is completed.

[0066] In summary, in an embodiment of the present invention, a grayscale image of a care object and a target area of ​​the care object in the grayscale image of the care object are obtained. The target motion amplitude is calculated based on the difference between the area of ​​the target area of ​​the care object and the pixel points on the boundary, and the difference in the target motion amplitude of the grayscale images of the care object in adjacent frames is compared to obtain the target motion risk coefficient. According to the distance between the center of mass of the target area of ​​the care object and the pixel points on its boundary, a target surrounding area is constructed, and according to the corner points of the target surrounding area, the target collision risk coefficient of each frame of the grayscale image of the care object is calculated. Combining the target collision risk coefficient and the target motion risk coefficient, the target care factor is obtained, and then an enhanced image is generated and identified. The present invention improves the clarity of the grayscale image of the care object by performing feature enhancement on the grayscale image of the care object, thereby improving the accuracy of recognition.

[0067] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent monitoring method for a rehabilitation accompanying robot, characterized in that: The method comprises the following steps: Acquire several frames of grayscale images of the accompanying object and a target area of ​​the accompanying object in the grayscale images of the accompanying object; According to the area of ​​the target region of the accompanying object in each frame of the accompanying object grayscale image and the position difference of all pixel points on the boundary, the target motion amplitude of each frame of the accompanying object grayscale image is obtained; According to the difference between the target action amplitude of each frame of the grayscale image of the accompanying object and the target action amplitude of the grayscale image of the accompanying object in the next frame, the target action risk coefficient of each frame of the grayscale image of the accompanying object is obtained; Constructing a target surrounding area corresponding to each frame of the accompanying object grayscale image according to the distance from the centroid of the accompanying object target area of ​​each frame of the accompanying object grayscale image to the pixel point on the boundary of the accompanying object target area; Obtain a number of corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image; obtain the target collision risk coefficient of each frame of the accompanying object grayscale image according to the difference in distances between all corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image and the centroid of the accompanying object target area; According to the target collision risk coefficient and the target action danger coefficient of each frame of the accompanying object grayscale image, the target attention factor of each frame of the accompanying object grayscale image is obtained; the target attention factor of each frame of the accompanying object grayscale image is used as the contrast gain value, and the target surrounding area corresponding to each frame of the accompanying object grayscale image is linearly enhanced to obtain an enhanced accompanying object grayscale image; the trained convolutional neural network is used to judge the dangerous behavior of the accompanying object in each frame of the enhanced accompanying object grayscale image to obtain the dangerous behavior judgment result.

2. The intelligent monitoring method of a rehabilitation accompanying robot according to claim 1, characterized in that: The specific steps of obtaining the target motion amplitude of each frame of the grayscale image of the escorted object according to the area of ​​the target region of the escorted object in each frame of the grayscale image of the escorted object and the position difference of all pixels on the boundary are as follows: Using contour detection technology to calculate the target area of ​​the escorted object in each frame of the grayscale image of the escorted object, and obtaining the minimum circumscribed rectangle of the target area of ​​the escorted object in each frame of the grayscale image of the escorted object; According to The minimum circumscribed rectangle area of ​​the escort object grayscale image of the escort object and the position difference of all pixels on the boundary are obtained. The frame accompanies the target motion amplitude of the object grayscale image.

3. The intelligent monitoring method of a rehabilitation accompanying robot according to claim 2, characterized in that: According to the The minimum circumscribed rectangle area of ​​the escort object grayscale image of the escort object and the position difference of all pixels on the boundary are obtained. The specific calculation formula for the target motion amplitude of the grayscale image of the frame escort object is: in, Indicates The target motion amplitude of the frame accompanying the grayscale image of the object, Indicates The first The distance from the pixel point to the centroid of the target area of ​​the escort object; Indicates The first The distance from the pixel point to the centroid of the target area of ​​the escort object; Indicates The number of pixels on the boundary of the escort object target area of ​​the frame escort object grayscale image; The area of ​​the minimum circumscribed rectangle of the target area of ​​the escort object in the grayscale image of the escort object; Indicates The area of ​​the target region of the escort object in the frame escort object grayscale image; represents the hyperbolic tangent function; express Belong to the interval All internal time The variance of .

4. The intelligent monitoring method of a rehabilitation accompanying robot according to claim 1, characterized in that: The step of obtaining the target action risk coefficient of each frame of the grayscale image of the accompanying object according to the difference between the target action amplitude of each frame of the grayscale image of the accompanying object and the grayscale image of the accompanying object in the next frame includes the following specific steps: According to the position of the centroid of the target area of ​​the escorted object in the grayscale image of the escorted object in adjacent frames, the centroid interval distance of the grayscale image of the escorted object in each frame is obtained; According to The grayscale image of the frame accompanying the object is The difference in the target motion amplitude of the grayscale image of the frame accompanying the object and the The centroid distance of the grayscale image of the frame accompanying the object is obtained. The target action risk factor of the grayscale image of the frame escort object.

5. The intelligent monitoring method of a rehabilitation accompanying robot according to claim 4, characterized in that: The method of obtaining the centroid interval distance of each frame of the grayscale image of the escorted object according to the position of the centroid of the target area of ​​the escorted object in the grayscale image of the escorted object in adjacent frames includes the following specific steps: In each frame of the grayscale image of the accompanying object, a coordinate system is obtained by taking the vertex at the upper left corner of each frame of the grayscale image of the accompanying object as the origin, horizontally to the right as the positive direction of the horizontal axis, and vertically downward as the positive direction of the vertical axis; In the coordinate system, The coordinates of the centroid of the target area of ​​the accompanying object in the grayscale image of the accompanying object in the frame are The Euclidean distance between the coordinates of the centroid of the target area of ​​the accompanying object in the grayscale image of the accompanying object in the frame is recorded as The centroid distance between the grayscale images of the frame accompanying the object.

6. The intelligent monitoring method of a rehabilitation accompanying robot according to claim 4, characterized in that: According to the The grayscale image of the frame accompanying the object is The difference in the target motion amplitude of the grayscale image of the frame accompanying the object and the The centroid distance of the grayscale image of the frame accompanying the object is obtained. The specific calculation formula for the target action risk coefficient of the grayscale image of the frame escort object is: in, No. The target action risk factor of the grayscale image of the frame escort object; Indicates The centroid distance between the grayscale images of the frame escort objects; Indicates The target motion amplitude of the frame chaperone object grayscale image; Indicates The target motion amplitude of the frame chaperone object grayscale image; represents the hyperbolic tangent function.

7. The intelligent monitoring method of a rehabilitation accompanying robot according to claim 1, characterized in that: The method of constructing the target surrounding area corresponding to each frame of the accompanying object grayscale image according to the distance from the centroid of the accompanying object target area of ​​each frame of the accompanying object grayscale image to the pixel point on the boundary of the accompanying object target area includes the following specific steps: Will be The centroid of the target area of ​​the escort object grayscale image of the frame escort object is the center of the circle, and the maximum value of the distance from all pixels on the boundary of the target area of ​​the escort object to the center of the circle plus The sum of the values ​​of the circle with radius is denoted as The frame accompanies the target surrounding area corresponding to the grayscale image of the object; wherein, is the preset threshold length.

8. The intelligent monitoring method of a rehabilitation accompanying robot according to claim 1, characterized in that: The specific steps of obtaining the target collision risk coefficient of each frame of the accompanying object grayscale image according to the difference in distances between all corner points in the target surrounding area corresponding to each frame of the accompanying object grayscale image and the centroid of the accompanying object target area are as follows: The first The grayscale image of the object in the frame corresponds to the surrounding area of ​​the target Corner point to The distance between the centroids of the target area of ​​the accompanying object in the grayscale image of the accompanying object in the frame is recorded as The grayscale image of the frame accompanying the object The distance between the corner points; According to The grayscale image of the frame accompanying the object is The difference in the target motion amplitude of the grayscale image of the frame accompanying the object and the The distance between all corner points of the grayscale image of the frame accompanying the object is obtained. Target collision risk coefficient of the grayscale image of the frame escort object.

9. The intelligent monitoring method of a rehabilitation accompanying robot according to claim 8, characterized in that: According to the The grayscale image of the frame accompanying the object is The difference in the target motion amplitude of the grayscale image of the frame accompanying the object and the The distance between all corner points of the grayscale image of the frame accompanying the object is obtained. The specific calculation formula of the target collision risk coefficient of the grayscale image of the frame escort object is: in, Indicates Target collision risk factor of the grayscale image of the frame escort object; Indicates The number of corner points in the target surrounding area corresponding to the grayscale image of the frame escort object; Indicates The grayscale image of the frame accompanying the object The distance between the corner points; Represents an exponential function with a natural constant as its base.

10. The intelligent monitoring method of a rehabilitation accompanying robot according to claim 1, characterized in that: The target care factor of each frame of the grayscale image of the escort object is obtained according to the target collision risk coefficient and the target action risk coefficient of each frame of the grayscale image of the escort object, and the specific steps include the following: The first The product of the target collision risk coefficient and the target action risk coefficient of the grayscale image of the frame escort object is recorded as The target illumination factor of the grayscale image of the frame accompaniment object.

Images