Action recognition method, device, equipment and medium based on edge separation

Abstract

The embodiment of the present invention discloses a behavior recognition method, device, equipment and medium based on edge separation, the method comprising: obtaining image data of a preset range through an edge computing device; segmenting objects in a target area according to the image data, and obtaining surface features of the segmented objects; obtaining monitoring data corresponding to the image data through an edge computing device, and judging the action data of a target person within a preset range through an action recognition model; determining the target object corresponding to the action data based on the action data, surface features and a preset surface feature database, and determining the behavior information of the target person based on the target object, action data and preset behavior data. The behavior of the target person is determined by combining the surface features and the action data of the target person, thereby improving the accuracy of behavior recognition, obtaining only image data and monitoring data within a preset range, avoiding excessive exposure of unnecessary personal space, and strengthening the protection of personal privacy.

Description

Technical Field

[0001] The present invention relates to the field of intelligent monitoring technology, and in particular to an edge separation-based action recognition method, device, equipment and medium. Background Art

[0002] With the aging of the population and the changes in social structure, home-based elderly care has become the choice of more and more people. In this context, the development of smart home technology and the Internet of Things has provided new possibilities for home-based elderly care services.

[0003] In traditional surveillance technology, in order to identify the interaction between the human body and objects, such as the relationship between the human body and the bed, it is often necessary to conduct a detailed analysis of the video content. However, this practice may infringe on personal privacy, especially in a home environment. To solve this problem, some technologies try to protect privacy by converting the background environment into a sketch. Such processing can effectively hide sensitive information in the environment, such as the bank account password that an elderly person with memory impairment may write on a noteboard at home. The sketch filter removes text information that may cause privacy leakage by simplifying the background, but this method is too simplified, sacrificing the details of the relationship between objects and the environment in the scene, and cannot provide enough spatial information.

[0004] In addition, when relying on manual observation, such as the interpretation of surveillance images by caregivers and guardians, this approach is also not conducive to effective visualization management while protecting privacy due to the lack of intuitive display of environmental depth and spatial relationships. This not only limits the accurate understanding of behavior, but also makes it difficult to meet the needs of modern surveillance systems for details and environmental perception.

[0005] With the development of image recognition technology, video surveillance can also be used in intelligent monitoring scenarios of the ward, so that the situation of the ward can be understood in real time. However, most of the existing technologies identify the behavior of the ward based on the monitoring image of the ward. However, when the same action interacts with different objects, different behaviors will be formed. For example, when the tilting action interacts with the bed, the behavior at this time can be understood as lying down or falling. Therefore, the method of identifying the behavior of the ward based on the monitoring image of the ward in the existing technology has the problem of inaccurate recognition. Summary of the invention

[0006] In view of this, the present invention provides an edge separation-based action recognition method, device, equipment and medium, which are used to solve the problem of inaccurate behavior recognition in the prior art.

[0007] To achieve one, part, or all of the above purposes or other purposes, the present invention provides an action recognition method based on edge separation, comprising:

[0008] Acquire image data of a preset range through an edge computing device;

[0009] Segmenting objects in the target area according to the image data, and acquiring surface features of the segmented objects;

[0010] Acquire monitoring data corresponding to the image data through an edge computing device, and determine motion data of a target person within a preset range through a motion recognition model based on the monitoring data;

[0011] A target object corresponding to the action data is determined based on the action data, the surface features and a preset surface feature database, and behavior information of the target person is determined based on the target object, the action data and preset behavior data.

[0012] Optionally, before the step of acquiring image data of a preset range through the edge computing device, the method further includes:

[0013] Obtaining identity information of the edge computing device, and matching the identity information with preset authorization data;

[0014] If the identity information matches the preset authorization data, then continue to execute the step of acquiring image data of a preset range through the edge computing device;

[0015] If the identity information does not match the preset permission data, obtaining permission information of the edge computing device;

[0016] When the permission information meets the preset conditions, the edge computing device is authorized through a preset authorization method. The authorized edge computing device is used to obtain image data within a preset range. The preset authorization method includes a local authorization method, a formal authorization method, and a remote authorization method.

[0017] Optionally, the step of segmenting the objects in the target area according to the image data and acquiring surface features of the segmented objects includes:

[0018] Segmenting the objects in the image data using semantic segmentation technology to obtain an initial image of the object;

[0019] Identifying the edge of the object in the initial image of the object by edge detection technology, and marking the edge to obtain the segmented object image;

[0020] Determining surface data of the object in the object image according to a surface detection technique;

[0021] The surface data is segmented by deep learning technology to obtain the surfaces of the segmented objects, and the area data of each surface is calculated as the surface features of the segmented objects.

[0022] Optionally, before the step of segmenting the objects in the target area according to the image data and acquiring the surface features of the segmented objects, the method further includes:

[0023] Segmenting the image data into a human body image and a background image;

[0024] Identify the human body image, determine the naked area and the clothed area in the human body image, and perform image privacy processing on the naked area to obtain a target human body image;

[0025] Performing background privacy processing on the background image to obtain a target background image;

[0026] Target image data is obtained according to the target human image and the target background image, and the target image data is used to segment objects in a target area.

[0027] Optionally, the monitoring data includes at least one of visual modality data and non-visual modality data, and the step of determining the motion data of the target person within the preset range through a motion recognition model based on the monitoring data includes:

[0028] When the monitoring data includes the visual modality data and the non-visual modality data, extracting first feature data of the visual modality data by a first convolutional neural network;

[0029] Extracting second feature data of the non-visual modality data by a second convolutional neural network;

[0030] Performing feature fusion on the first feature data and the second feature data to obtain fused feature data;

[0031] The fused feature data is input into the action recognition model to determine the action data of the target person within the preset range.

[0032] Optionally, the step of determining a target object corresponding to the action data based on the action data, the surface feature and a preset surface feature database, and determining behavior information of the target person based on the target object, the action data and preset behavior data includes:

[0033] Selecting, in the preset surface feature database, a set of preset surface features of different objects corresponding to the action data based on the action data;

[0034] Matching the surface feature with a preset surface feature in the preset surface feature set to obtain a preset surface feature that matches the surface feature, and taking an object corresponding to the preset surface feature as a target object;

[0035] The preset behavior data is searched for a preset behavior type corresponding to the target object and the action data as the behavior information of the target person.

[0036] Optionally, the method further includes:

[0037] Determining the damage level of the target person according to the behavior information and preset classification rules;

[0038] If the damage level is greater than or equal to the preset level, the behavior information is sent to the first receiving end, and alarm information is generated based on the damage level, the behavior information and the location information of the target person, and the alarm information is sent to the second receiving end.

[0039] On the other hand, an embodiment of the present application provides an edge separation-based action recognition device, the device comprising:

[0040] A data acquisition module, used to obtain image data of a preset range through an edge computing device;

[0041] A segmentation module, used to segment objects in the target area according to the image data and obtain surface features of the segmented objects;

[0042] A first recognition module is used to obtain monitoring data corresponding to the image data through an edge computing device, and determine the motion data of the target person within the preset range through a motion recognition model based on the monitoring data;

[0043] The second recognition module is used to determine the target object corresponding to the action data based on the action data, the surface features and a preset surface feature database, and to determine the behavior information of the target person based on the target object, the action data and preset behavior data.

[0044] On the other hand, an embodiment of the present application provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the edge separation-based action recognition method as described above are performed.

[0045] On the other hand, an embodiment of the present application provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned edge separation-based action recognition method are executed.

[0046] Implementing the embodiments of the present invention will have the following beneficial effects:

[0047] The image data of a preset range is obtained through the edge computing device; the objects in the target area are segmented according to the image data, and the surface features of the segmented objects are obtained; the monitoring data corresponding to the image data is obtained through the edge computing device, and the action data of the target person in the preset range is judged through the action recognition model based on the monitoring data; the target object corresponding to the action data is determined based on the action data, the surface features and the preset surface feature database, and the behavior information of the target person is determined based on the target object, the action data and the preset behavior data. The behavior of the target person is determined by combining the surface features and the action data of the target person, which improves the accuracy of behavior recognition, only obtains the image data and monitoring data within the preset range, avoids excessive exposure of unnecessary personal space, and strengthens the protection of personal privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] in:

[0050] Figure 1 is a flow chart of an action recognition method based on edge separation provided in an embodiment of the present application;

[0051] Figure 2 It is a structural schematic diagram of a seasonal electricity consumption data monitoring device provided in an embodiment of the present application;

[0052] Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0053] Figure 4 It is a structural schematic diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] like Figure 1As shown, the embodiment of the present application provides an action recognition method based on edge separation, comprising:

[0056] S101, acquiring image data of a preset range through an edge computing device;

[0057] Exemplarily, the edge computing device is embedded hardware, and the range parameters of the preset range can be adjusted by setting the angle of the edge computing device. The image captured by the edge computing device can also be cropped according to preset requirements to obtain image data of the preset range. For example, the preset required image data is a range with the edge computing device as the center and 5 meters as the radius. In this case, the image captured by the edge computing device is cropped to obtain image data with the edge computing device as the center and 5 meters as the radius.

[0058] S102, segmenting objects in the target area according to the image data, and acquiring surface features of the segmented objects;

[0059] In a possible implementation, the step of segmenting the objects in the target area according to the image data and acquiring surface features of the segmented objects includes:

[0060] Segmenting the objects in the image data using semantic segmentation technology to obtain an initial image of the object;

[0061] Identifying the edge of the object in the initial image of the object by edge detection technology, and marking the edge to obtain the segmented object image;

[0062] Determining surface data of the object in the object image according to a surface detection technique;

[0063] The surface data is segmented by deep learning technology to obtain the surfaces of the segmented objects, and the area data of each surface is calculated as the surface features of the segmented objects.

[0064] Exemplarily, the process of identifying the edges of the object in the initial image of the object is: using computer vision technology to detect the edges in the image; the process of segmenting the objects in the image data is: using semantic segmentation technology to segment the objects in the image; the process of marking the edges is: marking the edges of the segmented objects, especially finely marking the edges of the ground, and excluding the edges of objects above the ground.

[0065] Exemplarily, the process of determining the surface data of the object in the object image according to the surface detection technology is: based on the result of object segmentation (initial image of the object), the position and shape of the surface of the object are detected using depth information or point cloud data. Specifically, the surface detection technology includes: Depth map analysis: analyzing the depth information of the surface of the object from the depth map obtained by the depth camera. Point cloud processing: filtering, aligning and smoothing the point cloud data. Surface detection: using plane fitting, cluster segmentation and other methods to detect the position and shape of the surface of the object. Surface extraction: extracting a point set of a specific object surface from the point cloud data. Surface parameterization: parameterizing the extracted surface to obtain a mathematical representation of the surface.

[0066] S103, acquiring monitoring data corresponding to the image data through an edge computing device, and determining motion data of a target person within a preset range through a motion recognition model based on the monitoring data;

[0067] In a possible implementation, the step of determining the motion data of the target person within the preset range by using the motion recognition model includes:

[0068] When the monitoring data includes the visual modality data and the non-visual modality data, extracting first feature data of the visual modality data by a first convolutional neural network;

[0069] Extracting second feature data of the non-visual modality data by a second convolutional neural network;

[0070] Performing feature fusion on the first feature data and the second feature data to obtain fused feature data;

[0071] The fused feature data is input into the action recognition model to determine the action data of the target person within the preset range.

[0072] For example, in order to avoid the problem of inaccurate recognition using a single form of data, multiple forms of data are fused and used together as the basis for action recognition, thereby ensuring the accuracy of action recognition.

[0073] S104, determining a target object corresponding to the action data based on the action data, the surface feature and a preset surface feature database, and determining behavior information of the target person based on the target object, the action data and preset behavior data.

[0074] In a possible implementation, the step of determining a target object corresponding to the action data based on the action data, the surface feature, and a preset surface feature database, and determining behavior information of the target person based on the target object, the action data, and preset behavior data includes:

[0075] Selecting, in the preset surface feature database, a set of preset surface features of different objects corresponding to the action data based on the action data;

[0076] Matching the surface feature with a preset surface feature in the preset surface feature set to obtain a preset surface feature that matches the surface feature, and taking an object corresponding to the preset surface feature as a target object;

[0077] The preset behavior data is searched for a preset behavior type corresponding to the target object and the action data as the behavior information of the target person.

[0078] Exemplarily, the surface features of different objects under different actions are obtained, and then a preset surface feature set is constructed according to the surface features of different objects under different actions. For example, when the action data is tipping, the preset surface features of different objects when tipping occurs are searched in the preset surface feature database to obtain a preset surface feature set, and the preset surface features of different objects are matched with the surface features, for example, the preset surface features of the ground are matched with the actual surface features of the ground, and the preset surface features of the sofa are matched with the actual surface features of the sofa, and the object corresponding to the preset surface feature is used as the target object.

[0079] Exemplarily, the preset behavior data includes different actions occurring on different objects. For example, the action of falling on the bed is called lying, and the action of falling on the bed is called falling.

[0080] The image data of a preset range is obtained through the edge computing device; the objects in the target area are segmented according to the image data, and the surface features of the segmented objects are obtained; the monitoring data corresponding to the image data is obtained through the edge computing device, and the action data of the target person in the preset range is judged through the action recognition model; the target object corresponding to the action data is determined based on the action data, the surface features and the preset surface feature database, and the behavior information of the target person is determined based on the target object, the action data and the preset behavior data. The behavior of the target person is determined by combining the surface features and the action data of the target person, which improves the accuracy of behavior recognition, only obtains the image data and monitoring data within the preset range, avoids excessive exposure of unnecessary personal space, and strengthens the protection of personal privacy.

[0081] In a possible implementation, before the step of acquiring image data of a preset range through an edge computing device, the method further includes:

[0082] Obtaining identity information of the edge computing device, and matching the identity information with preset authorization data;

[0083] If the identity information matches the preset authorization data, then continue to execute the step of acquiring image data of a preset range through the edge computing device;

[0084] If the identity information does not match the preset permission data, obtaining permission information of the edge computing device;

[0085] When the permission information meets the preset conditions, the edge computing device is authorized through a preset authorization method. The authorized edge computing device is used to obtain image data within a preset range. The preset authorization method includes a local authorization method, a formal authorization method, and a remote authorization method.

[0086] Exemplary, local authorization method: local authorization operation is performed through Bluetooth connection; formal authorization method: requires the system or device administrator to enter a password or give formal authorization; remote authorization method: supports remote authorization operations through the network, which also requires the formal permission of the administrator.

[0087] Exemplarily, the authorization method can also be a combination of local authorization and remote authorization. For example, the administrator user performs camera calibration or background acquisition mode settings within the Bluetooth range of the camera, ensuring the safety of operation and the correct configuration of the camera. Once the camera calibration is completed, the administrator user can set the system to support remote operation, that is, allowing background acquisition or camera calibration operations outside the Bluetooth range of the camera.

[0088] In a possible implementation manner, before the step of segmenting the objects in the target area according to the image data and acquiring the surface features of the segmented objects, the method further includes:

[0089] Segmenting the image data into a human body image and a background image;

[0090] Identify the human body image, determine the naked area and the clothed area in the human body image, and perform image privacy processing on the naked area to obtain a target human body image;

[0091] Performing background privacy processing on the background image to obtain a target background image;

[0092] Target image data is obtained according to the target human image and the target background image, and the target image data is used to segment objects in the target area.

[0093] Exemplarily, the process of background privacy processing can be: displaying the edges of objects in solid colors, simplifying the background, making the edges of objects in the monitoring screen clear, and presenting the background in a single color, thereby reducing exposure to privacy information; the process of background privacy processing can also be: reconstructing the shape of the object through a 3D mesh model, and highlighting the spatial sense of the object and the spatial sense of the background environment in solid colors or various colorful display states, while protecting privacy, retaining the three-dimensional spatial information of the scene, which helps the monitoring system to understand and analyze the scene more accurately.

[0094] Exemplarily, the image privacy processing can be implemented in a variety of technical ways, including but not limited to:

[0095] Mosaic method: Hide details by covering key parts with mosaics. Solid color or color coverage: Use solid color or color to cover key parts while retaining the outline and movement information of the human body. Edge processing: Use machine vision technology to distinguish the edges of the person to protect the privacy of the monitoring area environment. 3D mesh model reconstruction: Use 3D mesh model to reconstruct the human body shape and display the various parts or torso of the human body in solid color or various colors, which not only protects privacy but also retains three-dimensional spatial information. Data desensitization technology: Involves processing sensitive data to reduce or eliminate the risk of leakage of personal privacy information. In the field of images and videos, this usually means blurring or pixelating sensitive areas in the image. Anonymization processing: Privacy protection is achieved by removing or replacing identifiable information in images and videos. Applicable to images and videos that need to be published or shared publicly, but do not want to expose the information of individuals or specific entities. Encryption technology: An effective means to protect image and video data from unauthorized access. By using encryption algorithms, only individuals who know the correct key can decrypt and access the original content. Differential privacy: By making small changes to the original data (adding noise) to mask the details of individual inputs while maintaining the interpretability of the data. Zero-knowledge proof: A simple question of whether something is true or false can be answered without revealing any other information.

[0096] Exemplarily, in certain scenarios, based on specific user requests, you can choose to perform privacy processing on human body images and / or background images to obtain target image data, or you can choose not to perform privacy processing on human body images and / or background images and retain the image data based on specific user requests.

[0097] In a possible implementation, the monitoring data includes at least one of visual modality data and non-visual modality data, and the step of determining the motion data of the target person within the preset range through a motion recognition model based on the monitoring data includes:

[0098] When the monitoring data includes the visual modality data and the non-visual modality data, extracting first feature data of the visual modality data by a first convolutional neural network;

[0099] Extracting second feature data of the non-visual modality data by a second convolutional neural network;

[0100] Performing feature fusion on the first feature data and the second feature data to obtain fused feature data;

[0101] The fused feature data is input into the action recognition model to determine the action data of the target person within the preset range.

[0102] Exemplarily, the visual modality data includes but is not limited to images in standard image format, skeleton data, depth information, infrared images, point cloud data, event streams, etc.; the non-visual modality data includes but is not limited to audio signals, acceleration data, radar signals, wireless (WiFi) signals, etc.

[0103] Exemplarily, when the monitoring data includes only one of the visual modality data and the non-visual modality data, the data type of the monitoring data is identified as the target type, and based on the target type, the convolutional neural network corresponding to the monitoring data is selected as the target convolutional neural network in the first convolutional neural network and the second convolutional neural network, and feature data of the monitoring data is extracted through the target convolutional neural network. Based on the feature data of the monitoring data, the action data of the target person within the preset range is judged by a motion recognition model trained with the data of the target type.

[0104] In a possible implementation, the method further includes:

[0105] Determining the damage level of the target person according to the behavior information and preset classification rules;

[0106] If the damage level is greater than or equal to the preset level, the behavior information is sent to the first receiving end, and alarm information is generated based on the damage level, the behavior information and the location information of the target person, and the alarm information is sent to the second receiving end.

[0107] Exemplarily, when the behavior information is a bump, the damage level corresponding to the bump is determined according to the preset classification rules. If the damage level corresponding to the bump is greater than or equal to the preset level, the behavior information is sent to a first receiving end, which can be understood as the guardian end of the target person, so that the guardian can timely understand the dynamics of the target person; based on the damage level, the behavior information and the location information of the target person, an alarm message is generated, and the alarm message is sent to a second receiving end, which can be understood as a medical end. Through the alarm information, medical staff can timely understand the location of the target person, the behavior, that is, the degree of damage, so as to provide timely assistance to the target person.

[0108] Exemplarily, before the step of generating alarm information based on the damage level, the behavior information and the location information of the target person, it also includes: obtaining request information fed back by the first receiving end, and if the request information is used to represent the cancellation of the alarm information, sending a notice of cancellation of treatment to the second receiving end to avoid wasting medical resources.

[0109] In a possible implementation, Figure 2 As shown, an embodiment of the present application provides an edge separation-based action recognition device, the device comprising:

[0110] The data acquisition module 201 is used to obtain image data of a preset range through an edge computing device;

[0111] A segmentation module 202 is used to segment objects in the target area according to the image data and obtain surface features of the segmented objects;

[0112] A first recognition module 203 is used to obtain monitoring data corresponding to the image data through an edge computing device, and determine the motion data of the target person within the preset range through a motion recognition model based on the monitoring data;

[0113] The second recognition module 204 is used to determine the target object corresponding to the action data based on the action data, the surface features and a preset surface feature database, and to determine the behavior information of the target person based on the target object, the action data and preset behavior data.

[0114] The image data of a preset range is obtained through the edge computing device; the objects in the target area are segmented according to the image data, and the surface features of the segmented objects are obtained; the monitoring data corresponding to the image data is obtained through the edge computing device, and the action data of the target person in the preset range is judged through the action recognition model; the target object corresponding to the action data is determined based on the action data, the surface features and the preset surface feature database, and the behavior information of the target person is determined based on the target object, the action data and the preset behavior data. The behavior of the target person is determined by combining the surface features and the action data of the target person, which improves the accuracy of behavior recognition, only obtains the image data and monitoring data within the preset range, avoids excessive exposure of unnecessary personal space, and strengthens the protection of personal privacy.

[0115] In a possible implementation, the data acquisition module 201 is used to:

[0116] Obtaining identity information of the edge computing device, and matching the identity information with preset authorization data;

[0117] If the identity information matches the preset authorization data, then continue to execute the step of acquiring image data of a preset range through the edge computing device;

[0118] If the identity information does not match the preset permission data, obtaining permission information of the edge computing device;

[0119] When the permission information meets the preset conditions, the edge computing device is authorized through a preset authorization method. The authorized edge computing device is used to obtain image data within a preset range. The preset authorization method includes a local authorization method, a formal authorization method, and a remote authorization method.

[0120] In a possible implementation, the segmentation module 202 is used to:

[0121] Segmenting the objects in the image data using semantic segmentation technology to obtain an initial image of the object;

[0122] Identifying the edge of the object in the initial image of the object by edge detection technology, and marking the edge to obtain the segmented object image;

[0123] Determining surface data of the object in the object image according to a surface detection technique;

[0124] The surface data is segmented by deep learning technology to obtain the surfaces of the segmented objects, and the area data of each surface is calculated as the surface features of the segmented objects.

[0125] In a possible implementation, the monitoring data includes at least one of visual modality data and non-visual modality data.

[0126] In a possible implementation manner, the first identification module 203 is used to:

[0127] When the monitoring data includes the visual modality data and the non-visual modality data, extracting first feature data of the visual modality data by a first convolutional neural network;

[0128] Extracting second feature data of the non-visual modality data by a second convolutional neural network;

[0129] Performing feature fusion on the first feature data and the second feature data to obtain fused feature data;

[0130] The fused feature data is input into the action recognition model to determine the action data of the target person within the preset range.

[0131] In a possible implementation manner, the second identification module 204 is configured to:

[0132] Selecting, in the preset surface feature database, a set of preset surface features of different objects corresponding to the action data based on the action data;

[0133] Matching the surface feature with a preset surface feature in the preset surface feature set to obtain a preset surface feature that matches the surface feature, and taking an object corresponding to the preset surface feature as a target object;

[0134] The preset behavior data is searched for a preset behavior type corresponding to the target object and the action data as the behavior information of the target person.

[0135] In a possible implementation manner, the second identification module 204 is configured to:

[0136] Determining the damage level of the target person according to the behavior information and preset classification rules;

[0137] If the damage level is greater than or equal to the preset level, the behavior information is sent to the first receiving end, and alarm information is generated based on the damage level, the behavior information and the location information of the target person, and the alarm information is sent to the second receiving end.

[0138] In a possible implementation, Figure 3As shown, an embodiment of the present application provides an electronic device 300, including: a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, it implements: obtaining image data of a preset range through an edge computing device; segmenting objects in the target area according to the image data, and obtaining surface features of the segmented objects; obtaining monitoring data corresponding to the image data through an edge computing device, and judging the action data of a target person within the preset range through a motion recognition model; determining a target object corresponding to the action data based on the action data, the surface features, and a preset surface feature database, and determining the behavior information of the target person based on the target object, the action data, and preset behavior data.

[0139] The image data of a preset range is obtained through the edge computing device; the objects in the target area are segmented according to the image data, and the surface features of the segmented objects are obtained; the monitoring data corresponding to the image data is obtained through the edge computing device, and the action data of the target person in the preset range is judged through the action recognition model; the target object corresponding to the action data is determined based on the action data, the surface features and the preset surface feature database, and the behavior information of the target person is determined based on the target object, the action data and the preset behavior data. The behavior of the target person is determined by combining the surface features and the action data of the target person, which improves the accuracy of behavior recognition, only obtains the image data and monitoring data within the preset range, avoids excessive exposure of unnecessary personal space, and strengthens the protection of personal privacy.

[0140] In a possible implementation, Figure 4 As shown, an embodiment of the present application provides a computer-readable storage medium 400, on which a computer program 411 is stored. When the computer program 411 is executed by a processor, it implements: obtaining image data of a preset range through an edge computing device; segmenting objects in the target area according to the image data, and obtaining surface features of the segmented objects; obtaining monitoring data corresponding to the image data through an edge computing device, and judging the motion data of a target person within the preset range through a motion recognition model; determining a target object corresponding to the motion data based on the motion data, the surface features and a preset surface feature database, and determining the behavior information of the target person based on the target object, the motion data and preset behavior data.

[0141] The image data of a preset range is obtained through the edge computing device; the objects in the target area are segmented according to the image data, and the surface features of the segmented objects are obtained; the monitoring data corresponding to the image data is obtained through the edge computing device, and the action data of the target person in the preset range is judged through the action recognition model; the target object corresponding to the action data is determined based on the action data, the surface features and the preset surface feature database, and the behavior information of the target person is determined based on the target object, the action data and the preset behavior data. The behavior of the target person is determined by combining the surface features and the action data of the target person, which improves the accuracy of behavior recognition, only obtains the image data and monitoring data within the preset range, avoids excessive exposure of unnecessary personal space, and strengthens the protection of personal privacy.

[0142] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0143] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0144] The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, etc., or any suitable combination of the foregoing.

[0145] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0146] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0147] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described above, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

[0148] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. An action recognition method based on edge separation, characterized in that: include: Acquire image data of a preset range through an edge computing device; Segmenting objects in the target area according to the image data, and acquiring surface features of the segmented objects; Acquire monitoring data corresponding to the image data through an edge computing device, and determine motion data of a target person within a preset range through a motion recognition model based on the monitoring data; A target object corresponding to the action data is determined based on the action data, the surface features and a preset surface feature database, and behavior information of the target person is determined based on the target object, the action data and preset behavior data.

2. The action recognition method based on edge separation according to claim 1, characterized in that: The step of acquiring image data of a preset range through an edge computing device includes: Obtaining identity information of the edge computing device, and matching the identity information with preset authorization data; If the identity information matches the preset authorization data, then continue to execute the step of acquiring image data of a preset range through the edge computing device; If the identity information does not match the preset permission data, obtaining permission information of the edge computing device; When the permission information meets the preset conditions, the edge computing device is authorized through a preset authorization method. The authorized edge computing device is used to obtain image data within a preset range. The preset authorization method includes a local authorization method, a formal authorization method, and a remote authorization method.

3. The action recognition method based on edge separation as claimed in claim 1, characterized in that: The step of segmenting the objects in the target area according to the image data and acquiring surface features of the segmented objects comprises: Segmenting the objects in the image data using semantic segmentation technology to obtain an initial image of the object; Identifying the edge of the object in the initial image of the object by edge detection technology, and marking the edge to obtain the segmented object image; Determining surface data of the object in the object image according to a surface detection technique; The surface data is segmented by deep learning technology to obtain the surfaces of the segmented objects, and the area data of each surface is calculated as the surface features of the segmented objects.

4. The method for motion recognition based on edge separation according to claim 1, characterized in that: Before the step of segmenting the objects in the target area according to the image data and obtaining the surface features of the segmented objects, the method further includes: Segmenting the image data into a human body image and a background image; Identify the human body image, determine the naked area and the clothed area in the human body image, and perform image privacy processing on the naked area to obtain a target human body image; Performing background privacy processing on the background image to obtain a target background image; Target image data is obtained according to the target human image and the target background image, and the target image data is used to segment objects in a target area.

5. The method for motion recognition based on edge separation as claimed in claim 1, characterized in that: The monitoring data includes at least one of visual modality data and non-visual modality data, and the step of determining the motion data of the target person within the preset range through the motion recognition model based on the monitoring data includes: When the monitoring data includes the visual modality data and the non-visual modality data, extracting first feature data of the visual modality data by a first convolutional neural network; Extracting second feature data of the non-visual modality data by a second convolutional neural network; Performing feature fusion on the first feature data and the second feature data to obtain fused feature data; The fused feature data is input into the action recognition model to determine the action data of the target person within the preset range.

6. The method for motion recognition based on edge separation according to claim 1, characterized in that: The step of determining a target object corresponding to the action data based on the action data, the surface feature and a preset surface feature database, and determining behavior information of the target person based on the target object, the action data and preset behavior data comprises: Selecting, based on the action data, a preset surface feature set of different objects corresponding to the action data in the preset surface feature database; Matching the surface feature with a preset surface feature in the preset surface feature set to obtain a preset surface feature that matches the surface feature, and taking an object corresponding to the preset surface feature as a target object; The preset behavior data is searched for a preset behavior type corresponding to the target object and the action data as the behavior information of the target person.

7. The method for motion recognition based on edge separation according to claim 1, characterized in that: The method further comprises: Determining the damage level of the target person according to the behavior information and preset classification rules; If the damage level is greater than or equal to the preset level, the behavior information is sent to the first receiving end, and alarm information is generated based on the damage level, the behavior information and the location information of the target person, and the alarm information is sent to the second receiving end.

8. An edge separation-based motion recognition device, characterized in that: The device comprises: A data acquisition module, used to obtain image data of a preset range through an edge computing device; A segmentation module, used to segment objects in the target area according to the image data, and obtain surface features of the segmented objects; A first recognition module is used to obtain monitoring data corresponding to the image data through an edge computing device, and determine the motion data of the target person within the preset range through a motion recognition model; The second recognition module is used to determine the target object corresponding to the action data based on the action data, the surface features and a preset surface feature database, and to determine the behavior information of the target person based on the target object, the action data and preset behavior data.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the edge separation-based action recognition method as described in any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the edge separation-based action recognition method according to any one of claims 1 to 7 are executed.

Images