Personnel behavior identification method and device based on three-dimensional target detection

Abstract

The invention discloses a personnel behavior recognition method and device based on three-dimensional target detection, and the method comprises the steps: obtaining a to-be-recognized image of a target space, and enabling the to-be-recognized image to be a two-dimensional image; performing three-dimensional target detection on the to-be-recognized image, and determining a three-dimensional frame of a corresponding person; determining a plurality of target points of the three-dimensional frame, analyzing three-dimensional coordinates of the target points, and performing point location matching to obtain orientation features of the three-dimensional frame; and on the basis of analysis and comparison of orientation features of the three-dimensional frame, identification of personnel behaviors is completed. Three-dimensional target detection is performed on the to-be-recognized image to obtain the three-dimensional frame of the personnel in the target space, and the orientation features are extracted from the three-dimensional frame for analysis and comparison, so that the behavior recognition of the personnel in the target space is completed, the influence of visual difference caused by an imaging angle on behavior recognition is reduced, and the detection precision of behavior detection is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of behavior detection, and in particular relates to a method and device for human behavior recognition based on three-dimensional target detection. Background Art

[0002] Distribution stations are a crucial component of the power system, located at the end of the power grid, connecting to substations above and various power-consuming devices below. They convert high-voltage electricity into low-voltage energy and deliver it to various users via distribution facilities, lines, and equipment. They also ensure stable power supply and safe use.

[0003] When workers are working in the high-voltage or ultra-high-voltage environment of the power distribution room, they may suffer accidents such as electric shock. How to detect abnormal behavior of workers in a timely manner is a current problem.

[0004] At present, the behavior detection of staff in the distribution room mainly focuses on fall detection. The relevant technologies are mainly divided into two directions. One is based on target detection, which targets staff and then classifies images of the detected personnel to determine whether there are falls. The other is based on posture recognition, which detects the joint points of the personnel and classifies the multi-dimensional matrix composed of the positions of the joint points to determine whether there are falls.

[0005] Since the direction in which the staff falls is not fixed and the angle of the staff being photographed is fixed, there is visual difference caused by the imaging angle. How to deal with the visual difference caused by the imaging angle and improve the detection accuracy of behavior detection is a problem that needs to be solved at present. Summary of the Invention

[0006] In response to the defects in the above-mentioned prior art, the present invention provides a method and device for identifying human behavior based on three-dimensional target detection. The method includes: obtaining an image to be identified in a target space, wherein the image to be identified is a two-dimensional image; performing three-dimensional target detection on the image to be identified to determine a three-dimensional frame of the corresponding person; determining multiple target points in the three-dimensional frame, analyzing the three-dimensional coordinates of the target points and performing point matching to obtain the orientation features of the three-dimensional frame; and completing the identification of human behavior based on the analysis and comparison of the orientation features of the three-dimensional frame. By performing three-dimensional target detection on the image to be identified to obtain a three-dimensional frame of the person in the target space, and extracting orientation features from the three-dimensional frame for analysis and comparison, the identification of human behavior in the target space is completed, thereby reducing the impact of visual differences caused by imaging angles on behavior recognition and improving the detection accuracy of behavior detection.

[0007] In one aspect, the present invention provides a method for human behavior recognition based on three-dimensional target detection, comprising:

[0008] Acquire an image to be identified in the target space, wherein the image to be identified is a two-dimensional image;

[0009] Perform 3D target detection on the image to be identified and determine the 3D frame of the corresponding person;

[0010] Determine multiple target points in the 3D frame, analyze the 3D coordinates of the target points and perform point matching to obtain the orientation characteristics of the 3D frame;

[0011] Based on the analysis and comparison of the orientation features of the three-dimensional framework, the identification of personnel behavior is completed.

[0012] In one possible implementation, performing three-dimensional object detection on the image to be recognized and determining the three-dimensional frame of the corresponding person specifically includes:

[0013] Perform 3D target detection on the image to be identified and give a rectangular detection frame of the corresponding person;

[0014] Select a point from the vertices of the cuboid detection box as the origin, and determine the three-dimensional coordinate system based on the origin and the edges connected to the origin;

[0015] Based on the three-dimensional coordinate system, the three-dimensional coordinates of each vertex on the rectangular detection frame are given to determine the three-dimensional frame of the corresponding person.

[0016] In one possible implementation, based on a three-dimensional coordinate system, the three-dimensional coordinates of each vertex on the cuboid detection frame are given to determine the three-dimensional frame of the corresponding person, specifically including:

[0017] Based on the three-dimensional coordinate system, the coordinates of the points on the coordinate axis of the cuboid detection box are given;

[0018] The coordinates of the points on the coordinate axis of the cuboid detection frame are analyzed to obtain the three-dimensional coordinates of other vertices on the cuboid detection frame and determine the three-dimensional frame.

[0019] In one possible implementation, the orientation feature of the three-dimensional frame includes at least one of a length feature, a height feature, and a width feature;

[0020] Determine multiple target points in the 3D frame, analyze the 3D coordinates of the target points and perform point matching to obtain the orientation characteristics of the 3D frame, including:

[0021] Give multiple target points in the three-dimensional frame and the three-dimensional coordinates of each target point;

[0022] Normalize the three-dimensional coordinates to obtain standard coordinates;

[0023] Point matching is performed according to standard coordinates to obtain length features, height features and width features.

[0024] In a possible implementation, point matching is performed according to standard coordinates to obtain length features, height features, and width features, specifically including:

[0025] According to the standard coordinates, the maximum and minimum values ​​of the vertices of the three-dimensional frame on each coordinate axis are obtained;

[0026] Determine the matching points based on the maximum and minimum values ​​on each coordinate axis;

[0027] Based on the matching points, length features, height features and width features are obtained.

[0028] In one possible implementation, the identification of human behavior is completed based on the analysis and comparison of the orientation features of the three-dimensional frame, specifically including:

[0029] Based on the high feature in the orientation feature, the maximum height difference in the orientation feature is used as the first eigenvalue;

[0030] The maximum value of the length feature and the width feature in the orientation feature is taken as the second eigenvalue;

[0031] The first eigenvalue and the second eigenvalue are compared to determine a recognition result.

[0032] In a possible implementation, the first eigenvalue is specifically expressed as:

[0033] h=max(abs(z1-z2),abs(z2-z3),abs(z1-z3))

[0034] Among them, h is the first eigenvalue, abs() is the absolute value function, and z1, z2, and z3 are the coordinate values ​​of the target point on the z-axis.

[0035] In a possible implementation, the second eigenvalue is specifically expressed as:

[0036] γ=max(abs(x1-x2),abs(x2-x3),abs(x1-x3),abs(y1

[0037] -y2),abs(y2-y3),abs(y1-y3))

[0038] Among them, γ is the second eigenvalue, abs() is the absolute value function, x1, x2, x3 are the coordinate values ​​of the target point on the x-axis, and y1, y2, y3 are the coordinate values ​​of the target point on the y-axis.

[0039] In one possible implementation, comparing the first eigenvalue and the second eigenvalue to determine the recognition result specifically includes:

[0040] When the first characteristic value is less than the second characteristic value, it is determined that the recognition result is that a person has fallen;

[0041] When the first characteristic value is greater than or equal to the second characteristic value, it is determined that the recognition result is that there is no person falling.

[0042] On the other hand, the present invention also discloses a human behavior recognition device based on three-dimensional target detection, comprising:

[0043] An image acquisition module is used to acquire an image to be identified in the target space, wherein the image to be identified is a two-dimensional image;

[0044] A 3D detection module is used to perform 3D target detection on the image to be identified and determine the 3D frame of the corresponding person;

[0045] The feature determination module is used to determine multiple target points of the 3D frame, analyze the 3D coordinates of the target points and perform point matching to obtain the orientation features of the 3D frame;

[0046] The feature analysis module is used to identify human behavior based on the analysis and comparison of the orientation features of the three-dimensional framework.

[0047] In one possible implementation, performing three-dimensional object detection on the image to be recognized and determining the three-dimensional frame of the corresponding person specifically includes:

[0048] Perform 3D target detection on the image to be identified and give a rectangular detection frame of the corresponding person;

[0049] Select a point from the vertices of the cuboid detection box as the origin, and determine the three-dimensional coordinate system based on the origin and the edges connected to the origin;

[0050] Based on the three-dimensional coordinate system, the three-dimensional coordinates of each vertex on the rectangular detection frame are given to determine the three-dimensional frame of the corresponding person.

[0051] In one possible implementation, based on a three-dimensional coordinate system, the three-dimensional coordinates of each vertex on the cuboid detection frame are given to determine the three-dimensional frame of the corresponding person, specifically including:

[0052] Based on the three-dimensional coordinate system, the coordinates of the points on the coordinate axis of the cuboid detection box are given;

[0053] The coordinates of the points on the coordinate axis of the cuboid detection frame are analyzed to obtain the three-dimensional coordinates of other vertices on the cuboid detection frame and determine the three-dimensional frame.

[0054] In one possible implementation, the orientation feature of the three-dimensional frame includes at least one of a length feature, a height feature, and a width feature;

[0055] Determine multiple target points in the 3D frame, analyze the 3D coordinates of the target points and perform point matching to obtain the orientation characteristics of the 3D frame, including:

[0056] Give multiple target points in the three-dimensional frame and the three-dimensional coordinates of each target point;

[0057] Normalize the three-dimensional coordinates to obtain standard coordinates;

[0058] Point matching is performed according to standard coordinates to obtain length features, height features and width features.

[0059] In a possible implementation, point matching is performed according to standard coordinates to obtain length features, height features, and width features, specifically including:

[0060] According to the standard coordinates, the maximum and minimum values ​​of the vertices of the three-dimensional frame on each coordinate axis are obtained;

[0061] Determine the matching points based on the maximum and minimum values ​​on each coordinate axis;

[0062] Based on the matching points, length features, height features and width features are obtained.

[0063] In one possible implementation, the identification of human behavior is completed based on the analysis and comparison of the orientation features of the three-dimensional frame, specifically including:

[0064] Based on the high feature in the orientation feature, the maximum height difference in the orientation feature is used as the first eigenvalue;

[0065] The maximum value of the length feature and the width feature in the orientation feature is taken as the second eigenvalue;

[0066] The first eigenvalue and the second eigenvalue are compared to determine a recognition result.

[0067] In a possible implementation, the first eigenvalue is specifically expressed as:

[0068] h=max(abs(z1-z2),abs(z2-z3),abs(z1-z3))

[0069] Among them, h is the first eigenvalue, abs() is the absolute value function, and z1, z2, and z3 are the coordinate values ​​of the target point on the z-axis.

[0070] In a possible implementation, the second eigenvalue is specifically expressed as:

[0071] γ=max(abs(x1-x2),abs(x2-x3),abs(x1-x3),abs(y1-y2),abs(y2-y3),abs(y1-y3))

[0072] Among them, γ is the second eigenvalue, abs() is the absolute value function, x1, x2, x3 are the coordinate values ​​of the target point on the x-axis, and y1, y2, y3 are the coordinate values ​​of the target point on the y-axis.

[0073] In one possible implementation, comparing the first eigenvalue and the second eigenvalue to determine the recognition result specifically includes:

[0074] When the first characteristic value is less than the second characteristic value, it is determined that the recognition result is that a person has fallen;

[0075] When the first characteristic value is greater than or equal to the second characteristic value, it is determined that the recognition result is that there is no person falling.

[0076] The present invention provides a method and device for human behavior recognition based on three-dimensional target detection, which has at least the following beneficial effects:

[0077] By performing three-dimensional target detection on the image to be identified, a three-dimensional frame of the person in the target space is obtained, and orientation features are extracted from the three-dimensional frame for analysis and comparison to complete the identification of the person's behavior. This reduces the impact of visual differences caused by imaging angles on behavior recognition and improves the detection accuracy of behavior detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 A flowchart of a method for human behavior recognition based on three-dimensional target detection provided by an embodiment of the present invention;

[0079] Figure 2 An example diagram of a rectangular parallelepiped detection frame provided by an embodiment of the present invention;

[0080] Figure 3 A schematic diagram of the structure of a rectangular parallelepiped detection frame provided by an embodiment of the present invention;

[0081] Figure 4 This is a structural block diagram of a method and device for human behavior recognition based on three-dimensional target detection provided by an embodiment of the present invention.

[0082] Among them, 201 is an image acquisition module; 202 is a three-dimensional detection module; 203 is a feature determination module; and 204 is a feature analysis module. DETAILED DESCRIPTION

[0083] To better understand the above technical solution, the following will be described in detail with reference to the accompanying drawings and specific implementation methods. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0084] The terms used in the embodiments of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The singular forms "a," "an," "the," and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, and unless the context clearly indicates otherwise, "a plurality" generally includes at least two.

[0085] It should also be noted that the terms "include," "comprises," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a product or device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such product or device. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the product or device comprising the element.

[0086] Distribution stations play a crucial role in the power system. Using transformers, they convert high-voltage electricity into low-voltage power suitable for users, ensuring their regular access to electricity. In modern society, various facilities, such as hospitals, schools, and transportation hubs, require electricity, making distribution stations a crucial component of urban infrastructure. Furthermore, distribution stations can automatically adjust voltage to ensure stable and high-quality power supply even when user power consumption fluctuates dramatically. Furthermore, protective devices within distribution stations can detect power system faults and take timely action to prevent power outages, thereby ensuring safe electricity use.

[0087] The safety of the distribution substation building, where a power distribution station is located, not only directly impacts the safety of distribution facilities but also impacts the user experience and safety of electricity use. Security inspections in distribution substations have always been a key security priority in power systems, and increasingly sophisticated security inspection systems constantly safeguard the safety of equipment and personnel within distribution substations.

[0088] With the development of surveillance technology, a variety of monitoring devices are deployed in distribution substations to monitor the status of equipment and personnel, and to detect abnormal behavior. To more accurately detect abnormal behavior, monitoring devices equipped with depth sensors are being used, but most distribution substations still use standard monitoring devices. Improving the visual differences caused by imaging angles based on images captured by standard monitoring devices and improving the accuracy of behavior detection are currently challenges.

[0089] like Figure 1 As shown, the embodiment of the present invention provides a method for human behavior recognition based on three-dimensional target detection, and the specific steps are as follows:

[0090] S101: Acquire an image to be recognized in a target space.

[0091] Specifically, the image to be identified is a two-dimensional image. In the example provided by the present invention, the target space is a power distribution station. In other embodiments, it can be other areas or locations where human behavior recognition is required, and this is not limited to this. It should be understood that the above-mentioned image to be identified is collected by a traditional power distribution station camera, and there is no need to use a camera with a depth sensor. The present invention does not require special hardware support during implementation, which reduces the large-scale deployment of equipment and also reduces the cost of the solution.

[0092] S102: Perform three-dimensional target detection on the image to be recognized to determine the three-dimensional frame of the corresponding person.

[0093] Specifically, 3D object detection is performed on the image to be identified, generating a rectangular detection box corresponding to the personnel in the distribution station. A point is selected from the vertices of the rectangular detection box as the origin, and a 3D coordinate system is determined based on the origin and the edges connecting it. Based on this 3D coordinate system, the 3D coordinates of each vertex on the rectangular detection box are determined to determine the 3D frame corresponding to the personnel in the distribution station.

[0094] Furthermore, based on the three-dimensional coordinate system, the coordinates of the points on the coordinate axis of the cuboid detection frame are given. The coordinates of the points on the coordinate axis of the cuboid detection frame are analyzed to obtain the three-dimensional coordinates of other vertices on the cuboid detection frame and determine the three-dimensional frame.

[0095] Reference Figure 2 The figure shows a real-life image of the staff in the distribution station, that is, the image captured by the camera when the staff is working normally in the distribution station. By performing three-dimensional target detection on the image, we can obtain the rectangular detection frame corresponding to the staff in the distribution station.

[0096] In a specific example, referring to Figure 3In a typical two-dimensional image, a two-dimensional plane coordinate system is generally established with the upper left corner of the image as the origin. The same method can be used to determine the origin when establishing a three-dimensional coordinate system. In other embodiments, other vertices can be used as the origin, and the X-axis, Y-axis, and Z-axis are determined based on the distance from the origin to the edge of the rectangular parallelepiped detection frame. It can be understood that the spatial coordinates of any point on a surface perpendicular to a coordinate axis and having only one intersection with the coordinate axis have the same coordinate value corresponding to that coordinate axis. For example, the OAao and CBbc surfaces are both perpendicular to the X-axis and have only one intersection with the X-axis, namely, points O and C. Therefore, the x-axis coordinates of all points on the OAao surface are the same as the x-coordinate value of point O, and the x-axis coordinates of all points on the CBbc surface are the same as the x-coordinate value of point C. Similarly, the z-axis coordinates of all points on the OABC and oabc surfaces, and the y-axis coordinates of all points on the AaBb and OoCc surfaces can be obtained. This can be used to obtain the coordinates of the eight vertices of the rectangular parallelepiped detection frame. At this point, the three-dimensional coordinate system has been established.

[0097] It should be noted that in this embodiment, the 3D object detection process described above is implemented using the ImVoxelNet algorithm. Because the objects in the image to be identified are personnel in a power distribution station, which is a common phenomenon, algorithm training does not require actual data collection in the power distribution station. Model training can be performed using the publicly available data used in the ImVoxelNet algorithm, and the trained ImVoxelNet model is then used for 3D object detection. 3D object detection generates eight vertices, which define a spatial framework, or 3D frame.

[0098] In a specific embodiment, the ImVoxelNet algorithm is used to perform three-dimensional target detection on the image to be identified. For each person, the ImVoxelNet algorithm outputs the three-dimensional coordinates of eight points, and the corresponding three-dimensional coordinates are (x O ,y O ,z O ), (x A ,y A ,z A ), (x B ,y B ,z B ), (x C ,y C ,z C ), (x o ,y o ,z o ), (x a ,y a ,z a ), (x b ,y b,z b ), (x c ,y c ,z c ). The cuboid detection frame is obtained based on the coordinates of the eight points.

[0099] S103: Determine multiple target points of the three-dimensional frame, analyze the three-dimensional coordinates of the target points and perform point matching to obtain the orientation features of the three-dimensional frame.

[0100] Specifically, the orientation feature of the 3D frame includes at least one of a length feature, a height feature, and a width feature. Multiple target points in the 3D frame and the corresponding 3D coordinates of each target point are given. The 3D coordinates are normalized to obtain standard coordinates. Point matching is performed based on the standard coordinates to obtain the length feature, the height feature, and the width feature.

[0101] Furthermore, based on the standard coordinates, the maximum and minimum values ​​of the vertices of the 3D frame on each coordinate axis are obtained. Based on the maximum and minimum values ​​on each coordinate axis, matching points are determined. Based on the matching points, length features, height features, and width features are obtained.

[0102] In a specific example, by (x min ,y min ,z min ) conditional filter matches point O, through (x max ,y max ,z min ) condition to match point B, through (x max ,y max ,z max ) condition to match point b, thus determining the three matching points on the 3D frame. min =min(x O ,x A ,x B ,x C ,x o ,x a ,x b ,x c ), x max =max(x O ,x A ,x B ,x C ,x o ,x a ,x b ,x c ),

[0103] y min =min(y O ,y A ,y B ,yC ,y o ,y a ,y b ,y c ), y max =max(y O ,y A ,y B ,y C ,y o ,y a ,y b ,y c ), z min =min(z O ,z A ,z B ,z C ,z o ,z a ,z b ,z c ), z max =max(z O ,z A ,z B ,z C ,z o ,z a ,z b ,z c ). According to the above matching process, three matching points are obtained. The coordinates of the three matching points are (x min ,y min ,z min )、(x max ,y max ,z min ) and (x max ,y max ,z max ). (x min ,y min ,z min ) and (x max ,y max ,z min ) is considered as the bottom of the cuboid detection frame, so the side length of the bottom corresponds to the length and width of the cuboid detection frame. max ,y max ,z max The plane parallel to the bottom plane where point b is located is the top plane. By calculating the height difference between the top and bottom planes, we can get the height (height feature) of the cuboid detection frame. In the same way, we can get the length and width features.

[0104] Since the detection target is a person, each individual has certain differences in height, body shape, etc. Therefore, the three-dimensional frame directly detected may have large differences due to different objects. The three-dimensional frame with large differences has a certain impact on the accuracy of the subsequent process. Therefore, the three-dimensional frame obtained by the three-dimensional target detection needs to be normalized. All position coordinates in the three-dimensional frame are normalized, that is, all position coordinates in the three-dimensional frame are normalized to within 1. In a specific example, all position coordinates in the three-dimensional frame are divided by the maximum value in the detection point, that is, when the x coordinate is normalized, it is divided by the maximum value of the x coordinate value, when the y coordinate is normalized, it is divided by the maximum value of the y coordinate value, and when the z coordinate is normalized, it is divided by the maximum value of the z coordinate value to complete the normalization process.

[0105] Normalize the 3D coordinates of the three matching points. The normalized 3D coordinates are between 0 and 1. Normalization can reduce the difference in pixel size caused by imaging distance and target size.

[0106] It is understandable that for a normal upright person, the height is perpendicular to the ground, and the height of the rectangular detection frame formed is significantly greater than the length and width. When a person squats, the difference between the height, length and width of the rectangular detection frame formed is often not too large. When a person falls, the length or width of the rectangular detection frame formed will be much greater than the height. Therefore, before determining whether the staff member has fallen, point matching is required to obtain the length, height and width features in the three-dimensional frame.

[0107] S104: Based on the analysis and comparison of the orientation features of the three-dimensional frame, the identification of the personnel behavior is completed.

[0108] Specifically, based on the height feature in the orientation feature, the maximum height difference in the orientation feature is used as the first eigenvalue. The maximum value of the length feature and the width feature in the orientation feature is used as the second eigenvalue. The first eigenvalue and the second eigenvalue are compared to determine the recognition result.

[0109] Furthermore, the first eigenvalue is specifically expressed as:

[0110] h=max(abs(z1-z2),abs(z2-z3),abs(z1-z3))

[0111] Wherein, h is the first eigenvalue, abs() is the absolute value function, z1, z2, and z3 are the coordinate values ​​of the target point on the z axis. In a specific embodiment, z1, z2, and z3 are the coordinate values ​​of the matching point on the z axis.

[0112] Furthermore, the second eigenvalue is specifically expressed as:

[0113] γ=max(abs(x1-x2),abs(x2-x3),abs(x1-x30,abs(y1

[0114] -y2),abs(y2-y3),abs(y1-y3))

[0115] Wherein, γ is the second eigenvalue, abs() is the absolute value function, x1, x2, and x3 are the coordinate values ​​of the target point on the x-axis, and y1, y2, and y3 are the coordinate values ​​of the target point on the y-axis. In a specific embodiment, x1, x2, and x3 are the coordinate values ​​of the matching point on the x-axis, and y1, y2, and y3 are the coordinate values ​​of the matching point on the y-axis.

[0116] In one specific embodiment, because the camera imaging angle is relatively fixed, under normal circumstances, the person's upright orientation is the same as or similar to the vertical orientation of the image. Considering that a person's movement while standing significantly affects the width and length in three-dimensional space, if a person falls, one of the length or width dimensions may be significantly greater than the height. Therefore, the larger of the length and width is taken as the second eigenvalue, and the first eigenvalue is compared with the second eigenvalue to determine whether a person has fallen.

[0117] Furthermore, when the first eigenvalue is less than the second eigenvalue, the recognition result is determined to be that a person has fallen. When the first eigenvalue is greater than or equal to the second eigenvalue, the recognition result is determined to be that no person has fallen.

[0118] It is understood that after obtaining the orientation characteristics of the three-dimensional frame, calculating the first eigenvalue and the second eigenvalue is to calculate the ratio between the length, width and height of the three-dimensional frame. Based on the first eigenvalue and the second eigenvalue, the relationship between the length, width and height can be determined to determine whether the detected person has fallen. When the first eigenvalue is less than the second eigenvalue, it means that the height of the three-dimensional frame is less than the length or width of the three-dimensional frame, and the person has fallen. When the first eigenvalue is greater than or equal to the second eigenvalue, it means that the height of the three-dimensional frame is greater than the length or width of the three-dimensional frame, and the person has not fallen.

[0119] Reference Figure 4 , an embodiment of the present invention provides a human behavior recognition device based on three-dimensional target detection, comprising:

[0120] An image acquisition module 201 is used to acquire an image to be identified in a target space, wherein the image to be identified is a two-dimensional image;

[0121] A three-dimensional detection module 202 is used to perform three-dimensional object detection on the image to be recognized and determine the three-dimensional frame of the corresponding person;

[0122] The feature determination module 203 is used to determine multiple target points of the three-dimensional frame, analyze the three-dimensional coordinates of the target points and perform point matching to obtain the orientation features of the three-dimensional frame;

[0123] The feature analysis module 204 is used to identify human behavior based on analysis and comparison of the orientation features of the three-dimensional framework.

[0124] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0125] It should be noted that the computer-readable medium of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0126] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0127] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0128] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified functions or operations, or can be implemented using a combination of dedicated hardware and computer instructions.

[0129] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0130] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the invention. Thus, the present invention is intended to include such changes and modifications as fall within the scope of the claims and their equivalents.

Claims

1. A method for identifying human behavior based on three-dimensional target detection, characterized in that: include: Acquire an image to be identified in the target space, wherein the image to be identified is a two-dimensional image; Perform 3D target detection on the image to be identified and determine the 3D frame of the corresponding person; Determine multiple target points of the three-dimensional frame, analyze the three-dimensional coordinates of the target points and perform point matching to obtain the orientation characteristics of the three-dimensional frame; Based on the analysis and comparison of the orientation features of the three-dimensional framework, the identification of personnel behavior is completed.

2. The method for identifying human behavior based on three-dimensional target detection according to claim 1, characterized in that: Perform 3D target detection on the image to be identified and determine the 3D frame of the corresponding person, including: Perform 3D target detection on the image to be identified and give a rectangular detection frame of the corresponding person; Select a point from the vertices of the cuboid detection box as the origin, and determine a three-dimensional coordinate system based on the origin and the edges connected to the origin; Based on the three-dimensional coordinate system, the three-dimensional coordinates of each vertex on the rectangular detection frame are given to determine the three-dimensional frame of the corresponding person.

3. The method for identifying human behavior based on three-dimensional target detection according to claim 2, characterized in that: Based on the three-dimensional coordinate system, the three-dimensional coordinates of each vertex on the cuboid detection frame are given to determine the three-dimensional frame of the corresponding person, including: Based on the three-dimensional coordinate system, the coordinates of the points on the coordinate axis of the cuboid detection frame are given; The coordinates of the points on the coordinate axis of the cuboid detection frame are analyzed to obtain the three-dimensional coordinates of other vertices on the cuboid detection frame and determine the three-dimensional frame.

4. The method for identifying human behavior based on three-dimensional target detection according to claim 1, characterized in that: The orientation feature of the three-dimensional frame includes at least one of a length feature, a height feature, and a width feature; Determine multiple target points of the 3D frame, analyze the 3D coordinates of the target points and perform point matching to obtain the orientation features of the 3D frame, including: Give multiple target points in the three-dimensional frame and the three-dimensional coordinates of each target point; Normalize the three-dimensional coordinates to obtain standard coordinates; Point matching is performed according to standard coordinates to obtain length features, height features and width features.

5. The method for identifying human behavior based on three-dimensional target detection according to claim 4, characterized in that: According to the standard coordinates, point matching is performed to obtain length features, height features and width features, including: According to the standard coordinates, the maximum and minimum values ​​of the vertices of the three-dimensional frame on each coordinate axis are obtained; Determine the matching points according to the maximum and minimum values ​​on each coordinate axis; Based on the matching points, length features, height features and width features are obtained.

6. The method for identifying human behavior based on three-dimensional target detection according to claim 1, characterized in that: Based on the analysis and comparison of the orientation features of the three-dimensional framework, the identification of personnel behavior is completed, including: Based on the high feature in the orientation feature, the maximum height difference in the orientation feature is used as the first eigenvalue; The maximum value of the length feature and the width feature in the orientation feature is taken as the second eigenvalue; The first eigenvalue and the second eigenvalue are compared to determine a recognition result.

7. The method for identifying human behavior based on three-dimensional target detection according to claim 6, characterized in that: The first eigenvalue is specifically expressed as: h=max(abs(z1-z2),abs(z2-z3),abs(z1-z3)) Among them, h is the first eigenvalue, abs() is the absolute value function, z1, z2, z3 are the coordinate values ​​of the target point on the z-axis.

8. The method for identifying human behavior based on three-dimensional target detection according to claim 6, characterized in that: The second eigenvalue is specifically expressed as: γ=max(abs(x1-x2),abs(x2-x3),abs(x1-x3),abs(y1 -y2),abs(y2-y3),abs(y1-y3)) Among them, γ is the second eigenvalue, abs() is the absolute value function, x1, x2, x3 are the coordinate values ​​of the target point on the x-axis, and y1, y2, y3 are the coordinate values ​​of the target point on the y-axis.

9. The method for identifying human behavior based on three-dimensional target detection according to claim 6, characterized in that: Comparing the first eigenvalue and the second eigenvalue to determine the recognition result specifically includes: When the first characteristic value is less than the second characteristic value, it is determined that the recognition result is that a person falls; When the first characteristic value is greater than or equal to the second characteristic value, it is determined that the recognition result is that there is no person falling.

10. A human behavior recognition device based on three-dimensional target detection, characterized in that: The method for identifying human behavior based on three-dimensional target detection according to any one of claims 1 to 9 comprises: An image acquisition module is used to acquire an image to be identified in the target space, wherein the image to be identified is a two-dimensional image; A three-dimensional detection module is used to perform three-dimensional target detection on the image to be identified and determine the three-dimensional frame of the corresponding person; A feature determination module is used to determine multiple target points of the three-dimensional frame, analyze the three-dimensional coordinates of the target points and perform point matching to obtain the orientation features of the three-dimensional frame; The feature analysis module is used to identify human behavior based on the analysis and comparison of the orientation features of the three-dimensional framework.

Images