Method for identifying sleep behavior of identified behavior object back to camera

Abstract

The invention provides a method and a device for identifying a sleep behavior of an identified behavior object back to a camera, electronic equipment and a non-instantaneous computer readable storage medium, the identification method comprises the following steps: in response to a sleep behavior identification instruction, dividing a target identification area into a workbench area and a human body trunk contour area; respectively determining a first plane to which the workbench area belongs and a second plane to which the human body trunk contour area belongs by utilizing the workbench area and the human body trunk contour area; calculating a plane included angle between the first plane and the second plane; and recognizing the sleep behavior back to the camera according to the plane included angle. According to the embodiment of the invention, the sleep behavior of the behavior recognition object is recognized by calculating the included angle between the plane where the human body trunk contour is located and the plane where the workbench is located, so that the sleep behavior back to the camera is recognized.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular, to a method and device for recognizing sleep behavior when the recognized behavior object has its back to the camera, an electronic device, and a non-transitory computer-readable storage medium. Background Art

[0002] In fields such as security, safety supervision, chemical industry, and gas stations, during the night, the duty personnel need to be in a working state all the time, and the occurrence of sleep behavior may lead to the inability to handle abnormal emergencies in a timely manner.

[0003] With the development of artificial intelligence technology and the increasing popularity of monitoring devices, the picture data resources are growing rapidly. Images, as the core of information dissemination, contain rich information, and in recent years, machine learning and deep learning technologies have developed rapidly. However, the current behavior monitoring methods can only recognize the sleeping behavior facing the camera, and cannot effectively recognize the sleeping-on-duty behavior with the back to the camera. Summary of the Invention

[0004] The present invention aims to provide a method and device for recognizing sleep behavior when the recognized behavior object has its back to the camera, an electronic device, and a non-transitory computer-readable storage medium, so as to solve the problem that the prior art cannot effectively recognize the sleeping-on-duty behavior with the back to the camera.

[0005] According to one aspect of the present invention, a method for recognizing sleep behavior when the recognized behavior object has its back to the camera is provided, including: in response to a sleep behavior recognition instruction, dividing a target recognition area into a workbench area and a human torso contour area; respectively determining a first plane to which the workbench area belongs and a second plane to which the human torso contour area belongs by using the workbench area and the human torso contour area; calculating a plane angle between the first plane and the second plane; and recognizing the sleep behavior with the back to the camera according to the plane angle.

[0006] According to some embodiments, respectively determining the first plane to which the workbench area belongs and the second plane to which the human torso contour area belongs by using the workbench area and the human torso contour area includes: respectively determining a first point cloud position of the workbench area and a second point cloud position of the human torso contour area according to depth values of the target recognition area; determining the first plane according to the first point cloud position; and determining the second plane according to the second point cloud position.

[0007] According to some embodiments, before recognizing the sleep behavior with the back to the camera according to the plane angle, it further includes: determining head key points of the recognized behavior object; and calculating a distance from the head key points to the first plane.

[0008] According to some embodiments, identifying the sleep behavior of a person with their back to the camera based on the plane angle includes: comparing the plane angle with a preset plane angle threshold; comparing the distance with a preset distance threshold; when the plane angle is less than the plane angle threshold and / or the distance is less than the distance threshold, it is confirmed that a sleep behavior exists.

[0009] According to some embodiments, before identifying the sleep behavior of a person with their back to the camera based on the plane angle, it further includes: determining the neck key point and the waist key point of the object to be identified; using the head key point, the neck key point, and the waist key point to calculate the pose angles of the head-neck and neck-waist.

[0010] According to some embodiments, identifying the sleep behavior of a person with their back to the camera based on the plane angle includes: comparing the plane angle with a preset plane angle threshold; comparing the distance with a preset distance threshold; comparing the pose angle with a preset pose angle threshold; when the plane angle is less than the plane angle threshold, the distance is less than the distance threshold, and / or the pose angle is less than the pose angle threshold, it is confirmed that a sleep behavior exists.

[0011] According to some embodiments, identifying the sleep behavior of a person with their back to the camera based on the plane angle includes: identifying the sleep behavior of a person with their back to the camera based on the plane angles calculated from consecutive multiple frames of images.

[0012] According to one aspect of the present invention, there is provided an apparatus for identifying the sleep behavior of an object to be identified with their back to the camera, including: a region identification unit for dividing a target identification region into a workbench region and a human torso contour region in response to a sleep behavior identification instruction; a plane determination unit for respectively determining a first plane to which the workbench region belongs and a second plane to which the human torso contour region belongs by using the workbench region and the human torso contour region; a plane angle calculation unit for calculating the plane angle between the first plane and the second plane; and a sleep behavior identification unit for identifying the sleep behavior of a person with their back to the camera based on the plane angle.

[0013] According to one aspect of the present invention, there is provided an electronic device, including: a processor; and a memory storing a computer program, which when executed by the processor, causes the processor to execute the method as described in any one of the previous items.

[0014] According to one aspect of the present invention, there is provided a non-transitory computer-readable storage medium having stored thereon computer-readable instructions, which when executed by a processor, cause the processor to execute the method as described in any one of the previous embodiments.

[0015] According to an embodiment of the present invention, when the recognized behavior object has its back to the camera, the plane angle between the plane where the human torso contour is located and the plane where the workbench is located is calculated, and the sleep behavior of the recognized behavior object is recognized according to the calculated plane angle.

[0016] The recognition process according to the embodiment of the present invention not only reduces the computational complexity but also has high recognition accuracy.

[0017] It should be understood that the above general description and the following detailed description are only exemplary and do not limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. By referring to the drawings and describing its exemplary embodiments in detail, the above and other objects, features and advantages of the present invention will become more obvious.

[0019] Figure 1 FIG. shows a flowchart of a method for recognizing the sleep behavior of a recognized behavior object when the object has its back to the camera according to an exemplary embodiment of the present invention.

[0020] Figure 2 FIG. shows a flowchart of another method for recognizing the sleep behavior of a recognized behavior object when the object has its back to the camera according to an exemplary embodiment of the present invention.

[0021] Figure 3 FIG. shows a flowchart of another method for recognizing the sleep behavior of a recognized behavior object when the object has its back to the camera according to an exemplary embodiment of the present invention.

[0022] Figure 4 FIG. shows a flowchart of another method for recognizing the sleep behavior of a recognized behavior object when the object has its back to the camera according to an exemplary embodiment of the present invention.

[0023] Figure 5 FIG. shows a schematic diagram of key points according to an exemplary embodiment of the present invention.

[0024] Figure 6 FIG. shows a block diagram of a device for recognizing the sleep behavior of a recognized behavior object when the object has its back to the camera according to an exemplary embodiment of the present invention.

[0025] Figure 7 FIG. shows an electronic device according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. Identical reference numerals in the figures denote identical or similar parts, and thus their repetitive description will be omitted.

[0027] The features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present disclosure. However, those skilled in the art will realize that the technical solutions of the present disclosure can be practiced without one or more of these specific details, or other methods, components, materials, devices, or operations, etc. may be employed. In these cases, well-known structures, methods, devices, implementations, materials, or operations will not be shown or described in detail.

[0028] The flowcharts shown in the accompanying drawings are merely illustrative and not necessarily include all the contents and operations / steps, nor are they necessarily executed in the order described. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined, so the actual execution order may change according to the actual situation.

[0029] The terms "first", "second", etc. in the specification and claims of the present invention and the above accompanying drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0030] The following will, with reference to the accompanying drawings, elaborate on the specific embodiments according to the present invention in detail.

[0031] Figure 1 A flowchart of a method for identifying a sleeping behavior when the recognized behavioral object has its back to the camera according to an example embodiment of the present invention is shown, as Figure 1 The shown recognition method includes steps S101, S103, S105, and S107. The following will, taking Figure 1 as an example, elaborate on a method for identifying a sleeping behavior with the back to the camera according to an example embodiment of the present invention in detail.

[0032] As Figure 1As shown, in step S101, in response to a sleep behavior recognition instruction, the target recognition area is divided into a workbench area and a human torso contour area.

[0033] According to an embodiment of the present invention, before step S101, it is necessary to obtain image data of the recognition behavior object.

[0034] In a specific embodiment, in order to capture the recognition behavior object, a monocular camera with a resolution greater than 1080p is used, and the top-down angle, focal length, and position of the camera are kept fixed, and the top-down angle of the camera is kept between 15° and 45°.

[0035] In some embodiments, a video stream is obtained from the camera and the video stream is decoded to obtain color image data. Among them, the video stream includes but is not limited to RTSP stream, MP4, and / or RTMP.

[0036] According to an embodiment of the present invention, in step S101, first, based on the color image data, at least one recognition area is determined. Among them, the recognition area needs to be determined according to a specific application scenario. In this embodiment, the recognition behavior object needs to be on duty beside the workbench. Therefore, the determined recognition area is the workbench area including the recognition behavior object, the coordinates of multiple points of the recognition area are obtained, and a polygon area is generated based on the obtained point coordinates. Then, the polygon area is cut, human detection and monocular depth estimation are performed to obtain the workbench area excluding the human area, the human area, and the depth values corresponding to all pixel points of the entire color image. Finally, human key point detection and human segmentation are performed according to the obtained human area to obtain the human torso contour area.

[0037] In step S103, the first plane to which the workbench area belongs and the second plane to which the human torso contour area belongs are respectively determined by using the workbench area and the human torso contour area.

[0038] According to an embodiment of the present invention, in step S103, first, the first point cloud position of the workbench area and the second point cloud position of the human torso contour area are respectively determined according to the depth values of the target recognition area; then, the first plane is determined according to the first point cloud position and the second plane is determined according to the second point cloud position.

[0039] In a specific embodiment, the point cloud position is represented by three-dimensional coordinates, and the first plane to which the workbench area belongs and the second plane to which the human torso contour area belongs are determined by the plane variance. In some embodiments, the RANSAC (Random Sample Consensus) method can be used to fit the plane.

[0040] In step S105, the plane angle between the first plane and the second plane is calculated.

[0041] In this embodiment, the posture of the recognized behavior object is determined by calculating the plane angle between the first plane and the second plane.

[0042] In step S107, the sleeping behavior with the back to the camera is recognized according to the plane angle.

[0043] In this embodiment, in order to improve the recognition accuracy of the sleeping behavior of the recognized behavior object, the sleeping behavior with the back to the camera is recognized according to the plane angles of a series of consecutive frames of images calculated. And a sleeping behavior warning is given only when the number of images with the plane angle less than the preset plane angle threshold is greater than the preset plane number threshold.

[0044] It should be noted here that the preset thresholds in this embodiment can be set according to specific application scenarios and are not specifically limited herein. For example, when strict control over the sleeping behavior of the recognized behavior object is required, the plane angle threshold can be increased and / or the plane number threshold can be decreased.

[0045] According to Figure 1 the embodiment shown, the sleeping behavior of the recognized behavior object is recognized by calculating the angle between the plane where the human torso contour is located and the plane where the workbench is located, so as to recognize the sleeping behavior with the back to the camera.

[0046] Figure 2 FIG. shows a flowchart of another method for recognizing the sleeping behavior of a recognized behavior object with the back to the camera according to an exemplary embodiment of the present invention. Compared with Figure 1 compared, Figure 2 the embodiment shown also includes steps S106 and S107. Among them, step S106 is executed before step S107. For the sake of simplicity, only the differences from Figure 1 will be described in this embodiment, and the same parts will not be repeated.

[0047] According to Figure 2 the embodiment shown, in step S101, it is also necessary to detect the human body to obtain the head key points of the recognized behavior object.

[0048] In step 106, the distance from the head key point to the first plane is calculated.

[0049] In step S107, the sleeping behavior with the back to the camera is recognized according to the plane angle and the distance.

[0050] For example, when the plane angle is less than the preset plane angle threshold and / or the distance from the head key point to the first plane is less than the preset distance threshold, it is determined that the recognized behavior object has a sleeping behavior.

[0051] In this embodiment, in order to improve the recognition accuracy of the sleep behavior of the recognized behavior object, the sleep behavior with the back to the camera is recognized according to the calculated planar angle of consecutive multiple frames of images. And only when the number of images with the planar angle less than the preset planar angle threshold is greater than the preset planar number threshold, and when the number of images with the distance from the head key point to the first plane less than the preset distance threshold is greater than the preset distance number threshold, a sleep behavior warning is issued.

[0052] In Figure 2 the embodiment shown, the sleep behavior of the recognized behavior object is recognized by calculating the distance from the head key point to the first plane and the angle between the plane where the human torso contour is located and the plane where the workbench is located, so as to improve the recognition accuracy of the sleep behavior with the back to the camera.

[0053] Figure 3 shows a flowchart of another method for recognizing the sleep behavior of the recognized behavior object with the back to the camera according to an exemplary embodiment of the present invention. Compared with Figure 2 that Figure 3 shown in the embodiment, in step S101, it is also necessary to determine the neck key point and the waist key point of the recognized behavior object; and in step S106, it is also necessary to calculate the postural angles of the head and neck and the neck and waist by using the head key point, the neck key point and the waist key point. Among them, the straight line where the neck and waist are located is the straight line where the midpoint of the neck and the waist is located.

[0054] Finally, in step S107, the sleep behavior with the back to the camera is recognized according to the planar angle, the distance and the postural angle.

[0055] For example, when the planar angle is less than the preset planar angle threshold, the distance from the head key point to the first plane is less than the preset distance threshold, and / or the postural angle is less than the preset postural angle threshold, it is determined that the recognized behavior object has a sleep behavior.

[0056] In this embodiment, in order to improve the recognition accuracy of the sleep behavior of the recognized behavior object, the sleep behavior with the back to the camera is recognized according to the calculated planar angle of consecutive multiple frames of images. And only when the number of images with the planar angle less than the preset planar angle threshold is greater than the preset planar number threshold, the number of images with the distance from the head key point to the first plane less than the preset distance threshold is greater than the preset distance number threshold, and / or the number of images with the postural angle less than the preset postural angle threshold is greater than the preset postural number threshold, a sleep behavior warning is issued.

[0057] In Figure 3In the illustrated embodiment, the sleep behavior of the recognized behavioral object is recognized by calculating the posture angles between the head and neck and between the neck and waist, calculating the distances from the key points of the head to the first plane, and the angle between the plane where the human torso contour is located and the plane where the workbench is located, further improving the recognition accuracy of the sleep behavior with the back facing the camera.

[0058] Figure 4 FIG. shows a flowchart of another method for recognizing the sleep behavior of a recognized behavioral object with the back facing the camera according to an exemplary embodiment of the present invention. In the embodiment, the recognized object is a staff member on duty at a desk.

[0059] As Figure 4 shown, first, color image data is obtained from the camera. For example, a monocular camera with a resolution greater than 1080p is used, and the camera's top-down angle, focal length, and position are kept fixed. An RTSP stream is obtained by a camera with a top-down angle between 15° and 45° to obtain color image data.

[0060] Then, based on the color image data, at least one algorithm recognition region is set, the coordinates of multiple points in the algorithm recognition region are obtained, and a polygon region can be generated based on these point coordinates.

[0061] It should be noted here that the present invention does not limit the setting method of the coordinate system. As long as the positional relationship of the algorithm recognition region can be described, it is applicable to the embodiments of the present invention.

[0062] After that, based on the obtained color image data, desktop segmentation (excluding the human body region), human body detection, and monocular depth estimation are performed to obtain the desktop contour region (there may be multiple), the human body region, and the depth values corresponding to the pixel points in the color image, respectively.

[0063] After that, based on the obtained desktop contour region and the depth values corresponding to the pixel points in the color image, the depth values of all points within the desktop contour are obtained, and human key point detection and human body segmentation are performed on the human body region to obtain human key points and the human body contour region, respectively.

[0064] Figure 5 FIG. shows a schematic diagram of key points according to an exemplary embodiment of the present invention. As Figure 5 shown, to obtain 17 key points, the head, neck, and waist key points are extracted therefrom, and there are 2 waist key points. Based on the human shoulder key points and waist key points, and the human body contour region, the human torso contour region can be obtained, and based on the depth values corresponding to the pixel points in the color image, the depth values of the pixel points within the human body contour are obtained, so as to obtain the depth values of the head, neck, and waist key points and the depth values of the pixel points within the human torso contour.

[0065] After that, using the depth values of all the points within the desktop contour, the depth values of the head, neck, and waist key points (two waist key points), and the depth values of all the points within the human torso contour region, calculate the three-dimensional point cloud coordinates of the pixel points within the desktop contour, the three-dimensional point cloud coordinates of the head, neck, and waist key points, and the three-dimensional point cloud coordinates of the pixel points within the human torso contour, respectively. Among them, the calculation of the point cloud coordinates is shown in Formula (1).

[0066] z = z c depth_scale

[0067] x = (μ - μ 0 ) * zf x

[0068] y = (ν - ν 0 ) * zf y (1)

[0069] Among them, (μ, ν) are the pixel coordinates of the pixel point, z c is the depth value of the pixel point (μ, ν), (u 0 , v 0 ) are the principal point coordinates, fx is the length of the focal length in the x-axis direction described by pixels, f y is the length of the focal length in the y-axis direction described by pixels, depth_scale is the scale factor corresponding to the depth map, which is the ratio of the value stored in the depth map to the true depth (in meters), and is used to calibrate the ratio relationship between the true distance of the object from the camera and the depth value. Among them, the principal point coordinates can be obtained according to the internal parameter matrix of the camera.

[0070] After that, using the three-dimensional point cloud coordinates of the pixel points within the desktop contour and the three-dimensional point cloud coordinates of the pixel points within the human torso contour, obtain the plane equation of the plane to which the desktop belongs (as shown in Formula 2) and the plane equation of the plane to which the human torso contour belongs (as shown in Formula 3); using the three-dimensional point cloud coordinates of the head, neck, and waist key points, obtain the straight line equation formed by the head and neck key points (the head key point and the neck key point) and the straight line equation formed by the midpoint of the neck and waist (the midpoint of the neck key point and the two waist key points).

[0071] A1x + B1y + C1z + D1 = 0 (2)

[0072] A2x + B2y + C2z + D2 = 0 (3)

[0073] Among them, the normal vectors of the first plane to which the desktop belongs and the second plane to which the human torso contour belongs are shown in Formulas (4) and (5), respectively.

[0074]

[0075] The included angle between the first plane to which the desktop belongs and the second plane to which the human torso contour belongs is as shown in formula (6).

[0076]

[0077] After that, using the three-dimensional point cloud coordinates of the head key point and the plane equation of the desktop, the distance from the head key point to the plane of the desktop is obtained. Using the straight-line equation formed by the head and neck key points and the straight-line equation formed by the midpoint of the neck and waist, the included angle between these two straight lines is obtained. And using the plane equation of the desktop and the plane equation of the human torso contour, the included angle between the two planes can be obtained.

[0078] Assume that the three-dimensional point cloud coordinates of the head key point are (x 0 , y 0 , z 0 ), and the distance from the head key point to the plane of the desktop is as shown in formula (7).

[0079]

[0080] Assume that the included angle between the two straight lines is θ1, and the three-dimensional point cloud coordinates of the head key point, the neck key point, and the midpoint of the waist (the midpoint of the two waist key points) are (x 0 , y 0 , z 0 ), (x 1 , y 1 , z 1 ), (x 2 , y 2 , z 2 ), respectively. Then the straight-line equations formed by the head and neck key points and the straight-line equation formed by the midpoint of the neck and waist are as shown in formula (8). Among them, the direction vectors of the two straight lines are respectively: and

[0081]

[0082] The included angle between the straight line formed by the head and neck key points and the straight line formed by the midpoint of the neck and waist is as shown in formula (9).

[0083]

[0084] After that, continuously count the number M1 of the distances from the head key point to the plane of the desktop in N consecutive frames that are less than threshold 1, the number M2 of the included angles between the straight line formed by the head and neck key points and the straight line formed by the midpoint of the neck and waist that are less than threshold 2, and the number M3 of the included angles between the first plane to which the desktop belongs and the second plane to which the human torso contour belongs that are less than threshold 3.

[0085] If M1 / N > k1, M2 / N > k2, and / or M3 / N > k3, then further combine the spatio-temporal features of multiple frames to perform dozing video classification on the multiple-frame images to further reduce false alarms. If the output result is yes, then an alarm is generated; otherwise, no alarm is generated.

[0086] In a specific embodiment, the dozing video classification method includes but is not limited to 3DCNN, Two-stream CNN, and / or TSN.

[0087] In some embodiments, the thresholds k1, k2, and k3, the thresholds 1, 2, and 3, and the sizes of M1, M2, and M3 can be specifically set according to requirements and are not limited herein.

[0088] The above mainly introduces the embodiments of the present invention from the perspective of methods. Those skilled in the art should easily realize that, in combination with the operations or steps of the examples described in the embodiments disclosed herein, the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Those skilled in the art can use different methods to implement the described functions for each specific operation or method, and such implementation should not be considered to exceed the scope of the present invention.

[0089] The device embodiments of the present invention are described below. For details not described in the device embodiments of the present invention, reference can be made to the method embodiments of the present invention.

[0090] Figure 6 A block diagram of a device for identifying the sleeping behavior of an identified behavioral object with its back to the camera according to an exemplary embodiment of the invention is shown, as Figure 6 The shown identification device includes a region identification unit 601, a plane determination unit 603, a plane angle calculation unit 605, and a sleeping behavior identification unit 607. Among them, the region identification unit 601 is configured to divide the target identification region into a workbench region and a human torso contour region in response to a sleeping behavior identification instruction; the plane determination unit 603 is configured to respectively determine a first plane to which the workbench region belongs and a second plane to which the human torso contour region belongs by using the workbench region and the human torso contour region; the plane angle calculation unit 605 is configured to calculate the plane angle between the first plane and the second plane; and the sleeping behavior identification unit 607 is configured to identify the sleeping behavior with the back to the camera according to the plane angle.

[0091] Figure 7 An electronic device according to an exemplary embodiment of the present invention is shown. The following refers to Figure 7 to describe the electronic device 200 according to this embodiment of the present invention. Figure 7 The shown electronic device 200 is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention.

[0092] As Figure 7 shown, the electronic device 200 is presented in the form of a general-purpose computing device. The components of the electronic device 200 may include, but are not limited to: at least one processing unit 210, at least one storage unit 220, a bus 230 connecting different system components (including the storage unit 220 and the processing unit 210), a display unit 240, etc.

[0093] Among them, the storage unit stores program code, and the program code can be executed by the processing unit 210, so that the processing unit 210 executes the methods according to various exemplary embodiments of the present invention described in this specification. For example, the processing unit 210 can execute the method as Figure 1 shown in

[0094] The storage unit 220 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 2201 and / or a cache storage unit 2202, and may further include a read-only storage unit (ROM) 2203.

[0095] The storage unit 220 may further include a program / utilities 2204 having a set (at least one) of program modules 2205. Such program modules 2205 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0096] The bus 230 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus structures.

[0097] The electronic device 200 can also communicate with one or more external devices 300 (such as a keyboard, a pointing device, a Bluetooth device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 200, and / or communicate with any device that enables the electronic device 200 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 250. Moreover, the electronic device 200 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 260. The network adapter 260 can communicate with other modules of the electronic device 200 through the bus 230. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0098] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or can be implemented by a combination of software and necessary hardware. The technical solutions according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the above methods according to the embodiments of the present invention.

[0099] The software product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0100] A computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable storage medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0101] The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., by using an Internet service provider to connect through the Internet).

[0102] The above computer-readable medium carries one or more programs, which, when executed by one of the devices, cause the computer-readable medium to implement the foregoing functions.

[0103] Those skilled in the art can understand that the above-mentioned modules may be distributed in the device according to the description of the embodiments, or may be correspondingly changed and distributed in one or more devices that are uniquely different from the present embodiment. The modules of the above embodiments may be combined into one module, or may be further split into multiple sub-modules.

[0104] According to an embodiment of the present invention, a computer program is provided, including a computer program or instruction, which, when executed by a processor, can perform the method described above.

[0105] The above has introduced the embodiments of the present invention in detail. Specific examples are used herein to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, any changes or deformations made by those skilled in the art based on the idea of the present invention, within the specific implementation manners and application scope of the present invention, fall within the scope of protection of the present invention. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A method for identifying the sleeping behavior of an identified behavior object when the object faces away from the camera, characterized in that: include: In response to the sleep behavior recognition instruction, the target recognition area is divided into a workbench area and a human body trunk contour area; Using the workbench area and the human body trunk contour area, respectively determine a first plane to which the workbench area belongs and a second plane to which the human body trunk contour area belongs; calculating a plane angle between the first plane and the second plane; The sleeping behavior of the person with his back to the camera is identified according to the plane angle.

2. The method according to claim 1, characterized in that Using the workbench area and the human body trunk contour area to respectively determine a first plane to which the workbench area belongs and a second plane to which the human body trunk contour area belongs, comprising: Determine the first point cloud position of the workbench area and the second point cloud position of the human body trunk contour area respectively according to the depth value of the target recognition area; Determine the first plane according to the first point cloud position; The second plane is determined according to the position of the second point cloud.

3. The method according to claim 1, characterized in that Before identifying the sleeping behavior of the person facing away from the camera according to the plane angle, the method further includes: Determine the key points of the head of the recognition behavior object; The distance between the head key point and the first plane is calculated.

4. The method according to claim 3, characterized in that The sleep behavior of the person facing away from the camera is identified according to the plane angle, including: Comparing the plane angle with a preset plane angle threshold; comparing the distance to a preset distance threshold; When the plane angle is smaller than the plane angle threshold and / or the distance is smaller than the distance threshold, it is confirmed that sleeping behavior exists.

5. The method according to claim 3, characterized in that: Before identifying the sleeping behavior of the person facing away from the camera according to the plane angle, the method further includes: Determine the neck key points and waist key points of the recognition behavior object; The head key points, the neck key points and the waist key points are used to calculate the posture angles of the head and neck and the neck and waist.

6. The method according to claim 5, characterized in that The sleep behavior of the person facing away from the camera is identified according to the plane angle, including: Comparing the plane angle with a preset plane angle threshold; comparing the distance to a preset distance threshold; comparing the posture angle with a preset posture angle threshold; When the plane angle is smaller than the plane angle threshold, the distance is smaller than the distance threshold, and / or the posture angle is smaller than the posture angle threshold, the presence of sleeping behavior is confirmed.

7. The method according to claim 1, characterized in that The sleep behavior of the person facing away from the camera is identified according to the plane angle, including: The sleeping behavior of the subject facing away from the camera is identified based on the calculated plane angle of multiple consecutive image frames.

8. A device for identifying the sleeping behavior of an identified behavior subject when the subject faces away from a camera, characterized in that: include: A region recognition unit, for dividing a target recognition region into a workbench region and a human body trunk contour region in response to a sleep behavior recognition instruction; A plane determining unit, used to respectively determine a first plane to which the workbench area belongs and a second plane to which the human body trunk contour area belongs by using the workbench area and the human body trunk contour area; A plane angle calculation unit, used to calculate the plane angle between the first plane and the second plane; The sleeping behavior recognition unit is used to recognize the sleeping behavior of the person facing away from the camera according to the plane angle.

9. An electronic device, comprising: processor; as well as A memory storing a computer program, which, when executed by the processor, enables the processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having computer-readable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 7.

Images