Method and device for assessing hand contact risk, electronic equipment and storage medium

Abstract

This application discloses a method, apparatus, electronic device, and storage medium for assessing hand contact risk. The method includes: acquiring video data of an indoor scene using a depth camera, wherein the video data includes two-dimensional color images and depth distance data; calculating the coordinates of 25 human skeleton points and up to 21 key hand skeleton points in a first video stream and a second video stream, respectively, as well as the ROI of the object for assessing surface contact risk and the mask coordinates of the object's surface within the ROI, thereby identifying the contact risk value of each person and the contact exposure dose value of any object in the indoor scene at any given time. This solves the problems of related technologies being unable to quantify the hand contact risk of personnel in indoor scenes and quantify the contact exposure dose of object surfaces in specific scenarios.

Description

Technical Field

[0001] This application relates to the field of risk assessment technology, and in particular to a method, apparatus, electronic device and storage medium for assessing hand contact risk. Background Technology

[0002] Currently, people are paying increasing attention to hand hygiene and safety. For hospitals, the risk of hand hygiene infections is even greater, and hand hygiene has become one of the important performance indicators in hospital infection control. At the same time, technologies for testing hand hygiene have emerged.

[0003] The relevant technology can use an RGB (Red-Green-Blue) camera to acquire a color RGB two-dimensional image of the target to evaluate depth information, thereby determining whether the person being identified has their hands in contact with an object and thus detecting the hand hygiene status.

[0004] However, the relevant technologies only assess depth information through two-dimensional images, which makes it difficult to effectively improve the accuracy of RGB image recognition. Furthermore, they cannot quantify the risk of hand contact in indoor scenarios or the exposure dose to object surfaces in specific scenarios, which urgently needs to be addressed. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for assessing hand contact risk, in order to solve the problems that related technologies cannot quantify the risk of hand contact for personnel in indoor scenarios and quantify the contact exposure dose of object surfaces in specific scenarios.

[0006] The first aspect of this application provides a method for assessing hand contact risk, comprising the following steps: acquiring video data of an indoor scene using a depth camera, wherein the video data includes a two-dimensional color image and depth distance data; calculating the coordinates of 25 human skeleton points and up to 21 key hand skeleton points, as well as the Region of Interest (ROI) of the object for assessing surface contact risk and the mask coordinates of the object's surface within the ROI, in a first video stream and a second video stream, respectively; and identifying the contact risk value of each person and the contact exposure dose value of any object in the indoor scene at any given time period based on the coordinates of the 25 human skeleton points, the coordinates of the up to 21 key hand skeleton points, the ROI of the object for assessing surface contact risk, and the mask coordinates of the object's surface within the ROI.

[0007] Optionally, in one embodiment of this application, the step of calculating 25 human skeleton point coordinates and at most 21 hand skeleton key point coordinates, as well as the ROI of the object for assessing surface contact risk and the mask coordinates of the object surface within the ROI, in the first video stream and the second video stream respectively, includes: assigning a unique identity to each detected person, binding the human skeleton point coordinates to the identity, performing single-person position tracking for all persons in the camera's field of view, and setting a contact risk value for each identity; for persons facing forward and whose viewing angle meets preset conditions, using the extracted at most 21 hand skeleton key point coordinates as annotation points for evaluating hand contact risk; for persons whose hands are obscured and whose viewing angle does not meet the preset conditions, estimating the hand contact annotation point positions using the left / right elbow and left / right wrist points in the human skeleton.

[0008] Optionally, in one embodiment of this application, the step of calculating the coordinates of 25 human skeleton points and at most 21 hand skeleton key points, as well as the ROI of the object for assessing surface contact risk and the Mask coordinates of the object's surface within the ROI, in the first and second video streams respectively, includes: assigning each detected object a unique object ID (Identity Document); binding the Mask coordinates to the object ID; tracking all objects in the camera's field of view; and setting a contact exposure dose value for each object ID; and using the depth distance data to mark distance information for each pixel.

[0009] Optionally, in one embodiment of this application, the step of identifying the contact risk value of each person and the contact exposure dose value of any object in the indoor scene at any time period based on the coordinates of the 25 human skeleton points, the coordinates of the at most 21 hand skeleton key points, the ROI of the object for assessing surface contact risk, and the mask coordinates of the object's surface within the ROI, includes: determining whether the coordinates of the hand annotation points in each frame of the image overlap with the mask coordinates of each object within the ROI; if there is overlap, calculating the difference between the depth distance of the hand annotation point coordinates and the depth distance in the original depth information, and determining the contact state based on the difference.

[0010] Optionally, in one embodiment of this application, determining the contact state based on the gap includes: when the gap is less than or equal to 0.1m, determining the contact state as a contact state; otherwise, determining the contact state as a non-contact state.

[0011] A second aspect of this application provides a device for assessing hand contact risk, comprising: a data acquisition module for acquiring video data of an indoor scene using a depth camera, wherein the video data includes a two-dimensional color image and depth distance data; a calculation module for calculating the coordinates of 25 human skeleton points and up to 21 key hand skeleton points, as well as the ROI of the object for assessing surface contact risk and the mask coordinates of the object's surface within the ROI, in a first video stream and a second video stream, respectively; and an identification module for identifying the contact risk value of each person and the contact exposure dose value of any object in the indoor scene at any time period based on the coordinates of the 25 human skeleton points, the coordinates of the up to 21 key hand skeleton points, the ROI of the object for assessing surface contact risk, and the mask coordinates of the object's surface within the ROI.

[0012] Optionally, in one embodiment of this application, the calculation module includes: a first tracking unit, used to assign a unique identity to each detected person, bind the coordinates of human skeleton points to the identity, perform single-person position tracking for all persons in the camera's field of view, and set a contact risk value for each identity; an extraction unit, used to extract up to 21 hand skeleton key point coordinates as annotation points for evaluating hand contact risk for persons who are facing forward and whose viewing angle meets preset conditions; and an estimation unit, used to estimate the position of hand contact annotation points for persons whose hands are obstructed and whose viewing angle does not meet the preset conditions, using the left / right elbow and left / right wrist points in the human skeleton.

[0013] Optionally, in one embodiment of this application, the calculation module further includes: a second tracking unit, configured to assign a unique object ID to each detected object, bind the mask coordinates to the object identifier, track all objects in the camera's field of view, and set a contact exposure dose value for each object ID; and a marking unit, configured to mark distance information for each pixel using the depth distance data.

[0014] Optionally, in one embodiment of this application, the recognition module includes: a judgment unit, used to judge whether the coordinates of the hand annotation point in each frame image overlap with the mask coordinates of each object in the ROI; and a determination unit, used to calculate the difference between the depth distance of the hand annotation point coordinates and the depth distance in the original depth information when there is overlap, and to determine the contact state based on the difference.

[0015] Optionally, in one embodiment of this application, the determining unit is specifically used to determine that the contact state is a contact state when the gap is less than or equal to 0.1m, otherwise the contact state is a non-contact state.

[0016] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the hand contact risk assessment method as described in the above embodiments.

[0017] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for assessing hand contact risks.

[0018] Therefore, the embodiments of this application have the following beneficial effects:

[0019] The embodiments of this application can utilize a depth camera to collect video data of indoor scenes. The video data includes two-dimensional color images and depth distance data. The two-dimensional color images are used in a first video stream and a second video stream to calculate the coordinates of 25 human skeletal points and up to 21 key hand skeletal points, as well as the Region of Interest (ROI) of the object assessing surface contact risk and the mask coordinates of the object's surface within the ROI. This allows for the identification of the contact risk value for each person and the contact exposure dose value for any object within the indoor scene at any given time. This effectively improves the accuracy of RGB image recognition and achieves higher precision contact risk assessment. Therefore, it solves the problems of related technologies being unable to quantify the contact risk of personnel's hands in indoor scenes and quantify the contact exposure dose of object surfaces in specific scenarios.

[0020] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0021] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0022] Figure 1 A flowchart illustrating a method for assessing hand contact risk according to an embodiment of this application;

[0023] Figure 2 This is a schematic diagram of a camera installation position according to an embodiment of this application;

[0024] Figure 3 This is a schematic diagram illustrating the execution logic of a method for assessing hand contact risk according to an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of a coordinate extraction logic according to an embodiment of this application;

[0026] Figure 5 This is a schematic diagram of human skeleton and hand marking points according to an embodiment of this application;

[0027] Figure 6 This is a schematic diagram of a model training process according to an embodiment of this application;

[0028] Figure 7 This is a schematic diagram of an exposure risk assessment process according to an embodiment of this application;

[0029] Figure 8 This is an example diagram of a hand contact risk assessment device according to an embodiment of this application;

[0030] Figure 9 A schematic diagram of the structure of the electronic device provided in the application embodiment.

[0031] Explanation of reference numerals in the attached figures:

[0032] Hand contact risk assessment device-10; data acquisition module-100, calculation module-200, identification module-300; memory-901, processor-902, communication interface-903. Detailed Implementation

[0033] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0034] The following description, with reference to the accompanying drawings, describes a method, apparatus, electronic device, and storage medium for assessing hand contact risk according to embodiments of this application. Addressing the problems mentioned in the background art, this application provides a method for assessing hand contact risk. In this method, a depth camera is used to collect video data of an indoor scene. The video data includes two-dimensional color images and depth distance data. The coordinates of 25 human skeletal points and up to 21 key hand skeletal points, as well as the ROI of the object assessing surface contact risk and the mask coordinates of the object's surface within the ROI, are calculated in a first video stream and a second video stream, respectively. This allows for the identification of the contact risk value for each person at any given time in the indoor scene and the contact exposure dose value for any object within the scene, thereby effectively improving the accuracy of RGB image recognition and achieving a higher precision contact risk assessment. This solves the problems of related technologies being unable to quantify the hand contact risk of personnel in indoor scenes and quantify the contact exposure dose of object surfaces in specific scenarios.

[0035] Specifically, Figure 1 A flowchart illustrating a method for assessing hand contact risk provided in an embodiment of this application.

[0036] like Figure 1 As shown, the method for assessing the risk of hand contact includes the following steps:

[0037] In step S101, a depth camera is used to collect video data of the indoor scene, wherein the video data includes two-dimensional color images and depth distance data.

[0038] As one possible implementation method, embodiments of this application can use a depth camera (including a color sensor and a depth sensor) to acquire indoor scene video through two channels. The RGB channel records two-dimensional color RGB images, and a depth channel records depth distance data. Coordinate correction and alignment processing is performed on the images from the two channels.

[0039] Furthermore, it should be noted that the embodiments of this application can set a certain number of the aforementioned depth cameras at multiple suitable locations indoors, so that the entire indoor space is within the field of view of the cameras.

[0040] For example, when installing the aforementioned depth cameras in a hospital ward, cameras can be installed at different locations on each bed, such as the headboard, middle, and left and right sides of the footboard. The distance between the cameras can be set within 4 meters, and multiple videos can be recorded from different angles. The camera arrangement is as follows: Figure 2 As shown, this ensures that the indoor space and objects within it, such as hand sanitizer, hospital beds, and fixed medical equipment, are all within the camera's field of view, providing a reliable video data source for subsequent assessment of hand contact risks.

[0041] It should be noted that, in the specific implementation process, those skilled in the art can set up multiple cameras at appropriate intervals in different locations in the ward according to the actual situation. The specific location, interval and number of cameras are not specifically limited here.

[0042] In step S102, the coordinates of 25 human skeleton points and up to 21 key hand skeleton points are calculated in the first video stream and the second video stream, respectively. The ROI of the object for assessing surface contact risk and the Mask coordinates of the object's surface within the ROI are also calculated.

[0043] It should be noted that after acquiring video data of an indoor scene using a depth camera, embodiments of this application can perform calculations on the color images in two separate video streams to obtain the coordinates of human skeletal points and key points of the hand bones, thereby providing accurate and reliable coordinate information and other data for subsequent hand contact risk assessment. Figure 3 As shown.

[0044] Optionally, in one embodiment of this application, the coordinates of 25 human skeleton points and up to 21 hand skeleton key points, as well as the ROI of the object for assessing surface contact risk and the mask coordinates of the object's surface within the ROI, are calculated in the first and second video streams respectively. This includes: assigning a unique identity to each detected person, binding the human skeleton point coordinates to the identity, performing single-person position tracking for all persons in the camera's field of view, and setting a contact risk value for each identity; for persons facing forward and whose viewing angle meets preset conditions, using the extracted up to 21 hand skeleton key point coordinates as annotation points for evaluating hand contact risk; for persons whose hands are obstructed and whose viewing angle does not meet preset conditions, estimating the hand contact annotation point positions using the left / right elbow and left / right wrist points in the human skeleton.

[0045] Specifically, embodiments of this application can call the OpenPose framework in the first video stream to extract 25 human skeleton point coordinates and up to 21 hand skeleton keypoint coordinates, such as... Figure 4 As shown.

[0046] When detecting human body parts, each detected person can be assigned a unique ID, the coordinates of human skeletal points can be bound to the ID, the person's position can be tracked individually in the camera's field of view, and a contact risk value can be set for each ID, with the initial value of the contact risk value set to 0.

[0047] Furthermore, for individuals facing forward with a good viewing angle, embodiments of this application use up to 21 key hand skeletal coordinates extracted as markers to assess hand contact risk. For individuals whose hands are obscured or whose viewing angle is poor, the left and right elbows (e.g., from the human skeletal system) are used as markers. Figure 5 (as shown in point 6, 3) and left / right wrists (as shown in point 6, 3) Figure 5 The positions of the hand contact points are estimated using the method shown in points 7 and 4. Simultaneously, embodiments of this application employ a method of proportionally extending the forearm to estimate the positions of the left and right hand contact points. Specifically, the coordinates of the left hand palm contact point are the coordinates at a point 40% longer than point 6-7, and similarly, the coordinates of the right hand palm contact point are the coordinates at a point 40% longer than point 3-4.

[0048] Therefore, the embodiments of this application accurately obtain the coordinates of human skeletal points and key points of hand bones by calling the OpenPose framework in the first video stream, ensuring the accuracy of subsequent hand contact risk assessment.

[0049] Optionally, in one embodiment of this application, the coordinates of 25 human skeleton points and up to 21 key hand skeleton points, as well as the ROI of the object for assessing surface contact risk and the Mask coordinates of the object's surface within the ROI, are calculated in the first and second video streams respectively. This includes: assigning a unique object ID to each detected object, binding the Mask coordinates to the object identifier, tracking all objects in the camera's field of view, and setting a contact exposure dose value for each object ID; and using depth distance data to mark distance information for each pixel.

[0050] Specifically, in embodiments of this application, for different indoor scenarios, objects in specific scenarios can be labeled and then trained in the Mask R-CNN network framework, such as... Figure 6 As shown, the ROI of the object whose surface contact risk needs to be assessed is marked, along with the Mask coordinates of the object's surface within the ROI.

[0051] Specifically, when detecting objects, each detected object is assigned a unique ID, the Mask coordinates are bound to the ID, the object is tracked in the camera's field of view, and a contact exposure dose value is set for each ID, with the initial value of the contact exposure dose set to 0.

[0052] Furthermore, for depth channel images, embodiments of this application utilize a depth camera to extract depth distance data synchronized with the color RGB image. The depth distance image has the same resolution as the color RGB image, and distance information is marked for each pixel. When there are no people in the indoor scene, the depth information of the objects in the indoor scene is saved as the original depth information, thereby compensating for the shortcomings of two-dimensional image evaluation in the depth direction misjudgment and effectively improving the accuracy of RGB image recognition.

[0053] In step S103, based on the coordinates of 25 human skeletal points and up to 21 hand skeletal key points, the ROI of the object for assessing surface contact risk, and the Mask coordinates of the object's surface within the ROI, the contact risk value of each person and the contact exposure dose value of any object in the indoor scene at any time period are identified.

[0054] After acquiring the coordinates of human skeletal points and key points of hand bones, and evaluating the ROI of objects with surface contact risk and the Mask coordinates of the object's surface within the ROI, the embodiments of this application can further identify the contact risk value of each person and the contact exposure dose value of any object in an indoor scene at any time period. This enables the quantitative assessment of personnel's hand contact risk and the quantification of contact exposure dose of object surfaces in a specific scene, which not only provides protection for personnel's health and safety but also provides reliable technical support for issues such as tracing the source of epidemic transmission.

[0055] Optionally, in one embodiment of this application, the contact risk value of each person and the contact exposure dose value of any object in an indoor scene at any time period are identified based on 25 human skeleton point coordinates, at most 21 hand skeleton key point coordinates, the ROI of the object for assessing surface contact risk, and the mask coordinates of the object's surface within the ROI. This includes: determining whether the hand annotation point coordinates in each frame of the image overlap with the mask coordinates of each object within the ROI; if there is overlap, calculating the difference between the depth distance of the hand annotation point coordinates and the depth distance in the original depth information, and determining the contact state based on the difference.

[0056] Specifically, after acquiring the coordinates of human skeletal points and key hand skeletal points, and evaluating the ROI of objects with surface contact risks and the mask coordinates of objects within the ROI belonging to the surface of those objects, embodiments of this application calculate for each frame to determine whether the coordinates of the hand annotation points in the first video stream in that frame overlap with the mask coordinates of each object in the second video stream. If there is no overlap, the calculation continues for the next frame; if there is overlap, the depth distance of the hand annotation point coordinates in the depth channel image of that frame is calculated and compared with the depth distance of that point in the original depth information. Figure 7 As shown, this enables the determination of whether the hand annotation points in each frame of the image overlap with the mask coordinates of each object.

[0057] Optionally, in one embodiment of this application, determining the contact state based on the gap includes: when the gap is less than or equal to 0.1m, determining the contact state as a contact state; otherwise, determining the contact state as a non-contact state.

[0058] After determining whether the coordinates of the hand annotation points in the first video stream and the mask coordinates of each object in the second video stream overlap in each frame of the image, embodiments of this application can analyze whether the two are in contact by selecting an appropriate threshold range.

[0059] Specifically, if the difference between the two is less than or equal to 0.1m, it is considered contact, and the contact risk value of the person's ID to which the hand belongs is increased by 1, and the contact exposure dose value of the object being contacted is also increased by 1. Subsequent contact between the person's ID and the object is not considered until the person's ID is determined to be without contact, at which point the calculation can be recalculated. If the difference is greater than 0.1m, it is considered no contact.

[0060] Therefore, by calculating and evaluating a certain time period in an indoor setting, based on the exposure risk value of each person and the exposure dose value of a specific object in the setting, the risk of hand contact and the exposure dose of object surfaces can be quantitatively assessed.

[0061] According to the hand contact risk assessment method proposed in this application, a depth camera is used to collect video data of an indoor scene. The video data includes two-dimensional color images and depth distance data. The coordinates of 25 human skeleton points and up to 21 key hand skeleton points are calculated in the first and second video streams of the two-dimensional color images, as well as the ROI of the object for assessing surface contact risk and the mask coordinates of the object's surface within the ROI. This allows for the identification of the contact risk value of each person and the contact exposure dose value of any object in the indoor scene at any time period, thereby effectively improving the accuracy of RGB image recognition and achieving a higher accuracy contact risk assessment.

[0062] Next, with reference to the accompanying drawings, an assessment device for hand contact risk proposed according to an embodiment of this application is described.

[0063] Figure 8 This is a block diagram of a hand contact risk assessment device according to an embodiment of this application.

[0064] like Figure 8 As shown, the hand contact risk assessment device 10 includes: a data acquisition module 100, a calculation module 200, and an identification module 300.

[0065] The acquisition module is used to acquire video data of indoor scenes using a depth camera. The video data includes two-dimensional color images and depth distance data.

[0066] The calculation module is used to calculate the coordinates of 25 human skeleton points and up to 21 hand skeleton key points in the first and second video streams respectively, as well as the ROI of the object for assessing surface contact risk and the mask coordinates of the object's surface within the ROI.

[0067] The identification module is used to identify the exposure risk value of each person and the exposure dose value of any object in an indoor scene at any time period based on the coordinates of 25 human skeletal points and up to 21 hand skeletal key points, the ROI of the object for assessing surface contact risk, and the mask coordinates of the object's surface within the ROI.

[0068] Optionally, in one embodiment of this application, the calculation module 200 includes: a first tracking unit, an extraction unit, and an estimation unit.

[0069] The first tracking unit is used to assign a unique identity to each detected person, bind the coordinates of the human skeleton points to the identity, track the location of all people individually in the field of view of the camera, and set a contact risk value for each identity.

[0070] The extraction unit is used to extract up to 21 key hand skeletal coordinates from a person facing forward and whose viewing angle meets preset conditions, and use them as annotation points to evaluate the risk of hand contact.

[0071] The estimation unit is used to estimate the position of the hand contact marker point for people whose hands are obscured or whose viewing angle does not meet the preset conditions, by using the left / right elbow and left / right wrist points in the human skeleton.

[0072] Optionally, in one embodiment of this application, the calculation module further includes a second tracking unit and a tagging unit.

[0073] The second tracking unit is used to assign a unique object ID to each detected object, bind the mask coordinates to the object identifier, track all objects in the camera's field of view, and set a contact exposure dose value for each object ID.

[0074] The tagging unit is used to tag distance information for each pixel using depth distance data.

[0075] Optionally, in one embodiment of this application, the identification module 300 includes a judgment unit and a determination unit.

[0076] The judgment unit is used to determine whether the coordinates of the hand annotation points in each frame of the image overlap with the mask coordinates of each object within the ROI.

[0077] The determination unit is used to calculate the difference between the depth distance of the hand annotation point coordinates and the depth distance in the original depth information when there is overlap, and to determine the contact state based on the difference.

[0078] Optionally, in one embodiment of this application, the determining unit is specifically used to determine the contact state as a contact state when the gap is less than or equal to 0.1m, and otherwise the contact state as a non-contact state.

[0079] It should be noted that the explanation of the aforementioned method embodiment for assessing hand contact risk also applies to the hand contact risk assessment device of this embodiment, and will not be repeated here.

[0080] The hand contact risk assessment device proposed in this application utilizes depth data to improve a color RGB two-dimensional image and can synchronize the depth and color RGB images. Using a depth distance detector, a three-dimensional field of view can be established to determine the true distance between a person and an object, thereby assessing whether contact has occurred. The depth distance information assistance can compensate for the shortcomings of using only two-dimensional images for assessment in terms of misjudgment in the depth direction, effectively improving the accuracy of RGB image recognition.

[0081] Figure 9 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0082] The memory 901, the processor 902, and the computer program stored on the memory 901 and capable of running on the processor 902.

[0083] When the processor 902 executes the program, it implements the hand contact risk assessment method provided in the above embodiments.

[0084] Furthermore, electronic devices also include:

[0085] Communication interface 903 is used for communication between memory 901 and processor 902.

[0086] The memory 901 is used to store computer programs that can run on the processor 902.

[0087] The memory 901 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0088] If the memory 901, processor 902, and communication interface 903 are implemented independently, then the communication interface 903, memory 901, and processor 902 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0089] Optionally, in a specific implementation, if the memory 901, processor 902, and communication interface 903 are integrated on a single chip, then the memory 901, processor 902, and communication interface 903 can communicate with each other through an internal interface.

[0090] The processor 902 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0091] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for assessing hand contact risks.

[0092] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0093] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0094] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0095] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0096] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0097] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0098] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0099] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for assessing the risk of hand contact, characterized in that, Includes the following steps: Video data of an indoor scene is acquired using a depth camera, wherein the video data includes two-dimensional color images and depth distance data; The two-dimensional color image is used to calculate the coordinates of 25 human skeleton points and up to 21 key hand skeleton points in the first and second video streams, respectively, as well as the ROI of the object used to assess surface contact risk and the mask coordinates of the object's surface within the ROI; and Based on the coordinates of the 25 human skeletal points, the coordinates of the at most 21 hand skeletal key points, the ROI of the object for assessing surface contact risk, and the Mask coordinates of the object's surface within the ROI, the contact risk value of each person and the contact exposure dose value of any object in the indoor scene at any time period are identified. The step of calculating the coordinates of 25 human skeleton points and at most 21 hand skeleton key points in the first and second video streams, respectively, and the ROI of the object for assessing surface contact risk, as well as the mask coordinates within the ROI belonging to the surface of the object, includes: Each detected person is assigned a unique identifier, and the coordinates of the human skeleton points are bound to the identifier. Individual positions of all persons are tracked within the camera's field of view, and a contact risk value is set for each identifier. For individuals facing forward and whose viewing angle meets the preset conditions, up to 21 key hand skeletal coordinates will be extracted as markers to assess hand contact risk. For individuals whose hands are obscured or whose viewing angle does not meet the preset conditions, the position of the hand contact marker is estimated using the left / right elbow and left / right wrist points in the human skeleton. The step of calculating the coordinates of 25 human skeleton points and up to 21 hand skeleton key points in the first and second video streams respectively, as well as the ROI of the object for assessing surface contact risk and the mask coordinates of the object's surface within the ROI, includes: Each detected object is assigned a unique object ID, the mask coordinates are bound to the object ID, all objects are tracked in the field of view of the camera, and a contact exposure dose value is set for each object ID. The depth distance data is used to mark distance information for each pixel.

2. The method according to claim 1, characterized in that, The step of identifying the exposure risk value of each person and the exposure dose value of any object in the indoor scene at any time period based on the coordinates of the 25 human skeletal points, the coordinates of at most 21 hand skeletal key points, the ROI of the object for assessing surface contact risk, and the mask coordinates of the object's surface within the ROI, includes: Determine whether the coordinates of the hand annotation points in each frame of the image overlap with the mask coordinates of each object within the ROI; If there is overlap, the difference between the depth distance of the hand marking point coordinates and the depth distance in the original depth information is calculated, and the contact state is determined based on the difference.

3. The method according to claim 2, characterized in that, Determining the contact state based on the gap includes: When the gap is less than or equal to 0.1m, the contact state is determined to be a contact state; otherwise, the contact state is a non-contact state.

4. A device for assessing hand contact risk, characterized in that, include: The acquisition module is used to acquire video data of an indoor scene using a depth camera, wherein the video data includes two-dimensional color images and depth distance data; The calculation module is used to calculate the coordinates of 25 human skeleton points and up to 21 hand skeleton key points in the first and second video streams, respectively, as well as the ROI of the object for assessing surface contact risk and the mask coordinates of the object's surface within the ROI; and The identification module is used to identify the contact risk value of each person and the contact exposure dose value of any object in the indoor scene at any time period based on the coordinates of the 25 human skeleton points, the coordinates of the at most 21 hand skeleton key points, the ROI of the object for assessing surface contact risk, and the Mask coordinates of the object's surface within the ROI. The calculation module includes: The first tracking unit is used to assign a unique identity to each detected person, bind the coordinates of the human skeleton points to the identity, track the position of all people individually in the field of view of the camera, and set a contact risk value for each identity. The extraction unit is used to extract up to 21 key hand bone coordinates for personnel facing forward and whose viewing angle meets preset conditions, and use them as annotation points to evaluate the risk of hand contact. The estimation unit is used to estimate the position of the hand contact marker point for a person whose hand is obscured and whose viewing angle does not meet the preset conditions, using the left / right elbow and left / right wrist points in the human skeleton. The computing module also includes: The second tracking unit is used to assign a unique object ID to each detected object, bind the mask coordinates to the object ID, track all objects in the field of view of the camera, and set a contact exposure dose value for each object ID. A marking unit is used to mark distance information for each pixel using the depth distance data.

5. The apparatus according to claim 4, characterized in that, The identification module includes: The judgment unit is used to determine whether the coordinates of the hand annotation points in each frame of the image overlap with the mask coordinates of each object in the ROI; The determining unit is used to calculate the difference between the depth distance of the hand marking point coordinates and the depth distance in the original depth information when there is overlap, and to determine the contact state based on the difference.

6. The apparatus according to claim 5, characterized in that, The determining unit is specifically used for, When the gap is less than or equal to 0.1m, the contact state is determined to be a contact state; otherwise, the contact state is a non-contact state.

7. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the hand contact risk assessment method as described in any one of claims 1-3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the method for assessing hand contact risk as described in any one of claims 1-3.

Images