Method and device for detecting intrusion of personnel into operating room

Abstract

The present application provides a method and device for detecting intrusion of personnel in an operation room, wherein the method for detecting intrusion of personnel in an operation room comprises: identifying a target video frame associated with the operation room, and determining a door target contained in the target video frame; determining the door state of the door target according to the area of ​​the door target, wherein the door state includes an open state and a closed state; when the door state is the closed state, identifying the personnel target contained in the target video frame; identifying a reference object target in the target video frame, and determining the positional relationship between the personnel target and the door target according to the reference object target, wherein the positional relationship is an intrusion relationship or a non-intrusion relationship; generating an alarm instruction according to the positional relationship and reporting it to a preset alarm device. Accurate and real-time detection of whether there is intrusion of personnel in the operation room is achieved, reducing missed detections and false detections, and providing reliable safety protection for industrial production.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and more particularly to a method and device for detecting intrusion of personnel into an operating room. The present application also relates to a computing device and a computer-readable storage medium. Background Art

[0002] In modern industrial production, the door opening and closing area of ​​the operation room often involves the entry and exit of personnel and the operation of mechanical equipment, such as the operation of a robotic arm. In this environment, it is crucial to ensure the safety of personnel and the normal operation of equipment.

[0003] However, traditional monitoring systems mostly rely on basic sensors and manual inspections, and have problems such as insufficient detection accuracy and slow response speed. They are unable to timely identify the switch status of doors and the entry of people, thus causing safety hazards. Summary of the invention

[0004] In view of this, the embodiment of the present application provides a method for detecting intrusion of personnel in an operation room to solve the technical defects existing in the prior art. The embodiment of the present application also provides an apparatus for detecting intrusion of personnel in an operation room, a computing device, and a computer-readable storage medium.

[0005] According to a first aspect of an embodiment of the present application, a method for detecting intrusion of a person into an operating room is provided, comprising:

[0006] identifying a target video frame associated with the operating room, and determining a door target contained in the target video frame;

[0007] Determining a door state of the door target according to the area of ​​the door target, wherein the door state includes an open state and a closed state;

[0008] When the door state is the closed state, identifying a human target contained in the target video frame;

[0009] Identifying a reference object in the target video frame, and determining a positional relationship between the person object and the door object according to the reference object, wherein the positional relationship is an intrusion relationship or a non-intrusion relationship;

[0010] According to the position relationship, an alarm instruction is generated and reported to a preset alarm device.

[0011] Optionally, the identifying a target video frame associated with the operating room and determining a door target included in the target video frame includes:

[0012] Acquiring candidate video frames associated with the operating room;

[0013] Processing the candidate video frame based on a preset brightness equalization algorithm and a noise filtering algorithm to obtain the target video frame;

[0014] The gate target contained in the target video frame is identified.

[0015] Optionally, determining the door state of the door target according to the area of ​​the door target includes:

[0016] Query the preset historical video frame library to determine the gate object with the largest area as the largest gate object;

[0017] Calculating the ratio between the gate target and the maximum gate target corresponding to the target video frame to obtain bounding box ratio data, and storing the target video frame in the historical video frame library;

[0018] Determine whether the bounding box ratio data is less than a preset first threshold,

[0019] If yes, the door state of the door target is an open state;

[0020] If not, the door state of the door object is a closed state.

[0021] Optionally, the reference object target is a door opening warning light target or a marking line target, wherein:

[0022] The door opening warning light target displays a first warning state when the door target of the operating room is opened, and displays a second warning state when the door target during the operation is closed;

[0023] The marking line target is arranged on the ground and is vertical to the horizontal plane where the door target of the operating room is located.

[0024] Optionally, when the reference object target is the door opening warning light target, determining the positional relationship between the person target and the door target according to the reference object target includes:

[0025] Determine a person boundary box of the person target, and determine a person center point according to the person boundary box;

[0026] Determine an external bounding box associated with the door object, and determine the door direction of the door object;

[0027] Determining an intrusion bounding box according to the size of the circumscribed bounding box and the door direction;

[0028] Based on the ray method, determine whether the center point of the person is within the intrusion boundary box.

[0029] If yes, when the door opening warning light target is in the first warning state, the position relationship is the intrusion relationship, and when the door opening warning light target is in the second warning state, the position relationship is the non-intrusion relationship;

[0030] If not, the position relationship is determined to be the non-intrusion relationship.

[0031] Optionally, when the reference object target is a marked line target, determining the positional relationship between the person target and the door target according to the reference object target includes:

[0032] Determine a person boundary box of the person target, and determine a person center point according to the person boundary box;

[0033] Determine an external bounding box associated with the door object, and determine the door direction of the door object;

[0034] Determining an intrusion bounding box according to the size of the circumscribed bounding box and the door direction;

[0035] Based on the ray method, determine whether the center point of the person is within the intrusion boundary box.

[0036] If yes, determine the vertical distance between the personnel target and the door target according to the marking line target, if the vertical distance is less than a preset second threshold, the position relationship is the intrusion relationship, if the vertical distance is greater than or equal to the second threshold, the position relationship is the non-intrusion relationship;

[0037] If not, the position relationship is determined to be the non-intrusion relationship.

[0038] Optionally, generating an alarm instruction according to the position relationship and reporting it to a preset alarm device includes:

[0039] When the position relationship is the intrusion relationship, based on a preset number of video frame acquisitions, a video frame sequence after the target video frame is acquired;

[0040] Selecting a video frame in the video frame sequence as a first video frame;

[0041] Determine that a video frame before the first video frame is a second video frame, and calculate an intersection-over-union ratio of a person bounding box in the first video frame and a person bounding box in the second video frame;

[0042] The step of selecting a video frame in the video frame sequence as the first video frame is performed until all the video frames in the video frame sequence are selected to obtain all the intersection-union ratios;

[0043] Take all the intersection over union ratios greater than a preset third threshold as the target intersection over union ratio. If the number of the target intersection over union ratios is greater than a preset fourth threshold, generate the alarm instruction and report it to the alarm device.

[0044] Optionally, the generating the alarm instruction and reporting it to the alarm device includes:

[0045] Perform a secondary inspection on the video frame sequence according to a pre-trained object detection model. If the secondary inspection meets a preset inspection standard, generate the alarm instruction and report it to the alarm device.

[0046] Optionally, the performing a secondary inspection on the video frame sequence according to a pre-trained object detection model includes:

[0047] Identify the door target and the person target of the target video frame through the object detection model, and process the recognition result through a preset intersection over union algorithm to determine the secondary inspection result.

[0048] According to a second aspect of the embodiments of the present application, there is provided an operating room personnel intrusion detection device, including:

[0049] A first recognition module, configured to recognize a target video frame associated with an operating room and determine a door target included in the target video frame;

[0050] A first determination module, configured to determine a door state of the door target according to the area of the door target, where the door state includes an open state and a closed state;

[0051] A second recognition module, configured to recognize a person target included in the target video frame when the door state is the closed state;

[0052] A second determination module, configured to recognize a reference object target in the target video frame and determine a position relationship between the person target and the door target according to the reference object target, where the position relationship is an intrusion relationship or a non-intrusion relationship;

[0053] An alarm module, configured to generate an alarm instruction according to the position relationship and report it to a preset alarm device.

[0054] According to a third aspect of the embodiments of the present application, there is provided a computing device, including:

[0055] A memory and a processor;

[0056] The memory is used to store computer-executable instructions, and when the processor executes the computer-executable instructions, the steps of the operating room personnel intrusion detection method are implemented.

[0057] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the method for detecting intrusion of personnel into an operating room are implemented.

[0058] According to a fifth aspect of the embodiments of the present application, a chip is provided, which stores a computer program, and when the computer program is executed by the chip, the steps of the method for detecting intrusion of personnel into the operating room are implemented.

[0059] The method for detecting intrusion of personnel in the operation room provided by the present application is to identify the target video frame associated with the operation room, determine the door target contained in the target video frame; determine the door state of the door target according to the area of ​​the door target, wherein the door state includes an open state and a closed state; when the door state is the closed state, identify the personnel target contained in the target video frame; identify the reference object target in the target video frame, and determine the positional relationship between the personnel target and the door target according to the reference object target, wherein the positional relationship is an intrusion relationship or a non-intrusion relationship; generate an alarm instruction according to the positional relationship and report it to a preset alarm device. The method realizes real-time identification of the switch state of the door, locates the specific position of the door, detects whether there is a person entering or staying in the designated area, reduces missed detection and false detection, and provides reliable safety protection for industrial production. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 This is a flow chart of a method for detecting intrusion of a person into an operating room provided by an embodiment of the present application;

[0062] Figure 2 It is a processing flow chart of a method for detecting intrusion of a person into an operating room provided by an embodiment of the present application;

[0063] Figure 3 It is a structural schematic diagram of an operating room personnel intrusion detection device provided by an embodiment of the present application;

[0064] Figure 4 It is a structural block diagram of a computing device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0065] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0066] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms of "a", "said" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.

[0067] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0068] First, the terms involved in one or more embodiments of the present invention are explained.

[0069] YOLO: A single CNN model is used to implement end-to-end target detection. The core idea is to use the entire image as the input of the network and directly regress the position of the bounding box and its category in the output layer.

[0070] ViT: is an innovative model that introduces the Transformer model into the field of computer vision. The ViT model divides the image into multiple small patches (patches) and then treats these small patches as tokens and inputs them into the Transformer encoder, thereby achieving efficient image processing and classification.

[0071] In the present application, a method for detecting intrusion of a person in an operation room is provided. The present application also relates to an intrusion detection device for a person in an operation room, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0072] Figure 1 A flowchart of a method for detecting intrusion of a person into an operating room according to an embodiment of the present application is shown, which specifically includes the following steps:

[0073] Step S102: identifying a target video frame associated with the operating room, and determining a door target included in the target video frame;

[0074] Step S104: determining the door state of the door target according to the area of ​​the door target, wherein the door state includes an open state and a closed state;

[0075] Step S106: when the door state is the closed state, identifying a human target included in the target video frame;

[0076] Step S108: identifying a reference object in the target video frame, and determining a positional relationship between the person object and the door object according to the reference object, wherein the positional relationship is an intrusion relationship or a non-intrusion relationship;

[0077] Step S110: Generate an alarm instruction according to the position relationship and report it to a preset alarm device.

[0078] Among them, the target video frame is collected by a visual sensor arranged outside the door of the operation room. The visual sensor collects images in real time, and the image collected at the current moment is the target video frame; the operation room is engineering machinery, such as a tower crane, a forklift, a bulldozer, an excavator, a dump truck, a crane, a cement pump truck, a ship crane, a pile driver and a long-arm excavator, or a monitoring room, etc. The specific type of the operation room is determined by the actual use scenario and is not limited in this embodiment; the alarm device is composed of one or more of warning lights, speakers, displays and other devices. The specific composition is determined by the actual use scenario and is not limited in this embodiment.

[0079] Based on this, the target video frames obtained by the acquisition are subjected to target detection through the target detection algorithm. In order to ensure the detection efficiency, the processing speed is used as the selection index of the target detection algorithm. For example, the YOLO series of target detection algorithms are used to identify the target video frames and determine the door targets contained in the target video frames; according to the area of ​​the door target, it is determined whether the door state corresponding to the door target is in the open state or the closed state.

[0080] When the door target is in the closed state, the target video frame is also identified by the YOLO series target detection algorithm to determine the human target contained in the target video frame; the target video frame is also identified by the YOLO series target detection algorithm to determine the reference target contained in the target video frame, and the positional relationship between the human target and the door target is determined to be an intrusion relationship or a non-intrusion relationship according to the reference target; finally, an alarm instruction is generated according to the positional relationship. When there is a human intrusion, that is, the positional relationship is an intrusion relationship, the alarm device is reported, and the alarm device issues an alarm.

[0081] Furthermore, in step S102, the process of identifying the target video frame associated with the operating room and determining the door target contained in the target video frame is specifically implemented as follows in this embodiment:

[0082] The candidate video frames associated with the operating room are collected; the candidate video frames are processed based on a preset brightness equalization algorithm and a noise filtering algorithm to obtain the target video frame; and the door target contained in the target video frame is identified.

[0083] Among them, the candidate video frame is the initial picture collected by the visual sensor. After preprocessing, the target video frame can be obtained. The preprocessing process includes brightness balance and denoising.

[0084] Based on this, the brightness of the candidate video frame is balanced by a brightness balance algorithm, and the candidate video frame is denoised by a noise filtering algorithm to ensure that the image is clear and stable under different lighting conditions. Specifically, for the candidate video frame , the first target video frame obtained after preprocessing x Line y The pixel value of the column , as shown in the following formula:

[0085] ,

[0086] in, Candidate video frames No. x Line y The pixel value of the column, Candidate video frames The minimum pixel value, Candidate video frames The maximum pixel value, t is the time corresponding to the target video frame.

[0087] Furthermore, in step S104, the process of determining the door state of the door target according to the area of ​​the door target is specifically implemented as follows in this embodiment:

[0088] Query a preset historical video frame library to determine that the gate target with the largest area is the maximum gate target; calculate the ratio between the gate target corresponding to the target video frame and the maximum gate target to obtain the bounding box proportion data, and store the target video frame in the historical video frame library; determine whether the bounding box proportion data is less than a preset first threshold, if so, the door state of the gate target is open; if not, the door state of the gate target is closed.

[0089] Among them, the target video frame is the image acquired at the current moment and needs to be stored in the historical video frame library in real time. For the video frames contained in the historical video frame library, the gate target with the largest area contained therein is the largest gate target.

[0090] Based on this, Figure 2 As shown in the processing flow chart of a method for detecting intrusion of operating room personnel provided, continuous video frames are obtained, that is, the target video frame at the current moment is obtained in real time, and then the door opening and closing detection is performed and the low confidence is filtered out. Specifically, the area of ​​the door target in the target video frame is calculated. , and the area of ​​the maximum gate target The ratio between them is used to obtain the bounding box ratio data. The process is shown in the following formula:

[0091] ,

[0092] When the bounding box accounts for the data Less than the first threshold In the case of, the door state of the door target is judged to be open; otherwise, when The door state of the door target is closed. It should be noted that the first threshold .

[0093] Furthermore, the reference target is a door opening warning light target or a marking line target, wherein the door opening warning light target displays a first warning state when the door target of the operating room is opened, and displays a second warning state when the door target is closed during the operation; the marking line target is arranged on the ground and is perpendicular to the horizontal plane where the door target of the operating room is located.

[0094] The first warning state corresponds to one of the door opening warning light target being on or off, and the second warning state corresponds to the other of the door opening warning light target being on or off, and the first warning state indicates that the operating room door is open, and the second warning state indicates that the operating room door is closed. The marking line target is a sign painted or posted on the ground. In order to facilitate the recognition of the relevant algorithm, it can be arranged in a color that is significantly different from the ground color.

[0095] Furthermore, when the reference object target is a door opening warning light target, in step S108, the process of determining the positional relationship between the person target and the door target according to the reference object target is specifically implemented as follows in this embodiment:

[0096] Determine a personnel boundary box of the personnel target, and determine a personnel center point based on the personnel boundary box; determine an external boundary box associated with the door target, and determine a door direction of the door target; determine an intrusion boundary box based on the size of the external boundary box and the door direction; based on the ray method, determine whether the personnel center point is within the intrusion boundary box; if so, when the door opening warning light target is in the first warning state, the positional relationship is the intrusion relationship, and when the door opening warning light target is in the second warning state, the positional relationship is the non-intrusion relationship; if not, determine that the positional relationship is the non-intrusion relationship.

[0097] Among them, Figure 2 As shown in the processing flow chart of a method for detecting intrusion of personnel into an operating room, after a door closing state exists, personnel target detection is performed to filter low confidence, such as identifying the personnel target contained in the target video frame in step S106, and then personnel alarm area filtering is performed to retain personnel staying in the suspected door closing state.

[0098] Based on this, in order to filter the personnel alarm area and retain the process of people staying in the suspected closed door state, first determine the personnel boundary box of the personnel target , determine the person bounding box The geometric center of , personnel center The determination process is shown in the following formula:

[0099] ,

[0100] in, Person Bounding Box The minimum value of the horizontal axis in Person Bounding Box The maximum value of the horizontal axis in Person Bounding Box The minimum value of the vertical coordinate in Person Bounding Box The maximum value of the vertical axis in .

[0101] It should be noted that in order to determine whether there is a person breaking into the operating room door, it is necessary to determine whether the person has the action or tendency to open the operating room door. In actual scenarios, whether it is a single-door or double-door, the person's position will be close to the door handle and away from the door hinge when opening the door. Therefore, the intrusion boundary box that represents the person breaking into the operating room door range and having the tendency to open the operating room door does not completely overlap with the external boundary box associated with the door target. The width of the external rectangular box is multiplied by a preset first proportional coefficient to obtain the width of the intrusion boundary box. The specific value of the first proportional coefficient is determined by the actual application scenario, such as 0.9, 1.1, etc., and is not limited in this embodiment.

[0102] In addition, the geometric centers of the intrusion bounding box and the external bounding box do not coincide with each other. After determining the handle direction and hinge direction of the door target through the door direction, the position of the intrusion bounding box is translated toward the handle direction, that is, away from the hinge direction, compared with the position of the external bounding box. The translation distance is determined by multiplying the width of the external bounding box by a preset second proportional coefficient. The specific value setting of the second proportional coefficient is also determined by the actual application scenario, such as 0.1, 0.15, etc., and is not limited in this embodiment.

[0103] Then, the ray method is used to determine whether the center point of the person is within the intrusion boundary box. If the center point of the person is within the intrusion boundary box, a secondary judgment is further performed through the door opening warning light target. If the door opening warning light target is in the first warning state, indicating that the operating room door is opened, then it is determined that the person target has opened the operating room door without authorization, and the position relationship is an intrusion relationship. On the contrary, if the door opening warning light target is in the second warning state, indicating that the operating room door is not opened, then it is determined that the person target has not opened the operating room door without authorization, and the position relationship is a non-intrusion relationship; and when the center point of the person is outside the intrusion boundary box, there is no need to perform a secondary judgment through the door opening warning light target, and the position relationship is directly determined to be a non-intrusion relationship.

[0104] Furthermore, when the reference object target is a marking line target, in step S108, the process of determining the positional relationship between the person target and the door target according to the reference object target is specifically implemented as follows in this embodiment:

[0105] Determine a personnel boundary box of the personnel target, and determine a personnel center point based on the personnel boundary box; determine an external boundary box associated with the door target, and determine a door direction of the door target; determine an intrusion boundary box based on the size of the external boundary box and the door direction; based on the ray method, determine whether the center point of the personnel is within the scope of the intrusion boundary box, if so, determine a vertical distance between the personnel target and the door target based on the marking line target, if the vertical distance is less than a preset second threshold, the position relationship is the intrusion relationship, if the vertical distance is greater than or equal to the second threshold, the position relationship is the non-intrusion relationship; if not, determine that the position relationship is the non-intrusion relationship.

[0106] Among them, the process of judging whether the center point of the person is within the scope of the intrusion boundary box is similar to the judgment process of the reference object target being the door opening warning light target, and will not be repeated in this embodiment. When it is judged that the center point of the person is within the scope of the intrusion boundary box, a secondary judgment is made through the marking line target. Specifically, the relative position relationship between the marking line target and the person target is determined. According to the determination result, the vertical distance between the person target and the door target can be determined, that is, the center point of the person target is perpendicular to the plane where the door target is located, and the length of the perpendicular line is recorded as the vertical distance. The reference object target has a scale line, and the scale line aggregates the scale of the conversion between the target video frame and the actual physical space distance.

[0107] Based on this, the specific value of the vertical distance is determined by the actual application scenario, such as 0.4 meters, 0.5 meters, etc. When the vertical distance is greater than or equal to the second threshold, it proves that the distance between the personnel target and the door target is far away and there is no possibility of breaking into the operation room. In this case, the position relationship is determined to be a non-intrusion relationship; on the contrary, when the vertical distance is less than the second threshold, it proves that the distance between the personnel target and the door target is close and there is a possibility of breaking into the operation room. In this case, the position relationship is determined to be an intrusion relationship. Similarly, when the center point of the personnel is outside the intrusion boundary box, there is no need to make a secondary judgment through the marking line target, and the position relationship is directly determined to be a non-intrusion relationship.

[0108] Furthermore, in step S110, the process of generating an alarm instruction according to the position relationship and reporting it to a preset alarm device is specifically implemented as follows in this embodiment:

[0109] When the positional relationship is the intrusion relationship, based on a preset number of video frame acquisitions, a video frame sequence after the target video frame is acquired; a video frame in the video frame sequence is selected as the first video frame; a video frame previous to the first video frame is determined to be the second video frame, and an intersection-and-union ratio between a person boundary box in the first video frame and a person boundary box in the second video frame is calculated; the step of selecting a video frame in the video frame sequence as the first video frame is performed until all video frames in the video frame sequence are selected to obtain all the intersection-and-union ratios; among all the intersection-and-union ratios, the intersection-and-union ratios that are greater than a preset third threshold are used as target intersection-and-union ratios, and if the number of the target intersection-and-union ratios is greater than a preset fourth threshold, the alarm instruction is generated and reported to the alarm device.

[0110] Among them, Figure 2 As shown in the processing flow chart of the provided method for detecting intrusion of personnel into the operating room, after retaining the suspected state of the person staying in the door-closed state, that is, determining that the position relationship is an intrusion relationship, it is also necessary to execute a multi-frame confirmation process of the closed door person intrusion, which is achieved through the execution steps of image information and alarm information box list-multi-frame confirmation.

[0111] Specifically, the multi-frame confirmation process is based on a preset video frame sequence of the number of video frames collected after the target video frame is collected. It should be noted that the setting of the number of video frames collected is determined by the actual usage scenario, such as 10 frames, 15 frames, etc., and this embodiment does not limit it. , determine the person bounding box of the previous video frame of this video frame , calculate the intersection and union ratio of the two , as shown in the following formula:

[0112] ,

[0113] When the intersection-and-union ratio calculation result is greater than the third threshold In the case of a target I / O ratio, the I / O ratio is recorded as the target I / O ratio. After calculating the I / O ratios corresponding to all video frames in the video frame sequence and comparing them with the third threshold, the number of all target I / O ratios is determined. When the number is greater than the fourth threshold, the multi-frame confirmation result shows that the positional relationship corresponding to the target video frame is the intrusion relationship, and the determination is correct. On the contrary, if the number of target I / O ratios is less than or equal to the fourth threshold, then it is proved that the determination is wrong and there is no need to generate the alarm instruction. It should be noted that the value of the fourth threshold is less than or equal to the number of video frame acquisitions.

[0114] Furthermore, in step S110, the process of generating an alarm instruction and reporting it to the alarm device is specifically implemented as follows in this embodiment:

[0115] According to the pre-trained target detection model, the video frame sequence is subjected to a secondary inspection. If the secondary inspection meets the preset inspection standard, the alarm instruction is generated and reported to the alarm device.

[0116] Furthermore, according to the pre-trained target detection model, the video frame sequence is subjected to a secondary inspection. In this embodiment, the specific implementation method is as follows:

[0117] The door target and the person target of the target video frame are identified through the target detection model, and the identification result is processed through a preset intersection-over-union algorithm to determine the secondary inspection result.

[0118] Among them, Figure 2 As shown in the processing flow chart of the provided method for detecting intrusion of personnel into the operating room, after multiple frames confirm that the person intruding with the door closed, a secondary inspection is required. The secondary inspection is carried out through ViT (Co-DETR) door closing detection and IOU with the door closing after multiple frames confirmation, as well as ViT (Co-DETR) personnel detection and IOU with the personnel after multiple frames confirmation, to achieve ViT secondary verification to confirm the intrusion of the person closing the door, and issue a warning for the intrusion of the person closing the door.

[0119] Specifically, in order to ensure the detection accuracy of the YOLO algorithm, a target detection model is added for secondary inspection. Since the occurrence of human intrusion is an accidental event, even if the model used in the secondary inspection has a lower inspection efficiency, it does not affect the overall inspection efficiency. Therefore, the accuracy of the secondary inspection is the selection criterion for the target detection model, so the target detection model adopts the ViT model.

[0120] Based on this, after multiple frames are confirmed, the position relationship corresponding to the target video frame is an intrusion relationship. Then, for the target video frame, the door target recognition and the person target recognition are performed respectively through the ViT model, and the door target of the ViT model is calculated. Recognition results and gate targets after multi-frame confirmation The intersection ratio , calculate the personnel target of the ViT model Recognition results and human targets after multi-frame confirmation The intersection ratio , the specific calculation is shown in the following formula:

[0121] ,

[0122] ,

[0123] in, The first condition is that the value is greater than the preset fifth threshold value. The second condition is that the value is greater than the preset sixth threshold value. When both the first and second conditions are met, the secondary inspection result indicates that the positional relationship corresponding to the target video frame is an intrusion relationship. Conversely, if either the first or second condition is not met, the secondary inspection result indicates that the multi-frame confirmation result is a misjudgment, and the positional relationship corresponding to the target video frame is a non-intrusion relationship.

[0124] Corresponding to the above method embodiment, the present application also provides an embodiment of a device for detecting intrusion of personnel into an operating room, Figure 3 FIG. 2 shows a schematic diagram of a structure of a device for detecting intrusion of personnel into an operating room provided by an embodiment of the present application. Figure 3 As shown, the device comprises:

[0125] The first recognition module 302 is configured to recognize a target video frame associated with the operating room and determine a door target contained in the target video frame;

[0126] A first determination module 304 is configured to determine a door state of the door target according to the area of ​​the door target, wherein the door state includes an open state and a closed state;

[0127] A second recognition module 306 is configured to recognize a human target included in the target video frame when the door state is the closed state;

[0128] A second determination module 308 is configured to identify a reference object in the target video frame, and determine a positional relationship between the person object and the door object according to the reference object, wherein the positional relationship is an intrusion relationship or a non-intrusion relationship;

[0129] The alarm module 310 is configured to generate an alarm instruction according to the position relationship and report it to a preset alarm device.

[0130] In an optional embodiment, the first identification module 302 is further configured to:

[0131] The candidate video frames associated with the operating room are collected; the candidate video frames are processed based on a preset brightness equalization algorithm and a noise filtering algorithm to obtain the target video frame; and the door target contained in the target video frame is identified.

[0132] In an optional embodiment, the first determining module 304 is further configured to:

[0133] Query a preset historical video frame library to determine that the gate target with the largest area is the maximum gate target; calculate the ratio between the gate target corresponding to the target video frame and the maximum gate target to obtain the bounding box proportion data, and store the target video frame in the historical video frame library; determine whether the bounding box proportion data is less than a preset first threshold, if so, the door state of the gate target is open; if not, the door state of the gate target is closed.

[0134] In an optional embodiment, in the operation room personnel intrusion detection device, the reference target is a door opening warning light target or a marking line target, wherein the door opening warning light target displays a first warning state when the door target of the operation room is opened, and displays a second warning state when the door target is closed during the operation; the marking line target is arranged on the ground and is perpendicular to the horizontal plane where the door target of the operation room is located.

[0135] In an optional embodiment, the second determining module 308 is further configured to:

[0136] Determine a personnel boundary box of the personnel target, and determine a personnel center point based on the personnel boundary box; determine an external boundary box associated with the door target, and determine a door direction of the door target; determine an intrusion boundary box based on the size of the external boundary box and the door direction; based on the ray method, determine whether the personnel center point is within the intrusion boundary box; if so, when the door opening warning light target is in the first warning state, the positional relationship is the intrusion relationship, and when the door opening warning light target is in the second warning state, the positional relationship is the non-intrusion relationship; if not, determine that the positional relationship is the non-intrusion relationship.

[0137] In an optional embodiment, the second determining module 308 is further configured to:

[0138] Determine a personnel boundary box of the personnel target, and determine a personnel center point based on the personnel boundary box; determine an external boundary box associated with the door target, and determine a door direction of the door target; determine an intrusion boundary box based on the size of the external boundary box and the door direction; based on the ray method, determine whether the center point of the personnel is within the scope of the intrusion boundary box, if so, determine a vertical distance between the personnel target and the door target based on the marking line target, if the vertical distance is less than a preset second threshold, the position relationship is the intrusion relationship, if the vertical distance is greater than or equal to the second threshold, the position relationship is the non-intrusion relationship; if not, determine that the position relationship is the non-intrusion relationship.

[0139] In an optional embodiment, the alarm module 310 is further configured to:

[0140] When the positional relationship is the intrusion relationship, based on a preset number of video frame acquisitions, a video frame sequence after the target video frame is acquired; a video frame in the video frame sequence is selected as the first video frame; a video frame previous to the first video frame is determined to be the second video frame, and an intersection-and-union ratio between a person boundary box in the first video frame and a person boundary box in the second video frame is calculated; the step of selecting a video frame in the video frame sequence as the first video frame is performed until all video frames in the video frame sequence are selected to obtain all the intersection-and-union ratios; among all the intersection-and-union ratios, the intersection-and-union ratios that are greater than a preset third threshold are used as target intersection-and-union ratios, and if the number of the target intersection-and-union ratios is greater than a preset fourth threshold, the alarm instruction is generated and reported to the alarm device.

[0141] In an optional embodiment, the alarm module 310 is further configured to:

[0142] According to the pre-trained target detection model, the video frame sequence is subjected to a secondary inspection. If the secondary inspection meets the preset inspection standard, the alarm instruction is generated and reported to the alarm device.

[0143] In an optional embodiment, the alarm module 310 is further configured to:

[0144] The door target and the person target of the target video frame are identified through the target detection model, and the identification result is processed through a preset intersection-over-union algorithm to determine the secondary inspection result.

[0145] The operating room personnel intrusion detection device provided by the present application determines the door target contained in the target video frame by identifying the target video frame associated with the operating room; determines the door state of the door target according to the area of ​​the door target, wherein the door state includes an open state and a closed state; when the door state is the closed state, identifies the personnel target contained in the target video frame; identifies the reference object target in the target video frame, and determines the positional relationship between the personnel target and the door target according to the reference object target, wherein the positional relationship is an intrusion relationship or a non-intrusion relationship; generates an alarm instruction according to the positional relationship and reports it to a preset alarm device. It realizes real-time identification of the switch state of the door, locates the specific position of the door, detects whether there is a person entering or staying in the designated area, reduces missed detection and false detection, and provides reliable safety protection for industrial production.

[0146] The above is a schematic scheme of an operating room personnel intrusion detection device of this embodiment. It should be noted that the technical scheme of the operating room personnel intrusion detection device and the technical scheme of the operating room personnel intrusion detection method mentioned above belong to the same concept. For the details not described in detail in the technical scheme of the operating room personnel intrusion detection device, please refer to the description of the technical scheme of the operating room personnel intrusion detection method mentioned above. In addition, each component in the device embodiment should be understood as a functional module that must be established to implement each step of the program flow or each step of the method, and each functional module is not an actual functional division or separation definition. The device claim defined by such a group of functional modules should be understood as a functional module architecture that mainly implements the solution through the computer program recorded in the specification, and should not be understood as a physical device that mainly implements the solution through hardware.

[0147] Figure 4 The block diagram of a computing device 400 according to an embodiment of the present application is shown. The components of the computing device 400 include but are not limited to a memory 410 and a processor 420. The processor 420 is connected to the memory 410 via a bus 430, and the database 450 is used to store data.

[0148] The computing device 400 also includes an access device 440 that enables the computing device 400 to communicate via one or more networks 460. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 440 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0149] In one embodiment of the present application, the above components of the computing device 400 and Figure 4 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 4 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0150] Computing device 400 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. Computing device 400 may also be a mobile or stationary server.

[0151] The processor 420 is used to execute computer executable instructions of each step of the method for detecting intrusion of operating room personnel.

[0152] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned operation room personnel intrusion detection method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned operation room personnel intrusion detection method.

[0153] An embodiment of the present application further provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, are used to execute the steps of the method for detecting intrusion of a person into an operating room.

[0154] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned operation room personnel intrusion detection method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned operation room personnel intrusion detection method.

[0155] An embodiment of the present application further provides a chip storing a computer program, which implements the steps of the method for detecting intrusion of personnel into an operating room when the computer program is executed by the chip.

[0156] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0157] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0158] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0159] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0160] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the present application. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can understand and use the present application well. The present application is only limited by the claims and their full scope and equivalents.

Claims

1. A method for detecting intrusion of personnel into an operating room, characterized in that: include: identifying a target video frame associated with the operating room, and determining a door target contained in the target video frame; Determining a door state of the door target according to the area of ​​the door target, wherein the door state includes an open state and a closed state; When the door state is the closed state, identifying a human target contained in the target video frame; Identify the reference object target in the target video frame, and determine the positional relationship between the personnel target and the door target based on the reference object target, wherein the positional relationship is an intrusion relationship or a non-intrusion relationship, and the reference object target is a door opening warning light target or a marking line target. The door opening warning light target displays a first warning state when the door target of the operating room is opened, and displays a second warning state when the door target of the operating room is closed. Specifically, when the reference object target is the door opening warning light target, determine the personnel bounding box of the personnel target, and determine the personnel center point based on the personnel bounding box, determine the external bounding box associated with the door target, and determine the door direction of the door target, determine the intrusion bounding box based on the size of the external bounding box and the door direction, and determine whether the personnel center point is within the intrusion bounding box based on the ray method. If so, when the door opening warning light target is in the first warning state, the position The relationship is the intrusion relationship. When the door opening warning light target is the second warning state, the position relationship is the non-intrusion relationship. If not, the position relationship is determined to be the non-intrusion relationship. When the reference object target is the marked line target, the person boundary box of the person target is determined, and the person center point is determined according to the person boundary box, the external bounding box associated with the door target is determined, and the door direction of the door target is determined. According to the size of the external bounding box and the door direction, the intrusion boundary box is determined. Based on the ray method, it is judged whether the person center point is within the range of the intrusion boundary box. If so, according to the marked line target, the vertical distance between the person target and the door target is determined. If the vertical distance is less than a preset second threshold, the position relationship is the intrusion relationship. If the vertical distance is greater than or equal to the second threshold, the position relationship is the non-intrusion relationship. If not, the position relationship is determined to be the non-intrusion relationship. According to the position relationship, an alarm instruction is generated and reported to a preset alarm device.

2. The method according to claim 1, characterized in that The identifying the target video frame associated with the operating room and determining the door target contained in the target video frame includes: Acquiring candidate video frames associated with the operating room; Processing the candidate video frame based on a preset brightness equalization algorithm and a noise filtering algorithm to obtain the target video frame; The gate target contained in the target video frame is identified.

3. The method according to claim 1, characterized in that The determining the door state of the door target according to the area of ​​the door target includes: Query the preset historical video frame library to determine the gate object with the largest area as the largest gate object; Calculating the ratio between the gate target and the maximum gate target corresponding to the target video frame to obtain bounding box ratio data, and storing the target video frame in the historical video frame library; Determine whether the bounding box ratio data is less than a preset first threshold, If yes, the door state of the door target is an open state; If not, the door state of the door object is a closed state.

4. The method according to claim 1, characterized in that: The reference object target is a door opening warning light target or a marking line target, wherein: The door opening warning light target displays a first warning state when the door target of the operating room is opened, and displays a second warning state when the door target of the operating room is closed; The marking line target is arranged on the ground and is vertical to the horizontal plane where the door target of the operating room is located.

5. The method according to claim 1, characterized in that The generating of an alarm instruction according to the position relationship and reporting to a preset alarm device includes: When the position relationship is the intrusion relationship, based on a preset number of video frame acquisitions, a video frame sequence after the target video frame is acquired; Selecting a video frame in the video frame sequence as a first video frame; Determine that a video frame before the first video frame is a second video frame, and calculate an intersection-over-union ratio of a person bounding box in the first video frame and a person bounding box in the second video frame; The step of selecting a video frame in the video frame sequence as the first video frame is performed until all the video frames in the video frame sequence are selected to obtain all the intersection-union ratios; The intersection-and-union ratios greater than the preset third threshold value among all the intersection-and-union ratios are taken as target intersection-and-union ratios. If the number of the target intersection-and-union ratios is greater than the preset fourth threshold value, the alarm instruction is generated and reported to the alarm device.

6. The method according to claim 5, characterized in that The generating the alarm instruction and reporting it to the alarm device comprises: According to the pre-trained target detection model, the video frame sequence is subjected to a secondary inspection. If the secondary inspection meets the preset inspection standard, the alarm instruction is generated and reported to the alarm device.

7. The method according to claim 6, characterized in that The performing a secondary inspection on the video frame sequence according to the pre-trained target detection model comprises: The door target and the person target of the target video frame are identified through the target detection model, and the identification result is processed through a preset intersection-over-union algorithm to determine the secondary inspection result.

8. An operating room personnel intrusion detection device, characterized in that: include: A first recognition module is configured to recognize a target video frame associated with the operating room and determine a door target contained in the target video frame; A first determination module is configured to determine a door state of the door target according to an area of ​​the door target, wherein the door state includes an open state and a closed state; A second recognition module is configured to recognize a human target contained in the target video frame when the door state is the closed state; The second determination module is configured to identify the reference object target in the target video frame, and determine the positional relationship between the personnel target and the door target according to the reference object target, wherein the positional relationship is an intrusion relationship or a non-intrusion relationship, and the reference object target is a door opening warning light target or a marking line target. The door opening warning light target displays a first warning state when the door target of the operating room is opened, and displays a second warning state when the door target of the operating room is closed. Specifically, when the reference object target is the door opening warning light target, the personnel bounding box of the personnel target is determined, and the personnel center point is determined according to the personnel bounding box, the external bounding box associated with the door target is determined, and the door direction of the door target is determined, and the intrusion bounding box is determined according to the size of the external bounding box and the door direction. Based on the ray method, it is judged whether the center point of the personnel is within the range of the intrusion bounding box. If so, when the door opening warning light target is in the first warning state , then the position relationship is the intrusion relationship. When the door opening warning light target is the second warning state, the position relationship is the non-intrusion relationship. If not, the position relationship is determined to be the non-intrusion relationship. When the reference object target is the marked line target, the personnel boundary box of the personnel target is determined, and the personnel center point is determined according to the personnel boundary box. The external boundary box associated with the door target is determined, and the door direction of the door target is determined. According to the size of the external boundary box and the door direction, the intrusion boundary box is determined. Based on the ray method, it is determined whether the center point of the personnel is within the range of the intrusion boundary box. If so, according to the marked line target, the vertical distance between the personnel target and the door target is determined. If the vertical distance is less than the preset second threshold, the position relationship is the intrusion relationship. If the vertical distance is greater than or equal to the second threshold, the position relationship is the non-intrusion relationship. If not, the position relationship is determined to be the non-intrusion relationship. The alarm module is configured to generate an alarm instruction according to the position relationship and report it to a preset alarm device.

Images