Method, device, electronic device, and storage medium for detecting risky passenger behavior

Abstract

A method, device, electronic device, and storage medium for detecting risky behavior of passengers, the detection method including, when it is detected that the elevator doors are closed, collecting an image inside the car with a sensor and identifying a door vicinity area from the image (S101), identifying a passenger area from the door vicinity area and identifying a position of the passenger area in the door vicinity area (S102), calculating an occupancy rate of the passenger area in the door vicinity area based on the position (S103), and, when the occupancy rate is greater than a predetermined occupancy rate threshold, identifying that there is a passenger leaning against the door in the car (S104), and, when a door open command is received, identifying a target hand entrapment risk area to which the passenger area belongs out of at least two hand entrapment risk areas in the door vicinity area based on the position (S105), and controlling the door based on the target hand entrapment risk area (S106).

Description

[Technical Field]

[0001] This application claims priority to a Chinese patent application bearing application number 202211626696.X, filed with the China Patent Office on December 16, 2022, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the technical field of elevator control, and relates to, for example, a method, a device, an electronic device, and a storage medium for detecting dangerous behavior of passengers. [Background technology]

[0003] Elevators are used in high-rise buildings, and at certain times (e.g., rush hour), when the elevator car is full and crowded, passengers inside the car may lean against the door due to the crowding, and if the door suddenly opens, there is a risk that the passenger leaning against the door will fall out of the car and cause an accident.Alternatively, if a passenger is leaning against the door and the door opens upon arrival at the destination floor, there is a risk that the passenger's fingers, clothing, etc. will be caught in the gap between the door and the front wall of the car.

[0004] In related technologies, hand entrapment prevention detection focuses on the action of a hand getting caught in the gap between two doors as the doors close. When detecting a hand getting caught in the front wall of the car as the doors open, an alarm is usually detected by detecting the distance between the passenger and the door or taking an image of the door area. When the door closes again after the hand entrapment action occurs, the passenger's hand may already be caught and injured. Furthermore, there is no reliable method for detecting and preventing door leaning, which makes it impossible to ensure the safety of passengers when riding in the elevator. Summary of the Invention [Problem to be solved by the invention]

[0005] The present application provides a reliable detection method to prevent hand entrapment and door leaning, and provides a method, device, electronic device and storage medium for detecting dangerous passenger behavior to ensure safety when passengers board elevators. [Means for solving the problem]

[0006] This application is When detecting that the elevator doors are closed, an image of the inside of the car is acquired by a sensor, and an area near the doors is identified from the image; Identifying a passenger area from the near-door area and identifying a position of the passenger area in the near-door area; calculating an occupancy rate of the near-door region of the passenger area based on the location; Identifying a passenger leaning against the door in the car if the occupancy rate is greater than a preset occupancy rate threshold; When a door opening command is received, identifying a target hand entrapment risk area to which the passenger area belongs among at least two hand entrapment risk areas in the door vicinity area based on the position; and controlling the door based on a target hand entrapment risk area to which the passenger area belongs. A method for detecting risky passenger behavior is provided.

[0007] This application is a door vicinity area identification module configured to, when detecting that the elevator doors are in a closed state, capture an image of the interior of the car using a sensor and identify a door vicinity area from the image; a passenger area identification module configured to identify a passenger area from the near-door area and identify a location of the passenger area in the near-door area; an occupancy calculation module configured to calculate an occupancy rate in the near-door region of the passenger area based on the location; a door leaning identification module configured to identify a passenger leaning on the door in the car if the occupancy rate is greater than a preset occupancy rate threshold; a hand entrapment risk area identification module configured to identify a target hand entrapment risk area to which the passenger area belongs among at least two hand entrapment risk areas in the door vicinity area based on the position when a door opening command is received; a door control module configured to control the door based on a target hand entrapment risk area to which the passenger area belongs. To provide a device for detecting risky behavior of passengers.

[0008] This application is at least one processor; a memory communicatively coupled to the at least one processor; An electronic device comprising: the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to execute the method for detecting risky passenger behavior described herein; Provide electronic devices.

[0009] The present application provides a method for detecting risky passenger behavior, the method comprising: storing computer instructions for, when executed by a processor, implementing the method for detecting risky passenger behavior described herein; A computer-readable storage medium is provided.

[0010] It should be understood that the descriptions herein are not intended to identify key or important features of the embodiments of the present application, nor are they intended to limit the scope of the present application. [Brief explanation of the drawings]

[0011] The drawings that must be used in the description of the embodiments will now be described.

[0012] [Figure 1A] 1 is a flowchart of a method for detecting risky behavior of passengers according to a first embodiment of the present invention. [Figure 1B]FIG. 2 is a schematic diagram of a sensor attached in Example 1 of the present application. [Figure 1C] FIG. 2 is a schematic diagram of a door vicinity area in the first embodiment of the present invention. [Figure 1D] FIG. 2 is a schematic diagram of a region at risk of hand entrapment in Example 1 of the present application. [Figure 2] 10 is a flowchart of a method for detecting risky behavior of passengers according to a second embodiment of the present invention. [Figure 3] FIG. 10 is a structural schematic diagram of a passenger risky behavior detection device according to a third embodiment of the present invention. [Figure 4] FIG. 10 is a structural schematic diagram of an electronic device according to a fourth embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, the technical solutions in the embodiments of the present application will be described with reference to the drawings in the embodiments of the present application.

[0014] [Example 1] 1A is a flowchart of a method for detecting risky behavior of passengers according to a first embodiment of the present application, which is applied to detecting risky behavior of passengers leaning on doors while riding in an elevator and getting their hands caught as the doors open, and the method can be performed by a device for detecting risky behavior of passengers, which can be realized in the form of hardware and / or software, and which can be configured in electronic equipment, for example, in an elevator controller. As shown in FIG. 1A, the method for detecting risky behavior of passengers includes the following steps:

[0015] In S101, when it is detected that the elevator doors are closed, an image of the inside of the car is taken by a sensor, and the area near the door is identified from the image.

[0016] As shown in FIG. 1B, in this embodiment, a sensor 3 is attached to the elevator car. The sensor may be a general camera such as a black-and-white camera or a red-green-blue (RGB) camera, or an active optical sensor such as a time-of-flight (TOF) depth sensor or a structured light sensor, i.e., a sensor that senses by emitting and receiving light to obtain a depth image. Of course, the sensor may also be a radar, etc., and this embodiment does not limit the type of sensor.

[0017] In one example, the sensor of this embodiment may be an active optical sensor, and the sensor 3 may be attached to the car 1 such that the detection area of ​​the sensor 3 covers the area inside the car 1 and the area at the landing 2, and preferably, the sensor 3 may be attached to the header of the door of the car 1, for example, at a midpoint of the door header, such that the detection area of ​​the sensor 3 covers the area inside the car 1 and the area at the landing 2.

[0018] In one example, the number of sensors 3 may be one, i.e., the sensor 3 may be an active optical sensor with a large angle of view. When the number of sensors 3 is one, the angle of the sensor 3 may be adjustable or fixed, i.e., the angle between the light-emitting axis of the sensor 3 and the vertical direction may be variable, i.e., the angle of the sensor 3 may be adjusted so that the detection area of ​​the sensor 3 at the landing 2 may be enlarged or reduced.

[0019] In another example, the number of sensors 3 may be two, with the light-emitting axis of one of the two sensors 3 facing into car 1 to capture an image of the area within car 1, and the light-emitting axis of the other sensor 3 facing toward hall 2 to capture an image of the area at hall 2. Preferably, the detection ranges of the two sensors 3 overlap to ensure that images of the entire area from car 1 to hall 2 can be captured. Furthermore, if the sensor is an active optical sensor with one light-emitting axis facing vertically, this avoids the problem of overexposure that occurs after the elevator doors close when simultaneously capturing images of the entire area from car 1 to hall 2. Illustratively, the angles between the light-emitting axes of the two sensors 3 and the vertical direction can be adjusted to adjust the detection areas of the two sensors 3. Of course, the angles between the light-emitting axes of the two sensors 3 and the vertical direction may be fixed.

[0020] In this embodiment, the closed state of the door may mean that the door is completely closed or at least partially closed (the door is in the process of opening or closing), in which case the sensor 3 can be controlled to capture images of the inside of the car 1 according to a preset frame rate, and the images may be depth images, black and white images, color images, etc., and here the frame rate at which the sensor 3 captures the images may be a fixed frame rate or a dynamically adjustable frame rate.

[0021] The door vicinity area may be an area near the car door, and may be, for example, a rectangular area whose length is equal to the door and whose width is a preset dimension. As shown in Fig. 1C, the door 4 can be opened and closed between the car front wall 5, and the area surrounded by the car front wall 5 and the car side wall 7 is the area inside the car. The length of the door vicinity area 6 refers to the dimension of the door vicinity area in the direction in which the door 4 opens and closes, and the width of the door vicinity area 6 refers to the dimension of the door vicinity area in the direction perpendicular to the door 4. As shown in Fig. 1C, the length of the door vicinity area 4 is equal to the length of the two doors 4 after they are closed, and the width can be set arbitrarily. For example, the width may be 6 cm, 10 cm, etc.

[0022] In this embodiment, the door vicinity area 6 can be identified in the image in conjunction with the imaging principle based on the mounting height, focal length, etc. of the sensor 3. When the sensor 3 is fixed, pixel points in the captured image correspond one-to-one to multiple positions in the door vicinity area, and a preset pixel point area in the image may be identified as the door vicinity area.

[0023] In S102, a passenger area is identified from the door vicinity area, and the position of the passenger area in the door vicinity area is identified.

[0024] For example, the image captured by the sensor is a depth image. Since passengers have a certain height relative to the bottom of the car, the depth of the bottom of the car is set to 0, and the depth value of each pixel point in the area near the door in the image can be obtained. Pixel points with depth values ​​greater than a preset depth threshold are identified as pixel points corresponding to passengers, and the image area occupied in the area near the door by pixel points corresponding to passengers is identified as the passenger area.

[0025] In another embodiment, the sensor is an active optical sensor. When the passengers have a certain height and the intensity of the light reflected by the passengers is greater than the intensity of the light reflected by the car bottom, the image can be binarized based on the light energy value of the pixel points in the image to obtain the passenger area. For example, pixel points whose light intensity is greater than a preset energy threshold are colored black, and vice versa, are colored white to obtain the passenger area.

[0026] Of course, if the captured image is a color image, the image can be input into a target detection model to identify the occupant area from the image, and the present embodiment does not limit the manner of identifying the occupant area from the image.

[0027] As shown in FIG. 1C, a passenger area 8 is identified within the door vicinity area 6, and the number of passenger areas 8 may be one or more, and the passenger area 8 may be an area formed by a person within the door vicinity area 6, or an area formed by a pet, a wheelchair, etc.

[0028] After the passenger area is identified, the distances to the doors of the multiple portions of the passenger area can be identified in a direction perpendicular to the doors. As shown in FIG. 1C , the door near area 6 can be divided into multiple door near sub-areas according to the distances to the doors 4. In one example, the door near area 6 can be divided into three door near sub-areas (61, 62, 63). For example, the door near area 6 includes multiple rows of pixel points, and N rows of pixel points can be divided into one sub-area. For example, if the door near area 6 includes 6 rows of pixel points, the pixel points can be divided into one sub-area every two rows.

[0029] Identifying the position of the passenger area in the door vicinity region may involve identifying the passenger area belonging to each door vicinity sub-region, where as shown in FIG. 1C, the passenger area 8 is divided by three door vicinity sub-regions (61, 62, 63), and identifying pixel points of the passenger area 8 included in each door vicinity sub-region.

[0030] Of course, the position of the passenger area may be the coordinates of each pixel point in the passenger area in the area near the door, with the coordinates taking the gap between the two doors 4 as the origin, the direction along the opening and closing of the doors 4 as the x coordinate, and the direction perpendicular to the doors 4 toward the inside of the car as the y direction, and the values ​​of the x and y coordinates being expressed as the number of pixel points.

[0031] In S103, the occupancy rate of the passenger area in the door vicinity area is calculated based on the position.

[0032] The occupancy rate represents the reliability that a passenger is leaning against the door, and the higher the occupancy rate, the higher the possibility that a passenger is leaning against the door. In one embodiment, the occupied area of ​​the passenger area in each door vicinity sub-area is calculated, and the product of the preset weight and the occupied area of ​​each door vicinity sub-area is calculated to obtain the occupancy rate in each door vicinity sub-area of ​​the passenger area. Then, the sum of the occupancy rates in the multiple door vicinity sub-areas of the passenger area is calculated to obtain the occupancy rate in the door vicinity area of ​​the passenger area.

[0033] As shown in Figure 1C, different weights may be set for different door near sub-regions, and the closer a door near sub-region is to door 4, the greater its weight, indicating a greater possibility that a passenger located in that door near sub-region is leaning against door 4. In one embodiment, the occupied area of ​​the passenger region in each door near sub-region may be calculated by counting the number of pixel points in the passenger region located in each door near sub-region. In another preferred embodiment, the occupied area of ​​the passenger region in each door near sub-region may be calculated by calculating the physical area of ​​the passenger region in each door near sub-region. Then, the occupancy rate of the passenger region in the door near sub-region is calculated by multiplying the occupied area by the weight of the door near sub-region, and the sum of the occupancy rates of all door near sub-regions is calculated to obtain the occupancy rate of the passenger region in the door near region.

[0034] In S104, if the occupancy rate is greater than a preset occupancy rate threshold, it is determined that there is a passenger leaning against the door in the car.

[0035] If the occupancy rate in the door-near area of ​​the passenger area is greater than the occupancy rate threshold, it is determined that the passenger corresponding to the passenger area is leaning against the door.

[0036] In another embodiment, if the occupancy rates of the door-near areas of the passenger area in consecutive N frame images are all greater than the occupancy rate threshold, the passenger corresponding to the passenger area can be determined to be leaning against the door, thereby improving the accuracy of detecting that a passenger is leaning against the door.

[0037] If it is determined that a passenger is leaning against the door, warning information is generated to warn the passenger to move away from the door, thereby preventing an accident from occurring when the passenger leans against the door and the door opens. Here, the warning information may be audio information, for example, announced by an audio announcement device inside the car, or in another example, the warning information may be lamp information, for example, lamp information is displayed on a display, indicator light, etc. inside the car to warn the passenger to move away from the door.

[0038] In S105, when a door opening command is received, a target hand entrapment risk area to which the passenger area belongs is identified from among at least two hand entrapment risk areas in the door vicinity area based on the position.

[0039] For example, when the car arrives at the destination floor, a door opening command for controlling the opening of the door can be received, and in this case, a target hand entrapment risk area to which the passenger area in the area near the door belongs can be identified based on multiple frames of images currently taken.

[0040] 1D , the hand entrapment risk area may be divided into areas according to the distance from each area to the car front wall in the door opening / closing direction. As shown in FIG. 1D , the hand entrapment risk area includes a door corner area 64, a door panel area 66, and a door gap area 65. Because the door corner area 64 is close to the car front wall 5, the door corner area 64 has the highest risk of having a hand entrapped when the door 4 opens. The door gap area 65 is located near the door gap formed by the two doors 4. If a passenger leans against the door when the door 4 opens, the passenger is not at risk of having their hand entrapped, but is at high risk of falling out of the car. In the door panel area 66, if a passenger leans against the door in the door panel area 66 when the door 4 opens, the passenger will be pulled by the door panel and slide toward the car front wall 5, giving the passenger enough time to react and move away from the door. Therefore, the risk of having a hand entrapped or falling out of the car is lower in the door panel area 66 than in the door corner area 64 and the door gap area 65.

[0041] After the position of the passenger area is identified, the target hand entrapment risk area to which the passenger area belongs can be identified. In one preferred embodiment, a weight is preset for each hand entrapment risk area, the occupied area of ​​the passenger area in each hand entrapment risk area is calculated, the product of the preset weight and the occupied area of ​​each hand entrapment risk area is calculated to obtain a score for each hand entrapment risk area of ​​the passenger area, and the hand entrapment risk area with the highest score can be identified as the target hand entrapment risk area to which the passenger area in the door vicinity belongs.

[0042] In another example, when a passenger area is distributed among multiple hand entrapment risk areas, the hand entrapment risk area that contains the most pixel points of the passenger area can be identified as the target hand entrapment risk area to which the passenger area in the door vicinity belongs.

[0043] In a further example, if the passenger area is distributed among multiple hand entrapment risk areas, the area with the highest risk level can be identified as the target hand entrapment risk area to which the passenger area in the door vicinity belongs.

[0044] In S106, the door is controlled based on the target hand entrapment risk area to which the passenger area belongs.

[0045] If the target hand entrapment risk area to which the passenger area belongs does not change in the images of consecutive N frames, the door is controlled based on the target hand entrapment risk area.

[0046] As shown in FIG. 1D, when the target hand entrapment risk area is the door corner area 64, the door opening is paused or the door is controlled to close in reverse, thereby preventing the limbs or clothing of a passenger positioned in the door corner area 64 from being entrapped in the gap between the door 4 and the car front wall 5 when the door continues to open.

[0047] If the target hand entrapment risk area is the door panel area 66, the door is controlled to open at a first door opening speed, which may be slower than the speed at which the door opens normally, so that a passenger positioned in the door panel area 66 has enough time to react and move away from the door 4, and avoids the door opening too quickly, which could result in the passenger's limbs or clothing being caught in the gap between the car front wall 5 and the door 4 or falling out of the car.

[0048] If the target hand entrapment risk area is the door gap area 65, the door is controlled to open at a second door opening speed, where the second door opening speed is smaller than the first door opening speed, i.e., the door is opened at the slowest speed, so that passengers have enough time to react and move away from the door 4, and the door is suddenly opened to avoid passengers located in the door gap area 65 getting out of the car.

[0049] In an embodiment of the present application, when it is detected that the elevator doors are closed, an image of the inside of the car is taken by a sensor and a near-door area is identified from the image, a passenger area is identified from the near-door area, the position of the passenger area in the near-door area is identified, and an occupancy rate is calculated based on the position of the passenger within the near-door area, and if the occupancy rate is greater than a threshold, it is determined that the passenger is leaning against the door, and if the passenger is leaning against the door, a hand entrapment risk area to which the passenger belongs is identified based on the passenger's position, so that when it is detected that the passenger is leaning against the door, the passenger is warned to move away from the door, and the door opening is controlled to be paused or opened slowly, etc., depending on the hand entrapment risk area in which the passenger is located, so as to prevent the door from opening suddenly when a passenger is leaning against the door and thus causing the passenger to exit the car, and if there is a risk of their hand being entrapped, the door opening can be paused in advance or the door can be opened slowly, so that passengers have sufficient reaction time to avoid an accident in which their hand is entrapped when the door opens, ensuring the safety of passengers when riding in the elevator.

[0050] [Example 2] FIG. 2 is a flowchart of a method for detecting risky behavior of passengers according to Example 2 of the present application. This example is optimized based on Example 1 above. As shown in FIG. 2, the method for detecting risky behavior of passengers includes the following steps:

[0051] In S201, when it is detected that the elevator doors are closed, an image of the inside of the car is acquired by a sensor, and a door vicinity area is identified from the image, and the door vicinity area includes at least two door vicinity sub-areas that are not at equal distances to the door, and at least two hand entrapment risk areas.

[0052] As shown in Figures 1C and 1D, the door vicinity area 6 is an area near the car door 4, and the length of the door vicinity area 6 is equal to the sum of the lengths of the two doors 4, and the width can be customized. For example, the width may be 6 cm. In this embodiment, the definition of the door vicinity area 6 is not limited.

[0053] 1C , the door vicinity region 6 can be divided into at least two door vicinity sub-regions in a direction perpendicular to the door 4 according to the distance from each portion of the door vicinity region 6 to the door 4. Taking FIG. 1C as an example, the door vicinity region 6 can be divided into a first door vicinity sub-region 61 close to the door 4, a third door vicinity sub-region 63 away from the door 4, and a second door vicinity sub-region 62 located between the first door vicinity sub-region 61 and the third door vicinity sub-region 63. Each door vicinity sub-region 61 may include at least one row of pixels. A weight may be assigned to each door vicinity sub-region, representing the degree to which a passenger in the door vicinity sub-region is leaning against the door. Here, the weights of the third door vicinity sub-region 63, the second door vicinity sub-region 62, and the first door vicinity sub-region 61 increase in order. For example, the weight of the third door vicinity sub-region 63 is 1, the weight of the second door vicinity sub-region 62 is 2, and the weight of the first door vicinity sub-region 61 is 4.

[0054] As shown in Figure 1D, the door vicinity area 6 can be divided into at least two hand entrapment risk areas in the opening and closing direction of the door 4 depending on the distance from each part of the door vicinity area 6 to the car front wall 5. Taking Figure 1D as an example, the door vicinity area 6 can be divided into a door corner area 64 close to the car front wall 5, a door gap area 65 away from the car front wall 5 and located near the gap between the two doors 4, and a door panel area 66 located between the door corner area 64 and the door gap area 65. A weight may be set for each hand entrapment risk area, and this weight represents the degree of risk of a passenger's hand being entrapped in the hand entrapment risk area when the door opens, where the door corner area 64 is a high-level risk, the door gap area 65 is a medium-level risk, and the door panel area 66 is a low-level risk.

[0055] The above division of the door-nearby sub-areas and hand-entrapment risk areas is merely an example, and in actual applications, a person skilled in the art may provide any number of door-nearby sub-areas and hand-entrapment risk areas.

[0056] In this embodiment, the door vicinity area 6 can be identified in the image in conjunction with the imaging principle based on the mounting height, focal length, etc. of the sensor 3. When the sensor 3 is fixed, pixel points in the captured image correspond one-to-one to multiple positions in the door vicinity area, and a preset pixel point area in the image may be identified as the door vicinity area.

[0057] In S202, a passenger area is identified from the door vicinity area, and at least one door vicinity sub-area to which the passenger area belongs in the door vicinity area is identified.

[0058] For example, the image captured by the sensor is a depth image. Since passengers have a certain height relative to the bottom of the car, the depth of the bottom of the car is set to 0, and the depth value of each pixel point in the area near the door in the image can be obtained. Pixel points with depth values ​​greater than a preset depth threshold are identified as pixel points corresponding to passengers, and the image area occupied in the area near the door by pixel points corresponding to passengers is identified as the passenger area.

[0059] In another embodiment, the sensor is an active optical sensor. When the passengers have a certain height and the intensity of the light reflected by the passengers is greater than the intensity of the light reflected by the car bottom, the image can be binarized based on the light energy value of the pixel points in the image to obtain the passenger area. For example, pixel points whose light intensity is greater than a preset energy threshold are colored black, and vice versa, are colored white to obtain the passenger area.

[0060] Of course, if the captured image is a color image, the image can be input into a target detection model to identify the occupant area from the image, and the present embodiment does not limit the manner of identifying the occupant area from the image.

[0061] As shown in FIG. 1C, a passenger area 8 is identified within the door vicinity area 6, and the number of passenger areas 8 may be one or more, and the passenger area 8 may be an area formed by a person within the door vicinity area 6, or an area formed by a pet, a wheelchair, etc.

[0062] After identifying the passenger area, the distance to the door of each part of the passenger area in the direction perpendicular to the door can be identified, and the position of the passenger area in the door vicinity area can be identified, as shown in FIG. 1C, and the passenger area belonging to each door vicinity sub-area can be identified. As shown in FIG. 1C, the passenger area 8 is divided by three door vicinity sub-areas (61, 62, 63), and the pixel points of the passenger area 8 included in each door vicinity sub-area are identified.

[0063] In S203, the occupied area of ​​the passenger area in each door vicinity sub-area is calculated, and the product of the preset weight of each door vicinity sub-area and the occupied area is calculated to obtain the occupancy rate of the passenger area in each door vicinity sub-area.

[0064] The occupied area of ​​the passenger area in each door vicinity sub-region may be the number of pixel points of the passenger area included in each door vicinity sub-region. For example, in FIG. 1C , the occupied area of ​​the passenger area in the first door vicinity sub-region 61, the second door vicinity sub-region 62, and the third door vicinity sub-region 63 may be calculated by counting the number of pixel points of the passenger area 8 included in the first door vicinity sub-region 61, the second door vicinity sub-region 62, and the third door vicinity sub-region 63, respectively. Then, the product of the number of pixel points of the passenger area 8 in the first door vicinity sub-region 61, the second door vicinity sub-region 62, and the third door vicinity sub-region 63 and the weight of the corresponding door vicinity sub-region is calculated, thereby obtaining the occupancy rate of the passenger area 8 in the first door vicinity sub-region 61, the second door vicinity sub-region 62, and the third door vicinity sub-region 63.

[0065] For example, as shown in FIG. 1C , if the number of pixel points of the passenger area 8 in the first door vicinity sub-region 61 is 150, the number of pixel points of the passenger area 8 in the second door vicinity sub-region 62 is 400, and the number of pixel points of the passenger area 8 in the third door vicinity sub-region 63 is 350, the occupancy rate of the passenger area 8 in the first door vicinity sub-region 61 is 150×4=600, the occupancy rate of the passenger area 8 in the second door vicinity sub-region 62 is 400×2=800, and the occupancy rate of the passenger area 8 in the third door vicinity sub-region 63 is 350×1=350.

[0066] In S204, the sum of the occupancy rates in the plurality of door-nearby sub-regions of the passenger region is calculated to obtain the occupancy rate in the door-nearby region of the passenger region.

[0067] Taking the first door vicinity sub-area 61, the second door vicinity sub-area 62 and the third door vicinity sub-area 63 in S203 as an example, the sum of the occupancy rates in the three door vicinity sub-areas of the passenger area is 600+800+350=1750, that is, the occupancy rate in the door vicinity areas of the passenger area is 1750.

[0068] In S205, if the occupancy rate of the passenger area near the door in N frames of images collected continuously is greater than a preset occupancy rate threshold, it is determined that there is a passenger leaning against the door in the car.

[0069] For example, if the occupancy rate in the passenger area near the door in 10 consecutively acquired frames of images is equal to or greater than a preset occupancy rate threshold, it is determined that the passenger corresponding to that passenger area in the elevator is leaning against the door, and in this case, a warning message is generated and announced by an acousto-optical alarm device to warn the passenger to move away from the door.

[0070] In S206, when a door opening command is received, the occupied area of ​​the passenger area in each hand entrapment risk area is calculated for each hand entrapment risk area, and the product of the preset weight and the occupied area of ​​each hand entrapment risk area is calculated to obtain a score for each hand entrapment risk area in the passenger area.

[0071] As shown in FIG. 1D , the hand entrapment risk area in the door vicinity area 6 includes a door corner area 64 close to the car front wall, a door gap area 65 away from the car front wall 5, and a door panel area 66 located between the door corner area 64 and the door gap area 65. A preset weight is assigned to each hand entrapment risk area, and the weight represents the importance of the passenger area within the hand entrapment risk area. The larger the weight, the higher the score of the passenger area in the hand entrapment risk area, and the higher the possibility that the passenger area belongs to the hand entrapment risk area.

[0072] In FIG. 1D , the passenger area 8 is distributed in the door corner area 64 and the door panel area 66. To determine whether the target hand entrapment risk area to which the passenger area 8 belongs is the door corner area 64 or the door panel area 66, the number of pixel points located in the door corner area 64 and the door panel area 66 of the passenger area 8 are respectively statistically analyzed, and the number of pixel points of the passenger area 8 in the door corner area 64 and the weight of the door corner area 64 are calculated to obtain a score for the passenger area 8 belonging to the door corner area 64. Similarly, the score for the passenger area 8 belonging to the door panel area 66 can be calculated.

[0073] As shown in FIG. 1D , assuming that the weight of the door corner region 64 is 5, the weight of the door panel region 66 is 1, the weight of the door gap region 65 is 3, the number of pixel points in the door corner region 64 of the passenger region 8 is 200, the number of pixel points in the door panel region 66 of the passenger region 8 is 700, and the number of pixel points in the door gap region 65 is 0, then the score for the passenger region 8 belonging to the door corner region 64 is 200×5=1000, the score for the passenger region 8 belonging to the door panel region 66 is 700×1=700, and the score for the passenger region 8 belonging to the door gap region 65 is 0×3=0.

[0074] In S207, the hand entrapment risk area with the highest score is identified as the target hand entrapment risk area to which the passenger area in the door vicinity area belongs.

[0075] Taking the door corner area 64, door panel area 66 and door gap area 65 in S206 as an example, if the scores of the passenger area 8 belonging to the door corner area 64, door panel area 66 and door gap area 65 are 1000, 700 and 0 respectively, the target hand entrapment risk area to which the passenger area 8 belongs can be identified as the door corner area 64.

[0076] If there are two or more hand entrapment risk areas with the same score, the hand entrapment risk area with the highest risk level among the hand entrapment risk areas with the same score is identified as the target hand entrapment risk area.

[0077] If the target hand entrapment risk area to which the passenger area belongs does not change in the images of the consecutive N frames, the door can be controlled based on the target hand entrapment risk area, specifically as shown in S208 to S210.

[0078] In S208, if the target hand entrapment risk area is a door corner area, the door opening is temporarily stopped or the door is controlled to be closed in reverse.

[0079] As shown in FIG. 1D , if the target hand entrapment risk area is the door corner area 64, the door opening is paused or the door is controlled to close in reverse, and an audio or indicator light is used to warn the user to move away from the door, so as to prevent the limbs or clothing of a passenger positioned in the door corner area 64 from being entrapped in the gap between the door 4 and the car front wall 5 when the door continues to open.

[0080] In S209, if the target hand entrapment risk area is the door panel area, the door is controlled to open at a first door opening speed.

[0081] If the target hand entrapment risk area is the door panel area 66, the door is controlled to open at a first speed, and a sound or an indicator light is used to warn the user to move away from the door, where the first door speed may be slower than the speed at which the door opens normally, so that the passengers positioned in the door panel area 66 have enough time to react and move away from the door 4, and avoid the door opening too quickly causing the passengers' limbs or clothing to be caught in the gap between the car front wall 5 and the door 4 or to fall out of the car.

[0082] In S210, if the target hand entrapment risk area is a door gap area, the door is controlled to open at a second door opening speed, where the second door opening speed is lower than the first door opening speed.

[0083] If the target hand entrapment risk area is the door gap area 65, the door is controlled to open at a second door opening speed, and a sound or an indicator light is used to warn the user to move away from the door, where the second door opening speed is smaller than the first door opening speed, i.e., the door is opened at the slowest speed, so that passengers have enough time to react and move away from the door 4, and avoid the door suddenly opening and passengers located in the door gap area 65 getting out of the car.

[0084] In this embodiment, the occupancy rate is calculated by assigning different weights to passengers located in different door vicinity sub-areas. If the total occupancy rate is greater than the occupancy rate threshold, it is determined that the passenger is leaning against the door. After it is determined that the passenger is leaning against the door, different weights are assigned to the different hand entrapment risk areas in which the passenger is located to calculate a score for each hand entrapment risk area in the passenger area. The hand entrapment risk area with the highest score is determined as the target hand entrapment risk area in which the passenger is located. By assigning different weights to different positions based on the passenger's location, it is possible to detect whether the passenger is leaning against the door and to identify the hand entrapment risk area to which the passenger belongs when the door opens. This achieves high accuracy in detecting passengers leaning against the door and the risk area in which their hands are entrapped. On the one hand, if it is detected that a passenger is leaning against the door, a warning message can be sent to warn the passenger to move away from the door. On the other hand, the doors can be controlled to open using different strategies based on the hand entrapment risk area in which the passenger is located, ensuring the safety of passengers when riding in the elevator.

[0085] Different door-nearby sub-areas have different weights, and the occupancy rate of each door-nearby sub-area of ​​the passenger area is calculated based on the weight and area, and then the occupancy rates corresponding to all door-nearby sub-areas are added together to obtain the total occupancy rate. Different hand entrapment risk areas have different weights, and the score to which each passenger area belongs to each hand entrapment risk area is calculated based on the area and weight of each hand entrapment risk area of ​​the passenger area. This allows for a detailed analysis of the impact of each position in the passenger area on the risk of door leaning and hand entrapment, and the accuracy of door leaning detection and hand entrapment risk area identification is high.

[0086] When the door opens, the door opening can be controlled based on whether the passenger area belongs to the door corner area, the door panel area, or the door gap area, and the door opening can be differentially controlled depending on whether the passenger is located in a different hand entrapment risk area, ensuring the safety of passengers when they board the elevator and also optimizing the door opening method.

[0087] [Example 3] FIG. 3 is a structural schematic diagram of a detection device for detecting risky behavior of passengers according to a third embodiment of the present invention. 3, the passenger risk behavior detection device includes: a door vicinity area identification module 301 configured to, when detecting that the elevator doors are closed, collect an image inside the car using a sensor and identify a door vicinity area from the image; a passenger area identification module 302 configured to identify a passenger area from the door vicinity area and identify a position of the passenger area in the door vicinity area; an occupancy rate calculation module 303 configured to calculate an occupancy rate of the passenger area in the door vicinity area based on the position; a door leaning identification module 304 configured to identify a passenger leaning against the door in the car if the occupancy rate is greater than a predetermined occupancy rate threshold; a hand entrapment risk area identification module 305 configured, when receiving a door opening command, to identify a target hand entrapment risk area to which the passenger area belongs among at least two hand entrapment risk areas in the door vicinity area based on the position; and a door control module 306 configured to control the door based on the target hand entrapment risk area to which the passenger area belongs.

[0088] Preferably, the passenger area identification module 302 includes a depth value acquisition unit configured to acquire a depth value of each pixel point in the door vicinity area in the image, and a passenger area generation unit configured to identify pixel points having a depth value greater than a predetermined depth threshold as pixel points corresponding to the passenger, and to identify an image area occupied in the door vicinity area by the pixel points corresponding to the passenger as a passenger area.

[0089] Preferably, the door near region includes at least two door near sub-regions that are not equal in distance to the door, and the passenger region identification module 302 further includes a door near sub-region identification unit configured to identify at least one door near sub-region to which the passenger region in the door near region belongs.

[0090] Preferably, the occupancy calculation module 303 includes: a sub-area occupancy calculation unit configured to calculate an occupancy area of ​​the passenger area in each door-near sub-area among the at least one door-near sub-area, calculate a product of a preset weight of each door-near sub-area and the occupancy area to obtain an occupancy rate in each door-near sub-area of ​​the passenger area; and a total occupancy calculation unit configured to calculate a sum value of occupancy rates in the at least one door-near sub-area of ​​the passenger area, to obtain an occupancy rate in the door-near sub-area of ​​the passenger area.

[0091] Preferably, the door leaning identification module 304 includes a door leaning identification unit configured to identify that there is a passenger leaning on the door in the car when the occupancy rate in the door vicinity area of ​​the passenger area is greater than a preset occupancy rate threshold in N frames of images collected continuously.

[0092] Preferably, the apparatus further comprises a door leaning warning module configured to warn passengers to stay away from the door by generating warning information and announcing the warning information by an acoustic-optical warning device.

[0093] Preferably, the hand entrapment risk area identification module 305 includes a score identification unit configured, when receiving a door opening command, to calculate the occupied area of ​​the passenger area in each hand entrapment risk area for each hand entrapment risk area, calculate the product of a preset weight of each hand entrapment risk area and the occupied area, and obtain a score for each hand entrapment risk area in the passenger area, and a target hand entrapment risk area identification unit configured to identify the hand entrapment risk area with the highest score as the target hand entrapment risk area to which the passenger area in the door vicinity belongs.

[0094] Preferably, the hand entrapment risk area includes a door corner area close to the car front wall, a door gap area away from the car front wall, and a door panel area located between the door corner area and the door gap area, and the door control module 306 includes: a first door control unit configured to pause the opening of the door or control the door to close in reverse, when a target hand entrapment risk area to which the passenger area belongs does not change in N frames of images continuously acquired and the target hand entrapment risk area is the door corner area; a second door control unit configured to control the door to open at a first door opening speed, when the target hand entrapment risk area is the door panel area; and a third door control unit configured to control the door to open at a second door opening speed, when the target hand entrapment risk area is the door gap area, wherein the second door opening speed is slower than the first door opening speed.

[0095] The passenger risk behavior detection device according to the embodiments of the present application can execute the passenger risk behavior detection method according to the first and second embodiments of the present application, and has functional modules and effects corresponding to the execution of the method.

[0096] [Example 4] 4 shows a structural schematic diagram of an electronic device 40 that can be used to implement embodiments of the present application. The electronic device 40 may be any of various types of digital computers, such as a desktop computer, a workstation, a server, a blade server, a mainframe, etc. The components, their connections, relationships, and their functions shown herein are merely exemplary and do not limit the implementation of the present application as described and / or claimed herein.

[0097] As shown in FIG. 4 , electronic device 40 includes at least one processor 41 and memory communicatively coupled to at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., where the memory stores computer programs executable by the at least one processor, and processor 41 can perform various appropriate operations and processes based on the computer programs stored in read-only memory (ROM) 42 or loaded from storage device 48 into random access memory (RAM) 43. RAM 43 further stores various programs and data necessary for the operation of electronic device 40. Processor 41, ROM 42, and RAM 43 are connected to one another via a bus 44. An input / output (I / O) interface 45 is also connected to bus 44.

[0098] The components in the electronic device 40 are connected to an I / O interface 45, which comprises input units 46 such as a keyboard, mouse, sensors, etc., output units 47 such as various displays, speakers, etc., storage units 48 such as a magnetic disk, optical disk, etc., and communication units 49 such as a network card, modem, wireless communication transceiver, etc. The communication units 49 allow the electronic device 40 to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks.

[0099] The processor 41 may be a general-purpose and / or dedicated processing component having processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, processors that execute various machine learning model algorithms, digital signal processing (DSP), and any suitable processor, controller, microcontroller, etc. The processor 41 executes the various methods and processes described above, for example, a method for detecting risky passenger behavior.

[0100] In some embodiments, the method for detecting risky passenger behavior can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, some or all of the computer program can be loaded and / or installed into electronic device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, it can perform one or more steps of the method for detecting risky passenger behavior. Alternatively, in other embodiments, processor 41 can be configured to perform the method for detecting risky passenger behavior in any other suitable manner (e.g., via firmware).

[0101] Various embodiments of the systems and techniques described herein may be realized in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: embodied in one or more computer programs that can be executed and / or interpreted by a programmable system including at least one programmable processor, which may be a special purpose or general purpose programmable processor, capable of receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device;

[0102] Computer programs for implementing the methods of the present application can be written in one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that, when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are performed. The computer program can be executed entirely on the device, partially on the device, as a separate software package partially on the device and partially on a remote device, or entirely on a remote device or server.

[0103] In this specification, a computer-readable storage medium may be a tangible medium that can contain or store a program for use in or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A machine-readable storage medium may include an electrical connection of one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0104] To provide for user interaction, the systems and techniques described herein can be implemented in an electronic device having a display device (e.g., a Cathode Ray Tube (CRT) or Liquid Crystal Display (LCD) monitor) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse or trackball) by which the user can provide input to the electronic device. Other types of devices can be used to provide further user interaction. For example, the feedback provided to the user can be any form of sensing feedback (e.g., visual feedback, auditory feedback, or haptic feedback), and can receive input from the user in any form (including sound, speech, or tactile input).

[0105] The systems and techniques described herein can be implemented in a computing system that includes a back-end component (e.g., a data server), or a computing system that includes middleware (e.g., an application server), or a computing system that includes a front-end component (e.g., a user computer having a graphical user interface or network browser through which a user can interact with embodiments of the systems and techniques described herein), or a computing system that includes any combination of such back-end, middleware, or front-end components. The components of the system can be connected to each other by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0106] A computing system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. The client-server relationship is established by running computer programs on corresponding computers. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system and is used to solve the shortcomings of traditional physical host and virtual private server (VPS) services, such as difficulty in management and poor traffic scalability.

[0107] It should be understood that steps can be rearranged, added, or deleted using the various types of flows shown above. For example, the steps described herein may be performed in parallel, sequentially, or in a different order, and this application is not limited thereto as long as the desired results of the technical solution disclosed herein can be achieved.

Claims

1. When detecting that the elevator doors are closed, an image of the inside of the car is acquired by a sensor, and an area near the doors is identified from the image; Identifying a passenger area from the near-door area and identifying a position of the passenger area in the near-door area; calculating an occupancy rate of the near-door region of the passenger area based on the location; Identifying a passenger leaning against the door in the car if the occupancy rate is greater than a preset occupancy rate threshold; When a door opening command is received, identifying a target hand entrapment risk area to which the passenger area belongs among at least two hand entrapment risk areas in the door vicinity area based on the position; and controlling the door based on a target hand entrapment risk area to which the passenger area belongs. A method for detecting risky passenger behavior.

2. The step of identifying a passenger area from the door vicinity area includes: obtaining a depth value of each pixel point in the area near the door in the image; Identifying pixel points having depth values ​​greater than a preset depth threshold as pixel points corresponding to the passengers, and identifying an image region occupied by the pixel points corresponding to the passengers in the door vicinity region as a passenger region. The method for detecting risky passenger behavior according to claim 1 .

3. the near-door region includes at least two near-door sub-regions that are not equal in distance to the door, and identifying a location in the near-door region of the passenger region includes: identifying at least one door-nearby sub-region to which the passenger region belongs in the door-nearby region; The method for detecting risky passenger behavior according to claim 1 .

4. Calculating an occupancy rate of the near-door region of the passenger region based on the position includes: calculating an occupied area of ​​the passenger area in each of the at least one door proximate sub-areas, and calculating a product of a preset weight of each of the door proximate sub-areas and the occupied area to obtain an occupancy rate of the passenger area in each of the door proximate sub-areas; calculating a sum of occupancy rates in the at least one near-door sub-region of the passenger region to obtain an occupancy rate in the near-door region of the passenger region; The method for detecting risky passenger behavior according to claim 3.

5. Identifying that there is a passenger leaning against the door in the car when the occupancy rate is greater than a preset occupancy rate threshold, and determining that there is a passenger leaning against the door in the car when an occupancy rate in a door-near area of ​​the passenger area is greater than a preset occupancy rate threshold in N frames of images acquired continuously. The method for detecting risky behavior of passengers according to any one of claims 1 to 4.

6. After determining that there is a passenger in the car leaning against the door, generating warning information and announcing the warning information by an acoustic-optical warning device to warn passengers to move away from the door. The method for detecting risky behavior of passengers according to any one of claims 1 to 4.

7. When the door opening command is received, identifying a target hand entrapment risk area to which the passenger area belongs among at least two hand entrapment risk areas in the door vicinity area based on the position, When a door opening command is received, for each hand entrapment risk area, calculate an occupied area of ​​the passenger area in each hand entrapment risk area, calculate a product of a preset weight of each hand entrapment risk area and the occupied area, and obtain a score for each hand entrapment risk area of ​​the passenger area; and identifying the hand entrapment risk area having the largest score as a target hand entrapment risk area to which the passenger area in the door vicinity area belongs. The method for detecting risky passenger behavior according to claim 1 .

8. the hand entrapment risk area includes a door corner area close to a car front wall, a door gap area away from the car front wall, and a door panel area located between the door corner area and the door gap area, and controlling the door based on the target hand entrapment risk area to which the passenger area belongs includes: When a target hand entrapment risk area to which the passenger area belongs does not change among the N frames of images continuously acquired and the target hand entrapment risk area is the door corner area, the opening of the door is temporarily stopped or the door is closed in reverse; If the target hand entrapment risk area is the door panel area, controlling the door to open at a first door opening speed; When the target hand entrapment risk area is the door gap area, controlling the door to open at a second door opening speed; the second door opening speed is smaller than the first door opening speed; The method for detecting risky passenger behavior according to claim 1 .

9. a door vicinity area identification module configured to, when detecting that the elevator doors are in a closed state, capture an image of the interior of the car using a sensor and identify a door vicinity area from the image; a passenger area identification module configured to identify a passenger area from the near-door area and identify a location of the passenger area in the near-door area; an occupancy calculation module configured to calculate an occupancy rate in the near-door region of the passenger area based on the location; a door leaning identification module configured to identify a passenger leaning on the door in the car if the occupancy rate is greater than a preset occupancy rate threshold; a hand entrapment risk area identification module configured to identify a target hand entrapment risk area to which the passenger area belongs, among at least two hand entrapment risk areas in the door vicinity area, based on the position when a door opening command is received; a door control module configured to control the door based on a target hand entrapment risk area to which the passenger area belongs. A device for detecting dangerous passenger behavior.

10. at least one processor; a memory communicatively coupled to the at least one processor; An electronic device comprising: The memory stores a computer program executable by the at least one processor, and the computer program, when executed by the at least one processor, enables the at least one processor to execute the method for detecting risky behavior of passengers according to any one of claims 1 to 8. electronic equipment.

11. 9. A method for detecting risky passenger behavior comprising: storing computer instructions that, when executed by a processor, implement the method for detecting risky passenger behavior according to any one of claims 1 to 8; A computer-readable storage medium.

Images