Personnel fall risk detection method, system and storage medium

Abstract

The present invention relates to the field of fall protection technology, and discloses a personnel fall risk detection method, system and storage medium, wherein a first enhanced image and a second enhanced image are randomly cropped to obtain a first cropped image and a second cropped image, respectively, and the first cropped image and the second cropped image are spliced ​​to obtain a training image; the training image is input into a neural network model to be trained to obtain a target neural network model; based on the acquired image to be detected, fall location information and personnel position information are generated, and the fall location information is input into the target neural network model for judgment to generate a first judgment result; when the first judgment result is a prone to fall location, the personnel position information is input into the target neural network model for judgment to generate a second judgment result; when the second judgment result is a prone to fall location, an alarm message is issued; the present invention solves the problems of slow recognition speed and low recognition accuracy of existing detection and recognition methods.

Description

Technical Field

[0001] The present invention relates to the field of fall protection technology, and in particular to a personnel fall risk detection method, system and storage medium. Background Art

[0002] With the rapid development of construction technology and the continuous improvement of construction standards, more and more high-rise buildings are appearing in cities. As the number of high-rise buildings continues to increase, in order to meet the construction and maintenance needs of high-rise buildings, the number of high-altitude construction workers is also increasing. High-altitude construction workers need to perform corresponding construction and maintenance operations at a higher position from the ground. At present, when high-altitude construction workers are performing construction and maintenance operations, if there is an irregular operation, there will be a risk of falling. By collecting the movements of high-altitude construction workers in real time, when high-altitude construction workers make dangerous actions that are prone to falling risks, timely warnings and stop are effective reminders. However, the terrain in areas prone to falling incidents is relatively complex, and the process of falling incidents often occurs in a short period of time. Therefore, there are very high requirements for the speed and accuracy of recognition, which are difficult to meet with existing detection and recognition methods. Summary of the invention

[0003] The present invention provides a personnel fall risk detection method, system and storage medium to solve the problems of slow recognition speed and low recognition accuracy of existing detection and recognition methods.

[0004] In order to achieve the above object, the present invention is implemented by the following technical solutions:

[0005] In a first aspect, the present invention provides a method for detecting a risk of a person falling, comprising:

[0006] Acquire a first training image and a second training image, and remove effective areas in the first training image and the second training image to obtain a first enhanced image and a second enhanced image, wherein the effective area includes a first removed area and a second removed area;

[0007] Randomly cropping the first enhanced image and the second enhanced image respectively to obtain a first cropped image and a second cropped image, and splicing the first cropped image and the second cropped image to obtain a training image;

[0008] Inputting the training image into the neural network model to be trained for training to obtain a target neural network model;

[0009] Generate falling location information and personnel location information based on the acquired image to be detected, and input the falling location information into the target neural network model for judgment to generate a first judgment result;

[0010] When the first judgment result is a location prone to falling, inputting the personnel location information into a target neural network model for judgment to generate a second judgment result;

[0011] When the second judgment result is a prone-to-fall position, an alarm message is issued.

[0012] Optionally, before removing effective areas in the first training image and the second training image to obtain the first enhanced image and the second enhanced image, the method further includes:

[0013] generating a first rejection area based on a first area of ​​the first training image, and comparing a first ratio of the first rejection area to the first area with a preset maximum rejection threshold and a minimum rejection threshold;

[0014] When the first ratio is greater than a minimum rejection threshold and less than a maximum rejection threshold, determining the first rejection area to be a valid area in the first training image;

[0015] When the first ratio is less than a minimum rejection threshold or greater than a maximum rejection threshold, regenerating a new first rejection area based on the first area;

[0016] generating a second rejection area based on a second area of ​​the second training image, and comparing a second ratio of the second rejection area to the second area with a preset maximum rejection threshold and a minimum rejection threshold;

[0017] When the second ratio is greater than a minimum rejection threshold and less than a maximum rejection threshold, determining the second rejection area as a valid area in the first training image;

[0018] When the second ratio is smaller than the minimum rejection threshold or larger than the maximum rejection threshold, a new second rejection area is regenerated based on the second area.

[0019] Optionally, removing effective areas in the first training image and the second training image to obtain a first enhanced image and a second enhanced image includes:

[0020] Establishing a coordinate system with the vertex of the first side of the first training image as the origin of the coordinate system;

[0021] A point in the first training image is selected as a first coordinate point, the first coordinate point is used as the upper left corner coordinate point of the first eliminated image, the horizontal coordinate of the first coordinate point is added to the height and length of the first eliminated image, the vertical coordinate of the first coordinate point is added to the width and length of the first eliminated image, and the lower right corner coordinate point of the first eliminated image is determined as the second coordinate point;

[0022] Comparing the abscissa of the second coordinate point with the width and length of the first training image, and comparing the ordinate of the second coordinate point with the height and length of the first training image, and when the abscissa of the second coordinate point is less than or equal to the width and length of the first training image, and the ordinate of the second coordinate point is less than or equal to the height and length of the first training image, eliminating the first eliminated image from the first training image to obtain a first enhanced image;

[0023] The height of the first eliminated image is obtained by taking the square root of the product of the first eliminated area and the aspect ratio of the first eliminated image;

[0024] The width and length of the first eliminated image is obtained by taking the square root of the ratio of the first eliminated area to the aspect ratio of the first eliminated image.

[0025] Optionally, the removing effective areas in the first training image and the second training image to obtain the first enhanced image and the second enhanced image further includes:

[0026] Establishing a coordinate system with the vertex of the first side of the second training image as the origin of the coordinate system;

[0027] A point in the second training image is selected as a third coordinate point, the third coordinate point is used as the upper left corner coordinate point of the second eliminated image, the horizontal coordinate of the third coordinate point is added to the height and length of the second eliminated image, the vertical coordinate of the third coordinate point is added to the width and length of the second eliminated image, and the lower right corner coordinate point of the second eliminated image is determined as the fourth coordinate point;

[0028] Comparing the abscissa of the fourth coordinate point with the width and length of the second training image, and comparing the ordinate of the fourth coordinate point with the height and length of the second training image, when the abscissa of the fourth coordinate point is less than or equal to the width and length of the second training image, and the ordinate of the fourth coordinate point is less than or equal to the height and length of the second training image, eliminating the second eliminated image from the second training image to obtain a second enhanced image;

[0029] The height of the second eliminated image is obtained by taking the square root of the product of the second eliminated area and the aspect ratio of the second eliminated image;

[0030] The width and length of the second eliminated image is obtained by taking the square root of the ratio of the second eliminated area to the aspect ratio of the second eliminated image.

[0031] Optionally, the method further includes:

[0032] Acquire the size of a first person outline region in the first enhanced image, and compare the size of the first person outline region with a preset outline size threshold; when the size of the first person outline region is greater than or equal to the preset outline size threshold, determine that the first enhanced image is valid; when the size of the first person outline region is less than the preset outline size threshold, discard the first enhanced image and regenerate a first rejection area;

[0033] Obtain the size of the second person contour area in the second enhanced image, and compare the size of the second person contour area with a preset contour size threshold. When the size of the second person contour area is greater than or equal to the preset contour size threshold, determine that the second enhanced image is valid; when the size of the second person contour area is less than the preset contour size threshold, discard the second enhanced image and regenerate a second elimination area.

[0034] Optionally, inputting the training image into a neural network model to be trained for training includes:

[0035] The neurons in the neural network model to be trained are randomly divided into training neurons and standby neurons, and after the training image is input into the neural network model to be trained, the training neurons are trained based on the training image. After the training is completed, the trained neural network model to be trained is verified based on the training image by the training neurons and the standby neurons.

[0036] Optionally, the method further includes:

[0037] Inputting the personnel position information in the next frame of the image to be detected into the target neural network model for judgment, and generating a third judgment result;

[0038] The third judgment result is compared with the second judgment result to generate a comparison value, and the comparison value is compared with a preset similarity threshold. When the comparison value is greater than the preset similarity threshold, no alarm message is issued, and when the comparison value is less than the preset similarity threshold, an alarm message is issued.

[0039] Optionally, the alarm information includes an on-site loudspeaker reminder and a mobile Internet reminder.

[0040] In a second aspect, an embodiment of the present application provides a personnel fall risk detection system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0041] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method steps described in the first aspect.

[0042] Beneficial effects:

[0043] The personnel fall risk detection method provided by the present invention eliminates the effective area from the first training image and the second training image, so that the target recognition contour in the training image is missing, and uses the missing target recognition contour for training, which can effectively improve the recognition speed and accuracy of the real target contour in the detection process, thereby improving the speed and accuracy of judging the personnel position information; at the same time, the cropped first cropped image and the second cropped image are spliced ​​to generate a training image for training, which can improve the accuracy of judging the fall location information in the detection process.

[0044] The personnel fall risk detection method provided by the present invention can detect the personnel position information of high-altitude construction personnel during high-altitude operations by judging the falling location information and personnel position information in the detected image, and make timely alarm reminders for the positions of personnel who may be in danger of falling, thereby reducing the occurrence of falling accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flow chart of a method for detecting a risk of falling of a person according to a preferred embodiment of the present invention;

[0046] Figure 2 It is a logic block diagram of a personnel fall risk detection method according to a preferred embodiment of the present invention;

[0047] Figure 3 This is an architecture diagram of a personnel fall risk detection system according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0048] The technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Unless otherwise defined, the technical terms or scientific terms used in the present invention shall have the usual meanings understood by persons with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "one" do not indicate quantity restrictions, but indicate the existence of at least one. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship also changes accordingly.

[0050] Embodiment 1:

[0051] See also Figure 1-2 , the embodiment of the present application provides a method for detecting a risk of falling of a person, comprising:

[0052] Acquire a first training image and a second training image, and remove effective areas in the first training image and the second training image to obtain a first enhanced image and a second enhanced image, wherein the effective area includes a first removed area and a second removed area;

[0053] Randomly cropping the first enhanced image and the second enhanced image respectively to obtain a first cropped image and a second cropped image, and splicing the first cropped image and the second cropped image to obtain a training image;

[0054] Inputting the training image into the neural network model to be trained for training to obtain a target neural network model;

[0055] Generate falling location information and personnel location information based on the acquired image to be detected, and input the falling location information into the target neural network model for judgment to generate a first judgment result;

[0056] When the first judgment result is a location prone to falling, inputting the personnel location information into a target neural network model for judgment to generate a second judgment result;

[0057] When the second judgment result is a prone-to-fall position, an alarm message is issued.

[0058] In the above embodiment, the effective area is the image area eliminated from the first training image and the second training image, and the size of the eliminated image area is randomly generated. After the size of the image area for elimination is generated, the size of the image area for elimination is compared with the area size of the first training image and the area size of the second training image, respectively, to ensure that the ratio of the two is neither too large nor too small, so as to avoid that the remaining image after random erasure is not too large to achieve the effect of random erasure, and is not too small to achieve the training effect. When it is judged that the size of the image area for elimination is appropriate, the randomly generated image area for elimination can be determined as the effective area; after the effective area is eliminated from the first training image and the second training image, there will be a lack of recognition target contour in the remaining image. Using the missing recognition target contour for recognition training can improve the accuracy of target recognition, ensure that in actual application, even in complex conditions, the target can still be quickly and accurately identified, and the recognition speed is improved. This is the effect to be achieved by randomly erasing the image.

[0059] By detecting the location information of high-altitude construction workers during high-altitude operations, and making timely alarm reminders for the locations of people who may be in danger of falling, the occurrence of falling accidents can be reduced. At the same time, in the process of training the neural network model, the analysis model is trained by the data enhancement method, which improves the accuracy of the detection method in the actual detection process; the training images in the training process are mainly combined by combining random cropping and mixing different original images. The goal of this method is to combine the initial two samples to generate new samples for training, so as to achieve the purpose of data enhancement. The combination formula of this operation can be expressed as:

[0060]

[0061]

[0062] Where M represents a binary mask, M∈{0,1} W*H , the mask indicates the locations cropped and filled from the two images, 1 indicates the part where the mask is filled with 1, λ indicates the combination ratio between the two data points, which is sampled from the Beta distribution Beta(0,δ) (here δ is set to 1), (x A ,y A ) and (x B ,y B ) represent the original training samples, represents the newly generated training sample, ⊙ represents bit-by-bit multiplication;

[0063] To sample the binary mask M, we first sample the bounding box coordinates B = (r x ,ry ,r w ,r h ) for sampling, and the regional coordinates are uniformly sampled according to the following method:

[0064]

[0065]

[0066] Among them, (r x ,r y ) represents the coordinates of the upper left corner of region B, Unif(0,W) refers to r x Should be sampled uniformly randomly between 0 and W. w is the width of region B, r h is the height of region B. λ represents the combination ratio, which is randomly and uniformly sampled in (0,1). In this experiment, in order to keep the shape of region B the same as the original image, the ratio of the cropped region is

[0067] Area B represents x A and x B The clipping area, x A The area B in is removed and then filled with B The patch cut from region B in . That is, for the binary mask M∈{0,1} W*H , which fills 0 in area B and 1 in the rest of the positions.

[0068] During the application process, the image to be detected is input into the target neural network model, and the target neural network model will first judge the falling location information in the image to be detected. This method only needs to detect whether personnel are at risk of falling in dangerous areas such as high altitudes, slopes, balconies, etc., and in other safe areas without falling risks, no matter what actions the high-altitude construction workers take, no falling accidents will occur. Therefore, only after the falling location information in the image to be detected is judged, and the first judgment result generated is a prone to falling location, the personnel position information in the image to be detected will be further detected, and the personnel position information will be judged to generate a second judgment result. According to the second judgment result, it is determined whether there is a risk of falling. When the second judgment result is a prone to falling location, it means that the high-altitude construction workers are performing dangerous actions that are prone to falling accidents, and an alarm message is issued to remind them.

[0069] Optionally, before removing effective areas in the first training image and the second training image to obtain the first enhanced image and the second enhanced image, the method further includes:

[0070] generating a first rejection area based on a first area of ​​the first training image, and comparing a first ratio of the first rejection area to the first area with a preset maximum rejection threshold and a minimum rejection threshold;

[0071] When the first ratio is greater than a minimum rejection threshold and less than a maximum rejection threshold, determining the first rejection area to be a valid area in the first training image;

[0072] When the first ratio is less than a minimum rejection threshold or greater than a maximum rejection threshold, regenerating a new first rejection area based on the first area;

[0073] generating a second rejection area based on a second area of ​​the second training image, and comparing a second ratio of the second rejection area to the second area with a preset maximum rejection threshold and a minimum rejection threshold;

[0074] When the second ratio is greater than a minimum rejection threshold and less than a maximum rejection threshold, determining the second rejection area as a valid area in the first training image;

[0075] When the second ratio is smaller than the minimum rejection threshold or larger than the maximum rejection threshold, a new second rejection area is regenerated based on the second area.

[0076] In the above embodiment, in order to make the target detection more accurate, a data enhancement method is used, including randomly erasing part of the area in the image before training. The effective area in the embodiment is the randomly erased area. The first eliminated area and the second eliminated area are two different names of the effective area in the first training image and the second training image, respectively. The data enhancement method uses a random erasing operation to eliminate the effective area from the image, and uses the eliminated image for training to improve the accuracy of target recognition and achieve the effect of data enhancement, thereby improving the generalization ability of the model. The randomly erased area is randomly generated, and there is a maximum threshold and a minimum threshold to avoid the randomly erased area being too large or too small. The random erasure ratio formula can be expressed as:

[0077]

[0078] Among them, S e represents the area of ​​random erasure, S represents the area of ​​the entire image, and Q represents the ratio between the two; by comparing the ratio Q with the maximum threshold and the minimum threshold, the randomly erased area S is considered to be the same only when the ratio Q is greater than the minimum threshold and less than the maximum threshold. e Only then can normal simulated occlusion operations be performed. Otherwise, the randomly erased area may be too large, resulting in a small identifiable area, or the randomly erased area may be too small, resulting in no simulated occlusion effect.

[0079] Optionally, removing effective areas in the first training image and the second training image to obtain a first enhanced image and a second enhanced image includes:

[0080] Establishing a coordinate system with the vertex of the first side of the first training image as the origin of the coordinate system;

[0081] A point in the first training image is selected as a first coordinate point, the first coordinate point is used as the upper left corner coordinate point of the first eliminated image, the horizontal coordinate of the first coordinate point is added to the height and length of the first eliminated image, the vertical coordinate of the first coordinate point is added to the width and length of the first eliminated image, and the lower right corner coordinate point of the first eliminated image is determined as the second coordinate point;

[0082] Comparing the abscissa of the second coordinate point with the width and length of the first training image, and comparing the ordinate of the second coordinate point with the height and length of the first training image, and when the abscissa of the second coordinate point is less than or equal to the width and length of the first training image, and the ordinate of the second coordinate point is less than or equal to the height and length of the first training image, eliminating the first eliminated image from the first training image to obtain a first enhanced image;

[0083] The height of the first eliminated image is obtained by taking the square root of the product of the first eliminated area and the aspect ratio of the first eliminated image;

[0084] The width and length of the first eliminated image is obtained by taking the square root of the ratio of the first eliminated area to the aspect ratio of the first eliminated image.

[0085] Optionally, the removing effective areas in the first training image and the second training image to obtain the first enhanced image and the second enhanced image further includes:

[0086] Establishing a coordinate system with the vertex of the first side of the second training image as the origin of the coordinate system;

[0087] A point in the second training image is selected as a third coordinate point, the third coordinate point is used as the upper left corner coordinate point of the second eliminated image, the horizontal coordinate of the third coordinate point is added to the height and length of the second eliminated image, the vertical coordinate of the third coordinate point is added to the width and length of the second eliminated image, and the lower right corner coordinate point of the second eliminated image is determined as the fourth coordinate point;

[0088] Comparing the abscissa of the fourth coordinate point with the width and length of the second training image, and comparing the ordinate of the fourth coordinate point with the height and length of the second training image, when the abscissa of the fourth coordinate point is less than or equal to the width and length of the second training image, and the ordinate of the fourth coordinate point is less than or equal to the height and length of the second training image, eliminating the second eliminated image from the second training image to obtain a second enhanced image;

[0089] The height of the second eliminated image is obtained by taking the square root of the product of the second eliminated area and the aspect ratio of the second eliminated image;

[0090] The width and length of the second eliminated image is obtained by taking the square root of the ratio of the second eliminated area to the aspect ratio of the second eliminated image.

[0091] In the above embodiment, after the effective area for random erasure is determined, it is necessary to convert the effective area into a removed image in the original image. Therefore, it is necessary to determine the height, length and width of the randomly erased image according to the randomly erased area. The height and length of the randomly erased image can be expressed as:

[0092]

[0093] Among them, S e represents the area of ​​random erasure, r e represents the aspect ratio of the randomly erased rectangular area, H e Indicates the height and length of the randomly erased image;

[0094] The width and length of the randomly erased image can be expressed as:

[0095]

[0096] Among them, S e represents the area of ​​random erasure, r e represents the aspect ratio of the randomly erased rectangular area, W e Indicates the width and length of the randomly erased image;

[0097] The left side of the image is taken as the first side of the image. After the coordinate system is established with the vertex of the first side of the original image as the origin of the coordinate system, a point P = (x e ,y e ) as the vertex coordinates of the first side of the randomly erased image, when x e +W e ≤W and y e +H e When ≤H, it means that all vertices of the randomly erased image of the rectangular area are located in the original image. At this time, the randomly erased image can be used as the culled image of the original image to meet the conditions of the random erasing operation. Otherwise, it is necessary to randomly select point P again.

[0098] Optionally, the method further includes:

[0099] Acquire the size of a first person outline region in the first enhanced image, and compare the size of the first person outline region with a preset outline size threshold; when the size of the first person outline region is greater than or equal to the preset outline size threshold, determine that the first enhanced image is valid; when the size of the first person outline region is less than the preset outline size threshold, discard the first enhanced image and regenerate a first rejection area;

[0100] Obtain the size of the second person contour area in the second enhanced image, and compare the size of the second person contour area with a preset contour size threshold. When the size of the second person contour area is greater than or equal to the preset contour size threshold, determine that the second enhanced image is valid; when the size of the second person contour area is less than the preset contour size threshold, discard the second enhanced image and regenerate a second elimination area.

[0101] In the above embodiment, after randomly erasing the image, because random erasing may result in the situation where all identifiable areas are deleted or all are retained, in order to ensure a balance between deleted information and retained information, noise is introduced to achieve a balance and achieve a better effect. The introduction of noise can be achieved through the following expression:

[0102]

[0103] k=1-(1-R) 2 =2R-R 2 ;

[0104] l = r × d;

[0105] d=random(d min ,d max );

[0106] δ x (δ y )=random(0,dl);

[0107] Where k is the retention ratio of the given mask M, which is composed of the binary mask M∈{0,1} W*H The sum(M) of 1s in the image is divided by the number of pixels H×W. H is the height of the image, and W is the width of the image. It represents the ratio of the area between the retained image and the original masked image. R is the ratio of the image input, r is the ratio of the two occluded parts in the unit, and d is a unit length. In the parameter d min ,d max The random(x,y) function represents a random value in the range of x and y. The length of the unit d does not affect the retention rate, but it determines the size of the square that falls. l is the length of the unit length that is not blocked. δx and δ y are the boundaries of the complete image unit (δ x and δ y Masks of r and d can be randomly given so that the masks cover all possible cases).

[0108] Optionally, inputting the training image into a neural network model to be trained for training includes:

[0109] The neurons in the neural network model to be trained are randomly divided into training neurons and standby neurons, and after the training image is input into the neural network model to be trained, the training neurons are trained based on the training image. After the training is completed, the trained neural network model to be trained is verified based on the training image by the training neurons and the standby neurons.

[0110] In the above embodiment, the neurons in the neural network model to be trained are divided into training neurons and standby neurons by random division, and the corresponding parameters in the training neurons are trained. During the training process, the parameters in the standby neurons are shielded, thereby improving the training efficiency of the neural network model to be trained. At the same time, in the verification process after the training is completed, both the training neurons and the standby neurons participate in the update, thereby ensuring the accuracy of the target neural network model obtained after training.

[0111] Optionally, the method further includes:

[0112] Inputting the personnel position information in the next frame of the image to be detected into the target neural network model for judgment, and generating a third judgment result;

[0113] The third judgment result is compared with the second judgment result to generate a comparison value, and the comparison value is compared with a preset similarity threshold. When the comparison value is greater than the preset similarity threshold, no alarm message is issued, and when the comparison value is less than the preset similarity threshold, an alarm message is issued.

[0114] In the above embodiment, after a segment of video stream data is input, the algorithm will split the video stream into frames for analysis, and perform target detection on each frame. After the detection is completed, the frame images are combined into a video stream of fixed length for output; this processing method means that the algorithm may frequently alarm for a fall risk of the same object; in order to avoid repeated alarms, the system expands the path tracking algorithm based on the target recognition algorithm; when the model detects the fall risk, the algorithm will compare the target recognition results of different frames. If the comparison results show that the targets of different frames have a similarity higher than the threshold, the algorithm will regard this group of targets in different frames as the same fall risk of the same target, thereby reducing the number of alarms.

[0115] Optionally, the alarm information includes an on-site loudspeaker reminder and a mobile Internet reminder.

[0116] In the above embodiment, the detected risk of falling persons is promptly notified to relevant personnel. The warning method includes timely reminders through the tweeters on site when there are tweeters on site. When there are no tweeters on site, reminders can also be made through SMS notifications, WeChat and phone calls, etc. Relevant personnel include the persons at risk of falling on site, the person in charge of the scene area, security guards and police, etc.

[0117] Embodiment 2:

[0118] The present invention provides a method for detecting the risk of a person falling. The specific implementation steps are as follows:

[0119] Step 1: Output the HD camera video stream of the specified area to the server.

[0120] Step 2: Detect the dangerous area in the video where there may be a risk of falling according to the video stream data obtained in step 1. After the dangerous area is detected, a virtual line is drawn on the dangerous area to be used in the subsequent steps to determine the risk of falling of personnel.

[0121] Step 3: Before the risk prediction of falling personnel is performed, it is necessary to identify personnel in the area. In the identification, the present invention adopts a method of converting the target detection problem into a single regression problem of directly extracting bounding boxes and class probabilities from the image. Only one operation is required to detect the target category and location information. The input image is divided into S×S grids. If the center of a target falls into a grid, the grid is responsible for detecting the target.

[0122] Step 4: For each grid, predict the bounding box and the confidence value of the box. The confidence value represents the confidence that the box contains an object. The confidence value definition formula is as follows:

[0123]

[0124] in, represents the confidence of the jth bounding box of the i-th grid, and Pr(object) represents the probability of whether the current box has an object. Represents the IOU ratio between the real detection box and the predicted detection box;

[0125] Each grid also predicts C conditional class probabilities: Pr(Class i |object);

[0126] The probability of a certain class appearing in the box and the degree of fit of the prediction box are expressed as:

[0127]

[0128] Among them, Class i Represents the i-th category.

[0129] Step 5: In order to achieve higher recognition rate and accuracy of target detection, the original training of the picture is carried out by combining the two methods of random cropping and mixing different original pictures. The goal of this method is to combine the initial two samples to generate new samples for training, so as to achieve the purpose of data enhancement;

[0130] New sample coordinates The expression is:

[0131]

[0132] Where M∈{0,1} W*H represents a binary mask indicating the locations cropped from the two images and filled, 1 represents the part where the mask is filled with 1, x A and x B Respectively represent the initial sample abscissa;

[0133] New sample coordinates The expression is:

[0134]

[0135] Where λ represents the combined ratio between two data points, which is sampled from the Beta distribution Beta(0,δ) (here δ is set to 1), y A and B Represent the initial sample ordinates respectively.

[0136] Step 6: To sample the binary mask M described above, we first sample the bounding box coordinates B = (r x ,r y ,r w ,r h ) for sampling, x A The area B in is removed and x is used B The patch cut from the area B in the sampled rectangular mask M has an aspect ratio proportional to the original image, and the coordinate B is at (x A ,x B ) is uniformly sampled as follows:

[0137]

[0138]

[0139] Among them, the cutting area ratio needs to satisfy λ represents the combination ratio between two data points.

[0140] Step 7: After the previous steps, the dangerous areas and people in the area are identified, and then the risk of people falling needs to be detected. In order to avoid repeated alarms, this system expands the path tracking algorithm based on the target recognition algorithm. When the model detects the risk of falling, the algorithm will compare the target recognition results of different frames. If the comparison results show that the targets of different frames have a similarity higher than the threshold, the algorithm will regard this group of targets in different frames as the same target with the same risk of falling, thereby reducing the number of alarms.

[0141] Step 8: The detected risk of falling persons will be promptly notified to relevant personnel. The warning methods include on-site loudspeaker reminders, SMS notifications, WeChat and phone calls, etc. The relevant personnel include the person at risk of falling on site, the person in charge of the scene area, security guards and police, etc.

[0142] See also Figure 3 An embodiment of the present application also provides a personnel fall risk detection system, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-mentioned methods when executing the computer program.

[0143] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the method steps described above when the program is executed by a processor.

[0144] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for detecting a risk of falling, characterized in that: include: Acquire a first training image and a second training image, and remove effective areas in the first training image and the second training image to obtain a first enhanced image and a second enhanced image, wherein the effective area includes a first removed area and a second removed area; Randomly cropping the first enhanced image and the second enhanced image respectively to obtain a first cropped image and a second cropped image, and splicing the first cropped image and the second cropped image to obtain a training image; Inputting the training image into the neural network model to be trained for training to obtain a target neural network model; Generate falling location information and personnel location information based on the acquired image to be detected, and input the falling location information into the target neural network model for judgment to generate a first judgment result; When the first judgment result is a location prone to falling, inputting the personnel location information into a target neural network model for judgment to generate a second judgment result; When the second judgment result is a prone-to-fall position, an alarm message is issued.

2. The method for detecting the risk of falling of a person according to claim 1, characterized in that: Before removing effective areas in the first training image and the second training image to obtain the first enhanced image and the second enhanced image, the method further includes: generating a first rejection area based on a first area of ​​the first training image, and comparing a first ratio of the first rejection area to the first area with a preset maximum rejection threshold and a minimum rejection threshold; When the first ratio is greater than a minimum rejection threshold and less than a maximum rejection threshold, determining the first rejection area to be a valid area in the first training image; When the first ratio is less than a minimum rejection threshold or greater than a maximum rejection threshold, regenerating a new first rejection area based on the first area; generating a second rejection area based on a second area of ​​the second training image, and comparing a second ratio of the second rejection area to the second area with a preset maximum rejection threshold and a minimum rejection threshold; When the second ratio is greater than a minimum rejection threshold and less than a maximum rejection threshold, determining the second rejection area as a valid area in the first training image; When the second ratio is smaller than the minimum rejection threshold or larger than the maximum rejection threshold, a new second rejection area is regenerated based on the second area.

3. The method for detecting the risk of falling of a person according to claim 1, characterized in that: The removing effective areas in the first training image and the second training image to obtain the first enhanced image and the second enhanced image comprises: Establishing a coordinate system with the vertex of the first side of the first training image as the origin of the coordinate system; A point in the first training image is selected as a first coordinate point, the first coordinate point is used as the upper left corner coordinate point of the first eliminated image, the horizontal coordinate of the first coordinate point is added to the height and length of the first eliminated image, the vertical coordinate of the first coordinate point is added to the width and length of the first eliminated image, and the lower right corner coordinate point of the first eliminated image is determined as the second coordinate point; Comparing the abscissa of the second coordinate point with the width and length of the first training image, and comparing the ordinate of the second coordinate point with the height and length of the first training image, and when the abscissa of the second coordinate point is less than or equal to the width and length of the first training image, and the ordinate of the second coordinate point is less than or equal to the height and length of the first training image, eliminating the first eliminated image from the first training image to obtain a first enhanced image; The height of the first eliminated image is obtained by taking the square root of the product of the first eliminated area and the aspect ratio of the first eliminated image; The width and length of the first eliminated image is obtained by taking the square root of the ratio of the first eliminated area to the aspect ratio of the first eliminated image.

4. The method for detecting the risk of falling of a person according to claim 1, characterized in that: The removing effective areas in the first training image and the second training image to obtain the first enhanced image and the second enhanced image also includes: Establishing a coordinate system with the vertex of the first side of the second training image as the origin of the coordinate system; A point in the second training image is selected as a third coordinate point, the third coordinate point is used as the upper left corner coordinate point of the second eliminated image, the horizontal coordinate of the third coordinate point is added to the height and length of the second eliminated image, the vertical coordinate of the third coordinate point is added to the width and length of the second eliminated image, and the lower right corner coordinate point of the second eliminated image is determined as the fourth coordinate point; Comparing the abscissa of the fourth coordinate point with the width and length of the second training image, and comparing the ordinate of the fourth coordinate point with the height and length of the second training image, when the abscissa of the fourth coordinate point is less than or equal to the width and length of the second training image, and the ordinate of the fourth coordinate point is less than or equal to the height and length of the second training image, eliminating the second eliminated image from the second training image to obtain a second enhanced image; The height of the second eliminated image is obtained by taking the square root of the product of the second eliminated area and the aspect ratio of the second eliminated image; The width and length of the second eliminated image is obtained by taking the square root of the ratio of the second eliminated area to the aspect ratio of the second eliminated image.

5. The method for detecting the risk of falling of a person according to claim 1, characterized in that: The method further comprises: Acquire the size of a first person outline region in the first enhanced image, and compare the size of the first person outline region with a preset outline size threshold; when the size of the first person outline region is greater than or equal to the preset outline size threshold, determine that the first enhanced image is valid; when the size of the first person outline region is less than the preset outline size threshold, discard the first enhanced image and regenerate a first rejection area; Obtain the size of the second person contour area in the second enhanced image, and compare the size of the second person contour area with a preset contour size threshold. When the size of the second person contour area is greater than or equal to the preset contour size threshold, determine that the second enhanced image is valid; when the size of the second person contour area is less than the preset contour size threshold, discard the second enhanced image and regenerate a second elimination area.

6. The method for detecting the risk of falling of a person according to claim 1, characterized in that: The step of inputting the training image into the neural network model to be trained for training comprises: The neurons in the neural network model to be trained are randomly divided into training neurons and standby neurons, and after the training image is input into the neural network model to be trained, the training neurons are trained based on the training image. After the training is completed, the trained neural network model to be trained is verified based on the training image by the training neurons and the standby neurons.

7. The method for detecting the risk of falling of a person according to claim 1, characterized in that: The method further comprises: Inputting the personnel position information in the next frame of the image to be detected into the target neural network model for judgment, and generating a third judgment result; The third judgment result is compared with the second judgment result to generate a comparison value, and the comparison value is compared with a preset similarity threshold. When the comparison value is greater than the preset similarity threshold, no alarm message is issued, and when the comparison value is less than the preset similarity threshold, an alarm message is issued.

8. The method for detecting the risk of falling of a person according to claim 1, characterized in that: The alarm information includes on-site loudspeaker reminders and mobile Internet reminders.

9. A personnel fall risk detection system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method steps described in any one of claims 1 to 8 are implemented.

Images