Personnel safety movement detection method and system based on visual identification technology

Abstract

The invention provides a personnel safety movement detection method and system based on a visual identification technology, and the method comprises the following steps: S1, carrying out the image collection of a video image, and recognizing and obtaining a walking route region in the video image; s2, identifying and acquiring a target human body area in the video picture through a human body detection model pre-trained based on YOLOv5, and identifying and acquiring a target vehicle area in the video picture through a vehicle detection model pre-trained based on YOLOv5; and S3, judging whether the target human body area is located in the target vehicle area or the walking route area or not, and when the target human body area is located outside the target vehicle area and the walking route area at the same time, judging that the target human body is located outside the safe walking route, and triggering an alarm. According to the invention, the method can achieve the detection of the moving path of a person in a safe walking route, and can effectively avoid the interference of a vehicle driver on a detection result.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and artificial intelligence technology, and in particular relates to a method and system for detecting safe movement of personnel based on visual recognition technology. Background Art

[0002] In some high-risk operation areas or traffic roads, personnel must walk along the prescribed safe routes to protect their safety. Under the existing technology, the current method of detecting the walking route of personnel usually uses the target detection model to identify and locate the human body in the image or video, and intelligently judge whether it is within the set area range coordinates. However, the target detection model usually requires a large amount of high-quality, accurately labeled data for training. The data set is not only costly, but also time-consuming and laborious. In addition, the existing target detection model has poor adaptability to environmental changes, especially when there are changes in lighting conditions, weather conditions, background complexity, etc., which may lead to performance degradation. At the same time, the existing target detection model is also difficult to handle the impact of human data in the vehicle on the judgment process, which is prone to misidentification problems. Summary of the invention

[0003] The present invention aims to provide a method and system for detecting safe movement of personnel based on visual recognition technology, so as to solve the technical problems that conventional personnel movement route recognition technology has high requirements on the software and hardware of the detection equipment and poor recognition accuracy under the existing technology.

[0004] To solve the above problems, the technical solution of the present invention is: a method for detecting safe movement of personnel based on visual recognition technology, comprising the following steps: S1: Capture the video image and identify the walking route area in the video image; S2: The target human region in the video is obtained by recognizing the human detection model based on the YOLOv5 pre-training, and the target vehicle region in the video is obtained by recognizing the vehicle detection model based on the YOLOv5 pre-training; S3: Determine whether the target human body area is located inside the target vehicle area or the walking route area. When the target human body area is located outside both the target vehicle area and the walking route area, determine that the target human body is outside the safe walking route and trigger an alarm.

[0005] Preferably, the step of capturing an image of a video screen in S1 specifically includes the following steps: S11: Acquire a video stream, extract single-frame video images from the video stream at fixed intervals, and pre-process the single-frame video images in sequence.

[0006] Preferably, identifying and acquiring the walking route area in the video picture in S1 specifically includes the following steps: S12: Recognize and obtain the pedestrian traffic sign in the video screen through the walking route detection model pre-trained based on YOLOv5, automatically construct the walking route area based on the extended path of the pedestrian traffic sign, and obtain a plurality of first coordinate values ​​of the boundary of the walking route area in the video screen based on the walking route detection model; The pedestrian traffic signs include zebra crossings and temporary guide tubes; Alternatively, the walking route area is manually framed in a video frame with a fixed shooting angle, and a plurality of first coordinate values ​​of the boundary of the walking route area in the video frame are acquired based on the walking route detection model.

[0007] Preferably, in S2, the target human body region and the target vehicle region in the video image are obtained by identifying the human body detection model and the vehicle detection model, which specifically includes the following steps: S21: Acquire a plurality of second coordinate values ​​of the boundary of the target human region in the video screen based on the human body detection model, and acquire a plurality of third coordinate values ​​of the boundary of the target vehicle region in the video screen based on the vehicle detection model.

[0008] Preferably, judging in S3 whether the target human body area is located inside the walking route area specifically comprises the following steps: S31: Obtain the vertical height and horizontal width of the target human body area. When the ratio of the vertical height to the horizontal width of the target human body area reaches a first preset threshold, determine that the bottom edge of the target human body area is the contact position between the feet of the target human body and the ground of the walking route. When the coordinates of the bottom edge of the target human body area are outside the coordinates of the walking route area, determine that the target human body is outside the safe walking route. When the ratio of the vertical height to the horizontal width of the target human body area does not reach the first preset threshold, and when any position coordinate of the target human body area is outside the coordinates of the walking route area, it is determined that the target human body is outside the safe walking route.

[0009] Preferably, judging in S3 whether the target human body area is located inside the target vehicle area specifically comprises the following steps: S32: Based on the coordinates of the target human area and the target vehicle area, the overlap ratio of the target human area and the target vehicle area is calculated by using the IoU intersection over union algorithm; when the overlap ratio of the target human area and the target vehicle area exceeds a second preset threshold, it is determined that the target human is inside the target vehicle.

[0010] Preferably, in S3, when it is determined that the target human body is outside the safe walking route, an alarm is triggered, which specifically includes the following steps: S33: Record the facial image of the target human body area and its moving path in the continuous frame video images, and remind the target human body that it is not within the safe walking route through an audible and visual alarm at the video monitoring site.

[0011] Based on the same concept, the present invention also provides a personnel safe movement detection system based on visual recognition technology, which executes any one of the above-mentioned personnel safe movement detection methods based on visual recognition technology, including: Video monitoring module, used to capture and process video stream data within the target detection area; A target detection module, wherein the target detection module is provided with a walking route detection model, a human body detection model and a vehicle detection model, and is used to obtain a walking route area, a target human body area and a target vehicle area in a video screen; A position calculation module, used to determine the relative position relationship between the walking route area, the target human area and the target vehicle area; The alarm module is used to trigger an alarm when the target human body is outside the safe walking route.

[0012] Based on the same concept, the present invention also provides a personnel safety movement detection device based on visual recognition technology, comprising: Memory for storing computer programs; A processor is used to implement the method for detecting safe movement of personnel based on visual recognition technology as described in any one of the above when executing the computer program.

[0013] Based on the same concept, the present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for detecting safe movement of personnel based on visual recognition technology as described in any one of the above is implemented.

[0014] Due to the adoption of the above technical solution, the present invention has the following advantages and positive effects compared with the prior art: The present invention provides a personnel safe movement detection method and system based on visual recognition technology, which supports automatic acquisition of fixed walking routes or temporary walking routes, and respectively acquires the walking route area, target human area and target vehicle area in a single-frame video screen through a walking route detection model, a human body detection model and a vehicle detection model, and determines whether the target human body is in a safe walking route only by judging the relative position relationship between the walking route area, the target human body area and the target vehicle area. Through the present invention, the requirements of the detection system for the intelligent performance of the walking route detection model, the human body detection model and the vehicle detection model can be effectively reduced, and the overall calculation amount of the detection system can be reduced. At the same time, the interference of the human body in the vehicle to the detection process can be effectively eliminated, and the accurate identification of personnel in the walking route area can be realized, which is suitable for dangerous scenes prone to accidents such as conventional traffic intersections, ports, and logistics warehousing. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The present invention provides a flow chart of a method for detecting safe movement of personnel based on visual recognition technology. DETAILED DESCRIPTION

[0016] The following is a further detailed description of a method and system for detecting safe movement of personnel based on visual recognition technology proposed by the present invention in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following description and claims.

[0017] First embodiment See also Figure 1 This embodiment provides a method for detecting safe movement of personnel based on visual recognition technology, which is used to automatically and accurately identify whether a person is walking within a safe walking route, and specifically includes the following steps: S1: Capture the video image and identify the walking route area in the video image.

[0018] S2: A human detection model based on YOLOv5 pre-training and a vehicle detection model based on YOLOv5 pre-training are provided. For the same video screen in S1, the target human area in the video screen is obtained by the human detection model recognition, and the target vehicle area in the video screen is obtained by the vehicle detection model recognition.

[0019] S3: Determine whether the target human body area is located inside the target vehicle area or the walking route area. When the target human body area is located outside the target vehicle area and the walking route area at the same time, it proves that the target human body is neither inside the safe walking route nor in the driving vehicle state. Therefore, it is determined that the target human body is outside the safe walking route and an alarm is triggered.

[0020] The following is a detailed description of the specific implementation steps and functions of a method for detecting safe movement of personnel based on visual recognition technology provided in this embodiment: Preferably, in this embodiment, the image acquisition of the video screen in S1 specifically includes the following steps: S11: The video stream data shot at the target site is obtained by local reading or network transmission, and single-frame video images are extracted from the video stream at fixed intervals. The fixed interval can be selected to extract a single-frame video image every 10 frames. People usually walk at a slower speed, and the video images between consecutive frames change little. Therefore, extracting single-frame video images at fixed intervals can remove redundant and repeated information. Under the premise of ensuring the continuity and accuracy of video image data reading, the amount of video image data that needs to be processed can be significantly reduced, thereby saving computing resources and time.

[0021] Then, the acquired single-frame video images are preprocessed in sequence, and the preprocessing includes: According to the preset width and height parameters, adjust the size of the cropped video image so that the video image can be correctly read by the walking route detection model, human body detection model and vehicle detection model described later.

[0022] The video image is denoised through Gaussian blur, bilateral filtering, median filtering and other methods to reduce random noise in the image and improve image quality.

[0023] The contrast of the video image is enhanced by histogram equalization, adaptive histogram equalization and other methods, making the target features in the video image more obvious.

[0024] Preferably, in this embodiment, identifying and acquiring the walking route area in the video picture in S1 specifically includes the following steps: S12: A walking route detection model based on YOLOv5 pre-training is provided. The walking route detection model is used to identify and obtain pedestrian traffic signs in the video screen. The pedestrian traffic signs include zebra crossings and temporary guide tubes, etc. Then, the walking route detection model automatically constructs a walking route area according to the extended path of the pedestrian traffic signs, and the walking route detection model obtains several first coordinate values ​​of the boundaries of the walking route area in the video screen.

[0025] Specifically, taking a traffic intersection as an example, the walking route detection model can identify the zebra crossing graphic structure on the ground, and determine that the area with zebra crossings is the walking route area based on the extension length of multiple zebra crossings. Taking a temporary pedestrian walkway without zebra crossings as an example, in the temporary pedestrian walkway, several temporary guide tubes will be placed at the intersections at both ends of the temporary pedestrian walkway and on both sides of the middle road. The walking route detection model can identify the temporary guide tube device and determine the rectangular structure formed by the several temporary guide tube devices as the walking route area.

[0026] Therefore, in this application, it supports automatic acquisition of fixed walking routes and automatic acquisition of temporary walking routes. At the same time, the walking route area can also be manually selected in the video screen with a fixed shooting angle, so as to realize more flexible and controllable setting of the walking route area.

[0027] Preferably, in this embodiment, in S2, the target human body area and the target vehicle area in the video screen are obtained by identifying the human body detection model and the vehicle detection model, which specifically includes the following steps: S21: Acquire a number of second coordinate values ​​of the boundary of the target human region in the video image based on the human body detection model, and acquire a number of third coordinate values ​​of the boundary of the target vehicle region in the video image based on the vehicle detection model.

[0028] Specifically, in the same frame of video, the center of the video image is used as the coordinate origin, and several first coordinate values ​​of the boundary of the walking route area, several second coordinate values ​​of the boundary of the target human body area, and several third coordinate values ​​of the boundary of the target vehicle area are respectively obtained. Therefore, in this embodiment, only the coordinate values ​​of the walking route area, the target human body area, and the target vehicle area need to be compared and calculated to determine the relative position relationship among the walking route area, the target human body area, and the target vehicle area, which can effectively reduce the computational complexity of data processing in the video image, as well as the requirements for the intelligent performance of the walking route detection model, the human body detection model, and the vehicle detection model.

[0029] Preferably, in this embodiment, determining in S3 whether the target human body area is located inside the walking route area specifically includes the following steps: S31: Obtain the vertical height and horizontal width of the target human body area. When the ratio of the vertical height to the horizontal width of the target human body area reaches a first preset threshold, determine that the bottom edge of the target human body area is the contact position between the target human body's feet and the ground of the walking route. When the bottom edge coordinates of the target human body area are outside the coordinates of the walking route area, determine that the target human body is outside the safe walking route.

[0030] Specifically, taking the case where the camera device is located on the upper left side or right side of the walking route area as an example, there is a large difference between the vertical height and the horizontal width of the target human body area obtained at this time, that is, the ratio of the vertical height to the horizontal width of the target human body area reaches the first preset threshold value. At this time, the bottom edge of the target human body area can be understood as the contact position between the feet of the target human body and the ground of the walking route. By judging whether the feet of the target human body are located inside the walking route area, it can be determined whether the target human body is in a safe walking route. The body of the target human body is affected by the positional relationship between the camera device and the walking route area, and the overlap ratio deviation with the walking route area is large (when the camera device is closer to the upper side of the walking route area, the overlap ratio of the body of the target human body and the walking route area is higher, and when the camera device is farther away from the upper side of the walking route area, the overlap ratio of the body of the target human body and the walking route area is lower). Therefore, in this embodiment, when the ratio of the vertical height to the horizontal width of the target human body area reaches the first preset threshold value, the overlap relationship between the body of the target human body and the walking route area is no longer considered, thereby improving the recognition accuracy of whether the target human body area is located inside the walking route area.

[0031] Further, when the ratio of the vertical height to the horizontal width of the target human body area does not reach the first preset threshold, when any position coordinate of the target human body area is outside the coordinate of the walking route area, it is determined that the target human body is outside the safe walking route.

[0032] Specifically, taking the case where the camera device is located directly above the walking route area as an example, the difference between the vertical height and the lateral width of the target human body area obtained at this time is small (especially when the target human body is located directly below the camera device, the vertical height and the lateral width of the target human body area obtained are basically the same), that is, the ratio of the vertical height to the lateral width of the target human body area does not reach the first preset threshold. At this time, it is necessary to determine whether any position of the target human body area is within the walking route area, so as to improve the recognition accuracy of whether the target human body area is within the walking route area.

[0033] Therefore, in this embodiment, the camera device can automatically switch the judgment standard for whether the target human body area is located inside the walking route area based on different setting positions of the walking route area, thereby improving the judgment accuracy in different complex situations.

[0034] Preferably, in this embodiment, determining in S3 whether the target human body area is located inside the target vehicle area specifically includes the following steps: S32: Based on the coordinates of the target human area and the target vehicle area, the overlap ratio of the target human area and the target vehicle area is calculated by using the IoU intersection over union algorithm. When the overlap ratio of the target human area and the target vehicle area exceeds a second preset threshold, it is determined that the target human is inside the target vehicle.

[0035] In this embodiment, the overlap ratio between the target human body area and the target vehicle area is determined by the IoU intersection-over-union algorithm to determine whether the target human body is inside the target vehicle, thereby effectively eliminating the interference of the people in the vehicle on the detection process.

[0036] In one embodiment, it is further proposed to calculate the overlap ratio of the target vehicle area and the walking route area through the IoU intersection over union algorithm. When the overlap ratio of the target vehicle area and the walking route area exceeds the third preset threshold, it is determined that the target vehicle is inside the walking route area. At this time, there may be a situation where the target human body is outside the target vehicle, but the target human body area overlaps with the target vehicle area. Therefore, the human body detection model further detects the posture of the target human body located inside the walking route area. When the posture of the target human body is consistent with the walking posture, the target human body area is overlapped with the walking route area. When the posture of the target human body is consistent with the sitting posture, the target human body area is first overlapped with the target vehicle area to determine whether the target human body is sitting inside the target vehicle. If the target human body is not sitting inside the target vehicle, the human body area is overlapped with the walking route area, thereby reducing the interference of the vehicle passing through the walking route area to the detection process.

[0037] Preferably, in this embodiment, in S3, when it is determined that the target human body is outside the safe walking route, an alarm is triggered, which specifically includes the following steps: S33: Record the facial image of the target human body area and its moving path in the continuous frame video images, and remind the target human body that it is not within the safe walking route through an audible and visual alarm at the video monitoring site.

[0038] In summary, this embodiment provides a method for detecting safe movement of personnel based on visual recognition technology, which supports automatic acquisition of fixed walking routes or temporary walking routes in video images, and respectively acquires walking route areas, target human areas and target vehicle areas in single-frame video images through walking route detection models, human body detection models and vehicle detection models, and then uses the coordinate data of walking route areas, target human body areas and target vehicle areas to determine whether the target human body is in a safe walking route only by judging the relative position relationship between the walking route areas, target human body areas and target vehicle areas. Through the present invention, the requirements of the detection system for the intelligent performance of the walking route detection model, human body detection model and vehicle detection model can be effectively reduced, and the overall calculation amount of the detection system can be reduced. At the same time, the interference of the human body in the vehicle with the detection process can be effectively eliminated, and the accurate identification of personnel in the walking route area can be realized, which is suitable for dangerous scenes prone to accidents such as conventional traffic intersections, ports, and logistics warehousing.

[0039] Second embodiment Based on the same concept, the present invention also provides a personnel safe movement detection system based on visual recognition technology, which executes the personnel safe movement detection method based on visual recognition technology as described in any one of the first embodiments, including: The video monitoring module is used to capture and obtain video stream data within the target detection area, extract single-frame video images at fixed intervals, and pre-process the video images.

[0040] The target detection module is provided with a walking route detection model, a human body detection model and a vehicle detection model, which are used to obtain the walking route area, the target human body area and the target vehicle area in the video screen, and the coordinate data of the walking route area, the target human body area and the target vehicle area.

[0041] The position calculation module is used to calculate and determine the relative position relationship between the walking route area, the target human area and the target vehicle area according to the coordinate data of the walking route area, the target human area and the target vehicle area.

[0042] The alarm module is used to trigger an alarm when the target human body is outside the safe walking route.

[0043] In one embodiment, the present invention also provides a personnel safe movement detection device based on visual recognition technology, including a processor and a memory, wherein an executable computer program is stored in the memory. When the processor calls the computer program in the memory, the processor is used to execute the personnel safe movement detection method based on visual recognition technology as described in the first embodiment.

[0044] In one embodiment, the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for detecting safe movement of personnel based on visual recognition technology as described in the first embodiment is implemented.

[0045] The embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they still fall within the protection scope of the present invention.

Claims

1. A method for detecting safe movement of personnel based on visual recognition technology, characterized in that: The steps include: S1: Capture the video image and identify the walking route area in the video image; S2: The target human region in the video is obtained by recognizing the human detection model based on the YOLOv5 pre-training, and the target vehicle region in the video is obtained by recognizing the vehicle detection model based on the YOLOv5 pre-training; S3: Determine whether the target human body area is located inside the target vehicle area or the walking route area. When the target human body area is located outside both the target vehicle area and the walking route area, determine that the target human body is outside the safe walking route and trigger an alarm.

2. The method for detecting safe movement of personnel based on visual recognition technology according to claim 1, characterized in that: In S1, the video screen is captured, which specifically includes the following steps: S11: Acquire a video stream, extract single-frame video images from the video stream at fixed intervals, and pre-process the single-frame video images in sequence.

3. The method for detecting safe movement of personnel based on visual recognition technology according to claim 1, characterized in that: In S1, the walking route area in the video image is identified and obtained, which specifically includes the following steps: S12: Recognize and obtain the pedestrian traffic sign in the video screen through the walking route detection model pre-trained based on YOLOv5, automatically construct the walking route area based on the extended path of the pedestrian traffic sign, and obtain a plurality of first coordinate values ​​of the boundary of the walking route area in the video screen based on the walking route detection model; The pedestrian traffic signs include zebra crossings and temporary guide tubes; Alternatively, the walking route area is manually framed in a video frame with a fixed shooting angle, and a plurality of first coordinate values ​​of the boundary of the walking route area in the video frame are acquired based on the walking route detection model.

4. The method for detecting safe movement of personnel based on visual recognition technology according to claim 1, characterized in that: In S2, the target human body region and the target vehicle region in the video image are obtained by identifying the human body detection model and the vehicle detection model, which specifically includes the following steps: S21: Acquire a plurality of second coordinate values ​​of the boundary of the target human region in the video screen based on the human body detection model, and acquire a plurality of third coordinate values ​​of the boundary of the target vehicle region in the video screen based on the vehicle detection model.

5. The method for detecting safe movement of personnel based on visual recognition technology according to claim 4, characterized in that: In S3, it is determined whether the target human body area is located inside the walking route area, which specifically includes the following steps: S31: Obtain the vertical height and horizontal width of the target human body area. When the ratio of the vertical height to the horizontal width of the target human body area reaches a first preset threshold, determine that the bottom edge of the target human body area is the contact position between the feet of the target human body and the ground of the walking route. When the coordinates of the bottom edge of the target human body area are outside the coordinates of the walking route area, determine that the target human body is outside the safe walking route. When the ratio of the vertical height to the horizontal width of the target human body area does not reach the first preset threshold, and when any position coordinate of the target human body area is outside the coordinates of the walking route area, it is determined that the target human body is outside the safe walking route.

6. The method for detecting safe movement of personnel based on visual recognition technology according to claim 5, characterized in that: In S3, it is determined whether the target human body area is located inside the target vehicle area, which specifically includes the following steps: S32: Based on the coordinates of the target human area and the target vehicle area, the overlap ratio of the target human area and the target vehicle area is calculated by using the IoU intersection over union algorithm; when the overlap ratio of the target human area and the target vehicle area exceeds a second preset threshold, it is determined that the target human is inside the target vehicle.

7. The method for detecting safe movement of personnel based on visual recognition technology according to claim 1, characterized in that: In S3, when it is determined that the target person is outside the safe walking route, an alarm is triggered, which specifically includes the following steps: S33: Record the facial image of the target human body area and its moving path in the continuous frame video images, and remind the target human body that it is not within the safe walking route through an audible and visual alarm at the video monitoring site.

8. A personnel safety movement detection system based on visual recognition technology, characterized in that: The method for detecting safe movement of personnel based on visual recognition technology as described in any one of claims 1 to 7 above comprises: Video monitoring module, used to capture and process video stream data within the target detection area; A target detection module, wherein the target detection module is provided with a walking route detection model, a human body detection model and a vehicle detection model, and is used to obtain a walking route area, a target human body area and a target vehicle area in a video screen; A position calculation module, used to determine the relative position relationship between the walking route area, the target human area and the target vehicle area; The alarm module is used to trigger an alarm when the target human body is outside the safe walking route.

9. A personnel safety movement detection device based on visual recognition technology, characterized in that: include: Memory for storing computer programs; A processor is used to implement the method for detecting safe movement of personnel based on visual recognition technology as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method for detecting safe movement of personnel based on visual recognition technology as described in any one of claims 1 to 7 is implemented.

Images