High-altitude falling object detection method and system based on machine vision

Abstract

The invention relates to the field of image processing, in particular to a high-altitude falling object detection method and system based on machine vision. The high-altitude falling object detection method based on machine vision comprises the following steps that a visual area is determined, a visual picture of the visual area is obtained, and the visual picture is captured by at least one imaging device; identifying a first falling object in the visual picture, and evaluating the risk degree of the first falling object; according to the risk degree of the first falling object, a second falling object in the visual picture is recognized, and the risk degree of the second falling object exceeds a risk degree threshold value; and predicting a falling point range of the second falling object, and triggering early warning of the high-altitude falling object according to the number of personnel in the falling point range. According to the invention, all-around monitoring and early warning can be carried out on high-altitude falling objects in complex environments such as a building construction site, so that the personnel and property safety guarantee is effectively improved, the occurrence of safety accidents is reduced, and the defects of traditional safety measures can be made up.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method and system for detecting high-altitude falling objects based on machine vision. Background Art

[0002] In the field of construction engineering, falling objects are one of the common safety hazards on construction sites, which can cause serious personal injury and property loss. Due to the complex and changeable environment of construction sites, frequent and uncertain high-altitude operations, although traditional safety measures can provide a certain degree of protection, there are many limitations in practical applications, such as physical barriers cannot cover all risk areas, and personnel monitoring is limited by factors such as viewing angle, reaction time and fatigue. Summary of the invention

[0003] The present invention provides a high-altitude falling object detection method and system based on machine vision, aiming to solve the deficiencies of traditional safety measures in the field of construction engineering and improve the detection and early warning capabilities of high-altitude falling objects.

[0004] In a first aspect, the present invention provides a method for detecting falling objects from a high altitude based on machine vision, comprising the following steps:

[0005] Determine a visual area, and obtain a visual picture of the visual area, wherein the visual picture is captured by at least one imaging device;

[0006] identifying a first falling object in the visual image, and assessing a risk level of the first falling object;

[0007] identifying a second falling object in the visual image according to the risk level of the first falling object, the risk level of the second falling object exceeding a risk level threshold;

[0008] The landing point range of the second falling object is predicted, and based on the number of people within the landing point range, a high-altitude falling object warning is triggered.

[0009] Furthermore, the obtaining of the visual image of the visual area comprises the following steps:

[0010] Dividing the visual area to obtain a plurality of visual sub-areas, wherein projections of the plurality of visual sub-areas on a vertical plane completely cover a projection of the falling object space on the vertical plane;

[0011] The plurality of imaging devices are matched one by one with the plurality of visual sub-areas, and the imaging devices are used to obtain visual sub-pictures of the corresponding visual sub-areas respectively, and the projection of any visual sub-picture on the vertical plane covers a part of the projection of the falling object space on the vertical plane.

[0012] Furthermore, the identifying the first falling object in the visual picture comprises the following steps:

[0013] The first falling object in each visual sub-screen is identified respectively, wherein the first falling object identification in any visual sub-screen includes the following steps:

[0014] Acquire a first visual sub-picture, where the first visual sub-picture is a previous visual sub-picture of two adjacent visual sub-picture frames;

[0015] Acquire a second visual sub-picture, where the second visual sub-picture is a later visual sub-picture of two adjacent frames of visual sub-pictures;

[0016] Obtaining a difference image through pixel distribution of the first visual sub-image and the second visual sub-image, wherein the pixel value at any position in the difference image is the pixel difference value at the corresponding position of the first visual sub-image and the second visual sub-image;

[0017] According to the pixel distribution in the binarized difference image, the moving area and the static area in the visual sub-picture are divided, and the first falling object is marked based on the edge of the moving area.

[0018] Furthermore, the marking of the first falling object based on the edge of the motion area comprises the following steps:

[0019] Edge pixels in the difference image are extracted, and based on the edge pixels, a minimum enclosing rectangle is generated, where the minimum enclosing rectangle is used to mark the corresponding first pendant.

[0020] Furthermore, the first visual sub-picture, the second visual sub-picture and the third visual sub-picture are acquired by the same imaging device.

[0021] Furthermore, the frame number corresponding to the second visual sub-picture is continuous with the frame number corresponding to the third visual sub-picture.

[0022] Furthermore, the triggering of the high-altitude falling object warning according to the number of people within the falling point range includes the following triggering rules:

[0023] When the number of people within the falling point range is greater than zero, a high-altitude falling object warning is triggered.

[0024] In a second aspect, the high-altitude falling object detection system based on machine vision provided by the present invention includes an input device, a processor, a memory and an output device, wherein the input device, the processor, the memory and the output device are interconnected, wherein the memory is used to store a computer program, and the computer program includes program instructions, and the processor is configured to call the program instructions to execute the high-altitude falling object detection method based on machine vision described in the first aspect.

[0025] The high-altitude falling object detection method and system based on machine vision provided by the present invention have the following benefits:

[0026] The present invention uses machine vision technology to achieve automatic identification and tracking of falling objects, and can predict the range of the landing point, and then provide timely warnings based on the distribution of personnel in the landing area, thereby improving the safety management efficiency of the construction site and reducing the risk of personal injury and property loss. Compared with traditional safety measures, the present invention can conduct all-round monitoring and early warning of falling objects in complex environments such as construction sites, thereby effectively improving the safety of personnel and property, reducing the occurrence of safety accidents, and making up for the shortcomings of traditional safety measures. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A flow chart of a high-altitude falling object detection method based on machine vision provided by an embodiment of the present invention;

[0028] Figure 2 A schematic diagram of visual area division provided by an embodiment of the present invention;

[0029] Figure 3 Schematic diagram of a high-altitude falling object detection system based on machine vision provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures and technologies are proposed to facilitate a thorough understanding of the embodiments of the present application.

[0031] However, it should be clear to those skilled in the art that the present application can be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present application with unnecessary details.

[0032] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections.

[0033] In order to simplify the drawings, each figure only schematically shows the parts related to the present invention, which do not represent the actual structure of the product.

[0034] In addition, in order to make the drawings simple and easy to understand, in some figures, only one of the components with corresponding structures or functions is schematically drawn or marked; in this article, "one" not only means "only one" but also can mean "more than one".

[0035] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0036] In the embodiments shown in the drawings, the indications of directions (such as up, down, left, right, front and back) used to explain the structure and movement of the various components of the present invention are not absolute but relative. These descriptions are appropriate when these components are in the positions shown in the drawings. If the descriptions of the positions of these components change, the indications of these directions also change accordingly.

[0037] In addition, in the description of the present application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0038] In order to more clearly illustrate the embodiments of the present invention or the prior art, the specific implementation of the present invention will be described below with reference to the accompanying drawings. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings and other implementations can be obtained based on these drawings without creative work.

[0039] In one embodiment, see Figure 1 , Figure 1 A flow chart of a high-altitude falling object detection method based on machine vision provided in an embodiment of the present invention.

[0040] like Figure 1 As shown, the high-altitude falling object detection method based on machine vision provided by an embodiment of the present invention includes the following steps:

[0041] S01. Determine a visual area, and obtain a visual picture of the visual area, where the visual picture is captured by at least one imaging device.

[0042] In this embodiment, the visual area refers to the spatial range for high-altitude falling object detection through machine vision at a construction site, and its size at least covers the area with the risk of high-altitude falling objects.

[0043] It is understandable that the spatial scope of the visual area needs to be determined based on the actual situation of the construction site to ensure all-round monitoring of falling objects from high altitudes.

[0044] Furthermore, in order to achieve comprehensive monitoring of the area with the risk of falling objects in the visual area, the step of obtaining the visual image of the visual area in step S01 includes the following steps:

[0045] S011. Divide the visual area to obtain multiple visual sub-areas, wherein projections of the multiple visual sub-areas on the vertical plane completely cover the projection of the falling object space on the vertical plane.

[0046] It should be noted that the vertical plane described in this embodiment refers to a two-dimensional plane perpendicular to the horizontal ground, which is used to represent the vertical movement projection of falling objects from high altitude in three-dimensional space, such as a vertical plane with the outer wall of a high-rise building as a reference in a high-rise construction project.

[0047] It should also be noted that the falling object space described in this embodiment refers to the three-dimensional space area within the construction site that falling objects from high altitudes may pass through or fall into. It indicates the range that potential falling objects from high altitudes may occupy during the falling process and is the key area that the visual picture needs to cover.

[0048] S012. Match the multiple imaging devices to the multiple visual sub-areas one by one, and use the imaging devices to obtain visual sub-screens of the corresponding visual sub-areas respectively, and the projection of any visual sub-screen on the vertical plane covers the partial projection of the falling object space on the vertical plane.

[0049] In this example, see Figure 2 , Figure 2 A schematic diagram of visual area division provided in an embodiment of the present invention. Figure 2 The solid rectangle in the middle represents the building, the black dots represent the imaging devices arranged on the outer wall of the building, the visible range on one side of the outer wall where the imaging device is installed belongs to the visual area of ​​this example, the rectangular area framed by the dotted lines and the solid lines of the outer wall represents the falling object space, and the spatial coordinate system XYZ is constructed based on the layout position of the imaging device O, its Y axis is parallel to the direction of gravity, the XOZ plane is perpendicular to the Y axis, and it is usually used to represent the horizontal ground in actual applications.

[0050] like Figure 2 As shown, the projection of the visual area corresponding to the building exterior wall on the vertical plane XOZ can completely cover the projection of the falling object space on the vertical plane XOZ; further, the visual picture corresponding to the visual area can be divided into multiple visual sub-pictures, and any visual sub-picture can cover the partial projection of the falling object space on the vertical plane.

[0051] It should be noted that for a visual area, each imaging device deployed to obtain its visual image has a fixed shooting angle in the process of obtaining the corresponding visual image. It should be further noted that for two visual areas, the number, position, shooting angle, shooting focal length and other parameters of the imaging devices deployed to obtain the corresponding visual images may be the same or different, which needs to be determined according to the actual situation of the construction site to ensure all-round monitoring of falling objects from high altitude.

[0052] S02: Identify a first falling object in the visual image, and assess the risk level of the first falling object.

[0053] In this embodiment, the first falling object described in step S02 refers to an object with obvious falling characteristics that is detected for the first time in the visual picture in the high-altitude environment of the construction site. It usually starts from a high place, such as a scaffold, a hanging basket, the edge of a floor, a tower crane arm and other high-altitude locations.

[0054] Furthermore, the step of identifying the first pendant in the visual picture in step S02 includes the following steps: identifying the first pendant in each visual sub-picture respectively.

[0055] Furthermore, the first falling object recognition in any visual sub-picture includes the following steps:

[0056] S021. Acquire a first visual sub-picture, where the first visual sub-picture is a previous visual sub-picture of two adjacent visual sub-picture frames.

[0057] S022: Acquire a second visual sub-picture, where the second visual sub-picture is a later visual sub-picture of two adjacent frames of visual sub-pictures.

[0058] It should be noted that the subsequent visual sub-picture frame mentioned here is the current visual sub-picture frame.

[0059] S023. Obtain a difference image through pixel distribution of the first visual sub-picture and the second visual sub-picture, wherein the pixel value at any position in the difference image is the pixel difference value of the first visual sub-picture and the second visual sub-picture at the corresponding position.

[0060] It should be noted that, in order to facilitate data calculation, the calculation of the pixel difference of the difference image in step S023 is based on the pixel value distribution of the grayscaled first visual sub-image and the second visual sub-image.

[0061] S024. Divide the visual sub-picture into a moving area and a static area according to the pixel distribution in the binarized difference image, and mark the first falling object based on the edge of the moving area.

[0062] In this embodiment, the dynamic area refers to the area where the pixel value changes significantly in two consecutive frames of visual sub-pictures, which is formed by the movement trajectory of the first pendant; the static area refers to the area where the pixel value remains basically unchanged in two consecutive frames of visual sub-pictures, such as the background environment.

[0063] Furthermore, in the step of binarizing the difference image in step S024, the binarization grayscale threshold is set according to a specific scenario.

[0064] Furthermore, in step S024, marking the first falling object based on the edge of the motion area includes the following steps:

[0065] The edge detection algorithm is used to extract edge pixels in the difference image, and based on the edge pixels, a minimum enclosing rectangle is generated, and the minimum enclosing rectangle is used to mark the corresponding first pendant.

[0066] In this embodiment, the risk level described in step S02 is a preliminary assessment of the potential threat level posed by the first falling object falling to the ground to on-site personnel based on the position characteristics, two-dimensional characteristics and motion characteristics of the first falling object.

[0067] Specifically, the risk level of any first falling object satisfies the following assessment model: Among them, E represents the risk level, S c represents the minimum enclosing rectangular area of ​​the first pendant, Δh represents the vertical displacement of the center of the minimum enclosing rectangle of the first pendant in the second visual sub-screen and the third visual sub-screen, Δt represents the recording time difference between the second visual sub-screen and the third visual sub-screen, g represents the gravitational acceleration, and H represents the height of the imaging device for capturing the second visual sub-screen and the third visual sub-screen and the horizontal ground.

[0068] Furthermore, Δh=y a -y b , where y a Indicates the vertical center height of the minimum enclosing rectangle of the first falling object in the second visual sub-image, y b Indicates the vertical center point height of the minimum enclosing rectangle of the first pendant in the third visual sub-picture.

[0069] It should be noted that the third visual sub-picture is another frame of visual sub-picture captured by the corresponding imaging device after the second visual sub-picture marking the first pendant, and its frame number may be continuous or discontinuous with the frame number of the second visual sub-picture.

[0070] For example, if the second visual sub-picture is counted as the second frame visual sub-picture, the third visual sub-picture may be any one of the third frame, the fourth frame, ..., and the nth frame visual sub-picture.

[0071] It should be further noted that the first visual sub-picture, the second visual sub-picture and the third visual sub-picture are obtained by the same imaging device.

[0072] S03. According to the risk level of the first falling object, identify a second falling object in the visual image, wherein the risk level of the second falling object exceeds a risk level threshold.

[0073] In this embodiment, the second falling object refers to a high-risk falling object selected from among multiple first falling objects through a screening mechanism after the initial detection of multiple first falling objects.

[0074] Furthermore, the risk level threshold is a critical value for screening high-risk falling objects. For the multi-target detection results, other irrelevant image interference is effectively filtered out to ensure the accuracy and effectiveness of multi-target detection, such as the first falling object with a very small minimum enclosing rectangle, and the first falling object that is in motion but not continuously falling. Specifically, the risk level threshold can be set by fixed, dynamic or statistical methods to ensure the accuracy and effectiveness of multi-target monitoring.

[0075] S04. Predict the landing point range of the second falling object, and trigger a high-altitude falling object warning based on the number of people within the landing point range.

[0076] In this embodiment, the landing point range represents the area range where the second falling object may eventually contact the ground or building structure during the falling process, which is usually expressed as a circular, elliptical or rectangular area on the horizontal ground perpendicular to the above-mentioned vertical plane, and is used to cover all locations where the falling object may land.

[0077] Furthermore, the falling point range S of the second falling object is E Satisfies the following model:

[0078] (x-x0) 2 +(z-z0) 2 ≤R 2 , where (x,z) represents the point range S E A point within, (x0, z0) represents the projection point of the center of the minimum enclosing rectangle of the second falling object on the horizontal ground perpendicular to the vertical plane, the origin of the horizontal ground is the projection point of the imaging device layout position of the second visual sub-screen on the horizontal ground, and R is the risk radius.

[0079] Furthermore, the risk radius R can be fixed by empirical values ​​or safety regulations. For example, a 1-2 meter safety boundary is usually set as the risk radius at a construction site.

[0080] In this embodiment, the step S04 described in which a high-altitude falling object warning is triggered according to the number of people within the landing point range includes the following triggering rules: when the number of people within the landing point range is greater than zero, a high-altitude falling object warning is triggered.

[0081] Furthermore, the persons within the landing point range can be counted by machine vision algorithms (such as YOLO, etc.), and on-site personnel can be notified by visual, sound or communication methods to remind the persons within the landing point range to pay attention and take protective measures quickly.

[0082] In one embodiment, see Figure 3 , Figure 3 Schematic diagram of a high-altitude falling object detection system based on machine vision provided in an embodiment of the present invention.

[0083] like Figure 3 As shown, the high-altitude falling object detection system based on machine vision provided by the present invention includes an input device, a processor, a memory and an output device, and the input device, the processor, the memory and the output device are interconnected.

[0084] Furthermore, the input device described in this embodiment is used to obtain visual images of the visual area and transmit these images to the processor for further processing.

[0085] Typically, the input device is an industrial camera or other type of imaging device; in some embodiments, it may include multiple industrial cameras to cover a wide range of monitoring areas.

[0086] Furthermore, the processor described in this embodiment is the computing and analysis core of the system, and is used to perform tasks such as image processing, target detection, risk assessment, and early warning triggering. It executes a method for detecting falling objects from high altitudes by calling program instructions in a memory.

[0087] It is understandable that the processor needs to have high-performance computing capabilities to process large amounts of image data acquired by the input device in real time; in some embodiments, in order to reduce the computing pressure of the central processing unit, some edge processors are also provided to preliminarily process the image data.

[0088] Furthermore, the memory is used to store a computer program, which includes program instructions, and the processor is configured to call the program instructions to execute the high-altitude falling object detection method based on machine vision provided by the present invention.

[0089] Specifically, the memory includes short-term storage (such as RAM) and long-term storage (such as SSD); in some embodiments, the memory also includes cloud storage, which can synchronize a large amount of image data to the cloud for storage.

[0090] Furthermore, the output device is responsible for transmitting the detection results, risk assessment and warning signals to relevant personnel or systems to achieve information visualization and warning communication.

[0091] In this embodiment, the output device supports multiple warning modes, including visual output, sound output and communication output. Specifically, the output device includes a display screen, a speaker, and a mobile terminal (such as a smart phone, a smart watch, etc.) connected to the processor signal via a wireless network (such as Wi-Fi, 4G / 5G).

[0092] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0093] It should be noted that the above embodiments can be freely combined as needed. The above is only a preferred implementation of the present invention; it should be noted that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting falling objects from a high altitude based on machine vision, characterized in that: The steps include: Determine a visual area, and obtain a visual picture of the visual area, wherein the visual picture is captured by at least one imaging device; identifying a first falling object in the visual image, and assessing a risk level of the first falling object; identifying a second falling object in the visual image according to the risk level of the first falling object, the risk level of the second falling object exceeding a risk level threshold; The landing point range of the second falling object is predicted, and based on the number of people within the landing point range, a high-altitude falling object warning is triggered.

2. The high-altitude falling object detection method based on machine vision according to claim 1 is characterized in that: The step of obtaining the visual image of the visual area comprises the following steps: The visual area is divided to obtain a plurality of visual sub-areas, and the projections of the plurality of visual sub-areas on the vertical plane completely cover the projection of the falling object space on the vertical plane.

3. The high-altitude falling object detection method based on machine vision according to claim 2 is characterized in that: The step of obtaining the visual image of the visual area comprises the following steps: The plurality of imaging devices are matched one by one with the plurality of visual sub-areas, and the imaging devices are used to obtain visual sub-pictures of the corresponding visual sub-areas respectively, and the projection of any visual sub-picture on the vertical plane covers a part of the projection of the falling object space on the vertical plane.

4. The high-altitude falling object detection method based on machine vision according to claim 3 is characterized in that: The step of identifying the first falling object in the visual picture comprises the following steps: The first falling object in each visual sub-screen is identified respectively, wherein the first falling object identification in any visual sub-screen includes the following steps: A first visual sub-picture is obtained, where the first visual sub-picture is a previous visual sub-picture of two adjacent visual sub-picture frames.

5. The high-altitude falling object detection method based on machine vision according to claim 4 is characterized in that: The step of identifying the first falling object in the visual picture comprises the following steps: Acquire a second visual sub-picture, where the second visual sub-picture is a later visual sub-picture of two adjacent frames of visual sub-pictures; Obtaining a difference image through pixel distribution of the first visual sub-image and the second visual sub-image, wherein the pixel value at any position in the difference image is the pixel difference value at the corresponding position of the first visual sub-image and the second visual sub-image; According to the pixel distribution in the binarized difference image, the moving area and the static area in the visual sub-picture are divided, and the first falling object is marked based on the edge of the moving area.

6. The high-altitude falling object detection method based on machine vision according to claim 5 is characterized in that: The step of marking the first falling object based on the edge of the motion area comprises the following steps: Edge pixels in the difference image are extracted, and based on the edge pixels, a minimum enclosing rectangle is generated, where the minimum enclosing rectangle is used to mark the corresponding first pendant.

7. The high-altitude falling object detection method based on machine vision according to claim 6 is characterized in that: The first visual sub-picture, the second visual sub-picture and the third visual sub-picture are acquired by a same imaging device.

8. The high-altitude falling object detection method based on machine vision according to claim 7 is characterized in that: The frame number corresponding to the second visual sub-picture is continuous with the frame number corresponding to the third visual sub-picture.

9. The high-altitude falling object detection method based on machine vision according to any one of claims 1 to 8, characterized in that: The triggering of the high-altitude falling object warning according to the number of people within the falling point range includes the following triggering rules: When the number of people within the falling point range is greater than zero, a high-altitude falling object warning is triggered.

10. A high-altitude falling object detection system based on machine vision, characterized in that: It includes an input device, a processor, a memory and an output device, wherein the input device, the processor, the memory and the output device are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the high-altitude falling object detection method based on machine vision as described in any one of claims 1 to 9.

Images