A method, apparatus and device for out-of-boundary alarm

Abstract

The application discloses a kind of border crossing warning method, device and equipment.The method comprises: receiving the feature image information corresponding to at least one mobile object in target area, and receiving the position information of mobile object on the camera device deployed on monitoring vehicle system and the distance information between mobile object and camera device;For each mobile object, according to the distance information of current mobile object and the pose information of camera device, determine the current three-dimensional point cloud data of current mobile object in target area, based on feature image information, pre-set operation information, current three-dimensional point cloud data of current mobile object and the region three-dimensional point cloud data corresponding to target area, determine whether current mobile object is out of bounds;If yes, send early warning prompt, through the feature image information and three-dimensional point cloud data analysis processing of mobile object, realize the real-time monitoring of mobile object, and timely early warning to out of bounds behavior, improve operation safety, reduce monitoring cost.

Description

Technical Field

[0001] This invention relates to the field of safety risk management technology, and in particular to a boundary crossing alarm method, device and equipment. Background Technology

[0002] Safety is an eternal theme. In any industry, ensuring operational safety is the foundation for the survival and development of an enterprise. Therefore, real-time monitoring of the work site and alarms for boundary crossings play an important role in safe production.

[0003] Traditional boundary crossing alarm methods involve deploying a large amount of hardware equipment at the work site to locate moving objects based on positioning base stations and positioning tags, or based on monocular two-dimensional cameras.

[0004] The former method requires a large number of devices to be set up on site, which is costly and takes up a lot of space, making it unsuitable for operation. The latter method may result in false alarms or missed alarms due to issues with shooting angle and camera accuracy. Summary of the Invention

[0005] This invention provides a boundary crossing alarm method, device, and equipment to achieve real-time monitoring of moving objects and timely warning of boundary crossing behavior, thereby improving operational safety and reducing monitoring costs.

[0006] In a first aspect, embodiments of the present invention provide an out-of-bounds alarm method, the method comprising:

[0007] The system receives feature image information corresponding to at least one moving object within a target area, and receives location information corresponding to the moving object and distance information between the moving object and the camera device uploaded by a camera device deployed on a monitoring vehicle system; wherein the moving object includes at least one of a worker, a work vehicle, and a work tool; the work tool refers to the auxiliary tools used by the worker during the work process;

[0008] For each moving object, the current three-dimensional point cloud data of the moving object in the target area is determined based on the distance information of the moving object and the pose information of the camera device. The pose information includes the shooting position information and shooting orientation information of the camera device.

[0009] Based on the feature image information, the pre-set operation information, the current 3D point cloud data of the currently moving object, and the corresponding 3D point cloud data of the target area, it is determined whether the currently moving object has crossed the boundary; wherein, the operation information includes at least one of the following: the characteristic information of the operator, the operation time, the operation location information, the status information of the nearby electrical equipment, and the characteristic information of the operation vehicle;

[0010] If so, a warning notification will be sent.

[0011] Secondly, embodiments of the present invention also provide a boundary crossing alarm device, the device comprising:

[0012] The data receiving module is used to receive feature image information corresponding to at least one moving object within the target area, and to receive location information corresponding to the moving object and distance information between the moving object and the camera device uploaded by the camera device deployed on the monitoring vehicle system; wherein, the moving object includes at least one of the following: workers, work vehicles, and work tools; the work tools refer to the auxiliary tools used by the workers during the work process;

[0013] The point cloud data determination module is used to determine the current three-dimensional point cloud data of each moving object within the target area based on the distance information of the moving object and the pose information of the camera device. The pose information includes the shooting position information and shooting orientation information of the camera device.

[0014] The boundary crossing judgment module is used to determine whether the current moving object has crossed the boundary based on the feature image information, the pre-set operation information, the current three-dimensional point cloud data of the current moving object, and the regional three-dimensional point cloud data corresponding to the target area; wherein, the operation information includes at least one of the following: the characteristic information of the operator, the operation time, the operation location information, the status information of the adjacent electrical equipment, and the characteristic information of the operation vehicle;

[0015] The warning notification sending module is used to send a warning notification if the condition is met.

[0016] Thirdly, the present invention also provides an electronic device comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the out-of-bounds alarm method according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the out-of-bounds alarm method according to any embodiment of the present invention.

[0021] The technical solution of this invention receives feature image information corresponding to at least one moving object within a target area, and receives location information of the moving object and distance information between the moving object and the camera device uploaded by a camera device deployed on a monitoring vehicle system. For each moving object, based on the distance information of the current moving object and the pose information of the camera device, the current three-dimensional point cloud data of the current moving object within the target area is determined. Based on the feature image information, pre-set operation information, the current three-dimensional point cloud data of the current moving object, and the corresponding three-dimensional point cloud data of the target area, it is determined whether the current moving object has crossed the boundary; if so, a warning is sent. This solves the problems of high cost, large space occupation, and inconvenience for operation due to the deployment of a large number of devices on site, as well as the problems of false alarms or missed alarms due to camera shooting angle and accuracy issues. It achieves real-time monitoring of moving objects, timely warnings for boundary crossing behavior, improves operational safety, and reduces monitoring costs.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of an out-of-bounds alarm method provided in Embodiment 1 of the present invention;

[0025] Figure 2 This is a method for determining a region-related data set according to Embodiment 1 of the present invention;

[0026] Figure 3 This is a flowchart of an out-of-bounds alarm method provided in Embodiment 2 of the present invention;

[0027] Figure 4 This is a flowchart of an out-of-bounds alarm method provided in Embodiment 3 of the present invention;

[0028] Figure 5 This is a schematic diagram of the structure of a boundary crossing alarm device provided in Embodiment 4 of the present invention;

[0029] Figure 6 This is a schematic diagram of the structure of an electronic device that implements the boundary crossing alarm method of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] Before introducing the technical solution of this embodiment, the monitoring vehicle system with boundary crossing alarm is first described: The monitoring vehicle system consists of a mobile vehicle, an onboard internal memory, a vehicle locator, a multi-axis sensor, a camera device, a snapshot camera, an onboard human-machine interface platform, an onboard information processor, an onboard alarm device, and a wireless communication module. The onboard internal memory stores the corresponding 3D point cloud data of each device within the work area. The vehicle locator determines the current location information of the monitoring vehicle system. The multi-axis sensor, installed in the camera device, determines the direction and speed of movement of the monitoring vehicle system, as well as the camera's shooting orientation. The camera device collects distance information of the moving object relative to the camera device. The snapshot camera collects feature image information of the moving object. The onboard human-machine interface platform allows operators to confirm whether the moving object on site matches the information on the work order. The onboard information processor processes the information collected by the vehicle locator, multi-axis sensor, camera device, and snapshot camera, and sends the processing results to the backend intranet processor via the wireless communication module. Furthermore, the intranet processor is used to store the 3D point cloud data corresponding to each device within the work area, as well as the moving object information on the work ticket.

[0033] Example 1

[0034] Figure 1This is a flowchart of a boundary crossing alarm method provided in Embodiment 1 of the present invention. This embodiment can be applied to the situation of determining whether a moving object has crossed the boundary. The method can be executed by a boundary crossing alarm device, which can be implemented in hardware and / or software and can be configured in an electronic device.

[0035] like Figure 1 As shown, the method includes:

[0036] S110. Receive feature image information corresponding to at least one moving object within the target area, and receive location information corresponding to the moving object and distance information between the moving object and the camera device uploaded by the camera device deployed on the monitoring vehicle system.

[0037] The target area refers to the area where boundary crossing judgments need to be made for moving objects within the area. A moving object refers to an object and / or person that can move relative to fixed equipment within the target area. For example, a moving object can be, but is not limited to, workers, vehicles, and tools. Furthermore, tools refer to auxiliary equipment used by workers during operations. Feature image information refers to photographs containing the features of the moving object. This feature image information can be acquired by a capture camera deployed on a monitoring vehicle system. The monitoring vehicle system refers to a vehicle system that issues boundary crossing alarms for moving objects. The camera device refers to the device in the monitoring vehicle system that can capture images of moving objects, calculate their size, and measure the distance between the moving object and the camera device. For example, the camera device can be a binocular camera, a tri-lens camera, or a multi-lens camera. Position information refers to the coordinate information of each point on the surface of the moving object relative to the camera device's reference frame. Distance information refers to the distance information between each point on the surface of the moving object and the camera device.

[0038] Specifically, the vehicle-mounted information processor receives feature image information of moving objects captured by the camera, as well as location information of the moving objects and distance information between the moving objects and the camera uploaded by the camera device.

[0039] It should be noted that the number of objects to be moved varies depending on the specific task, so there can be one or more objects to be moved.

[0040] For example, if the moving object in target area A is worker A, then facial feature image information of worker A can be captured by a camera and received by an onboard information processor. Simultaneously, the onboard information processor receives the location information of worker A and the distance information between worker A and the camera device, both captured by the camera.

[0041] S120. For each moving object, determine the current three-dimensional point cloud data of the moving object within the target area based on the distance information of the current moving object and the pose information of the camera device.

[0042] The pose information includes the camera device's shooting position information and shooting orientation information. Shooting position information refers to the camera device's geographical location, and shooting orientation information refers to the camera device's shooting direction information. In this embodiment, a locator and a multi-axis sensor can be installed on the camera device to determine the camera device's pose information based on the locator and the multi-axis sensor. A locator is a device capable of locating the camera device and acquiring shooting position information in real time. A multi-axis sensor is a sensor capable of determining the shooting orientation information of the shooting device. Three-dimensional point cloud data refers to a collection of massive point position data on the surface of an object under the same spatial reference frame. For example, Pi = (x, y, z) represents a point in space, and Point Cloud = (P1, P2, P3, P4, ... Pi) represents i points in space.

[0043] Specifically, since the method for determining the 3D point cloud data corresponding to each moving object is the same, the determination of the 3D point cloud data of one moving object will be explained below: The camera device outputs the coordinate information and distance information of each point on the surface of the moving object relative to the camera device's reference frame, and sends them to the vehicle-mounted information processor. The vehicle-mounted information processor establishes a transformation function between the camera device's reference frame and the satellite reference frame based on the shooting position information and shooting direction information. The transformation function is used to transform the coordinate information of each point on the surface of the current moving object in the camera device's reference frame to obtain the current 3D point cloud data corresponding to the current moving object.

[0044] S130. Based on the feature image information, the pre-set job information, the current three-dimensional point cloud data of the currently moving object, and the regional three-dimensional point cloud data corresponding to the target area, determine whether the currently moving object has crossed the boundary.

[0045] The operation information can include at least one of the following: characteristic information of the moving object, operation time, operation location information, status information of nearby energized equipment, characteristic information of the operation tool, and characteristic information of the operation vehicle. Further, the characteristic information of the moving object can include facial features of the operator, license plate number of the operation vehicle, etc. Operation location information refers to the geographical coordinates of the operation location; for example, the operation location information can be the coordinates of the center of the area where the operation location is located. Nearby energized equipment refers to energized equipment such as overhead busbars and lead wires above the operation location. Nearby energized equipment status information indicates the energization status of nearby energized equipment within the operation time range. For example, the nearby energized equipment status information can be 1, indicating that the nearby energized equipment is in an energized state; the nearby energized equipment status information can be 0, indicating that the nearby energized equipment is in a de-energized state. The regional 3D point cloud data corresponding to the target area refers to the data set of geographical coordinates corresponding to each point in the target area.

[0046] Specifically, it can be determined whether the current moving object is a pre-defined moving object in the task information by comparing the feature image information with the feature information of the moving object in the pre-set task information. If not, it is determined that the current moving object has crossed the boundary. Furthermore, if the current 3D point cloud data of the current moving object is in the corresponding 3D point cloud data of the target area, it is determined that the current moving object has not crossed the boundary; otherwise, it is determined that the current moving object has crossed the boundary.

[0047] S140. If so, send a warning notification.

[0048] Among them, early warning prompts refer to prompts made by issuing early warning information, controlling flashing lights, and sounding alarms through alarm devices.

[0049] For example, if the vehicle information processor determines that the moving object has crossed the boundary, it will issue an alarm through the vehicle alarm device and send a warning to the background through the wireless communication module, and issue a boundary crossing alarm in the background.

[0050] The technical solution of this invention receives feature image information corresponding to at least one moving object within a target area, and receives location information of the moving object and distance information between the moving object and the camera device uploaded by a camera device deployed on a monitoring vehicle system. For each moving object, based on the distance information of the current moving object and the pose information of the camera device, the current three-dimensional point cloud data of the current moving object within the target area is determined. Based on the feature image information, pre-set operation information, the current three-dimensional point cloud data of the current moving object, and the corresponding three-dimensional point cloud data of the target area, it is determined whether the current moving object has crossed the boundary; if so, a warning is sent. This solves the problems of high cost, large space occupation, and inconvenience for operation due to the deployment of a large number of devices on site, as well as the problems of false alarms or missed alarms due to camera shooting angle and accuracy issues. It achieves real-time monitoring of moving objects, timely warnings for boundary crossing behavior, improves operational safety, and reduces monitoring costs.

[0051] In this embodiment, before receiving the feature image information corresponding to at least one moving object within the target area, and before receiving the location information of the moving object and the distance information between the moving object and the camera device uploaded by the camera device deployed on the monitoring vehicle system, the method further includes establishing a regional association data set corresponding to the working area where the target area is located. See [link to relevant documentation]. Figure 2 :

[0052] S101. Perform lidar scanning on each device and the environment within the work area where the target area is located to obtain three-dimensional point cloud data corresponding to each device.

[0053] LiDAR scanning involves using a LiDAR scanner to scan the surface of an object with a line laser emitted from a line laser, obtaining spatial location information for each point on the object's surface. Furthermore, a LiDAR scanner is a system that combines three technologies: laser, GPS, and inertial measurement unit.

[0054] Specifically, a LiDAR scanner is used to scan the equipment and environment in the work area where the target area is located, and to obtain the three-dimensional point cloud data of all equipment in the work area where the target area is located.

[0055] For example, the target area is located in substation A. A LiDAR scanner is used to scan the ground, walls, capacitors, wires, etc. in the substation to obtain the three-dimensional point cloud data corresponding to each device.

[0056] S102. Based on the three-dimensional point cloud data, determine a three-dimensional spatial model corresponding to the work area.

[0057] Specifically, the obtained 3D point cloud data corresponding to the equipment and environment of the work area are input into 3D modeling software, and finally a 3D spatial model of the work area is output. Furthermore, in this embodiment, the method for determining the corresponding 3D spatial model of the work area can also be a large-scale scene modeling method based on images; this embodiment does not impose any limitations on this method.

[0058] Based on the above example, the obtained 3D point cloud data of each device and environment in substation A is input into the modeling software to obtain a 3D spatial model of the interior of substation A. In the 3D spatial model, the distribution of each device can be clearly observed.

[0059] S103. Divide the corresponding operation sub-regions according to the equipment intervals of each device in the three-dimensional space model, and determine the monitoring area associated with each operation sub-region.

[0060] Here, equipment interval refers to the blank area between equipment. A work sub-area refers to the spatial area corresponding to each equipment interval. The monitoring area refers to the area where monitoring vehicles can move freely while monitoring the target area.

[0061] For example, in a substation, three work sub-areas are divided according to equipment bays: work sub-area A, work sub-area B, and work sub-area C. Users can define corresponding monitoring areas for each work sub-area based on the monitoring range of the monitoring vehicle. For instance, the monitoring area corresponding to work sub-area A is monitoring area A, the monitoring area corresponding to work sub-area B is monitoring area B, and the monitoring area corresponding to work sub-area C is monitoring area C.

[0062] S104. Based on the three-dimensional point cloud data of each operation sub-region and the three-dimensional point cloud data of the monitoring area associated with each operation sub-region, establish a regional association data set.

[0063] Among them, the regional association data set is a data set that can represent the three-dimensional point cloud data association relationship between the operation sub-region and the monitoring region.

[0064] Based on the above example, the 3D point cloud data [P1,P2,P3] corresponding to the operation sub-region A is associated with the 3D point cloud data [M1,M2,M3] corresponding to the monitoring area A to obtain [P1,P2,P3,M1,M2,M3]. The same processing is performed on the 3D point cloud data of each operation sub-region and the corresponding monitoring area to obtain the regional association data set.

[0065] In this embodiment, the purpose of determining the associated data set is to enable the monitoring vehicle system to determine the target area and the monitoring area associated with the target area based on the work location information in the work information and the regional associated data set.

[0066] The technical solution of this invention involves scanning the equipment and environment within the work area where the target area is located using LiDAR to obtain three-dimensional point cloud data corresponding to each equipment. Based on the three-dimensional point cloud data, a three-dimensional spatial model corresponding to the work area is determined. The work area is then divided into corresponding sub-regions according to the equipment spacing of each device in the three-dimensional spatial model, and a monitoring area associated with each sub-region is determined. Based on the three-dimensional point cloud data of each sub-region and the three-dimensional point cloud data of the monitoring area associated with each sub-region, a regional association data set is established. This allows the monitoring vehicle system to determine the target area and the associated monitoring area based on the work location information in the work information and the regional association data set. The monitoring vehicle system can determine the corresponding target area and monitoring area based on the work location information and move to the monitoring area during the work period to monitor the moving object, thus achieving unmanned automatic monitoring of the target area.

[0067] Example 2

[0068] Figure 3 This is a flowchart of a boundary crossing alarm method provided in Embodiment 2 of the present invention. Based on the foregoing embodiments, the determination of whether the current moving object has crossed the boundary can be further refined based on the feature image information, the pre-set operation information, the current three-dimensional point cloud data of the current moving object, and the regional three-dimensional point cloud data corresponding to the target area. For the specific implementation method, please refer to the detailed description of the embodiments of the present invention. The technical terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0069] like Figure 3 As shown, the method includes:

[0070] S210. Obtain the job information.

[0071] Specifically, the operation information is obtained through the vehicle-mounted human-machine interaction platform and displayed on the platform. On-site operators can further confirm on the platform whether the moving object is the same as the moving object in the preset operation information.

[0072] For example, the work information includes the name of worker A, a facial feature image of worker A, a license plate feature image of work vehicle A, and the license plate number of work vehicle A. By displaying the work information through a human-machine interaction platform, the worker can further verify whether the on-site situation matches the work information. If they match, the worker clicks the confirmation button; if they do not match, the worker enters the actual work information into the human-machine interaction platform and sends it to the vehicle information processor for correction.

[0073] S220. Determine at least one of the moving objects based on the job information.

[0074] Specifically, the vehicle information processor can identify at least one moving object by using the feature information of the moving object in the job information. Since the job information may contain one or more moving objects, at least one moving object can be identified based on the job information.

[0075] For example, the feature information of the moving objects in the work information includes the name of worker A, the facial feature image of worker A, the license plate feature image of work vehicle A, and the license plate number of work vehicle A. Therefore, the moving objects can be identified as worker A and work vehicle A.

[0076] S230. Based on the work location information in the work information and the regional association data set, determine the target area and the monitoring area associated with the target area.

[0077] Specifically, the onboard data processor traverses the regional association data set to find the target area containing the work location information and the monitoring area associated with the target area. This allows the monitoring vehicle system to move to the monitoring area.

[0078] S240: Receive feature image information corresponding to at least one moving object within the target area.

[0079] The moving objects include at least one of the following: workers, vehicles, and tools; the tools refer to the auxiliary equipment used by the workers during their work.

[0080] S250. Based on the feature matching result between the feature image information and the feature information of the moving object in the job information, determine whether the current moving object has crossed the boundary.

[0081] The feature matching result can be the similarity between the feature image information and the corresponding feature information of the moving object in the task information. For example, if the feature matching result is greater than a certain threshold, it means the match is successful and the current moving object has not crossed the boundary. Conversely, if the feature matching result is less than the threshold, the match fails and it is determined that the current moving object has crossed the boundary.

[0082] S260. If so, send a warning notification.

[0083] The technical solution of this invention involves acquiring the operation information; determining at least one moving object based on the operation information; determining the target area and the monitoring area associated with the target area based on the operation location information in the operation information and the regional association data set; receiving feature image information corresponding to at least one moving object within the target area; determining whether the current moving object has crossed the boundary based on the feature matching result between the feature image information and the feature information of the moving object in the operation information; and matching the feature information image of the current moving object with the feature information of the moving object in the operation information to determine whether the current moving object has crossed the boundary. This achieves real-time monitoring of moving objects, improves operation safety, and further reduces monitoring costs.

[0084] Example 3

[0085] Figure 4 This is a flowchart of a boundary crossing alarm method provided in Embodiment 3 of the present invention. Based on the foregoing embodiments, the determination of whether the current moving object has crossed the boundary can be further refined based on the feature image information, the pre-set operation information, the current three-dimensional point cloud data of the current moving object, and the regional three-dimensional point cloud data corresponding to the target area. For the specific implementation method, please refer to the detailed description of the embodiments of the present invention. Technical terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0086] like Figure 4 As shown, the method includes:

[0087] S310. Receive the location information of the moving object and the distance information between the moving object and the camera device uploaded by the camera device deployed on the monitoring vehicle system.

[0088] S320. For each moving object, determine the current three-dimensional point cloud data of the moving object within the target area based on the distance information of the current moving object and the pose information of the camera device.

[0089] The pose information includes the shooting position information and shooting orientation information of the camera device.

[0090] S330. Determine the distance information corresponding to the nearby energized equipment based on the status information of the nearby energized equipment.

[0091] Among them, distance information refers to the minimum distance between a moving object and a nearby live equipment.

[0092] Specifically, when the status information of the nearby energized equipment is energized, the corresponding distance limit can be preset to 2m; when the status information of the nearby energized equipment is de-energized, the corresponding distance limit can be 0m.

[0093] S340. Based on the regional three-dimensional point cloud data corresponding to the target area and the distance information, determine the spatial three-dimensional point cloud data corresponding to the target activity space of the moving object.

[0094] The target activity space refers to the space obtained by updating the target area based on the status information of different nearby energized devices. The spatial 3D point cloud data refers to the 3D point cloud data corresponding to each location within the target activity space.

[0095] Specifically, the on-board information processor deletes the point cloud data of the target area corresponding to the regional three-dimensional point cloud data that are less than or equal to the distance limit information between the target area and the adjacent electrical equipment, thereby obtaining the spatial three-dimensional point cloud data corresponding to the target activity space.

[0096] S350. Determine whether the current moving object has crossed the boundary based on the current three-dimensional point cloud data of the current moving object and the spatial three-dimensional point cloud data corresponding to the target activity space.

[0097] Specifically, the onboard information processor searches within the spatial 3D point cloud data corresponding to the target activity space. If the current 3D point cloud data of the moving object can be found in the spatial 3D point cloud data corresponding to the target activity space, it means that the moving object is within the target activity space. Conversely, if the current 3D point cloud data of the moving object cannot be found in the spatial 3D point cloud data corresponding to the target activity space, it means that the moving object has crossed the boundary.

[0098] S360, if yes, then send a warning notification.

[0099] Optionally, in this embodiment, S360 may include providing early warning based on the alarm device on the monitoring vehicle system; and capturing the video frame corresponding to the early warning information when it is issued by the camera device and uploading it to the background for display.

[0100] Among them, a video frame refers to a photograph taken by a camera device when a moving object crosses the boundary.

[0101] Specifically, if it is determined that the moving object has crossed the boundary, an alarm can be sounded by the alarm device on the monitoring vehicle system, or a warning light can be flashed by controlling the alarm device. Video frames of the moving object are captured by a camera and uploaded to the backend for display.

[0102] The technical solution of this invention determines the distance limit information corresponding to the nearby energized equipment based on the status information of the nearby energized equipment; determines the spatial three-dimensional point cloud data corresponding to the target activity space of the moving object based on the regional three-dimensional point cloud data corresponding to the target area and the distance limit information; determines whether the current moving object has crossed the boundary based on the current three-dimensional point cloud data of the moving object and the spatial three-dimensional point cloud data corresponding to the target activity space; if so, a warning is issued based on the alarm device on the monitoring vehicle system; and the video frame corresponding to the issuance of the warning information is captured by the camera device and uploaded to the background for display. By determining the target activity space under different status information according to the status information of the nearby energized equipment, and by judging whether the current three-dimensional point cloud data of the moving object is in the spatial three-dimensional point cloud data corresponding to the target activity space, the accuracy of boundary crossing judgment is improved. Furthermore, by using different alarm methods, timely warnings are issued for boundary crossing behavior, and users can identify the moving object that has crossed the boundary.

[0103] Example 4

[0104] Figure 5 This is a schematic diagram of a boundary crossing alarm device provided in Embodiment 4 of the present invention.

[0105] like Figure 5 As shown, the device includes:

[0106] The data receiving module 410 is used to receive feature image information corresponding to at least one moving object within the target area, and to receive location information of the moving object and distance information between the moving object and the camera device uploaded by the camera device deployed on the monitoring vehicle system; wherein, the moving object includes at least one of the following: workers, work vehicles, and work tools; the work tools refer to auxiliary tools used by the workers during the work process; the point cloud data determination module 420 is used to determine, for each moving object, the location of the current moving object within the target area based on the distance information of the current moving object and the pose information of the camera device. The current 3D point cloud data within the target area includes pose information including the shooting position information and shooting orientation information of the camera device; the boundary judgment module 430 is used to determine whether the current moving object has crossed the boundary based on the feature image information, pre-set operation information, the current 3D point cloud data of the current moving object, and the regional 3D point cloud data corresponding to the target area; wherein the operation information includes at least one of the characteristic information of the operator, operation time, operation location information, status information of nearby electrical equipment, and characteristic information of the operation vehicle; the warning prompt sending module 440 is used to send a warning prompt if the boundary is crossed.

[0107] Based on the above technical solutions, the boundary crossing alarm device also includes:

[0108] The associated data establishment module is used to perform LiDAR scanning on each device and the environment within the work area where the target area is located to obtain three-dimensional point cloud data corresponding to each device; based on the three-dimensional point cloud data, a three-dimensional spatial model corresponding to the work area is determined; the work sub-areas are divided according to the device spacing of each device in the three-dimensional spatial model, and the monitoring areas associated with each work sub-area are determined; based on the three-dimensional point cloud data of each work sub-area and the three-dimensional point cloud data of the monitoring areas associated with each work sub-area, a regional association data set is established, so that the monitoring vehicle system can determine the target area and the monitoring areas associated with the target area based on the work location information in the work information and the regional association data set.

[0109] Based on the above technical solutions, the boundary crossing alarm device also includes:

[0110] The area determination module is used to acquire the operation information; determine at least one moving object based on the operation information; and determine the target area and the monitoring area associated with the target area based on the operation location information in the operation information and the area association data set.

[0111] Based on the above technical solutions, the feature image information is obtained by a capture camera deployed on the monitoring vehicle system.

[0112] Based on the above technical solutions, the method further includes installing a locator and a multi-axis sensor on the camera device to determine the pose information of the camera device based on the locator and the multi-axis sensor.

[0113] Based on the above technical solutions, the boundary judgment module can be used for:

[0114] Based on the feature matching result between the feature image information and the feature information of the moving object in the job information, it is determined whether the current moving object has crossed the boundary.

[0115] Based on the above technical solutions, the boundary crossing detection module can also be used for:

[0116] Based on the status information of the nearby energized equipment, the distance limit information corresponding to the nearby energized equipment is determined; based on the regional three-dimensional point cloud data corresponding to the target area and the distance limit information, the spatial three-dimensional point cloud data corresponding to the target activity space of the moving object is determined; based on the current three-dimensional point cloud data of the current moving object and the spatial three-dimensional point cloud data corresponding to the target activity space, it is determined whether the current moving object has crossed the boundary.

[0117] Based on the above technical solutions, the early warning notification sending module can also be used for:

[0118] The system provides early warnings based on the alarm devices on the monitored vehicle system; it also captures the video frames corresponding to the early warning information when the camera device issues the warning and uploads them to the backend for display.

[0119] The technical solution of this invention receives feature image information corresponding to at least one moving object within a target area, and receives location information of the moving object and distance information between the moving object and the camera device uploaded by a camera device deployed on a monitoring vehicle system. For each moving object, based on the distance information of the current moving object and the pose information of the camera device, the current three-dimensional point cloud data of the current moving object within the target area is determined. Based on the feature image information, pre-set operation information, the current three-dimensional point cloud data of the current moving object, and the corresponding three-dimensional point cloud data of the target area, it is determined whether the current moving object has crossed the boundary; if so, a warning is sent. This solves the problems of high cost, large space occupation, and inconvenience for operation due to the deployment of a large number of devices on site, as well as the problems of false alarms or missed alarms due to camera shooting angle and accuracy issues. It achieves real-time monitoring of moving objects, timely warnings for boundary crossing behavior, improves operational safety, and reduces monitoring costs.

[0120] The boundary crossing alarm device provided in this embodiment of the invention can execute the boundary crossing alarm method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0121] Example 5

[0122] Figure 6 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0123] like Figure 6As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0124] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0125] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the out-of-bounds alarm method.

[0126] In some embodiments, the boundary crossing alarm method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the boundary crossing alarm method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the boundary crossing alarm method by any other suitable means (e.g., by means of firmware).

[0127] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0128] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0129] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0130] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0131] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0132] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0133] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0134] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method of out-of-boundary alarm, characterized by, include: The system receives feature image information corresponding to at least one moving object within a target area, and also receives location information of the moving object and distance information between the moving object and the camera device uploaded by a camera device deployed on the monitoring vehicle system. The moving object includes at least one of a worker, a vehicle, and a tool. The tool refers to auxiliary equipment used by the worker during operation. The feature image information is acquired by a capture camera deployed on the monitoring vehicle system. For each moving object, the current three-dimensional point cloud data of the moving object in the target area is determined based on the distance information of the moving object and the pose information of the camera device. The pose information includes the shooting position information and shooting orientation information of the camera device. Based on the feature image information, the pre-set operation information, the current 3D point cloud data of the currently moving object, and the regional 3D point cloud data corresponding to the target area, it is determined whether the currently moving object has crossed the boundary; wherein, the operation information includes at least one of the feature information of the moving object, operation time, operation location information, status information of nearby electrical equipment, and feature information of the operation vehicle; If so, a warning notification will be sent.

2. The method of claim 1, wherein, Before receiving feature image information corresponding to at least one moving object within the target area, and receiving location information corresponding to the moving object and distance information between the moving object and the camera device uploaded by the camera device deployed on the monitoring vehicle system, the method further includes: LiDAR scanning is performed on each device and the environment within the work area where the target area is located to obtain three-dimensional point cloud data corresponding to each device. Based on the three-dimensional point cloud data, a three-dimensional spatial model corresponding to the work area is determined; The corresponding work sub-regions are divided according to the equipment intervals of each device in the three-dimensional spatial model, and the monitoring areas associated with each work sub-region are determined. Based on the 3D point cloud data of each operation sub-region and the 3D point cloud data of the monitoring area associated with each operation sub-region, a regional association data set is established so that the monitoring vehicle system can determine the target area and the monitoring area associated with the target area based on the operation location information in the operation information and the regional association data set.

3. The method of claim 2, wherein, Before receiving feature image information corresponding to at least one moving object within the target area, and receiving location information corresponding to the moving object and distance information between the moving object and the camera device uploaded by the camera device deployed on the monitoring vehicle system, the method further includes: Obtain the job information; At least one of the moving objects is determined based on the job information; The target area and the monitoring area associated with the target area are determined based on the operation location information in the operation information and the regional association data set.

4. The method of claim 1, wherein, A locator and a multi-axis sensor are installed on the camera device to determine the pose information of the camera device based on the locator and the multi-axis sensor.

5. The method of claim 1, wherein, The step of determining whether the currently moving object has crossed the boundary based on the feature image information, pre-set job information, the current 3D point cloud data of the currently moving object, and the corresponding 3D point cloud data of the target area includes: Based on the feature matching result between the feature image information and the feature information of the moving object in the job information, it is determined whether the current moving object has crossed the boundary.

6. The method of claim 1, wherein, The step of determining whether the currently moving object has crossed the boundary based on the feature image information, pre-set job information, the current 3D point cloud data of the currently moving object, and the corresponding 3D point cloud data of the target area includes: further including: Based on the status information of the adjacent energized equipment, determine the distance information corresponding to the adjacent energized equipment; Based on the regional three-dimensional point cloud data corresponding to the target area and the distance limiting information, determine the spatial three-dimensional point cloud data corresponding to the target activity space of the moving object; Based on the current 3D point cloud data of the currently moving object and the spatial 3D point cloud data corresponding to the target activity space, it is determined whether the currently moving object has crossed the boundary.

7. The method of claim 1, wherein If so, a warning notification will be sent, including: Early warning prompts are issued based on the alarm devices on the vehicle monitoring system; The camera device captures the video frames corresponding to the issuance of early warning information and uploads them to the backend for display.

8. A device for out-of-boundary alarm, characterized by comprising: include: The data receiving module is used to receive feature image information corresponding to at least one moving object within the target area, and to receive location information corresponding to the moving object and distance information between the moving object and the camera device uploaded by the camera device deployed on the monitoring vehicle system; wherein, the moving object includes at least one of the following: workers, work vehicles, and work tools; the work tools refer to auxiliary tools used by the workers during the work process; the feature image information is acquired by the capture camera deployed on the monitoring vehicle system; The point cloud data determination module is used to determine the current three-dimensional point cloud data of each moving object within the target area based on the distance information of the moving object and the pose information of the camera device. The pose information includes the shooting position information and shooting orientation information of the camera device. The boundary crossing judgment module is used to determine whether the current moving object has crossed the boundary based on the feature image information, the pre-set operation information, the current three-dimensional point cloud data of the current moving object, and the regional three-dimensional point cloud data corresponding to the target area; wherein, the operation information includes at least one of the following: the characteristic information of the operator, the operation time, the operation location information, the status information of the adjacent electrical equipment, and the characteristic information of the operation vehicle; The warning notification sending module is used to send a warning notification if the condition is met.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the out-of-bounds alarm method according to any one of claims 1-7.

Images