Point cloud three-dimensional scene video monitoring and shooting method, system and device and storage medium

Abstract

The embodiment of the invention provides a point cloud three-dimensional scene video monitoring and shooting method, system and device and a storage medium, and relates to the technical field of power transmission equipment monitoring, and the method comprises the steps: obtaining a target object selected by a user from a pre-established three-dimensional point cloud image; obtaining a three-dimensional coordinate and a visualization range of the target object based on the three-dimensional point cloud image; calculating a rotation angle of the monitoring and photographing equipment based on the three-dimensional coordinate of the target object; based on the visual range of the target object, the zoom multiple of the monitoring and photographing equipment is calculated; and controlling the monitoring equipment to shoot the target object based on the rotation angle and the zoom multiple so as to obtain a monitoring video corresponding to the target object. In this way, the rotation angle and the zoom multiple of the monitoring and photographing equipment are respectively controlled through the three-dimensional coordinate and the visual range of the target object, so that the monitoring and photographing equipment rotates to a specified direction and carries out photographing according to a specified magnification multiple, and the situation that the monitoring and photographing equipment is frequently controlled to rotate to carry out position adjustment is avoided; therefore, the monitoring efficiency and accuracy of the monitoring and photographing equipment are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power transmission equipment monitoring, in particular to a point cloud three-dimensional scene video monitoring method, system, device and storage medium. BACKGROUND

[0002] With the wide application of smart grid and Internet of Things technology, the high-voltage transmission line monitoring system as a key component of power infrastructure is increasingly important. High-voltage transmission lines are often distributed in various harsh natural environments. A monitoring device is usually located on a power transmission tower to monitor the 360-degree scene around the tower body, wires, insulators, etc.

[0003] There are a large number of towers on a line around a power transmission tower, and each tower needs to observe different points. In some power transmission lines across transportation hubs or construction areas, there are external uncertain factors such as construction machinery, bridges, and passing vehicles, which need to be set by up, down, left, and right control of the holder to set the preset position to monitor and shoot the scene around the power transmission tower.

[0004] However, power transmission lines are usually distributed in remote areas with complex environments, and signals are often unstable. Adjusting the holder position point by point through up, down, left, and right control of the holder not only has low operation efficiency and easy video transmission lag, but also is difficult to ensure the accuracy of manual adjustment to the preset position, resulting in low monitoring efficiency and accuracy of the holder. SUMMARY

[0005] Therefore, the present application provides a point cloud three-dimensional scene video monitoring method, system, device and storage medium. The three-dimensional coordinates and visual range of the selected frame or selected point are calculated to obtain the angle and magnification required for the monitoring device to rotate, so that the monitoring device after obtaining the angle and magnification can accurately rotate to the specified direction and perform specified magnification, avoiding frequent preset position adjustment by up, down, left, and right control of the holder, thereby improving the monitoring efficiency and accuracy of the holder.

[0006] To achieve the above purpose, in a first aspect, the present application provides a point cloud three-dimensional scene video monitoring method, which comprises: obtaining a target object selected by a user in a pre-established three-dimensional point cloud image; the three-dimensional point cloud image is established according to three-dimensional point cloud data of a monitoring device; obtaining three-dimensional coordinates and visual range of the target object based on the three-dimensional point cloud image; calculating the rotation angle of the monitoring device based on the three-dimensional coordinates of the target object; calculating the zoom factor of the monitoring device based on the visual range of the target object; controlling the monitoring device to shoot the target object based on the rotation angle and the zoom factor to obtain the monitoring video corresponding to the target object.

[0007] In the embodiment, the target object selected by the user in the three-dimensional point cloud image is acquired, so as to calculate the rotation angle and the zoom factor of the monitoring and shooting device according to the three-dimensional coordinates and the visual range of the target object in the three-dimensional point cloud image, so as to control the monitoring and shooting device to shoot the target object through the rotation angle and the zoom factor, and obtain the monitoring and shooting video corresponding to the target object. In this way, the rotation angle and the zoom factor of the monitoring and shooting device are controlled according to the three-dimensional coordinates and the visual range of the target object, so that the monitoring and shooting device can rotate to a specified direction and shoot at a specified magnification, avoiding frequent position adjustment by rotating the monitoring and shooting device up, down, left and right, thereby improving the monitoring efficiency and accuracy of the monitoring and shooting device.

[0008] In some embodiments, the target object includes a selection box, and the three-dimensional coordinates and the visual range of the target object are acquired based on the three-dimensional point cloud image, including: acquiring the three-dimensional coordinates of the top-left corner vertex and the three-dimensional coordinates of the bottom-right corner vertex of the selection box based on the three-dimensional point cloud image; calculating the three-dimensional coordinates of the center point between the top-left corner vertex and the bottom-right corner vertex based on the three-dimensional coordinates of the top-left corner vertex and the three-dimensional coordinates of the bottom-right corner vertex, and taking the three-dimensional coordinates of the center point as the three-dimensional coordinates of the selection box; acquiring the size range of the selection box based on the three-dimensional point cloud image; and obtaining the visual range of the selection box according to the size range of the selection box.

[0009] In this way, the three-dimensional coordinates and the visual range of the selection box are acquired, and then the monitoring and shooting device is accurately controlled to shoot according to the three-dimensional coordinates and the visual range of the selection box.

[0010] In some embodiments, the rotation angle of the monitoring and shooting device includes a horizontal angle and a pitch angle, and the rotation angle of the monitoring and shooting device is calculated based on the three-dimensional coordinates of the target object, including: calculating the horizontal angle based on the horizontal coordinate and the vertical coordinate of the center point; calculating a first target distance from the center point to the monitoring and shooting device based on the horizontal coordinate, the vertical coordinate and the vertical coordinate of the center point; and calculating the pitch angle based on the vertical coordinate of the center point and the first target distance.

[0011] In this way, the horizontal angle and the pitch angle for controlling the action of the monitoring and shooting device can be calculated based on the three-dimensional coordinates of the center point of the selection box, so as to control the monitoring and shooting device to rotate to a specified position to shoot the picture corresponding to the selection box according to the parameters of the horizontal angle and the pitch angle.

[0012] In some embodiments, the zoom factor of the monitoring and shooting device is calculated based on the visual range of the target object, including: acquiring a first field of view angle of the selection box in the three-dimensional point cloud image according to the visual range of the selection box; and calculating the zoom factor of the monitoring and shooting device according to the first field of view angle.

[0013] In this way, the zooming multiple of the monitoring device can be calculated according to the visual range of the selection frame, so that the monitoring device is controlled to zoom to a specified multiple according to the parameter of the zooming multiple, and the picture corresponding to the selection frame is shot.

[0014] In some embodiments, the target object includes a selection point, and the obtaining of the three-dimensional coordinates and the visual range of the target object based on the three-dimensional point cloud image comprises: obtaining the three-dimensional coordinates of the selection point based on the three-dimensional point cloud image; obtaining the image frame size in which the selection point is located based on the three-dimensional point cloud image; and obtaining the visual range of the selection point based on the image frame size in which the selection point is located.

[0015] In this way, the three-dimensional coordinates and the visual range of the selection point are obtained, and then the monitoring device is controlled to shoot according to the three-dimensional coordinates and the visual range of the selection point.

[0016] In some embodiments, the rotation angle of the monitoring device includes a horizontal angle and a pitch angle, and the calculation of the rotation angle of the monitoring device based on the three-dimensional coordinates of the target object comprises: calculating the horizontal angle based on the horizontal coordinate and the vertical coordinate of the selection point; calculating a second target distance from the selection point to the monitoring device based on the horizontal coordinate, the vertical coordinate and the vertical coordinate of the selection point; and calculating the pitch angle based on the vertical coordinate of the selection point and the second target distance.

[0017] In this way, the horizontal angle and the pitch angle of the monitoring device can be calculated according to the three-dimensional coordinates of the selection point, so that the monitoring device is controlled to rotate to a specified position according to the parameters of the horizontal angle and the pitch angle, and the picture corresponding to the selection point is shot.

[0018] In some embodiments, the calculation of the zooming multiple of the monitoring device based on the visual range of the target object comprises: obtaining a second field of view angle of the selection point in the three-dimensional point cloud image according to the visual range of the selection point; and calculating the zooming multiple of the monitoring device according to the second field of view angle.

[0019] In this way, the zooming multiple of the monitoring device can be calculated according to the visual range of the selection frame, so that the monitoring device is controlled to zoom to a specified multiple according to the parameter of the zooming multiple, and the picture corresponding to the selection frame is shot.

[0020] Secondly, embodiments of the present invention provide a point cloud 3D scene video monitoring system, the system comprising: an acquisition module, configured to acquire a target object selected by a user in a pre-established 3D point cloud image; the 3D point cloud image is established based on 3D point cloud data of a monitoring device; the 3D coordinates and visualization range of the target object are acquired based on the 3D point cloud image; a processing module, configured to calculate the rotation angle of the monitoring device based on the 3D coordinates of the target object; and calculate the zoom factor of the monitoring device based on the visualization range of the target object; and a playback module, configured to control the monitoring device to capture images of the target object based on the rotation angle and the zoom factor, so as to obtain a monitoring video corresponding to the target object.

[0021] Thirdly, embodiments of the present invention provide an electronic device, including a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the point cloud 3D scene video monitoring method as described in the first aspect.

[0022] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the point cloud 3D scene video monitoring method as described in the first aspect.

[0023] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the structural composition of the monitoring device provided in an embodiment of the present invention; Figure 2 A flowchart of a point cloud 3D scene video monitoring method provided in an embodiment of the present invention; Figure 3 for Figure 2 Flowchart of sub-steps S201~S204 of step S200; Figure 4 A schematic diagram of the three-dimensional coordinates of the center point provided in an embodiment of the present invention; Figure 5 for Figure 2 Flowchart of sub-steps S205~S207 of step S200; Figure 6 A function module schematic diagram of the point cloud three-dimensional scene video monitoring system provided by the embodiment of the present application is shown in the figure. Figure 7 A block schematic diagram of the electronic device 2000 provided by the embodiment of the present application is shown in the figure.

[0026] Icon: 1000-point cloud three-dimensional scene video monitoring system; 1100-acquisition module; 1200-processing module; 1300-playback module; 2000-electronic device; 2100-processor; 2200-memory; 2300-bus; 2400-communication interface. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0028] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0029] It should be noted that the relational terms such as "first" and "second" and the like are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0030] Some concepts that may be involved in the embodiments of the present application are briefly introduced below.

[0031] The existing power transmission monitoring field device has the capabilities of video push stream, pan-tilt rotation, video zoom and automatic focusing, and can preset some points that need to be focused on to quickly control the pan-tilt to the target point. After the device is connected to the server, a management platform is usually used to manage all devices, and the platform has functions of real-time video pull stream, pan-tilt control, device data viewing, etc. Each device corresponds to a tower, and the points of interest of each tower are different. Different preset positions are usually set for the devices of each tower to facilitate subsequent rapid observation of a point.

[0032] As described in the background, power transmission lines are usually distributed in remote areas with complex environment and unstable signals. Adjusting the pan-tilt position point by point through up, down, left and right control of the pan-tilt not only has low operation efficiency and easy video transmission lag, but also is difficult to ensure the accuracy of manual adjustment to the preset position, thereby resulting in low monitoring efficiency and accuracy of the pan-tilt.

[0033] Therefore, on the hardware level, reference is made to Figure 1 , Figure 1 The monitoring device structure provided by the embodiment of the present application provides a point cloud three-dimensional scene video monitoring device, which comprises a remote management platform and a monitoring device, and automatically adjusts the shooting position and shooting picture of the monitoring device through the operation of the remote management platform. The remote management platform is mainly used for managing all connected devices, including but not limited to displaying a three-dimensional point cloud image near the tower where the monitoring device is located, watching real-time video, controlling the pan-tilt, setting the preset position, obtaining device working state data, image capturing, and alarm prompting. The monitoring device is distributed in remote areas and connected to the remote management platform through signal transmission, so that the user can remotely control the monitoring device on the remote management platform, adjust the monitoring position and monitoring picture of the monitoring device, and play the video corresponding to the monitoring picture.

[0034] In some embodiments, the monitoring device further comprises a power supply unit, a main control unit, a pan-tilt motor and a network camera. The main control unit is used to receive the control instruction of the remote management platform, execute the corresponding operation according to the instruction issued by the remote management platform, and make a reply. Each monitoring device contains an independent 4G network and a power supply unit. The power supply unit contains a rechargeable lithium iron phosphate battery and a solar panel. The power supply unit provides power for the main control unit, the pan-tilt motor and the network camera to ensure the independent operation of the monitoring device. The main control unit communicates with the pan-tilt motor through the network to control the rotation of the pan-tilt motor and adjust the angle and direction of the network camera. The pan-tilt motor is divided into two groups of cameras with horizontal 0-360 degrees and pitch ±90 degrees to realize panoramic 360-degree video. When the network camera is installed, the horizontal 0° and the pitch 0° are aligned with the power transmission line, which is the initial direction of the network camera.

[0035] In some embodiments, when the monitoring device is powered on and initialized, the remote management platform is accessed actively, and the basic information of the monitoring device such as power information, working voltage, working current, and working temperature is uploaded, and the zoom range and field of view angle of the network camera are uploaded, which are used to match the maximum magnification of the point cloud image and the range limit of the framed region.

[0036] In the present embodiment, the remote management platform is connected with multiple monitoring devices. Each device corresponds to a point cloud data, and the server reads the point cloud data corresponding to the monitoring device from the storage, and at the same time, taking the installation position of the current monitoring device as the rotation center of the point cloud, the initial orientation of the network camera is defined as the positive direction of the x-axis of the point cloud, the left direction as the positive direction of the y-axis, and the vertical upward direction as the positive direction of the z-axis. Then, according to the zoom magnification of the network camera, the maximum magnification of the point cloud image is limited, and according to the field of view angle of the network camera, the visualization range of the point cloud image is limited, and finally, based on the zoom magnification and the field of view angle of the network camera, a three-dimensional point cloud image adapted to the monitoring device is generated. In order to drag the three-dimensional point cloud image, the 360-degree point cloud image around the monitoring device can be viewed with the monitoring device as the center. Therefore, at the software level, the present embodiment provides a point cloud three-dimensional scene video monitoring method applied to a remote management platform, as shown in Figure 2 , Figure 2 A point cloud three-dimensional scene video monitoring method flowchart provided by the present embodiment controls the rotation angle and zoom magnification of the monitoring device through the three-dimensional coordinates and visualization range of the target object, so that the monitoring device can be rotated to a specified direction and photographed at a specified magnification, avoiding frequent position adjustment by rotating the monitoring device up, down, left, and right, thereby improving the monitoring efficiency and accuracy of the monitoring device. The method comprises steps S100-S500: S100, obtaining a target object selected by a user in a pre-established three-dimensional point cloud image; the three-dimensional point cloud image is established according to the three-dimensional point cloud data of the monitoring device.

[0037] In the present embodiment, the three-dimensional point cloud image is defined as a set of three-dimensional space points generated by laser radar scanning or image reconstruction technology, with the monitoring device as the origin, for representing the three-dimensional structure of the environment around the device. After receiving the initialization information of the monitoring device, the remote management platform retrieves the point cloud data corresponding to the monitoring device from the server, and establishes a three-dimensional coordinate system with the installation position of the monitoring device as the origin, wherein the x-axis points to the initial orientation of the monitoring device (straight ahead), the y-axis points to the left side, and the z-axis points vertically upward. The remote management platform renders the point cloud data into a visualized three-dimensional point cloud image for user interaction on the interface, so that the user can select the target object by clicking or dragging the mouse.

[0038] S200, obtaining the three-dimensional coordinates and visualization range of the target object based on the three-dimensional point cloud image.

[0039] In the embodiment, the three-dimensional point cloud image is established based on the rendering of the monitoring device as the center, so that the three-dimensional coordinates of the target object can be obtained through the point cloud data corresponding to the monitoring device in the three-dimensional point cloud image; and the visualization range of the target object is obtained by the remote management platform based on the image range occupied by the target object in the three-dimensional point cloud image, the visualization range being determined by the image scaling ratio and the field of view angle range, which is used for subsequent calculation of the zoom multiple of the network camera. It should be noted that the scaling state of the three-dimensional point cloud image has a one-to-one correspondence with the field of view angle, and the remote management platform can determine the corresponding field of view angle value according to the current scaling state, thereby providing input data for zoom calculation, which is helpful to realize the linkage between the three-dimensional point cloud image and the network camera.

[0040] S300, calculating the rotation angle of the monitoring device based on the three-dimensional coordinates of the target object.

[0041] In the embodiment, after obtaining the three-dimensional coordinates of the target object through the three-dimensional point cloud image, the rotation angle of the monitoring device is calculated through the horizontal coordinate, the vertical coordinate, the vertical coordinate of the target object and the trigonometric function operation, so as to accurately control the rotation angle of the monitoring device.

[0042] S400, calculating the zoom multiple of the monitoring device based on the visualization range of the target object.

[0043] In the embodiment, the field of view FOV (Field Of View) of the target object in the three-dimensional point cloud image can be obtained according to the visualization range of the target object. The field of view angle represents the range of the scene that can be shot by the camera at present, which is usually represented by the angle of horizontal, vertical or diagonal direction. Further, based on the relationship between the field of view angle and the network camera parameters (including sensor size, default focal length), the zoom multiple to be used by the monitoring device is calculated. Specifically, the field of view angle and the focal length are inversely proportional, and the focal length and the zoom multiple are proportional, so the current focal length f can be derived through the formula FOV = 2 × arctan(d / (2 × f)), and then the zoom multiple = f / f0 is calculated in combination with the default focal length f0. It should be noted that if the given field of view angle exceeds the support range of the network camera, the network camera will automatically adopt the maximum or minimum field of view angle for adjustment to ensure the feasibility of zoom operation. In this way, the field of view matching between the three-dimensional point cloud image and the actual shooting picture is realized, and the consistency of the monitoring picture and the target region is improved.

[0044] S500, controlling the monitoring device to shoot the target object based on the rotation angle and the zoom multiple, to obtain the monitoring video corresponding to the target object.

[0045] In the embodiment, the calculated horizontal angle, pitch angle and zoom factor are sent to the monitoring device through the network. The gimbal motor of the monitoring device rotates to the specified angle according to the instruction, the network camera starts shooting after adjusting to the specified zoom factor, and pushes the video data to the streaming media server. Then the video picture is superimposed on the corresponding area of the three-dimensional point cloud image to form a fusion image picture for the user to view. Further, when the video playback is over or timeout, the video streaming is stopped and the operability of the three-dimensional point cloud image is restored. Thus, the conversion from the virtual three-dimensional point cloud image to the real video picture is completed, the visual monitoring function is realized, and the operation intuitiveness and monitoring efficiency are improved.

[0046] In some embodiments, referring to Figure 3 , Figure 3 for Figure 2 the flowchart of steps S201-S204 of step S200, the target object includes a selection frame, and step S200 can include steps S201-S204: S201, obtaining the three-dimensional coordinates of the top-left corner point and the three-dimensional coordinates of the bottom-right corner point of the selection frame based on the three-dimensional point cloud image.

[0047] In the embodiment, the user frames the target area in the three-dimensional point cloud image interface of the remote management platform by the mouse drag method, and the selection frame appears as a rectangular area on the image interface. Then the top-left corner point coordinate P1(x1, y1, z1) and the bottom-right corner point coordinate P2(x2, y2, z2) of the rectangle are recorded according to the user's operation behavior. It should be noted that each point in the three-dimensional point cloud image corresponds to a position in the three-dimensional space, so the two-dimensional positions of the top-left corner and the bottom-right corner of the framed area on the image can be mapped to the specific coordinate points in the three-dimensional space.

[0048] S202, calculating the three-dimensional coordinates of the center point between the top-left corner point and the bottom-right corner point based on the three-dimensional coordinates of the top-left corner point and the three-dimensional coordinates of the bottom-right corner point, and taking the three-dimensional coordinates of the center point as the three-dimensional coordinates of the selection frame.

[0049] In the embodiment, referring to Figure 4 , Figure 4 for the three-dimensional coordinate diagram of the center point provided by the embodiment of the application, the center point coordinate P3(x3, y3, z3) between the top-left corner point coordinate P1(x1, y1, z1) and the bottom-right corner point coordinate P2(x2, y2, z2) is calculated, and the calculation formula is: P3=( , , ) = (x3, y3, z3). Further, the center point coordinate P3 is taken as the representative coordinate of the selection frame, which is used for subsequent calculation of the rotation angle of the PTZ of the monitoring device. Specifically, when calculating the horizontal angle and the pitch angle of the monitoring device, the center point is taken as the target position for angle derivation, so as to ensure that the camera of the network camera is aligned with the center of the selected region of the three-dimensional point cloud image after the PTZ is rotated. It should be noted that the center point of the selection frame is representative and can effectively reflect the spatial position of the target region, thereby avoiding control errors caused by deviation of the edge points of the selection frame.

[0050] S203, obtaining the size range of the selection frame based on the three-dimensional point cloud image.

[0051] In this embodiment, the size range of the selection frame is obtained according to the image size of the selection frame selected by the user in the three-dimensional point cloud image. The size range is determined by the width and height of the selection frame on the image, and is usually expressed in pixel units or image scaling ratio. Further, the size range is related to the magnification of the three-dimensional point cloud image. The higher the image magnification, the more detailed the point cloud details covered by the selection frame, and the smaller the corresponding visual range. Conversely, the lower the image magnification, the wider the coverage range of the selection frame, and the larger the visual range. It should be noted that the size range of the selection frame is not only used for determining the visual range, but also used for calculating the field of view angle and the zoom ratio to be used by the network camera, so as to ensure that the captured image matches the display region of the three-dimensional point cloud image.

[0052] S204, obtaining the visual range of the selection frame according to the size range of the selection frame.

[0053] In this embodiment, the visual range of the selection frame in the three-dimensional point cloud image is obtained according to the size range of the selection frame. The visual range represents the spatial region that should be covered by the monitoring device when capturing the target object. Specifically, according to the mapping relationship between the scaling state of the current point cloud image and the field of view angle, the field of view angle corresponding to the selection frame is determined, which is used to control the zoom ratio of the network camera. Further, if the selection frame occupies a larger proportion in the point cloud image, the corresponding field of view angle is wider, and the zoom ratio is lower. Conversely, if the selection frame is smaller, the field of view angle is narrower, and the zoom ratio is higher. It should be noted that the zoom parameter to be issued can be automatically adjusted according to the range of the field of view angle supported by the network camera, so as to ensure the feasibility and rationality of the zoom operation.

[0054] In some embodiments, in the case where the target object is the selection frame, the rotation angle of the monitoring device includes a horizontal angle and a pitch angle, which can be calculated based on the three-dimensional coordinates of the selection frame.

[0055] Exemplarily, the horizontal angle is calculated based on the horizontal coordinate and the vertical coordinate of the center point; the first target distance from the center point to the monitoring device is calculated based on the horizontal coordinate, the vertical coordinate and the vertical coordinate of the center point; and the pitch angle is calculated based on the vertical coordinate of the center point and the first target distance.

[0056] For example, the horizontal angle is calculated based on the horizontal coordinate x3 and the vertical coordinate y3 in the center point coordinate P3(x3, y3, z3) , and the calculation formula is: atan2(y3, x3). If the calculation result is negative, 360° is added to ensure that the angle is within the range of [0°, 360°]. The horizontal angle represents the angle of rotation of the gimbal left and right, and the positive direction is from the x-axis counterclockwise to the y-axis direction. It can be understood that the rotation direction corresponds to the direction of the positive half of the x-axis to the positive half of the y-axis, then to the negative half of the x-axis, then to the negative half of the y-axis, and then to the positive half of the x-axis (counterclockwise direction). Further, the first target distance r from the center point to the monitoring device is calculated based on the three coordinates x3, y3 and z3 of the center point, and the calculation formula is: sqrt(x3*x3 + y3*y3 + z3*z3). The distance r represents the distance of the center point in the three-dimensional space relative to the origin of the monitoring device, which is used for subsequent calculation of the pitch angle. Next, the pitch angle is calculated based on the z3 coordinate of the center point and the distance r , and the calculation formula is: asin(z3 / r). The pitch angle represents the angle of rotation of the gimbal up and down, and the value range is [-90°, 90°], and the positive direction is upward rotation and the negative direction is downward rotation. It can be understood that the rotation direction corresponds to the direction of the positive half of the z-axis to the positive half of the x-axis, and then to the negative half of the z-axis. The positive half of the z-axis corresponds to the +90 degree direction, and the negative half corresponds to the -90 degree direction. Thus, through three-dimensional coordinate and trigonometric function operation, accurate control of the rotation angle of the gimbal is realized, ensuring that the camera can accurately aim at the center position of the selection frame.

[0057] In some embodiments, in the case of a target object being a selection frame, the zoom ratio of the monitoring device can be calculated based on the visualizable range of the selection frame.

[0058] Exemplarily, a first field of view angle of the selection frame in the three-dimensional point cloud image is obtained according to the visualizable range of the selection frame; and the zoom ratio of the monitoring device is calculated according to the first field of view angle.

[0059] For example, the first field of view angle FOV1 of the selection frame in the three-dimensional point cloud image is obtained based on the size range of the selection frame, and the camera field of view angle (FOV) calculation formula is known: FOV = 2*arctan( ); wherein: d represents the horizontal, vertical or diagonal dimension of the camera sensor, in mm; f is the current focal length of the camera, in mm. The field of view represents the range of the scene that the camera can capture at the current time, usually represented by the angle in the horizontal direction. Further, based on the known relationship between the sensor size d, the default focal length f0 in the non-zoomed state and the field of view, the focal length f to be used at the current time is calculated according to the formula: f = d tan (f0 / 2) / tan (f / 2). . Subsequently, the zoom factor z is calculated based on the relationship between the current focal length f and the default focal length f0 according to the formula: z = f0 / f. .

[0060] In some embodiments, referring to Figure 5 , Figure 5 as Figure 2 the flowchart of steps S205-S207 of step S200 in S205, obtaining the three-dimensional coordinates of the selected point based on the three-dimensional point cloud image.

[0061] In the present embodiment, the user selects a target point on the three-dimensional point cloud image interface of the remote management platform through a single-click operation, which is represented as a specific position on the image interface. Then, the corresponding three-dimensional coordinates P(x, y, z) are obtained according to the mapping relationship of the position in the point cloud data. Further, the two-dimensional image coordinates clicked by the user are converted into three-dimensional space coordinates through the index and coordinate mapping mechanism of the point cloud data, so as to obtain the accurate spatial position of the selected point. It should be noted that the three-dimensional coordinates are the basic parameters for subsequent calculation of the pan-tilt rotation angle and the zoom factor, and the accuracy directly affects the alignment accuracy of the monitoring equipment to the target point.

[0062] S206, obtaining the image frame size where the selected point is located based on the three-dimensional point cloud image.

[0063] In the present embodiment, the image frame size of the selected point is obtained according to the image frame area where the selected point is located. The image frame size represents the frame range where the selected point is located in the current point cloud image, usually represented by the image scaling ratio or the pixel coverage range of the area around the selected point. Further, the image frame size is related to the scaling state of the point cloud image. The higher the scaling ratio, the more focused and smaller the coverage range of the frame around the selected point. The lower the scaling ratio, the wider and larger the coverage range of the frame around the selected point. The image frame size of the selected point can be calculated by recording the scaling ratio of the current point cloud image in the image rendering process and combining the relative position of the selected point in the image.

[0064] S207, obtaining the visualization range of the selected point based on the image frame size where the selected point is located.

[0065] In the embodiment, the visual range of the selected point is obtained based on the image screen size where the selected point is located, and the visual range represents a space region that should be covered by the monitoring device when shooting the target point, and is usually represented by the field of view FOV of the camera. Further, according to the mapping relationship between the image screen size and the point cloud image zoom state, the field of view value corresponding to the selected point is determined, and the field of view is used to control the zoom multiple of the camera, so as to ensure that the shooting screen is consistent with the point cloud image display region. After the field of view is sent to the monitoring device, the corresponding zoom multiple is calculated by the monitoring device according to the sensor size and the default focal length known by the monitoring device, and the screen matching is realized.

[0066] In some embodiments, in the case that the target object is the selected point, the rotation angle of the monitoring device includes a horizontal angle and a pitch angle, and the horizontal angle and the pitch angle of the monitoring device can be calculated through the three-dimensional coordinates of the selected point.

[0067] For example, the horizontal angle is calculated based on the horizontal coordinate and the vertical coordinate of the selected point; the second target distance from the selected point to the monitoring device is calculated based on the horizontal coordinate, the vertical coordinate and the vertical coordinate of the selected point; and the pitch angle is calculated based on the vertical coordinate of the selected point and the second target distance.

[0068] In the embodiment, in the case that the target object is the selected point, the way of calculating the horizontal angle and the pitch angle according to the three-dimensional coordinates of the selected point is the same as the way of calculating the horizontal angle and the pitch angle according to the three-dimensional coordinates of the center point, which has been described above and will not be repeated here.

[0069] In some embodiments, in the case that the target object is the selected point, the zoom multiple of the monitoring device can be calculated through the visual range of the selected point.

[0070] For example, the second field of view of the selected point in the three-dimensional point cloud image is obtained according to the visual range of the selected point; and the zoom multiple of the monitoring device is calculated according to the second field of view.

[0071] In the embodiment, the second field of view FOV2 of the selected point in the three-dimensional point cloud image is obtained based on the image screen size where the selected point is located, and the field of view represents the current scene range that can be shot by the camera, and is usually represented by the angle in the horizontal direction. In the case that the target object is the selected point, the image screen size is obtained according to the image screen region where the selected point is located, which is different from the image screen size in the case that the target object is the selected frame but has the same data obtaining method, so the way of calculating the zoom multiple of the monitoring device according to the second field of view FOV2 is the same as the way of calculating the zoom multiple of the monitoring device according to the first field of view FOV1, which has been described above and will not be repeated here.

[0072] Based on the above method, the embodiment of the application further provides a system corresponding to the above method, as shown in Figure 6 Figure 6 ​A function module schematic diagram of the point cloud three-dimensional scene video monitoring and shooting system 1000 provided by the embodiment of the present application is shown. It should be noted that the point cloud three-dimensional scene video monitoring and shooting system 1000 provided by the embodiment has the same basic principle and technical effects as the above-mentioned method embodiments, and for brief description, the part not mentioned in the embodiment can refer to the corresponding content in the method embodiments.

[0073] In the embodiment, the point cloud three-dimensional scene video monitoring and shooting system 1000 includes an acquisition module 1100, a processing module 1200 and a playing module 1300; the acquisition module 1100 is configured to acquire a target object selected by a user in a pre-established three-dimensional point cloud image; the three-dimensional point cloud image is established according to three-dimensional point cloud data of a monitoring and shooting device; and the three-dimensional coordinates and the visual range of the target object are acquired based on the three-dimensional point cloud image. It can be understood that the acquisition module 1100 is configured to perform the above steps S100-S200.

[0074] The processing module 1200 is configured to calculate a rotation angle of the monitoring and shooting device based on the three-dimensional coordinates of the target object; and calculate a zoom multiple of the monitoring and shooting device based on the visual range of the target object. It can be understood that the processing module 1200 is also configured to perform the above steps S300-S400.

[0075] The playing module 1300 is configured to control the monitoring and shooting device to shoot the target object based on the rotation angle and the zoom multiple, so as to obtain a monitoring and shooting video corresponding to the target object. It can be understood that the playing module 1300 is also configured to perform the above step S500.

[0076] In some embodiments, the target object includes a selection box. The acquisition module 1100 is configured to acquire the three-dimensional coordinates of the top-left corner vertex and the three-dimensional coordinates of the bottom-right corner vertex of the selection box based on the three-dimensional point cloud image; calculate the three-dimensional coordinates of the center point between the top-left corner vertex and the bottom-right corner vertex based on the three-dimensional coordinates of the top-left corner vertex and the three-dimensional coordinates of the bottom-right corner vertex, and take the three-dimensional coordinates of the center point as the three-dimensional coordinates of the selection box; and acquire the size range of the selection box based on the three-dimensional point cloud image. It can be understood that the acquisition module 1100 is also configured to perform the above steps S201-S204.

[0077] In some embodiments, the rotation angle of the monitoring and shooting device includes a horizontal angle and a pitch angle. The processing module 1200 is configured to calculate the horizontal angle based on the horizontal coordinate and the vertical coordinate of the center point; calculate a first target distance from the center point to the monitoring and shooting device based on the horizontal coordinate, the vertical coordinate and the vertical coordinate of the center point; calculate the pitch angle based on the vertical coordinate of the center point and the first target distance; and obtain the visual range of the selection box according to the size range of the selection box.

[0078] In some embodiments, when the target object comprises the selection box, the processing module 1200 is further configured to: acquire a first field of view angle of the selection box in the three-dimensional point cloud image according to the visual range of the selection box; and calculate the zoom factor of the monitoring device according to the first field of view angle.

[0079] In some embodiments, the target object comprises the selection point. The acquisition module 1100 is configured to: acquire the three-dimensional coordinates of the selection point based on the three-dimensional point cloud image; acquire the image frame size in which the selection point is located based on the three-dimensional point cloud image; and obtain the visual range of the selection point based on the image frame size in which the selection point is located. It can be understood that the acquisition module 1100 is further configured to perform steps S205-S207 described above.

[0080] In some embodiments, the rotation angle of the monitoring device comprises a horizontal angle and a pitch angle. The processing module 1200 is configured to: calculate the horizontal angle based on the horizontal coordinate and the vertical coordinate of the selection point; calculate a second target distance from the selection point to the monitoring device based on the horizontal coordinate, the vertical coordinate and the vertical coordinate of the selection point; and calculate the pitch angle based on the vertical coordinate of the selection point and the second target distance.

[0081] In some embodiments, when the target object comprises the selection point, the processing module 1200 is configured to: acquire a second field of view angle of the selection point in the three-dimensional point cloud image according to the visual range of the selection point; and calculate the zoom factor of the monitoring device according to the second field of view angle.

[0082] Based on the same inventive concept disclosed above, the embodiments of the present application also provide a block schematic diagram of an electronic device 2000 for performing the above method. Please refer to Figure 7 , Figure 7 The block schematic diagram of the electronic device 2000 provided by the embodiments of the present application comprises a processor 2100, a memory 2200, a bus 2300 and a communication interface 2400. The processor 2100 and the memory 2200 are connected through the bus 2300, and the processor 2100 communicates with external devices through the communication interface 2400.

[0083] The processor 2100 can be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by integrated logic circuits or instructions in the form of software in the processor 2100. The processor 2100 described above can be a general-purpose processor 2100, including a central processing unit 2100 (CPU), a network processor 2100 (NP), etc.; can also be a digital signal processor 2100 (DSP), an application-specific integrated circuit (ASIC), a ready-to-program gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0084] The memory 2200 is configured to store a computer program, for example, the point cloud three-dimensional scene video monitoring system 1000 in the embodiments of the present application, including at least one software function module stored in the memory 2200 in the form of software or firmware, and the processor 2100 executes the program to realize the point cloud three-dimensional scene video monitoring method in the embodiments of the present application after receiving an execution instruction.

[0085] The memory 2200 can include a high-speed random access memory 2200 (RAM: Random Access Memory), and can also include a non-volatile memory 2200. Optionally, the memory 2200 can be a storage device built in the processor 2100, or a storage device independent of the processor 2100.

[0086] The bus 2300 can be an ISA bus 2300, a PCI bus 2300, or an EISA bus 2300, etc. Figure 7 Only one bidirectional arrow is used to represent, but it does not mean that there is only one bus 2300 or one type of bus 2300.

[0087] The electronic device 2000 can be a mobile phone, a tablet computer, a notebook computer, a desktop computer, etc.

[0088] Based on the same inventive concept, the embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program is executed by the processor 2100 to realize the point cloud three-dimensional scene video monitoring method as described above. The computer readable storage medium can include a U disk, a mobile hard disk, a read-only memory 2200 (ROM), a random access memory 2200 (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0089] The above only describes the preferred embodiments of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for video monitoring of point cloud 3D scenes, characterized in that, The method includes: The system acquires the target object selected by the user in a pre-built 3D point cloud image; the 3D point cloud image is built based on the 3D point cloud data of the monitoring device. The three-dimensional coordinates and visualization range of the target object are obtained based on the three-dimensional point cloud image; The rotation angle of the monitoring device is calculated based on the three-dimensional coordinates of the target object; The zoom level of the monitoring device is calculated based on the visual range of the target object; Based on the rotation angle and the zoom level, the monitoring device is controlled to capture images of the target object to obtain a monitoring video corresponding to the target object.

2. The method according to claim 1, characterized in that, The target object includes a selection box, and obtaining the three-dimensional coordinates and visualization range of the target object based on the three-dimensional point cloud image includes: Based on the three-dimensional point cloud image, obtain the three-dimensional coordinates of the upper left corner vertex and the lower right corner vertex of the selection box; The three-dimensional coordinates of the center point between the top left and bottom right vertices are calculated based on the three-dimensional coordinates of the top left and bottom right vertices, and the three-dimensional coordinates of the center point are used as the three-dimensional coordinates of the selection box. The size range of the selection box is obtained based on the three-dimensional point cloud image; The visual range of the selection box is obtained based on the size range of the selection box.

3. The method according to claim 2, characterized in that, The rotation angle of the monitoring device includes a horizontal angle and a pitch angle, and the calculation of the rotation angle of the monitoring device based on the three-dimensional coordinates of the target object includes: Calculate the horizontal angle based on the x and y coordinates of the center point; Calculate the first target distance from the center point to the monitoring device based on the horizontal, vertical, and axial coordinates of the center point; The pitch angle is calculated based on the vertical coordinate of the center point and the distance to the first target.

4. The method according to claim 2, characterized in that, The calculation of the zoom level of the monitoring device based on the visualization range of the target object includes: The first field of view of the selection box in the three-dimensional point cloud image is obtained based on the visualization range of the selection box; The zoom level of the monitoring device is calculated based on the first field of view.

5. The method according to claim 1, characterized in that, The target object includes selected points, and obtaining the three-dimensional coordinates and visualization range of the target object based on the three-dimensional point cloud image includes: The three-dimensional coordinates of the selected point are obtained based on the three-dimensional point cloud image; The image size of the selected point is obtained based on the three-dimensional point cloud image; The visualization range of the selected point is obtained based on the size of the image frame where the selected point is located.

6. The method according to claim 5, characterized in that, The rotation angle of the monitoring device includes a horizontal angle and a pitch angle, and the calculation of the rotation angle of the monitoring device based on the three-dimensional coordinates of the target object includes: Calculate the horizontal angle based on the x and y coordinates of the selected point; The second target distance from the selected point to the monitoring device is calculated based on the x-coordinate, y-coordinate, and y-coordinate of the selected point. The pitch angle is calculated based on the vertical coordinates of the selected point and the distance to the second target.

7. The method according to claim 5, characterized in that, The calculation of the zoom level of the monitoring device based on the visualization range of the target object includes: The second field of view of the selected point in the three-dimensional point cloud image is obtained based on the visualization range of the selected point; The zoom level of the monitoring device is calculated based on the second field of view.

8. A point cloud 3D scene video monitoring system, characterized in that, The system includes: The acquisition module is used to acquire the target object selected by the user in a pre-established 3D point cloud image; the 3D point cloud image is established based on the 3D point cloud data of the monitoring device; and the 3D coordinates and visualization range of the target object are acquired based on the 3D point cloud image. The processing module is used to calculate the rotation angle of the monitoring device based on the three-dimensional coordinates of the target object; and to calculate the zoom ratio of the monitoring device based on the visualization range of the target object. The playback module is used to control the monitoring device to capture the target object based on the rotation angle and the zoom level, so as to obtain the monitoring video corresponding to the target object.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program that can be executed by the processor, the processor being able to execute the computer program to implement the point cloud 3D scene video monitoring method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the point cloud 3D scene video monitoring method as described in any one of claims 1-7.

Images