A multi-plane reconstruction method and device, and an electronic device

Abstract

The application discloses a multi-plane reconstruction method and device and electronic equipment, and the multi-plane reconstruction method comprises the following steps: acquiring height information of a preset target and a plurality of first image information of the preset target on a target plane; determining a three-dimensional coordinate set of a foot point of the preset target on the target plane according to the height information of the preset target and the plurality of first image information; and determining plane information of the target plane according to the three-dimensional coordinate set of the foot point. The application can conveniently complete multi-plane reconstruction through simple walking of a pedestrian on different planes, and does not increase additional cost, and has great landing use value.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of passenger flow statistics, and particularly relates to a multi-plane reconstruction method and device and electronic equipment. BACKGROUND

[0002] Currently, a common method for monocular passenger flow counting is as follows: a calibration board is placed under a monocular RGB camera, the internal and external parameters of the camera are calibrated, then the camera is used to track and collect a target to obtain a two-dimensional picture of the target, the three-dimensional position of the target is determined according to the foot point position of the target, and the movement trajectory of the foot point of the target in the three-dimensional space is obtained. After the trajectory is obtained, a line is usually drawn, and when the three-dimensional trajectory of the target passes through the line, the target is counted, and passenger flow counting is completed. However, this passenger flow counting method needs to assume that the ground height is consistent, and if there are steps or exhibition stands on the ground, the ground height is inconsistent, which will cause the passenger flow counting algorithm to fail. Because before passenger flow counting, it is assumed that the height of the target foot point is the same as the height of the calibration plane, and when the actual ground height is inconsistent with the height of the calibration plane, this assumption will fail, and the calculated three-dimensional trajectory of the target is wrong, and thus the passenger flow counting is also wrong. Currently, the solution to the problem that the monocular passenger flow counting algorithm fails when the ground height is inconsistent is usually to scan the entire scene by using a radar or a depth camera to reconstruct multiple planes. This method is too high in cost and cumbersome, and is not suitable for landing use. SUMMARY

[0003] Therefore, the present application provides a multi-plane reconstruction method and device and electronic equipment to solve the problem that the current multi-plane reconstruction process is complex and high in cost.

[0004] To achieve the above object, the present application adopts the following technical scheme:

[0005] The embodiment of the present application discloses a multi-plane reconstruction method, comprising: obtaining height information of a preset target and a plurality of first image information of the preset target on a target plane; determining a three-dimensional coordinate set of a foot point of the preset target on the target plane according to the height information of the preset target and the plurality of first image information; and determining plane information of the target plane according to the three-dimensional coordinate set of the foot point.

[0006] The present application obtains the height information of the preset target and the plurality of first image information of the preset target on the target plane, and determines the three-dimensional coordinate set of the foot point of the preset target on the target plane according to the height information of the preset target and the plurality of first image information, and then determines the plane information of the target plane according to the three-dimensional coordinate set of the foot point. The present application completes the multi-plane reconstruction conveniently by using the walking of pedestrians on different planes, and does not increase additional cost, and has great landing use value.

[0007] Optionally, the obtaining the height information of the preset target comprises: determining second image information of the preset target captured by the camera on a preset plane; obtaining parameter information of the camera and three-dimensional coordinates of the camera, and processing the second image information through the parameter information and the three-dimensional coordinates of the camera to determine the height information of the preset target.

[0008] The application can calculate the height of the preset target by using the parameter information of the camera, the three-dimensional coordinates of the camera and the image information of the preset target captured by the camera, and the process is simple and fast without additional cost.

[0009] Optionally, the determining the second image information of the preset target captured by the camera on the preset plane comprises: obtaining a second image of the preset target captured by the camera on the preset plane; inputting the second image into a preset image recognition model to identify head point coordinates and foot point coordinates of the preset target in the second image through the image recognition model, and taking the head point coordinates and the foot point coordinates as the determined second image information.

[0010] Optionally, the determining the height information of the preset target comprises: obtaining a first straight line by using the three-dimensional coordinates of the camera and the foot point coordinates of the preset target on the second image; obtaining plane information of the preset plane, taking three-dimensional coordinates of an intersection point of the first straight line and the preset plane as three-dimensional coordinates of a foot point of the preset target; obtaining a second straight line by using the three-dimensional coordinates of the foot point of the preset target and a straight line perpendicular to the preset plane; obtaining a third straight line by using the three-dimensional coordinates of the camera and head point coordinates of the preset target on the second image; taking three-dimensional coordinates of an intersection point of the second straight line and the third straight line as three-dimensional coordinates of a head point of the preset target; and determining the height information of the preset target according to the three-dimensional coordinates of the foot point of the preset target and the three-dimensional coordinates of the head point of the preset target.

[0011] The application obtains a first straight line by using the three-dimensional coordinates of the camera and the foot point coordinates of the preset target on the second image, then calculates an intersection point of the first straight line and the preset plane according to the obtained plane information of the preset plane to obtain the three-dimensional coordinates of the foot point of the preset target, further obtains a second straight line by using the three-dimensional coordinates of the foot point of the preset target and a straight line perpendicular to the preset plane, simultaneously obtains a third straight line by using the three-dimensional coordinates of the camera and the head point coordinates of the preset target on the second image, then calculates an intersection point of the second straight line and the third straight line to obtain the three-dimensional coordinates of the head point of the preset target, and finally calculates the height information of the preset target according to the three-dimensional coordinates of the head point of the preset target and the three-dimensional coordinates of the foot point of the preset target. The height of the preset target calculated by the above method is simple and fast without additional cost.

[0012] Optionally, the first image information comprises head point coordinates of the preset target on the first image and foot point coordinates of the preset target on the first image; and the determining of the three-dimensional coordinate set of the foot point of the preset target on the target plane according to the height information of the preset target and the plurality of first image information comprises: obtaining a fourth straight line by using the three-dimensional coordinates of the camera and the head point coordinates of the preset target on the first image; obtaining a fifth straight line by using the three-dimensional coordinates of the camera and the foot point coordinates of the preset target on the first image; searching for a line segment which intersects with the fourth straight line and the fifth straight line and matches the height information in a direction perpendicular to the horizontal plane between the fourth straight line and the fifth straight line, and taking the three-dimensional coordinates of the intersection point of the searched line segment and the fifth straight line as the foot point information of the preset target on the target plane; and traversing the plurality of first image information of the preset target to determine the foot point information of the preset target on the target plane in each of the first image information, and taking each of the foot point information as the three-dimensional coordinate set of the foot point of the preset target on the target plane.

[0013] The application determines the head point coordinates of the preset target on the first image and the foot point coordinates of the preset target on the first image according to the first image information, obtains a fourth straight line by using the three-dimensional coordinates of the camera and the head point coordinates of the preset target on the first image, further obtains a fifth straight line by using the three-dimensional coordinates of the camera and the foot point coordinates of the preset target on the first image, and determines the foot point information of the preset target on the target plane according to the fourth straight line, the fifth straight line and the height information of the preset target, and generates the three-dimensional coordinate set of the foot point of the preset target on the target plane. The application generates the three-dimensional coordinate set of the foot point of the preset target on the target plane according to the height information of the preset target and the first image information, and then facilitates the determination of the plane information by using the three-dimensional coordinate set of the foot point of the preset target on the target plane, and greatly improves the work efficiency.

[0014] Optionally, the determining of the plane information of the target plane according to the three-dimensional coordinate set of the foot point comprises: determining the plane information of the target plane according to the three-dimensional coordinate set of the foot point by using a random sample consensus algorithm.

[0015] The application determines the plane information of the target plane according to the three-dimensional coordinate set of the foot point of the preset target on the target plane by using a random sample consensus algorithm. The three-dimensional coordinate set of the foot point is obtained by performing noise reduction processing on the action trajectory of the preset target on the target plane, high-precision parameters can be estimated from the three-dimensional coordinate set containing a large number of inaccurate outliers, and more reliable plane information can be obtained.

[0016] Optionally, the multi-plane reconstruction method further comprises: determining a moving track of the to-be-identified target moving on the target plane according to the plane information of the target plane and a plurality of images of the to-be-identified target on the target plane collected by the camera; and if the moving track represents that the to-be-identified target passes through a preset straight line on the target plane, counting the to-be-identified target.

[0017] In the technical solution provided in the application, a straight line is set on the target plane, and the passenger flow is counted by judging whether the to-be-identified target passes through the straight line. When the to-be-identified target passes through the preset straight line, the to-be-identified target is counted. The technical solution provided in the application solves the problem that, in the current common situation, only when the heights of all planes are consistent, the target passing through the straight line can be counted.

[0018] The application counts the passenger flow by constructing a multi-plane space model and using a monocular passenger flow camera, avoids the problem that in the general technology, a multi-plane is reconstructed by scanning the whole scene by a radar or a depth camera, which is high in cost and complicated, and is not suitable for landing use, reduces the cost of passenger flow counting, and has high landing use value.

[0019] According to a second aspect, the application further discloses a multi-plane reconstruction method and device, comprising: an acquisition module configured to acquire height information of a preset target and a plurality of first image information of the preset target on a target plane; a three-dimensional coordinate set determination module configured to determine a three-dimensional coordinate set of a foot point of the preset target on the target plane according to the height information of the preset target and the plurality of first image information; and a plane information determination module configured to determine plane information of the target plane according to the three-dimensional coordinate set of the foot point.

[0020] According to a third aspect, the application further discloses an electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the multi-plane reconstruction method steps according to the first aspect or any optional implementation manner of the first aspect.

[0021] According to a fourth aspect, the application further discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the multi-plane reconstruction method steps according to the first aspect or any optional implementation manner of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the description of the embodiments or the prior art. Obviously, the drawings described below are some embodiments of the present application, and the other drawings can be obtained by those skilled in the art without creative effort based on these drawings.

[0023] Figure 1 A flowchart of a multi-plane reconstruction method in an embodiment of the present application;

[0024] Figure 2 A schematic diagram of a multi-plane reconstruction method in an embodiment of the present application;

[0025] Figure 3 Another schematic diagram of a multi-plane reconstruction method in an embodiment of the present application;

[0026] Figure 4 Another schematic diagram of a multi-plane reconstruction method in an embodiment of the present application;

[0027] Figure 5 A structural schematic diagram of a multi-plane reconstruction device in an embodiment of the present application;

[0028] Figure 6 A structural schematic diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solutions of the present application will be described clearly and completely below with reference to the drawings. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative effort belong to the protection scope of the present application.

[0030] In the description of the present application, it should be noted that the term "and / or" used in the present application and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0031] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as there is no conflict.

[0032] The embodiments of the present application disclose a multi-plane reconstruction method, as shown in the figure, the method comprises the following steps: Figure 1

[0033] Step 101, obtaining height information of a preset target and a plurality of first image information of the preset target on a target plane. ​

[0034] Specifically, the height information of the preset target is acquired, including: determining second image information of the preset target on a preset plane collected by a camera; acquiring parameter information of the camera and three-dimensional coordinates of the camera, and processing the second image information through the parameter information and the three-dimensional coordinates of the camera to determine the height information of the preset target.

[0035] In addition, in actual application, the height information of the preset target can also be known in advance, and the height information of the preset target can be stored in a specified path in the form of a key-value pair. By reading information in the specified path, the height information of the preset target can be acquired.

[0036] As a specific embodiment, in order to acquire accurate height more quickly, the second image of the preset target walking on the calibrated preset plane can be detected by using a target detection model, and then the second image is identified by using key point detection to obtain the second image information of the preset target on the preset plane. The target detection model can be a model capable of human body recognition. By analyzing the image captured by the camera, the image containing the human body can be recognized. Specifically, the target detection model can be a centerNet network, and the image features of the preset target are obtained by detecting the preset target. The key point detection can further identify the image containing the preset target collected by the target detection model, and then identify the key point position of the preset target from the image. In actual application, the key point detection can be realized by the following method: using a regression method to train and test image features in a deep regression model to obtain the mapping of key point positions, and then using a sparse matrix to estimate the key points not mapped on the current second image according to the mapping of the key point positions of the training set. The key point positions can be the head position and foot position of the human body. When performing key point detection, the image samples in the training set are trained in the above manner, and an image recognition model capable of recognizing the head position and foot position in the image can be obtained. Subsequently, by inputting the second image into the image recognition model, the head position and foot position in the second image can be detected.

[0037] Specifically, the second image information of the preset target on the preset plane collected by the camera is determined, specifically: the second image of the preset target on the preset plane collected by the camera is acquired; the second image is input into a preset image recognition model, so as to recognize the head point coordinates and foot point coordinates of the preset target in the second image through the image recognition model, and the head point coordinates and the foot point coordinates are taken as the determined second image information.

[0038] The second image information includes but is not limited to a head point coordinate of the preset target on the second image and a foot point coordinate of the preset target on the second image, and the like. According to the second image information, the height information of the preset target is determined, specifically, a first straight line is calculated by using the three-dimensional coordinates of the camera and the foot point coordinate of the preset target on the second image; then, the plane information of the preset plane is obtained, since the preset target is standing on the preset plane, and since the three-dimensional coordinates of the camera, the foot point coordinate of the preset target on the second image and the foot point position of the preset target in the world coordinate system are on a straight line according to the imaging principle of the camera, the intersection of the first straight line and the preset plane can be calculated, so that the three-dimensional coordinates of the foot point of the preset target on the preset plane are obtained. At this time, a second straight line perpendicular to the preset plane can be drawn at the three-dimensional coordinates of the foot point of the preset target, since the preset target is standing upright on the preset plane, so the head point of the preset target is on the second straight line at this time.

[0039] It should be noted that the preset plane described above can be used as a reference plane when the camera is calibrated, and the plane information of the preset plane in the world coordinate system is known. That is, the preset plane in the world coordinate system can be represented by a known plane function.

[0040] Since the head point coordinate of the preset target on the second image is known, and since the three-dimensional coordinates of the camera, the head point coordinate of the preset target on the second image and the head coordinate of the preset target in the world coordinate system are on a straight line according to the imaging principle of the camera, a third straight line can be obtained by using the three-dimensional coordinates of the camera and the head point coordinate of the preset target on the second image, and the intersection of the second straight line and the third straight line can be calculated, so that the three-dimensional coordinates of the head point of the preset target on the preset plane are obtained.

[0041] It should be noted that the two straight lines described in the specification of the present application will intersect as described above in a pure mathematical model, and the intersection between the two straight lines will be generated. However, in actual application, the two straight lines may not strictly intersect, but are close enough. In this case, the intersection between the two straight lines can be understood as the closest position of the two straight lines.

[0042] Finally, the height information of the preset target is obtained by calculating the Euclidean distance between the three-dimensional coordinates of the foot point and the three-dimensional coordinates of the head point of the preset target.

[0043] For example, as shown in FIG. 4, the height information of the preset target is obtained by calculating the Euclidean distance between the three-dimensional coordinates of the foot point and the three-dimensional coordinates of the head point of the preset target. Figure 2As shown, when the preset target walks on the preset plane, the second image information of the preset target can be collected by the camera, wherein the second image information includes the head point coordinate h2 of the preset target on the second image and the foot point coordinate f2 of the preset target on the second image, and then the straight line camera-h2 is calculated by using the three-dimensional coordinates of the camera and the head point coordinate h2 of the preset target on the second image, and the straight line camera-f2 is calculated by using the three-dimensional coordinates of the camera and the foot point coordinate f2 of the preset target on the second image, and the foot point f3 coordinate (x2, y2, z2) of the preset target on the preset plane is calculated according to the known preset plane and the straight line camera-f2. A straight line perpendicular to the preset plane is drawn at the foot point f3 coordinate, and the intersection of the straight line and the straight line camera-h2 can be used as the head point h3 coordinate (x1, y1, z1) of the preset target. Finally, the height information d of the preset target can be obtained by calculating the Euclidean distance between the three-dimensional coordinates (x2, y2, z2) of the foot point and the three-dimensional coordinates (x1, y1, z1) of the head point.

[0044]

[0045] Wherein, x2 is the coordinate of the human foot point in the x-axis direction, x1 is the coordinate of the human head point in the x-axis direction, y2 is the coordinate of the human foot point in the y-axis direction, y1 is the coordinate of the human head point in the y-axis direction, z2 is the coordinate of the human foot point in the z-axis direction, z1 is the coordinate of the human head point in the z-axis direction, and d is the Euclidean distance between the human foot point and the human head point of the preset target.

[0046] By using the above method, the height of the preset target can be calculated by using the parameter information of the camera, the three-dimensional coordinates of the camera and the image information collected by the camera on the preset target, the process is simple and fast, and no additional cost is increased.

[0047] The first image information of the preset target on the target plane can be the foot point coordinate on the first image collected when the preset target walks on the target plane and the head point coordinate on the first image.

[0048] In step 102, the three-dimensional coordinate set of the foot point of the preset target on the target plane is determined according to the height information of the preset target and the plurality of first image information.

[0049] Specifically, according to the height information of the preset target and the plurality of first image information, the three-dimensional coordinate set of the foot point of the preset target on the preset plane is determined as follows: according to the first image information, the head point coordinate of the preset target on the first image and the foot point coordinate of the preset target on the first image are determined, then the three-dimensional coordinates of the camera and the head point coordinate of the preset target on the first image are used to obtain a fourth straight line, and the three-dimensional coordinates of the camera and the foot point coordinate of the preset target on the first image are used to obtain a fifth straight line, and then a line segment that intersects the fourth straight line and the fifth straight line and matches the height information is searched in the direction perpendicular to the horizontal plane between the fourth straight line and the fifth straight line, and the three-dimensional coordinates of the intersection point of the searched line segment and the fifth straight line are taken as the foot point information of the preset target on the target plane. Finally, the plurality of first image information of the preset target is traversed to determine the foot point information of the preset target on the target plane in each first image information, and each foot point information is taken as the three-dimensional coordinate set of the foot point of the preset target on the target plane. The horizontal plane is a known plane.

[0050] As a specific embodiment, when the preset target walks on other planes, a plurality of first image information is obtained by using a camera to capture images of the preset target. The first image information can be the head point coordinate of the preset target on the first image and the foot point coordinate of the preset target on the first image. Specifically, the acquisition method of the first image information can be similar to the acquisition method of the second image information described above. A target detection model is used to detect the first image of the preset target walking on the target plane, and then key point detection is used to identify the first image to obtain the first image information of the preset target on the first image. The target detection model can be a centerNet network. By detecting the preset target, the image features of the preset target are obtained. The key point detection can be realized by the following method: in a deep regression model, a regression method is used to train and test image features to obtain the mapping of key point positions, and then a sparse matrix is used to estimate the key points on the current first image that are not mapped according to the mapping of the key point positions of the training set.

[0051] Furthermore, since the first image of the preset target is obtained by capturing the preset target on the target plane using a camera, the camera's 3D coordinates, the head coordinates of the preset target in the first image, and the 3D coordinates of the preset target's head in the world coordinate system are all on the same straight line. Similarly, the camera's 3D coordinates, the foot coordinates of the preset target in the first image, and the 3D coordinates of the preset target's feet in the world coordinate system are also on the same straight line. Using the camera's 3D coordinates and the preset target's head coordinates in the first image, a fourth straight line can be obtained. Simultaneously, using the camera's 3D coordinates and the preset target's foot coordinates in the first image, a fifth straight line can be obtained. A straight line perpendicular to the horizontal plane can be drawn; this line intersects the fourth line at the first intersection point, and the fifth line at the second intersection point. When the Euclidean distance between the first and second intersection points equals the height of the preset target, the first intersection point can be determined as the head coordinates of the preset target on the target plane, and the second intersection point as the foot coordinates of the preset target on the target plane. However, since a plane cannot be determined based on the coordinates of a single foot point, it is necessary to obtain multiple image information of the preset target walking on the target plane, and then obtain the three-dimensional coordinate set of the preset target's foot points on the target plane.

[0052] Step 103: Determine the planar information of the target plane based on the set of three-dimensional coordinates of the foot points.

[0053] Specifically, since noise points inevitably enter the target plane during its movement, these noise points may be due to the unevenness of the target plane itself or the undulations caused by the target's movement within the target plane. Therefore, a random sampling consensus algorithm can be used to determine the local points of the target. For example, as... Figure 3 As shown, Figure 3 The set of three-dimensional coordinates of the target's foot points on the target plane is used as the initial set of coordinates. This set is input into a random sampling consensus algorithm, such as RANSAC. RANSAC uses a small set of randomly assumed inhabitants as initial values. A model is then fitted using these inhabitants. This model fits the assumed inhabitants, and all unknown parameters can be calculated from these inhabitants. The resulting model is then used to test all other data. If a point fits the estimated model, it is considered an inhabitant, and the inhabitant list is expanded. If enough points are classified as assumed inhabitants, the estimated model is considered sufficiently reasonable. Finally, a parameterized model is obtained, for example... Figure 4 The straight line in the plane can be used to determine another straight line using the RANSAC algorithm based on other points in the plane. Finally, based on these two straight lines, the plane equation of the plane containing the preset target can be determined.

[0054] Finally, the above method can be used to reconstruct the multi-plane in the current space, and the multi-plane can be used for passenger flow statistics.

[0055] Specifically, after determining the plane information of the target plane, the passenger flow of a specific area on the target plane can be counted. For example, the specific area on the target plane can be a store opened on the target plane, or a gate deployed on the target plane. In actual application, a straight line can be set at the entrance / exit of the specific area, and the passenger flow can be counted by judging whether the to-be-identified target passes through the straight line. When the to-be-identified target passes through the preset straight line, the to-be-identified target is counted, so that the passenger flow of the specific area can be counted. The technical scheme provided in the application solves the problem that in the current general case, only when the heights of all planes are consistent, the target passing through the straight line can be counted.

[0056] Specifically, after determining the plane information of the target plane, the action trajectory of the target can be determined according to the image of the target, and then it is judged whether the target passes through the preset straight line according to the action trajectory of the target. Therefore, according to the plane information of the target plane and the multiple images of the to-be-identified target on the target plane collected by the camera, the action trajectory of the to-be-identified target acting on the target plane can be determined. If the action trajectory represents that the to-be-identified target passes through the preset straight line on the target plane, the to-be-identified target can be counted.

[0057] The application counts the targets on each plane respectively, realizes passenger flow counting by using a monocular passenger flow camera, avoids the problem that in the general technology, a multi-plane is reconstructed by scanning the whole scene by a radar or a depth camera, which is too high in cost and too complicated, and is not suitable for landing use, reduces the cost of passenger flow counting, and has high landing use value.

[0058] The application also provides a multi-plane reconstruction device, as shown in the figure, the device comprises: Figure 5 The acquisition module 51 is used to acquire the height information of a preset target and multiple first image information of the preset target on a target plane, and the detailed content is described with reference to step 101.

[0059] The three-dimensional coordinate set determination module 52 is used to determine a three-dimensional coordinate set of a foot point of the preset target on the target plane according to the height information of the preset target and the multiple first image information, and the detailed content is described with reference to step 102.

[0060] The plane information determination module 53 is used to determine the plane information of the target plane according to the three-dimensional coordinate set of the foot point, and the detailed content is described with reference to step 103.

[0061] The application also provides a multi-plane reconstruction method, as shown in the figure, the method comprises the following steps:

[0062] The embodiment of the present application also provides an electronic device, such as Figure 6 As shown in the figure, the electronic device can include a processor 601 and a memory 602, wherein the processor 601 and the memory 602 can be connected through a bus or other means, Figure 6 For example, the connection through the bus.

[0063] The processor 601 can be a central processing unit (CPU). The processor 601 can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination thereof.

[0064] The memory 602 as a non-transitory computer readable storage medium can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the program instructions / modules corresponding to the multi-plane reconstruction method and the key shielding method in the embodiment of the present application. The processor 601 executes various functions of the processor and data processing by running the non-transitory software programs, instructions and modules stored in the memory 602, that is, realizes the multi-plane reconstruction method in the above method embodiment.

[0065] The memory 602 can include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function; the data storage area can store data created by the processor 601 and the like. In addition, the memory 602 can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory 602 can optionally include a memory remotely arranged with respect to the processor 601, and these remote memories can be connected to the processor 601 through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0066] The one or more modules are stored in the memory 602, and when executed by the processor 601, the one or more modules perform the multi-plane reconstruction method in the embodiment as shown in the figure. Figures 1-4

[0067] The above electronic device specific details can be referred to in the figure Figures 1-4 ​The corresponding descriptions and effects of the embodiments shown are understood, and will not be repeated here.

[0068] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The program can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc. The storage medium can also include a combination of the above-mentioned types of memories.

[0069] Although the embodiments of the present application are described in conjunction with the drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A method of multiplanar reformation, characterized by, The method comprises the following steps: acquiring height information of a preset target and a plurality of first image information of the preset target on a target plane; the first image information comprises head point coordinates of the preset target on a first image and foot point coordinates of the preset target on the first image; and the target plane is a preset plane other than a calibrated plane; determining a three-dimensional coordinate set of a foot point of the preset target on the target plane according to the height information of the preset target and the plurality of first image information; determining plane information of the target plane according to the three-dimensional coordinate set of the foot point; wherein the determining of the three-dimensional coordinate set of the foot point of the preset target on the target plane according to the height information of the preset target and the plurality of first image information comprises: obtaining a fourth straight line by using a three-dimensional coordinate of a camera and the head point coordinates of the preset target on the first image; obtaining a fifth straight line by using the three-dimensional coordinate of the camera and the foot point coordinates of the preset target on the first image; searching for a line segment that intersects with both the fourth straight line and the fifth straight line and matches the height information in a direction perpendicular to a horizontal plane between the fourth straight line and the fifth straight line, and taking a three-dimensional coordinate of an intersection point of the searched line segment and the fifth straight line as foot point information of the preset target on the target plane; traversing the plurality of first image information of the preset target to determine the foot point information of the preset target on the target plane in each of the first image information, and taking each of the foot point information as the three-dimensional coordinate set of the foot point of the preset target on the target plane; the determining of the plane information of the target plane according to the three-dimensional coordinate set of the foot point comprises determining the plane information of the target plane according to the three-dimensional coordinate set of the foot point by using a random sample consensus algorithm.

2. The method of claim 1, wherein, The acquiring of the height information of the preset target comprises: determining second image information of the preset target on a preset plane collected by a camera; acquiring parameter information of the camera and a three-dimensional coordinate of the camera, and processing the second image information by using the parameter information and the three-dimensional coordinate of the camera to determine the height information of the preset target.

3. The method of claim 2, wherein, The determining of the second image information of the preset target on the preset plane collected by the camera comprises: acquiring a second image of the preset target on the preset plane collected by the camera; inputting the second image into a preset image recognition model to identify head point coordinates and foot point coordinates of the preset target in the second image by using the image recognition model, and taking the head point coordinates and the foot point coordinates as the determined second image information.

4. The method of claim 3, wherein, The determining of the height information of the preset target comprises: obtaining a first straight line by using the three-dimensional coordinate of the camera and the foot point coordinates of the preset target on the second image; acquiring plane information of the preset plane, and taking a three-dimensional coordinate of an intersection point of the first straight line and the preset plane as a three-dimensional coordinate of a foot point of the preset target; obtaining a second straight line by drawing a straight line perpendicular to the preset plane with the three-dimensional coordinate of the foot point of the preset target; and A third straight line is obtained by using the three-dimensional coordinates of the camera and the head point coordinates of the preset target on the second image; A three-dimensional coordinate of an intersection point of the second straight line and the third straight line is taken as a three-dimensional coordinate of the head point of the preset target; Height information of the preset target is determined according to the three-dimensional coordinates of the foot point of the preset target and the three-dimensional coordinates of the head point of the preset target.

5. The method of claim 1, wherein, The method further comprises: According to the plane information of the target plane and a plurality of images of a to-be-identified target on the target plane collected by a camera, an action trajectory of the to-be-identified target acting on the target plane is determined; If the action trajectory represents that the to-be-identified target crosses a preset straight line on the target plane, the to-be-identified target is counted.

6. A multi-planar reformation apparatus characterized by comprising: Comprise: An acquisition module is configured to acquire height information of a preset target and a plurality of first image information of the preset target on a target plane; The first image information comprises head point coordinates of the preset target on a first image and foot point coordinates of the preset target on the first image; and the target plane is a preset plane other than a calibrated plane; A three-dimensional coordinate set determination module is configured to determine a three-dimensional coordinate set of a foot point of the preset target on the target plane according to the height information of the preset target and the plurality of first image information; A plane information determination module is configured to determine plane information of the target plane according to the three-dimensional coordinate set of the foot point; The three-dimensional coordinate set determination module is specifically configured to obtain a fourth straight line by using three-dimensional coordinates of a camera and the head point coordinates of the preset target on the first image; A fifth straight line is obtained by using the three-dimensional coordinates of the camera and the foot point coordinates of the preset target on the first image; A line segment that intersects the fourth straight line and the fifth straight line and matches the height information is searched for along a direction perpendicular to a horizontal plane between the fourth straight line and the fifth straight line, and a three-dimensional coordinate of an intersection point of the searched line segment and the fifth straight line is taken as foot point information of the preset target on the target plane; The plurality of first image information of the preset target is traversed to determine the foot point information of the preset target on the target plane in each of the first image information, and each of the foot point information is taken as the three-dimensional coordinate set of the foot point of the preset target on the target plane; The plane information determination module is specifically configured to determine the plane information of the target plane according to the three-dimensional coordinate set of the foot point by using a random sample consensus algorithm.

7. An electronic device, comprising: Comprise: At least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform steps of the multi-plane reconstruction method according to any one of claims 1-5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement steps of the multi-plane reconstruction method according to any one of claims 1-5.

Images