Cargo quantity monitoring method and system

Abstract

The application provides a cargo quantity monitoring method and system, the method comprising: collecting a first visible light image of a to-be-monitored area by a first collection device; collecting a first laser point cloud of the to-be-monitored area by a second collection device; obtaining pixel coordinates of a plurality of target feature points in the first visible light image and obtaining point cloud coordinates of the corresponding plurality of target feature points in the first laser point cloud to form a plurality of pixel-point cloud coordinate groups; obtaining a conversion matrix between the image and the point cloud according to the pixel-point cloud coordinate groups; obtaining a second visible light image of the to-be-monitored area by the first collection device; obtaining a second laser point cloud of the to-be-monitored area by the second collection device; identifying a pixel set of the cargo in the to-be-monitored area according to the second visible light image; matching to obtain a point cloud set of the cargo in the second laser point cloud according to the conversion matrix, the second laser point cloud and the pixel set of the cargo; and calculating the volume of the cargo according to the point cloud set of the cargo to realize cargo quantity monitoring.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a method and system for monitoring cargo volume. Background Technology

[0002] Fireworks are classified as hazardous materials, and relevant regulations and rules impose strict requirements on their storage quantity. Therefore, it is necessary to monitor the storage quantity of fireworks in the warehouse in real time to prevent the storage quantity from exceeding the standard and causing safety hazards.

[0003] With the development of computer vision technology, computer vision-based monitoring is gradually replacing manual monitoring. Computer vision-based monitoring not only provides convenient, fast, accurate, and 24 / 7 monitoring, but also allows the transmission of detected anomalies to the monitoring center or the monitoring personnel's terminal via the internet. Technicians in related fields know that images captured by cameras, combined with relatively mature image target recognition technology, are beneficial for effectively identifying targets, but cannot determine the quantity of the target; while point cloud data acquired by radar can calculate the volume of a target, but its accuracy in target identification is relatively low. How to effectively, or even perfectly, combine these two technologies is a technical problem that technicians in related fields are constantly exploring. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a cargo quantity monitoring method and system that can accurately match images and laser point clouds, thereby effectively acquiring and monitoring the cargo quantity in the monitored area based on accurate identification of the cargo in the monitored area.

[0005] The technical solution adopted in this invention is as follows:

[0006] A method for monitoring cargo quantity includes the following steps: acquiring a first visible light image of a monitored area using a first acquisition device; acquiring a first laser point cloud of the monitored area using a second acquisition device; obtaining pixel coordinates of multiple target feature points in the first visible light image and point cloud coordinates of corresponding multiple target feature points in the first laser point cloud to form multiple pixel-point cloud coordinate groups; obtaining a transformation matrix between the image and the point cloud based on the pixel-point cloud coordinate groups; acquiring a second visible light image of the monitored area using the first acquisition device; acquiring a second laser point cloud of the monitored area using the second acquisition device; identifying the pixel set of cargo in the monitored area based on the second visible light image; matching the transformation matrix, the second laser point cloud, and the pixel set of cargo to obtain the point cloud set of cargo in the second laser point cloud; calculating the volume of cargo based on the point cloud set of cargo to achieve monitoring of cargo quantity in the monitored area.

[0007] The target feature points in the first visible light image are automatically selected by identifying fixed objects in the area to be monitored.

[0008] Obtaining the point cloud coordinates of multiple target feature points in the first laser point cloud specifically includes: manually selecting initial points in the first laser point cloud based on the location of each target feature point in the first visible light image; constructing a cube with the initial points as the center and a preset length as the side length; dividing the cube into several sub-cubes; performing surface fitting on the points in each sub-cube to obtain the equation of the fitted surface in each sub-cube; calculating the feature point selection parameters of the fitted surface in each sub-cube based on the equation of the fitted surface in each sub-cube; and determining the maximum point of the fitted surface with the largest feature point selection parameters as the corresponding target feature point in the first laser point cloud.

[0009] The equation of the fitted surface is:

[0010]

[0011] in,( x , y , z () represents the coordinates of a point. λ i Let be the coefficient, where i Choose 1, 2, 3, 4, 5, 6;

[0012] The feature point selection parameters for the fitted surface are:

[0013]

[0014] in, F P Parameters are selected for the feature points of the fitted surface. a Let be the side length of the sub-cube.

[0015] The goods are pyrotechnic agents.

[0016] A cargo quantity monitoring system includes: a first acquisition device for acquiring a first visible light image of a region to be monitored; a second acquisition device for acquiring a first laser point cloud of the region to be monitored; a first acquisition module for acquiring pixel coordinates of multiple target feature points in the first visible light image and point cloud coordinates of corresponding multiple target feature points in the first laser point cloud, to form multiple pixel-point cloud coordinate groups; a second acquisition module for acquiring a transformation matrix between the image and the point cloud based on the pixel-point cloud coordinate groups; the first acquisition device is also used to acquire a second visible light image of the region to be monitored; the second acquisition device is also used to acquire a second laser point cloud of the region to be monitored; an identification module for identifying a set of pixels of cargo in the region to be monitored based on the second visible light image; a matching module for matching a set of points of cargo in the second laser point cloud based on the transformation matrix, the second laser point cloud, and the set of pixels of cargo; and a calculation module for calculating the volume of cargo based on the set of points of cargo, to monitor the quantity of cargo in the region to be monitored.

[0017] The target feature points in the first visible light image are automatically selected by the first acquisition module through the identification of fixed objects in the area to be monitored.

[0018] The first acquisition module is specifically used for: after manually selecting corresponding initial points in the first laser point cloud based on the location of each target feature point in the first visible light image, constructing a cube with the initial points as the center and a preset length as the side length; dividing the cube into several sub-cubes; performing surface fitting on the points in each sub-cube to obtain the equation of the fitted surface in each sub-cube; calculating the feature point selection parameters of the fitted surface in each sub-cube based on the equation of the fitted surface in each sub-cube; and determining the maximum point of the fitted surface with the largest feature point selection parameters as the corresponding target feature point in the first laser point cloud.

[0019] The equation of the fitted surface is:

[0020]

[0021] in,( x , y , z () represents the coordinates of a point. λ i Let be the coefficient, where i Choose 1, 2, 3, 4, 5, 6;

[0022] The feature point selection parameters for the fitted surface are:

[0023]

[0024] in, F P Parameters are selected for the feature points of the fitted surface. a Let be the side length of the sub-cube.

[0025] The goods are pyrotechnic agents.

[0026] The beneficial effects of this invention are:

[0027] This invention first obtains the corresponding target feature points in image data and laser point cloud data to establish an accurate matching relationship between them. Then, during actual monitoring, it identifies the goods through the image and obtains the point cloud set of the goods based on the matching relationship. Finally, it calculates the volume of the goods based on the point cloud set. Thus, it can accurately match the image and the laser point cloud, thereby effectively obtaining and monitoring the quantity of goods in the area to be monitored based on accurate identification of the goods in the area to be monitored. Attached Figure Description

[0028] Figure 1 This is a flowchart of the cargo volume monitoring method according to an embodiment of the present invention;

[0029] Figure 2 This is a block diagram of a cargo volume monitoring system according to an embodiment of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] like Figure 1 As shown, the cargo volume monitoring method of this invention includes the following steps:

[0032] S1, acquire the first visible light image of the area to be monitored through the first acquisition device.

[0033] S2, the first laser point cloud of the area to be monitored is acquired through the second acquisition device.

[0034] In one embodiment of the present invention, the goods to be monitored are pyrotechnic agents, and the area to be monitored is a warehouse storing the pyrotechnic agents. The first acquisition device and the second acquisition device can be a camera and a lidar, respectively. The area to be monitored, targeted by the first visible light image and the first lidar point cloud, includes not only the goods but also some fixed objects, such as building structures like walls, ground, and beams, as well as some necessary tools and equipment.

[0035] S3, obtain the pixel coordinates of multiple target feature points in the first visible light image, and obtain the point cloud coordinates of the corresponding multiple target feature points in the first laser point cloud, so as to form multiple pixel-point cloud coordinate groups.

[0036] In one embodiment of the present invention, before obtaining the pixel coordinates of multiple target feature points in the first visible light image, a specific sub-region in the first visible light image can be cropped first. This sub-region contains fixed objects that will not move during subsequent actual monitoring, such as corners of walls, floors, etc.

[0037] The target feature points in the first visible light image are automatically selected by identifying fixed objects in the area to be monitored. Feature point detection algorithms such as the Harris algorithm can be applied here.

[0038] By using feature points in fixed objects in the area to be monitored as target feature points, there is no need to use special calibration objects such as checkerboard patterns. Furthermore, the positions, angles, and working environments of the first and second acquisition devices are the same as those in the subsequent actual monitoring process, which helps to ensure the accuracy of matching between the image and the point cloud.

[0039] After obtaining the pixel coordinates of multiple target feature points in the first visible light image, a preliminary selection point can be manually selected in the first laser point cloud based on the location of each target feature point in the first visible light image. Assuming the target feature point in the first visible light image is point P, its corresponding preliminary selection point is a point in the first laser point cloud that is approximately at the same or similar position as point P.

[0040] Then, construct a cube with the initially selected point as the center and a preset length as the side length. This preset length should ensure that the corresponding actual target feature point is within the cube, but it should not be too large to avoid excessive computation or the inclusion of other target feature points.

[0041] Next, the cube can be divided into several sub-cubes, and surface fitting can be performed on the points within each sub-cube to obtain the equation of the fitted surface within each sub-cube. The number of sub-cubes is preferably 8 or 27. The surface fitting method can be linear least squares. In one embodiment of the invention, the equation of the fitted surface is:

[0042]

[0043] in,( x , y , z () represents the coordinates of a point. λ i Let be the coefficient, where i Choose 1, 2, 3, 4, 5, 6.

[0044] Furthermore, the feature point selection parameters for the fitted surface within each sub-cube can be calculated based on the equation of the fitted surface within each sub-cube. In one embodiment of the present invention, the feature point selection parameters for the fitted surface are:

[0045]

[0046] in, F P Select parameters for the feature points of the fitted surface. a Let be the side length of the cube.

[0047] Finally, the maximum point of the fitted surface with the largest parameter can be selected as the target feature point in the first laser point cloud.

[0048] Following the above method, the target feature point corresponding to each target feature point in the first visible light image is determined in the first laser point cloud. The combination of the pixel coordinates of each target feature point in the first visible light image and the point cloud coordinates of the target feature point corresponding to it in the first laser point cloud is called the pixel-point cloud coordinate group.

[0049] S4, obtain the transformation matrix between the image and the point cloud based on the pixel-point cloud coordinate set.

[0050] The pixel-to-point cloud coordinate sets obtained in step S3 are typically at least four sets. Therefore, the PnP algorithm can be applied, combining camera intrinsics and the pixel-to-point cloud coordinate sets, to obtain the transformation matrix between the image and the point cloud. This transformation matrix includes a rotation matrix and a translation matrix. It should be understood that, given sufficient computing power, the more pixel-to-point cloud coordinate sets there are, the more accurately the transformation matrix can reflect the matching relationship between the image and the point cloud.

[0051] The above steps S1~S4 are the matching stage of the data acquired by the first acquisition device and the second acquisition device, and the following steps S5~S9 are the actual cargo quantity monitoring stage.

[0052] S5, acquire a second visible light image of the area to be monitored through the first acquisition device.

[0053] S6, the second laser point cloud of the area to be monitored is acquired through the second acquisition device.

[0054] S7, Identify the pixel set of goods in the area to be monitored based on the second visible light image.

[0055] Here, a deep learning-based target recognition algorithm can be used to identify the goods in the area to be monitored, thereby obtaining the pixel set of the goods in the area to be monitored. The second visible light image here, together with the first visible light image obtained in step S1, can be processed by image filtering, image correction, etc., if necessary before application.

[0056] S8. Based on the transformation matrix, the second laser point cloud, and the pixel set of the cargo, the point cloud set of the cargo in the second laser point cloud is obtained by matching.

[0057] By using a transformation matrix, points in the point cloud can be mapped to pixels in the image. For each pixel in the pixel set of the cargo, all points in the second laser point cloud that can be mapped to that pixel are obtained. Then, the set of points that can be mapped to all pixels in the pixel set can be used to form the point cloud set of the cargo.

[0058] S9 calculates the volume of goods based on the point cloud of goods in order to monitor the quantity of goods in the area to be monitored.

[0059] In one specific embodiment of the present invention, since the stored pyrotechnic agents generally do not contain concave surfaces, the convex hull method can be used to calculate the volume of the smallest convex polyhedron containing all points, thereby obtaining the volume of the cargo. In other embodiments of the present invention, Poisson surface reconstruction algorithms or similar methods can also be used to calculate the cargo volume.

[0060] When the amount of goods in the monitored area is about to exceed the limit or has already exceeded the limit, an alarm message can be sent to the monitoring center or the monitoring personnel's terminal.

[0061] According to the cargo quantity monitoring method of the present invention, firstly, by accurately acquiring the corresponding target feature points in image data and laser point cloud data, an accurate matching relationship between the image data and the laser point cloud data is obtained. Then, during actual monitoring, cargo is identified through the image, and the point cloud set of the cargo is obtained by combining the above matching relationship. Finally, the volume of the cargo is calculated based on the point cloud set of the cargo. Thus, the image and laser point cloud can be accurately matched, thereby effectively acquiring and monitoring the cargo quantity in the area to be monitored based on accurate identification of the cargo in the area to be monitored.

[0062] Corresponding to the cargo volume monitoring method in the above embodiments, the present invention also proposes a cargo volume monitoring system.

[0063] like Figure 2As shown, the cargo volume monitoring system of this embodiment includes a first acquisition device 10, a second acquisition device 20, a first acquisition module 30, a second acquisition module 40, an identification module 50, a matching module 60, and a calculation module 70. The system comprises the following components: a first acquisition device 10 for acquiring a first visible light image of the area to be monitored; a second acquisition device 20 for acquiring a first laser point cloud of the area to be monitored; a first acquisition module 30 for acquiring the pixel coordinates of multiple target feature points in the first visible light image and the point cloud coordinates of the corresponding multiple target feature points in the first laser point cloud, to form multiple pixel-point cloud coordinate groups; a second acquisition module 40 for acquiring a transformation matrix between the image and the point cloud based on the pixel-point cloud coordinate groups; a first acquisition device 10 for acquiring a second visible light image of the area to be monitored; a second acquisition device 20 for acquiring a second laser point cloud of the area to be monitored; a recognition module 50 for recognizing the pixel set of goods in the area to be monitored based on the second visible light image; a matching module 60 for matching the point cloud set of goods in the second laser point cloud based on the transformation matrix, the second laser point cloud, and the pixel set of goods; and a calculation module 70 for calculating the volume of goods based on the point cloud set of goods, thereby achieving the monitoring of the quantity of goods in the area to be monitored.

[0064] In one embodiment of the present invention, the goods to be monitored are pyrotechnic agents, and the area to be monitored is a warehouse storing the pyrotechnic agents. The first acquisition device 10 and the second acquisition device 20 can be a camera and a lidar, respectively. The area to be monitored, targeted by the first visible light image and the first lidar point cloud, includes not only the goods but also some fixed objects, such as building structures like walls, ground, and beams, as well as some necessary tools and equipment.

[0065] In one embodiment of the present invention, before acquiring the pixel coordinates of multiple target feature points in the first visible light image, the first acquisition module 30 may first capture a specific sub-region in the first visible light image. This sub-region contains fixed objects that will not move during subsequent actual monitoring, such as corners of walls, floors, etc.

[0066] The target feature points in the first visible light image are automatically selected by the first acquisition module 30 through the identification of fixed objects in the area to be monitored. Feature point detection algorithms such as the Harris algorithm can be applied here.

[0067] By using feature points in fixed objects in the area to be monitored as target feature points, there is no need to use special calibration objects such as checkerboard patterns. Furthermore, the positions, angles, and working environments of the first acquisition device 10 and the second acquisition device 20 are the same as those in the subsequent actual monitoring process, which helps to ensure the accuracy of matching between the image and the point cloud.

[0068] After manually selecting initial points in the first laser point cloud based on the location of each target feature point in the first visible light image, the first acquisition module 30 can construct a cube with the initial points as the center and a preset length as the side length. Assuming the target feature point in the first visible light image is point P, its corresponding initial point is a point in the first laser point cloud that is approximately at the same or similar position as point P. This preset length should ensure that the corresponding actual target feature point is within the cube, but it should not be too large to avoid excessive computation or the inclusion of other target feature points.

[0069] Next, the first acquisition module 30 can divide the cube into several sub-cubes and perform surface fitting on points within each sub-cube to obtain the equation of the fitted surface within each sub-cube. The number of sub-cubes is preferably 8 or 27. The surface fitting method can be linear least squares. In one embodiment of the invention, the equation of the fitted surface is:

[0070]

[0071] in,( x , y , z () represents the coordinates of a point. λ i Let be the coefficient, where i Choose 1, 2, 3, 4, 5, 6.

[0072] Furthermore, the first acquisition module 30 can calculate the feature point selection parameters of the fitted surface within each sub-cube based on the equation of the fitted surface within each sub-cube. In one embodiment of the present invention, the feature point selection parameters of the fitted surface are:

[0073]

[0074] in, F P Select parameters for the feature points of the fitted surface. a Let be the side length of the cube.

[0075] Finally, the first acquisition module 30 can determine the maximum point of the fitted surface with the largest feature point selection parameter as the corresponding target feature point in the first laser point cloud.

[0076] Following the above method, the target feature point corresponding to each target feature point in the first visible light image is determined in the first laser point cloud. The combination of the pixel coordinates of each target feature point in the first visible light image and the point cloud coordinates of the target feature point corresponding to it in the first laser point cloud is called the pixel-point cloud coordinate group.

[0077] The first acquisition module 30 typically acquires at least four pixel-to-point cloud coordinate sets. Therefore, the second acquisition module 40 can apply the PnP algorithm, combining camera intrinsic parameters and the pixel-to-point cloud coordinate sets, to obtain the transformation matrix between the image and the point cloud. This transformation matrix includes a rotation matrix and a translation matrix. It should be understood that, given sufficient computing power, the more pixel-to-point cloud coordinate sets there are, the more accurately the transformation matrix can reflect the matching relationship between the image and the point cloud.

[0078] The identification module 50 can use a deep learning-based target recognition algorithm to identify goods in the area to be monitored, thereby obtaining a pixel set of goods in the area to be monitored. During the data matching stage with the second acquisition device and the actual monitoring stage, the visible light images acquired by the first acquisition device 10 can be processed, such as image filtering and image correction, if necessary before application.

[0079] The matching module 60 can map the points in the point cloud to the pixels of the image through the transformation matrix. For each pixel in the pixel set of the goods, it obtains all the points in the second laser point cloud that can be mapped to that pixel. Then, the set of all the points that can be mapped to the pixel set can be used to form the point cloud set of the goods.

[0080] In one specific embodiment of the present invention, since the stored pyrotechnic agents generally do not contain concave surfaces, the calculation module 70 can use the convex hull method to calculate the volume of the smallest convex polyhedron containing all points of the point cloud, thereby obtaining the volume of the cargo. In other embodiments of the present invention, the calculation module 70 can also use Poisson surface reconstruction algorithms or the like to calculate the cargo volume.

[0081] The cargo quantity monitoring system of this invention may also include an alarm module. When the cargo quantity in the monitored area is about to exceed the limit or has already exceeded the limit, the alarm module can send alarm information to the monitoring center or the monitoring personnel terminal.

[0082] According to the cargo quantity monitoring system of the present invention, the system first obtains the corresponding target feature points in the image data and laser point cloud data to obtain the accurate matching relationship between the image data and the laser point cloud data. Then, during actual monitoring, the system identifies the cargo through the image and obtains the point cloud set of the cargo by combining the above matching relationship. Finally, the system calculates the volume of the cargo based on the point cloud set of the cargo. Thus, the system can accurately match the image and the laser point cloud, thereby effectively obtaining and monitoring the cargo quantity in the area to be monitored based on accurate identification of the cargo in the area to be monitored.

[0083] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. "A plurality of" means two or more, unless otherwise explicitly specified.

[0084] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0085] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0086] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0087] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0088] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0089] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0090] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0091] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0092] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for monitoring cargo volume, characterized in that, Includes the following steps: The first visible light image of the area to be monitored is acquired by the first acquisition device; The first laser point cloud of the area to be monitored is acquired by the second acquisition device; Obtain the pixel coordinates of multiple target feature points in the first visible light image, and obtain the point cloud coordinates of multiple target feature points in the first laser point cloud to form multiple pixel-point cloud coordinate groups; The transformation matrix between the image and the point cloud is obtained based on the pixel-point cloud coordinate set; The first acquisition device acquires a second visible light image of the area to be monitored; The second laser point cloud of the area to be monitored is acquired by the second acquisition device; The pixel set of goods in the area to be monitored is identified based on the second visible light image; Based on the transformation matrix, the second laser point cloud, and the pixel set of the cargo, the point cloud set of the cargo in the second laser point cloud is obtained by matching; The volume of the goods is calculated based on the point cloud of the goods, so as to monitor the quantity of goods in the area to be monitored. The target feature points in the first visible light image are automatically selected by identifying fixed objects in the area to be monitored, and the point cloud coordinates of multiple target feature points in the first laser point cloud are obtained. Specifically, this includes: manually selecting a preliminary point in the first laser point cloud based on the location of each target feature point in the first visible light image; constructing a cube with the preliminary point as the center and a preset length as the side length; dividing the cube into several sub-cubes; performing surface fitting on the points in each sub-cube to obtain the equation of the fitted surface in each sub-cube; calculating the feature point selection parameters of the fitted surface in each sub-cube based on the equation of the fitted surface in each sub-cube; and determining the maximum point of the fitted surface with the largest feature point selection parameters as the corresponding target feature point in the first laser point cloud.

2. The cargo volume monitoring method according to claim 1, characterized in that, The equation of the fitted surface is: ； Where (x,y,z) represents the coordinates of a point, and λ i Let be the coefficient, where i takes values ​​of 1, 2, 3, 4, 5, 6; The feature point selection parameters for the fitted surface are: ； Among them, F P Parameters are selected for the feature points of the fitted surface, where a is the side length of the sub-cube.

3. The cargo volume monitoring method according to claim 1 or 2, characterized in that, The goods are pyrotechnic agents.

4. A cargo volume monitoring system, characterized in that, include: A first acquisition device is used to acquire a first visible light image of the area to be monitored. The second acquisition device is used to acquire the first laser point cloud of the area to be monitored. The first acquisition module is used to acquire the pixel coordinates of multiple target feature points in the first visible light image and the point cloud coordinates of the corresponding multiple target feature points in the first laser point cloud, so as to form multiple pixel-point cloud coordinate groups. The second acquisition module is used to acquire the transformation matrix between the image and the point cloud based on the pixel-point cloud coordinate group. The first acquisition device is also used to acquire a second visible light image of the area to be monitored; The second acquisition device is also used to acquire a second laser point cloud of the area to be monitored; The identification module is used to identify the pixel set of goods in the area to be monitored based on the second visible light image; A matching module is configured to match the point cloud set of the cargo in the second laser point cloud based on the transformation matrix, the second laser point cloud, and the pixel set of the cargo; The calculation module is used to calculate the volume of the goods based on the point cloud of the goods, so as to monitor the quantity of goods in the area to be monitored. The target feature points in the first visible light image are automatically selected by the first acquisition module through the identification of fixed objects in the area to be monitored. The first acquisition module is specifically used for: manually selecting corresponding initial points in the first laser point cloud based on the location of each target feature point in the first visible light image; constructing a cube with the initial points as the center and a preset length as the side length; dividing the cube into several sub-cubes; performing surface fitting on the points in each sub-cube to obtain the equation of the fitted surface in each sub-cube; calculating the feature point selection parameters of the fitted surface in each sub-cube based on the equation of the fitted surface in each sub-cube; and determining the maximum point of the fitted surface with the largest feature point selection parameters as the corresponding target feature point in the first laser point cloud.

5. The cargo volume monitoring system according to claim 4, characterized in that, The equation of the fitted surface is: ； Where (x,y,z) represents the coordinates of a point, and λ i Let be the coefficient, where i takes values ​​of 1, 2, 3, 4, 5, 6; The feature point selection parameters for the fitted surface are: ； Among them, F P Parameters are selected for the feature points of the fitted surface, where a is the side length of the sub-cube.

6. The cargo volume monitoring system according to claim 4 or 5, characterized in that, The goods are pyrotechnic agents.

Images