Box volume measurement method, device, system and electronic equipment

Abstract

The present disclosure relates to a method, system, device and electronic device for measuring the volume of a box. A monocular camera obtains a calibration plate image on a calibration plate and image information of an object to be measured placed on the calibration plate. The monocular camera is arranged near the calibration plate. The monocular camera and a computing unit perform data and communication interaction. The present disclosure can realize the calculation of the volume of the box by obtaining the calibration plate image and the box image, and uses the box image to perform projection calculation and coordinate conversion to obtain the length and width of the box, and then obtain the height of the box by the height difference, and finally obtain the volume of the box. Compared with the prior art binocular 3D or TOF 3D, the present disclosure has low implementation cost, relatively low requirements on the computing power of the equipment, convenient scene construction and high measurement efficiency.

Description

Technical Field

[0001] The present invention relates to a box volume measurement method, device, system and electronic equipment. Background Art

[0002] With the rapid development of the Internet of Things, the number of express delivery orders has increased rapidly. During the process of entering and leaving the warehouse, these express parcels need to collect important data information such as size, volume and weight. Compared with the weight of the parcel, it is more difficult to quickly obtain the parcel size information. Manual measurement errors are large and time-consuming and labor-intensive. Considering that most of the current express parcels are regular-shaped rectangular cartons, if the volume is measured by TOF 3D or binocular 3D depth cameras, the cost is high and it is difficult to popularize and apply on a large scale. Summary of the invention

[0003] The present disclosure provides a volume measurement method, device, system and electronic device.

[0004] According to one aspect of the present disclosure, a method for measuring the volume of a box is provided, including: acquiring a calibration plate image, and calculating a first height between the origin of a camera coordinate system and the calibration plate according to the calibration plate image, wherein the camera coordinate system is a coordinate system in which a monocular camera that captures the calibration plate image is located; acquiring a box image captured by the monocular camera, wherein the box is placed on the calibration plate, and performing a projection operation on the box image to obtain the length and width of the box in the world coordinate system of the calibration plate and the coordinates of the box vertices on the upper surface of the box; establishing a new world coordinate system with one of the box vertices as the origin to obtain a second height of the monocular camera relative to the upper surface of the box; taking the height difference between the first height and the second height as the height of the box, and calculating the volume of the box according to the length, width and height of the box.

[0005] According to at least one embodiment of the present disclosure, the step of acquiring the calibration plate image and calculating the first height between the origin of the camera coordinate system and the calibration plate based on the calibration plate image further includes: acquiring the origin of the camera coordinate system and the camera extrinsic parameters of the monocular camera, and obtaining a first translation vector and a first rotation vector of the monocular camera relative to the calibration plate through the calibration plate image; converting the first rotation vector into a corresponding rotation matrix to obtain first depth information of the monocular camera, and determining a first height based on the first depth information, wherein the first height is specifically the height of the origin of the camera coordinate system from the calibration plate.

[0006] According to at least one embodiment of the present disclosure, the converting the first rotation vector into a corresponding rotation matrix to obtain the first depth information of the monocular camera further includes: establishing a calibration plate world coordinate system on the calibration plate, wherein the Z axis of the calibration plate world coordinate system is perpendicular to the horizontal plane; the calibration plate image is a checkerboard image, the origin of the checkerboard image is set, the checkerboard size is used as a coordinate distance reference for the X-axis and Y-axis of the world coordinate system, the three-dimensional coordinates of the checkerboard corner points are defined by the origin of the checkerboard pattern, and the pixel coordinates of the corresponding checkerboard corner points are obtained; a first translation vector and a first rotation vector of the monocular camera relative to the calibration plate are obtained, the first rotation vector is converted into a rotation matrix, and the first depth information of the camera is obtained; the first depth information is set to zero, and a first height from the camera to the calibration plate plane is obtained according to a mapping relationship between the camera coordinate system and the calibration plate world coordinate system.

[0007] According to at least one embodiment of the present disclosure, the acquiring of the box image taken by the monocular camera, wherein the box is placed on the calibration plate, and performing a projection operation on the box image to obtain the length and width of the box in the world coordinate system of the calibration plate and the coordinates of the box vertices on the upper surface of the box, further includes: acquiring the box image of the box placed on the calibration plate, and obtaining the pixel coordinates of the length and width vertices of the lower surface of the box corresponding to the calibration plate; performing a projection operation on the pixel coordinates of the length and width vertices, projecting the pixel coordinates of the length and width vertices into the world coordinate system of the calibration plate, and calculating the pixel coordinates of the length and width vertices of the box in the world coordinate system of the calibration plate to obtain the length and width of the box.

[0008] According to at least one embodiment of the present disclosure, the projection operation is performed on the pixel coordinates of the length and width vertices, the pixel coordinates of the length and width vertices are projected into the world coordinate system of the calibration plate, and the pixel coordinates of the length and width vertices of the box are calculated in the world coordinate system of the calibration plate to obtain the length and width of the box, further including: using the Zhang Zhengyou calibration model to obtain the intrinsic parameters and distortion parameters of the monocular camera through the chessboard pattern of the calibration plate; according to the intrinsic parameters and extrinsic parameters of the monocular camera, the pixel coordinates of the length and width vertices of the box are projected and transformed into the world coordinate system of the calibration plate, and the coordinates of the pixel coordinates of the length and width vertices in the camera coordinate system projected into the world coordinate system of the calibration plate are obtained; and the length and width of the box are calculated using the coordinates of the world coordinate system of the calibration plate.

[0009] According to at least one embodiment of the present disclosure, the pixel coordinates of the length and width vertices of the box are projected and transformed into the world coordinate system of the calibration plate based on the intrinsic parameters and extrinsic parameters of the monocular camera, and the pixel coordinates of the length and width vertices in the camera coordinate system are projected into the world coordinate system of the calibration plate. Specifically, the pixel coordinates of the length and width vertices of the box are projected into the world coordinate system of the calibration plate by performing a projection operation using the first translation vector and the first rotation vector; the length and width of the box are calculated in the world coordinate system of the calibration plate using the world coordinates of the length and width vertices of the box in the world coordinate system of the calibration plate after the projection transformation, combined with the extrinsic parameters of the position of the monocular camera relative to the calibration plate, and the known intrinsic parameters of the camera.

[0010] According to at least one embodiment of the present disclosure, establishing a new world coordinate system with one of the vertices of the box as the origin to obtain the second height of the monocular camera relative to the upper surface of the box further includes: selecting a vertex on the upper surface of the box to establish a new world coordinate system, and defining the three-dimensional coordinates of four vertices on the upper surface of the box based on the new world coordinate system; and calculating the second height of the monocular camera relative to the new world coordinate system based on known camera intrinsic parameters and distortion parameters.

[0011] According to at least one embodiment of the present disclosure, in the process of establishing a new world coordinate system for a vertex on the upper surface of the box, the coordinates of the remaining three vertices on the upper surface of the box are defined, and the pixel coordinate values ​​of the remaining three vertices are respectively obtained; the third translation vector and the third rotation vector are obtained through the calibration plate image, and the third rotation vector is converted into a corresponding rotation matrix by performing a matrix transposition operation, and the rotation matrix is ​​multiplied by the third translation vector to obtain the second depth information of the monocular camera, and the second depth information is specifically the second height.

[0012] According to another aspect of the present disclosure, a box volume measurement device is provided, including: a first height calculation module, used to obtain a calibration plate image, and calculate a first height between the origin of a camera coordinate system and the calibration plate according to the calibration plate image, wherein the camera coordinate system is a coordinate system in which a monocular camera that captures the calibration plate image is located; a projection operation module, used to obtain a box image captured by the monocular camera, wherein the box is placed on the calibration plate, and perform a projection operation on the box image to obtain the length and width of the box in the world coordinate system of the calibration plate and the coordinates of the box vertices on the upper surface of the box; a second height calculation module, used to establish a new world coordinate system with one of the box vertices as the origin, and obtain a second height of the monocular camera relative to the upper surface of the box; and a box volume calculation module, used to take the height difference between the first height and the second height as the height of the box, and to calculate the volume of the box according to the length, width and height of the box.

[0013] According to another aspect of the present disclosure, a box volume measurement system is provided, comprising: a monocular camera, used to capture images and arranged to be inclined relative to the calibration plate; a calculation unit, used to calculate the size of the box; wherein the calculation unit obtains the calibration plate image, and calculates a first height between the origin of the camera coordinate system and the calibration plate according to the calibration plate image, wherein the camera coordinate system is a coordinate system in which the monocular camera that captures the calibration plate image is located; the calculation unit obtains the box image captured by the monocular camera, wherein the box is placed on the calibration plate, and performs a projection operation on the box image to obtain the length and width of the box in the world coordinate system of the calibration plate and the coordinates of the box vertices on the upper surface of the box; the calculation unit establishes a new world coordinate system with one of the box vertices as the origin, and obtains the second height of the monocular camera relative to the upper surface of the box; the calculation unit uses the height difference between the first height and the second height as the height of the box, and calculates the volume of the box according to the length, width and height of the box.

[0014] According to another aspect of the present disclosure, an electronic device is provided, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, so that the processor executes the box volume measurement method.

[0015] According to another aspect of the present disclosure, a readable storage medium is provided, wherein the readable storage medium stores execution instructions, and the execution instructions are used to implement the box volume measurement method when executed by a processor.

[0016] According to other aspects of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the box volume measurement method is implemented.

[0017] The technical effect of the present invention is that when measuring the volume of a box, only a monocular camera needs to be calibrated with a fixed calibration plate. The volume of a box or other rectangular object can be calculated by obtaining a photo. The length and width of the box can be obtained through projection calculation. Different coordinate systems are established and the coordinate systems are converted to obtain the first height between the camera coordinate system and the calibration plate in different coordinate systems, as well as the second height between the camera coordinate system and the upper surface of the box. The height of the box is obtained by calculating the height difference between the first and second heights. Finally, the volume of the box is calculated by the length, width and height of the box. Compared with binocular 3D or TOF 3D, the implementation cost is low, the requirement for equipment computing power is relatively low, the scene is easy to build, and the measurement efficiency is high.

[0018] Since the monocular camera and the calibration plate will be calibrated with external parameters after being fixed, there are no special requirements for the distance of the monocular camera relative to the calibration plate and the size of the calibration plate image (chessboard). It has higher flexibility and can be freely adjusted according to the size of the measurement site and the size of the box to be measured, and has wide adaptability.

[0019] Compared with the existing monocular camera calculation method that offsets the height of the object by the height above the ground, the height calculation result error of the box volume measurement method provided by the present invention is smaller, which greatly improves the measurement accuracy and achieves the effect of low cost, high adaptability and high precision measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principle of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of the specification of the present disclosure.

[0021] Figure 1 A schematic diagram of an application scenario of a box measurement method according to some embodiments of the present disclosure is shown.

[0022] Figure 2 An interactive schematic diagram of tank volume measurement according to some embodiments of the present disclosure is shown.

[0023] Figure 3 The overall flow chart of the box volume measurement method according to some other embodiments of the present disclosure is shown.

[0024] Figure 4 An overall flow chart of a box volume measurement method according to some other embodiments of the present disclosure is shown.

[0025] Figure 5 The figure shows an overall flow chart of a box volume measurement method according to some further embodiments of the present disclosure.

[0026] Figure 6 The overall flow chart of the box volume measurement method of some other embodiments of the present disclosure is shown.

[0027] Figure 7 A schematic block diagram of the structure of an electronic device equipped with a box volume measuring device according to some embodiments of the present disclosure is shown.

[0028] Figure 8 The overall flow chart of the box volume measurement method with the computing unit as the execution body in some embodiments of the present disclosure is shown.

[0029] Fig. 9 A schematic diagram of a monocular camera and a calibration board is shown.

[0030] Fig.10A schematic diagram showing the establishment of a world coordinate system on the checkerboard pattern of a calibration plate is shown.

[0031] Fig.11 A schematic diagram showing the placement of boxes on the checkerboard pattern of the calibration plate.

[0032] Fig.12 A schematic diagram showing the vertices of the upper and lower surfaces of the box.

[0033] Fig.13 A schematic diagram showing the establishment of a world coordinate system at the vertices on the upper surface of the box. DETAILED DESCRIPTION

[0034] The present disclosure is further described in detail below in conjunction with the accompanying drawings and examples. It is understood that the specific examples described herein are only used to explain the relevant content, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0035] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0036] In the logistics industry, the Internet of Things and other related fields, the volume of express orders has grown rapidly. In order to effectively understand the express parcels, it is necessary to collect the size, volume, and weight of the express parcels during the process of entering and leaving the warehouse. It is relatively difficult to obtain the size information of the express parcels, and it is difficult to achieve low-cost and high-intelligence effects. In the prior art, the volume measurement of express parcels is achieved by using TOF 3D or binocular 3D depth cameras. The process of using TOF (Time of Flight) 3D depth cameras or binocular 3D depth cameras to measure the volume of express parcels mainly includes: obtaining depth information, image processing, three-dimensional reconstruction, object segmentation, volume calculation, precision optimization, etc. There are many steps to calculate, and it requires the participation of high-cost hardware equipment, consumes more computing resources, and cannot satisfy people's desire to reduce costs.

[0037] In the present disclosure, other cameras or devices similar to monocular cameras may be used, such as digital cameras, webcams, security cameras, industrial cameras, sports cameras, scientific professional cameras (such as Phase One's medium format cameras, Red Epic Dragon), computer vision cameras, etc. The calibration board mainly uses a checkerboard calibration board, that is, a checkerboard calibration board composed of black and white grids, which is easy to detect and identify corners. Other calibration boards with functions of correcting lens distortion, determining physical size and converting pixels may also be used, such as a circular network calibration board, an ArUco calibration board (ArUcoCalibration Board), a CharuCo calibration board (combining the characteristics of checkerboards and ArUco markers), a random pattern calibration board, a lidar calibration board, a custom calibration board (which can be calibrated using software tools such as Halcon), and the like. The computing unit may be a software computing unit in a general-purpose hardware device, or a computing unit that combines software and hardware, such as a central processing unit, a graphics processing unit, a dedicated integrated circuit, a field programmable gate array, a digital signal processor, a neural network processor, a system-on-chip, a high-performance computing unit (HPC, for example, a computing unit in a supercomputer containing a large number of CPUs and GPUs for performing large-scale parallel computing), a finite element computing unit, and the like.

[0038] Figure 1 A schematic diagram of an application scenario of a box measurement method of some embodiments of the present disclosure is shown. This application scenario is a basic application scenario M100. The basic application scenario M100 includes a calibration board 100, a monocular camera 110, and a computing unit 120. The calibration board 100 is used to provide a calibration board image. Various calibration board images can be set on the calibration board 100, such as: a circular network calibration board, an ArUco calibration board (ArUco Calibration Board), a CharuCo calibration board, a random pattern calibration board, a laser radar calibration board, a custom calibration board, etc. In the embodiments of the present disclosure, the calibration board image of a checkerboard pattern is mainly used as an example for description, but this does not mean that the calibration board image of the calibration board is limited to a checkerboard image. By establishing various coordinate systems on the calibration board 100, the projection conversion between the world coordinate system and the camera coordinate system can be realized, and then the volume of the object to be measured placed on the calibration board 100 can be measured, for example, in the embodiments of the present disclosure, the volume of a rectangular box is measured. The monocular camera 110 is used to obtain the calibration plate image on the calibration plate 100, and obtain the image information of the object to be measured placed on the calibration plate 100. In the present disclosure, the monocular camera 110 is used as an example to illustrate the measurement process of the box volume. The monocular camera 110 should be set near the calibration plate 100 to capture the full view of the calibration plate 100 and the complete image of the object to be measured (such as a box) placed on the calibration plate 100.

[0039] The computing unit 120 is connected to the monocular camera 110. Here, the connection relationship between the computing unit 120 and the monocular camera 110 can be either a wired connection or a wireless connection, as long as data and communication interaction can be achieved between the computing unit 120 and the monocular camera 110. The main task of the computing unit 120 is to perform calculations by acquiring the calibration plate image and the box image information. The computing unit 120 can calculate the length and width of the box placed on the calibration plate 100 through a projection conversion algorithm or model, and calculate the first height between the monocular camera 110 and the calibration plate 100, and then calculate the second height of the monocular camera 110 relative to the upper surface of the box. The height of the box is obtained by the height difference between the first and second heights. After obtaining the length, width and height of the box, the volume of the box is calculated.

[0040] Figure 2 An interactive schematic diagram of measuring the volume of a box in some embodiments of the present disclosure is shown. Figure 2 The detailed description is intended to provide a clearer understanding of the technical solution of the present disclosure and should not be considered as limiting the scope of protection of the present disclosure. In the process of implementing the technical idea of ​​the present disclosure, one or some steps may be omitted, or other alternatives may be adopted. Figure 1 , Figure 2 A memory 130 is added, but the memory 130 is not essential. For some computing units with built-in memories, the existence of the memory 130 can be ignored. The purpose of adding the memory 130 is to target special scenarios of some computing units that do not have storage functions.

[0041] Step S101, obtaining a calibration plate image: obtaining a calibration plate image on the calibration plate 100 through the monocular camera 110. In step S101, the camera coordinate system is the coordinate system where the monocular camera that captures the calibration plate image is located.

[0042] Step S102, calculate the first height: calculate the first height between the origin of the camera coordinate system and the calibration plate according to the calibration plate image, the monocular camera 110 obtains the corresponding calibration plate image by photographing the calibration plate 100, and sends the calibration plate image information to the calculation unit 120, so that the calculation unit 120 can perform calculations. One of the ways for the calculation unit 120 to calculate the first height is to use the PnP (Perspective-n-Point) algorithm calculation method to estimate the camera's posture from the 2D projection on the calibration plate image of the calibration plate 100, and obtain the first height between the monocular camera 110 and the calibration plate 100 according to the translation vector and rotation vector output by the PnP algorithm. After the calculation unit 120 calculates the first height, if it has a storage unit, it stores it by itself, and if it does not have a storage unit, it sends the first height to the memory 130 of the external device for storage.

[0043] Step S103, obtaining the box image: the monocular camera 110 captures the box on the calibration plate 100, and sends the acquired box image to the computing unit 120, so that the computing unit 120 performs a projection operation based on the box image information to obtain the length and width of the box, and the coordinates of the box vertices on the upper surface of the box.

[0044] Step S104, projection operation: The process of calculating the length and width of the box by the calculation unit 120 can be briefly described as establishing a coordinate system on the calibration plate 100, and then placing the box on the calibration plate 100 to obtain the pixel coordinates of the length and width vertices of the lower surface of the box corresponding to the calibration plate 100. Taking the calibration plate image of the checkerboard pattern as an example, the calculation unit 120 performs a projection operation based on the pixel coordinates of the length and width vertices of the box corresponding to the checkerboard pattern to obtain the length and width of the box in the world coordinate system of the calibration plate, wherein the world coordinate system of the calibration plate is established on the checkerboard pattern on the calibration plate 100. The length and width of the box can be obtained in step S104. Since the box is approximately in the shape of a rectangular parallelepiped, the area of ​​the bottom or top surface of the box can be obtained in step S104, which provides a basis for further calculating the volume value.

[0045] In step S104, the pixel coordinates of the length and width vertices of the box are projected into the world coordinate system of the calibration plate by using a projection operation: the pixel coordinates of the length and width vertices of the box are projected into the world coordinate system of the calibration plate according to the internal and external parameters of the monocular camera, and the pixel coordinates of the length and width vertices in the camera coordinate system are projected into the coordinates of the world coordinate system of the calibration plate on the calibration plate, thereby calculating the length and width of the box. The first translation vector and the first rotation vector are used to perform a projection operation, and the coordinates of the length and width vertices of the box are projected into the world coordinate system of the calibration plate to calculate the length and width of the box.

[0046] Step S105, calculate the second height: select a vertex on the upper surface of the box to establish a new world coordinate system, and define the three-dimensional coordinates of the four vertices on the upper surface of the box based on the new world coordinate system. Since the box is a rectangular parallelepiped, the bottom and top surfaces are both rectangles, so there are three vertices on the top surface in addition to the vertex used to establish the new world coordinate system, and obtain the coordinate values ​​of the remaining three vertices respectively. Through the PnP algorithm and combined with the camera's intrinsic parameters, distortion parameters, and the translation vector and rotation vector corresponding to the calibration plate image, and then through corresponding matrix transposition and other operations, the second depth information is obtained. The second depth information is the vertical information from the camera to the upper surface of the box, that is, the second height.

[0047] Step S106, calculating the volume of the box: The length, width and height of the box placed on the calibration plate 100 are finally obtained through the calculation from step S101 to step S105. According to the calculation formula of a rectangular parallelepiped, the volume formula of the box is volume = length * width * height, and the volume of the box can be finally obtained.

[0048] Figure 3 The overall process diagram of the box volume measurement method of some other embodiments of the present disclosure is shown. Figure 3 The box volume measurement method M200 includes step S201, step S202, step S203, and step S204. The box volume measurement method M200 can be executed by a general electronic device, or by a dedicated electronic device such as a dedicated integrated circuit, a field programmable gate array, a digital signal processor, a neural network processor, a system-level chip, etc.

[0049] Step S201, obtaining a calibration plate image, and calculating a first height between the origin of a camera coordinate system and the calibration plate according to the calibration plate image, where the camera coordinate system is a coordinate system where a monocular camera that captures the calibration plate image is located.

[0050] The calibration plate image of the calibration plate is obtained by a monocular camera. The monocular camera is usually an image acquisition device with an image shooting function. The embodiments of the present disclosure are described by taking a monocular camera as an example. It does not exclude the use of other image acquisition devices with an image shooting function. In some specific occasions, a device with a video acquisition function can also be used to collect video information, and then the video information is converted into usable image information by a known method. The monocular camera has its own coordinate system, and the position of the monocular camera is usually used as the origin of the coordinate system.

[0051] In this step, the monocular camera with known intrinsic parameters can be tilted and fixed relative to the calibration plate and an image of the checkerboard calibration plate can be taken. The translation vector and rotation vector of the monocular camera relative to the calibration plate can be obtained by using the calibration plate image and the PnP algorithm can be used. The first height of the origin of the camera coordinate system from the calibration plate can be calculated in combination with the camera extrinsic parameters. Calculating the first height can provide a height numerical basis for calculating the height of the box in the subsequent steps, realizing the function of obtaining depth information without a binocular camera or a depth camera, and making up for the defect of the monocular camera in the depth information acquisition function.

[0052] Step S202, obtaining a box image taken by a monocular camera, wherein the box is placed on a calibration board, performing a projection operation on the box image, and obtaining the length and width of the box in the world coordinate system of the calibration board and the coordinates of the box vertices on the upper surface of the box.

[0053] One of the purposes of obtaining the box image is to determine the vertices of the upper and lower surfaces of the box, and project each vertex obtained to the world coordinate system of the calibration plate through projection operation to obtain the corresponding coordinates. By performing projection operation on the coordinates of the obtained multiple vertices, the length and width vertices of the box are projected to the world coordinate system of the calibration plate, and the length and width of the box can be obtained, which provides a numerical basis for measuring the length and width of the box volume.

[0054] When using a monocular camera for projection calculation, the box should be placed on the calibration plate, and the box and the calibration plate should be kept in contact. At least seven vertex pixel coordinates of the box should be picked up, of which four are the vertex pixel coordinates of the upper surface of the box and three are the vertex pixel coordinates of the lower surface of the box. Then, the projection calculation is performed in combination with the intrinsic and extrinsic parameters of the monocular camera. When performing the projection calculation, the Zhang Zhengyou calibration model can be used for distortion correction to complete the projection transformation of the box vertices from pixel coordinates to the world coordinate system.

[0055] Since obtaining the box image through a monocular camera is a low-cost operation and does not require additional hardware investment, the length and width of the box can be easily obtained by performing projection calculations on the box image and establishing the calibration plate world coordinate system on the calibration plate.

[0056] Step S203, establishing a new world coordinate system with one of the vertices of the box as the origin, and obtaining a second height of the monocular camera relative to the upper surface of the box.

[0057] The vertices obtained from step S202 are used in step S203 to establish a new world coordinate system. In the new world coordinate system, the vertical distance of the monocular camera relative to the upper surface of the box is calculated using the vertices on the upper surface of the box, that is, the second height is obtained.

[0058] When a monocular camera is used as a tool for acquiring the calibration plate image, the third translation vector and the third rotation vector of the monocular camera relative to the new world coordinate system can be calculated based on the known camera intrinsic parameters and distortion parameters, and then the third translation vector and the third rotation vector are subjected to matrix operation to calculate the depth information of the monocular camera. The depth information here is the vertical distance of the monocular camera relative to the upper surface of the box. Step S203 achieves the purpose of obtaining the depth information without the need for relatively expensive equipment such as a binocular camera or TOF.

[0059] Step S204: taking the height difference between the first height and the second height as the height of the box, and calculating the volume of the box according to the length, width and height of the box.

[0060] In step S201 to step S203, the length and width of the box are obtained, and the height difference between the first and second heights is calculated to obtain the height of the box. Therefore, in step S204, the volume of the box can be easily calculated = length * width * height, and finally the volume of the box is obtained.

[0061] The implementation method provided by steps S201 to S204 only requires the calibration of the monocular camera with a fixed calibration plate. In addition to obtaining an image of the calibration plate, it is only necessary to take an image of the box to calculate the volume of the box. The calibration plate image and the box image are used to perform projection operations and coordinate transformation calculations to obtain the length, width and height of the box, where the height difference is calculated through the depth information in steps S201 and S203, and finally the volume of the box is obtained in step S204.

[0062] Figure 4 The overall flow chart of the box volume measurement method of other embodiments of the present disclosure is shown. Figure 4 In the illustrated embodiment, step S201 of the box volume measurement method M200 is further optimized and improved to form step S2011 and step S2021.

[0063] Step S2011, obtaining the origin of the camera coordinate system and the camera extrinsic parameters of the monocular camera, and obtaining a first translation vector and a first rotation vector of the monocular camera relative to the calibration plate through the calibration plate image;

[0064] Step S2012, converting the first rotation vector into a corresponding rotation matrix to obtain first depth information of the monocular camera, and determining a first height according to the first depth information, where the first height is specifically the height of the origin of the camera coordinate system from the calibration plate.

[0065] In step S2011, a calibration plate world coordinate system is established on the calibration plate, and the Z axis of the calibration plate world coordinate system is perpendicular to the horizontal plane. In step S2012, in order to obtain the first depth information, the origin of the chessboard pattern in the calibration plate image is first set, and the chessboard size is used as the coordinate distance reference of the X axis and Y axis of the world coordinate system. The three-dimensional coordinates of the chessboard corner points are defined by the origin of the chessboard pattern, and the corresponding chessboard corner point pixel coordinates are obtained. After obtaining the chessboard corner point pixel coordinates, the first translation vector and the first rotation vector of the monocular camera relative to the calibration plate can be calculated by combining the internal and external parameters of the monocular camera, and the first rotation vector is converted into a rotation matrix to obtain the first depth information of the monocular camera. Then further calculation is performed to make the first depth information zero, and the first height from the camera to the calibration plate plane is obtained according to the mapping relationship between the camera coordinate system and the world coordinate system.

[0066] Figure 5 The overall flow chart of the box volume measurement method of some other embodiments of the present disclosure is shown. Figure 5 In the illustrated embodiment, step S202 of the box volume measurement method M200 is further optimized and improved to form step S2021 and step S2022.

[0067] Step S2021, obtain the box image of the box placed on the calibration plate, and obtain the pixel coordinates of the length and width vertices of the lower surface of the box corresponding to the calibration plate.

[0068] Step S2022, perform projection operation on the pixel coordinates of the length and width vertices, project the pixel coordinates of the length and width vertices into the world coordinate system of the calibration plate, calculate the world coordinates of the length and width vertices of the box in the world coordinate system of the calibration plate, and obtain the length and width of the box.

[0069] Step S2021 acts on the lower surface of the box, and simultaneously obtains the pixel coordinates of the three vertices of the lower surface of the box, and then performs projection operation in combination with the translation vector and the rotation vector to "move" the object to the origin of the camera coordinate system, and find out the position of the object in the world coordinate system of the origin of the camera coordinate system, that is, let the origin of the camera coordinate system be represented by coordinates in the world coordinate system of the calibration plate. In the process of projection operation, the Zhang Zhengyou calibration model is used to obtain the intrinsic parameters and distortion parameters of the monocular camera through the chessboard pattern of the calibration plate, and the pixel coordinates of the length and width vertices of the box are projected and transformed into the world coordinate system of the calibration plate according to the intrinsic parameters and external parameters of the monocular camera, and the pixel coordinates of the length and width vertices are projected onto the coordinates of the world coordinate system of the calibration plate on the calibration plate, and finally the length and width of the box are calculated using the coordinates of the world coordinate system of the calibration plate.

[0070] Through the operation of step S2021 and step S2022, the length and width of the lower surface of the box are obtained. Since the box is generally rectangular, the length and width of the upper surface of the box are almost the same as the lower surface, so the calculation results obtained in step S2021 and step S2022 can be used in the volume calculation formula of the rectangle. In the calculation process from step S2021 to step S2022, the Zhang Zhengyou model is also applied for distortion correction, which minimizes the image distortion caused by camera distortion, thereby improving the box image quality and the accuracy of the image processing task.

[0071] Figure 6 The overall flow chart of the box volume measurement method of other embodiments of the present disclosure is shown. Figure 6 In the illustrated embodiment, step S203 of the box volume measurement method M200 is further optimized and improved to form step S2031 and step S2032.

[0072] Step S2031, selecting a vertex on the upper surface of the box to establish a new world coordinate system, and defining the three-dimensional coordinates of the four vertices on the upper surface of the box based on the new world coordinate system.

[0073] Step S2032: Calculate the second height of the monocular camera relative to the new world coordinate system based on the known camera intrinsic parameters and distortion parameters.

[0074] Steps S2031 to S2032 establish a new world coordinate system at a vertex on the upper surface of the box, and then define the coordinates of the remaining three vertices on the upper surface through the new world coordinate system, and obtain the coordinate values ​​of the remaining three vertices respectively. Then, by obtaining the corresponding translation vector and rotation vector and rotation operation, the second depth information of the monocular camera is obtained, where the second depth information is the second height in step S203.

[0075] At this point, the three elements for calculating the volume of the box: length, width and height have all been calculated, and the volume of the box can be calculated in the subsequent step S204.

[0076] Figure 7 It is a schematic block diagram of the structure of an electronic device equipped with the box volume measuring device of the present disclosure according to one embodiment of the present disclosure.

[0077] The box volume measuring device includes:

[0078] A first height calculation module 1001 is used to obtain a calibration plate image and calculate a first height between the origin of a camera coordinate system and the calibration plate according to the calibration plate image, wherein the camera coordinate system is a coordinate system where a monocular camera that captures the calibration plate image is located;

[0079] The projection operation module 1002 obtains a box image taken by the monocular camera, wherein the box is placed on the calibration board, and performs a projection operation on the box image to obtain the length and width of the box in the world coordinate system of the calibration board and the coordinates of the box vertices on the upper surface of the box;

[0080] A second height calculation module 1003 establishes a new world coordinate system with one of the vertices of the box as the origin to obtain a second height of the monocular camera relative to the upper surface of the box;

[0081] The box volume calculation module 1004 uses the height difference between the first height and the second height as the height of the box, and calculates the volume of the box according to the length, width and height of the box.

[0082] The box volume measuring device may be in the form of computer software, and each module included therein may be in the form of software modules.

[0083] The implementation process of the functions and effects of each module in the above box volume measurement device can be specifically seen in the implementation process of the corresponding steps in the box volume measurement method M200, which will not be repeated here. The box volume measurement device can be stored in the memory 1300 of the electronic device 1000 in the form of software.

[0084] The hardware structure of the electronic device 1000 can be implemented using a bus architecture. The bus architecture can include any number of interconnected buses and bridges, depending on the specific application and overall design constraints of the hardware. The bus 1100 connects various circuits including one or more processors 1200, memory 1300 and / or hardware modules together. The bus 1100 can also connect various other circuits 1400 such as peripherals, voltage regulators, power management circuits, external antennas, etc.

[0085] The bus 1100 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the figure only uses one connecting line, but does not mean that there is only one bus or one type of bus.

[0086] The processor 1200 may be a central processing unit (CPU). The processor 1200 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0087] The memory 1300 can be used as a non-transitory computer-readable storage medium, and can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions of the computer program in the embodiment of the present disclosure. The processor 1200 implements the vehicle signal processing method by running the non-transitory software programs, instructions and modules stored in the memory 1300.

[0088] The memory 1300 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created by the processor 1200, such as structural data, model data, sample data, stiffness evaluation results, etc. of the bridge. In addition, the memory 1300 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1300 may optionally include a memory remotely arranged relative to the processor 1200, and these remote memories may be connected to the processor 1200 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0089] Figure 8 A schematic diagram of the overall flow of a box volume measurement method corresponding to a calculation unit (which may be a processor 1200 of an electronic device 1000 equipped with the above-mentioned box volume measurement device) in some embodiments of the present disclosure is shown. Figure 8 The method for measuring the box volume by a computing unit includes steps S301, S302, S303, and S304. The box volume measurement method M300 can be executed by general electronic devices such as computers, or by special electronic devices such as dedicated integrated circuits, field programmable gate arrays, digital signal processors, neural network processors, and system-level chips.

[0090] In step S301, a calculation unit receives a calibration plate image sent from a monocular camera, and calculates a first height between the origin of a camera coordinate system and the calibration plate according to the calibration plate image. The camera coordinate system is a coordinate system where the monocular camera that captures the calibration plate image is located.

[0091] In step S302, the computing unit receives a box image taken by a monocular camera, wherein the box is placed on a calibration plate, and performs a projection operation on the box image to obtain the length and width of the box in the world coordinate system of the calibration plate and the coordinates of the box vertices on the upper surface of the box.

[0092] Step S303: The calculation unit establishes a new world coordinate system with one of the vertices of the box as the origin to obtain a second height of the monocular camera relative to the upper surface of the box.

[0093] Step S304: the calculation unit uses the height difference between the first height and the second height as the height of the box, and calculates the volume of the box according to the length, width and height of the box.

[0094] Steps S301 to S304 provided by the box volume measurement method M300 use the calculation unit 120 as the execution body. The calculation unit 120 receives image information from the monocular camera 110. The image information sent by the monocular camera 110 includes the calibration plate image of the calibration plate 100 and the box placed on the calibration plate 100. The calculation unit performs projection calculations and coordinate transformations to obtain the length and width of the box, and then obtains the height of the box through the height difference between the first and second heights, wherein the height difference is obtained through the depth information in steps S301 and S303, and finally the volume of the box is obtained in step S304.

[0095] Figures 9 to 13 The embodiment of the box volume measurement method of the present disclosure in some specific application scenarios is shown. The specific steps of the method for measuring the volume of a cuboid based on a monocular camera provided by the present disclosure can be summarized as follows:

[0096] Firstly, the monocular camera with known intrinsic parameters is tilted and fixed relative to the calibration plate, and a calibration plate image of the checkerboard calibration plate is taken. The translation vector and rotation vector of the monocular camera relative to the calibration plate are obtained by using the PnP algorithm through the calibration plate image. Since the Z axis of the world coordinate system established on the calibration plate is perpendicular to the horizontal ground, the height of the origin of the monocular camera coordinate system from the calibration plate can be calculated according to the extrinsic parameters of the monocular camera.

[0097] Then, ensure that the positions of the monocular camera and the calibration plate remain unchanged, place the measured cuboid (for example, a cuboid box) on the calibration plate, capture an image containing the length, width, height corresponding to a lower vertex of the cuboid and the four upper vertices, and pick up the pixel coordinates of the seven vertices of the cuboid involved.

[0098] According to the pixel coordinates of the length and width vertices of the picked cuboid close to the lower surface of the calibration plate, a projection operation is performed in combination with the internal and external parameters of the monocular camera, and the length and width vertices of the cuboid are projected onto the world coordinate system of the calibration plate. The length and width dimensions of the cuboid can be calculated by the coordinates of the length and width vertices of the cuboid in the world coordinate system obtained by projection.

[0099] In the previous step, the length and width of the cuboid are known. Since the upper and lower surfaces of the cuboid are parallel, that is, parallel to the plane of the calibration plate, the PnP algorithm is calculated again according to the four vertices of the upper surface of the cuboid picked up to obtain the translation vector and rotation vector of the monocular camera relative to the upper surface of the cuboid. Then, the height of the monocular camera relative to the upper surface of the measured cuboid can be calculated. At this time, the height of the cuboid is the height of the camera from the calibration plate minus the height of the camera from the upper surface of the cuboid.

[0100] Given the length, width and height, we can finally find the volume of the cuboid.

[0101] From the above records, it can be known that in the embodiment of the present invention, only a single camera needs to be calibrated with a fixed calibration plate, and the volume of a rectangular object can be calculated through a photo. Compared with binocular 3D and TOF 3D, the implementation cost is lower, the requirements for equipment computing power are relatively low, the scene is easy to build, and the measurement efficiency is high.

[0102] Since the monocular camera and the calibration plate will be calibrated with external parameters after being fixed, there are no special requirements for the distance of the camera relative to the calibration plate and the size of the calibration plate's checkerboard. It is more flexible and can be freely adjusted according to the size of the measurement site and the size of the box to be tested.

[0103] Compared with the existing monocular camera calculation method that offsets the height of the object by the height above the ground, this aspect has a smaller height calculation error, greatly improving the measurement accuracy.

[0104] exist Fig. 9 In the method, a monocular camera with known intrinsic parameters is tilted and fixed at a certain pitch angle parallel to the horizontal viewing angle of a checkerboard calibration plate of known size, and an image of the calibration plate is taken to obtain the translation vector and rotation vector of the camera relative to the calibration plate in this posture, and then the vertical height of the origin of the camera coordinate system of the monocular camera from the plane of the calibration plate is calculated.

[0105] exist Fig.10In , the PnP algorithm is used to measure the translation vector and rotation vector of the camera relative to the calibration plate, and a world coordinate system is established on the calibration plate. The Z axis is perpendicular to the horizontal working surface and upward, with the corner point (0,0) of the checkerboard calibration plate as the origin O, the horizontal direction along the checkerboard corner point as the X axis, and the vertical direction as the Y axis. Then, based on the known checkerboard size w as the reference for the x-axis and y-axis coordinate distance of the world coordinate system, let z = 0, starting from (0,0,0) through (w,0,0)... to (n*w,n*w,0) and ending, the three-dimensional coordinates of the n*n checkerboard corner points are defined respectively. The Harris corner detection algorithm is further used to obtain the pixel coordinates of the corresponding checkerboard corner points, and then the translation vector tvec and rotation vector rvec of the camera relative to the calibration plate are calculated by iteration based on the known camera intrinsic parameters and distortion parameters, and then the rotation vector rvec is converted into a rotation matrix R, and then the depth information P of the camera is calculated. cam .

[0106] P cam =R*P world +tvec

[0107] Among them, P cam Represents the coordinates of the object in the camera coordinate system, P world Represents the coordinates of the object in the world coordinate system, R and the rotation vector rvec represent the transfer of the target point from the world coordinate system to the camera coordinate system. The PnP algorithm calculates exactly this mapping relationship.

[0108] Make P cam = 0, it means that the object "moves" to the origin of the camera coordinate system, and the obtained P world Represents the position of the origin of the camera coordinate system in the world coordinate system.

[0109] 0=R*P world +tvec

[0110] P world = -inverse(R)*tvec

[0111] P world The Z-axis coordinate is the depth information, that is, the vertical distance depth1 from the camera to the plane of the calibration plate, that is, the first height.

[0112] After obtaining the vertical distance depth1 (first height) of the calibration plate plane, ensure that the position of the monocular camera and the calibration plate remains unchanged, place the measured cuboid on the calibration plate, and capture an image containing the length, width, height and four upper vertices corresponding to a lower vertex of the cuboid. Pick up the pixel coordinates of the seven vertices of the involved cuboid, namely, vertex a, vertex b, vertex c, vertex d, vertex e, vertex f and vertex g.

[0113] According to the pixel coordinates of the length and width vertices of the picked cuboid close to the lower surface of the calibration plate, namely vertices e, f and g, a projection operation is performed in combination with the intrinsic parameters of the monocular camera and the previously calculated extrinsic parameters (including the translation vector and the rotation vector), and the length and width vertices of the cuboid are projected onto the world coordinate system O-XYZ of the calibration plate. The length width and width height of the cuboid can be calculated by the coordinates of the length and width vertices of the cuboid in the world coordinate system obtained by projection.

[0114] Then, distortion correction is performed according to Zhang Zhengyou's calibration model. The calculation formula of Zhang Zhengyou's calibration model is as follows:

[0115] x dist =x(1+k1r 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 )

[0116] y dist =y(1+k1r 2 +k2r 4 +k3r 6 )+p1(r 2 +2y 2 )+2p2xy

[0117] Where (x dist ,y dist ) is the coordinate of the pixel point in the distorted image, (x, y) is the coordinate of the corresponding pixel point in the dedistorted image, k1, k2, k3 are the radial distortion coefficients, p1, p2 are the tangential distortion coefficients, and r is the distance from the center of the image to the point.

[0118] Then, based on the camera's intrinsic and extrinsic parameters, the length and width vertices of the box are projected from the pixel coordinate system to the world coordinate system (O-XYZ).

[0119] The conversion formula from pixel coordinates to world coordinates is:

[0120]

[0121] It can be concluded that:

[0122] Z c =X w *R 31 +Y w *R 32 +Z w *R 33 +T3

[0123] Where (x, y) is the pixel coordinate point, is the known camera intrinsic parameter matrix, To obtain the camera external parameter matrix, R is the rotation matrix, T is the translation matrix, (X w , Y w , Z w ) is the world coordinate point, and R in the camera extrinsic matrix ij (From R 11 to R 33 ) describes how each axis in the world coordinate system is projected onto an axis in the camera coordinate system.

[0124] Based on the premise of establishing the world coordinate system, Z w =0, the formula can be simplified to:

[0125] Z c =X w *R 31 +Y w *R 32 +T3

[0126] According to the obtained Z c , the external parameters of the camera's relative position to the calibration plate, the internal parameters of the known camera, and the length and width vertices of the captured cuboid close to the calibration plate, the world coordinate system coordinates (X we , Y we ,0)、(X wf , Y wf ,0)、(X wg , Y wg , 0).

[0127] Then the length of the object being measured is:

[0128]

[0129] width:

[0130]

[0131] According to the pixel coordinates of the four vertices on the upper surface of the measured object and the obtained side length, a new world coordinate system O′-X′Y′Z′ is re-established with point a as the origin O′.

[0132] Let Z′=0, and define the three-dimensional coordinates of the four points on the upper surface of the object to be measured as a(0,0,0), b(width,0,0), c(0,height,0), and d(width,height,0). Given the pixel coordinates of the four points and the corresponding world coordinate system coordinates, use the PnP iterative algorithm again to calculate the translation vector tvec′ and rotation vector rvec′ of the camera relative to the world coordinate system where the upper surface of the object to be measured is located based on the known camera intrinsic parameters and distortion parameters, and then convert the rotation vector rvec′ into a rotation matrix R′ to calculate the depth information P of the camera. world ′:

[0133] P world ′=-inverse(R′)*tvec′

[0134] P world The Z-axis coordinate of ′ is the depth information, that is, the vertical distance depth2 from the camera to the upper surface plane of the measured cuboid, that is, the second height.

[0135] Finally, the volume V of the measured cuboid is obtained:

[0136] V=width*height*(depth1-depth2)

[0137] The box volume measuring device involved in any embodiment of the present disclosure can be configured in the electronic device described above in the present disclosure. The box volume measuring device is used in the electronic device to measure the volume of a rectangular box transported or transferred via other components.

[0138] It should be understood and comprehended that in the box volume measurement method, device, system and electronic device provided in the present disclosure, the so-called rectangle refers to a rectangle in the engineering sense, not a rectangle strictly defined in the mathematical geometry sense. Any box that can be identified as a rectangle by equipment and devices in the prior art is applicable to the box volume measurement method, device, system and electronic device provided in the present disclosure.

[0139] The present disclosure also provides a readable storage medium, in which a computer program is stored, and the computer program is used to implement the above method when executed by a processor. "Readable storage medium" can be any device that can contain, store, communicate, propagate or transmit a program for use in an instruction execution system, device or equipment or in combination with these instruction execution systems, devices or equipment. More specific examples of readable storage media include the following: an electrical connection portion (electronic device) with one or more wirings, a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable read-only memory (CDROM), etc.

[0140] The present disclosure also provides a computer program product. The method of the present disclosure can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed, the process or function of the present disclosure is executed in whole or in part.

[0141] The computer program or instructions may be stored in a readable storage medium or transmitted from one readable storage medium to another readable storage medium, for example, the computer program or instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless means. The readable storage medium may be any available medium that can be accessed or a data storage device such as a server, data center, etc. that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; it may also be an optical medium, such as a digital video disk; it may also be a semiconductor medium, such as a solid state drive. The computer readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0142] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0143] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0144] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0146] In the description of this specification, the description with reference to the terms "one embodiment / method", "some embodiments / methods", "example", "specific example", or "some examples" etc. means that the specific features, structures, or characteristics described in conjunction with the embodiment / method or example are included in at least one embodiment / method or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment / method or example. Moreover, the specific features, structures, or characteristics described may be combined in any one or more embodiments / methods or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments / methods or examples described in this specification and features of different embodiments / methods or examples, unless they are contradictory.

[0147] Those skilled in the art should understand that the above embodiments are only for the purpose of clearly illustrating the present disclosure, and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications may be made based on the above disclosure, and these changes or modifications are still within the scope of the present disclosure.

Claims

1. A method for measuring box volume, characterized in that: include: Acquire a calibration plate image, and calculate a first height between the origin of a camera coordinate system and the calibration plate according to the calibration plate image, wherein the camera coordinate system is a coordinate system where a monocular camera that captures the calibration plate image is located; Acquire a box image taken by the monocular camera, wherein the box is placed on the calibration plate, perform a projection operation on the box image, and obtain the length and width of the box in the world coordinate system of the calibration plate and the coordinates of the box vertices on the upper surface of the box; A new world coordinate system is established with one of the vertices of the box as the origin to obtain the second height of the monocular camera relative to the upper surface of the box; The height difference between the first height and the second height is taken as the height of the box, and the volume of the box is calculated according to the length, width and height of the box.

2. The box volume measurement method according to claim 1, characterized in that: The acquiring the calibration plate image and calculating the first height between the origin of the camera coordinate system and the calibration plate according to the calibration plate image further includes: Obtain the origin of the camera coordinate system and the camera extrinsic parameters of the monocular camera, and obtain the first translation vector and the first rotation vector of the monocular camera relative to the calibration plate through the calibration plate image; The first rotation vector is converted into a corresponding rotation matrix to obtain first depth information of the monocular camera, and a first height is determined according to the first depth information, where the first height is specifically the height of the origin of the camera coordinate system from the calibration plate.

3. The box volume measurement method according to claim 2, characterized in that: The converting the first rotation vector into a corresponding rotation matrix to obtain first depth information of the monocular camera further includes: Establishing a calibration plate world coordinate system on the calibration plate, wherein the Z axis of the calibration plate world coordinate system is perpendicular to the horizontal plane; The calibration plate image is a checkerboard image, the origin of the checkerboard image is set, the size of the checkerboard is used as the coordinate distance reference of the X-axis and Y-axis of the world coordinate system, the three-dimensional coordinates of the checkerboard corner points are defined by the origin of the checkerboard pattern, and the pixel coordinates of the corresponding checkerboard corner points are obtained; Calculate the first translation vector and the first rotation vector of the monocular camera relative to the calibration plate, convert the first rotation vector into a rotation matrix, and calculate the first depth information of the camera; The first depth information is set to zero, and a first height from the camera to the plane of the calibration plate is obtained according to a mapping relationship between the camera coordinate system and the world coordinate system of the calibration plate.

4. The box volume measurement method according to claim 3, characterized in that: The method further comprises: obtaining a box image captured by the monocular camera, wherein the box is placed on the calibration plate, performing a projection operation on the box image, and obtaining the length and width of the box in the world coordinate system of the calibration plate and the coordinates of the box vertices on the upper surface of the box. Obtain the box image of the box placed on the calibration plate, and obtain the pixel coordinates of the length and width vertices of the lower surface of the box corresponding to the calibration plate; The pixel coordinates of the length and width vertices are projected into the world coordinate system of the calibration plate, and the world coordinates of the length and width vertices of the box are calculated in the world coordinate system of the calibration plate to obtain the length and width of the box.

5. The box volume measurement method according to any one of claims 1 to 4, characterized in that: The projection operation is performed on the pixel coordinates of the length and width vertices, the pixel coordinates of the length and width vertices are projected into the world coordinate system of the calibration plate, the pixel coordinates of the length and width vertices of the box are calculated in the world coordinate system of the calibration plate, and the length and width of the box are obtained, further comprising: Use Zhang Zhengyou's calibration model to obtain the intrinsic parameters and distortion parameters of the monocular camera through the chessboard pattern of the calibration plate; According to the internal and external parameters of the monocular camera, the pixel coordinates of the length and width vertices of the box are projected into the world coordinate system of the calibration plate, and the pixel coordinates of the length and width vertices in the camera coordinate system are projected into the world coordinate system of the calibration plate. Calculate the length and width of the box using the coordinates of the world coordinate system of the calibration plate; Optionally, the pixel coordinates of the length and width vertices of the box are projected and transformed into the world coordinate system of the calibration plate according to the intrinsic and extrinsic parameters of the monocular camera, and the pixel coordinates of the length and width vertices in the camera coordinate system are projected into the world coordinate system of the calibration plate, specifically: Performing a projection operation by using the first translation vector and the first rotation vector to project the pixel coordinates of the length and width vertices of the box into the world coordinate system of the calibration plate; The length and width of the box are calculated in the world coordinate system of the calibration plate using the vertex pixel coordinates of the length and width of the box in the world coordinate system of the calibration plate after projection transformation, combined with the external parameters of the monocular camera relative to the calibration plate and the known internal parameters of the camera. Optionally, the step of establishing a new world coordinate system with one of the vertices of the box as the origin to obtain a second height of the monocular camera relative to the upper surface of the box further includes: A vertex on the upper surface of the box is selected to establish a new world coordinate system, and three-dimensional coordinates of four vertices on the upper surface of the box are defined based on the new world coordinate system; Calculate the second height of the monocular camera relative to the new world coordinate system based on the known camera intrinsic parameters and distortion parameters; Optionally, in the process of establishing a new world coordinate system for a vertex on the upper surface of the box, coordinates of the remaining three vertices on the upper surface of the box are defined to obtain pixel coordinate values ​​of the remaining three vertices respectively; A third translation vector and a third rotation vector are obtained through the calibration plate image, and a matrix transposition operation is performed on the third rotation vector to convert it into a corresponding rotation matrix. The rotation matrix is ​​multiplied by the third translation vector to obtain second depth information of the monocular camera, where the second depth information is specifically a second height.

6. A box volume measuring device, characterized in that: include: A first height calculation module, used for acquiring a calibration plate image, and calculating a first height between the origin of a camera coordinate system and the calibration plate according to the calibration plate image, wherein the camera coordinate system is a coordinate system where a monocular camera that captures the calibration plate image is located; A projection operation module is used to obtain a box image taken by the monocular camera, wherein the box is placed on the calibration board, and perform a projection operation on the box image to obtain the length and width of the box in the world coordinate system of the calibration board and the coordinates of the box vertices on the upper surface of the box; A second height calculation module is used to establish a new world coordinate system with one of the vertices of the box as the origin, and obtain a second height of the monocular camera relative to the upper surface of the box; The box volume calculation module uses the height difference between the first height and the second height as the height of the box, and calculates the volume of the box according to the length, width and height of the box.

7. A box volume measurement system, characterized in that: include: A monocular camera, used for capturing images and arranged to be tilted relative to the calibration plate; A calculation unit, used for calculating the size of the box; The calculation unit obtains a calibration plate image, and calculates a first height between the origin of a camera coordinate system and the calibration plate according to the calibration plate image, wherein the camera coordinate system is a coordinate system where a monocular camera that captures the calibration plate image is located; The computing unit acquires a box image taken by the monocular camera, wherein the box is placed on the calibration board, performs a projection operation on the box image, and obtains the length and width of the box in the world coordinate system of the calibration board and the coordinates of the box vertices on the upper surface of the box; The computing unit establishes a new world coordinate system with one of the vertices of the box as the origin, and obtains a second height of the monocular camera relative to the upper surface of the box; The calculation unit uses the height difference between the first height and the second height as the height of the box, and calculates the volume of the box according to the length, width and height of the box.

8. An electronic device, characterized in that: include: A memory storing execution instructions; as well as A processor, wherein the processor executes the execution instructions stored in the memory, so that the processor executes the box volume measurement method according to any one of claims 1 to 5.

9. A readable storage medium, characterized in that: The readable storage medium stores execution instructions, which are used to implement the box volume measurement method according to any one of claims 1 to 5 when executed by the processor.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the box volume measurement method according to any one of claims 1 to 5 is implemented.

Images