Method, device, electronic device and computer program product for measuring loading volume

Abstract

The present application is applicable to the field of transportation and logistics technology, and provides a method, device, electronic device and computer program product for measuring loading volume. The method includes: when the loading state of the carriage is in the ready state, judging whether the cargo exceeds the target coverage area based on the depth of the carriage point cloud collected by the point cloud acquisition device; if the cargo does not exceed the target coverage area, the carriage point cloud is used as the target point; if the cargo exceeds the target coverage area, based on the carriage image collected by the image acquisition device, the pixel point of the cargo area in the carriage image is determined as the target point; based on the depth of the target point, the loading volume of the carriage is determined. The present application can realize the real-time measurement of the loading volume during the loading process, and improve the efficiency of loading volume measurement.

Description

Technical Field

[0001] The present application belongs to the field of transportation and logistics technology, and in particular, relates to a method, device, electronic equipment and computer program product for measuring loading volume. Background Art

[0002] With the development of the logistics industry, intelligent logistics is increasingly being used in basic activities such as transportation, warehousing, distribution, packaging, loading and unloading in the logistics industry. In the process of trucks carrying goods, intelligent measurement of the loading rate of truck compartments is one of the inevitable requirements for the logistics industry to improve efficiency. When calculating the loading rate, it is usually necessary to measure the loading volume of the compartment (that is, the volume of loaded goods). When measuring the loading volume, the relevant technology first collects the empty truck as a template, and then collects and aligns it again after the cargo is fully loaded to calculate the volume change. This solution cannot achieve real-time measurement during the loading process, and requires at least two collection and measurement. The operation is cumbersome and the measurement efficiency is low. Summary of the invention

[0003] The embodiments of the present application provide a method, device, electronic device and computer program product for measuring loading volume, which can realize real-time measurement of loading volume during loading and improve the measurement efficiency of loading volume.

[0004] In a first aspect, an embodiment of the present application provides a method for measuring a loading volume, which is applied to an electronic device, and the method includes:

[0005] When the loading state of the carriage is the ready state, judging whether the cargo exceeds the target coverage area based on the depth of the carriage point cloud collected by the point cloud collection device, the target coverage area is the maximum coverage range of the field of view of the point cloud collection device on the lower side wall of the carriage, and the field of view of the point cloud collection device covers a part of the area in the carriage;

[0006] If the cargo does not exceed the target coverage area, the carriage point cloud is used as the target point;

[0007] If the cargo exceeds the target coverage area, the pixel point of the cargo area in the carriage image captured by the image acquisition device is determined as the target point, and the field of view of the image acquisition device covers the entire area in the carriage;

[0008] Based on the depth of the target point, the loading volume of the carriage is determined.

[0009] In an embodiment of the present application, when the loading status of the car is in the ready status, it can be judged whether the cargo exceeds the maximum coverage range of the field of view of the point cloud acquisition device on the lower side wall of the car (i.e., the target coverage area) based on the depth of the car point cloud acquired by the point cloud acquisition device, and the field of view of the point cloud acquisition device covers part of the area in the car. If the cargo does not exceed the target coverage area, the car point cloud is used as the target point. If the cargo exceeds the target coverage area, the pixel point of the area where the cargo is located in the car image acquired by the image acquisition device is used as the target point, and the field of view of the image acquisition device covers the entire area in the car. The loading volume of the car can be determined based on the depth of the target point. This solution can realize real-time measurement of the loading volume during the loading process by measuring the loading volume of the carriage when the loading status of the carriage is in the ready status. Through the complementary installation of point cloud acquisition equipment and image acquisition equipment, the far end (i.e., the area in the carriage that is far away from the electronic equipment) is mainly covered by the point cloud acquisition equipment, and the near end (i.e., the area in the carriage that is close to the electronic equipment) is mainly covered by the image acquisition equipment. There is no need to cover the entire carriage with point clouds. On the basis of saving hardware deployment costs, the loading volume can be measured fully automatically and without any sense during the loading process. There is no need to collect empty car data in advance. The operation is relatively simple, which improves the measurement efficiency of the loading volume.

[0010] In a second aspect, an embodiment of the present application provides a device for measuring a loading volume, which is applied to an electronic device, and the device includes:

[0011] A cargo position determination module, used to determine whether the cargo exceeds a target coverage area based on the depth of the point cloud of the carriage collected by the point cloud collection device when the loading state of the carriage is a ready state, wherein the target coverage area is the maximum coverage range of the field of view of the point cloud collection device on the lower side wall of the carriage, and the field of view of the point cloud collection device covers a part of the area in the carriage;

[0012] A target point determination module, configured to use the carriage point cloud as a target point if the cargo does not exceed the target coverage area;

[0013] An image processing module, configured to use the pixel points of the area where the goods are located in the carriage image captured by an image acquisition device as the target point if the goods exceed the target coverage area, and the field of view of the image acquisition device covers the entire area in the carriage;

[0014] The loading volume determination module is used to determine the loading volume of the carriage based on the depth of the target point.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the method described in the first aspect above.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method described in the first aspect above is implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, wherein when the computer program is executed, the method described in the first aspect above is executed.

[0018] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 is a schematic diagram of a system for measuring a loading volume provided in an embodiment of the application;

[0021] Figure 2 is a diagram showing an example of the structure of an electronic device;

[0022] Figure 3 This is a top view example diagram of a point cloud acquisition device and an image acquisition device;

[0023] Figure 4 is a schematic flow chart of a method for measuring a loading volume provided in an embodiment of the present application;

[0024] Figure 5 is a diagram showing an example of deployment of electronic equipment;

[0025] Figure 6 is an example image of a carriage;

[0026] Figure 7 is another example of a carriage image;

[0027] Figure 8 This is a side view of a vehicle;

[0028] Fig. 9This is an example diagram of the first square;

[0029] Fig.10 is a schematic structural diagram of a loading volume measuring device provided in an embodiment of the present application;

[0030] Fig.11 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0032] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0033] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0034] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0035] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0036] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0037] The loading volume measurement method provided in the embodiment of the present application can be applied to electronic devices such as mobile phones, tablet computers, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiment of the present application does not impose any restrictions on the specific types of electronic devices.

[0038] For example, Figure 1 Schematic diagram of a system for measuring loading volume provided in an embodiment of the present application is shown. Figure 1 , the measurement system includes an electronic device 101 and a vehicle 102 .

[0039] The electronic device 101 can automatically complete the measurement of the loading volume of the vehicle 102's compartment through the complementary installation of the point cloud acquisition device and the image acquisition device without modifying the compartment or accessing an external system. The specific implementation scheme can be found in the loading volume measurement method embodiment below.

[0040] As an example but not limitation, the point cloud acquisition device may be a time-of-flight (ToF) camera, a laser radar, or other device capable of acquiring point clouds. The image acquisition device may be a wide-angle camera, a fixed-focus camera, or other device capable of acquiring visible light images.

[0041] The field of view of the point cloud acquisition device covers part of the area in the carriage, and the field of view of the image acquisition device covers the entire area in the carriage. The far end of the carriage is mainly covered by the point cloud acquisition device, and the near end is mainly covered by the image acquisition device. It is not necessary for the entire carriage of the vehicle 102 to be covered with point clouds. On the basis of saving the deployment cost of hardware, it is possible to realize the fully automatic and non-contact measurement of the loading volume during the loading process, and there is no need to collect the empty car in advance. The operation is relatively simple, which improves the measurement efficiency of the loading volume.

[0042] In order to achieve accurate measurement of the loading volume of the compartment of the vehicle 102, in one embodiment, the point cloud acquisition device and the image acquisition device can be integrated into the electronic device 101, and the axis of the optical axis of the point cloud acquisition device and the axis of the optical axis of the image acquisition device are parallel, ensuring that the deployment positions of the point cloud acquisition device and the image acquisition device in the electronic device 101 are as close as possible to avoid measurement errors caused by structural deformation. Of course, it is understandable that the point cloud acquisition device and the image acquisition device may not be integrated into the electronic device 101, and this application does not limit this.

[0043] In order to ensure that the point cloud acquisition device and the image acquisition device in the electronic device 101 can accurately measure the loading volume, the deployment position of the electronic device 101 can be determined based on the height of the lower side wall (i.e., the bottom surface of the compartment) and the height of the upper side wall (i.e., the top surface of the compartment) of the vehicle 102 (for example, if the height between the lower side wall and the upper side wall of the compartment is 5 meters, then the electronic device 101 can be deployed at a position about 2 meters from the lower side wall of the compartment), and the axis of the optical axis of the point cloud acquisition device and the axis of the optical axis of the image acquisition device are parallel to the longitudinal direction of the compartment (i.e., the length direction of the compartment). Since the depth measured by the point cloud acquisition device is relatively accurate, when deploying the electronic device 101, in the length direction of the compartment, the maximum coverage area of ​​the field of view of the point cloud acquisition device on the lower side wall of the compartment is kept to be more than half of the length of the compartment. For a shorter compartment, the length of the coverage area of ​​the point cloud acquisition device can be kept unchanged to shorten the length of the coverage blind area of ​​the point cloud acquisition device. On this basis, this solution can ensure the measurement accuracy and scene adaptability of the loading volume at a limited cost, adapt to multi-sized carriages, and realize the measurement of the loading volume of carriages of different sizes.

[0044] like Figure 2 The figure shows an example structure of the electronic device 101. The electronic device 101 further includes a computing chip, which may be a processor. The computing chip is used to measure the loading volume of the carriage based on the carriage point cloud collected by the point cloud collection device and the carriage image collected by the image collection device.

[0045] like Figure 3 Shown is a top view example of a point cloud acquisition device and an image acquisition device. Figure 3 The field of view of the point cloud acquisition device is smaller than that of the image acquisition device, so that the far end of the car can be mainly covered by the point cloud acquisition device, and the near end can be mainly covered by the image acquisition device. There is no need for the entire car 102 to be covered with point clouds, saving hardware deployment costs.

[0046] See also Figure 4 , Figure 4The flow chart of the method for measuring the loading volume provided in the embodiment of the present application is shown, which is applied to an electronic device. As an example but not limitation, the method includes the following steps:

[0047] Step 401, when the loading state of the carriage is in the ready state, it is determined whether the cargo exceeds the target coverage area based on the depth of the carriage point cloud collected by the point cloud collection device.

[0048] The target coverage area is the maximum coverage range of the field of view of the point cloud acquisition device on the lower wall of the carriage, and the field of view of the point cloud acquisition device covers part of the area in the carriage. The above-mentioned carriage point cloud is the point cloud collected by the point cloud acquisition device. In this embodiment, the point cloud acquisition device performs point cloud acquisition on the carriage, so the point cloud collected by the point cloud acquisition device can be called the carriage point cloud.

[0049] like Figure 5 The following is an example diagram of the deployment of electronic equipment. Figure 5 The tof area in the figure is the target coverage area, and the camera area is the coverage blind area of ​​the point cloud acquisition device.

[0050] The above-mentioned ready state may include a loading start state and a loading in progress state. In this embodiment, there is no need to distinguish between the loading start state and the loading in progress state. As long as the loading state of the carriage is detected to be the ready state, the task of measuring the loading volume of the carriage can be automatically triggered, thereby realizing real-time measurement of the loading volume during the entire loading process.

[0051] As an example and not limitation, when the loading status of the carriage is in the ready state, the electronic device measures the loading volume of the carriage at preset time intervals and reports the measured loading volume in real time (for example, reports to the user device so that the user can understand the loading progress of the goods).

[0052] In a possible implementation, a method for determining that the loading state of the carriage is a ready state includes:

[0053] Based on the carriage image, determine the motion displacement of the four corner points of the rear of the carriage;

[0054] If the movement displacement of the four corner points is less than or equal to the preset displacement threshold and the compartment door is in an open state, it is determined that the loading state of the compartment is a ready state.

[0055] The above-mentioned carriage image is an image captured by an image acquisition device. In this embodiment, the image acquisition device performs image acquisition on the carriage, so the image captured by the image acquisition device can be referred to as a carriage image. The above-mentioned four corner points may include a corner point at the upper left corner, a corner point at the lower left corner, a corner point at the upper right corner, and a corner point at the lower right corner at the rear of the carriage.

[0056] The electronic device can detect the four corner points at the rear of the carriage based on the carriage image, and when the four corner points are detected in the carriage images of two adjacent frames, the motion displacement of the four corner points can be determined, and the motion displacement of the four corner points can be continuously determined based on the continuous acquisition of the carriage image by the image acquisition device. On this basis, it is continuously judged whether the motion displacement of the four corner points is less than or equal to the preset displacement threshold; if the motion displacement of the four corner points is less than or equal to the preset displacement threshold, it can be determined that the vehicle is in a stopped state, and since the four corner points at the rear of the carriage can be identified in the carriage image acquired by the image acquisition device, it means that the vehicle has been parked in the designated area. On this basis, combined with whether the carriage door is in an open state, it can be further judged whether the loading state of the carriage is in a ready state. Among them, the above-mentioned designated area can be the parking area of ​​the vehicle when loading goods. The electronic device is deployed near the above-mentioned designated area to ensure that the electronic device can measure the loading volume of the carriage of the vehicle parked in the above-mentioned designated area.

[0057] In order to reduce the amount of data calculation, the electronic device can first determine whether the movement displacement of the four corner points is less than or equal to a preset displacement threshold; if the movement displacement of the four corner points is less than or equal to the preset displacement threshold, then determine whether the car door is in an open state, if the car door is in an open state, determine that the loading state of the car is in a ready state; if the movement displacement of the four corner points is greater than the preset displacement threshold, then no longer determine whether the car door is in an open state, directly determine that the loading state of the car is not in a ready state, and at this time there is no need to measure the loading volume of the car. Since there is no need to detect the state of the car door in this case, the detection of the state of the car door is reduced, and the amount of data calculation generated thereby is reduced; if the car door is in a closed state, then determine that the loading state of the car is not in a ready state, and at this time there is no need to measure the loading volume of the car.

[0058] In a possible implementation, a method for determining whether the carriage door is in an open state includes:

[0059] When the first condition is met, determining that the carriage door is in an open state;

[0060] The first condition includes: segmenting the four car walls of the car from the target image, or detecting four lines from the target image and matching the positional relationship of the four lines with the positional relationship of the intersection lines between the four preset car walls, the four car walls include the left side wall, right side wall, upper side wall and lower side wall of the car, and the target image is the image located within the four corner points of the car image.

[0061] Based on the detection of four corner points from the car image, the electronic device can determine whether four car wall surfaces can be segmented from the target image based on a semantic segmentation algorithm, and can detect whether there are lines in the target image based on a line detection algorithm.

[0062] The positional relationship of the intersection lines between the four compartment walls may include an up-down positional relationship and a left-right positional relationship. Figure 6 The following is an example of a car compartment image. Figure 6 The four circular black dots in the figure are the four corner points. The dotted lines represent the intersections between the four compartment walls. Figure 6 The intersection line between the upper side wall and the left side wall and the intersection line between the lower side wall and the left side wall are in a vertical position relationship, the intersection line between the upper side wall and the right side wall and the intersection line between the lower side wall and the right side wall are in a vertical position relationship, the intersection line between the upper side wall and the left side wall and the intersection line between the upper side wall and the right side wall are in a left-right position relationship, and the intersection line between the lower side wall and the left side wall and the intersection line between the lower side wall and the right side wall are in a left-right position relationship. The above-mentioned positional relationship of the four lines detected from the target image matches the positional relationship of the intersection lines between the preset four car walls, which can mean that the positional relationship of the four lines detected from the target image is as follows: Figure 6 The up-down position relationship and the left-right position relationship shown in .

[0063] In another possible implementation, a method for determining whether the carriage door is in an open state includes:

[0064] When the second condition is met, determining that the carriage door is in an open state;

[0065] The second condition includes: the car point cloud located within the four corner points does not form a plane, or the car point cloud located within the four corner points forms a plane and the plane is not a vertical plane, or the car point cloud located within the four corner points forms a vertical plane and the distance between the vertical plane and the electronic device is greater than a preset distance threshold.

[0066] The electronic device can calculate the flatness of the carriage point cloud located at the four corner points based on the depth of the carriage point cloud located at the four corner points; if the flatness of the carriage point cloud located at the four corner points is greater than a preset flatness threshold, it is determined that the carriage point cloud located within the four corner points does not form a plane, and it is determined that the carriage door is in an open state; if the flatness of the carriage point cloud located at the four corner points is less than or equal to the preset flatness threshold, it is determined that the carriage point cloud located within the four corner points forms a plane, and on this basis, it is further determined whether the plane formed by the carriage point cloud located within the four corner points Whether it is a vertical plane; if the plane formed by the car point cloud located within the four corner points is not a vertical plane, it is determined that the car door is in an open state; if the plane formed by the car point cloud located within the four corner points is a vertical plane, then based on the depth of the car point cloud located within the four corner points, the distance between the vertical plane and the electronic device is determined; if the distance between the vertical plane and the electronic device is greater than the preset distance threshold, it is determined that the car door is in an open state; if the distance between the vertical plane and the electronic device is less than or equal to the preset distance threshold, it can be determined that the car door is in a closed state. Optionally, the preset distance threshold can be determined based on the shortest distance between the designated area where the vehicle is parked and the electronic device when measuring the loading volume. For example, considering that it is difficult to ensure that the vehicle is parked exactly at the position closest to the electronic device when parking the vehicle in the designated area, the preset distance threshold can be set to a value slightly larger than the shortest distance between the designated area and the electronic device.

[0067] The above flatness can indicate whether the car point clouds located in the four corner points are in the same plane. The greater the flatness, the greater the probability that the car point clouds located in the four corner points are in the same plane, and the smaller the flatness, the smaller the probability that the car point clouds located in the four corner points are in the same plane.

[0068] In order to make the judgment result of the state of the car door more accurate, in another possible implementation, it can be determined that the car door is in the open state when the first condition and the second condition are met.

[0069] It should be noted that the present application does not limit the order of determining the first condition and the second condition. For example, it is possible to first determine whether the first condition is satisfied, and then determine whether the second condition is satisfied if the first condition is satisfied.

[0070] When the electronic device detects for the first time that the loading state of the carriage is a ready state, it can be determined that the ready state is a loading start state, at which time no cargo has been loaded in the carriage. In this case, the carriage size and carriage posture can be measured based on the carriage image.

[0071] The electronic equipment can calculate the cabin volume based on the cabin dimensions.

[0072] Electronic devices can perform correction processing on vehicles parked in a designated area based on the carriage posture, thereby facilitating subsequent calculation of the loading volume. Electronic devices can first establish a correction coordinate system so that the carriage posture is 0 when the carriage image is projected into the correction coordinate system to achieve correction processing. On this basis, the carriage point cloud is reprojected into the correction coordinate system to obtain the mapping relationship between the pixel points in the carriage image and the carriage point cloud, thereby obtaining the depth of the pixel points in the carriage image corresponding to the carriage point cloud.

[0073] Among them, the size of the car includes the height, height and length of the car. The car posture may include the coordinate information and rotation information of the car in the coordinate system of the image acquisition device. The coordinate information of the car in the coordinate system of the image acquisition device includes the horizontal axis coordinate, the vertical axis coordinate and the vertical axis coordinate of the car in the coordinate system of the image acquisition device. The rotation information of the car in the coordinate system of the image acquisition device includes the angle of rotation of the car around the horizontal axis, the angle of rotation of the car around the vertical axis and the angle of rotation of the car around the vertical axis.

[0074] As an example but not limitation, the coordinate system of the image acquisition device may be a Cartesian coordinate system. Since the ground is horizontal, it can be determined that the angle of rotation of the carriage around the longitudinal axis and the angle of rotation of the carriage around the vertical axis are both 0°, and the angle of rotation of the carriage around the horizontal axis is the angle between the right wall or the left wall of the carriage and the axis of the optical axis of the image acquisition device.

[0075] In one embodiment, the height and width of the carriage are determined by:

[0076] Classify the carriage point cloud to determine the point clouds belonging to the four carriage walls, which include the left wall, the right wall, the upper wall and the lower wall of the carriage;

[0077] Calculate the width of the carriage based on the depth of the point clouds belonging to the right wall and the left wall respectively;

[0078] The height of the cabin is calculated based on the depths of the point clouds belonging to the upper and lower side wall surfaces, respectively.

[0079] In one embodiment, after calculating the height and width of the carriage, the distance between the rear of the carriage and the electronic device can be determined by monocular distance measurement based on the height, width and position information of the four corner points of the carriage in the carriage image using the internal parameters of the image acquisition device. On this basis, after determining that the loading state of the carriage is in the ready state, it can be determined whether the loading state of the carriage is in the terminated state, and the determination method includes:

[0080] After determining that the loading state of the carriage is in the ready state, if the third condition or the fourth condition is satisfied, the loading state of the carriage is determined to be in the terminated state, and information indicating the termination of loading is reported.

[0081] Among them, the third condition includes: the car point cloud forms a vertical plane and the distance between the vertical plane and the electronic device matches the distance between the rear of the car and the electronic device. The distance between the vertical plane and the electronic device matches the distance between the rear of the car and the electronic device, which means that the absolute value of the difference between the distance between the vertical plane and the electronic device and the distance between the rear of the car and the electronic device is less than a preset absolute value. Optionally, the specific value of the preset absolute value can be set according to the scene requirements, and this application does not limit this.

[0082] If the third condition is met, it means that the vehicle is fully loaded or the door of the vehicle is closed. In this case, it can be determined that the loading is terminated.

[0083] The fourth condition includes: the movement displacement of the four corner points is greater than a preset displacement threshold. If the fourth condition is met, it means that the vehicle moves, and in this case, it can be determined that the loading is terminated.

[0084] In one embodiment, the length of the carriage is determined by:

[0085] Determine a carriage head region in the carriage image;

[0086] Classify the car point cloud and determine the point cloud belonging to the head area of ​​the car;

[0087] Based on the depth of the point cloud belonging to the head area of ​​the car, the length of the car is calculated.

[0088] The electronic device can obtain four compartment walls by performing semantic segmentation on the compartment image. The area surrounded by the four compartment walls is the compartment head area. Based on the external parameters of the image acquisition device and the external parameters of the point cloud acquisition device, the compartment point cloud can be classified to divide the point clouds belonging to the four compartment walls and the compartment head area respectively. On this basis, the compartment size can be calculated.

[0089] Based on the depth of the point cloud belonging to the head area of ​​the car, the electronic device can calculate the distance between the head area of ​​the car and the electronic device. The length of the car can be obtained by subtracting the distance between the rear of the car and the electronic device from the distance between the head area of ​​the car and the electronic device.

[0090] Based on the car size calculated by the electronic device, the coordinate information of the car in the coordinate system of the image acquisition device can be determined. That is, the horizontal axis coordinate of the car in the coordinate system of the image acquisition device is the height of the car, the vertical axis coordinate is the width of the car, and the vertical axis coordinate is the distance between the head area of ​​the car and the electronic device.

[0091] Step 402: If the cargo does not exceed the target coverage area, the car point cloud is used as the target point.

[0092] If the cargo does not exceed the target coverage area, it can be determined that the cargo is within the coverage range of the carriage point cloud. In this case, the loading volume of the carriage can be determined based on the depth of the carriage point cloud.

[0093] Step 403: If the cargo exceeds the target coverage area, the pixel points of the area where the cargo is located in the carriage image captured by the image acquisition device are used as target points.

[0094] Among them, the field of view of the image acquisition device covers the entire area inside the car.

[0095] If the cargo exceeds the target coverage area, it can be determined that the cargo exceeds the coverage of the point cloud of the vehicle system. In this case, the depth of the pixel points in the area where the cargo is located in the vehicle compartment image can be determined based on the vehicle compartment image and the vehicle compartment point cloud. Based on the depth of the pixel points in the area where the cargo is located, the loading volume of the vehicle compartment can be determined.

[0096] In a possible implementation, the depth of the pixel point in the area where the goods are located is determined by:

[0097] Determine the depth of a first pixel point and the depth of a second pixel point, the first pixel point is a pixel point on the edge of the area where the cargo is located, and the second pixel point is a pixel point in the area where the cargo is located corresponding to the carriage point cloud;

[0098] Based on the depth of the first pixel and the depth of the second pixel, the depth of the third pixel is determined, where the third pixel is a pixel other than the first pixel and the second pixel in the area where the goods are located.

[0099] Based on the height and width of the carriage and the position information of the edge of the cargo area in the carriage image, the electronic device performs monocular ranging using the internal parameters of the image acquisition device. It can determine the distance between the edge of the cargo area and the electronic device. This distance is the depth of the pixel points on the edge of the cargo area.

[0100] Since the field of view of the point cloud acquisition device is small, the coverage of the measured car point cloud is small, resulting in unknown depth of the pixel points between the car point cloud and the edge of the cargo area in the car image. Since cargo loading is continuous, the pixel point with unknown depth (i.e., the third pixel point) can be interpolated based on the depth of the first pixel point and the depth of the second pixel point (for example, smooth interpolation, of course, other interpolation methods can also be used, which is not limited in this application), so as to obtain the depth of all pixel points in the cargo area.

[0101] like Figure 7 Shown is an example of a car compartment image. Figure 7 The carriage image in the figure is taken by the image acquisition device from the perspective of the rear of the carriage. Figure 7The area surrounded by the red dotted line in the figure is the area where the vehicle point cloud is located, the red solid line is the edge of the area where the cargo is located, and the area surrounded by the red solid line is the area where the cargo is located. The pixels between the red dotted line and the red solid line are the pixels with unknown depth.

[0102] like Figure 8 Shown is an example view of the side of a vehicle. Figure 8 The area surrounded by the red dotted line in the figure is the area where the vehicle point cloud is located, and the red solid line is the edge of the area where the cargo is located. Figure 8 The area with unknown depth between the red dotted line and the red solid line (that is, the area where the pixel points with unknown depth are located) can be displayed more intuitively.

[0103] Step 404, determining the loading volume of the carriage based on the depth of the target point.

[0104] Based on the depth of the target point, the electronic equipment can calculate the empty capacity of the car; the load volume can be obtained by subtracting the empty capacity from the car volume.

[0105] In a possible implementation, the electronic device can calculate the volume of each target point based on the depth, width and height of each target point, and the free capacity of the carriage can be obtained by adding the volumes of each target point. The width and height of the target point are related to the resolution of the point cloud acquisition device. Optionally, the width and height of the target point can be pre-set based on the resolution of the point cloud acquisition device.

[0106] In another possible implementation, the electronic device may, when dividing the vehicle compartment in the vehicle compartment image into two or more first squares, determine the depth of each first square based on the depth of the target point; determine the depth of each second square based on the depth of each first square, the second squares including more than two first squares; and determine the loading volume of the vehicle compartment based on the depth of each second square.

[0107] Since the number of the first squares is relatively large, the electronic device determines the loading volume based on the depth of each second square, which can reduce the amount of calculation.

[0108] According to the dynamic characteristics of interference objects (such as loaders, loading aids, and other non-loaded goods) during the loading process, there is a large possibility of change on a short time scale. The loaded goods show a certain stability on a short time scale. In addition, in addition to being stable, the depth of the loaded goods is often greater than the depth of the foreground dynamic interference objects. Based on these related characteristics, the carriage in the carriage image can be divided into more than two first squares. For the convenience of calculation, the first square is usually a square, and the side length of the square can be the pixel size corresponding to the depth resolution of the point cloud acquisition device in the head area of ​​the carriage. On this basis, the dynamic interference objects can be first filtered out by the time domain sliding window method, and then the depth confidence of the first square can be calculated by the spatial domain load filter method and the interference occlusion smoothing filter method. Finally, the depth of each second square is calculated based on the depth confidence of the first square, and the loading volume is measured on this basis. It has high robustness and can reduce the impact of filtering dynamic interference objects such as loaders and loading aids on the measurement of the loading volume, thereby achieving accurate measurement of the loading volume during the loading process. As Fig. 9 The diagram shows an example of a first grid, where the carriage is divided into a large number of small grids, and the measurement of the loading volume is achieved based on a large grid including a plurality of small grids.

[0109] The electronic device can calculate the volume of each second square based on the depth and area of ​​each second square, add up the volumes of each second square to obtain the free capacity of the carriage, and subtract the free capacity from the volume of the carriage to obtain the loading volume.

[0110] The area of ​​the second square may be the sum of the areas of two or more first squares included in the second square.

[0111] In a possible implementation, before determining the depth of each second square based on the depth of each first square, dynamic interferences may be filtered using a time domain sliding window method, the method comprising:

[0112] For any first square, based on the time scale of k*q, in each k time window, the depth of the first square is updated to the maximum value of the k depths corresponding to the k time windows, where k is an integer greater than 1, and q is the sliding step of the time window, which is an integer greater than zero.

[0113] Since the depth of loaded goods is often greater than the depth of dynamic interferers, the electronic device updates the depth of the first square to the maximum value of k depths corresponding to k time windows, which can filter out dynamic interferers.

[0114] As an example but not a limitation, k is 10, q is 1 second, and for each first square, the maximum value is selected from the 10 depths every 10 seconds as the depth of the first square in this loading volume measurement. If a dynamic interferer appears in the car at the 5th second, the depth of the first square at the 5th second can be filtered out by selecting the maximum value, thereby filtering out the dynamic interferer.

[0115] In a possible implementation, before determining the depth of each second square based on the depth of each first square, the electronic device may determine the target depth confidence of each first square; and determine the depth of each second square based on the depth of each first square and the target depth confidence.

[0116] The depth confidence of the first square may represent the credibility of the depth of the first square. The depth confidence of the first square includes the target depth confidence of the first square and the first depth confidence and the second depth confidence described below.

[0117] The higher the depth confidence of the first square is, the smaller the possibility that the content of the first square is a dynamic distractor is; the lower the depth confidence of the first square is, the greater the possibility that the content of the first square is a dynamic distractor is.

[0118] For ease of calculation, the electronic device may normalize the target depth confidence of each first square, and then multiply the depth of each first square in the second square by the corresponding normalized target depth confidence and then add them together to obtain the depth of the second square.

[0119] In a possible implementation, before determining the target depth confidence of each first square, a spatial domain bearing filter method may be used to filter dynamic interferences, and the method includes:

[0120] Using a first sliding window with a scale of p*1, starting from the bottom first square of the p first squares in each column, the first sliding window is slid upward according to a preset step length to traverse each first square, where p is an integer greater than 1;

[0121] In each first sliding window, based on the depth of the first square at the bottom of the first sliding window, the depth of the first target square and the length of the carriage, a first depth confidence of the first target square is determined, where the first target square is the top first square in the first sliding window;

[0122] Determine that the first depth confidence of the first remaining square is a first preset value, the first remaining square is a square other than the first target square in the two or more first squares; the first target square and the first remaining square constitute each first square.

[0123] The present application does not limit the specific value of the preset step length, which can be preset according to scene requirements. As an example but not a limitation, in order to obtain more first depth confidences of the first target grid, the preset step length can be 1.

[0124] Since the cargo has a load-bearing characteristic (i.e., the cargo on the upper layer must be supported by the cargo on the lower layer), in each first sliding window, the first depth confidence is calculated based on the depth of the first square at the bottom of the first sliding window. In other words, the smaller the depth of the first target square is than the depth of the first square at the bottom of the first sliding window, the lower its first depth confidence is.

[0125] Since the first sliding window starts to slide from the bottom first square among the p first squares, the first remaining square is located below the first target square. Based on the carrying characteristics of the goods, it can be known that the lower the first square is, the higher the depth confidence is. Therefore, the above-mentioned first preset value can be 1.

[0126] The electronic device can calculate the depth percentage of the first target square (i.e., the ratio of the depth of the first target square to the length of the carriage) based on the depth of the first square at the bottom in the first sliding window and the length of the carriage; can calculate the depth percentage of the first square at the bottom in the first sliding window (i.e., the ratio of the depth of the first square at the bottom in the first sliding window to the length of the carriage) based on the depth of the first target square and the length of the carriage; and can calculate the first depth confidence of the first target square based on the depth percentage of the first target square and the depth percentage of the first square at the bottom in the first sliding window.

[0127] The calculation formula of the first depth confidence of the first target square is as follows:

[0128]

[0129] in, A first depth confidence level representing a first target square; Indicates the gain parameter, the default value is 1; Indicates the depth percentage of the first target square; Indicates the depth percentage of the first square at the bottom of the first sliding window.

[0130] In this manner, after obtaining the first depth confidence of each first square, the electronic device can determine the target depth confidence of the corresponding first square based on the first depth confidence of each first square. As an example and not a limitation, the target depth confidence of each first square can be determined as the first depth confidence of the corresponding first square. Of course, it can be understood that the target depth confidence of the corresponding first square can also be determined based on the first depth confidence of each first square in other ways, and this application does not limit this.

[0131] In another possible implementation, before determining the target depth confidence of each first square, an interference occlusion smoothing filter method may be used to filter dynamic interferences, and the method includes:

[0132] Using a second sliding window with a scale of m*n, sliding the second sliding window according to a preset rule to traverse each first square, where m and n are integers greater than 1;

[0133] In each second sliding window, determining the position difference between the deepest square in the second sliding window and the second target square, the deepest square is the first square with the largest depth in the second sliding window, and the second target square is the first square at the center of the second sliding window;

[0134] determining a second depth confidence of the second target square based on the position difference, the depth of the deepest square, the depth of the second target square, and the length of the carriage;

[0135] The second depth confidence of the second remaining square is determined to be a second preset value, and the second remaining square is the square other than the second target square in the two or more first squares; the first target square and the first remaining square constitute each first square.

[0136] Among them, the above-mentioned preset rules may include starting from the lower left corner of the carriage in the carriage image, first sliding horizontally according to the first step length, and then sliding vertically according to the second step length, or starting from the lower left corner of the carriage in the carriage image, first sliding vertically according to the second step length, and then sliding horizontally according to the first step length, or starting from the lower right corner of the carriage in the carriage image, first sliding horizontally according to the first step length, and then sliding vertically according to the second step length, or starting from the lower right corner of the carriage in the carriage image, first sliding vertically according to the second step length, and then sliding horizontally according to the first step length. The above-mentioned first step length and second step length are both preset, and can be the same or different, and this application does not limit this.

[0137] As an example but not a limitation, in order to obtain more first depth confidences of first target squares, the first step size and the second step size may be set to 1.

[0138] Since the depth of the dynamic interference is smaller than the actual depth of the loaded goods, the second depth confidence of the second target square is calculated based on the first square with the largest depth in the second sliding window in each second sliding window. That is to say, the smaller the depth of the second target square is than the depth of the deepest square, the greater the possibility that it is a dynamic interference.

[0139] Since the second sliding window starts to slide from the lower left corner or the lower right corner of the carriage, the second remaining square is usually located below the second target square. Based on the carrying characteristics of the cargo, it can be known that the lower the position of the first square, the higher the depth confidence, so the above-mentioned second preset value can be 1.

[0140] Based on the depth of the second target square and the length of the car, the electronic device can calculate the depth percentage of the second target square (i.e., the ratio of the depth of the second target square to the length of the car); based on the depth of the deepest square in the second sliding window and the length of the car, the electronic device can calculate the depth percentage of the deepest square (i.e., the ratio of the depth of the deepest square to the length of the car); based on the depth percentage of the second target square and the depth percentage of the second square of the deepest square, the electronic device can calculate the second depth confidence of the second target square.

[0141] The position difference between the second target square and the deepest square may refer to the sum of the position difference between the deepest square and the second target square in the horizontal direction and in the vertical direction.

[0142] The calculation formula of the second depth confidence of the second target square is as follows:

[0143]

[0144] in, A second depth confidence level representing a second target square; Indicates the gain parameter, the default value is 1; Indicates the depth percentage of the second target square; Indicates the depth percentage of the deepest square; It represents the position difference between the deepest square and the second target square. The larger the position difference is, the smaller its influence on the second depth confidence is; the smaller the position difference is, the greater its influence on the second depth confidence is.

[0145] In this manner, after obtaining the second depth confidence of each first square, the electronic device may determine the target depth confidence of the corresponding first square based on the second depth confidence of each first square.

[0146] In order to further improve the accuracy of the target depth confidence of the first square, in another possible implementation method, after obtaining the first depth confidence and the second depth confidence of each first square based on the above two methods, the target depth confidence can be obtained based on a weighted sum of the first depth confidence and the second depth confidence of the same first square.

[0147] It should be noted that the present application does not limit the weight values ​​corresponding to the first depth confidence and the second depth confidence. For example, the weight of the first depth confidence and the weight of the second depth confidence are both 0.5, or the weight of the first depth confidence is 0.4 and the weight of the second depth confidence is 0.6.

[0148] The embodiment of the present application measures the loading volume of the carriage when the loading state of the carriage is in the ready state, thereby realizing real-time measurement of the loading volume during the loading process. Moreover, through the complementary installation of the point cloud acquisition device and the image acquisition device, the far end (i.e., the area in the carriage that is far away from the electronic device) is mainly covered by the point cloud acquisition device, and the near end (i.e., the area in the carriage that is close to the electronic device) is mainly covered by the image acquisition device. There is no need to cover the entire carriage with point clouds. On the basis of saving hardware deployment costs, the loading volume can be measured fully automatically and without any sense during the loading process. There is no need to collect data on the empty car in advance, the operation is relatively simple, and the measurement efficiency of the loading volume is improved.

[0149] It should be noted that Figure 5 , Figure 6 , Figure 7 , Figure 8 and Fig. 9 L and D1 represent the length of the car, W represents the width of the car, H represents the height of the car, and D2 represents the distance between the rear of the car and the electronic equipment.

[0150] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0151] Corresponding to the measurement method of the loading volume described in the above embodiment, Fig.10 A schematic structural diagram of a loading volume measuring device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0152] Reference Fig.10 , the device comprises:

[0153] The cargo position determination module 1001 is used to determine whether the cargo exceeds the target coverage area based on the depth of the point cloud of the carriage collected by the point cloud collection device when the loading state of the carriage is the ready state, wherein the target coverage area is the maximum coverage range of the field of view of the point cloud collection device on the lower side wall of the carriage, and the field of view of the point cloud collection device covers a part of the area in the carriage;

[0154] A target point determination module 1002 is configured to use the carriage point cloud as a target point if the cargo does not exceed the target coverage area;

[0155] The image processing module 1003 is used to use the pixel points of the area where the goods are located in the carriage image captured by the image acquisition device as the target point if the goods exceed the target coverage area, and the field of view of the image acquisition device covers the entire area in the carriage;

[0156] The loading volume determination module 1004 is used to determine the loading volume of the carriage based on the depth of the target point.

[0157] Optionally, the above device further includes a target depth determination module, and the target depth determination module is specifically used to:

[0158] Determine the depth of a first pixel point and a second pixel point, wherein the first pixel point is a pixel point on the edge of the area where the cargo is located, and the second pixel point is a pixel point in the area where the cargo is located corresponding to the carriage point cloud;

[0159] Based on the depth of the first pixel and the depth of the second pixel, the depth of a third pixel is determined, where the third pixel is a pixel in the area where the goods are located except the first pixel and the second pixel.

[0160] Optionally, the above device further includes a readiness determination module, and the readiness determination module includes:

[0161] A displacement determination unit, used to determine the motion displacement of four corner points of the rear of the carriage based on the carriage image;

[0162] A ready determination unit is used to determine that the loading state of the carriage is the ready state if the movement displacement of the four corner points is less than or equal to a preset displacement threshold and the carriage door is in an open state.

[0163] Optionally, the above device further includes a first start-up determination module, and the first start-up determination module includes:

[0164] When the first condition is met, determining that the compartment door is in the open state;

[0165] The first condition includes: segmenting the four car walls of the car from the target image, or detecting four lines from the target image and matching the positional relationship of the four lines with the positional relationship of the preset intersection lines between the four car walls, the four car walls include the left wall, right wall, upper wall and lower wall of the car, and the target image is the image of the car image located within the four corner points.

[0166] Optionally, the above device further includes a second start-up determination module, and the second start-up determination module includes:

[0167] When the second condition is met, determining that the compartment door is in the open state;

[0168] The second condition includes: the car point cloud located within the four corner points does not form a plane, or the car point cloud located within the four corner points forms a plane and the plane is not a vertical plane, or the car point cloud located within the four corner points forms a vertical plane and the distance between the vertical plane and the electronic device is greater than a preset distance threshold.

[0169] Optionally, the above device further includes a third opening determination module, and the third opening determination module includes:

[0170] When the first condition and the second condition are met, determining that the compartment door is in the open state;

[0171] The first condition includes: segmenting the four compartment walls of the compartment from the target image, or detecting the four lines from the target image and matching the positional relationship of the four lines with the positional relationship of the intersection lines between the four compartment walls, the four compartment walls include the left side wall, the right side wall, the upper side wall and the lower side wall of the compartment, and the target image is an image located within the four corner points in the compartment image;

[0172] The second condition includes: the car point cloud located within the four corner points does not form a plane, or the car point cloud located within the four corner points forms a plane and the plane is not a vertical plane, or the car point cloud located within the four corner points forms a vertical plane and the distance between the vertical plane and the electronic device is greater than a preset distance threshold.

[0173] Optionally, the above device further comprises:

[0174] a termination determination module, configured to, after determining that the loading state of the carriage is in the ready state, determine that the loading state of the carriage is in the termination state if the third condition or the fourth condition is satisfied, and report information indicating the termination of loading;

[0175] The third condition includes: the car point cloud forms a vertical plane and the distance between the vertical plane and the electronic device matches the distance between the rear of the car and the electronic device;

[0176] The fourth condition includes: the movement displacement of the four corner points is greater than the preset displacement threshold.

[0177] Optionally, the above device further includes a distance determination module, and the distance determination module includes:

[0178] A width and height determination unit, used to determine the height and width of the carriage;

[0179] The distance determining unit is used to determine the distance between the rear of the carriage and the electronic device based on the height and width of the carriage and the position information of the four corner points in the carriage image.

[0180] Optionally, the width and height determining unit is specifically used for:

[0181] Classifying the carriage point cloud to determine point clouds belonging to four carriage walls, respectively, wherein the four carriage walls include a left wall, a right wall, an upper wall, and a lower wall of the carriage;

[0182] Calculating the width of the carriage based on the depths of the point clouds belonging to the right wall and the left wall respectively;

[0183] The height of the cabin is calculated based on the depths of the point clouds respectively belonging to the upper wall surface and the lower wall surface.

[0184] Optionally, the loading volume determination module 1004 includes:

[0185] A first determining unit, configured to determine the depth of each first square based on the depth of the target point when the carriage in the carriage image is divided into two or more first squares;

[0186] A second determining unit, configured to determine the depth of each second square based on the depth of each first square, wherein the second square includes more than two first squares;

[0187] The third determining unit is used to determine the loading volume of the carriage based on the depth of each second square.

[0188] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0189] Fig.11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.11 As shown, the electronic device of this embodiment includes: at least one processor 1100 ( Fig.11 Only one is shown in the figure), a memory 1101 and a computer program 1102 stored in the memory 1101 and executable on the at least one processor 1100, wherein the processor 1100 implements the steps of any of the above-mentioned method embodiments when executing the computer program 1102.

[0190] The electronic device may include, but is not limited to, a processor 1100 and a memory 1101. Those skilled in the art will appreciate that Fig.11These are merely examples of electronic devices and do not constitute a limitation on the electronic devices. The electronic devices may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, the electronic devices may also include input and output devices, network access devices, etc.

[0191] The processor 1100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0192] In some embodiments, the memory 1101 may be an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device. In other embodiments, the memory 1101 may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 1101 may also include both an internal storage unit of the electronic device and an external storage device. The memory 1101 is used to store an operating system, an application program, a boot loader (BootLoader), data and other programs, such as the program code of the computer program, etc. The memory 1101 may also be used to temporarily store data that has been output or is to be output.

[0193] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0194] An embodiment of the present application also provides an electronic device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps of any of the above-mentioned method embodiments when executing the computer program.

[0195] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0196] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0197] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0198] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0199] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0200] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for measuring loading volume, characterized in that: Applied to electronic equipment, the method comprises: When the loading state of the carriage is the ready state, judging whether the cargo exceeds the target coverage area based on the depth of the carriage point cloud collected by the point cloud collection device, the target coverage area is the maximum coverage range of the field of view of the point cloud collection device on the lower side wall of the carriage, and the field of view of the point cloud collection device covers a part of the area in the carriage; If the cargo does not exceed the target coverage area, the carriage point cloud is used as the target point; If the cargo exceeds the target coverage area, the pixel points of the cargo area in the carriage image captured by the image acquisition device are used as the target points, and the field of view of the image acquisition device covers the entire area in the carriage; determining a loading volume of the carriage based on the depth of the target point; The method for determining that the loading state of the carriage is the ready state includes: Based on the carriage image, determining the motion displacement of four corner points of the rear of the carriage; If the movement displacement of the four corner points is less than or equal to the preset displacement threshold and the compartment door is in an open state, it is determined that the loading state of the compartment is the ready state.

2. The method according to claim 1, characterized in that When the target point is a pixel point in the area where the goods are located, the depth of the target point is determined by: Determine the depth of a first pixel point and a second pixel point, wherein the first pixel point is a pixel point on the edge of the area where the cargo is located, and the second pixel point is a pixel point in the area where the cargo is located corresponding to the carriage point cloud; Based on the depth of the first pixel and the depth of the second pixel, the depth of a third pixel is determined, where the third pixel is a pixel in the area where the goods are located except the first pixel and the second pixel.

3. The method according to claim 1, characterized in that The method for determining that the carriage door is in the open state includes: When the first condition is met, determining that the compartment door is in the open state; The first condition includes: segmenting the four car walls of the car from the target image, or detecting four lines from the target image and matching the positional relationship of the four lines with the positional relationship of the preset intersection lines between the four car walls, the four car walls include the left wall, right wall, upper wall and lower wall of the car, and the target image is the image of the car image located within the four corner points.

4. The method according to claim 1, characterized in that The method for determining that the carriage door is in the open state includes: When the second condition is met, determining that the compartment door is in the open state; The second condition includes: the car point cloud located within the four corner points does not form a plane, or the car point cloud located within the four corner points forms a plane and the plane is not a vertical plane, or the car point cloud located within the four corner points forms a vertical plane and the distance between the vertical plane and the electronic device is greater than a preset distance threshold.

5. The method according to claim 1, characterized in that The method for determining that the carriage door is in the open state includes: When the first condition and the second condition are met, determining that the compartment door is in the open state; The first condition includes: segmenting four compartment walls of the compartment from the target image, or detecting four lines from the target image and matching the positional relationship of the four lines with the positional relationship of the intersection lines between the four compartment walls, the four compartment walls include the left side wall, the right side wall, the upper side wall and the lower side wall of the compartment, and the target image is an image located within the four corner points of the compartment image; The second condition includes: the car point cloud located within the four corner points does not form a plane, or the car point cloud located within the four corner points forms a plane and the plane is not a vertical plane, or the car point cloud located within the four corner points forms a vertical plane and the distance between the vertical plane and the electronic device is greater than a preset distance threshold.

6. The method according to any one of claims 3 to 5, characterized in that: Also includes: After determining that the loading state of the carriage is in the ready state, if the third condition or the fourth condition is satisfied, determining that the loading state of the carriage is in the terminated state, and reporting information indicating the termination of loading; The third condition includes: the car point cloud forms a vertical plane and the distance between the vertical plane and the electronic device matches the distance between the rear of the car and the electronic device; The fourth condition includes: the movement displacement of the four corner points is greater than the preset displacement threshold.

7. The method according to claim 6, characterized in that The method for determining the distance between the rear of the carriage and the electronic device includes: determining the height and width of the carriage; Based on the height and width of the carriage and the position information of the four corner points in the carriage image, the distance between the rear of the carriage and the electronic device is determined.

8. The method according to claim 7, characterized in that Determining the height and width of the carriage includes: Classifying the carriage point cloud to determine point clouds belonging to four carriage walls, respectively, wherein the four carriage walls include a left wall, a right wall, an upper wall, and a lower wall of the carriage; Calculating the width of the carriage based on the depths of the point clouds belonging to the right wall and the left wall respectively; The height of the cabin is calculated based on the depths of the point clouds respectively belonging to the upper wall surface and the lower wall surface.

9. The method according to any one of claims 1 to 5, characterized in that: The step of determining the loading volume of the carriage based on the depth of the target point comprises: In the case where the carriage in the carriage image is divided into two or more first squares, determining the depth of each first square based on the depth of the target point; Determining the depth of each second square based on the depth of each first square, wherein the second square includes more than two first squares; Based on the depths of the second squares, the loading volume of the carriage is determined.

10. A device for measuring loading volume, characterized in that: Applied to electronic equipment, the device comprises: A cargo position determination module, used to determine whether the cargo exceeds a target coverage area based on the depth of the point cloud of the carriage collected by the point cloud collection device when the loading state of the carriage is a ready state, wherein the target coverage area is the maximum coverage range of the field of view of the point cloud collection device on the lower side wall of the carriage, and the field of view of the point cloud collection device covers a part of the area in the carriage; A target point determination module, configured to use the carriage point cloud as a target point if the cargo does not exceed the target coverage area; An image processing module, configured to use the pixel points of the area where the goods are located in the carriage image captured by an image acquisition device as the target point if the goods exceed the target coverage area, and the field of view of the image acquisition device covers the entire area in the carriage; a loading volume determination module, configured to determine the loading volume of the carriage based on the depth of the target point; The apparatus further comprises a readiness determination module, wherein the readiness determination module comprises: A displacement determination unit, used to determine the motion displacement of four corner points of the rear of the carriage based on the carriage image; A ready determination unit is used to determine that the loading state of the carriage is the ready state if the movement displacement of the four corner points is less than or equal to a preset displacement threshold and the carriage door is in an open state.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the electronic device implements the method according to any one of claims 1 to 9.

12. A computer program product, characterized in that The invention comprises a computer program which, when executed, causes the method according to any one of claims 1 to 9 to be performed.

Images