Goods loading information generation method and device, equipment, medium and program product
By generating internal and external parameter information of cargo loading images and establishing a coordinate conversion model, the problem of inaccurate supervision of truck loading conditions is solved, and efficient and accurate generation of loading information is achieved.
Patent Information
- Application Number
- CN202410205247.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-23
- Publication Date
- 2025-08-26
AI Technical Summary
In the prior art, the supervision of truck loading conditions is not accurate enough, resulting in the determination of loading conditions being inefficient and accurate enough.
By acquiring the cargo loading image of the shooting device at the target position, generating the shooting internal parameter information and external parameter information, establishing a coordinate conversion model, and using this model to generate the cargo loading information.
It realizes efficient and accurate generation of cargo loading information, ensuring accurate determination of loading conditions.
Smart Images
Figure CN120543364A_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present disclosure relate to the field of computer technology, and more particularly to methods, devices, equipment, media, and program products for generating cargo loading information. Background Art
[0002] Currently, measuring truck loading plays a vital role in monitoring transportation routes and optimizing vehicle deployment. Determining truck loading is typically done by monitoring the loading status of the truck using a monocular camera.
[0003] However, the inventors have discovered that when the above method is used to determine the loading status of a truck, the following technical problems often arise:
[0004] The loading status of trucks cannot be effectively and efficiently monitored, resulting in inaccurate determination of the loading status of trucks.
[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention
[0006] The content of this disclosure is used to briefly introduce concepts that will be described in detail in the detailed description section below. The content of this disclosure is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.
[0007] Some embodiments of the present disclosure provide methods, devices, electronic devices, computer-readable media, and program products for generating cargo loading information to solve the technical problems mentioned in the above background technology section.
[0008] In a first aspect, some embodiments of the present disclosure provide a method for generating cargo loading information, including: obtaining a cargo loading image taken by a photographing device when a target truck is in a target position; generating shooting internal reference information corresponding to the photographing device based on the above-mentioned cargo loading image; determining shooting external reference information corresponding to the above-mentioned photographing device based on the above-mentioned shooting internal reference information and the above-mentioned cargo loading image; generating a coordinate transformation model based on the above-mentioned shooting internal reference information and the above-mentioned shooting external reference information, wherein the above-mentioned coordinate transformation model represents the coordinate transformation relationship between the pixel coordinate system and the world coordinate system; generating cargo loading information corresponding to the above-mentioned target truck based on the above-mentioned coordinate transformation model and the above-mentioned cargo loading image.
[0009] Optionally, generating shooting internal parameter information corresponding to the above-mentioned shooting device based on the above-mentioned cargo loading image includes: determining a shadow cancellation point set based on the above-mentioned cargo loading image; generating focal length information and principal point coordinate information corresponding to the above-mentioned shooting device based on the above-mentioned shadow cancellation point set; and generating the above-mentioned shooting internal parameter information based on the above-mentioned focal length information and the above-mentioned principal point coordinate information.
[0010] Optionally, the above-mentioned determining the shooting external reference information corresponding to the above-mentioned shooting device based on the above-mentioned shooting internal reference information and the above-mentioned cargo loading image includes: obtaining at least one first pixel point coordinate corresponding to at least one cargo box edge position point in the above-mentioned cargo loading image; obtaining at least one first world coordinate corresponding to the above-mentioned at least one cargo box edge position point; and determining the shooting external reference information corresponding to the above-mentioned shooting device based on the above-mentioned shooting internal reference information, the above-mentioned at least one first pixel point coordinate and the above-mentioned at least one first world coordinate.
[0011] Optionally, the cargo loading information is a cargo loading rate; and the cargo loading information corresponding to the target truck is generated based on the coordinate conversion model and the cargo loading image, including: determining at least one second pixel point corresponding to at least one cargo handover position point in the cargo loading image; bringing the at least one second pixel point into the coordinate conversion model to generate at least one second world coordinate for the at least one cargo handover position point; generating the free volume of the compartment of the target truck based on the at least one second world coordinate; and determining the cargo loading rate corresponding to the target truck based on the free volume of the compartment.
[0012] Optionally, the above-mentioned determining the shadow cancellation point set based on the above-mentioned cargo loading image includes: establishing a pixel coordinate system with a target cargo box edge position point among at least one cargo box edge position point as the origin; for each plane under the above-mentioned pixel coordinate system, determining the parallel lines corresponding to the above-mentioned plane; determining the shadow cancellation point corresponding to each parallel line in the obtained parallel line set, to obtain the shadow cancellation point set.
[0013] Optionally, the above-mentioned generating the free volume of the cabin of the above-mentioned target truck based on the above-mentioned at least one second world coordinate includes: for each world coordinate in the above-mentioned at least one second world coordinate, determining the coordinate value of the above-mentioned world point on the target coordinate axis; performing weighted summation processing on the obtained at least one coordinate value to obtain a weighted sum value; generating the above-mentioned free volume of the cabin based on the cabin width, cabin height and the above-mentioned weighted sum value of the above-mentioned target truck.
[0014] In the second aspect, some embodiments of the present disclosure provide a cargo loading information generating device, including: an acquisition unit, configured to acquire a cargo loading image taken by a shooting device when a target truck is in a target position; a first generation unit, configured to generate shooting internal reference information corresponding to the above-mentioned shooting device based on the above-mentioned cargo loading image; a determination unit, configured to determine the shooting external reference information corresponding to the above-mentioned shooting device based on the above-mentioned shooting internal reference information and the above-mentioned cargo loading image; a second generation unit, configured to generate a coordinate conversion model based on the above-mentioned shooting internal reference information and the above-mentioned shooting external reference information, wherein the above-mentioned coordinate conversion model represents the coordinate conversion relationship between the pixel coordinate system and the world coordinate system; a third generation unit, configured to generate cargo loading information corresponding to the above-mentioned target truck based on the above-mentioned coordinate conversion model and the above-mentioned cargo loading image.
[0015] Optionally, the first generation unit can be configured to: determine the shadow elimination point set based on the above-mentioned cargo loading image; generate the focal length information and principal point coordinate information corresponding to the above-mentioned shooting device based on the above-mentioned shadow elimination point set; and generate the above-mentioned shooting internal reference information based on the above-mentioned focal length information and the above-mentioned principal point coordinate information.
[0016] Optionally, the determination unit can be configured to: obtain at least one first pixel point coordinate corresponding to at least one cargo box edge position point in the above-mentioned cargo loading image; obtain at least one first world coordinate corresponding to the above-mentioned at least one cargo box edge position point; and determine the shooting external parameter information corresponding to the above-mentioned shooting device based on the above-mentioned shooting internal parameter information, the above-mentioned at least one first pixel point coordinate and the above-mentioned at least one first world coordinate.
[0017] Optionally, the cargo loading information is a cargo loading rate; and the third generation unit can be configured to: determine at least one second pixel point corresponding to at least one cargo handover position point in the cargo loading image; bring the at least one second pixel point into the coordinate conversion model to generate at least one second world coordinate for the at least one cargo handover position point; generate the empty volume of the compartment of the target truck based on the at least one second world coordinate; and determine the cargo loading rate corresponding to the target truck based on the empty volume of the compartment.
[0018] Optionally, the first generation unit can be configured to: establish a pixel coordinate system with a target cargo box edge position point among at least one cargo box edge position point as the origin; determine the parallel lines corresponding to each plane under the above pixel coordinate system; determine the shadow vanishing point corresponding to each parallel line in the obtained parallel line set to obtain a shadow vanishing point set.
[0019] Optionally, the third generation unit can be configured to: for each world coordinate in the above-mentioned at least one second world coordinate, determine the coordinate value of the above-mentioned world point on the target coordinate axis; perform weighted summation processing on the at least one obtained coordinate value to obtain a weighted sum value; and generate the above-mentioned car compartment vacant volume based on the car compartment width, car compartment height and the above-mentioned weighted sum value of the above-mentioned target truck.
[0020] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.
[0021] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.
[0022] In a fifth aspect, some embodiments of the present disclosure provide a computer program product, including a computer program, which implements the method described in any implementation manner in the first aspect when executed by a processor.
[0023] The above-described various embodiments of the present disclosure have the following advantageous effects: Through the cargo loading information generation methods of some embodiments of the present disclosure, cargo loading information can be generated efficiently and accurately. Specifically, the lack of accuracy and efficiency in generating cargo loading information is caused by the inability to effectively and efficiently monitor the loading status of a truck, resulting in inaccurate determination of the truck's loading status. Based on this, the cargo loading information generation methods of some embodiments of the present disclosure first obtain a cargo loading image captured by the camera with the target truck in the target position, for use in the subsequent determination of the camera's internal and external reference information. Then, based on the cargo loading image, the corresponding internal reference information can be accurately generated. The obtained internal reference information is not only used for the subsequent determination of the external reference information, but also for determining a coordinate transformation model. Furthermore, based on the internal reference information and the cargo loading image, the corresponding external reference information can be accurately determined. Furthermore, based on the internal and external reference information, a coordinate transformation model can be accurately generated, where the coordinate transformation model represents the coordinate transformation relationship between the pixel coordinate system and the world coordinate system. Finally, based on the coordinate transformation model and the cargo loading image, the cargo loading information corresponding to the target truck can be accurately generated. Here, the coordinate transformation model can be used to accurately reflect the actual loading conditions in subsequent cargo loading images to accurately generate the cargo loading information. In summary, by analyzing the image content of the cargo loading image, the internal and external reference information of the camera can be accurately determined. Furthermore, this internal and external reference information can be used to accurately generate a coordinate transformation model that can fully indirectly reflect the actual cargo loading conditions in the image. Thus, using the coordinate transformation model, the cargo loading information corresponding to the target truck can be precisely determined. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.
[0025] Figure 1 is a schematic diagram of an application scenario of a method for generating cargo loading information according to some embodiments of the present disclosure;
[0026] Figure 2 is a flow chart of some embodiments of a method for generating cargo loading information according to the present disclosure;
[0027] Figure 3is a schematic diagram of at least one cargo box edge position point in some embodiments of the cargo loading information generating method according to the present disclosure;
[0028] Figure 4 is a schematic diagram of at least one cargo handover location in some embodiments of the cargo loading information generating method according to the present disclosure;
[0029] Figure 5 is a flow chart of other embodiments of the method for generating cargo loading information according to the present disclosure;
[0030] Figure 6 is a schematic diagram of parallel lines corresponding to various planes in some embodiments of the cargo loading information generating method according to the present disclosure;
[0031] Figure 7 is a schematic structural diagram of some embodiments of the cargo loading information generating device according to the present disclosure;
[0032] Figure 8 It is a structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION
[0033] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.
[0034] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.
[0035] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0036] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".
[0037] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0038] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.
[0039] Figure 1 It is a schematic diagram of an application scenario of the cargo loading information generation method according to some embodiments of the present disclosure.
[0040] exist Figure 1 In this application scenario, the electronic device 101 can first obtain a cargo loading image 102 captured by the camera 103 with the target truck at the target position. Then, based on the cargo loading image 102, the electronic device 101 can generate intrinsic shooting reference information 104 corresponding to the camera 103. In this application scenario, the intrinsic shooting reference information 104 can be "focal length: 2.8mm; pixel size: 1080p." Next, based on the intrinsic shooting reference information 104 and the cargo loading image 102, the electronic device 101 can determine extrinsic shooting reference information 105 corresponding to the camera 103. In this application scenario, the extrinsic shooting reference information 105 can be "camera position: position A; camera rotation angle: angle B." Furthermore, based on the intrinsic shooting reference information 104 and the extrinsic shooting reference information 105, the electronic device 101 can generate a coordinate transformation model 106. The coordinate transformation model 106 represents the coordinate transformation relationship between the pixel coordinate system and the world coordinate system. Finally, the electronic device 101 may generate cargo loading information 107 corresponding to the target truck based on the coordinate transformation model 106 and the cargo loading image 102. In this application scenario, the cargo loading information 107 may be "loading rate: 60%."
[0041] It should be noted that the electronic device 101 can be hardware or software. When the electronic device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or it can be implemented as a single server or a single terminal device. When the electronic device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, for example, or it can be implemented as a single software or software module. No specific limitation is made here.
[0042] It should be understood that Figure 1 The number of electronic devices in the embodiment is merely illustrative. Any number of electronic devices may be provided according to implementation requirements.
[0043] Continue to refer Figure 2 , shows a process 200 of some embodiments of the cargo loading information generation method according to the present disclosure. The cargo loading information generation method includes the following steps:
[0044] Step 201 : obtaining a cargo loading image captured by a camera when a target truck is at a target position.
[0045] In some embodiments, the execution entity of the cargo loading information generation method (for example Figure 1 The electronic device 101 shown can obtain a cargo loading image captured by a camera when a target truck is at a target position via a wired or wireless connection. The target truck is the truck whose cargo loading status is currently to be identified. The camera can be a device that captures the cargo loading status of the target truck. For example, the camera can be a camera. The target position can be directly below the camera, with the rear of the target truck facing the camera. The cargo loading image can be a frame of a cargo loading video captured by the camera. The cargo loading image can be an image of the target truck loaded with cargo.
[0046] Step 202: Generate internal reference information corresponding to the photographing device based on the cargo loading image.
[0047] In some embodiments, the execution entity may generate internal shooting parameter information corresponding to the camera based on the cargo loading image. The internal shooting parameter information may include internal parameter information of various internal parameters of the camera. In practice, the internal parameter may include, but is not limited to, at least one of the following: focal length and principal point coordinates.
[0048] As an example, the execution entity may input the cargo loading image into an internal reference information generation model to generate internal reference information corresponding to the capture device. The internal reference information generation model may be a model for generating internal reference information. For example, the internal reference information generation model may be a recurrent neural network model with multiple layers of serial connections.
[0049] Step 203 : Determine the shooting external reference information corresponding to the shooting device according to the shooting internal reference information and the cargo loading image.
[0050] In some embodiments, the execution entity may determine external reference information corresponding to the camera based on the internal reference information and the cargo loading image. The external reference information may include external reference information of various external parameters of the camera. The external reference may include, but is not limited to, at least one of the following: the position of the camera and the rotation angle of the camera.
[0051] As an example, the execution entity may output the shooting internal reference information and the cargo loading image to an attention mechanism model based on a residual network model to generate shooting external reference information corresponding to the shooting device.
[0052] In some optional implementations of some embodiments, determining the shooting external reference information corresponding to the shooting device based on the shooting internal reference information and the cargo loading image may include the following steps:
[0053] The first step is to obtain at least one first pixel point coordinate corresponding to at least one cargo box edge position point in the cargo loading image.
[0054] The at least one cargo box edge location point may be at least one edge point at the rear of the cargo box in the cargo loading image. There is a one-to-one correspondence between the at least one cargo box edge location point and a first pixel coordinate in the at least one first pixel coordinate. The first pixel coordinate may be a pixel position of the cargo box edge location point in the cargo loading image. The first pixel coordinate may be a pixel coordinate in a pixel coordinate system.
[0055] See Figure 3 , showing the position of at least one container edge location point. Figure 3 Points A, B, C, and D are the edge points of the cargo box.
[0056] The second step is to obtain at least one first world coordinate corresponding to the at least one container edge location point. There is a one-to-one correspondence between the container edge location point in the at least one container edge location point and the world coordinate in the at least one first world coordinate. The first world coordinate may be the coordinate position of the container edge location point in a world coordinate system.
[0057] The third step is to determine the shooting extrinsic parameter information corresponding to the above-mentioned shooting device based on the above-mentioned shooting internal parameter information, the above-mentioned at least one first pixel point coordinate and the above-mentioned at least one first world coordinate.
[0058] As an example, first, the shooting intrinsic parameter information is converted into information in matrix form to obtain a shooting intrinsic parameter matrix. Then, the initial shooting extrinsic parameter matrix corresponding to the initial shooting extrinsic parameter information is determined. Next, the shooting intrinsic parameter matrix, the at least one first pixel point coordinate, and the at least one first world coordinate are substituted into the initial coordinate transformation model to generate a matrix value corresponding to the initial shooting extrinsic parameter matrix. The initial coordinate transformation model can be the following formula:
[0059]
[0060] in, Can be pixel coordinates. It can be a world coordinate. K can be the shooting intrinsic parameter matrix. Rt can be the initial shooting extrinsic parameter matrix. Rt can be a matrix composed of the R rotation matrix and the t translation matrix. The R rotation matrix can represent the rotation angle of the camera. The t translation matrix can represent the translation distance of the camera.
[0061] Step 204: Generate a coordinate transformation model based on the shooting internal reference information and the shooting external reference information.
[0062] In some embodiments, the execution entity may generate a coordinate transformation model based on the internal and external reference information. The coordinate transformation model represents the coordinate transformation relationship between the pixel coordinate system and the world coordinate system. In practice, the coordinate transformation model may be a deep convolutional neural network model. In practice, both the pixel coordinate system and the world coordinate system may be coordinate systems established with the target truck's location as the origin.
[0063] As an example, first, a pixel coordinate set is obtained. Then, the position of the camera is determined based on the internal and external shooting parameter information. Next, a world coordinate system is established with the camera's position as the center point to determine the world coordinates corresponding to each pixel coordinate, obtaining a world coordinate set. Then, using the pixel coordinate set as the training dataset and the corresponding world coordinate set as the label set, an initial deep convolutional neural network model is trained to obtain a deep convolutional neural network model, which serves as the coordinate transformation model.
[0064] Step 205 : Generate cargo loading information corresponding to the target truck based on the coordinate conversion model and the cargo loading image.
[0065] In some embodiments, the execution entity may generate cargo loading information corresponding to the target truck based on the coordinate transformation model and the cargo loading image. The cargo loading information may include the cargo loading status of the target truck. For example, the cargo loading information may include the cargo loading weight.
[0066] As an example, the execution entity may first determine the truck edge pixel coordinates corresponding to the truck edge point set and the cargo edge pixel coordinates corresponding to the cargo edge point set in the cargo loading image. Then, using the coordinate transformation model, the truck edge pixel coordinates and cargo edge pixel coordinates are converted into truck edge world coordinates and cargo edge world coordinates. These truck edge world coordinates and cargo edge world coordinates are then input into the multi-head attention model to generate cargo loading information corresponding to the target truck.
[0067] In some optional implementations of some embodiments, the cargo loading information is a cargo loading rate, wherein the cargo loading rate is a value between 0 and 1. A larger cargo loading rate value indicates a greater amount of cargo loaded on the target truck.
[0068] Optionally, generating the cargo loading information corresponding to the target truck based on the coordinate transformation model and the cargo loading image may include the following steps:
[0069] The first step is to determine at least one second pixel point corresponding to at least one cargo handover location in the cargo loading image. The cargo handover location may be the location where the cargo meets the truck. In practice, the cargo handover location may be the lower left or lower right intersection of the cargo and the truck.
[0070] See also Figure 4 , showing the specific location of at least one cargo delivery point. Figure 4 It includes cargo delivery location E and cargo delivery location F.
[0071] In a second step, the at least one second pixel point is brought into the coordinate transformation model to generate at least one second world coordinate for the at least one cargo handover location point.
[0072] The third step is to generate the free volume of the target truck compartment according to the at least one second world coordinate, wherein the free volume of the target truck compartment can represent the unused space of the target truck compartment.
[0073] As an example, first, the width and length of the target truck are determined. Then, at least one second world coordinate is converted into an array to obtain at least one array. Next, the at least one array is input into a multi-layer, serially connected fully connected layer to output the free volume of the vehicle compartment.
[0074] The fourth step is to determine the cargo loading rate corresponding to the target truck based on the empty volume of the truck compartment.
[0075] As an example, the execution entity may first determine the cargo volume of the target truck. Then, the unoccupied compartment volume is subtracted from the cargo volume to obtain a subtracted volume. Finally, the subtracted volume is divided by the cargo volume to obtain a cargo loading rate.
[0076] Optionally, generating the free volume of the target truck compartment according to the at least one second world coordinate may include the following steps:
[0077] In the first step, for each world coordinate in the at least one second world coordinate, a coordinate value of the world point on a target coordinate axis is determined. The target coordinate axis may be a coordinate axis corresponding to the length of the truck. The coordinate value may be a value on the coordinate axis corresponding to the length of the truck.
[0078] In the second step, a weighted summation process is performed on the obtained at least one coordinate value to obtain a weighted summation value.
[0079] The third step is to generate the above-mentioned free volume of the carriage according to the carriage width, carriage height and the above-mentioned weighted sum value of the above-mentioned target truck.
[0080] As an example, the execution entity may multiply the vehicle compartment width, the vehicle compartment height, and the weighted sum value to generate the vehicle compartment free volume.
[0081] The above-described various embodiments of the present disclosure have the following advantageous effects: Through the cargo loading information generation methods of some embodiments of the present disclosure, cargo loading information can be generated efficiently and accurately. Specifically, the lack of accuracy and efficiency in generating cargo loading information is caused by the inability to effectively and efficiently monitor the loading status of a truck, resulting in inaccurate determination of the truck's loading status. Based on this, the cargo loading information generation methods of some embodiments of the present disclosure first obtain a cargo loading image captured by the camera with the target truck in the target position, for use in the subsequent determination of the camera's internal and external reference information. Then, based on the cargo loading image, the corresponding internal reference information can be accurately generated. The obtained internal reference information is not only used for the subsequent determination of the external reference information, but also for determining a coordinate transformation model. Furthermore, based on the internal reference information and the cargo loading image, the corresponding external reference information can be accurately determined. Furthermore, based on the internal and external reference information, a coordinate transformation model can be accurately generated, where the coordinate transformation model represents the coordinate transformation relationship between the pixel coordinate system and the world coordinate system. Finally, based on the coordinate transformation model and the cargo loading image, the cargo loading information corresponding to the target truck can be accurately generated. Here, the coordinate transformation model can be used to accurately reflect the actual loading conditions in subsequent cargo loading images to accurately generate the cargo loading information. In summary, by analyzing the image content of the cargo loading image, the internal and external reference information of the camera can be accurately determined. Furthermore, this internal and external reference information can be used to accurately generate a coordinate transformation model that can fully indirectly reflect the actual cargo loading conditions in the image. Thus, using the coordinate transformation model, the cargo loading information corresponding to the target truck can be precisely determined.
[0082] Further references Figure 5 , shows a process 500 of another embodiment of the method for generating cargo loading information according to the present disclosure. The method for generating cargo loading information includes the following steps:
[0083] Step 501: Acquire a cargo loading image captured by a camera when a target truck is at a target position.
[0084] Step 502: Determine a shadow elimination point set based on the cargo loading image.
[0085] In some embodiments, the execution entity (e.g. Figure 1 The electronic device 101 shown can determine a set of shadow-elimination points based on the cargo loading image, wherein the shadow-elimination points can be in the form of coordinates.
[0086] As an example, the execution entity may first use Hough line detection to detect the edges of the carriage in the surveillance footage, obtaining three pairs of parallel lines in the cargo loading image. Then, the three pairs of parallel lines are marked in the cargo loading image. Finally, the cargo loading image with marked parallel lines is input into a shadow cancellation point generation model to generate a shadow cancellation point set. The shadow cancellation point generation model may be a YOLO model.
[0087] In some optional implementations of some embodiments, determining the shadow elimination point set based on the cargo loading image may include the following steps:
[0088] The first step is to establish a pixel coordinate system with the target edge point of at least one edge point of the cargo box as the origin. In practice, the target edge point of the cargo box can be the edge point of the lower left corner of the cargo box, or the edge point of the lower right corner of the cargo box.
[0089] The second step is to determine the parallel lines corresponding to each plane in the pixel coordinate system. The pixel coordinate system corresponds to the X-plane, Y-plane, and Z-plane. The X-plane corresponds to the plane of the X-axis. The Y-plane corresponds to the plane of the Y-axis. The Z-plane corresponds to the plane of the Z-axis. The X-axis corresponds to the length of the truck. The Y-axis corresponds to the width of the truck. The Z-axis corresponds to the height of the truck.
[0090] See also Figure 6 , showing the parallel lines corresponding to each plane. The parallel lines include: a parallel line formed by L1 and L2, a parallel line formed by L4 and L3, and a parallel line formed by L5 and L6.
[0091] The third step is to determine the shadow vanishing point corresponding to each parallel line in the obtained parallel line set to obtain the shadow vanishing point set.
[0092] As an example, the execution subject may determine the shadow vanishing point corresponding to each parallel line in the obtained parallel line set by using a shadow vanishing point determination formula to obtain a shadow vanishing point set.
[0093] Step 503: Generate the focal length information and principal point coordinate information corresponding to the above-mentioned shooting device according to the above-mentioned shadow elimination point set.
[0094] In some embodiments, the execution entity may generate the focal length information and principal point coordinate information corresponding to the shooting device based on the shadow elimination point set.
[0095] As an example, first, each shadow cancellation point in the shadow cancellation point set is converted into a matrix to obtain a shadow cancellation matrix set. Then, based on the shadow cancellation matrix set, an alignment matrix is determined. Finally, the focal length information and principal point coordinate information corresponding to the aforementioned camera device are determined based on the symmetric matrix.
[0096] In practice, the execution entity can determine the alignment matrix based on the shadow cancellation matrix set by the following first formula:
[0097] v1 t wv2=0
[0098] v1 t wv3=0
[0099] v2 t wv3=0,
[0100] Wherein, v1 can be the shadow cancellation matrix corresponding to the first shadow cancellation point in the shadow cancellation matrix set. v2 can be the shadow cancellation matrix corresponding to the second shadow cancellation point in the shadow cancellation matrix set. v3 can be the shadow cancellation matrix corresponding to the third shadow cancellation point in the shadow cancellation matrix set. w can be a symmetric matrix.
[0101] In practice, the execution entity may determine the focal length information and principal point coordinate information corresponding to the shooting device according to the symmetric matrix using the following second formula:
[0102] W=(KK T ) -1 ,
[0103]
[0104] Among them, K can be the shooting internal parameter matrix composed of focal length information and principal point coordinate information. X Can be focal length. cx . It can be the principal point coordinates. cy is also the principal point coordinates.
[0105] Step 504: Generate the shooting internal reference information according to the focal length information and the principal point coordinate information.
[0106] In some embodiments, the execution entity may generate the shooting internal reference information based on the focal length information and the principal point coordinate information.
[0107] As an example, the execution entity may fuse the focal length information and the principal point coordinate information to generate shooting internal reference information.
[0108] Step 505 : Determine the shooting external reference information corresponding to the shooting device according to the shooting internal reference information and the cargo loading image.
[0109] Step 506: Generate a coordinate transformation model based on the shooting internal reference information and the shooting external reference information.
[0110] Step 507 : Generate cargo loading information corresponding to the target truck based on the coordinate conversion model and the cargo loading image.
[0111] In some embodiments, the specific implementation of steps 501, 505-507 and the technical effects thereof can be referred to in Figure 2 Steps 201, 203-205 in the corresponding embodiment will not be repeated here.
[0112] from Figure 5 It can be seen that Figure 2 Compared with the description of some corresponding embodiments, Figure 5 In the corresponding process 500 of the cargo loading information generation method in some embodiments, by determining the shadow cancellation point set corresponding to the cargo loading image, the focal length information and the principal point coordinate information can be subsequently accurately determined, thereby quickly and efficiently generating the shooting internal reference information.
[0113] Further references Figure 7 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a cargo loading information generating device. These device embodiments are similar to Figure 2 Corresponding to the method embodiments shown, the cargo loading information generating device can be specifically applied to various electronic devices.
[0114] like Figure 7 As shown, a cargo loading information generating device 700 includes: an acquisition unit 701, a first generation unit 702, a determination unit 703, a second generation unit 704, and a third generation unit 705. The acquisition unit 701 is configured to acquire a cargo loading image captured by a camera with a target truck at a target position; the first generation unit 702 is configured to generate intrinsic reference information corresponding to the camera based on the cargo loading image; the determination unit 703 is configured to determine extrinsic reference information corresponding to the camera based on the intrinsic reference information and the cargo loading image; the second generation unit 704 is configured to generate a coordinate transformation model based on the intrinsic reference information and the extrinsic reference information, wherein the coordinate transformation model represents the coordinate transformation relationship between a pixel coordinate system and a world coordinate system; and the third generation unit 705 is configured to generate cargo loading information corresponding to the target truck based on the coordinate transformation model and the cargo loading image.
[0115] In some optional implementations of some embodiments, the first generation unit 702 can be further configured to: determine a shadow cancellation point set based on the above-mentioned cargo loading image; generate focal length information and principal point coordinate information corresponding to the above-mentioned shooting device based on the above-mentioned shadow cancellation point set; and generate the above-mentioned shooting internal parameter information based on the above-mentioned focal length information and the above-mentioned principal point coordinate information.
[0116] In some optional implementations of some embodiments, the determination unit 703 can be further configured to: obtain at least one first pixel point coordinate corresponding to at least one cargo box edge position point in the above-mentioned cargo loading image; obtain at least one first world coordinate corresponding to the above-mentioned at least one cargo box edge position point; determine the shooting external parameter information corresponding to the above-mentioned shooting device based on the above-mentioned shooting internal parameter information, the above-mentioned at least one first pixel point coordinate and the above-mentioned at least one first world coordinate.
[0117] In some optional implementations of some embodiments, the above-mentioned cargo loading information is a cargo loading rate; and the third generation unit 705 can be further configured to: determine at least one second pixel point corresponding to at least one cargo handover position point in the above-mentioned cargo loading image; bring the above-mentioned at least one second pixel point into the above-mentioned coordinate transformation model to generate at least one second world coordinate for the above-mentioned at least one cargo handover position point; generate the empty volume of the compartment of the above-mentioned target truck based on the above-mentioned at least one second world coordinate; and determine the cargo loading rate corresponding to the above-mentioned target truck based on the above-mentioned empty volume of the compartment.
[0118] In some optional implementations of some embodiments, the first generation unit 702 can be further configured to: establish a pixel coordinate system with a target cargo box edge position point among at least one cargo box edge position point as the origin; for each plane under the above pixel coordinate system, determine the parallel lines corresponding to the above plane; determine the shadow vanishing point corresponding to each parallel line in the obtained parallel line set to obtain a shadow vanishing point set.
[0119] In some optional implementations of some embodiments, the third generation unit 705 can be further configured to: for each world coordinate in the above-mentioned at least one second world coordinate, determine the coordinate value of the above-mentioned world point on the target coordinate axis; perform weighted summation processing on the at least one obtained coordinate value to obtain a weighted sum value; and generate the above-mentioned car compartment vacant volume based on the car compartment width, car compartment height and the above-mentioned weighted sum value of the above-mentioned target truck.
[0120] It is understood that the units recorded in the cargo loading information generating device 700 are similar to those in the reference Figure 2Therefore, the operations, features and beneficial effects described above for the method are also applicable to the cargo loading information generating device 700 and the units included therein, and will not be described in detail here.
[0121] Reference below Figure 8 , which shows an electronic device (eg, Figure 1 Schematic diagram of the structure of the electronic device 101)800. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0122] like Figure 8 As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory 802 or a program loaded from a storage device 808 into a random access memory 803. Various programs and data required for the operation of the electronic device 800 are also stored in the random access memory 803. The processing device 801, the read-only memory 802, and the random access memory 803 are connected to each other via a bus 804. An input / output interface 805 is also connected to the bus 804.
[0123] Typically, the following devices may be connected to the input / output interface 805: an input device 806 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 808 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead. Figure 8 Each block shown in the figure may represent one device, or may represent multiple devices as needed.
[0124] In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network via the communication device 809, or installed from the storage device 808, or installed from the read-only memory 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of some embodiments of the present disclosure are performed.
[0125] It should be noted that in some embodiments of the present disclosure, the computer-readable medium mentioned above may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.
[0126] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0127] The computer-readable medium may be included in the electronic device, or may exist independently and not be incorporated into the electronic device. The computer-readable medium carries one or more programs. When executed by the electronic device, the electronic device: obtains a cargo loading image captured by a camera with a target truck at a target location; generates internal reference information corresponding to the camera based on the cargo loading image; determines external reference information corresponding to the camera based on the internal reference information and the cargo loading image; generates a coordinate transformation model based on the internal reference information and the external reference information, wherein the coordinate transformation model represents the coordinate transformation relationship between a pixel coordinate system and a world coordinate system; and generates cargo loading information corresponding to the target truck based on the coordinate transformation model and the cargo loading image.
[0128] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0129] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0130] The units described in some embodiments of the present disclosure may be implemented in software or hardware. The units described may also be provided in a processor. For example, they may be described as follows: a processor includes an acquisition unit, a first generation unit, a determination unit, a second generation unit, and a third generation unit. The names of these units do not, in some cases, limit the units themselves. For example, the acquisition unit may also be described as a "unit for acquiring an image of cargo loading captured by a camera when a target truck is in a target position."
[0131] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
[0132] Some embodiments of the present disclosure further provide a computer program product, including a computer program, which implements any of the above-mentioned cargo loading information generation methods when executed by a processor.
[0133] The above description is only an illustration of some preferred embodiments of the present disclosure and the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.
Claims
1. A method for generating cargo loading information, comprising: Acquiring a cargo loading image captured by a camera when the target truck is at a target position; generating shooting internal reference information corresponding to the shooting device according to the cargo loading image; Determining shooting external reference information corresponding to the shooting device according to the shooting internal reference information and the cargo loading image; Generate a coordinate transformation model according to the shooting internal reference information and the shooting external reference information, wherein the coordinate transformation model represents a coordinate transformation relationship between a pixel coordinate system and a world coordinate system; Generate cargo loading information corresponding to the target truck according to the coordinate transformation model and the cargo loading image.
2. The method according to claim 1, wherein Generating shooting internal reference information corresponding to the shooting device according to the cargo loading image includes: determining a shadow elimination point set according to the cargo loading image; Generating focal length information and principal point coordinate information corresponding to the shooting device according to the shadow cancellation point set; The shooting internal reference information is generated according to the focal length information and the principal point coordinate information.
3. The method according to claim 1, wherein The determining, based on the shooting internal reference information and the cargo loading image, shooting external reference information corresponding to the shooting device includes: Obtaining at least one first pixel coordinate corresponding to at least one cargo box edge position point in the cargo loading image; Obtain at least one first world coordinate corresponding to the at least one cargo box edge position point; Determine shooting extrinsic parameter information corresponding to the shooting device according to the shooting intrinsic parameter information, the at least one first pixel point coordinate and the at least one first world coordinate.
4. The method according to claim 1, wherein The cargo loading information is the cargo loading rate; as well as Generating cargo loading information corresponding to the target truck according to the coordinate transformation model and the cargo loading image includes: determining at least one second pixel point corresponding to at least one cargo handover location point in the cargo loading image; Bringing the at least one second pixel point into the coordinate transformation model to generate at least one second world coordinate for the at least one cargo handover location point; generating a free volume of the compartment of the target truck according to the at least one second world coordinate; The cargo loading rate corresponding to the target truck is determined according to the free volume of the compartment.
5. The method according to claim 2, wherein: Determining a shadow elimination point set based on the cargo loading image includes: Establishing a pixel coordinate system with a target cargo box edge position point among at least one cargo box edge position point as the origin; For each plane in the pixel coordinate system, determining a parallel line corresponding to the plane; Determine the shadow vanishing point corresponding to each parallel line in the obtained parallel line set to obtain a shadow vanishing point set.
6. The method according to claim 4, wherein: The generating the free volume of the compartment of the target truck according to the at least one second world coordinate comprises: For each world coordinate in the at least one second world coordinate, determining a coordinate value of the world point on the target coordinate axis; Performing weighted sum processing on the obtained at least one coordinate value to obtain a weighted sum value; The compartment vacant volume is generated according to the compartment width, compartment height and the weighted sum value of the target truck.
7. An information generating device, comprising: an acquisition unit configured to acquire a cargo loading image captured by a camera when a target truck is at a target position; a first generating unit configured to generate shooting internal reference information corresponding to the shooting device based on the cargo loading image; a determining unit configured to determine shooting external reference information corresponding to the shooting device based on the shooting internal reference information and the cargo loading image; A second generating unit is configured to generate a coordinate transformation model according to the shooting internal reference information and the shooting external reference information, wherein the coordinate transformation model represents a coordinate transformation relationship between a pixel coordinate system and a world coordinate system; The third generating unit is configured to generate cargo loading information corresponding to the target truck according to the coordinate conversion model and the cargo loading image.
8. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.
9. A computer-readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
10. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.