Luggage compartment attitude recognition system, method, electronic device, and storage medium

Abstract

The present invention discloses a luggage compartment attitude recognition system, method, electronic device, and storage medium. The system includes a processor, a binocular camera, a first vehicle, a second vehicle, and a sign unit, the first vehicle includes a luggage compartment, the second vehicle is used to unload materials into the luggage compartment of the first vehicle, the sign unit is provided on the side of the luggage compartment, and the sign unit is provided with a number of reference points, and the binocular camera is provided on the second vehicle, the binocular camera collects an image including the sign unit and transmits the image to the processor, the processor recognizes and processes the image using an image recognition network model to obtain a target image including the recognition result of the number of reference points, determines three-dimensional coordinates of a target space for each reference point in the target image through a binocular camera stereoscopic geometric vision algorithm, and determines vehicle attitude information based on the three-dimensional coordinates of the target space of each reference point. The luggage compartment attitude recognition system has the characteristics of being simple in structure and suitable for large-scale outdoor unmanned driving scenes.

Description

[Technical field]

[0001] The present invention relates to the field of computer technology, and in particular to a luggage compartment attitude recognition system, method, electronic device and storage medium. [Background technology]

[0002] Construction machinery and equipment such as earthmoving machinery, mining machinery, and road machinery require the coordination of multiple machines in the upgrade of unmanned autonomous driving. For example, an excavator needs to unload cargo onto a truck. In such an application scenario, the excavator needs to have the ability to detect the real-time position and attitude of the truck compartment.

[0003] Usually, the position of the truck in the map coordinate system can be detected by the phase difference positioning principle of the real-time kinematic (RTK) carrier phase difference technology. Also, the map coordinate system position of the cargo compartment can be estimated by the size information of the truck itself. Furthermore, the excavator can obtain its own position in the map coordinate system by using RTK positioning. By comparing the two, the relative position of the truck cargo compartment and the excavator can be obtained, and then unloading can be completed.

[0004] Although this method can complete the outdoor unmanned autonomous driving of the excavator, the problem caused by it is the high cost. In order to obtain the position information of the truck and the excavator on the map, it is necessary to install a set of RTK mobile stations on the truck and the excavator, and also to install a public base station. In this case, the cost will be tens of thousands to hundreds of thousands of yuan, which makes it impossible to realize large-scale application. Therefore, this solution is not suitable for large-scale application of unmanned autonomous driving outdoors in construction machinery rooms. Summary of the Invention

[0005] The present invention aims to solve the technical problems that the systems used in conventional unmanned autonomous driving have complex structures, low levels of intelligence, high costs, and narrow ranges of application.

[0006] In order to solve the above problem, in one aspect, the present invention provides a luggage compartment attitude recognition system including a processor, a binocular camera, a first vehicle, a second vehicle, and a sign unit, The first vehicle includes a luggage compartment, the second vehicle is used to unload materials into the cargo compartment of the first vehicle; The sign portion is provided on a side surface of the luggage compartment, and a plurality of reference points are provided on the sign portion, the binocular camera is provided on the second vehicle, the binocular camera being used to collect an image including the sign portion and to transmit the image to the processor; The processor is communicatively connected to the binocular camera, and the processor uses an image recognition network model to recognize and process the image to obtain a target image including the recognition results of the multiple reference points, determines three-dimensional coordinates of a target space for each of the multiple reference points in the target image using a stereoscopic geometric vision algorithm of the binocular camera, and determines posture information of the luggage compartment based on the three-dimensional coordinates of the target space for each reference point.

[0007] Optionally, the second vehicle includes an unloading device and a body structure connected to each other; the unloading device is connected to a first side of the body structure; The binocular camera is provided on the first side.

[0008] Optionally, the first side includes a first attachment point and a second attachment point; the first attachment point and the second attachment point are located on an upper portion of the first side surface, a first preset distance exists between the first attachment point and the second attachment point; the binocular camera includes a first binocular camera and a second binocular camera; the first binocular camera is mounted at the first mounting point; The second binocular camera is mounted to the second mounting point.

[0009] Optionally, the sign portion includes at least two sub-sign portions; At least two of the sub-signal portions are located at a first position point and a second position point on a second side surface of the luggage compartment, respectively; the second side surface is a surface facing the first side surface, There is a second preset distance between the first position point and the second position point, the second preset distance being longer than half the length of the second side in a first predetermined direction, the first predetermined direction being perpendicular to the second predetermined direction, and the second predetermined direction being an extension of the height direction of the first vehicle.

[0010] Optionally, the processor includes an image recognition module and a position determination module; The image recognition module is used to recognize and process the image using an image recognition network model to obtain a target image, and transmit the target image to a position determination module, the target image being an image including the recognition result of the plurality of reference points; The position determination module determines three-dimensional coordinates in a target space for each of a plurality of reference points in the target image through a stereoscopic geometric vision algorithm of a binocular camera, and determines attitude information of the luggage compartment based on the three-dimensional coordinates in the target space of each reference point.

[0011] In another aspect, the present invention provides Collect images including the sign using a binocular camera; inputting the image into an image recognition network model to obtain a target image, the target image including a recognition result for each of the plurality of reference points; For each of the reference points, determining coordinates of the reference point in a pixel coordinate system based on the target image and the recognition result of the reference point; Obtaining camera parameters of the binocular camera; Determine a three-dimensional coordinate of the reference point in a target space according to the camera parameters of the binocular camera and the coordinate of the reference point in a pixel coordinate system through a binocular camera stereoscopic geometric vision algorithm; Determine the three-dimensional coordinates of the target space of the luggage compartment based on the three-dimensional coordinates of the target space of each reference point; determining attitude information of the luggage compartment based on three-dimensional coordinates of a target space of the luggage compartment; There is further provided a luggage compartment posture recognition method, including:

[0012] Preferably, the camera parameters include a distance between a left eye camera and a right eye camera of a binocular camera, mounting parameters and internal parameters.

[0013] Selectively, determining three-dimensional coordinates of the target space of the reference point based on camera parameters of the binocular camera and coordinates of the reference point in a pixel coordinate system, determining three-dimensional coordinates of the target space of the luggage compartment based on the target three-dimensional coordinates of each reference point, and determining attitude information of the luggage compartment based on the target three-dimensional coordinates of the luggage compartment; determining a three-dimensional coordinate of the reference point in a camera coordinate system based on a distance between a left eye camera and a right eye camera of the binocular camera, an internal parameter of the binocular camera, and a coordinate of the reference point in a pixel coordinate system; determining a first coordinate transformation matrix based on mounting parameters of the binocular camera; Transforming the three-dimensional coordinates of the reference point in the camera coordinate system into three-dimensional coordinates in a target vehicle coordinate system according to the first coordinate transformation matrix, the target vehicle coordinate system being a coordinate system in which the second vehicle is located; determining three-dimensional coordinates of the luggage compartment in the target vehicle coordinate system based on three-dimensional coordinates of each reference point in the target vehicle coordinate system; determining attitude information of the luggage compartment based on three-dimensional coordinates of the luggage compartment in the target vehicle coordinate system; Includes.

[0014] In another aspect, the present invention provides an electronic device including a processor and a memory, wherein at least one instruction, at least one program, code set or instruction set is stored in the memory, and wherein the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor, thereby realizing the luggage compartment posture recognition method.

[0015] In another aspect, the present invention provides a computer storage medium having at least one instruction or at least one program stored therein, the at least one instruction or at least one program being loaded and executed by a processor to realize the luggage compartment posture recognition method.

[0016] According to the above technical means, the luggage compartment attitude recognition method according to the present invention has the following beneficial effects. The luggage compartment attitude recognition system includes a processor, a binocular camera, a first vehicle, a second vehicle, and a sign unit, the first vehicle is used by the second vehicle to unload materials into the luggage compartment of the first vehicle, the sign unit is provided on the side of the luggage compartment, and the sign unit is provided with a plurality of reference points, the binocular camera is provided on the second vehicle, the binocular camera collects an image including the sign unit and transmits the image to the processor, the processor is connected to the binocular camera, the processor performs recognition processing on the image using an image recognition network model to obtain a target image, the target image is an image including the recognition result of the plurality of reference points, the binocular camera determines target three-dimensional coordinates for each of the plurality of reference points in the target image by a stereoscopic geometric vision algorithm, and determines attitude information of the vehicle based on the target three-dimensional coordinates of each reference point. The luggage compartment attitude recognition system according to the present invention has a simple structure and is applicable to large-scale outdoor unmanned driving scenes. [Brief description of the drawings]

[0017] In order to more clearly describe the technical means in the embodiments of the present invention, the following briefly introduces the drawings required in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and those skilled in the art can also obtain other drawings based on these drawings without creative efforts. [Figure 1] 1 is a structural schematic diagram of a selectable luggage compartment attitude recognition system according to the present invention; [Diagram 2] FIG. 2 is a structural schematic diagram of a selectable second vehicle of the present invention. [Diagram 3] FIG. 2 is a structural schematic diagram of a selectable first vehicle of the present invention. [Figure 4] FIG. 2 is a structural schematic diagram of a selectable label portion of the present invention. [Diagram 5] 3 is a flowchart of a selectable luggage compartment attitude recognition method of the present invention. [Figure 6] 4 is a flowchart of another selectable luggage compartment attitude recognition method of the present invention. [Figure 7] FIG. 13 is a diagram showing the relationship between multiple selectable coordinates of the present invention. [Figure 8] FIG. 2 is a schematic diagram of a selectable binocular stereoscopic vision camera model of the present invention. [Figure 9] 11 is a schematic diagram showing the relationship between a plurality of selectable coordinates according to another embodiment of the present invention.

[0018] 1-first vehicle; 101-luggage compartment; 2-second vehicle; 201-unloading device; 202-main body structure; 2021-first side; 3-signal section; 301-sub-signal section; 4-binocular camera; 401-first binocular camera; 402-second binocular camera; 5-processor DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, the technical means in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and are not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts are all included in the protection scope of the present invention.

[0020] The term "one embodiment" or "embodiment" as used herein means a particular feature, structure, or characteristic that may be included in at least one embodiment of the present invention. In the description of the present invention, the orientation or positional relationship indicated by terms such as "upper", "lower", "upper", "top", "bottom", etc., is based on the orientation or positional relationship shown in the drawings, and is merely for the purpose of illustrating and simplifying the description of the present invention, and does not indicate or suggest that the device or part referred to has a particular orientation and must be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first" and "second" are used merely for descriptive purposes, and cannot be understood as indicating or suggesting a relative importance or implicitly indicating the number of technical features shown. Thus, a feature qualified by "first" or "second" can explicitly or implicitly include one or more of the said features. In addition, the terms "first", "second", etc. are not intended to describe a particular order, but to distinguish between similar objects. Data used in this manner are interchangeable where appropriate, and the embodiments of the present invention described herein can be implemented in an order other than that shown or described herein.

[0021] Additionally, the terms "comprise" and "have" and any variations thereof are intended to cover exclusive inclusions, e.g., a process, method, system, product, or server comprising a series of steps or units is not necessarily limited to the steps or units explicitly recited, but may include other steps or units not explicitly shown or inherent to those processes, methods, products, or apparatus.

[0022] In general, the luggage compartment position detection method includes:

[0023] First, the position of the cargo compartment is detected visually at a fixed position. For example, several key points of the cargo compartment that moves back and forth between multiple positions are detected visually to determine the position of the cargo compartment, and then the work operation is performed. This method is applied to the position detection of the cargo compartment that is operated redundantly in a single scene, which belongs to repeated detection and has a low level of intelligence, and is not suitable for detecting the dynamic position and posture of the truck cargo compartment in the case of outdoor unmanned autonomous driving of an excavator.

[0024] Secondly, relative position correction is performed using a laser spot. In the automated guided vehicle (AGV) industry, a laser transmitter attached to the AGV and a laser receiver on the loading platform are used to correct the relative position between the AGV and the loading platform to guide the dump of the AGV. This type of sensing of the shelf position of the AGV can only be applied to precise terminal correction on a fixed route, and cannot be used to detect the dynamic position and attitude of the loading platform of a truck for outdoor unmanned autonomous driving of an excavator.

[0025] Third, the detection of trucks during unmanned driving. Currently, in unmanned driving, the truck in front is also detected to obtain the spatial relative position between the truck in front and the vehicle itself, but in unmanned driving, the exact position and posture of the truck's cargo compartment are not taken into account, and the truck is detected as a whole, and the rack is judged as an obstacle, so the truck detection used in unmanned driving cannot be directly used to accurately detect the cargo compartment position and posture of outdoor unmanned autonomous driving trucks of construction machinery.

[0026] Fourth, the RTK truck position and attitude detection described in the background art above. RTK can detect the position of the truck in the map coordinate system by the differential positioning principle, and can estimate the map coordinate system position of the cargo compartment by the size information of the truck itself. Furthermore, the excavator can obtain its own position in the map coordinate system by RTK positioning. By comparing the two, the relative position of the truck cargo compartment and the excavator can be obtained, and unloading can be completed. Although such a method can complete the unmanned autonomous driving of the excavator outdoors, the problem caused by it is high cost. In order to obtain the position information of the truck and the excavator on the map, it is necessary to install one set of RTK mobile stations on the truck and the excavator, and one public base station. In that case, the cost will be tens of thousands to hundreds of thousands of yuan, and large-scale application cannot be realized. Therefore, such a solution is not suitable for large-scale application of unmanned autonomous driving outdoors in construction machinery rooms.

[0027] The above luggage compartment detection method has the following drawbacks. 1) Low level of intelligence. Multi-position visual cargo compartment detection and point-to-point laser relative position correction detection do not have the ability of intelligent detection, and the level of intelligence is low, so they cannot be used for detecting the dynamic position and attitude of the truck cargo compartment of an unmanned autonomous driving excavator outdoors. 2) Complex structure and high cost: RTK positioning technology can be used for outdoor unmanned autonomous operation of construction machinery, but it is expensive, cannot be applied on a large scale, and is only suitable for early exploratory research. 3) Not suitable for outdoor operation of construction machinery. None of the existing mass production schemes can be applied to outdoor unmanned autonomous driving of construction machinery. The truck detection used for unmanned driving detects the truck, but detects the entire truck as an obstacle and does not pay attention to the exact position and posture of the cargo compartment. Other schemes, such as multi-position visual cargo compartment detection and point-to-point laser relative position correction detection, have a low level of intelligence and cannot be used to dynamically sense the cargo compartment position and posture of outdoor unmanned autonomous driving trucks of construction machinery. RTK positioning technology is expensive and cannot be applied to large-scale applications of outdoor unmanned autonomous driving of construction machinery.

[0028] For this purpose, please refer to Fig. 1, which is a structural schematic diagram of an optional luggage compartment attitude recognition system of the present invention. The present invention provides a luggage compartment attitude recognition system. The system includes a processor 5, a binocular camera 4, a first vehicle 1, a second vehicle 2, and a sign unit 3, the first vehicle 1 includes a cargo compartment 101, the second vehicle 2 is used to unload materials into the cargo compartment 101 of the first vehicle 1, the sign unit 3 is provided on a side of the cargo compartment 101, the sign unit 3 is provided with a plurality of reference points, the binocular camera 4 is provided on the second vehicle 2, the binocular camera 4 collects an image including the sign unit 3 and transmits the image to the processor 5, the processor 5 is connected in communication with the binocular camera 4, the processor 5 recognizes and processes the image using an image recognition network model to obtain a target image, the target image is an image including the recognition result of the plurality of reference points, the stereoscopic geometric vision algorithm of the binocular camera 4 determines target three-dimensional coordinates for each of the plurality of reference points in the target image, and determines posture information of the cargo compartment 101 based on the target three-dimensional coordinates of each reference point.

[0029] Alternatively, the first vehicle 1 may be a vehicle having a luggage compartment 101, for example a truck, and the second vehicle 2 may be a vehicle having a robotic arm, for example a shovel.

[0030] Alternatively, the processor 5 may be located in the second vehicle 2 or in a terminal or server separate from this vehicle.

[0031] Alternatively, the server may be an independent physical server, a server cluster or a distributed system consisting of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud audio recognition model training, middleware services, domain name services, security services, CDN (Content Delivery Network), big data and artificial intelligence platforms, etc. The operating system running on the server may include, but is not limited to, Android system, IOS system, Linux, windows, Unix, etc.

[0032] Optionally, the terminal may include, but is not limited to, a type of client such as a smartphone, a desktop computer, a tablet computer, a notebook computer, a smart speaker, a digital assistant, an augmented reality (AR) / virtual reality (VR) device, a smart wearable device, etc. The software executed on the client may be, for example, an application program, an applet, etc. Optionally, the operating system executed on the client may include, but is not limited to, an Android system, an IOS system, Linux, windows, Unix, etc.

[0033] In an optional example, refer to Fig. 2, which is a structural schematic diagram of an optional second vehicle of the present invention. The second vehicle 2 includes an unloading device 201 and a body structure 202 connected to each other, the unloading device 201 is connected to a first side 2021 of the body structure 202, and the binocular camera 4 is provided on the first side 2021.

[0034] To further ensure that the binocular camera can collect image information of all sign portions, in an optional example, the first side 2021 includes a first mounting point and a second mounting point, the first mounting point and the second mounting point are located at the top of the first side 2021, there is a first preset distance between the first mounting point and the second mounting point, and the binocular camera 4 includes a first binocular camera 401 and a second binocular camera 402, the first binocular camera 401 is mounted at the first mounting point and the second binocular camera 402 is mounted at the second mounting point.

[0035] In order to improve the width of the viewing angle of the binocular camera 4 and ensure the effect of the collected image, optionally, the first preset distance is longer than half the width of the first side 2021, and preferably, the first mounting point is located at the leftmost side of the first side 2021 and the second mounting point is located at the rightmost side of the first side 2021 to further improve the image collection effect.

[0036] In a selectable example, refer to FIG. 3, which is a structural schematic diagram of a selectable first vehicle of the present invention. In order to improve the accuracy of the luggage compartment attitude to be determined later, the sign unit 3 includes at least two sub-sign units 301, the at least two sub-sign units 301 are located at a first position point and a second position point of the second side surface of the luggage compartment 101, respectively, the second side surface is a surface facing the first side surface 2021, there is a second preset distance between the first position point and the second position point, the second preset distance is longer than half the length of the second side surface in a first predetermined direction, the first predetermined direction (x-axis direction in FIG. 3) is perpendicular to the second predetermined direction (y-axis direction in FIG. 3), and the second predetermined direction is an extension line direction of the height direction of the first vehicle 1. As a result, the processor 5 can determine the spatial three-dimensional coordinates of the multiple sub-sign units 301 and measure the attitude information of the luggage compartment 101, such as the distance between the luggage compartment 101 and the second vehicle 2 and the deflection angle with respect to the second vehicle 2.

[0037] In this embodiment, the second side may refer to multiple sides. In an actual scene, the first vehicle 1 does not necessarily stop directly in front of the second vehicle 2, and may have a certain angle with the second vehicle 2. In this case, the second side may be two sides corresponding to the first side 2021. In this case, the multiple sub-sign units 301 may be located on one of the two sides, or may be provided in numbers corresponding to the two sides. In this embodiment, a case in which all the sub-sign units 301 are located on the same side will be described as an example.

[0038] Optionally, referring to Fig. 4, Fig. 4 is a structural schematic diagram of a selectable sign unit of the present invention. The sign unit 3 is provided with five reference points, but in practice, it may be n, such as 5 or more, for example, 6, 7, 8, 9, 10, etc., as needed. The more the number of reference points, the more accurate the coordinate data of the sign unit 3 that is finally determined, but if the number of reference points is too large, the calculation time will further increase and the data processing will take too long, so the number of reference points can be set according to specific needs.

[0039] In a selectable example, the processor 5 includes an image recognition module and a position determination module, the image recognition module performs recognition processing on the image using an image recognition network model, obtains a target image including the recognition results of the multiple reference points, and transmits the target image to a position determination module, the position determination module determines target three-dimensional coordinates for each of the multiple reference points in the target image through a stereoscopic geometric vision algorithm of the binocular camera 4, and determines posture information of the luggage compartment 101 based on the target three-dimensional coordinates of each reference point. For details, please refer to the description of the luggage compartment posture recognition method described later.

[0040] The luggage compartment 101 posture recognition system according to the present invention has the following advantages. 1) High level of intelligence: Based on deep convolutional neural network technology in the field of artificial intelligence, the binocular camera 4 can utilize the images and the stereoscopic geometric vision of the binocular camera 4 to accurately detect the position and attitude of the truck compartment 101 relative to the truck under various weather conditions and in various working environments. 2) Single-end detection. The binocular camera 4 performs single-end detection, recognizing and measuring objects in the outside world like the human eye. There is no need to attach mobile stations to both the truck and the excavator, as with RTK. The RTK positioning results of the truck must be transmitted to the unmanned excavator via a communication terminal, and after receiving the truck positioning data, the unmanned excavator can obtain the relative positional relationship between the two only by comparing it with its own positioning data. 3) Low cost. Currently, the binocular camera 4 that has reached the vehicle grade has waterproof and dustproof capabilities, and the cost can be reduced to several thousand yuan in large-scale applications, which is low cost. 4) Meet the large-scale application of outdoor unmanned autonomous driving of construction machinery. Based on the deep convolutional neural network technology in the field of artificial intelligence, by utilizing images and binocular stereoscopic geometric vision, the binocular camera 4 can detect the exact position and attitude of the cargo compartment 101 of the truck under various weather conditions and in various working environments. At the same time, it is low cost and can realize the large-scale application of outdoor unmanned autonomous driving of construction machinery.

[0041] A specific embodiment of the luggage compartment posture recognition method of the present invention will be described below. FIG. 5 is a flowchart of a selectable luggage compartment posture recognition method of the present invention. This specification provides the operation steps of the method as in the embodiment or the flowchart, but may include more or less operation steps due to ordinary or non-inventive efforts. The step order given in the embodiment is one of many step execution orders, and does not represent the only execution order. When an actual system or server product executes, it can execute according to the method shown in the embodiment or the drawings or execute in parallel (such as a parallel processor 5 or multi-thread processing environment). Specifically, as shown in FIG. 5, the method can include the following steps.

[0042] S501: An image including the sign portion 3 is collected by the binocular camera 4.

[0043] In this embodiment, the manner of installing the binocular camera 4 is shown in FIG. 2, and the detailed description of the system is given above.

[0044] Optionally, the binocular camera 4 includes a left-eye camera and a right-eye camera. In step S501, specifically, a first image including the sign 3 is collected by the left-eye camera, the first image being a visible light image, and a second image including the sign 3 is collected by the right-eye camera, the second image being a visible light image. In the following step S503, the input images include the first image and the second image, and later, the target three-dimensional coordinates are calculated based on the recognition results of the sign in the two images.

[0045] S503: Input the image into an image recognition network model to obtain a target image, which includes a recognition result for each of the plurality of reference points.

[0046] Optionally, the image recognition network model includes a feature extraction network, a feature fusion network, and a predictive recognition network. Step S503 may include: performing a feature extraction operation on an image by a feature extraction network to obtain a feature image set (image collection), performing a feature fusion process on the feature image set by a feature fusion network to obtain a target feature image, and performing a prediction process on the target feature image by a predictive recognition network to obtain a target image.

[0047] Optionally, the feature extraction network includes an input layer and a sub-feature extraction network, where the input layer generally normalizes image data and inputs it into the neural network for inference. There are several normalization methods, one of which is to normalize image pixel values ​​from 0-255 to 0-1. The normalization formula for image pixel values ​​is as follows: X1=x / 255 Here, x represents a pixel value corresponding to a pixel point of the image, and X1 represents a value after normalization processing is performed on the pixel value of the pixel point.

[0048] For example, if x=200, normalization results in X1=0.78.

[0049] Next, image features are continuously extracted from the normalized input layer image data through convolution and activation operations.

[0050] Optionally, in the feature extraction process, the convolution kernel may be set to 1*1 or 3*3, and is not limited here.

[0051] After the convolution operation, the current output of the convolution layer is obtained by the activation function. There are several types of activation functions, usually ReLU and Sigmoid.

[0052] Optionally, the deep convolutional neural network Yolo5 KeyPoints used in this embodiment has generalization ability and can effectively identify specific pattern markers 3 and their five reference points in different weather conditions and different working environments. Yolo5 KeyPoints uses CSP-Darknet53 as the backbone feature extraction network, SPPF, CSP-PAN as the feature fusion network, and then uses the class prediction sub-network (class subnet) to predict the class of each network, the detection box regression sub-network (box subnet) regression detection box, and the reference point regression sub-network (key point subnet) regression reference point to obtain the final network prediction result. The reference point detection loss function Yolo5 KeyPoints uses the Wing Loss function. Wing Loss has the following characteristics: when the loss is large, the parameter gradient is small and it is not sensitive to outliers; when the loss is small, the parameter gradient is large, the model convergence is better, and the detection accuracy of the reference point is greatly improved.

[0053] As can be seen from the above description, the backbone feature extraction network is generally followed by a feature fusion network, which performs feature fusion on the different-scale feature images extracted by the backbone feature extraction network, and then performs further convolutional extraction features.

[0054] The feature fusion network may include an enhanced feature extraction network, which may be implemented by a pyramid stacking convolution operation of features. One typical enhanced feature extraction network is FPN. The FPN network fuses a lower-layer high-resolution feature image with a higher-layer high-semantic feature image, and makes predictions independently on the feature images of each layer after the fusion to obtain prediction results.

[0055] After feature fusion and enhanced feature extraction network, the network prediction results, i.e., detection box regression results, control point regression results and class confidence values, are generally obtained by 1x1 convolution.

[0056] Tests have shown that when the above-mentioned position and orientation recognition algorithm for the cargo compartment 101 is deployed on a CPU with a calculation speed of about 130 ms / time, it can meet the real-time requirements in low-speed scenes such as unmanned autonomous driving of a shovel, while when it is deployed on AI acceleration hardware (e.g., GPU, AI chip), the real-time performance can be further improved to within 30 ms / time.

[0057] The deep convolutional neural network Yolo5 KeyPoints can only be equipped with the ability to distinguish a specific pattern marker 3 and its five reference points through advance data collection, data labeling and model training.

[0058] Optionally, the process of training an image recognition network model to recognize the object is as follows. 1) A training sample dataset is obtained, the training sample dataset including each sample image in a plurality of sample images and a corresponding label image, each of the sample images including a sign portion 3 and a plurality of reference points located on the sign portion 3, and the label image is an image corresponding to a bounding box mark of the sign portion 3 in the sample image and a mark of each reference point in the sign portion 3.

[0059] In this embodiment, the sample images may be images of the sign unit 3 captured by a binocular camera under various weather conditions and in various working environments. After the image collection is completed, images that meet the requirements are manually selected and labeled. When selecting images, images that are meaningful in various weather conditions and various working environments and completely contain the pattern of the sign unit 3 are selected, and only one image that appears multiple times is retained.

[0060] In the process of generating the label image, data labeling is performed using the labelme tool to obtain the bounding box of each image label 3 and the label file of the five reference point positions.

[0061] In the specific operation process, click the Create Target Rectangle Frame button in the annotation software labelme, create a new target frame to frame the signboard 3, input the bounding box name "Signalboard", thereby obtaining the label of the signboard 3, then click the Create Point Target button, click the five reference points of the signboard 3 pattern, input the reference point names: point 1 (point1), point 2 (point2), point 3 (point3), point 4 (point4), point 5 (point5) to obtain the labels of the reference points, label all the signboards 3 and their five reference points in the complete image, and obtain the label image of this image.

[0062] 2) Construct a predetermined deep learning model, and determine the predetermined deep learning model as a current deep learning model.

[0063] 3) performing a label prediction operation on a sample image in a sample dataset based on the current deep learning model, and determining a label prediction result for the sample image;

[0064] 4) Determine a loss value based on the labels of the sample images and the label prediction results.

[0065] 5) if the loss value is greater than a predetermined threshold, perform backpropagation based on the loss value, perform model weight update on the current deep learning model to obtain a deep learning model after model weight update, and determine the deep learning model after model weight update as the current deep learning model, thereby completing one iteration of training. Based on the current deep learning model, perform a label prediction operation on the sample image, and determine a loss value between the label of the sample image and the annotation prediction result. If the loss value is greater than a threshold, perform backpropagation and repeat the steps of updating the model weight.

[0066] 6) If the loss value is less than or equal to the predetermined threshold, or if a predetermined maximum number of iterations is reached, determine the current deep learning model as the target image recognition network model.

[0067] In an alternative embodiment, the model training process may be as follows: After loading the pre-training weights, the labeled data is input to perform model training; The image data is normalized by a pre-processing module, and the normalized image data is sent to the network model to perform forward propagation to obtain a prediction result; and the prediction result and the target true value of the label file are used to obtain a loss value, which is the deviation between the model output and the target true value, by a damage function. The loss value includes three parts, namely, a target classification loss, a detection box regression loss, and a reference point regression loss. The loss is updated for the weights of each layer of the network by backpropagation, thereby completing one training; The model is converged by continuous iterative training, and when the convergence target or the maximum number of iterations is reached, the entire model training is completed.

[0068] Next, the trained Yolo5 KeyPoints model is deployed to a CPU processor or GPU or AI chip, and then the model can be used for inference and prediction to obtain the identification results of the sign 3 and its five reference points in the images captured in real time by the binocular camera 4.

[0069] The model training stage obtains a model with detection capabilities through field data training, and the model can be deployed to a CPU processor through the opencv library or libtorh library. The GPU and AI chips can also be deployed to GPUs and AI chips through the libraries and deployment requirements provided by the vendor. Generally, model format conversion is required.

[0070] The process of detecting the sign unit 3 and its five reference points includes three steps. When a real-time image is input, the image normalization operation is completed after the pre-processing step, the normalized image is further sent to the Yoo5 KeyPoints model, and the prediction result of each grid point on the image is obtained by forward inference, and finally, the prediction result of each grid point is post-processed, and the final prediction result is obtained if non-maximum suppression is performed.

[0071] In the autonomous operation of the unmanned excavator, the binocular camera 4 collects image data in real time, and after completing the recognition of the sign unit 3 and its five reference points, the following steps S505-S513 are carried out, that is, the image coordinates of the five reference points are calculated by the stereoscopic geometric vision of the binocular camera 4 to obtain the spatial three-dimensional coordinates of the five reference points of the sign unit 3. In this manner, the detection robustness is high and the accuracy is high.

[0072] S505: For each reference point, the coordinates of the reference point in the pixel coordinate system are determined based on the target image and the recognition result of the reference point.

[0073] In this embodiment, the coordinates of the reference point in the pixel coordinate system may be (u, v).

[0074] S507: The camera parameters of the binocular camera 4 are acquired.

[0075] In an alternative example, the camera parameters include the distance between the left and right cameras of the binocular camera 4, mounting parameters and internal parameters.

[0076] Optionally, the binocular camera 4 includes a left eye camera and a right eye camera, the distance between the left eye and the right eye is T, the mounting parameters of the binocular camera include a translation distance from the origin of the vehicle coordinate system of the left eye camera or the right eye camera, and a rotation angle relative to the origin of the vehicle coordinate system, and the internal parameters of the binocular camera include internal parameters of the left eye camera and internal parameters of the right eye camera.

[0077] S509: According to the stereoscopic geometric vision algorithm of the binocular camera 4, a target three-dimensional coordinate of the reference point is determined based on the camera parameters of the binocular camera 4 and the coordinate of the reference point in the pixel coordinate system.

[0078] S511: The target three-dimensional coordinates of the luggage compartment 101 are determined based on the target three-dimensional coordinates of each reference point.

[0079] S513: Based on the target three-dimensional coordinates of the luggage compartment 101, attitude information of the luggage compartment 101 is determined.

[0080] Referring to Figure 6, Figure 6 is a flow chart of another selectable luggage compartment posture recognition method of the present invention. Steps S509-S513 can be specifically detailed as follows:

[0081] S601: The coordinates of the reference point in the camera coordinate system are determined based on the distance between the left and right cameras of the binocular camera 4, the internal parameters of the binocular camera 4, and the coordinates of the reference point in the pixel coordinate system.

[0082] Optionally, the intrinsic parameters of the binocular cameras include intrinsic parameters of a left eye camera and intrinsic parameters of a right eye camera.

[0083] Optionally, the following provides an embodiment for determining the coordinates of the selectable reference point in the camera coordinate system. Refer to Fig. 7, which is a schematic diagram showing the relationship between the selectable coordinate systems of the present invention. World coordinate system: Xw, Yw, Zw; Camera coordinate system: Xc, Yc, Zc; Image coordinate system: x, y; Pixel coordinate system: u, v (reflecting the arrangement of the pixels of the camera CCD chip).

[0084] As can be seen from the figure, (u0, v0) represents coordinates in the uv coordinate system of O. If we assume that the length and width of one pixel are dx and dy, respectively, the relationship between the pixel coordinate system and the image coordinate system is as follows: JPEG2025507430000002.jpg30170If we combine equations (1) and (2) and write them as a matrix, we get the following. JPEG2025507430000003.jpg42170By using the above formula (3), it is possible to obtain the coordinates in the image coordinate system, i.e., (x, y), of any pixel point on the image.

[0085] FIG. 8 is a schematic diagram of a selectable camera model of the present invention. l , O r are the projection centers of the left and right eyes of the binocular camera 4, and the connecting line D between them is the binocular baseline, i.e., the distance between the center points of the two cameras. P is a point in space, and P l is the image point of point P in the left eye, and P r is the image point of point P in the right eye, and the disparity d=x l -x r It is. According to the similar triangle theorem, ΔPO l O r is ΔPP l P r The depth Z can be calculated using the following formula: JPEG2025507430000004.jpg19170 where f is the focal length of the camera.

[0086] After obtaining the depth information Z, i.e., Zc, using the above equation (4), it is possible to further determine the coordinates Xc and Yc in the target binocular camera coordinate system of P (binocular camera coordinate system, i.e., a coordinate system with the center of the left eye camera as the origin, i.e., the camera coordinate system in the coordinate relationship diagram 7).

[0087] 9, which is a schematic diagram of the relationship between another selectable coordinates of the present invention. In the left-eye camera coordinate system of the binocular camera, the image point P in the left-eye image of the spatial point P l From this, the horizontal coordinate X and vertical coordinate Y of spatial point P can be calculated using the similar triangle theorem. The coordinates of P(X,Y,Z) in the left camera coordinate system of the binocular camera are P(Xc,Yc,Zc), and P(x,y) are the image coordinates of the imaging point of spatial point P(Xc,Yc,Zc) in the left eye image coordinate system of the binocular camera.

[0088] ΔABO c ~ΔoCO c , ΔPBO c ~ΔpCO c Therefore, JPEG2025507430000005.jpg18170 can be derived. From the above derivation, Xc and Yc are It can be determined that the image is JPEG2025507430000006.jpg33170. At this point, all the coordinates of the spatial point P (Xc, Yc, Zc) have been obtained. In this manner, the three-dimensional coordinates of the five reference points on the reference point 3 in the camera coordinate system can be obtained in sequence, and the coordinates are in the left-eye camera coordinate system.

[0089] S603: Determine a first coordinate transformation matrix based on mounting parameters of the binocular camera.

[0090] S605: Transform the three-dimensional coordinates of the reference point in the camera coordinate system into three-dimensional coordinates in a target vehicle coordinate system according to the first coordinate transformation matrix, where the target vehicle coordinate system is the coordinate system in which the second vehicle 2 is located.

[0091] In this embodiment, assuming that the origin of the vehicle coordinate system is the rotation center point of the second vehicle 2, the translation distance and rotation angle of the binocular camera from the origin can be determined to determine the corresponding rotation matrix R and translation vector T. When the second vehicle is a shovel, the second vehicle includes a robot arm and a base, a rotation end of the robot arm is rotatably connected to the base, and the rotation center point of the second vehicle is located on the rotation axis on which the rotation end is located.

[0092] The coordinate transformation matrix for transforming the three-dimensional coordinates of the reference point in the camera coordinate system into three-dimensional coordinates in the second vehicle 2 coordinate system is expressed as follows. JPEG2025507430000007.jpg20170According to equation (7), the relationship between the camera coordinate system and the coordinate system of the second vehicle 2 is as follows: Alternatively, the above coordinate transformation matrix, i.e., equation (7), may be different depending on the definition of the vehicle coordinate system.

[0093] S607: The three-dimensional coordinates of the luggage compartment 101 in the target vehicle coordinate system are determined based on the three-dimensional coordinates of each reference point in the target vehicle coordinate system.

[0094] Alternatively, the sign 3 includes two sub-signs 301. For example, if each sub-sign 301 includes five reference points, a selection process is performed on the coordinates of the five reference points for each sub-sign 301, and the position coordinate data of the reference points with large numerical deviations are removed, and the coordinate data of the remaining reference points that meet the requirements are averaged to obtain the three-dimensional coordinate data of the sub-sign 301. In another alternative embodiment, weights are set for the coordinates of the reference points located at different positions on the sub-sign 301, and the three-dimensional coordinate data of the sub-sign 301 can be obtained by multiplying the coordinates of each reference point by the corresponding weight.

[0095] S609: The attitude information of the luggage compartment 101 is determined based on the three-dimensional coordinates of the luggage compartment 101 in the target vehicle coordinate system.

[0096] The attitude information of the luggage compartment 101 can be determined based on the three-dimensional coordinates of the two sub-sign portions 301 in the second vehicle-2 coordinate system.

[0097] This attitude information is, for example, information on the relative distance, angle, etc. between the luggage compartment 101 and the second vehicle 2.

[0098] In a possible embodiment, the image recognition module includes a feature extraction sub-module, a feature fusion sub-module, and a predictive recognition module. The feature extraction sub-module performs a feature extraction operation on an image through a feature extraction network to obtain a set of feature images. The feature fusion sub-module performs a feature fusion process on the set of feature images through a feature fusion network to obtain a target feature image. The predictive recognition module performs a prediction process on the target feature image through a predictive recognition network to obtain a target image.

[0099] In a possible embodiment, the position determining module includes a pixel coordinate determining module, a camera parameter acquiring module, a target 3D coordinate determining module, and an attitude information determining module. A pixel coordinate determination module is used for each reference point to determine a coordinate of the reference point in a pixel coordinate system based on the target image and the recognition result of the reference point. The camera parameter acquisition module is used to acquire the camera parameters of the binocular camera 4 . The target 3D coordinate determination module is used to determine the target 3D coordinates of the reference points based on the camera parameters of the binocular camera 4 and the coordinates of the reference points in the pixel coordinate system through the stereoscopic geometric algorithm of the binocular camera 4, and to determine the target 3D coordinates of the luggage compartment 101 based on the target 3D coordinates of each reference point. The attitude information determination module is used to determine attitude information of the luggage compartment 101 based on the target three-dimensional coordinates of the luggage compartment 101 .

[0100] In a possible embodiment, the target three-dimensional coordinate determining module includes a first coordinate determining module, a first coordinate transformation matrix, and a second coordinate determining module. The first coordinate determination module is used to determine the three-dimensional coordinates of the reference point in the camera coordinate system based on the distance between the left and right cameras of the binocular camera 4, the internal parameters of the binocular camera 4, and the coordinates of the reference point in the pixel coordinate system. The first coordinate transformation matrix is ​​used to determine a first coordinate transformation matrix based on mounting parameters of the binocular camera. The second coordinate determination module transforms the three-dimensional coordinates of the reference point in the camera coordinate system into three-dimensional coordinates in the target vehicle coordinate system according to the first coordinate transformation matrix, where the target vehicle coordinate system is the coordinate system in which the second vehicle 2 is located, and is used to determine the three-dimensional coordinates of the luggage compartment 101 in the target vehicle coordinate system based on the three-dimensional coordinates of each reference point in the target vehicle coordinate system. The attitude information determination module is used to determine the attitude information of the luggage compartment 101 based on the three-dimensional coordinates of the luggage compartment 101 in the target vehicle coordinate system.

[0101] The specific implementation process of the above modules is the same as that of the above method, so the description is omitted here. Note that the above modules may be actual sub-modules in a processor, or virtual modules configured by a program.

[0102] As described above, according to the luggage compartment posture recognition system provided in the present invention, the selectable operation process is as follows: That is, the trained deep convolutional neural network model Yolo5 KeyPoints is deployed to the CPU. At this time, the time required for the neural network model Yolo5 KeyPoints to perform one detection and calculation to obtain the sign unit 3 and its five reference points is about 130 ms. If the calculation unit includes an image calculation unit GPU or an AI accelerator chip, the neural network model Yolo5 KeyPoints can be deployed to the image calculation unit GPU or AI accelerator chip to improve real-time performance. In this case, the time required for the neural network model Yolo5 KeyPoints to perform one detection and calculation to obtain the sign unit 3 and its five reference points is about 30 ms. The five reference point space three-dimensional coordinate algorithm, which calculates the five reference points by binocular stereoscopic geometric vision, is deployed to the CPU.

[0103] When power is supplied to the system, the binocular camera 4 verifies the position of the cargo compartment 101 of the truck, the attitude sensing system is activated, the unmanned shovel control system is activated, and the unmanned shovel enters an unmanned autonomous operation state.

[0104] The driver parks the first vehicle 1 (eg, a truck) in a parking space and prepares to load.

[0105] After the truck parking is completed, the binocular camera 4 sends the captured left eye and right eye color images 1280x720 RGB images to the deep convolutional neural network Yoo5 KeyPoints model respectively. The neural network Yolo5 KeyPoints model preprocesses the left eye and right eye color images 1280x720 RGB images before inference. The preprocessing includes image normalization and image scaling. In the preprocessing of the Yolo5 KeyPoints model, the image normalization divides the RGB value of each pixel of the image by 127.5, subtracts 1, and normalizes all pixel values ​​to -1 to 1. The image scaling operation scales the original 1280x720 RGB image to a 1280x736 RGB figure to meet the requirements of the model input image size. After preprocessing, the image data is sent to the Yolo5 KeyPoints model for inference to obtain the model output. The model output is the object classification result of each anchor box at each lattice point, the detection box regression result, and the five control point regression results. Then, the Yolo5 KeyPoints model obtains the final sign 3 target detection box and its five reference point pixel coordinates through post-processing such as non-maximum suppression.

[0106] After obtaining the pixel coordinates of the five reference points of the sign unit 3 in the left and right eye images of the binocular camera 4 through neural network inference, the pixel coordinates of the five reference points are calculated through the stereoscopic geometric vision of the binocular camera 4 to obtain the three-dimensional spatial coordinates of the five reference points. The luggage compartment posture recognition system has a simple structure and is applicable to large-scale outdoor unmanned driving scenes.

[0107] An embodiment of the present invention further provides an electronic device including a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set, or instruction set, and the at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to realize the above-mentioned luggage compartment attitude recognition method.

[0108] An embodiment of the present invention further provides a computer storage medium, which is provided in a server and realizes at least one instruction, at least one program, code set, or instruction set related to the luggage compartment attitude recognition method in the method embodiment, and the at least one instruction, the at least one program, code set, or instruction set is loaded and executed by the processor to realize the luggage compartment attitude recognition method.

[0109] Optionally, in this embodiment, the storage medium may be located in at least one network server of a plurality of network servers of a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as a USB disk, a read-only memory (ROM), a random access memory (RAM), a portable hard disk, a magnetic disk, or an optical disk.

[0110] It should be noted that the order of the above-described embodiments of the present invention is merely for illustrative purposes and does not indicate superiority or inferiority of the embodiments. Specific embodiments of the present specification have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired results. Also, the processes depicted in the figures do not necessarily achieve the desired effects according to the particular order or sequential order depicted. In some embodiments, multitasking and parallel processing may also be performed or be advantageous.

[0111] Each embodiment in this specification will be described step by step, and the same or similar parts between each embodiment can be referred to each other. The parts that are emphasized in each embodiment are different from other embodiments. In particular, the device embodiment is basically the same as the method embodiment, so the explanation is relatively simple, and for the relevant parts, please refer to the explanation in the method embodiment.

[0112] It is understandable to those skilled in the art that all or part of the steps in the above embodiments may be realized by hardware, or may be completed by instructing relevant hardware by a program, and the program may be stored in a computer-readable storage medium, and the above storage medium may be a read-only memory, a magnetic disk, an optical disk, etc.

[0113] The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention should be included within the protection scope of the present invention.

Claims

1. A luggage compartment attitude recognition system including a processor (5), a binocular camera (4), a first vehicle (1), a second vehicle (2), and a sign unit (3), The first vehicle (1) includes a luggage compartment (101), The second vehicle (2) is used to unload materials into the luggage compartment (101) of the first vehicle (1); The sign portion (3) is provided on a side surface of the luggage compartment (101), and a plurality of reference points are provided on the sign portion (3); the binocular camera (4) is provided on the second vehicle (2), the binocular camera (4) being used to collect an image including the sign portion (3) and transmit the image to the processor (5); The processor (5) is communicatively connected to the binocular camera (4), and the processor (5) recognizes and processes the image using an image recognition network model to obtain a target image including the recognition results of the plurality of reference points, determines three-dimensional coordinates of a target space for each of the plurality of reference points in the target image using a stereoscopic geometric vision algorithm of the binocular camera (4), and determines posture information of the luggage compartment (101) based on the three-dimensional coordinates of the target space for each reference point.

2. The second vehicle (2) comprises an unloading device (201) and a body structure (202) connected to each other; The unloading device (201) is connected to a first side (2021) of the body structure (202); 2. The luggage compartment posture recognition system according to claim 1, wherein the binocular camera (4) is provided on the first side surface (2021).

3. The first side (2021) includes a first attachment point and a second attachment point; The first attachment point and the second attachment point are located on an upper portion of the first side surface (2021); a first preset distance exists between the first attachment point and the second attachment point; The binocular camera (4) includes a first binocular camera (401) and a second binocular camera (402), The first binocular camera (401) is mounted at the first mounting point; 3. The luggage compartment posture recognition system according to claim 2, wherein the second binocular camera (402) is provided at the second mounting point.

4. The sign portion (3) includes at least two sub-sign portions (301), At least two of the sub-signal portions (301) are located at a first position point and a second position point on a second side surface of the luggage compartment (101), respectively; The second side surface is a surface facing the first side surface (2021), The luggage compartment posture recognition system of claim 2, characterized in that there is a second preset distance between the first position point and the second position point, the second preset distance being longer than half the length of the second side in a first predetermined direction, the first predetermined direction being perpendicular to the second predetermined direction, and the second predetermined direction being an extension of the height direction of the first vehicle (1).

5. The processor (5) includes an image recognition module and a position determination module; The image recognition module is used to recognize and process the image using an image recognition network model to obtain a target image, and transmit the target image to a position determination module, the target image being an image including the recognition result of the plurality of reference points; The luggage compartment posture recognition system according to claim 1, characterized in that the position determination module determines three-dimensional coordinates in a target space for each of a plurality of reference points in the target image by a stereoscopic geometric vision algorithm of a binocular camera (4), and determines posture information of the luggage compartment (101) based on the three-dimensional coordinates in the target space of each reference point.

6. A method for recognizing a luggage compartment attitude by the luggage compartment attitude recognition system according to any one of claims 1 to 5, comprising: Collecting an image including the sign portion (3) using a binocular camera (4); inputting the image into an image recognition network model to obtain a target image, the target image including a recognition result for each of the plurality of reference points; For each of the reference points, determining coordinates of the reference point in a pixel coordinate system based on the target image and the recognition result of the reference point; Obtaining camera parameters of the binocular camera (4); determining, by a binocular camera stereoscopic geometric vision algorithm, a three-dimensional coordinate of the reference point in a target space based on the camera parameters of the binocular camera (4) and the coordinate of the reference point in a pixel coordinate system; Determine the three-dimensional coordinates of the target space of the luggage compartment (101) based on the three-dimensional coordinates of the target space of each reference point; Determining attitude information of the luggage compartment (101) based on three-dimensional coordinates of a target space of the luggage compartment (101); A luggage compartment posture recognition method comprising:

7. 7. The method for recognizing the posture of a luggage compartment according to claim 6, wherein the camera parameters include a distance between a left eye camera and a right eye camera of a binocular camera (4), an installation parameter, and an internal parameter.

8. determining three-dimensional coordinates of a target space of the reference point based on camera parameters of the binocular camera (4) and coordinates of the reference point in a pixel coordinate system, determining three-dimensional coordinates of a target space of the luggage compartment (101) based on the target three-dimensional coordinates of each reference point, and determining attitude information of the luggage compartment (101) based on the target three-dimensional coordinates of the luggage compartment (101); determining three-dimensional coordinates of the reference point in a camera coordinate system based on a distance between a left eye camera and a right eye camera of the binocular camera (4), internal parameters of the binocular camera, and coordinates of the reference point in a pixel coordinate system; determining a first coordinate transformation matrix based on mounting parameters of the binocular camera; Transforming the three-dimensional coordinates of the reference point in the camera coordinate system into three-dimensional coordinates in a target vehicle coordinate system according to the first coordinate transformation matrix, the target vehicle coordinate system being a coordinate system in which the second vehicle is located; determining three-dimensional coordinates of the luggage compartment in the target vehicle coordinate system based on three-dimensional coordinates of each reference point in the target vehicle coordinate system; Determining attitude information of the luggage compartment (101) based on three-dimensional coordinates of the luggage compartment in a target vehicle coordinate system; The method for recognizing the attitude of a luggage compartment according to claim 7, further comprising:

9. An electronic device, 9. An electronic device comprising: a processor; and a memory, wherein the memory stores at least one instruction, at least one program, code set, or instruction set, and wherein the at least one instruction, the at least one program, code set, or instruction set is loaded and executed by the processor to realize the luggage compartment posture recognition method according to any one of claims 6 to 8.

10. 1. A computer storage medium, comprising: The computer storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to realize the luggage compartment posture recognition method described in any one of claims 6 to 8.

Images