Recognition Method, Device, Electronic Device, and Storage Medium for Tractor and Trailer

Abstract

The present application discloses a recognition method, device, electronic device, and storage medium for a tractor and a trailer. The method includes collecting laser point cloud data when the tractor and the trailer are connected in a non-rigid state; converting the top view of the plurality of grids into a pseudo-image and then inputting it into a preset deep learning model; traversing the target grid to determine the category to which the target grid belongs according to the feature data of the tractor and / or the trailer output by the preset deep learning model; and determining the tractor and the trailer that satisfy a preset constraint relationship according to the category to which the target grid belongs. Through the present application, the deformation during the movement of non-rigid targets can be overcome, so as to better identify the tractor and the trailer. The present application can be used in scenarios such as airports.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular, to a method and device for identifying a tractor and a trailer, as well as an electronic device and a storage medium. Background Art

[0002] In current high-level autonomous driving technology solutions, lidar is used as the main sensor for autonomous driving. Lidar can provide accurate position information of targets in space and relatively accurate category information at the same time. Using lidar, an autonomous vehicle can accurately identify the positions and sizes of other vehicles on the road, thereby avoiding collisions with other vehicles and predicting the trajectories of other vehicles in the next period of time.

[0003] The target categories in a dedicated road or environment are quite different from those in an open road. There are a large number of tractors and trailers in an airport environment. During the driving process of a tractor on an airport road, a certain number of trailers will be towed. The trailers at the rear of the tractor are connected in series, and the pallets are connected by hinges. And usually, the tractor and the trailer can be up to 30 meters long. Different from traditional trucks on open roads, the tractor and the trailer are in a non-rigid state during the movement process, and deform greatly during the turning process. Therefore, they cannot be represented by a traditional single rotating polygon, so the commonly used 3D object detection algorithms cannot accurately identify the towing form of the tractor. Summary of the Invention

[0004] Embodiments of this application provide a method and device for identifying a tractor and a trailer, as well as an electronic device and a storage medium, so as to better identify the tractor and the trailer.

[0005] Embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, an embodiment of this application provides a method for identifying a tractor and a trailer, which is applied to an airport. The method includes: collecting lidar point cloud data when the tractor and the trailer are connected in a non-rigid state, where the lidar point cloud data is projected to obtain a top view with multiple grids, and the target grids in the top view contain lidar point cloud data of the tractor and / or the trailer; converting the top view of the multiple grids into a pseudo-image and then inputting it into a preset deep learning model, where the pseudo-image includes the attribute values of each point cloud data in the target grid, and the preset deep learning model is pre-trained through a deep learning neural network; traversing the target grids according to the feature data of the tractor and / or the trailer output by the preset deep learning model to determine the category to which the target grids belong; and determining the tractor and the trailer that satisfy a preset constraint relationship according to the category to which the target grids belong.

[0007] Second aspect, an embodiment of the present application further provides an identification device for a tractor and a trailer. When applied to an airport, the device includes: a point cloud data acquisition module, configured to acquire laser point cloud data when the tractor and the trailer are connected in a non-rigid state. After the laser point cloud data is projected, a top view with multiple grids is obtained, and the target grid in the top view contains laser point cloud data of the tractor and / or the trailer; a pseudo-image conversion module, configured to convert the top view of the multiple grids into a pseudo-image and then input it into a preset deep learning model, where the pseudo-image includes attribute values of each point cloud data in the target grid, and the preset deep learning model is pre-trained through a deep learning neural network; a traversal module, configured to traverse the target grid according to the feature data of the tractor and / or the trailer output by the preset deep learning model to determine the category to which the target grid belongs; a determination module, configured to determine the tractor and the trailer that meet a preset constraint relationship according to the category to which the target grid belongs.

[0008] Third aspect, an embodiment of the present application further provides an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, where the executable instructions, when executed, cause the processor to execute the above method.

[0009] Fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by an electronic device including multiple application programs, the electronic device is caused to execute the above method.

[0010] The above at least one technical solution adopted in the embodiment of the present application can achieve the following beneficial effects:

[0011] By acquiring the laser point cloud data when the tractor and the trailer are connected in a non-rigid state, converting the top view of the multiple grids into a pseudo-image and then inputting it into a preset deep learning model; then, according to the feature data of the tractor and / or the trailer output by the preset deep learning model, traversing the target grid to determine the category to which the target grid belongs, and finally, according to the category to which the target grid belongs, determining the tractor and the trailer that meet a preset constraint relationship. Through the present application, the deformation during the movement of the non-rigid target can be overcome, so as to better identify the tractor and the trailer. Description of the Drawings

[0012] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0013] Figure 1Schematic flow chart of the identification method for the tractor and trailer in the embodiments of the present application;

[0014] Figure 2 Schematic structural diagram of the identification device for the tractor and trailer in the embodiments of the present application;

[0015] Figure 3 Schematic structural diagram of an electronic device in the embodiments of the present application. Detailed implementation manners

[0016] To make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0017] Most autonomous vehicles use multi-beam rotary lidar, and there are currently a large number of object detection algorithms based on lidar. The lidar three-dimensional object detection algorithms can generally be divided into two categories: anchor box-based and non-anchor box-based detection algorithms.

[0018] For the anchor box-based object detection algorithm, a large number of prior anchor boxes of different sizes are preset in space. Using a deep neural network, in cooperation with a large amount of labeled point cloud data, through training, the network is made to judge whether these prior anchor boxes contain objects. If they contain the specified object, the size and position of the anchor box are simultaneously corrected to make the size and position of the anchor box closer to the size of the real target. Among them, PointPillars is a classic anchor box-based lidar three-dimensional object detection algorithm. This algorithm projects the lidar under the bird's-eye view, generates a large number of dense anchor boxes under the bird's-eye view, classifies and regresses these anchor boxes, and obtains the detection result of the final target.

[0019] Since the size and position of the anchor box seriously affect the final detection accuracy of the network, and the adjustment of the anchor box size requires a lot of experience, a three-dimensional object detection algorithm that does not require an anchor box has been derived. This type of algorithm does not need to preset prior anchor boxes and directly obtains the target result in the final feature map. CenterPoint is a center point-based object detection algorithm. In the finally output feature map, the target category features are represented in the form of a heat map, and the real size of the target is regressed in the heat map.

[0020] When the inventors were conducting research, they found that neither the 3D object detection algorithms based on anchor boxes nor those without anchor boxes could solve the deformation problem of non-rigid objects during movement. During the movement of a semi-trailer truck, the angle between the trailer boards will deform as the vehicle moves. Therefore, a new network model is needed to represent the shape and position of the semi-trailer truck.

[0021] In the laser 3D object detection algorithms in the related art, the true size and position of non-rigid objects cannot be accurately detected. To address the above problems, the recognition method for tractors and trailer trucks in the embodiments of the present application utilizes the characteristics of the fixed sizes of tractors and trailer trucks, as well as the prior information on the connection relationship between the trailer truck and the tractor, and uses a network model to detect the tractor and trailer truck in a non-rigid motion state, and detect the 3D position information of the tractor and trailer truck. Thus, the purpose of detecting the tractor and trailer truck is achieved, and the effect of detecting non-rigid connection structures is achieved.

[0022] The embodiments of the present application provide a recognition method for tractors and trailer trucks, as Figure 1 shown, provides a schematic flowchart of the recognition method for tractors and trailer trucks in the embodiments of the present application. The method at least includes the following steps S110 to step S140:

[0023] Step S110, collect the laser point cloud data when the tractor and the trailer truck are connected in a non-rigid state. After the laser point cloud data is projected, a top view with multiple grids is obtained, and the target grids in the top view contain the laser point cloud data of the tractor and / or the trailer truck.

[0024] By collecting the laser point cloud data when the tractor and the trailer truck are connected in a non-rigid state, and projecting the laser point cloud data, a top view with multiple grids is obtained.

[0025] It should be noted that the target grids in the top view contain the laser point cloud data of the tractor and / or the trailer truck.

[0026] For example, project the laser point cloud data onto the top view. The top view consists of 240 * 240 rectangular grids, each grid with a size of 0.25 m. The lidar can detect tractors and trailers within a radius of 30 m around the vehicle.

[0027] After the laser point cloud data is projected, according to the coordinates of the lidar on the X-axis and Y-axis, the laser point cloud is assigned to each grid. Each projected grid contains several point clouds. Due to the sparsity of the lidar point cloud, most grids do not contain point clouds, but the grids that contain point clouds are target grids.

[0028] Step S120: After converting the top view of the multiple grids into a pseudo-image, input it into a preset deep learning model, where the pseudo-image includes the attribute values of each point cloud data within the target grid, and the preset deep learning model is pre-trained through a deep learning neural network.

[0029] After converting the top view of the multiple grids into a pseudo-image and then inputting it into a preset deep learning model, it can be understood that the handrail containing multiple grids needs to be converted into a pseudo-image first before it can be input into the preset deep learning model for recognition. The preset deep learning model can be obtained by training a deep learning neural network in related technologies, and no specific limitation is made in the embodiments of the present application.

[0030] It should be noted that the pseudo-image includes the attribute values of each point cloud data within the target grid.

[0031] Step S130: According to the feature data of the tractor and / or trailer truck output by the preset deep learning model, traverse the target grid to determine the category to which the target grid belongs.

[0032] For the feature data of the tractor and / or trailer truck output by the output layer of the preset deep learning model, it is further necessary to determine the category to which the target grid belongs according to the feature data of the tractor and / or trailer truck. That is, there may be a tractor or a trailer truck in the target grid, and it is still necessary to further determine the category to which the target grid belongs.

[0033] For example, calculate the probability values of the tractor and the trailer truck in each grid, and take the maximum probability as the current grid category, which belongs to the tractor or the trailer truck.

[0034] Step S140: According to the category to which the target grid belongs, determine the tractors and trailer trucks that satisfy the preset constraint relationship.

[0035] According to the category to which the target grid belongs, it is possible to further determine which tractors and trailer trucks satisfy the preset constraint relationship. It can be understood that through the above process, a series of dense detection results of tractors and trailer trucks are obtained. For example, at the actual center positions of the tractors and trailer trucks, multiple overlapping tractors and trailer trucks will be generated simultaneously, and the redundant detection results need to be deleted.

[0036] The preset constraint relationship can be a constraint relationship established based on prior knowledge.

[0037] In an embodiment of the present application, the preset constraint relationship includes the prior value of the connection relationship between the tractor and the trailer, and the prior value of the size of each trailer. Determining the tractor and trailer that meet the preset constraint relationship according to the category to which the target grid belongs includes: determining the category to which the target grid belongs according to the preset probability value; obtaining the actual size of the tractor or trailer within the grid according to the type of the target grid; based on the actual size, deleting the tractors or trailers that do not meet the preset constraints according to the prior value of the connection relationship between the tractor and the trailer and / or the size of each trailer as the preset constraint relationship, so as to obtain the tractor and trailer that meet the preset constraint relationship.

[0038] During specific implementation, when filtering, determine the category to which the target grid belongs according to the preset probability value. For example, take the maximum probability as the current grid category, which belongs to the tractor or the trailer.

[0039] In addition, the actual size of the tractor or trailer within the grid can also be obtained according to the type of the target grid.

[0040] In order to obtain more accurate results, based on the actual size, delete the tractors or trailers that do not meet the preset constraints according to the prior value of the connection relationship between the tractor and the trailer and / or the size of each trailer as the preset constraint relationship, so as to obtain the tractor and trailer that meet the preset constraint relationship. For example, during the deletion process, the connection relationship between the trailer and the tractor is used as a constraint condition, and the tractors or trailers that do not meet the constraint condition will ultimately be deleted.

[0041] In an embodiment of the present application, after determining the tractor and trailer that meet the preset constraint relationship according to the category to which the target grid belongs, it further includes: starting from the center point of the tail of the target tractor, searching for multiple target trailers along the orientation information of the target grid; calculating the angle between the axis of the target trailer closest to the target tractor and the axis of the tractor; determining whether the angle meets the preset condition, and deleting the trailers that do not meet the preset condition. After determining the target trailer closest to the target tractor, then starting from the center point of the tail of this trailer, continue to search for subsequent target trailers; connecting the tractor and the multiple target trailers to obtain the detection result, and obtaining the position information of the tractor and the trailer according to the detection result.

[0042] During specific implementation, starting from the center point of the tail of the target tractor, search for multiple target trailers along the orientation information of the target grid. For example, the angle between the central axis of the nearest connected trailer and the central axis of the tractor can be calculated, and trailers that do not meet the constraint condition that the angle is between -30 degrees and +30 degrees are deleted. After determining the nearest trailer, use the center point of the tail of this trailer as the starting point to continue searching for subsequent trailers until there are no trailers within a range of 5m adjacent to each other at the end.

[0043] It should be noted that the above -30 degrees to +30 degrees are only examples and are not used for the protection scope in this application.

[0044] Connect the tractor and the multiple target trailers to obtain the detection result. Based on the detection result, obtain the position information of the tractor and the trailers. Finally, connect the tractor and the trailers to form an output of the detection result and obtain the final positions of the tractor and the trailers.

[0045] In an embodiment of this application, determining the category to which the target grid belongs according to the preset probability value includes: traversing all grids and calculating the probability values of the tractor and the trailers in each target grid, and taking the maximum probability value as the category to which the current target grid belongs, which belongs to the tractor or the trailer; filtering the probability values in the target grid again through a preset threshold, and only retaining the grids with probability values greater than the preset threshold. Among the grids with probability values greater than the preset threshold, there is a target grid with the maximum probability value of the tractor or the trailer. According to the length, width, and orientation information in the target grid, obtain the actual size of the tractor or the trailer within this target grid.

[0046] During specific implementation, first traverse all grids and calculate the probability values of the tractor and the trailers in each target grid. Take the maximum probability value as the category to which the current target grid belongs, which belongs to the tractor or the trailer. According to the traversal result, filter the probability values in the target grid again through a preset threshold, and only retain the grids with probability values greater than the preset threshold.

[0047] It should be noted that the actual size of the tractor or the trailer within the target grid can also be obtained according to the length, width, and orientation information in the target grid, which will not be elaborated here.

[0048] In an embodiment of the present application, a tractor and a trailer within a range of at least 30 m in radius around a preset center point are detected by a lidar, and the lidar point cloud data when the tractor and the trailer are connected in a non-rigid state is collected. After the lidar point cloud data is projected, a top view with multiple grids is obtained. The target grids in the top view contain the lidar point cloud data of the tractor and / or the trailer, including: after the lidar point cloud data is projected, according to the coordinates of the lidar on the X-axis and Y-axis, the lidar point data cloud is allocated to multiple grids; each grid after projection includes several point cloud data, and for the grids containing point cloud data, a preset attribute value of all the point cloud data in the grid is calculated.

[0049] In specific implementation, a tractor and a trailer within a range of at least 30 m in radius around a preset center point are detected by a lidar, and the range can also be expanded according to actual situations.

[0050] Projection is performed according to the lidar point cloud data obtained by lidar scanning. After the lidar point cloud data is projected, according to the coordinates of the lidar on the X-axis and Y-axis, the lidar point data cloud is allocated to multiple grids; each grid after projection includes several point cloud data, and for the grids containing point cloud data, a preset attribute value of all the point cloud data in the grid is calculated. For example, the point cloud generated by the lidar is projected in the top view, and the top view consists of 240*240 rectangular grids, and the size of each grid is 0.25 m.

[0051] In an embodiment of the present application, the preset attribute values of all the point cloud data at least include: the height value Z of the highest point cloud in the grid, the number of point clouds in the grid, and the average height of the point clouds in the grid. The conversion of the top view of the multiple grids into a pseudo-image and then inputting it into a preset deep learning model includes: based on the height value Z of the highest point cloud in the grid, the number of point clouds in the grid, and the average height of the point clouds in the grid, a 3-channel pseudo-image is generated, where the 3 channels respectively represent the preset attribute values; multiple pieces of the 3-channel pseudo-images are input into the preset deep learning model.

[0052] In specific implementation, the attribute values of all the point clouds in the grid are calculated: 1) the height value Z of the highest point cloud in the grid; 2) the number of point clouds in the grid; 3) the average height of the point clouds in the grid. Each grid after calculation will generate 3 values, and these 3 values serve as the attributes of the grid. It should be noted that for those grids that do not contain point clouds, these 3 attribute values are set to 0. Finally, all the grids will generate a 3-channel pseudo-image, and the 3 channels respectively represent the values of the above 3 different attributes.

[0053] In one embodiment of the present application, traversing the target grid with the feature data of the tractor and / or pallet truck output according to the preset deep learning model to determine the category to which the target grid belongs includes: according to the preset feature encoding backbone network and the encoding and decoding structure of the preset deep neural network in the preset deep learning model, and outputting preset feature data at the feature output layer, where the preset feature data includes at least one of the following features: the probability of the grid containing the center point of the tractor, the probability of the grid containing the pallet truck, the X coordinate of the center position of the pallet truck or the tractor, the Y coordinate of the center position of the pallet truck or the tractor, the length of the pallet truck or the tractor, the width of the pallet truck or the tractor, and the angular orientation of the pallet truck.

[0054] Specifically, according to the preset feature encoding backbone network and the encoding and decoding structure of the preset deep neural network in the preset deep learning model, and outputting preset feature data at the feature output layer.

[0055] For example, the pseudo-image passes through the feature encoding backbone network and the encoding and decoding structure of the deep neural network, and finally obtains a feature output layer with a size reduced by 8 times.

[0056] The preset feature data includes, but is not limited to, the probability of the grid containing the center point of the tractor, the probability of the grid containing the pallet truck, the X coordinate of the center position of the pallet truck or the tractor, the Y coordinate of the center position of the pallet truck or the tractor, the length of the pallet truck or the tractor, the width of the pallet truck or the tractor, and the angular orientation of the pallet truck.

[0057] The embodiment of the present application also provides an identification device 200 for the tractor and the pallet truck, as Figure 2 shown, which provides a schematic structural diagram of the identification device for the tractor and the pallet truck in the embodiment of the present application. The device 200 at least includes: a point cloud data acquisition module 210, a pseudo-image conversion module 220, a traversal module 230, and a determination module 240, where:

[0058] In one embodiment of the present application, the point cloud data acquisition module 210 is specifically configured to: acquire laser point cloud data when the tractor and the pallet truck are connected in a non-rigid state, where the laser point cloud data is projected to obtain a top view with multiple grids, and the target grid in the top view contains the laser point cloud data of the tractor and / or the pallet truck.

[0059] By acquiring the laser point cloud data when the tractor and the pallet truck are connected in a non-rigid state, and projecting the laser point cloud data to obtain a top view with multiple grids.

[0060] It should be noted that the target grid in the top view contains the laser point cloud data of the tractor and / or the pallet truck.

[0061] For example, project the lidar point cloud data onto the top view. The top view consists of 240 * 240 rectangular grids, each with a size of 0.25 m. The lidar can detect tractors and trailers within a range of 30 m around the vehicle.

[0062] After the lidar point cloud data is projected, according to the coordinates of the lidar in the X-axis and Y-axis, the lidar point cloud is assigned to each grid. Each grid after projection contains a number of point clouds. Due to the sparsity of the lidar point cloud, most grids do not contain point clouds, but the grids that contain point clouds are the target grids.

[0063] In an embodiment of the present application, the pseudo-image conversion module 220 is specifically configured to: convert the top view of the multiple grids into a pseudo-image and then input it into a preset deep learning model, where the pseudo-image includes the attribute values of each point cloud data within the target grid, and the preset deep learning model is pre-trained through a deep learning neural network.

[0064] Converting the top view of the multiple grids into a pseudo-image and then inputting it into a preset deep learning model, it can be understood that it is necessary to first convert the handrail containing multiple grids into a pseudo-image before it can be input into the preset deep learning model for recognition. The preset deep learning model can be trained through a deep learning neural network in related technologies, and is not specifically limited in the embodiments of the present application.

[0065] It should be noted that the pseudo-image includes the attribute values of each point cloud data within the target grid.

[0066] In an embodiment of the present application, the traversal module 230 is specifically configured to: traverse the target grids according to the feature data of the tractor and / or trailer output by the preset deep learning model to determine the category to which the target grid belongs.

[0067] For the feature data of the tractor and / or trailer output by the output layer of the preset deep learning model, it is further necessary to determine the category to which the target grid belongs according to the feature data of the tractor and / or trailer. That is, there may be a tractor or a trailer in the target grid, and it is still necessary to further determine the category to which the target grid belongs.

[0068] For example, calculate the probability values of the tractor and the trailer in each grid, and take the maximum probability as the current grid category, which belongs to the tractor or the trailer.

[0069] In an embodiment of the present application, the determination module 240 is specifically configured to: determine the tractor and the trailer that satisfy the preset constraint relationship according to the category to which the target grid belongs.

[0070] According to the category to which the target grid belongs, it can be further determined which of the tractors and trailers meet the preset constraint relationship. It can be understood that through the above process, a series of dense detection results of tractors and trailers are obtained. For example, at the actual center positions of the tractors and trailers, multiple overlapping tractors and trailers will be generated simultaneously, and the redundant detection results need to be deleted.

[0071] The preset constraint relationship can be a constraint relationship established based on prior knowledge.

[0072] It can be understood that the above-mentioned identification device for tractors and trailers can implement each step of the identification method for tractors and trailers provided in the foregoing embodiments. The relevant explanations regarding the identification method for tractors and trailers are applicable to the identification device for tractors and trailers, and will not be elaborated herein.

[0073] Figure 3 It is a schematic structural diagram of an electronic device according to an embodiment of the present application. Please refer to Figure 3 , at the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory, such as at least one disk memory, etc. Of course, the electronic device may also include other hardware required for other services.

[0074] The processor, network interface, and memory can be interconnected through the internal bus, and the internal bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity of representation, Figure 3 only a bidirectional arrow is used in [the figure] to represent it, but it does not mean that there is only one bus or one type of bus.

[0075] The memory is used to store programs. Specifically, the program may include program codes, and the program codes include computer operation instructions. The memory may include a memory and a non-volatile memory, and provide instructions and data to the processor.

[0076] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming an identification device for the tractor and the trailer at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0077] Collect the laser point cloud data when the tractor and the trailer are connected in a non-rigid state, wherein the laser point cloud data is projected to obtain a top view with multiple grids, and the target grids in the top view contain the laser point cloud data of the tractor and / or the trailer;

[0078] After converting the top view of the multiple grids into a pseudo-image, input it into a preset deep learning model, wherein the pseudo-image includes the attribute values of each point cloud data in the target grid, and the preset deep learning model is pre-trained through a deep learning neural network;

[0079] According to the feature data of the tractor and / or the trailer output by the preset deep learning model, traverse the target grid to determine the category to which the target grid belongs;

[0080] According to the category to which the target grid belongs, determine the tractor and the trailer that satisfy the preset constraint relationship.

[0081] The above is as in this application Figure 1The method executed by the identification device of the tractor and the trailer truck disclosed in the illustrated embodiment can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the above method can be completed by the integrated logic circuit of the hardware in the processor or the instructions in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed by the hardware decoding processor, or executed by the combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0082] The electronic device can also execute Figure 1 the method executed by the identification device of the tractor and the trailer truck in Figure 1 the illustrated embodiment, and implement the functions of the identification device of the tractor and the trailer truck in

[0083] The embodiments of the present application also propose a computer-readable storage medium, which stores one or more programs. The one or more programs include instructions that, when executed by an electronic device including a plurality of application programs, can enable the electronic device to execute Figure 1 the method executed by the identification device of the tractor and the trailer truck in the illustrated embodiment, and specifically used to execute:

[0084] Collect the laser point cloud data when the tractor and the trailer truck are connected in a non-rigid state. The laser point cloud data, after being projected, obtains a top view with a plurality of grids, and the target grids in the top view contain the laser point cloud data of the tractor and / or the trailer truck;

[0085] After converting the top view of the multiple grids into a pseudo-image, input it into a preset deep learning model, where the pseudo-image includes the attribute values of each point cloud data within the target grid, and the preset deep learning model is pre-trained through a deep learning neural network;

[0086] According to the feature data of the tractor and / or pallet truck output by the preset deep learning model, traverse the target grid to determine the category to which the target grid belongs;

[0087] According to the category to which the target grid belongs, determine the tractor and the pallet truck that satisfy the preset constraint relationship.

[0088] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0089] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0090] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including instruction means, and the instruction means implements the functions specified in one Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide for implementing the functions in the processFigure 1 one or more processes and / or blocks Figure 1 steps of the functions specified in one block or more blocks.

[0092] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0093] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0094] Computer-readable media includes permanent and non-permanent, removable and non-removable media implemented by any method or technology for information storage. The information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0095] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or apparatus comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or apparatus. Without further limitation, an element defined by the statement "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising the element.

[0096] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0097] The above are only the embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. An identification method for a tractor and a trailer, wherein, The method includes: Collecting laser point cloud data when a tractor and a trailer are connected in a non-rigid state, wherein the laser point cloud data, after being projected, obtains a top view with multiple grids, and a target grid in the top view contains laser point cloud data of the tractor and / or the trailer; Converting the top view of the multiple grids into a pseudo-image and then inputting it into a preset deep learning model, wherein the pseudo-image includes attribute values of each point cloud data in the target grid, and the preset deep learning model is pre-trained through a deep learning neural network; According to the feature data of the tractor and / or the trailer output by the preset deep learning model, traversing the target grid to determine the category to which the target grid belongs; According to the category to which the target grid belongs, determining the tractor and the trailer that satisfy the preset constraint relationship.

2. The method according to claim 1, wherein The preset constraint relationship includes a prior value of the connection relationship between the tractor and the trailer and a prior value of the size of each trailer. Determining the tractor and the trailer that satisfy the preset constraint relationship according to the category to which the target grid belongs includes: Determining the category to which the target grid belongs according to a preset probability value; Obtaining the actual size of the tractor or the trailer in the grid according to the category of the target grid; Based on the actual size, deleting the tractor or the trailer that does not satisfy the preset constraint according to the prior value of the connection relationship between the tractor and the trailer and / or the size of each trailer as the preset constraint relationship, to obtain the tractor and the trailer that satisfy the preset constraint relationship.

3. The method according to claim 2, wherein After determining the tractor and the trailer that satisfy the preset constraint relationship according to the category to which the target grid belongs, it further includes: Taking the center point of the tail of the target tractor as the starting point, searching for multiple target trailers along the orientation information of the target grid; Calculating the included angle between the axis of the target trailer closest to the target tractor and the axis of the tractor; Judging whether the included angle satisfies the preset condition, deleting the trailers that do not satisfy the preset condition. After determining the target trailer closest to the target tractor, taking the center point of the tail of this target trailer as the starting point, and continuing to search for subsequent target trailers; Connecting the tractor and the multiple target trailers to obtain a detection result, and obtaining the position information of the tractor and the trailer according to the detection result.

4. The method according to claim 2, wherein, Determining the category to which the target grid belongs according to the preset probability value includes: Traversing all grids and calculating the probability values of the tractor and the trailer in each target grid, and taking the maximum probability value as the category to which the current target grid belongs, which belongs to the tractor or the trailer; Filtering the probability values in the target grid again through a preset threshold, and only retaining the grids with probability values greater than the preset threshold. Among the grids with probability values greater than the preset threshold, the target grid with the largest probability value of the tractor or the trailer is included. According to the length and width dimension information and the orientation information in the target grid, the actual size of the tractor or the trailer in the target grid is obtained.

5. The method according to claim 1, wherein, Collect the laser point cloud data when the collection tractor and the trailer are connected in a non-rigid state, where the laser point cloud data is projected to obtain a top view with multiple grids, and the target grids in the top view contain the laser point cloud data of the tractor and / or the trailer, including: After the laser point cloud data is projected, the laser point cloud data is assigned to multiple grids according to the coordinates of the lidar in the X-axis and Y-axis; Each grid after projection includes several point cloud data, and for the grids containing point cloud data, calculate the preset attribute values of all the point cloud data in the grid.

6. The method according to claim 5, wherein The preset attribute values of all the point cloud data at least include: the height value of the highest point cloud in the grid, the number of point clouds in the grid, and the average height of the point clouds in the grid. The method of converting the top view of the multiple grids into a pseudo-image and then inputting it into a preset deep learning model includes: Based on the height value of the highest point cloud in the grid, the number of point clouds in the grid, and the average height of the point clouds in the grid, generate a 3-channel pseudo-image, where the three channels respectively represent the preset attribute values; Input multiple 3-channel pseudo-images into a preset deep learning model.

7. The method according to claim 1, wherein The method of traversing the target grids according to the feature data of the tractor and / or the trailer output by the preset deep learning model to determine the category to which the target grids belong includes: According to the preset feature encoding backbone network and the encoding and decoding structure of the preset deep neural network in the preset deep learning model, and output preset feature data at the feature output layer, where the preset feature data at least includes one of the following features: the probability of the grid containing the center point of the tractor, the probability of the grid containing the trailer, the X coordinate of the center position of the trailer or the tractor, the Y coordinate of the center position of the trailer or the tractor, the length of the trailer or the tractor, the width of the trailer or the tractor, and the angular orientation of the trailer.

8. An identification device for a tractor and a trailer, wherein, The device includes: A point cloud data acquisition module for collecting the laser point cloud data when the tractor and the trailer are connected in a non-rigid state, where the laser point cloud data is projected to obtain a top view with multiple grids, and the target grids in the top view contain the laser point cloud data of the tractor and / or the trailer; A pseudo-image conversion module for converting the top view of the multiple grids into a pseudo-image and then inputting it into a preset deep learning model, where the pseudo-image includes the attribute values of each point cloud data in the target grid, and the preset deep learning model is pre-trained through a deep learning neural network; A traversing module for traversing the target grids according to the feature data of the tractor and / or the trailer output by the preset deep learning model to determine the category to which the target grids belong; A determination module for determining the tractor and the trailer that satisfy the preset constraint relationship according to the category to which the target grids belong.

9. An electronic device, including: A processor; And A memory arranged to store computer-executable instructions that, when executed, cause the processor to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing one or more programs which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the method according to any one of claims 1 to 7.

Images