Target detection algorithm model training method and related device

Abstract

The invention discloses a target detection algorithm model training method and a related device, and relates to the field of artificial intelligence, point cloud data in a railway traffic scene is acquired through a laser radar, the point cloud data is labeled, and labeling results of multiple objects are acquired; acquiring non-ground point cloud data from the point cloud data; obtaining a target object located in a non-ground point cloud from the plurality of objects; obtaining a transformation object based on the target object; taking the point cloud coordinates of the points contained in the plurality of sample objects and the reflection intensity of the points as input, taking the marking results corresponding to the plurality of sample objects as training targets, and training to obtain a target detection algorithm model; and deploying the target detection algorithm model to terminal equipment arranged on the train. The object in the operation environment of the train can be detected through the target detection algorithm model deployed by the terminal equipment, so that the aim of detecting the object in the operation environment of the train in combination with the laser radar is fulfilled.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a training method and related devices for a target detection algorithm model. Background Art

[0002] LiDAR has unique advantages in target detection. Its main principle is to obtain information such as the position, shape, and size of an object by emitting a laser beam and receiving reflected light. It can accurately detect objects in complex environments and effectively avoid blind spots and dead angles. In addition, LiDAR has a high scanning rate and accuracy, and can accurately obtain information on a large number of objects in a short time.

[0003] The operating environment of the train, that is, the environment in the railway transportation scenario, is relatively complex. How to use lidar to detect objects in the operating environment of the train is a technical problem that urgently needs to be solved. Summary of the invention

[0004] In view of the above problems, this application provides a method for training a target detection algorithm model and a related device to achieve the purpose of detecting objects in the running environment of a train in combination with a laser radar. The specific scheme is as follows:

[0005] The first aspect of the present application provides a method for training a target detection algorithm model, comprising:

[0006] Acquire point cloud data in a railway traffic scene by using a laser radar, wherein the point cloud data includes point cloud coordinates of a plurality of points and reflection intensities corresponding to the plurality of points;

[0007] Annotating the point cloud data to obtain annotation results of multiple objects, wherein the annotation results of the objects include point cloud coordinates of points constituting the objects, category information of the objects, and position information of circumscribed 3D rectangular bounding boxes of the objects;

[0008] Determine, among the plurality of points included in the point cloud data, points whose distances from the first set ground model are greater than or equal to a first set threshold as non-ground point cloud data;

[0009] Acquire a target object located in the non-ground point cloud from the plurality of objects;

[0010] Acquire a transformation object based on the target object;

[0011] Taking the point cloud coordinates and the reflection intensity of the points included in the multiple sample objects as input, taking the annotation results respectively corresponding to the multiple sample objects as training targets, and training to obtain the target detection algorithm model; the multiple sample objects include the target object and the transformation object;

[0012] The target detection algorithm model is deployed on a terminal device installed on a train.

[0013] In a possible implementation, the method for obtaining the first set ground model includes:

[0014] Randomly determine three non-collinear points from the point cloud data, wherein the three non-collinear points do not all belong to the constructed candidate ground models;

[0015] A candidate ground model is formed by the three non-collinear points;

[0016] Dividing points whose distances from the candidate ground model are less than or equal to a second preset threshold among a plurality of points included in the point cloud data into an inner point set corresponding to the candidate ground model;

[0017] From the inlier point sets respectively corresponding to the plurality of candidate ground models, determine the candidate ground model corresponding to the inlier point set having the most points as the candidate optimal ground model;

[0018] If the current number of iterations is less than or equal to the preset number of iterations, returning to the step of randomly determining three non-collinear points from the point cloud data;

[0019] If the current number of iterations is greater than the preset number of iterations, the candidate optimal ground model is determined to be the first set ground model.

[0020] In a possible implementation, the step of acquiring a transformation object based on the target object includes:

[0021] Connecting the center of the circumscribed 3D rectangular bounding box of the target object with the vertices of the circumscribed 3D rectangular bounding box to obtain six regions;

[0022] For each of the six regions, determining the points contained in the region as a point set corresponding to the region, so as to obtain the point sets corresponding to the six regions respectively;

[0023] A first number of the point sets in the point sets corresponding to the six regions are randomly deleted to obtain the transformation object, where the first number is an integer greater than or equal to 1 and less than or equal to 5.

[0024] In a possible implementation, the step of acquiring a transformation object based on the target object includes:

[0025] For each of the target objects, connecting the center of a circumscribed 3D rectangular bounding box of the target object with the vertices of the circumscribed 3D rectangular bounding box to obtain six regions;

[0026] For each of the target objects, for each of the six regions of the target object, determining the points contained in the region as the point set corresponding to the region, so as to obtain the point sets corresponding to the six regions respectively;

[0027] For any two target objects belonging to the same category information, the point sets corresponding to the areas at the same positions in the circumscribed 3D rectangular bounding boxes of the two target objects are replaced to obtain the transformed objects.

[0028] In a possible implementation, the step of acquiring a transformation object based on the target object includes:

[0029] Connecting the center of the circumscribed 3D rectangular bounding box of the target object with the vertices of the circumscribed 3D rectangular bounding box to obtain six regions;

[0030] For each of the six regions, determining the points contained in the region as a point set corresponding to the region, so as to obtain the point sets corresponding to the six regions respectively;

[0031] Part of the points in the point set of at least one of the six areas is deleted to obtain the transformed object.

[0032] In a possible implementation, the target detection algorithm model includes a columnar division module, a sparse convolution module whose input end is connected to the output end of the columnar division module, a coordinate attention mechanism module whose input end is connected to the output end of the sparse convolution module, a neck network module whose input end is connected to the output end of the coordinate attention mechanism module, and a detection head module whose input end is connected to the output end of the neck network module; the input end of the columnar division module is the input end of the target detection algorithm model, and the output end of the detection head module is the output end of the target detection algorithm model;

[0033] The step of taking the point cloud coordinates of the points included in the plurality of sample objects and the reflection intensity of the points as input, taking the annotation results respectively corresponding to the plurality of sample objects as training targets, and training to obtain the target detection algorithm model comprises:

[0034] Inputting the point cloud coordinates of the points included in the plurality of sample objects and the reflection intensity of the points into the target detection algorithm model;

[0035] Acquire first feature information based on the point cloud coordinates of points included in the plurality of sample objects and the reflection intensity of the points through the columnar division module;

[0036] Acquire second feature information based on the first feature information through the sparse convolution module;

[0037] Acquire third feature information based on the second feature information through the coordinate attention mechanism module;

[0038] Acquiring fourth characteristic information based on the third characteristic information through the neck network module;

[0039] Acquire, by the detection head module, prediction results respectively corresponding to the plurality of sample objects based on the fourth feature information, the prediction results of the sample objects including point cloud coordinates of points constituting the sample objects, category information to which the sample objects belong, and position information of an external 3D rectangular bounding box of the sample objects;

[0040] Comparing the prediction results respectively corresponding to the plurality of sample objects and the annotation results respectively corresponding to the plurality of sample objects to obtain a comparison result;

[0041] The target detection algorithm model is updated based on the comparison result.

[0042] In a possible implementation, the step of acquiring the third feature information based on the second feature information through the coordinate attention mechanism module includes:

[0043] Obtaining a first feature matrix in an X-axis direction and a second feature matrix in a Y-axis direction based on the second feature information through a maximum pooling layer in the coordinate attention mechanism module, wherein the X-axis direction is the direction in which the train moves forward, and the Y-axis direction is a direction perpendicular to the X-axis;

[0044] Splicing the first feature matrix and the second feature matrix through the splicing layer in the coordinate attention mechanism module to obtain a target feature matrix;

[0045] The third feature information is obtained based on the target feature matrix through a multilayer perceptron in the coordinate attention mechanism module.

[0046] In a possible implementation, the method further includes:

[0047] Collecting the point cloud data to be measured by using the laser radar deployed on the train;

[0048] Determine, as non-ground point cloud data, points whose distances from a second set ground model are greater than or equal to the first set threshold value among a plurality of points included in the point cloud data to be tested;

[0049] Inputting the non-ground point cloud data in the point cloud data to be tested into the target detection algorithm model deployed by the terminal device, and obtaining the prediction result of the predicted object through the target detection algorithm model, wherein the prediction result includes the point cloud coordinates of the points constituting the predicted object, the category information to which the predicted object belongs, and the position information of the circumscribed 3D rectangular bounding box of the predicted object;

[0050] Control the terminal device to issue an early warning based on the prediction result of the prediction object.

[0051] A second aspect of the present application provides a training device for a target detection algorithm model, comprising:

[0052] A first acquisition module is used to acquire point cloud data in a railway traffic scene through a laser radar, wherein the point cloud data includes point cloud coordinates of a plurality of points and reflection intensities corresponding to the plurality of points respectively;

[0053] A second acquisition module is used to annotate the point cloud data to obtain annotation results of multiple objects, where the annotation results of the objects include point cloud coordinates of points constituting the objects, category information of the objects, and position information of circumscribed 3D rectangular bounding boxes of the objects;

[0054] A first determination module, configured to determine, among a plurality of points included in the point cloud data, points whose distances from a first set ground model are greater than or equal to a first set threshold as non-ground point cloud data;

[0055] A third acquisition module, used for acquiring a target object located in the non-ground point cloud from the multiple objects;

[0056] A fourth acquisition module, used for acquiring a transformation object based on the target object;

[0057] A training module, used to take the point cloud coordinates and reflection intensity of the points contained in the multiple sample objects as input, take the annotation results respectively corresponding to the multiple sample objects as training targets, and train to obtain a target detection algorithm model; the multiple sample objects include the target object and the transformation object;

[0058] A deployment module is used to deploy the target detection algorithm model on a terminal device installed on a train.

[0059] The third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the training method of the target detection algorithm model of the above-mentioned first aspect or any implementation method of the first aspect.

[0060] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0061] The memory is used to store computer programs;

[0062] The processor is used to execute the computer program so that the electronic device can implement the training method of the target detection algorithm model of the above-mentioned first aspect or any implementation method of the first aspect.

[0063] In a fifth aspect, the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can use the training method of the target detection algorithm model of the above-mentioned first aspect or any implementation method of the first aspect.

[0064] By means of the above technical scheme, the present application provides a method for training a target detection algorithm model, which obtains point cloud data in a railway traffic scene through a laser radar, annotates the point cloud data, and obtains the annotation results of multiple objects; determines the points in the multiple points contained in the point cloud data whose distance from the first set ground model is greater than or equal to the first set threshold as non-ground point cloud data; obtains the target object located in the non-ground point cloud from multiple objects; obtains the transformation object based on the target object; takes the point cloud coordinates of the points contained in the multiple sample objects and the reflection intensity of the points as input, and takes the annotation results corresponding to the multiple sample objects as training targets, and trains to obtain the target detection algorithm model; the multiple sample objects include target objects and transformation objects; and deploys the target detection algorithm model on a terminal device set on a train. The target detection algorithm model deployed by the terminal device can detect objects in the running environment of the train, thereby achieving the purpose of detecting objects in the running environment of the train in combination with the laser radar. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0066] Figure 1 A schematic diagram of a system architecture provided for this application;

[0067] Figure 2 A schematic diagram of an optional hardware structure of a terminal device 100 provided in this application;

[0068] Figure 3 A schematic diagram of the structure of a server 200 provided in this application;

[0069] Figure 4 A flowchart of a method for training a target detection algorithm model provided in an embodiment of the present application;

[0070] Figure 5A schematic diagram of six point sets corresponding to six regions provided in an embodiment of the present application;

[0071] Figure 6 A schematic diagram of the area at the same position in the circumscribed 3D rectangular bounding box of two target objects;

[0072] Figure 7 A schematic diagram of the structure of the target detection algorithm model provided in the embodiment of the present application;

[0073] Figure 8 A schematic diagram of the structure of the coordinate attention mechanism module provided in the embodiment of the present application;

[0074] Fig. 9 A schematic diagram of the structure of a training device for a target detection algorithm model provided in an embodiment of the present application;

[0075] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0077] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0078] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0079] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0080] See also Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. The system may include a terminal device 100, a server 200 and a laser radar 300. The server 200 may include one or more servers ( Figure 1 In the example, a server is included, and the server 200 can provide the method provided in the embodiment of the present application for one or more terminal devices.

[0081] The terminal device 100 and the laser radar 300 can be arranged on a train. The laser radar is installed at a suitable position of the train to ensure that it is stable and has a clear scanning angle.

[0082] Exemplarily, the terminal device 100 may be located in a driver's cab of a train. Figure 1 In the figure, a rectangular block is used to represent the terminal device 100 deployed in the driver's cab.

[0083] Exemplarily, the train can be deployed with one or more laser radars 300. After the laser radar 300 is deployed on the train, system debugging is required, including hardware connection, software configuration and initial calibration. The laser radar is synchronized with the data in the global positioning system (GPS) and the inertial measurement unit (IMU) to ensure that the geographic reference of the point cloud data is accurate.

[0084] In the embodiment of the present application, the point cloud data collected by the laser radar 300 has the train travel direction as the X-axis direction, the direction perpendicular to the X-axis direction as the Y-axis direction, and the Z-axis is perpendicular to the XOY plane.

[0085] Exemplarily, after the laser radar deployed on one or more trains collects point cloud data, it can be sent to the server 200; the staff can annotate the point cloud data through the server 200 and execute the training method of the target detection algorithm model provided in the embodiment of the present application. The server 200 can send the trained target detection algorithm model to the terminal device 100 deployed on one or more trains. In this way, the train's operating environment can be monitored in real time through the target detection algorithm model deployed in the terminal device 100 during the train's advancement.

[0086] For example, after the laser radar deployed on the train collects point cloud data, it can be sent to the terminal device 100; the terminal device 100 executes the training method of the target detection algorithm model provided in the embodiment of the present application, so that the trained target detection algorithm model can be deployed in the terminal device. In this way, when the train is moving forward, the target detection algorithm model deployed in the terminal device 100 can monitor the operating environment of the train in real time.

[0087] Next describe Figure 1 The product form of the middle terminal device 100;

[0088] The terminal device 100 in the embodiment of the present application can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiment of the present application does not impose any restrictions on this.

[0089] Figure 2 An optional hardware structure diagram of the terminal device 100 is shown.

[0090] refer to Figure 2 As shown, the terminal device 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), an earphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190 and other components. Those skilled in the art will appreciate that Figure 2 It is only an example of a terminal device and does not constitute a limitation on the terminal device. The terminal device may include more or fewer components than shown in the figure, or a combination of certain components, or different components.

[0091] The input unit 130 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the terminal device. Specifically, the input unit 130 may include a touch screen 131 (optional) and / or other input devices 132. The touch screen 131 can collect the user's touch operations on or near it (such as the user's operation on or near the touch screen using any suitable object such as fingers, joints, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the user's touch action on the touch screen, convert the touch action into a touch signal and send it to the processor 170, and can receive and execute the command sent by the processor 170; the touch signal at least includes the touch point coordinate information. The touch screen 131 can provide an input interface and an output interface between the terminal device 100 and the user. In addition, the touch screen can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch screen 131, the input unit 130 can also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.

[0092] Among them, the input device 132 can receive input data and the like.

[0093] The display unit 140 may be used to display information input by a user or provided to a user, various menus of the terminal device 100, an interactive interface, file display, and / or playback of any multimedia file.

[0094] The memory 120 can be used to store instructions and data. The memory 120 can mainly include an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files, texts, etc.; the instruction storage area can store software units such as operating systems, applications, instructions required for at least one function, or their subsets and extensions. It can also include a non-volatile random access memory; provide the processor 170 with hardware, software and data resources including management of computing and processing equipment, and support control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.

[0095] The processor 170 is the control center of the terminal device 100. It uses various interfaces and lines to connect various parts of the entire terminal device 100. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it executes various functions of the terminal device 100 and processes data, thereby controlling the terminal device as a whole. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 170. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips. The processor 170 may also be used to generate corresponding operation control signals, send them to corresponding components of the computing and processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0096] Among them, the memory 120 can be used to store software codes related to the training method of the target detection algorithm model, the processor 170 can execute the steps of the training method of the target detection algorithm model, and can also schedule other units (such as the above-mentioned input unit 130 and the display unit 140) to realize corresponding functions.

[0097] The radio frequency unit 110 (optional) can be used for receiving and sending information or receiving and sending signals during a call, for example, after receiving the downlink information of the base station, it is sent to the processor 170 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the radio frequency unit 110 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0098] In this embodiment of the present application, the RF unit 110 can send data to the server 200 and receive processing results sent by the server 200.

[0099] It should be understood that the radio frequency unit 110 is optional and can be replaced by other communication interfaces, such as a network port.

[0100] The terminal device 100 also includes a power supply 190 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0101] The terminal device 100 also includes an external interface 180 , which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal device 100 to communicate with other devices, or may be used to connect a charger to charge the terminal device 100 .

[0102] Although not shown, the terminal device 100 may also include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which are not described in detail here. Some or all of the methods described below may be applied in the following embodiments. Figure 2 In the terminal device 100 shown.

[0103] For example, the hardware configuration parameter settings of the terminal device developed using the Python 3.8+PyTorch framework can be as shown in Table 1.

[0104] Table 1

[0105] GPU NVIDIA GeForce RTX 4070 Video Mem 12GB CPU i7-13700KF RAM 16 GB Disk 1TB

[0106] Next describe Figure 1 The product form of the server 200;

[0107] Figure 3 A structural diagram of a server 200 is provided, such as Figure 3 As shown, the server 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.

[0108] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0109] The processor 202 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0110] The memory 204 may include a volatile memory, such as a random access memory (RAM). The memory 204 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0111] Among them, the memory 204 can be used to store software codes related to the training method of the target detection algorithm model, the processor 202 can execute the steps of the training method of the target detection algorithm model of the chip, and can also schedule other units to implement corresponding functions.

[0112] It should be understood that the above-mentioned terminal device 100 and server 200 can be centralized or distributed devices, and the processors in the above-mentioned terminal device 100 and server 200 (such as processor 170 and processor 202) can be hardware circuits (such as application specific integrated circuit (ASIC), field-programmable gate array (FPGA), general-purpose processor, digital signal processor (DSP), microprocessor or microcontroller, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0113] Reference Figure 4 , Figure 4 A flowchart of a method for training a target detection algorithm model provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, a training method for a target detection algorithm model provided in an embodiment of the present application may include steps S401 to S407, and these steps are described in detail below.

[0114] Step S401: Acquire point cloud data in a railway traffic scene through a laser radar.

[0115] The point cloud data includes point cloud coordinates of a plurality of points and reflection intensities respectively corresponding to the plurality of points.

[0116] By using LiDAR to collect point cloud data in railway traffic scenarios, we can obtain point clouds within a certain range from the perspective of the train's forward direction.

[0117] Step S402: annotate the point cloud data to obtain annotation results of multiple objects.

[0118] The annotation result of the object includes point cloud coordinates of points constituting the object, category information to which the object belongs, and position information of a circumscribed 3D rectangular bounding box of the object.

[0119] For example, the rosbag recording function of the ROS system can be used to obtain point cloud data in bag file format. The pcd file generation and bin file conversion are realized through the relevant format conversion tools. The generated bin file is manually annotated with the object annotation results using the SUSTechPOINTS tool.

[0120] For example, objects such as people, animals, and telephone poles in point cloud data may be labeled.

[0121] For example, the point cloud coordinates of a point may be (x, y, z); the reflection intensity may be represented by γ. For example, the category information may be represented by any character, for example, 1 represents a person, 2 represents an animal, and 3 represents a telephone pole.

[0122] Exemplarily, the position information of the circumscribed 3D rectangular bounding box includes: the length L, width W, height H, the center point coordinates (x, y, z) and the direction angle (θ) of the circumscribed 3D rectangular bounding box.

[0123] Step S403: Determine, among the plurality of points included in the point cloud data, points whose distances from the first set ground model are greater than or equal to a first set threshold as non-ground point cloud data.

[0124] It is understandable that the roads on which trains travel are often not flat. In addition to bumps and potholes, there are also large-scale uneven phenomena such as bridges, tunnels, ramps, etc. In these complex scenarios, the ground may be mistaken for an object. Based on this, the embodiment of the present application believes that "points whose distance from the first set ground model is greater than or equal to the first set threshold belong to the ground." Removing points belonging to the ground can avoid the situation where the ground is mistakenly taken as an object, thereby improving the accuracy of the target detection algorithm model.

[0125] Exemplarily, the first set ground model refers to the ground in the point cloud data. Exemplarily, the first set ground model can be a 1 x+b 1 y+c 1 z+d 1 =0 represents. Among them, (a 1 , b 1 , c 1 ) is the normal vector of the first set ground model. 1 is the preset offset.

[0126] Step S404: Acquire a target object located in the non-ground point cloud from the multiple objects.

[0127] Step S405: obtaining a transformation object based on the target object.

[0128] Step S406: taking the point cloud coordinates of the points included in the multiple sample objects and the reflection intensity of the points as input, taking the annotation results corresponding to the multiple sample objects as training targets, and training to obtain a target detection algorithm model.

[0129] The plurality of sample objects include the target object and the transformed object.

[0130] It is understandable that in order to improve the accuracy of the target detection algorithm, more sample objects are needed. In order to obtain more sample objects, the target object can be transformed.

[0131] Exemplarily, the point cloud within the circumscribed 3D rectangular bounding box of the target object may be scaled, translated, rotated, and flipped to obtain a transformed object.

[0132] Step S407: deploying the target detection algorithm model on a terminal device installed on the train.

[0133] The embodiment of the present application provides a method for training a target detection algorithm model, which obtains point cloud data in a railway traffic scene through a laser radar, annotates the point cloud data, and obtains annotation results of multiple objects; determines the points in the multiple points contained in the point cloud data whose distance from the first set ground model is greater than or equal to the first set threshold as non-ground point cloud data; obtains target objects located in the non-ground point cloud from multiple objects; obtains transformation objects based on the target objects; uses the point cloud coordinates of the points contained in the multiple sample objects and the reflection intensity of the points as inputs, and uses the annotation results corresponding to the multiple sample objects as training targets to train and obtain a target detection algorithm model; the multiple sample objects include target objects and transformation objects; and deploys the target detection algorithm model on a terminal device set on a train. The target detection algorithm model deployed by the terminal device can detect objects in the running environment of the train, thereby achieving the purpose of detecting objects in the running environment of the train in combination with the laser radar.

[0134] In an optional implementation, there are multiple methods for obtaining the first set ground model. The embodiment of the present application provides but is not limited to the following method, which includes the following steps A1 to A6 during implementation.

[0135] Step A1: randomly determine three non-collinear points from the point cloud data, wherein the three non-collinear points do not all belong to the constructed candidate ground models.

[0136] For example, the point cloud data P includes N points, denoted as P={p 1 ,p 2 ,p 3 ,…,p N}, where each point p i The point cloud coordinates are (xi ,y i ,z i ). Randomly select three non-collinear points from the point cloud data P, where i is a positive integer greater than or equal to 1 and less than or equal to N.

[0137] Step A2: construct a candidate ground model through the three non-collinear points.

[0138] Exemplarily, a candidate ground model is constructed using three non-collinear points. Exemplarily, the plane equation of the candidate ground model can be obtained by the normal vectors of the three non-collinear points: 2 x+b 2 y+c 2 z+d 2 =0. Among them, (a 2 , b 2 , c 2 ) is the normal vector of the first set ground model. 2 is the preset offset.

[0139] Step A3: Divide the points whose distances to the candidate ground model are less than or equal to a second preset threshold among the multiple points included in the point cloud data into an inner point set corresponding to the candidate ground model.

[0140] By calculating the vertical distance from other points in the point cloud data to the candidate ground model, the points whose vertical distance is less than or equal to the second preset threshold are defined as "inliers". Exemplarily, the second preset threshold is usually set according to the application requirements and data characteristics. The ground features in the track where the railway runs are complex, so the second preset threshold is appropriately enlarged to include more inliers. A candidate plane model with more inliers indicates that it is more consistent with the ground.

[0141] Step A4: From the inlier point sets respectively corresponding to the plurality of candidate ground models, determine the candidate ground model corresponding to the inlier point set having the most points as the candidate optimal ground model.

[0142] Step A5: If the current number of iterations is less than or equal to the preset number of iterations, return to step A11.

[0143] During multiple iterations, three new non-collinear points are randomly selected each time to obtain a candidate ground model. The quality of the candidate ground model is evaluated according to the number of inliers in the inlier set corresponding to the candidate ground model. The candidate ground model with the largest number of inliers after each iterative update will be temporarily recorded as the candidate optimal ground model.

[0144] Exemplarily, the preset number of iterations is usually determined according to the data scale and the ratio of inliers, and is limited by the set maximum number of iterations to prevent infinite iterations. Finally, the candidate ground model with the most inliers after the iteration is completed is the first set ground model.

[0145] Step A6: If the current number of iterations is greater than the preset number of iterations, the candidate optimal ground model is determined to be the first set ground model.

[0146] After the first set ground model is determined, the point cloud data is divided into ground point cloud data and non-ground point cloud data. Based on the first set ground model, the distance from each point to the first set ground model is calculated. If the distance is less than the first set threshold, the point is classified as ground point cloud data; if the distance is greater than or equal to the first set threshold, the point is classified as non-ground point cloud data. In this way, the point cloud data is divided into ground point cloud data and non-ground point cloud data, which is helpful for subsequent track anomaly monitoring, such as identifying possible dangerous obstacles such as pedestrians and electric poles.

[0147] In an optional implementation, there are multiple implementations of step S405, and the embodiments of the present application provide but are not limited to the following three methods.

[0148] The first method for implementing step S405 includes the following steps B11 to B13.

[0149] Step B11: Connect the center of the circumscribed 3D rectangular bounding box of the target object and the vertices of the circumscribed 3D rectangular bounding box to obtain six regions.

[0150] Step B12: for each of the six regions, determine the points contained in the region as the point set corresponding to the region, so as to obtain the point sets corresponding to the six regions respectively.

[0151] Step B13: Randomly delete a first number of the point sets corresponding to the six regions to obtain the transformation object, where the first number is an integer greater than or equal to 1 and less than or equal to 5.

[0152] In order for those skilled in the art to better understand the point sets corresponding to the six areas mentioned in the embodiments of the present application, the following examples are provided for illustration.

[0153] See also Figure 5 , which is a schematic diagram of six point sets corresponding to the six areas provided in an embodiment of the present application.

[0154] like Figure 5 As shown, when the center point of the circumscribed 3D rectangular bounding box is connected to the eight vertices of the circumscribed 3D rectangular bounding box, six quadrangular pyramids are formed, and these six quadrangular pyramids are six regions. The points located in each region constitute the point set corresponding to the region.

[0155] The following example illustrates step B13. It is assumed that the six regions of the circumscribed 3D rectangular bounding box 51 of the target object are the first region, the second region, the third region, the fourth region, the fifth region and the sixth region respectively; the point sets corresponding to the first region, the second region, the third region, the fourth region, the fifth region and the sixth region are point set A, point set B, point set C, point set D, point set E and point set F respectively; one, two, three, four or five point sets can be randomly deleted. Assuming that the first number is 1, point set E can be deleted. The exchange object 52 thus obtained includes the circumscribed 3D rectangular bounding box of the target object and point set A, point set B, point set C, point set D and point set F. From Figure 5 It can be seen that the point set E of the fifth region of the circumscribed 3D rectangular bounding box 52 of the exchange object has been deleted.

[0156] The exchange object is obtained by randomly deleting the first number of point sets corresponding to the six regions, which can simulate the situation in which some point cloud data is missing due to object occlusion, environmental interference or incomplete data collection in actual scenes. This random deletion operation effectively improves the detection ability of the target detection algorithm model under the condition of missing some point cloud data of the object, and enhances its robustness in the real environment.

[0157] The second method for implementing step S405 includes the following steps B21 to B23.

[0158] Step B21: For each target object, connect the center of the circumscribed 3D rectangular bounding box of the target object with the vertices of the circumscribed 3D rectangular bounding box to obtain six regions.

[0159] Step B22: for each of the target objects, for each of the six regions of the target object, determine the points contained in the region as the point set corresponding to the region, so as to obtain the point sets corresponding to the six regions respectively.

[0160] Step B23: for any two target objects belonging to the same category information, replace the point set corresponding to the area at the same position in the circumscribed 3D rectangular bounding boxes of the two target objects to obtain the transformed object.

[0161] In order for those skilled in the art to better understand the areas at the same position in the circumscribed 3D rectangular bounding boxes of the two target objects mentioned in the embodiments of the present application, an example is given below to illustrate.

[0162] like Figure 6 , which is a schematic diagram of the area at the same position in the circumscribed 3D rectangular bounding box of two target objects.

[0163] It is understandable that the circumscribed 3D rectangular bounding box of each target object can be divided into 6 regions. Exemplarily, for the circumscribed 3D rectangular bounding boxes of any two target objects, the regions at the same position refer to two quadrangular pyramids of similar size in the two circumscribed 3D rectangular bounding boxes. Exemplarily, for the circumscribed 3D rectangular bounding boxes of any two target objects, the regions at the same position refer to the regions containing the same sub-objects. Figure 6 As shown, the sub-object contained in the area 611 filled with diagonal lines in the circumscribed 3D rectangular bounding box 61 of the target object is the head of pedestrian AA, and the sub-object contained in the area 621 filled with grids in the circumscribed 3D rectangular bounding box 62 of the target object is the head of pedestrian BB, so area 611 and area 621 are areas in the same position.

[0164] Exemplarily, the point set corresponding to the area at the same position in the circumscribed 3D rectangular bounding box of the changed object and the target object can be replaced to obtain a new changed object. Exemplarily, the point set corresponding to the area at the same position in the circumscribed 3D rectangular bounding box of the changed object and the changed object can be replaced to obtain a new changed object.

[0165] Assume that both the target object and the transformed object have 6 regions, which are respectively called: the first region, the second region, the third region, the fourth region, the fifth region and the sixth region. Assume that the 6 point sets included in the target object CC are: point set A (corresponding to the first region), point set B (corresponding to the second region), point set C (corresponding to the third region), point set D (corresponding to the fourth region), point set E (corresponding to the fifth region) and point set F (corresponding to the sixth region). The 6 point sets included in the target object DD are: point set G (corresponding to the first region), point set H (corresponding to the second region), point set I (corresponding to the third region), point set J (corresponding to the fourth region), point set K (corresponding to the fifth region), and point set L (corresponding to the sixth region). Assuming that the position of the third region corresponding to point set C and the third region corresponding to point set I are the same, the two transformed objects obtained are: target object EE {point set A, point set B, point set I, point set D, point set E and point set F} and target object FF {point set G, point set H, point set C, point set J, point set K, point set L}.

[0166] It is understandable that the circumscribed 3D rectangular bounding box of the target object EE is the same as the circumscribed 3D rectangular bounding box of the target object CC, except that the point set corresponding to the third region has changed. Exemplarily, after obtaining the changed object, the point set of the changed object can be scaled or offset to obtain a new transformed object.

[0167] It is understandable that the sizes of the circumscribed 3D rectangular bounding box of the target object CC and the circumscribed 3D rectangular bounding box of the target object DD may be different. Therefore, when the point set I is placed in the third area of ​​the circumscribed 3D rectangular bounding box of the target object CC, it is necessary to scale the point set I. Exemplarily, the scaling ratio may be determined based on actual conditions and is not limited here.

[0168] By obtaining the transformed object through steps B21 to B23, the diversity of objects can be increased, and the adaptability of the target detection algorithm model to the detection of objects in different spatial forms can be enhanced. The point set exchange operation enables the target detection algorithm model to better recognize and understand the geometric features of objects at multiple angles and postures.

[0169] The third method for implementing step S405 includes the following steps B31 to B33.

[0170] Step B31: Connect the center of the circumscribed 3D rectangular bounding box of the target object and the vertices of the circumscribed 3D rectangular bounding box to obtain six regions.

[0171] Step B32: for each of the six regions, determine the points contained in the region as the point set corresponding to the region, so as to obtain the point sets corresponding to the six regions respectively.

[0172] Step B33: Delete some points in the point set of at least one of the six areas to obtain the transformed object.

[0173] It is understandable that for distant objects, the point cloud density acquired by the LiDAR is low. This application simulates the situation where part of the target object is far away from the LiDAR by deleting some points in the point set of at least one of the six areas of the circumscribed 3D rectangular bounding box of the target object. This improves the point cloud resolution of the target detection algorithm model for identifying distant objects. The above operations enhance the detection performance of the target detection algorithm model for objects at different distances, making the target detection algorithm model perform better in long-distance target detection.

[0174] like Figure 7 , which is a schematic diagram of the structure of the target detection algorithm model provided in an embodiment of the present application.

[0175] The target detection algorithm model includes a columnar division module 91, a sparse convolution module 92 whose input end is connected to the output end of the columnar division module 91, a coordinate attention mechanism module 93 whose input end is connected to the output end of the sparse convolution module 92, a neck network module 94 whose input end is connected to the output end of the coordinate attention mechanism module 93, and a detection head module 95 whose input end is connected to the output end of the neck network module 94; the input end of the columnar division module is the input end of the target detection algorithm model, and the output end of the detection head module is the output end of the target detection algorithm model.

[0176] Combination Figure 7 The process of step S406 is explained. The process of training the target detection algorithm model includes the following steps C1 to C7.

[0177] Step C1: inputting the point cloud coordinates of the points contained in the plurality of sample objects and the reflection intensity of the points into the target detection algorithm model.

[0178] Step C2: obtaining first feature information based on the point cloud coordinates of the points included in the multiple sample objects and the reflection intensity of the points through the columnar division module.

[0179] Exemplarily, the columnar division module can obtain point cloud data within a certain range of the forward viewing angle along the direction of the train's travel. Assume that the length of the acquired point cloud data in the X-axis direction is Lx, and the length in the Y-axis direction is Ly. Divide the point cloud data into equal-sized vertical columns (i.e., columns) along the XOY plane with a certain step length, and set the step lengths in the X-axis and Y-axis directions to Sx and Sy respectively, and set Nx and Ny to be the number of grids in the X-axis direction and the X-axis direction respectively, Nx = Lx / Sx, Ny = Ly / Sy, and the total number of columns Np = Nx*Ny.

[0180] Exemplarily, the feature information of each column includes: the coordinates of the point cloud of each column, the reflection intensity of each column, and the derived features; the derived features include: the geometric center coordinates (x c ,y z , z c ), the coordinates (x, y) of all points in the cylinder and the coordinates of the geometric center (x c ,y c ) p ,y p ), where x p =xx c ,y p =yy c .

[0181] Exemplarily, the feature information of each column can be compressed by maximum pooling to obtain a feature vector of a fixed length, namely, the first feature information. The first feature information includes the feature vectors of each column.

[0182] Step C3: obtaining second feature information based on the first feature information through the sparse convolution module.

[0183] It is understandable that, since point cloud data is usually highly sparse, the divided columns are used to extract key information using the sparse convolution module, thereby improving the overall performance of the target detection algorithm model. In the sparse convolution module, the data is stored in the form of a sparse matrix. Only columns containing features will be passed to the convolution layer, while empty columns will be ignored. The second feature information obtained by the sparse convolution module can be used in the subsequent deep learning network.

[0184] Step C4: Obtain third feature information based on the second feature information through the coordinate attention mechanism module.

[0185] Step C5: Obtain fourth characteristic information based on the third characteristic information through the neck network module.

[0186] pass Figure 7 It can be seen that the neck network module includes 5 sparse convolutional units. The 5 sparse convolutional units (from left to right and from top to bottom) can respectively obtain 2x downsampling feature maps, 4x downsampling feature maps, 8x downsampling feature maps, 16x downsampling feature maps, and 8x downsampling feature maps.

[0187] Through the neck network module, multi-scale feature fusion is performed, and the obtained 8x down-sampling feature map and 16x down-sampling feature map are fused to obtain a new 8x down-sampling feature map, which is then fused with the previously obtained 2x down-sampling feature map and 4x down-sampling feature map. The fused multi-scale feature map can improve the robustness of the target detection algorithm model, making it perform well under various conditions.

[0188] Step C6: Obtain prediction results corresponding to the multiple sample objects respectively based on the fourth feature information through the detection head module, and the prediction results of the sample objects include point cloud coordinates of points constituting the sample objects, category information of the sample objects, and position information of the circumscribed 3D rectangular bounding box of the sample objects.

[0189] Step C7: Compare the prediction results corresponding to the plurality of sample objects and the annotation results corresponding to the plurality of sample objects to obtain a comparison result.

[0190] Step C8: Update the target detection algorithm model based on the comparison result.

[0191] The structure of the coordinate attention mechanism module is explained below.

[0192] like Figure 8 As shown, it is a structural diagram of the coordinate attention mechanism module provided in an embodiment of the present application.

[0193] The coordinate attention mechanism module includes: a maximum pooling layer 1001, a concatenation layer 1002, and a multi-layer perceptron 1003. Figure 8 The implementation process of step C4 is described. The process includes the following steps D1 to D3.

[0194] Step D1: Obtain a first feature matrix in the X-axis direction and a second feature matrix in the Y-axis direction based on the second feature information through the maximum pooling layer in the coordinate attention mechanism module, wherein the X-axis direction is the direction in which the train moves forward, and the Y-axis direction is a direction perpendicular to the X-axis.

[0195] The purpose of using the Max Pooling layer is to extract the most significant features from the second feature information, by selecting the maximum value in each pooling area, thereby effectively reducing the dimension of the data while retaining important information. The result of this step is to generate two feature maps, namely the first feature matrix and the second feature matrix, which represent the feature information in the X-axis direction and the Y-axis direction respectively.

[0196] Step D2: splicing the first feature matrix and the second feature matrix through the splicing layer in the coordinate attention mechanism module to obtain a target feature matrix.

[0197] The width, height or number of channels of the first feature matrix and the second feature matrix are adjusted to perform dimension transformation, so as to ensure that the adjusted first feature matrix and the second feature matrix maintain a consistent shape in the concatenation operation.

[0198] Step D3: Acquire the third feature information based on the target feature matrix through the multi-layer perceptron in the coordinate attention mechanism module.

[0199] The target feature matrix is ​​convolved along the spatial dimension to further increase the dimension and extract higher-level feature information. The convolution operation processes the spliced ​​target feature matrix through a set of trainable convolution kernels, so that more complex feature representations can be learned. Finally, the sigmoid activation function is combined to map the output attention vector to the range of 0 to 1, and the result of the convolution operation is nonlinearly transformed to generate the final attention vector.

[0200] In an optional implementation, the following method is also included, and the method includes the following steps E1 to E4 during implementation.

[0201] Step E1: Collecting the point cloud data to be measured by the laser radar deployed on the train.

[0202] Step E2: Determine, as non-ground point cloud data, points whose distances from the second set ground model are greater than or equal to the first set threshold value among the multiple points included in the point cloud data to be tested.

[0203] Illustratively, the process of obtaining the second set ground model is the same as the process of obtaining the first set ground model, which will not be described in detail here.

[0204] Step E3: Input the non-ground point cloud data in the point cloud data to be tested into the target detection algorithm model deployed by the terminal device, and obtain the prediction result of the predicted object through the target detection algorithm model. The prediction result includes the point cloud coordinates of the points constituting the predicted object, the category information of the predicted object, and the position information of the circumscribed 3D rectangular bounding box of the predicted object.

[0205] Step E4: Control the terminal device to issue an early warning based on the prediction result of the prediction object.

[0206] The above introduces a training method for a target detection algorithm model provided in an embodiment of the present application. The following will introduce a device for executing the above training method for the target detection algorithm model.

[0207] See also Fig. 9 , is a schematic diagram of the structure of a training device for a target detection algorithm model provided in an embodiment of the present application. Fig. 9 As shown, the training device of the target detection algorithm model includes:

[0208] The first acquisition module 1101 is used to acquire point cloud data in a railway traffic scene by using a laser radar, wherein the point cloud data includes point cloud coordinates of a plurality of points and reflection intensities corresponding to the plurality of points;

[0209] The second acquisition module 1102 is used to annotate the point cloud data to obtain annotation results of multiple objects, where the annotation results of the objects include point cloud coordinates of points constituting the objects, category information of the objects, and position information of the circumscribed 3D rectangular bounding box of the objects;

[0210] A first determination module 1103 is used to determine, among the multiple points included in the point cloud data, points whose distances from the first set ground model are greater than or equal to a first set threshold as non-ground point cloud data;

[0211] A third acquisition module 1104 is used to acquire a target object located in the non-ground point cloud from the multiple objects;

[0212] A fourth acquisition module 1105 is used to acquire a transformation object based on the target object;

[0213] A training module 1106 is used to take the point cloud coordinates and reflection intensity of the points included in the multiple sample objects as input, and take the annotation results corresponding to the multiple sample objects as training targets, to train and obtain a target detection algorithm model; the multiple sample objects include the target object and the transformation object;

[0214] The deployment module 1107 is used to deploy the target detection algorithm model on a terminal device installed on the train.

[0215] In an optional implementation, the method further includes:

[0216] A second determination module is used to randomly determine three non-collinear points from the point cloud data, wherein the three non-collinear points do not all belong to the candidate ground model that has been constructed;

[0217] A construction module, configured to construct a candidate ground model through the three non-collinear points;

[0218] A partitioning module, used to partition points whose distances from the candidate ground model are less than or equal to a second preset threshold among a plurality of points included in the point cloud data into an inner point set corresponding to the candidate ground model;

[0219] A third determination module is used to determine, from the inlier point sets respectively corresponding to the plurality of candidate ground models, the candidate ground model corresponding to the inlier point set having the most points as the candidate optimal ground model;

[0220] A triggering module, configured to trigger the second determining module if the current number of iterations is less than or equal to a preset number of iterations;

[0221] The fourth determination module is used to determine that the candidate optimal ground model is the first set ground model if the current iteration number is greater than the preset iteration number.

[0222] In an optional implementation, the fourth acquisition module includes:

[0223] A first acquisition unit is used for connecting the center of a circumscribed 3D rectangular bounding box of the target object and the vertices of the circumscribed 3D rectangular bounding box for each target object to obtain six regions;

[0224] A first determining unit is used for determining, for each of the six regions of each of the target objects, points contained in the region as a point set corresponding to the region, so as to obtain the point sets corresponding to the six regions respectively;

[0225] The second acquisition unit is used to replace the point set corresponding to the area at the same position in the circumscribed 3D rectangular bounding box of any two target objects belonging to the same category information to obtain the transformed object.

[0226] In an optional implementation, the fourth acquisition module includes:

[0227] A third acquisition unit is used to connect the center of the circumscribed 3D rectangular bounding box of the target object and the vertices of the circumscribed 3D rectangular bounding box to obtain six areas;

[0228] A second determining unit is used to determine, for each of the six regions, points contained in the region as a point set corresponding to the region, so as to obtain the point sets corresponding to the six regions respectively;

[0229] The fourth acquisition unit is used to delete some points in the point set of at least one of the six areas to obtain the transformed object.

[0230] In an optional implementation, the fourth acquisition module includes:

[0231] a fifth acquisition unit, configured to connect the center of the circumscribed 3D rectangular bounding box of the target object with the vertices of the circumscribed 3D rectangular bounding box to obtain six regions;

[0232] A third determining unit is used to determine, for each of the six regions, points contained in the region as a point set corresponding to the region, so as to obtain the point sets corresponding to the six regions respectively;

[0233] A sixth acquisition unit is used to randomly delete a first number of the point sets from the point sets corresponding to the six regions respectively, to obtain the transformation object, where the first number is an integer greater than or equal to 1 and less than or equal to 5.

[0234] In an optional implementation, the target detection algorithm model includes a columnar division module, a sparse convolution module whose input end is connected to the output end of the columnar division module, a coordinate attention mechanism module whose input end is connected to the output end of the sparse convolution module, a neck network module whose input end is connected to the output end of the coordinate attention mechanism module, and a detection head module whose input end is connected to the output end of the neck network module; the input end of the columnar division module is the input end of the target detection algorithm model, and the output end of the detection head module is the output end of the target detection algorithm model; the training module includes:

[0235] An input unit, used to input the point cloud coordinates of the points included in the multiple sample objects and the reflection intensity of the points into the target detection algorithm model;

[0236] A seventh acquisition unit, configured to acquire first feature information based on point cloud coordinates of points included in the plurality of sample objects and reflection intensities of the points through the columnar division module;

[0237] an eighth acquisition unit, configured to acquire second feature information based on the first feature information through the sparse convolution module;

[0238] a ninth acquisition unit, configured to acquire third feature information based on the second feature information through the coordinate attention mechanism module;

[0239] a tenth acquiring unit, configured to acquire fourth feature information based on the third feature information through the neck network module;

[0240] an eleventh acquisition unit, configured to acquire, through the detection head module, prediction results respectively corresponding to a plurality of the sample objects based on the fourth feature information, the prediction results of the sample objects including point cloud coordinates of points constituting the sample objects, category information to which the sample objects belong, and position information of an external 3D rectangular bounding box of the sample objects;

[0241] a twelfth obtaining unit, configured to compare the prediction results respectively corresponding to the plurality of sample objects and the annotation results respectively corresponding to the plurality of sample objects, to obtain a comparison result;

[0242] An updating unit is used to update the target detection algorithm model based on the comparison result.

[0243] In an optional implementation, the ninth acquisition unit includes:

[0244] A first acquisition subunit is used to acquire a first feature matrix in an X-axis direction and a second feature matrix in a Y-axis direction based on the second feature information through a maximum pooling layer in the coordinate attention mechanism module, wherein the X-axis direction is a direction in which the train moves forward, and the Y-axis direction is a direction perpendicular to the X-axis;

[0245] A concatenation subunit, configured to concatenate the first feature matrix and the second feature matrix through a concatenation layer in the coordinate attention mechanism module to obtain a target feature matrix;

[0246] The second acquisition subunit is used to acquire the third feature information based on the target feature matrix through the multi-layer perceptron in the coordinate attention mechanism module.

[0247] In an optional implementation, the method further includes:

[0248] A collection module, used for collecting point cloud data to be measured through a laser radar deployed on the train;

[0249] A fourth determination module, configured to determine, among the plurality of points included in the point cloud data to be measured, points whose distances from the second set ground model are greater than or equal to the first set threshold as non-ground point cloud data;

[0250] a fifth acquisition module, configured to input the non-ground point cloud data in the point cloud data to be tested into the target detection algorithm model deployed by the terminal device, and obtain a prediction result of the predicted object through the target detection algorithm model, wherein the prediction result includes the point cloud coordinates of the points constituting the predicted object, the category information to which the predicted object belongs, and the position information of the circumscribed 3D rectangular bounding box of the predicted object;

[0251] A control module is used to control the terminal device to issue an early warning based on the prediction result of the prediction object.

[0252] The embodiment of the present application also provides an electronic device. The electronic device may be a terminal device 100 or a server 200. Fig.10 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Fig.10 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0253] like Fig.10 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage device 1208 to a random access memory (RAM) 1203. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 1203. The processing device 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0254] Typically, the following devices may be connected to the I / O interface 1205: an input device 1206 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1207 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1208 including, for example, a memory card, a hard disk, etc.; and a communication device 1209. The communication device 1209 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Fig.10An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0255] An embodiment of the present application also provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements a training method for any target detection algorithm model provided in the embodiment of the present application.

[0256] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When one or more computer programs are executed by an electronic device, the electronic device can implement a training method for any target detection algorithm model provided in an embodiment of the present application.

[0257] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0258] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0259] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0260] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for training a target detection algorithm model, characterized in that: include: Acquire point cloud data in a railway traffic scene by using a laser radar, wherein the point cloud data includes point cloud coordinates of a plurality of points and reflection intensities corresponding to the plurality of points respectively; Annotating the point cloud data to obtain annotation results of multiple objects, wherein the annotation results of the objects include point cloud coordinates of points constituting the objects, category information of the objects, and position information of circumscribed 3D rectangular bounding boxes of the objects; Determine, among the plurality of points included in the point cloud data, points whose distances from the first set ground model are greater than or equal to a first set threshold as non-ground point cloud data; Acquire a target object located in the non-ground point cloud from the plurality of objects; Acquire a transformation object based on the target object; Taking the point cloud coordinates and the reflection intensity of the points included in the multiple sample objects as input, taking the annotation results respectively corresponding to the multiple sample objects as training targets, and training to obtain the target detection algorithm model; the multiple sample objects include the target object and the transformation object; The target detection algorithm model is deployed on a terminal device installed on a train.

2. The training method of the target detection algorithm model according to claim 1, characterized in that: The method for obtaining the first set ground model includes: Randomly determine three non-collinear points from the point cloud data, wherein the three non-collinear points do not all belong to the constructed candidate ground models; A candidate ground model is formed by the three non-collinear points; Dividing points whose distances from the candidate ground model are less than or equal to a second preset threshold among a plurality of points included in the point cloud data into an inner point set corresponding to the candidate ground model; From the inlier point sets respectively corresponding to the plurality of candidate ground models, determine the candidate ground model corresponding to the inlier point set having the most points as the candidate optimal ground model; If the current number of iterations is less than or equal to the preset number of iterations, returning to the step of randomly determining three non-collinear points from the point cloud data; If the current number of iterations is greater than the preset number of iterations, the candidate optimal ground model is determined to be the first set ground model.

3. The training method of the target detection algorithm model according to claim 1, characterized in that: The step of acquiring a transformation object based on the target object comprises: Connecting the center of the circumscribed 3D rectangular bounding box of the target object with the vertices of the circumscribed 3D rectangular bounding box to obtain six regions; For each of the six regions, determining the points contained in the region as a point set corresponding to the region, so as to obtain the point sets corresponding to the six regions respectively; A first number of the point sets in the point sets corresponding to the six regions are randomly deleted to obtain the transformation object, where the first number is an integer greater than or equal to 1 and less than or equal to 5.

4. The training method of the target detection algorithm model according to claim 1, characterized in that: The step of acquiring a transformation object based on the target object comprises: For each of the target objects, connecting the center of a circumscribed 3D rectangular bounding box of the target object with the vertices of the circumscribed 3D rectangular bounding box to obtain six regions; For each of the target objects, for each of the six regions of the target object, determining the points contained in the region as the point set corresponding to the region, so as to obtain the point sets corresponding to the six regions respectively; For any two target objects belonging to the same category information, the point sets corresponding to the areas at the same positions in the circumscribed 3D rectangular bounding boxes of the two target objects are replaced to obtain the transformed objects.

5. The training method of the target detection algorithm model according to claim 1, characterized in that: The step of acquiring a transformation object based on the target object comprises: Connecting the center of the circumscribed 3D rectangular bounding box of the target object with the vertices of the circumscribed 3D rectangular bounding box to obtain six regions; For each of the six regions, determining the points contained in the region as a point set corresponding to the region, so as to obtain the point sets corresponding to the six regions respectively; Part of the points in the point set of at least one of the six areas is deleted to obtain the transformed object.

6. The method for training a target detection algorithm model according to any one of claims 1 to 5, characterized in that: The target detection algorithm model includes a columnar division module, a sparse convolution module whose input end is connected to the output end of the columnar division module, a coordinate attention mechanism module whose input end is connected to the output end of the sparse convolution module, a neck network module whose input end is connected to the output end of the coordinate attention mechanism module, and a detection head module whose input end is connected to the output end of the neck network module; the input end of the columnar division module is the input end of the target detection algorithm model, and the output end of the detection head module is the output end of the target detection algorithm model; The step of taking the point cloud coordinates of the points included in the plurality of sample objects and the reflection intensity of the points as input, taking the annotation results respectively corresponding to the plurality of sample objects as training targets, and training to obtain the target detection algorithm model comprises: Inputting the point cloud coordinates of the points included in the plurality of sample objects and the reflection intensity of the points into the target detection algorithm model; Acquire first feature information based on the point cloud coordinates of points included in the plurality of sample objects and the reflection intensity of the points through the columnar division module; Acquire second feature information based on the first feature information through the sparse convolution module; Acquire third feature information based on the second feature information through the coordinate attention mechanism module; Acquiring fourth characteristic information based on the third characteristic information through the neck network module; Acquire, by the detection head module, prediction results respectively corresponding to the plurality of sample objects based on the fourth feature information, the prediction results of the sample objects including point cloud coordinates of points constituting the sample objects, category information to which the sample objects belong, and position information of an external 3D rectangular bounding box of the sample objects; Comparing the prediction results respectively corresponding to the plurality of sample objects and the annotation results respectively corresponding to the plurality of sample objects to obtain a comparison result; The target detection algorithm model is updated based on the comparison result.

7. The training method of the target detection algorithm model according to claim 6, characterized in that: The step of obtaining the third feature information based on the second feature information through the coordinate attention mechanism module includes: Obtaining a first feature matrix in an X-axis direction and a second feature matrix in a Y-axis direction based on the second feature information through a maximum pooling layer in the coordinate attention mechanism module, wherein the X-axis direction is the direction in which the train moves forward, and the Y-axis direction is a direction perpendicular to the X-axis; Splicing the first feature matrix and the second feature matrix through the splicing layer in the coordinate attention mechanism module to obtain a target feature matrix; The third feature information is obtained based on the target feature matrix through a multilayer perceptron in the coordinate attention mechanism module.

8. The training method of the target detection algorithm model according to claim 1, characterized in that: Also includes: Collecting the point cloud data to be measured by using the laser radar deployed on the train; Determine, as non-ground point cloud data, points whose distances from a second set ground model are greater than or equal to the first set threshold value among a plurality of points included in the point cloud data to be tested; Inputting the non-ground point cloud data in the point cloud data to be tested into the target detection algorithm model deployed by the terminal device, and obtaining the prediction result of the predicted object through the target detection algorithm model, wherein the prediction result includes the point cloud coordinates of the points constituting the predicted object, the category information to which the predicted object belongs, and the position information of the circumscribed 3D rectangular bounding box of the predicted object; Control the terminal device to issue an early warning based on the prediction result of the prediction object.

9. A training device for a target detection algorithm model, characterized in that: include: A first acquisition module is used to acquire point cloud data in a railway traffic scene through a laser radar, wherein the point cloud data includes point cloud coordinates of a plurality of points and reflection intensities corresponding to the plurality of points respectively; A second acquisition module is used to annotate the point cloud data to obtain annotation results of multiple objects, where the annotation results of the objects include point cloud coordinates of points constituting the objects, category information of the objects, and position information of circumscribed 3D rectangular bounding boxes of the objects; A first determination module, configured to determine, among a plurality of points included in the point cloud data, points whose distances from a first set ground model are greater than or equal to a first set threshold as non-ground point cloud data; A third acquisition module, used for acquiring a target object located in the non-ground point cloud from the multiple objects; A fourth acquisition module, used for acquiring a transformation object based on the target object; A training module, used to take the point cloud coordinates and reflection intensity of the points contained in the multiple sample objects as input, take the annotation results respectively corresponding to the multiple sample objects as training targets, and train to obtain a target detection algorithm model; the multiple sample objects include the target object and the transformation object; A deployment module is used to deploy the target detection algorithm model on a terminal device installed on a train.

10. A computer program product, characterized in that It includes computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device implements the training method of the target detection algorithm model as described in any one of claims 1 to 8.

Images