Target paving path planning method and device based on deep learning and image recognition

Abstract

The application provides a target paving path planning method and device based on deep learning and image recognition, relates to the technical field of paving paths, and comprises the following steps: acquiring track plate state image information and environment dynamic image information; performing geometric modeling based on the environment dynamic image information to obtain a three-dimensional environment model; analyzing the track plate state image information and the three-dimensional environment model according to a deep learning algorithm, and predicting the paving path of the track plate based on a motion prediction algorithm; improving a particle swarm algorithm based on inertia weight and crossover mutation probability, and dynamically adjusting the parameters of the particle swarm algorithm based on a neural network and construction progress data to obtain a particle swarm optimization algorithm; inputting the predicted motion trajectory and path constraint conditions into the particle swarm optimization algorithm for solving; performing path paving according to the paving motion path and monitoring, and ending paving when the monitoring result meets a preset deviation threshold. The application solves the problem that path planning cannot be adjusted in time when facing dynamic environment changes.

Description

Technical Field

[0001] This invention relates to the field of paving path technology, and more specifically, to a method and apparatus for planning target paving paths based on deep learning and image recognition. Background Technology

[0002] In the current field of track paving path technology, track slab paving paths are mainly planned based on manual experience. This method can provide basic path guidance for construction in specific and relatively stable engineering environments, playing a certain role. However, in-depth analysis of existing technologies reveals that track slab paving path planning based on manual experience has significant accuracy deficiencies. In actual paving scenarios, the engineering environment is complex and variable, influenced by a combination of factors such as differences in geological conditions, the impact of surrounding existing buildings, and dynamic interference from construction. The low accuracy of track slab paving path planning in existing technologies leads to the problem of being unable to adjust the path planning in a timely manner in the face of dynamic environmental changes.

[0003] Therefore, there is an urgent need for a target paving path planning method and device based on deep learning and image recognition, which solves the problem of not being able to adjust path planning in a timely manner when facing dynamic environmental changes. Summary of the Invention

[0004] The purpose of this invention is to provide a method and apparatus for target paving path planning based on deep learning and image recognition, so as to improve the above-mentioned problems. To achieve the above objective, the technical solution adopted by this invention is as follows:

[0005] Firstly, this application provides a target paving path planning method based on deep learning and image recognition, including:

[0006] Acquire track slab status image information and environmental dynamic image information, wherein the environmental dynamic image information includes complete environmental images from at least two angles;

[0007] Based on the dynamic environmental image information, and combined with the obstacle movement trajectory and track slab deformation, a three-dimensional environment model is obtained through geometric modeling.

[0008] The track slab status image information and 3D environment model are analyzed based on deep learning algorithms, and the track slab paving path is predicted based on motion prediction algorithms to obtain the predicted motion trajectory.

[0009] The particle swarm optimization algorithm is improved based on inertia weight and crossover mutation probability, and the parameters of the particle swarm optimization algorithm are dynamically adjusted based on neural network and construction progress data to obtain the particle swarm optimization algorithm.

[0010] The predicted motion trajectory and preset path constraints are input into the particle swarm optimization algorithm to solve for the paving motion path.

[0011] The paving is carried out according to the paving movement path and monitored. When the monitoring result meets the preset deviation threshold, the paving ends.

[0012] Secondly, this application also provides a target paving path planning device based on deep learning and image recognition, including:

[0013] The acquisition module is used to acquire track slab status image information and environmental dynamic image information, wherein the environmental dynamic image information includes complete environmental images from at least two angles;

[0014] The modeling module is used to perform geometric modeling based on the dynamic image information of the environment, combined with the movement trajectory of obstacles and the deformation of the track slab, to obtain a three-dimensional environment model;

[0015] The analysis module is used to analyze the track slab status image information and 3D environment model based on deep learning algorithms, and to predict the track slab paving path based on motion prediction algorithms to obtain the predicted motion trajectory.

[0016] An improvement module is used to improve the particle swarm algorithm based on inertia weights and crossover mutation probabilities, and to dynamically adjust the parameters of the particle swarm algorithm based on neural networks and construction progress data, so as to obtain an optimized particle swarm algorithm.

[0017] The solution module is used to input the predicted motion trajectory and preset path constraints into the particle swarm optimization algorithm to solve for the paving motion path;

[0018] The monitoring module is used to perform path paving and monitor according to the paving movement path. When the monitoring result meets the preset deviation threshold, the paving ends.

[0019] The beneficial effects of this invention are as follows:

[0020] This invention generates optimized paving paths by combining track slab status images and dynamic environmental images with deep learning and motion prediction algorithms. Furthermore, a 3D environment model and dynamic obstacle trajectories are used to detect potential collision risks in advance, helping construction workers adjust the paving path in a timely manner and effectively preventing construction accidents. Simultaneously, this invention introduces an improved particle swarm optimization algorithm, which optimizes based on inertia weights and crossover mutation probabilities, and dynamically adjusts its parameters using neural networks and construction progress data. This allows for flexible adjustment of the paving path and optimization algorithm parameters based on actual construction progress and environmental conditions, quickly addressing work stoppages or rework caused by unreasonable path planning. In summary, this invention solves the problem of the inability to adjust path planning in a timely manner in the face of dynamic environmental changes.

[0021] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing embodiments of the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the target paving path planning method based on deep learning and image recognition described in an embodiment of the present invention;

[0024] Figure 2 This is a schematic diagram of the target paving path planning device based on deep learning and image recognition as described in an embodiment of the present invention.

[0025] The diagram is labeled as follows: 800, target paving path planning device based on deep learning and image recognition; 801, processor; 802, memory; 803, multimedia component; 804, I / O interface; 805, communication component. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0027] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0028] Example 1:

[0029] This embodiment provides a target paving path planning method based on deep learning and image recognition.

[0030] See Figure 1 The figure shows that the method includes steps S1 to S6, including:

[0031] S1: Acquire track slab status image information and environmental dynamic image information, wherein the environmental dynamic image information includes complete environmental images from at least two angles;

[0032] S2: Based on the dynamic image information of the environment, and combined with the obstacle movement trajectory and track deformation, geometric modeling is performed to obtain a three-dimensional environment model;

[0033] To clarify the specific method for obtaining the 3D environment model, step S2 includes S21 to S25, specifically:

[0034] S21: Based on the dynamic environmental image information, obstacle detection is performed, and the target coordinates of the obstacle are determined by matching the detection results with the feature points of the preset obstacle image.

[0035] In this step, obstacle detection is performed on the dynamic environmental image information based on the target detection algorithm. The bounding boxes of the obstacles are extracted, and key points and feature descriptions are extracted from both the detected obstacle image and a preset obstacle image by extracting image features of the obstacle region. Matching key point pairs are found using a feature matching algorithm to determine the coordinates of the obstacle target in the image. Feature point matching is used to accurately identify obstacles in the environment and determine the obstacle target coordinates, solving the problems of inaccurate obstacle detection and inability to accurately locate obstacles in traditional methods.

[0036] Preferably, the image features include color, texture, and shape.

[0037] S22: Transform the coordinates of the obstacle target into the global three-dimensional coordinate system according to the coordinate transformation algorithm, and construct a global three-dimensional model;

[0038] In this step, the image coordinates of the obstacle target are converted into 3D coordinates in the camera coordinate system using a coordinate transformation algorithm. Then, the camera's extrinsic parameters are used to transform the coordinates from the camera coordinate system to the global 3D coordinate system. The transformed coordinate points are added to the global 3D model to construct a complete global 3D model. This solves the problem of model construction difficulties caused by inconsistent local coordinate systems in traditional methods.

[0039] S23: Based on the obstacle's motion trajectory and the track slab's deformation, perform geometric correction on the global three-dimensional model to obtain an optimized three-dimensional point cloud model;

[0040] In this step, the position and shape of the relevant point cloud in the global 3D model are adjusted according to the obstacle's movement trajectory and the deformation of the track slab to obtain a corrected point cloud; the corrected point cloud is then optimized according to the point cloud processing algorithm to obtain an optimized 3D point cloud model, which is used to ensure the accuracy and integrity of the model.

[0041] S24: The optimized 3D point cloud model is meshed to generate a triangular mesh model. Texture maps are generated by mapping the texture information in the coordinates of the obstacle target onto the triangular mesh model.

[0042] In this step, the optimized 3D point cloud model is converted into a triangular mesh model based on the point cloud meshing algorithm, and the texture information of the obstacle target is extracted and mapped onto the triangular mesh model. Through meshing and texture mapping, a texture map with texture information is generated, which enhances the visual effect and detail of the model.

[0043] S25: Based on the incremental modeling algorithm, the optimized 3D point cloud model is dynamically updated using texture mapping and real-time obstacle detection data to obtain a 3D environment model.

[0044] In this step, based on an incremental modeling algorithm, the optimized 3D point cloud model is dynamically updated using texture mapping and real-time obstacle detection data, combined with real-time texture mapping and obstacle detection data. In each update, the geometric structure and texture information of the optimized 3D point cloud model are adjusted to obtain a 3D environment model. This 3D environment model improves the model's dynamic adaptability and real-time performance, solving the problems of traditional methods where 3D models cannot be updated in real time and cannot adapt to dynamic environmental changes.

[0045] S3: Analyze the track slab status image information and 3D environment model based on deep learning algorithm, and predict the track slab paving path based on motion prediction algorithm to obtain the predicted motion trajectory;

[0046] In this step, the three-dimensional environment model and the motion prediction algorithm are used to dynamically predict the movement trajectory of obstacles in advance to detect potential collision risks, helping construction workers to adjust the paving path in a timely manner, thereby effectively avoiding construction accidents.

[0047] To clarify the specific method for obtaining the predicted motion trajectory, step S3 includes S31 to S35, specifically:

[0048] S31: Obtain historical track slab paving data;

[0049] S32: Extract features from the track slab status image information using the convolutional neural network in the deep learning algorithm to obtain deep image features;

[0050] S33: The 3D environment model is segmented using the point cloud segmentation algorithm in deep learning to obtain multi-dimensional information about obstacles;

[0051] In this step, the 3D environment model is segmented according to the point cloud segmentation algorithm to identify obstacle regions and extract multi-dimensional information of obstacles. The point cloud segmentation algorithm is used to extract relevant information of obstacles from complex 3D environments, which solves the problems of inaccurate 3D environment model segmentation and inability to effectively extract obstacle information in traditional methods.

[0052] S34: Input the historical track slab paving data into the long short-term memory network model for training, evaluate the deviation between the prediction results and the actual path through the mean square error loss function, and obtain the optimized model;

[0053] To clarify the specific method for obtaining the optimization model, step S34 includes steps S341 to S344, specifically:

[0054] S341: Divide the historical track slab paving data to obtain a dataset, which includes a training set and a validation set;

[0055] S342: Input the training set into the long short-term memory network model for training. During the training process, the deviation between the prediction result and the actual path is evaluated by the mean squared error loss function to obtain the initial training model.

[0056] In this step, the training set is input into a Long Short-Term Memory (LSTM) network model for training. During training, the mean squared error loss function between the predicted result and the actual path is first calculated using forward propagation to evaluate the model's prediction bias. Subsequently, the weight and bias parameters of the LSTM network model are updated based on the calculated loss value using the backpropagation algorithm, gradually optimizing the model's performance. After multiple iterations of training, the initial training of the model is completed, resulting in the initial trained model.

[0057] S343: Based on the backpropagation algorithm, the weight parameters and bias parameters of the initial training model are iteratively optimized using the loss function value to obtain the parameter-adjusted model;

[0058] In each iteration, the weight and bias parameters of the initial trained model are calculated based on the backpropagation algorithm using the loss function value. The parameters of the initial trained model are then updated using an optimization algorithm, gradually reducing the loss function value. This iterative process is repeated until the loss function value converges or a predetermined number of iterations is reached, resulting in a parameter-adjusted model.

[0059] S344: Calculate the performance index of the parameter adjustment model based on the validation set, and optimize the hyperparameters of the parameter adjustment model using the performance index to obtain an optimized model.

[0060] In this step, the parameter tuning model is evaluated and its performance metrics are calculated based on the validation set. The hyperparameters of the parameter tuning model are then adjusted according to the performance metrics. The model is retrained using the optimized hyperparameters, and its performance is evaluated on the validation set to ensure that the performance metrics reach their optimal level, resulting in an optimized model. The validation set evaluates the performance metrics of the parameter tuning model, and the hyperparameters are adjusted based on these metrics to further optimize the model's performance, improving its generalization ability and prediction accuracy.

[0061] S35: Based on the deep features of the image and the multidimensional information of the obstacle, the data is input into the optimization model for prediction, generating a predicted motion trajectory.

[0062] In this step, the deep features of the image and the multidimensional information of the obstacle are fused together, and the fused features are input into the optimization model to generate a predicted motion trajectory, which is used to improve the reliability of path planning.

[0063] S4: The particle swarm optimization algorithm is improved based on inertia weight and crossover mutation probability, and the parameters of the particle swarm optimization algorithm are dynamically adjusted based on neural network and construction progress data to obtain the particle swarm optimization algorithm.

[0064] In this step, the particle swarm optimization algorithm is improved by using inertia weights and crossover mutation probabilities, and its parameters are dynamically adjusted by combining neural networks and construction progress data. This allows for dynamic and flexible adjustment of the paving path and optimization of algorithm parameters based on the actual construction progress and environment, quickly addressing work stoppages or rework caused by unreasonable path planning.

[0065] To clarify the specific method for obtaining the particle swarm optimization algorithm, step S4 includes S41 to S45, which are as follows:

[0066] S41: Based on the inertia weight and crossover mutation probability, a nonlinear adjustment model for the inertia weight is obtained.

[0067] To clarify the specific method for obtaining the nonlinear adjustment model of inertia weight, step S41 includes S411 to S415, specifically:

[0068] S411: The dynamic adjustment mechanism based on inertia weight calculates the difference between the current fitness value of the particle and the global optimal fitness value to obtain the nonlinear correction factor;

[0069] In this step, a dynamic adjustment mechanism based on inertia weights calculates the current fitness value and the global optimal fitness value for each particle. The difference between the current fitness value and the global optimal fitness value is then calculated to obtain the difference for each particle. A nonlinear correction factor is calculated based on this difference. This difference dynamically reflects the gap between the particle and the global optimal solution.

[0070] S412: Dynamically calibrate the initial inertia weight according to the nonlinear correction factor to obtain the adaptive inertia weight;

[0071] In this step, the adaptive inertia weight is the adaptive inertia weight for each particle. The inertia weight is dynamically adjusted through a nonlinear correction factor to better balance global and local search capabilities.

[0072] S413: The crossover probability is dynamically adapted according to the logistic function, and the adapted crossover probability is weighted and fused with the initial dynamic crossover probability to obtain the dynamic crossover probability;

[0073] In this step, the expression for the crossover probability after adaptation is:

[0074] (1);

[0075] In the above formula (1), The crossover probability after adaptation. For logistic functions, This is a nonlinear correction factor;

[0076] The expression for the dynamic crossover probability is:

[0077] (2);

[0078] In the above formula (2), For dynamic crossover probability, These are weighting coefficients. The crossover probability after adaptation. This represents the initial dynamic crossover probability.

[0079] S414: The mutation probability is dynamically adjusted based on the Gaussian function. The adjusted mutation probability is then weighted and fused with the initial dynamic mutation probability to obtain the dynamic mutation probability.

[0080] In this step, the expression for the dynamic mutation probability is:

[0081] (3);

[0082] In the above formula (3), This represents the dynamic mutation probability. These are weighting coefficients. This is the adjusted mutation probability. This represents the initial dynamic mutation probability.

[0083] S415: Based on the coupling coefficients of the adaptive inertia weight, the dynamic mutation probability, and the dynamic crossover probability, a nonlinear adjustment model for the inertia weight is obtained.

[0084] In this step, the coupling coefficient is used to comprehensively consider the effects of dynamic crossover probability and dynamic mutation probability.

[0085] The expression for the nonlinear adjustment model of the inertial weight is:

[0086] (4);

[0087] In the above formula (4), For the final adaptive inertia weights, As the current inertia weight, The coupling coefficient is... For dynamic crossover probability, This represents the dynamic mutation probability.

[0088] S42: Based on the aforementioned inertial weight nonlinear adjustment model, the velocity update mechanism of the particle swarm algorithm is improved to obtain the improved particle swarm algorithm;

[0089] To clarify the specific acquisition method of the improved particle swarm optimization algorithm, step S42 includes S421 to S423, specifically:

[0090] S421: Based on the nonlinear weights in the aforementioned inertial weight nonlinear adjustment model, the velocity update mechanism of the particle swarm algorithm is improved to obtain an improved velocity update formula;

[0091] In this step, inertial weights are used to nonlinearly adjust the nonlinear weights in the model. Replace the inertia weight in the traditional formula ;

[0092] The improved speed update formula is:

[0093] (5);

[0094] In the above formula (5), For particles In the The speed of the next iteration For non-linear weights, For particles In the The speed of the next iteration and As a learning factor, and It is a random number. For particles The optimal position of an individual For particles In the The position of the next iteration. This is the globally optimal position.

[0095] S422: Calculate the new velocity of the particle according to the improved velocity update formula, update the particle's position with the new velocity, and obtain the updated particle position;

[0096] In this step, the particle position update formula is:

[0097] (6);

[0098] In the above formula (6), For particles In the The position of the next iteration. For particles In the The position of the next iteration. For particles In the The speed of each iteration.

[0099] S423: Perform a dimensional cross operation on the information between particle swarms based on the updated particle positions to obtain an improved particle swarm algorithm.

[0100] In this step, for each particle, the position information of other particles in certain dimensions is selected and cross-referenced. The particle's position is then updated based on the cross-reference operation. This dimensional cross-reference operation enhances information exchange between the particle swarm, thereby improving the algorithm's global search capability and convergence speed.

[0101] The crossover operation expression is:

[0102] (7);

[0103] In the above formula (7), For particles In the The iteration of the ... The position of the dimension For particles In the The iteration of the ... The position of the dimension Cross factor For particles In the The iteration of the ... The position of the dimension.

[0104] S43: Divide the construction progress data into different map regions according to the convolutional neural network and extract key feature information to obtain optimized construction progress feature data;

[0105] In this step, the images in the construction progress data are analyzed using the convolutional neural network and divided into different image regions. Each image region represents a specific stage or feature area of ​​the construction progress. Then, key feature information is extracted for each image region. This key feature information reflects the core elements of the construction progress. Finally, by integrating the key feature information, optimized construction progress feature data is obtained, thereby providing a more accurate basis for the management and optimization of the construction progress.

[0106] S44: Input the optimized construction progress feature data into the improved particle swarm algorithm, and adjust the parameters of the improved particle swarm algorithm in reverse by calculating the error between the predicted progress and the actual progress to obtain the construction progress prediction model.

[0107] This step improves the particle swarm optimization algorithm to optimize parameters for more accurate construction progress prediction, thereby enhancing the accuracy of the prediction model. This solves the problems of low accuracy in construction progress prediction and the inability to adjust parameters in real time in traditional methods.

[0108] S45: Evaluate the error of the construction progress prediction model. When the error of the construction progress prediction model meets the preset reduction threshold, output the particle swarm optimization algorithm.

[0109] In each iteration, the error of the construction progress prediction model is calculated. If the error does not meet the preset threshold, the parameters of the particle swarm optimization algorithm are adjusted, and optimization continues. When the error meets the preset threshold, the iteration stops, and the optimized particle swarm optimization algorithm is output.

[0110] S5: Input the predicted motion trajectory and preset path constraints into the particle swarm optimization algorithm to solve for the paving motion path;

[0111] To clarify the specific method for obtaining the paving movement path, step S5 includes S51 to S54, specifically:

[0112] S51: Input the predicted motion trajectory into the particle swarm optimization algorithm to update the position and velocity of the particles and obtain a new motion path;

[0113] In this step, the predicted motion trajectory is input into the particle swarm optimization algorithm to initialize the particle swarm, the initial position and velocity of each particle are set, the improved velocity update formula in the particle swarm optimization algorithm is used to calculate the new velocity of each particle, and the position of each particle is updated according to the new velocity to obtain the new motion path.

[0114] S52: Calculate the fitness value of each particle in the particle swarm based on the current path of the fitness function, evaluate the fitness value through preset path constraints, and obtain the evaluation result;

[0115] S53: When the evaluation result does not meet the path constraint conditions, the particles that do not meet the path constraint conditions are penalized by the fitness function to obtain the corrected fitness value.

[0116] In this step, a penalty term is calculated for particles that do not meet the path constraints. The penalty item Add it to the fitness value to obtain the modified fitness value.

[0117] The expression for the corrected fitness value is:

[0118] (8);

[0119] In the above formula (8), To correct the fitness value, For fitness value, This is a penalty item.

[0120] S54: In each iteration, the new motion path is judged based on the modified fitness value to determine whether it meets the preset convergence condition. When the convergence condition is met, the motion path corresponding to the global optimal position of the particle swarm is finally used as the paving motion path.

[0121] In each iteration, the global optimum position is first updated based on the corrected fitness value. Then, it is checked whether the corrected fitness value meets the preset convergence condition. If the convergence condition is met, the iteration stops, and the motion path corresponding to the current global optimum position is taken as the final paving motion path. If the convergence condition is not met, the next iteration continues, and the above process is repeated until the convergence condition is met.

[0122] S6: Perform path paving and monitor according to the paving movement path. When the monitoring result meets the preset deviation threshold, the paving ends.

[0123] In this step, the track laying equipment is controlled to lay track slabs according to the paving motion path and the paving status is monitored in real time. When the monitoring result exceeds the preset deviation threshold, the motion parameters of the laying equipment are adjusted until the requirements are met.

[0124] Example 2:

[0125] This embodiment provides a target paving path planning device based on deep learning and image recognition, the device comprising:

[0126] The acquisition module is used to acquire track slab status image information and environmental dynamic image information, wherein the environmental dynamic image information includes complete environmental images from at least two angles;

[0127] The modeling module is used to perform geometric modeling based on the dynamic image information of the environment, combined with the movement trajectory of obstacles and the deformation of the track slab, to obtain a three-dimensional environment model;

[0128] To clarify the specific methods for obtaining the modeling module, the following are included:

[0129] The detection unit is used to detect obstacles based on the dynamic environmental image information, and to determine the coordinates of the obstacle target by matching the detection results with the feature points of the preset obstacle image.

[0130] The transformation unit is used to transform the coordinates of the obstacle target to the global three-dimensional coordinate system according to the coordinate transformation algorithm, and to construct a global three-dimensional model;

[0131] The correction unit is used to perform geometric correction on the global three-dimensional model based on the obstacle's motion trajectory and the track slab deformation to obtain an optimized three-dimensional point cloud model.

[0132] The processing unit is used to perform meshing processing on the optimized 3D point cloud model to generate a triangular mesh model, and to generate a texture map by mapping the texture information in the coordinates of the obstacle target onto the triangular mesh model.

[0133] The update unit is used to dynamically update the optimized 3D point cloud model based on the incremental modeling algorithm, using texture mapping and real-time obstacle detection data, to obtain a 3D environment model.

[0134] The analysis module is used to analyze the track slab status image information and 3D environment model based on deep learning algorithms, and to predict the track slab paving path based on motion prediction algorithms to obtain the predicted motion trajectory.

[0135] To clarify the specific methods for obtaining the analysis module, the following are included:

[0136] The acquisition unit is used to acquire historical track slab paving data;

[0137] The extraction unit is used to extract features from the track slab state image information based on the convolutional neural network in the deep learning algorithm to obtain deep image features;

[0138] The segmentation unit is used to segment the 3D environment model according to the point cloud segmentation algorithm in the deep learning algorithm to obtain multi-dimensional information of obstacles;

[0139] The training unit is used to input the historical track slab paving data into the long short-term memory network model for training, and to evaluate the deviation between the prediction results and the actual path through the mean square error loss function to obtain an optimized model.

[0140] To clarify the specific methods for obtaining training units, the following are included:

[0141] The historical track slab paving data is divided into sub-units to obtain a dataset, which includes a training set and a validation set.

[0142] The training subunit is used to input the training set into the long short-term memory network model for training. During the training process, the deviation between the prediction result and the actual path is evaluated by the mean squared error loss function to obtain the initial training model.

[0143] The optimization subunit is used to iteratively optimize the weight parameters and bias parameters of the initial training model based on the backpropagation algorithm and the loss function value to obtain the parameter-adjusted model.

[0144] The computational subunit is used to calculate performance indicators for the parameter adjustment model based on the validation set, and to optimize the hyperparameters of the parameter adjustment model using the performance indicators to obtain an optimized model.

[0145] The prediction unit is used to predict motion trajectories by inputting the deep features of the image and the multidimensional information of the obstacles into the optimization model.

[0146] An improvement module is used to improve the particle swarm algorithm based on inertia weights and crossover mutation probabilities, and to dynamically adjust the parameters of the particle swarm algorithm based on neural networks and construction progress data, so as to obtain an optimized particle swarm algorithm.

[0147] The solution module is used to input the predicted motion trajectory and preset path constraints into the particle swarm optimization algorithm to solve for the paving motion path;

[0148] The monitoring module is used to perform path paving and monitor according to the paving movement path. When the monitoring result meets the preset deviation threshold, the paving ends.

[0149] It should be noted that the specific manner in which each module performs its operation in the apparatus described in the above embodiments has been described in detail in the embodiments of the method, and will not be elaborated here.

[0150] Example 3:

[0151] Corresponding to the above method embodiments, this embodiment also provides a target paving path planning device based on deep learning and image recognition. The target paving path planning device based on deep learning and image recognition described below can be referred to in correspondence with the target paving path planning method based on deep learning and image recognition described above.

[0152] Figure 2 This is a block diagram illustrating a target paving path planning device 800 based on deep learning and image recognition, according to an exemplary embodiment. Figure 2 As shown, the target paving path planning device 800 based on deep learning and image recognition may include: a processor 801 and a memory 802. The target paving path planning device 800 may also include one or more of a multimedia component 803, an I / O interface 804, and a communication component 805.

[0153] The processor 801 controls the overall operation of the deep learning and image recognition-based target paving path planning device 800 to complete all or part of the steps in the aforementioned deep learning and image recognition-based target paving path planning method. The memory 802 stores various types of data to support the operation of the deep learning and image recognition-based target paving path planning device 800. This data may include, for example, instructions for any application or method operating on the deep learning and image recognition-based target paving path planning device 800, as well as application-related data, such as contact data, sent and received messages, images, audio, video, etc. The memory 802 can be implemented using any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 802 or transmitted via the communication component 805. The audio component also includes at least one speaker for outputting audio signals. I / O interface 804 provides an interface between processor 801 and other interface modules, such as keyboards, mice, and buttons. These buttons can be virtual or physical. Communication component 805 is used for wired or wireless communication between the deep learning and image recognition-based target paving path planning device 800 and other devices. Wireless communication includes Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination thereof. Therefore, the corresponding communication component 805 may include a Wi-Fi module, a Bluetooth module, and an NFC module.

[0154] In an exemplary embodiment, the target paving path planning device 800 based on deep learning and image recognition may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the target paving path planning method based on deep learning and image recognition described above.

[0155] Example 4:

[0156] Corresponding to the above method embodiments, this embodiment also provides a medium. The medium described below can be referred to in relation to the target paving path planning method based on deep learning and image recognition described above.

[0157] A medium storing a computer program, which, when executed by a processor, implements the steps of the target paving path planning method based on deep learning and image recognition described in the above method embodiments.

[0158] The medium can specifically be any medium capable of storing program code, such as a USB flash drive, external hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0159] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0160] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A target paving path planning method based on deep learning and image recognition, characterized in that, include: Acquire track slab status image information and environmental dynamic image information, wherein the environmental dynamic image information includes complete environmental images from at least two angles; Based on the dynamic environmental image information, and combined with the obstacle movement trajectory and track slab deformation, a three-dimensional environment model is obtained through geometric modeling. The specific methods for obtaining the three-dimensional environment model include: Obstacle detection is performed based on the dynamic environmental image information, and the coordinates of the obstacle target are determined by matching the detection results with the feature points of the preset obstacle image. The coordinates of the obstacle target are transformed into the global three-dimensional coordinate system using a coordinate transformation algorithm, and a global three-dimensional model is constructed. Based on the obstacle's motion trajectory and the track slab's deformation, the global 3D model is geometrically corrected to obtain an optimized 3D point cloud model. The optimized 3D point cloud model is meshed to generate a triangular mesh model. Texture maps are generated by mapping the texture information in the coordinates of the obstacle target onto the triangular mesh model. Based on the incremental modeling algorithm, the optimized 3D point cloud model is dynamically updated using texture mapping and real-time obstacle detection data to obtain a 3D environment model; The track slab status image information and 3D environment model are analyzed based on deep learning algorithms, and the track slab paving path is predicted based on motion prediction algorithms to obtain the predicted motion trajectory. The particle swarm optimization algorithm is improved based on inertia weight and crossover mutation probability, and the parameters of the particle swarm optimization algorithm are dynamically adjusted based on neural network and construction progress data to obtain the particle swarm optimization algorithm. The improvements to the particle swarm optimization algorithm include: A nonlinear adjustment model for inertia weights is obtained by constructing a nonlinear model based on inertia weights and crossover mutation probabilities. The velocity update mechanism of the particle swarm algorithm is improved based on the aforementioned inertial weight nonlinear adjustment model, resulting in an improved particle swarm algorithm. The predicted motion trajectory and preset path constraints are input into the particle swarm optimization algorithm to solve for the paving motion path. The paving is carried out according to the paving movement path and monitored. When the monitoring result meets the preset deviation threshold, the paving ends.

2. The target paving path planning method based on deep learning and image recognition according to claim 1, characterized in that, based on the deep learning algorithm, the track slab state image information and three-dimensional environment model are analyzed, and the paving path of the track slab is predicted based on the motion prediction algorithm to obtain the predicted motion trajectory, including: Obtain historical track slab paving data; The deep features of the image are obtained by extracting features from the track slab status image information using a convolutional neural network in a deep learning algorithm. The 3D environment model is segmented using the point cloud segmentation algorithm in deep learning to obtain multi-dimensional information about obstacles; The historical track slab paving data is input into a long short-term memory network model for training. The deviation between the predicted results and the actual path is evaluated using the mean square error loss function to obtain an optimized model. The deep features of the image and the multidimensional information of the obstacle are input into the optimization model for prediction, generating a predicted motion trajectory.

3. The target paving path planning method based on deep learning and image recognition according to claim 2, characterized in that the historical track slab paving data is input into a long short-term memory network model for training, and the deviation between the predicted result and the actual path is evaluated through the mean square error loss function to obtain an optimized model, including: The historical track slab paving data is divided to obtain a dataset, which includes a training set and a validation set. The training set is input into the Long Short-Term Memory network model for training. During the training process, the deviation between the prediction result and the actual path is evaluated by the mean squared error loss function to obtain the initial training model. Based on the backpropagation algorithm, the weight parameters and bias parameters of the initial training model are iteratively optimized using the loss function value to obtain the parameter-adjusted model; The performance index of the parameter adjustment model is calculated based on the validation set, and the hyperparameters of the parameter adjustment model are optimized using the performance index to obtain an optimized model.

4. The target paving path planning method based on deep learning and image recognition according to claim 1, characterized in that, based on neural networks and construction progress data, the parameters of the particle swarm optimization algorithm are dynamically adjusted to obtain the particle swarm optimization algorithm, including: The construction progress data is divided into different map regions by a convolutional neural network and key feature information is extracted to obtain optimized construction progress feature data. The optimized construction progress feature data is input into the improved particle swarm algorithm. The parameters of the improved particle swarm algorithm are adjusted in reverse by calculating the error between the predicted progress and the actual progress to obtain the construction progress prediction model. The error of the construction progress prediction model is evaluated. When the error of the construction progress prediction model meets the preset reduction threshold, the particle swarm optimization algorithm is output.

5. The target paving path planning method based on deep learning and image recognition according to claim 4, characterized in that, A nonlinear adjustment model for inertia weights is obtained by constructing a nonlinear model based on inertia weights and crossover mutation probabilities, including: The dynamic adjustment mechanism based on inertia weights calculates the difference between the current fitness value of a particle and the global optimal fitness value, and obtains the nonlinear correction factor. The initial inertia weight is dynamically calibrated based on the nonlinear correction factor to obtain the adaptive inertia weight; The crossover probability is dynamically adapted based on the logistic function, and the adapted crossover probability is weighted and fused with the initial dynamic crossover probability to obtain the dynamic crossover probability. The mutation probability is dynamically adjusted based on the Gaussian function, and the dynamic mutation probability is obtained by weighted fusion of the adjusted mutation probability and the initial dynamic mutation probability. A nonlinear adjustment model for the inertia weight is obtained by constructing a model based on the coupling coefficients of the adaptive inertia weight, the dynamic mutation probability, and the dynamic crossover probability.

6. A target paving path planning device based on deep learning and image recognition, characterized in that, include: The acquisition module is used to acquire track slab status image information and environmental dynamic image information, wherein the environmental dynamic image information includes complete environmental images from at least two angles; The modeling module is used to perform geometric modeling based on the dynamic image information of the environment, combined with the movement trajectory of obstacles and the deformation of the track slab, to obtain a three-dimensional environment model; The modeling module includes: The detection unit is used to detect obstacles based on the dynamic environmental image information, and to determine the coordinates of the obstacle target by matching the detection results with the feature points of the preset obstacle image. The transformation unit is used to transform the coordinates of the obstacle target to the global three-dimensional coordinate system according to the coordinate transformation algorithm, and to construct a global three-dimensional model; The correction unit is used to perform geometric correction on the global three-dimensional model based on the obstacle's motion trajectory and the track slab deformation to obtain an optimized three-dimensional point cloud model. The processing unit is used to perform meshing processing on the optimized 3D point cloud model to generate a triangular mesh model, and to generate a texture map by mapping the texture information in the coordinates of the obstacle target onto the triangular mesh model. The update unit is used to dynamically update the optimized 3D point cloud model based on the incremental modeling algorithm, using texture mapping and real-time obstacle detection data, to obtain a 3D environment model. The analysis module is used to analyze the track slab status image information and 3D environment model based on deep learning algorithms, and to predict the track slab paving path based on motion prediction algorithms to obtain the predicted motion trajectory. An improvement module is used to improve the particle swarm algorithm based on inertia weights and crossover mutation probabilities, and to dynamically adjust the parameters of the particle swarm algorithm based on neural networks and construction progress data, so as to obtain an optimized particle swarm algorithm. The particle swarm optimization algorithm in the improved module includes: A nonlinear adjustment model for inertia weights is obtained by constructing a nonlinear model based on inertia weights and crossover mutation probabilities. The velocity update mechanism of the particle swarm algorithm is improved based on the aforementioned inertial weight nonlinear adjustment model, resulting in an improved particle swarm algorithm. The solution module is used to input the predicted motion trajectory and preset path constraints into the particle swarm optimization algorithm to solve for the paving motion path; The monitoring module is used to perform path paving and monitor according to the paving movement path. When the monitoring result meets the preset deviation threshold, the paving ends.

7. The target paving path planning device based on deep learning and image recognition according to claim 6, characterized in that, The analysis module includes: The acquisition unit is used to acquire historical track slab paving data; The extraction unit is used to extract features from the track slab state image information based on the convolutional neural network in the deep learning algorithm to obtain deep image features; The segmentation unit is used to segment the 3D environment model according to the point cloud segmentation algorithm in the deep learning algorithm to obtain multi-dimensional information of obstacles; The training unit is used to input the historical track slab paving data into the long short-term memory network model for training, and to evaluate the deviation between the prediction results and the actual path through the mean square error loss function to obtain an optimized model. The prediction unit is used to predict motion trajectories by inputting the deep features of the image and the multidimensional information of the obstacles into the optimization model.

8. The target paving path planning device based on deep learning and image recognition according to claim 7, characterized in that, The training unit includes: The historical track slab paving data is divided into sub-units to obtain a dataset, which includes a training set and a validation set. The training subunit is used to input the training set into the long short-term memory network model for training. During the training process, the deviation between the prediction result and the actual path is evaluated by the mean squared error loss function to obtain the initial training model. The optimization subunit is used to iteratively optimize the weight parameters and bias parameters of the initial training model based on the backpropagation algorithm and the loss function value to obtain the parameter-adjusted model. The computational subunit is used to calculate performance indicators for the parameter adjustment model based on the validation set, and to optimize the hyperparameters of the parameter adjustment model using the performance indicators to obtain an optimized model.

Images