Dynamic target detection method, camera pose solving method, AGV map construction method, equipment and medium

Abstract

The invention discloses a dynamic target detection method, a camera pose solving method, an AGV map construction method, equipment and a medium, and relates to the field of visual positioning. The dynamic target detection method comprises the following steps: constructing a lightweight YOLOv5s network model; training and testing the lightweight YOLOv5s network model by using the training set and the test set to obtain a trained lightweight YOLOv5s network model; and carrying out dynamic target detection based on the trained lightweight YOLOv5s network model. According to the invention, the efficiency of dynamic target detection is improved.

Description

Technical Field

[0001] The present application relates to the field of visual positioning, and in particular to a dynamic target detection method, a camera posture solving method, an AGV map construction method, equipment and medium. Background Art

[0002] ORB-SLAM2 was published by Raúl Mur-Artal and Juan D.Tardós in IEEE Transactions on Robotics in 2017 and has become one of the classic algorithms in VSLAM based on feature point method. As a complete open source SLAM system, it supports monocular, binocular and RGB-D cameras. ORB-SLAM2 includes four threads, namely tracking thread, local mapping thread, loop closure detection thread, and a global BA optimization (Full BA) thread to optimize all keyframes and their map points. The fourth BA thread is only enabled after the loop is detected, and most of the time only the first three threads are run in parallel. The algorithm can run in real time indoors and outdoors, with extremely high positioning accuracy, up to centimeter level, and has achieved satisfactory performance. However, this algorithm is only applicable to static environments. Due to the inevitable presence of dynamic objects in real environments, such as walking people and moving vehicles, the camera pose state estimation and system positioning accuracy will be reduced.

[0003] In 2018, Bescos et al. proposed the DynaSLAM algorithm, which detects and removes dynamic objects through the semantic segmentation network MASK R-CNN and multi-view geometry methods. This method can not only remove dynamic objects, but also restore the background occluded by dynamic objects. Yu et al. proposed the DS-SLAM algorithm, which puts a real-time semantic segmentation network SegNet into an independent thread and filters out the dynamic part of the scene through motion consistency detection. However, since both DynaSLAM and DS-SLAM algorithms obtain semantic information through semantic segmentation and remove feature points on dynamic targets, although they can greatly improve the positioning accuracy of the system in dynamic scenes, such algorithms are usually time-consuming and difficult to ensure real-time performance. Summary of the invention

[0004] The purpose of this application is to provide a dynamic target detection method, a camera pose solving method, an AGV map construction method, a device and a medium, which can improve the efficiency of dynamic target detection.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a dynamic target detection method, the dynamic target detection method comprising:

[0007] Constructing a lightweight YOLOv5s network model; the lightweight YOLOv5s network model uses a lightweight ShuffleNetv2 network to replace the backbone feature extraction network of the YOLOv5s network model, and introduces a Stem module into the input layer of the backbone feature extraction network of the YOLOv5s network model;

[0008] The lightweight YOLOv5s network model is trained and tested using a training set and a test set to obtain a trained lightweight YOLOv5s network model; the training set and the test set both include image sequences of dynamic scenes with dynamic target markers;

[0009] Dynamic target detection is performed based on the trained lightweight YOLOv5s network model.

[0010] Optionally, the lightweight ShuffleNetv2 network includes: 3 basic units and 3 downsampling modules, the 3 basic units are respectively a first basic unit, a second basic unit and a third basic unit, the 3 downsampling modules are respectively: a first downsampling module, a second downsampling module and a third downsampling module, the first downsampling module and the third downsampling module each include 3 downsampling units connected in sequence, and the second downsampling module includes 7 downsampling units connected in sequence;

[0011] The Stem module, the first basic unit, the first down-sampling module, the second basic unit, the second down-sampling module, the third basic unit and the third down-sampling module are connected in sequence.

[0012] Optionally, the basic unit includes a channel separation module, a first 3-layer convolution module, a first splicing module and a first channel shuffling module;

[0013] The channel separation module is connected to the input channel of the basic unit, and the channel separation module is used to divide the input channel into two first branches, wherein one first branch is directly connected to the first splicing module, and the other first branch is connected to the first splicing module through the first 3-layer convolution module;

[0014] The first splicing module is used for splicing two first branches;

[0015] The first splicing module is connected to the first channel shuffling module; the first channel shuffling module is used to disrupt the order of output channels of the basic unit.

[0016] Optionally, the downsampling unit includes: a first 2-layer convolution module, a second 3-layer convolution module, a second splicing module and a second channel shuffling module;

[0017] The first 2-layer convolution module and the second 3-layer convolution module are used as two second branches, one end of the two second branches is connected to the input channel of the downsampling unit, the other end of the two second branches is connected to the second splicing module, and the second splicing module is connected to the second channel shuffling module;

[0018] The second splicing module is used for splicing two second branches;

[0019] The second channel shuffling module is used to disrupt the order of the output channels of the down-sampling unit.

[0020] Optionally, the Stem module includes: a second 2-layer convolution module, a maximum pooling layer, a third splicing module and a 1-layer convolution module;

[0021] The second 2-layer convolution module and the maximum pooling layer serve as two third branches, one end of the two third branches are connected to the input channel of the Stem module, the other end of the two third branches are connected to the third splicing module, and the third splicing module is connected to the 1-layer convolution module.

[0022] Optionally, the lightweight YOLOv5s network model is trained and tested using the training set and the test set to obtain a trained lightweight YOLOv5s network model, specifically including:

[0023] The Pascal VOC 2007 dataset is used as the training set;

[0024] Using the training set to train the lightweight YOLOv5s network model to obtain a trained lightweight YOLOv5s network model;

[0025] The dynamic sequence freiburg3_walking_xyz in the TUM RGB-D dataset is used as the test set;

[0026] Use the test set to test the trained lightweight YOLOv5s network model;

[0027] When the test fails, return to the step of "training the lightweight YOLOv5s network model using the training set to obtain a trained lightweight YOLOv5s network model";

[0028] When the test passes, the trained lightweight YOLOv5s network model is output as the trained lightweight YOLOv5s network model.

[0029] In a second aspect, the present application provides a camera pose solving method based on dynamic target detection, the camera pose solving method comprising:

[0030] Get two adjacent frames of RGB-D images taken by the RGB-D camera;

[0031] Extracting ORB feature points of each frame of RGB-D image; in a dynamic scene, the ORB feature points include dynamic feature points and static feature points;

[0032] Using the above dynamic target detection method, the dynamic target in each frame of RGB-D image is detected;

[0033] Based on the epipolar geometry constraint condition, the ORB feature points of the dynamic target in each frame of RGB-D image are eliminated to obtain the static ORB feature points of each frame of RGB-D image.

[0034] Based on the static ORB feature points of each RGB-D image frame, the position and posture of the RGB-D camera when obtaining each RGB-D image frame is solved.

[0035] In a third aspect, the present application provides an AGV map construction method based on dynamic target detection, the AGV map construction method comprising the following steps:

[0036] Control the AGV to move one circle in the target scene, and use the RGB-D camera set on the AGV to collect the RGB-D image of the target scene during the movement to obtain a multi-frame scene RGB-D image of the target scene;

[0037] The above-mentioned camera pose solving method is used to solve the pose of the RGB-D camera when acquiring each frame of the scene RGB-D image;

[0038] According to multiple scene RGB-D images and the pose of the RGB-D camera when acquiring each frame of the scene RGB-D image, the SLAM mapping package is used to build the AGV map of the target scene.

[0039] In a fourth aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned dynamic target detection method, or camera pose solving method, or AGV map construction method.

[0040] In a fifth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned dynamic target detection method, or camera pose solving method, or AGV map construction method.

[0041] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0042] The present application provides a dynamic target detection method, a camera pose solving method, an AGV map construction method, a device and a medium. The dynamic target detection method uses a lightweight YOLOv5s network model for dynamic target detection. The lightweight YOLOv5s network model uses a lightweight network ShuffleNetv2 to replace the backbone feature extraction network of YOLOv5s, thereby reducing model parameters, speeding up the detection speed, and greatly improving the real-time performance of the system. At the same time, in order to make up for the loss of feature extraction capability caused by the smaller network, the Stem module is introduced in the first layer of the network input to enhance the network feature extraction capability. Compared with the YOLOv5s network model, the lightweight YOLOv5s network model has a 87.67% reduction in parameters and an 88.89% increase in inference speed on the CPU, which can improve the efficiency of dynamic target detection.

[0043] While extracting ORB feature points, this camera pose solution method detects dynamic targets in the scene through a lightweight YOLOv5s network model, and uses epipolar geometry constraints to remove feature points on dynamic objects. Finally, the feature points on static objects are used to solve the camera pose, thereby improving the accuracy of camera pose solution.

[0044] The camera pose solution method was transplanted to the Jeston TX2 development board. By communicating with the RGB-D camera, a dense map of the laboratory in a dynamic scene was constructed to ensure the real-time detection of dynamic targets and improve the accuracy of AGV map construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 This is an application environment diagram of a dynamic target detection method provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of a flow chart of a dynamic target detection method provided in one embodiment of the present application;

[0048] Figure 3 A schematic diagram of the structure of a basic unit provided in one embodiment of the present application;

[0049] Figure 4 A schematic diagram of the structure of a downsampling unit provided in one embodiment of the present application;

[0050] Figure 5A schematic diagram of the structure of a Stem module provided in one embodiment of the present application;

[0051] Figure 6 A schematic diagram of the structure of a lightweight YOLOv5s network model provided in an embodiment of the present application;

[0052] Figure 7 A schematic diagram of test results of a lightweight YOLOv5s network model provided in an embodiment of the present application;

[0053] Figure 8 A schematic diagram of a camera pose solving method provided in one embodiment of the present application;

[0054] Fig. 9 A schematic diagram of epipolar geometry constraints provided in one embodiment of the present application;

[0055] Fig.10 A result graph of removing dynamic feature points provided by an embodiment of the present application;

[0056] Fig.11 A schematic diagram of a map construction process provided by an embodiment of the present application;

[0057] Fig.12 A diagram showing the laboratory map construction result provided in an embodiment of the present application;

[0058] Fig.13 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0061] The dynamic target detection method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, or it can be integrated on the server 104, or it can be placed on the cloud or other servers. The terminal 102 can send the image of the dynamic scene to the server 104. After the server 104 receives the image of the dynamic scene, the server 104 uses the trained lightweight YOLOv5s network model to perform dynamic target detection for the image of the dynamic scene. The server 104 can feed back the obtained detection results to the terminal 102. In addition, in some embodiments, the process of performing dynamic target detection using the trained lightweight YOLOv5s network model can also be implemented separately by the server 104 or the terminal 102, such as the terminal 102 can directly process the image of the dynamic scene, or the server 104 can obtain the image of the dynamic scene from the data storage system, and perform dynamic target detection on the image of the dynamic scene.

[0062] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0063] In an exemplary embodiment, Figure 2 As shown, a dynamic target detection method is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate, including the following steps 201 to 203. Among them:

[0064] Step 201, construct a lightweight YOLOv5s network model; the lightweight YOLOv5s network model uses a lightweight ShuffleNetv2 network to replace the backbone feature extraction network of the YOLOv5s network model, and is obtained by introducing a Stem module into the input layer of the backbone feature extraction network of the YOLOv5s network model.

[0065] Step 202, training and testing the lightweight YOLOv5s network model using a training set and a test set to obtain a trained lightweight YOLOv5s network model; the training set and the test set both include image sequences of dynamic scenes with dynamic target tags.

[0066] Step 203: Perform dynamic target detection based on the trained lightweight YOLOv5s network model.

[0067] In another exemplary embodiment of the present application, the improvement method of the YOLOv5s network model involved in the above step 201 specifically includes:

[0068] Replacing the backbone feature extraction network of YOLOv5s with the lightweight network ShuffleNetv2 reduces model parameters, speeds up detection, and greatly improves the real-time performance of the system.

[0069] The ShuffleNetv2 network consists of basic units and downsampling units.

[0070] like Figure 3 As shown, the basic unit includes a channel separation module, a first 3-layer convolution module, a first splicing module and a first channel shuffling module; the channel separation module is connected to the input channel of the basic unit, and the channel separation module is used to divide the input channel into two first branches, wherein one first branch is directly connected to the first splicing module, and the other first branch is connected to the first splicing module through the first 3-layer convolution module; the first splicing module is used to splice the two first branches; the first splicing module is connected to the first channel shuffling module; the first channel shuffling module is used to disrupt the order of the output channels of the basic unit. When the unit step size is 1, as shown in FIG. Figure 3 As shown in the figure, the unit uses channel splitting to divide the input channel into two branches. One branch is connected through a "shortcut", and the other branch is deepened through 3 layers of convolution. Finally, the left and right branches are concatenated together.

[0071] like Figure 4 As shown, the downsampling unit includes: a first 2-layer convolution module, a second 3-layer convolution module, a second splicing module and a second channel shuffling module; the first 2-layer convolution module and the second 3-layer convolution module are used as two second branches, one end of the two second branches are connected to the input channel of the downsampling unit, the other end of the two second branches are connected to the second splicing module, and the second splicing module is connected to the second channel shuffling module; the second splicing module is used to splice the two second branches; the second channel shuffling module is used to disrupt the order of the output channels of the downsampling unit. When the unit step size is 2, as shown in FIG. Figure 4As shown in the figure, the unit removes the channel separation operation and instead performs convolution operations on the two branches separately, reducing the width and height of the input features to achieve spatial downsampling. To compensate for the sudden reduction in parameters, the output results on the two branches are connected to double the channels. At the end of the two units, the order of the output channels needs to be disrupted by channel shuffle to ensure the exchange of feature information between the two branches. Given that ShuffleNetv2 has achieved good performance in the trade-off between speed and accuracy, this application replaces the backbone network of YOLOv5s with Figure 3 The basic unit and Figure 4 A lightweight ShuffleNetv2 network composed of downsampling units in .

[0072] The Stem module is introduced into the input layer of the backbone feature extraction network of the YOLOv5s network model to compensate for the loss of feature extraction capability caused by the smaller network. The Stem module is introduced to improve the network feature extraction capability.

[0073] The Stem module adopts the idea of ​​cross-layer information fusion. It first performs fast dimensionality reduction through the convolution layer, and then divides the downsampling step of the feature map into two paths: the first path is implemented through two convolution layers, and the second path is implemented through the maximum pooling layer (max poll). The results of the two paths are then concatenated to increase the number of channels of the output feature map, and finally the feature map is output through the convolution operation. The most significant advantage of the Stem module is that it introduces the idea of ​​the bottleneck layer. In the first path, the number of channels of the feature map is first reduced, and then downsampling is performed. The second path first performs the maximum pooling (max poll) operation on the feature map, and then concatenates the results of the first path to pass the valid information of the input feature map, ensuring that the final result still has enough semantic information, so that the network model can retain most of the valid information of the original feature map. Since the Stem module can simplify the model and improve the overall performance of the model without increasing too much computational cost, the Stem module is introduced in the first layer of YOLOv5s input to improve the network feature extraction capability. The Stem module is as follows Figure 5 As shown, it includes: a second 2-layer convolution module, a maximum pooling layer, a third splicing module and a 1-layer convolution module; the second 2-layer convolution module and the maximum pooling layer serve as two third branches, one end of the two third branches are connected to the input channel of the Stem module, the other end of the two third branches are connected to the third splicing module, and the third splicing module is connected to the 1-layer convolution module.

[0074] The structure of the lightweight YOLOv5s network model improved based on the YOLOv5s network model in this application is as follows Figure 6As shown. The lightweight ShuffleNetv2 network includes: 3 basic units and 3 downsampling modules, the 3 basic units are respectively the first basic unit, the second basic unit and the third basic unit, the 3 downsampling modules are respectively: the first downsampling module, the second downsampling module and the third downsampling module, the first downsampling module and the third downsampling module each include 3 downsampling units connected in sequence, the second downsampling module includes 7 downsampling units connected in sequence; the Stem module, the first basic unit, the first downsampling module, the second basic unit, the second downsampling module, the third basic unit and the third downsampling module are connected in sequence.

[0075] In another exemplary embodiment of the present application, the model training and testing process involved in the above step 202 specifically includes:

[0076] The lightweight YOLOv5s network model is trained using the PascalVOC 2007 dataset. The training initialization parameters are: batch_size is 64, the learning rate is initialized to 0.01, the cosine annealing hyperparameter is 0.2, the learning rate momentum is 0.937, and the weight decay coefficient is 0.0005. The network model is trained for 500 epochs. Compared with the YOLOv5s network, the number of parameters is reduced by 87.67%, and the inference speed on the CPU is increased by 88.89%.

[0077] In this example, the high-dynamic sequence freiburg3_walking_xyz in the TUM RGB-D public dataset is selected as the test dataset, and the trained lightweight YOLOv5s network model is used to perform target detection on the dataset. The detection results are as follows: Figure 7 As shown in (a) and (b) in the dataset, since the dynamic target in the scene is a person, the lightweight YOLOv5s network model is set to detect only the category of people, and the category parameter is set to 14. The detection results are shown in Figure 7 As shown in (c) and (d) in . Figure 7 It can be seen that the category of dynamic objects can be accurately selected, which verifies the feasibility of the improved network.

[0078] In another exemplary embodiment of the present application, the application method of the YOLOv5s network model involved in the above step 203 is, for example, to integrate the lightweight YOLOv5s network model into the ORB-SLAM2 visual odometry, and while the visual odometry extracts ORB feature points, the dynamic targets in the scene are detected through the lightweight YOLOv5s network model.

[0079] In an exemplary embodiment, Figure 8As shown, a camera pose solution method based on dynamic target detection is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together, such as Figure 8 As shown, in the embodiment of the present application, the camera posture solving method includes the following steps:

[0080] Get two adjacent frames of RGB-D images taken by an RGB-D camera.

[0081] Extract the ORB feature points of each frame of RGB-D image; in a dynamic scene, the ORB feature points include dynamic feature points and static feature points.

[0082] The above dynamic target detection method is used to detect dynamic targets in each frame of RGB-D image.

[0083] Based on the epipolar geometry constraint, the ORB feature points of dynamic targets in each frame of RGB-D image are eliminated to obtain the static ORB feature points of each frame of RGB-D image.

[0084] Based on the static ORB feature points of each RGB-D image frame, the position and posture of the RGB-D camera when obtaining each RGB-D image frame is solved.

[0085] like Figure 8 As shown in the camera pose solution method, the lightweight YOLOv5s network model (i.e. Figure 8 The YOLOv5s-light in the ORB-SLAM2 is integrated into the visual odometry. The dynamic targets in the scene are detected through the YOLOv5s-light network. While the ORB-SLAM2 visual odometry extracts ORB feature points, the feature points on dynamic objects are eliminated through the epipolar geometry constraints, and only the feature points on static objects are used to solve the camera pose.

[0086] In another exemplary embodiment of the present application, the specific implementation method of removing the ORB feature points of the dynamic target in each frame of RGB-D image based on the epipolar geometry constraint condition and obtaining the static ORB feature points of each frame of RGB-D image is:

[0087] First, epipolar geometry constraints are used to remove feature points on dynamic objects.

[0088] When the visual odometry matches feature points, the feature points on the dynamic object do not satisfy the epipolar geometry constraint. Therefore, the epipolar geometry constraint can be used to determine whether the matched feature points are on the moving object.

[0089] The schematic diagram of epipolar geometry constraint is as follows Fig. 9As shown, I1 and I2 are images between two adjacent frames. O1 and O2 are the centers of the two cameras. P is a spatial point, p1 and p2 are the projections of point P on the two frames of images respectively. The line connecting O1 and O2 is called the baseline, and the intersection points e1 and e2 with I1 and I2 are called poles. The plane formed by O1, O2 and the spatial point P is called the polar plane, and the straight lines l1 and l2 intersecting the two image planes are called polar lines.

[0090] If two feature points are correctly matched, the constraints will be satisfied, and the matched feature points will definitely fall on the epipolar line corresponding to the image plane. However, in fact, due to the existence of disturbances, the correctly matched feature points cannot fall exactly on the epipolar line. There will be a slight distance from the epipolar line, and there will be certain errors in the calculation. Therefore, the distance from the matching point to the epipolar line can be used to determine whether it is a correct match. If the distance from the matching point to its corresponding epipolar line is too large, the matching point is considered to be a dynamic feature point and is removed. This application judges the status of the matching point by setting a threshold value d = 0.5 pixels. If the distance from the matching point to its corresponding epipolar line is greater than this threshold, the matching point is considered to be a dynamic feature point and is removed. Otherwise, the feature point is considered to be static.

[0091] This example selects the high dynamic sequence freiburg3_walking_xyz in the TUM RGB-D public dataset as the test dataset. The test results are as follows: Fig.10 As shown in the figure, it can be seen that the dynamic target of the person in the dynamic scene has no matching feature points, which verifies that the method of the present application can effectively remove dynamic feature points and retain many "high-quality" feature points.

[0092] In an exemplary embodiment, an AGV map construction method based on dynamic target detection is provided. The AGV map construction method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the AGV map construction method includes the following steps:

[0093] The AGV is controlled to move one circle in the target scene, and an RGB-D camera set on the AGV is used to collect RGB-D images of the target scene during movement to obtain multi-frame scene RGB-D images of the target scene.

[0094] The above-mentioned camera pose solving method is used to solve the pose of the RGB-D camera when acquiring each frame of the scene RGB-D image.

[0095] According to multiple scene RGB-D images and the pose of the RGB-D camera when acquiring each frame of the scene RGB-D image, the SLAM mapping package is used to build the AGV map of the target scene.

[0096] As a specific implementation method, the camera pose solution method and Melodic version of ROS proposed in this application are ported to the JestonTX2 development board equipped with Ubuntu18.04 system, and the usb_cam camera call package is installed to communicate with the camera, and the SLAM mapping package is installed for mapping. The program file is integrated into the SLAM mapping package, compiled into a ROS node, and finally communication is established through ROS and usb_cam and other nodes. The map construction experiment is carried out in the internal scene of the laboratory, and there are moving personnel in the laboratory. The specific operation process is as follows:

[0097] 1. First, configure the AGV and PC as master and slave. The AGV is the host, mainly executing program operations, while the PC is the slave, mainly responsible for controlling the host and observing the scene mapping process in real time.

[0098] 2. Fill the intrinsic parameter matrix and distortion coefficient obtained by RGB-D camera calibration into the corresponding configuration files in the SLAM mapping package.

[0099] 3. Execute the roscore command in the terminal and start the ROS node manager.

[0100] 4. Execute the AGV's motion node and keyboard control node. At this time, the AGV can be controlled to move forward, backward, left, and right through the keyboard.

[0101] 5. Execute the usb_cam camera shooting node and the SLAM mapping node. When the AGV moves, the scene image is acquired through the RGB-D camera, and the laboratory environment map is constructed at the same time.

[0102] Control the AGV to move around the laboratory for one week. The map building process is as follows: Fig.11 As shown, Fig.11 (a) and (b) are scene images obtained at different moving angles. It can be seen that the method proposed in this application does not match the feature points of the dynamic target. Finally, after the AGV moves for one week, the successfully constructed laboratory scene map is as follows Fig.12 shown.

[0103] The method of this application and the traditional method were experimentally tested on the TUM high-dynamic dataset. The experimental results show that the method of this application can significantly improve the positioning accuracy of visual SLAM in dynamic scenes. Finally, the laboratory scene map was successfully constructed through the experimental platform, verifying the feasibility of the algorithm.

[0104] Based on the above embodiments, the technical method of the present application has the following advantages:

[0105] The lightweight YOLOv5s network model uses the lightweight network ShuffleNetv2 to replace the backbone feature extraction network of YOLOv5s, thereby reducing model parameters, speeding up detection, and greatly improving the real-time performance of the system. Step 2 In order to make up for the loss of feature extraction capability caused by the smaller network, the Stem module is introduced in the first layer of the network input to improve the network feature extraction capability, while also ensuring the accuracy of target detection.

[0106] The YOLOv5s-light network is integrated into the visual odometer. Dynamic target detection is performed while the visual odometer extracts feature points. The dynamic object detection frame can be obtained through the target detection algorithm. The detection frame is usually rectangular, but most dynamic objects are not rectangular, so this application uses epipolar geometry constraints to determine whether the current object is dynamic, accurately remove dynamic target feature points, and only retain static feature points.

[0107] The present application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned dynamic target detection method, camera pose solution method, or AGV map construction method. In an exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows: Fig.13 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store images of dynamic scenes. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above-mentioned dynamic target detection method, camera posture solution method, or AGV map construction method is implemented.

[0108] Those skilled in the art will understand that Fig.13The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0109] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0110] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0111] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0112] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0113] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0114] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A dynamic target detection method, characterized in that: The dynamic target detection method comprises: Constructing a lightweight YOLOv5s network model; the lightweight YOLOv5s network model uses a lightweight ShuffleNetv2 network to replace the backbone feature extraction network of the YOLOv5s network model, and introduces a Stem module into the input layer of the backbone feature extraction network of the YOLOv5s network model; The lightweight YOLOv5s network model is trained and tested using a training set and a test set to obtain a trained lightweight YOLOv5s network model; the training set and the test set both include image sequences of dynamic scenes with dynamic target markers; Dynamic target detection is performed based on the trained lightweight YOLOv5s network model.

2. The dynamic target detection method according to claim 1, characterized in that: The lightweight ShuffleNetv2 network includes: 3 basic units and 3 downsampling modules, the 3 basic units are respectively a first basic unit, a second basic unit and a third basic unit, the 3 downsampling modules are respectively: a first downsampling module, a second downsampling module and a third downsampling module, the first downsampling module and the third downsampling module each include 3 downsampling units connected in sequence, and the second downsampling module includes 7 downsampling units connected in sequence; The Stem module, the first basic unit, the first down-sampling module, the second basic unit, the second down-sampling module, the third basic unit and the third down-sampling module are connected in sequence.

3. The dynamic target detection method according to claim 2, characterized in that: The basic unit includes a channel separation module, a first 3-layer convolution module, a first splicing module and a first channel shuffling module; The channel separation module is connected to the input channel of the basic unit, and the channel separation module is used to divide the input channel into two first branches, wherein one first branch is directly connected to the first splicing module, and the other first branch is connected to the first splicing module through the first 3-layer convolution module; The first splicing module is used for splicing two first branches; The first splicing module is connected to the first channel shuffling module; the first channel shuffling module is used to disrupt the order of output channels of the basic unit.

4. The dynamic target detection method according to claim 2, characterized in that: The downsampling unit comprises: a first 2-layer convolution module, a second 3-layer convolution module, a second splicing module and a second channel shuffling module; The first 2-layer convolution module and the second 3-layer convolution module are used as two second branches, one end of the two second branches is connected to the input channel of the downsampling unit, the other end of the two second branches is connected to the second splicing module, and the second splicing module is connected to the second channel shuffling module; The second splicing module is used for splicing two second branches; The second channel shuffling module is used to disrupt the order of the output channels of the down-sampling unit.

5. The dynamic target detection method according to claim 2, characterized in that: The Stem module includes: a second 2-layer convolution module, a maximum pooling layer, a third splicing module and a 1-layer convolution module; The second 2-layer convolution module and the maximum pooling layer serve as two third branches, one end of the two third branches are connected to the input channel of the Stem module, the other end of the two third branches are connected to the third splicing module, and the third splicing module is connected to the 1-layer convolution module.

6. The dynamic target detection method according to claim 1, characterized in that: Use the training set and test set to train and test the lightweight YOLOv5s network model to obtain a trained lightweight YOLOv5s network model, including: The PascalVOC 2007 dataset is used as the training set; Using the training set to train the lightweight YOLOv5s network model to obtain a trained lightweight YOLOv5s network model; The dynamic sequence freiburg3_walking_xyz in the TUM RGB-D dataset is used as the test set; Use the test set to test the trained lightweight YOLOv5s network model; When the test fails, return to the step of "training the lightweight YOLOv5s network model using the training set to obtain the trained lightweight YOLOv5s network model"; When the test passes, the trained lightweight YOLOv5s network model is output as the trained lightweight YOLOv5s network model.

7. A camera pose solution method based on dynamic target detection, characterized in that: The camera pose solving method comprises: Get two adjacent frames of RGB-D images taken by the RGB-D camera; Extracting ORB feature points of each frame of RGB-D image; in a dynamic scene, the ORB feature points include dynamic feature points and static feature points; Using the dynamic target detection method according to any one of claims 1 to 6, detecting the dynamic target in each frame of the RGB-D image; Based on the epipolar geometry constraint condition, the ORB feature points of the dynamic target in each frame of RGB-D image are eliminated to obtain the static ORB feature points of each frame of RGB-D image. Based on the static ORB feature points of each RGB-D image frame, the position and posture of the RGB-D camera when obtaining each RGB-D image frame is solved.

8. An AGV map construction method based on dynamic target detection, characterized in that: The AGV map construction method comprises the following steps: Control the AGV to move one circle in the target scene, and use the RGB-D camera set on the AGV to collect the RGB-D image of the target scene during the movement to obtain a multi-frame scene RGB-D image of the target scene; The camera pose solving method according to claim 7 is used to solve the pose of the RGB-D camera when acquiring each frame of the scene RGB-D image; According to multiple scene RGB-D images and the pose of the RGB-D camera when acquiring each frame of the scene RGB-D image, the SLAM mapping package is used to build the AGV map of the target scene.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the dynamic target detection method described in any one of claims 1 to 6, or the camera pose solving method described in claim 7, or the AGV map building method described in claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the dynamic target detection method described in any one of claims 1 to 6, or the camera pose solving method described in claim 7, or the AGV map construction method described in claim 8.

Images