Automatic guiding parking space posture monitoring method, device and equipment and storage medium

Abstract

The invention discloses an automatic guided vehicle posture monitoring method, device and equipment and a storage medium, and relates to the technical field of positioning, and the method comprises the steps: carrying out the image collection of a current to-be-monitored automatic guided vehicle through a fixed camera, and obtaining a current monitoring image; inputting the current monitoring image into the pose estimation model; acquiring first pose information of the to-be-monitored automatic guided vehicle in the current monitoring image estimated by the pose estimation model, and determining second pose information of the current to-be-monitored automatic guided vehicle based on a laser instant positioning and mapping algorithm; and comparing the first pose information with the second pose information to determine whether the second pose information currently obtained based on the laser instant positioning and mapping algorithm is abnormal or not. According to the method, whether the second pose information obtained through the laser instant positioning and mapping algorithm is abnormal or not is judged through the first pose output by the trained pose estimation model, and the reliability of the second pose is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of positioning technology, and in particular to a method, device, equipment and storage medium for monitoring the position and posture of an automatic guided vehicle. Background Art

[0002] SLAM (simultaneous localization and mapping), also known as CML (Concurrent Mapping and Localization), is a real-time positioning and map construction, or concurrent mapping and positioning. SLAM technology has broad application prospects in the field of robotics.

[0003] The existing solution for positioning AGVs (Automated Guided Vehicles) is to install additional cameras on the vehicle and post specific coded images (such as QR codes, etc.) at a large number of locations in the factory to assist in SLAM positioning. Codes are set for specific markers, and when the camera on the AGV detects a specific marker, it can estimate its own posture. When obtaining the AGV posture in this way, there may be a problem of posture detection errors when the marker is contaminated, which makes it impossible to assist in SLAM positioning. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide an automatic guided vehicle posture monitoring method, device, equipment and storage medium, which can generate the first posture information by using the image collected by the fixed camera in the factory, and use the first posture information to correct the second posture information obtained based on the laser real-time positioning and mapping algorithm, thereby solving the problem of needing to install an additional camera on the automatic guided vehicle to assist in correcting the second posture information. The specific scheme is as follows:

[0005] In a first aspect, the present application provides a method for monitoring the posture of an automatic guided vehicle, comprising:

[0006] The image of the current automatic guided vehicle to be monitored is collected by a preset fixed camera to obtain a corresponding current monitoring image;

[0007] Inputting the current monitoring image into a posture estimation model obtained after training based on a preset training set; the preset training set includes historical monitoring images of any automatic guided vehicle collected by the preset fixed camera, a coordinate anchor frame of the any automatic guided vehicle obtained after target detection on the historical monitoring image, and corresponding image annotation information, wherein the image annotation information includes accurate historical posture information of the any automatic guided vehicle determined based on a laser real-time positioning and mapping algorithm;

[0008] Acquire the first pose information of the automatic guided vehicle to be monitored in the current monitoring image estimated by the pose estimation model, and determine the second pose information of the automatic guided vehicle to be monitored based on the laser real-time positioning and mapping algorithm;

[0009] By comparing the first posture information with the second posture information, it is determined whether there is any abnormality in the second posture information currently obtained based on the laser instant positioning and mapping algorithm.

[0010] Optionally, the step of collecting an image of the current automatic guided vehicle to be monitored by using a preset fixed camera to obtain a corresponding current monitoring image includes:

[0011] The preset fixed camera is used to collect images of the working area where the currently monitored automatic guided vehicle is located, and the collected initial images are input into the preset target detection model;

[0012] The preset target detection model is used to detect whether each of the initial images contains the automatic guided vehicle to be monitored, and the initial image containing the automatic guided vehicle to be monitored is screened out from each of the initial images based on the corresponding target detection result to obtain the corresponding current monitoring image.

[0013] Optionally, before collecting images of the current automatic guided vehicle to be monitored by a preset fixed camera, the method further includes:

[0014] Determine a working area corresponding to any automatic guided vehicle based on preset application requirements;

[0015] A preset laser radar is used to scan the working area corresponding to any automated guided vehicle, and a corresponding target map is constructed for the working area corresponding to any automated guided vehicle based on the corresponding scanning results and the laser real-time positioning and mapping algorithm, so as to obtain the historical accurate posture information according to the laser real-time positioning and mapping algorithm and the target map.

[0016] Optionally, before inputting the current monitoring image into a pose estimation model obtained after training based on a preset training set, the method further includes:

[0017] Labeling the historical monitoring image and the coordinate anchor frame with historical posture information labels corresponding to the historical accurate posture information to obtain the preset training set;

[0018] The historical monitoring images and the coordinate anchor frames in the preset training set are sent to the input end of the initial pose estimation model for training, and the historical pose information labels are sent to the output end of the initial pose estimation model for training to obtain a trained pose estimation model.

[0019] Optionally, the training process of the initial pose estimation model further includes:

[0020] The static object feature information in the working environment of the automatic guided vehicle is determined by the historical monitoring images in the preset training set, and the different working areas in the working environment are identified by using the static object feature information, so as to perform model training according to the corresponding working area identification results.

[0021] Optionally, the automatic guided vehicle posture monitoring method further includes:

[0022] The first posture information of the automatic guided vehicle to be monitored outputted in real time by the posture estimation model is stored in a preset storage space;

[0023] When the automatic guided vehicle to be monitored restarts, the first position and posture information of the automatic guided vehicle to be monitored stored in real time in the preset storage space is read, and the position and posture of the automatic guided vehicle to be monitored is initialized using the read first position and posture information.

[0024] In a second aspect, the present application discloses a posture estimation model training method, comprising:

[0025] Using the target detection model to detect the historical monitoring images collected by the preset fixed camera to obtain the coordinate anchor frame of any automatic guided vehicle in the historical monitoring images;

[0026] Determine the historical accurate posture information of the arbitrary automatic guided vehicle based on the laser real-time positioning and mapping algorithm, and determine the historical accurate posture information as the image annotation information corresponding to the historical monitoring image and the coordinate anchor frame;

[0027] A target training set is constructed according to the historical monitoring image, the coordinate anchor frame and the image annotation information, and an initial pose estimation model is trained by using the target training set to obtain a trained pose estimation model.

[0028] In a third aspect, the present application discloses a posture abnormality monitoring device, comprising:

[0029] An image acquisition module is used to acquire images of the current automatic guided vehicle to be monitored through a preset fixed camera to obtain a corresponding current monitoring image;

[0030] An image input module, used to input the current monitoring image into a posture estimation model obtained after training based on a preset training set; the preset training set includes historical monitoring images of any automatic guided vehicle collected by the preset fixed camera, a coordinate anchor frame of the any automatic guided vehicle obtained after target detection on the historical monitoring image, and corresponding image annotation information, wherein the image annotation information includes accurate historical posture information of the any automatic guided vehicle determined based on a laser instant positioning and mapping algorithm;

[0031] A posture information acquisition module, used to obtain the first posture information of the automatic guided vehicle to be monitored in the current monitoring image estimated by the posture estimation model, and determine the second posture information of the automatic guided vehicle to be monitored based on the laser real-time positioning and mapping algorithm;

[0032] The posture information comparison module is used to compare the first posture information with the second posture information to determine whether there is an abnormality in the second posture information currently obtained based on the laser instant positioning and mapping algorithm.

[0033] In a fourth aspect, the present application provides an electronic device, including:

[0034] Memory, used to store computer programs;

[0035] A processor is used to execute the computer program to implement the aforementioned method.

[0036] In a fifth aspect, the present application provides a computer-readable storage medium for storing a computer program, which implements the aforementioned method when executed by a processor.

[0037] The present application firstly acquires an image of the current automated guided vehicle to be monitored by a preset fixed camera to obtain a corresponding current monitoring image, and then inputs the current monitoring image into a posture estimation model obtained after training based on a preset training set; the preset training set includes historical monitoring images of any automated guided vehicle acquired by the preset fixed camera, a coordinate anchor frame of any automated guided vehicle obtained after target detection on the historical monitoring image, and corresponding image annotation information, wherein the image annotation information includes accurate historical posture information of any automated guided vehicle determined based on a laser instant positioning and mapping algorithm, and then obtains the first posture information of the automated guided vehicle to be monitored in the current monitoring image estimated by the posture estimation model, and determines the second posture information of the current automated guided vehicle to be monitored based on the laser instant positioning and mapping algorithm, and finally determines whether the second posture information currently obtained based on the laser instant positioning and mapping algorithm has an abnormality by comparing the first posture information with the second posture information. It can be seen that the present application obtains a pose estimation model through training, uses the monitoring images collected by fixed cameras in the factory and the pose estimation model to obtain the first pose information of the automatic guided vehicle, and uses the first pose information to determine whether there is an abnormality in the second pose information obtained based on the laser instant positioning and mapping algorithm, thereby avoiding the installation of additional cameras on the automatic guided vehicle and the posting of multiple auxiliary identification marks in the factory, thereby avoiding the problem that the erroneous pose detected when the marker is contaminated cannot assist the laser instant positioning and mapping algorithm in positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0039] Figure 1 A flow chart of a method for monitoring the position and posture of an automatic guided vehicle disclosed in this application;

[0040] Figure 2 This is an overall flow chart of an automatic guided vehicle posture monitoring method disclosed in this application;

[0041] Figure 3 A flow chart of a posture estimation model training method disclosed in this application;

[0042] Figure 4 This is a schematic diagram of the structure of a posture abnormality monitoring device disclosed in this application;

[0043] Figure 5This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] At present, the method of assisting in obtaining the position and posture of an automatic guided vehicle is to install a mobile camera on the automatic guided vehicle and lay a large number of camera recognition markers in the factory to assist in obtaining the position and posture of the automatic guided vehicle. However, this method has the problem that when the marker is contaminated, the mobile camera cannot obtain the position and posture information of the automatic guided vehicle. To this end, the present application provides a method for monitoring the position and posture of an automatic guided vehicle, which can obtain the position and posture of the automatic guided vehicle through a fixed camera in the factory to assist the laser real-time positioning and mapping algorithm, solving the problem that the position and posture cannot be obtained when the marker is contaminated.

[0046] See also Figure 1 As shown, an embodiment of the present invention discloses a method for monitoring the position and posture of an automatic guided vehicle, comprising:

[0047] Step S11: collect images of the current automatic guided vehicle to be monitored by a preset fixed camera to obtain a corresponding current monitoring image.

[0048] In this embodiment, it is necessary to use a fixed camera in the factory to collect images and obtain the corresponding current detection image. It can be understood that there are several fixed cameras in the factory for image collection, wherein not all monitoring cameras have AGVs in their monitoring range, that is, the images collected by the above-mentioned several fixed cameras include images with automatic guided vehicles in the picture and images without automatic guided vehicles in the picture. It is necessary to first detect the AGV by target, and then only further process the images with AGV. Correspondingly, the above-mentioned process of collecting images of the current automatic guided vehicle to be monitored by a preset fixed camera to obtain the corresponding current monitoring image can specifically include: collecting images of the working area where the current automatic guided vehicle to be monitored is located by a preset fixed camera, and inputting the collected initial images into a preset target detection model; detecting whether each initial image contains the automatic guided vehicle to be monitored by the preset target detection model, and filtering out the initial image containing the automatic guided vehicle to be monitored from each initial image based on the corresponding target detection result to obtain the corresponding current monitoring image.

[0049] Before using a fixed camera to collect images of the current automated guided vehicle to be monitored, it also includes: determining the working area corresponding to any automated guided vehicle based on preset application requirements; using a preset laser radar to scan the working area corresponding to any automated guided vehicle, and building a corresponding target map for the working area corresponding to any automated guided vehicle based on the corresponding scanning results and the laser instant positioning and mapping algorithm, so as to obtain historical accurate posture information according to the laser instant positioning and mapping algorithm and the target map. It can be understood that there are multiple automated guided vehicles in the factory environment. First, it is necessary to determine the corresponding working area according to the working needs of any automated guided vehicle, so as to scan the working area using the laser radar. This embodiment makes the process of generating the coordinate anchor frame more reliable by screening out images containing the current automated guided vehicle to be monitored; by determining the working area of ​​any automated guided vehicle and generating a corresponding target map for the above-mentioned working area, the problem of wasting computing resources when generating the target map due to the scanning area being too large is avoided.

[0050] Step S12, inputting the current monitoring image into a posture estimation model obtained after training based on a preset training set; the preset training set includes historical monitoring images of any automatic guided vehicle collected by the preset fixed camera, a coordinate anchor frame of the any automatic guided vehicle obtained after target detection on the historical monitoring image, and corresponding image annotation information, wherein the image annotation information includes accurate historical posture information of the any automatic guided vehicle determined based on a laser instant positioning and mapping algorithm.

[0051] In this embodiment, before the current monitoring image is input into the trained pose estimation model, it also includes: marking the historical monitoring image and the coordinate anchor frame with the historical pose information label corresponding to the historical accurate pose information to obtain a preset training set; sending the historical monitoring image and the coordinate anchor frame in the preset training set to the input end of the initial pose estimation model for training, and sending the historical pose information label to the output end of the initial pose estimation model for training to obtain the trained pose estimation model. It can be understood that the above-mentioned historical monitoring image is a historical image containing an automatic guided vehicle in the picture, and the historical accurate pose is the prior data in the model training process, which is used to calculate the loss function, and the model is trained by the above-mentioned loss function to reduce the error between the first pose information output by the model and the corresponding historical accurate pose information. It should be noted that the above-mentioned historical accurate pose information is obtained by using the laser instant positioning and mapping algorithm and the target map. In the early stage of the target map establishment, the environmental conditions in the working area corresponding to any automatic guided vehicle are relatively simple. At this time, the laser instant positioning and mapping algorithm will not be interfered by other environmental factors in the working area, and the historical accurate pose information corresponding to the automatic guided vehicle can be obtained. By marking the historical monitoring images and coordinate anchor frames with labels corresponding to the historical accurate pose information and inputting them into the initial pose estimation model for training, the reliability of the pose information output by the model is ensured.

[0052] It should be noted that before obtaining the coordinate anchor frame, it is also necessary to train a target detection model to obtain the corresponding coordinate anchor frame of any automatic guided vehicle after performing target detection on historical monitoring images; the specific type of the above target detection model can be determined according to actual application requirements; in a specific implementation, taking the ppyoloe model (a target detection model) as an example, it is first necessary to collect monitoring images of AGVs appearing in multiple factory areas in various operating time periods. The images may contain various types of AGVs, and then annotate the images, marking the circumscribed rectangular frame of the AGV as a training anchor frame, and input the corresponding annotation data and multiple monitoring images in which AGVs appear as training data into the initial target detection model to obtain a trained target detection model, so as to use the target detection model to provide training data for the pose estimation model.

[0053] Step S13, obtaining the first pose information of the automatic guided vehicle to be monitored in the current monitoring image estimated by the pose estimation model, and determining the second pose information of the automatic guided vehicle to be monitored based on the laser real-time positioning and mapping algorithm.

[0054] In this embodiment, the trained pose estimation model can output the first pose information of the currently monitored automatic guided vehicle, that is, the trained pose estimation model can directly estimate the AGV in the image based on the monitoring camera image, that is, the global coordinates of the automatic guided vehicle in the laser SLAM coordinate system, and the above laser SLAM coordinate system is a coordinate system obtained according to the target map; the second pose information of the currently monitored automatic guided vehicle can be determined by the laser instant positioning and mapping algorithm and the target map. It is understandable that due to changes in the working environment of the automatic guided vehicle, the second pose information determined by the laser instant positioning and mapping algorithm at this time may have problems such as measurement object errors.

[0055] Step S14: Compare the first posture information with the second posture information to determine whether there is any abnormality in the second posture information currently obtained based on the laser instant positioning and mapping algorithm.

[0056] In this embodiment, the posture estimation model after training no longer needs to input the posture obtained according to the target map, but outputs the posture for use on the SLAM side, that is, the first posture information output by the model is fed back to the verification module in the laser SLAM to assist the laser SLAM positioning, that is, the first posture information is compared with the second posture information to determine whether there is an abnormality in the second posture information currently obtained based on the laser instant positioning and mapping algorithm. It should be noted that the first posture information of the automatic guided vehicle output by the posture estimation model will not have the problem of estimation object error, but the accuracy is low, and the second posture information currently obtained based on the laser instant positioning and mapping algorithm may have the problem of measurement object error, but the obtained posture accuracy is high, so the first posture information is needed to judge the reliability of the second posture information. This embodiment can obtain the reliability evaluation of the second posture information by comparing the first posture information with the second posture information.

[0057] It should be noted that after the first position information of the current automatic guided vehicle to be monitored is obtained by using the position estimation model, the first position information of the current automatic guided vehicle to be monitored output by the position estimation model in real time is stored in the preset storage space; Figure 2 As shown, when the automatic guided vehicle to be monitored restarts, that is, when the current automatic guided vehicle to be monitored is turned on or restarted, the first position information of the automatic guided vehicle to be monitored stored in real time in the preset storage space is read, and the position of the automatic guided vehicle to be monitored is initialized using the read first position information; and for the AGV that has been positioned by the laser instant positioning and mapping algorithm, the difference between the position obtained by the position estimation model and the position obtained by the laser instant positioning and mapping algorithm can be compared in real time to determine whether the position information obtained by the laser instant positioning and mapping algorithm is mismatched.

[0058] It can be seen that the embodiment of the present application makes the process of generating the coordinate anchor frame more reliable by screening out images containing the current automated guided vehicle to be monitored; by determining the working area of ​​any automated guided vehicle and generating a corresponding target map for the above working area, the problem of waste of computing resources when generating the target map due to excessive scanning area is avoided; by marking the historical monitoring images and the coordinate anchor frames with labels corresponding to the historical accurate posture information, and inputting them into the initial posture estimation model for training, the reliability of the posture information output by the model is guaranteed; by using the first posture information to determine whether the second posture information obtained based on the laser instant positioning and mapping algorithm is abnormal, the installation of additional cameras on the automated guided vehicle and the posting of multiple auxiliary identification marks in the factory are avoided, thereby avoiding the problem that the erroneous posture detected when the marker is contaminated cannot assist the laser instant positioning and mapping algorithm in positioning.

[0059] Based on the previous embodiment, this application describes the process of obtaining the first pose information by using the current monitoring image and the pose estimation model, and comparing the first pose information with the second pose information obtained by the laser instant positioning and mapping algorithm. In order to make the technical method disclosed in this application more complete, the present application will describe the training process of the pose estimation model. Figure 3 As shown, the embodiment of the present application discloses a posture estimation model training method, comprising:

[0060] Step S21: Detect the historical monitoring images collected by the preset fixed camera using the target detection model to obtain the coordinate anchor frame of any automatic guided vehicle in the historical monitoring images.

[0061] Step S22: determining the historical accurate posture information of the arbitrary automatic guided vehicle based on the laser real-time positioning and mapping algorithm, and determining the historical accurate posture information as the image annotation information corresponding to the historical monitoring image and the coordinate anchor frame.

[0062] Step S23: construct a target training set according to the historical monitoring image, the coordinate anchor frame and the image annotation information, and train the initial pose estimation model through the target training set to obtain a trained pose estimation model.

[0063] In this embodiment, the coordinate anchor frame is input to make the pose estimation model focus on the correspondence between the pixel coordinate features and global features of the AGV; and the historical monitoring image is input to determine the static object feature information in the working environment of the automatic guided vehicle through the historical monitoring images in the preset training set, and use the static object feature information to identify different working areas in the working environment, so as to perform model training according to the corresponding working area identification results, thereby making the model training process faster and the pose information output by the model more accurate.

[0064] For the specific contents of the above-mentioned step S21 and step S22, reference may be made to the corresponding contents disclosed in the above-mentioned embodiment, which will not be repeated here.

[0065] It can be seen that the present application uses historical monitoring images, coordinate anchor frames and historical accurate pose information to train the initial pose estimation model. By inputting historical monitoring images, the initial pose estimation model determines the static object feature information in the working environment of the automatic guided vehicle, and uses the static object feature information to identify different working areas in the working environment. The model is trained according to the corresponding working area identification results, thereby ensuring the reliability of the pose information output by the trained pose estimation model.

[0066] See also Figure 4 As shown, the embodiment of the present application discloses a posture abnormality monitoring device, comprising:

[0067] An image acquisition module 11 is used to acquire an image of the current automatic guided vehicle to be monitored through a preset fixed camera to obtain a corresponding current monitoring image;

[0068] An image input module 12 is used to input the current monitoring image into a posture estimation model obtained after training based on a preset training set; the preset training set includes historical monitoring images of any automatic guided vehicle collected by the preset fixed camera, a coordinate anchor frame of the any automatic guided vehicle obtained after target detection on the historical monitoring image, and corresponding image annotation information, wherein the image annotation information includes accurate historical posture information of the any automatic guided vehicle determined based on a laser real-time positioning and mapping algorithm;

[0069] A posture information acquisition module 13 is used to obtain the first posture information of the automatic guided vehicle to be monitored in the current monitoring image estimated by the posture estimation model, and determine the second posture information of the automatic guided vehicle to be monitored based on the laser real-time positioning and mapping algorithm;

[0070] The posture information comparison module 14 is used to compare the first posture information with the second posture information to determine whether there is an abnormality in the second posture information currently obtained based on the laser real-time positioning and mapping algorithm.

[0071] It can be seen that the present application obtains a pose estimation model through training, uses the monitoring images collected by fixed cameras in the factory and the pose estimation model to obtain the first pose information of the automatic guided vehicle, and uses the first pose information to determine whether there is an abnormality in the second pose information obtained based on the laser instant positioning and mapping algorithm, thereby avoiding the installation of additional cameras on the automatic guided vehicle and the posting of multiple auxiliary identification marks in the factory, thereby avoiding the problem that the erroneous pose detected when the marker is contaminated cannot assist the laser instant positioning and mapping algorithm in positioning.

[0072] In some specific embodiments, the image acquisition module 11 may specifically include:

[0073] An image input unit is used to collect images of the working area where the automatic guided vehicle to be monitored is located through a preset fixed camera, and input the collected initial image into a preset target detection model;

[0074] An image screening unit is used to detect whether each of the initial images contains the automatic guided vehicle to be monitored through the preset target detection model, and to screen out the initial image containing the automatic guided vehicle to be monitored from each of the initial images based on the corresponding target detection result to obtain the corresponding current monitoring image.

[0075] In some specific embodiments, the image acquisition module 11 further includes:

[0076] A working area determination unit, configured to determine a working area corresponding to any automatic guided vehicle based on preset application requirements;

[0077] A map construction unit is used to scan the working area corresponding to any automatic guided vehicle using a preset laser radar, and to construct a corresponding target map for the working area corresponding to any automatic guided vehicle based on the corresponding scanning results and the laser real-time positioning and mapping algorithm, so as to obtain the historical accurate posture information according to the laser real-time positioning and mapping algorithm and the target map.

[0078] In some specific embodiments, the image input module 12 further includes:

[0079] A labeling unit, used for labeling the historical monitoring image and the coordinate anchor frame with a historical posture information label corresponding to the historical accurate posture information to obtain the preset training set;

[0080] The model training submodule is used to send the historical monitoring images and the coordinate anchor frames in the preset training set to the input end of the initial pose estimation model for training, and to send the historical pose information labels to the output end of the initial pose estimation model for training to obtain a trained pose estimation model.

[0081] In some specific embodiments, the model training submodule further includes:

[0082] A model training unit is used to determine the static object feature information in the working environment of the automatic guided vehicle through the historical monitoring images in the preset training set, and use the static object feature information to identify different working areas in the working environment, so as to perform model training according to the corresponding working area identification results.

[0083] In some specific embodiments, the abnormal posture monitoring device further includes:

[0084] A posture information storage unit, used for storing the first posture information of the automatic guided vehicle to be monitored output by the posture estimation model in real time into a preset storage space;

[0085] The posture initialization unit is used to read the first posture information of the automatic guided vehicle to be monitored stored in real time in the preset storage space when the automatic guided vehicle to be monitored restarts, and initialize the posture of the automatic guided vehicle to be monitored using the read first posture information.

[0086] Furthermore, the present application also discloses an electronic device. Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0087] Figure 5 A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the automatic guided vehicle posture monitoring method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0088] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0089] In addition, the memory 22 as a carrier for storing resources may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.

[0090] The operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the automatic guided vehicle posture monitoring method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0091] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned disclosed method for monitoring the posture of an automated guided vehicle is implemented. The specific steps of the method can be referred to the corresponding contents disclosed in the aforementioned embodiments, and will not be repeated here.

[0092] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0093] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0094] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0095] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0096] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for monitoring the position and posture of an automatic guided vehicle, characterized in that: include: The image of the current automatic guided vehicle to be monitored is collected by a preset fixed camera to obtain a corresponding current monitoring image; Inputting the current monitoring image into a posture estimation model obtained after training based on a preset training set; the preset training set includes historical monitoring images of any automatic guided vehicle collected by the preset fixed camera, a coordinate anchor frame of the any automatic guided vehicle obtained after target detection on the historical monitoring image, and corresponding image annotation information, wherein the image annotation information includes accurate historical posture information of the any automatic guided vehicle determined based on a laser real-time positioning and mapping algorithm; Acquire the first pose information of the automatic guided vehicle to be monitored in the current monitoring image estimated by the pose estimation model, and determine the second pose information of the automatic guided vehicle to be monitored based on the laser real-time positioning and mapping algorithm; By comparing the first posture information with the second posture information, it is determined whether there is any abnormality in the second posture information currently obtained based on the laser instant positioning and mapping algorithm.

2. The method for monitoring the position and posture of an automated guided vehicle according to claim 1, characterized in that: The method of collecting images of the current automatic guided vehicle to be monitored by a preset fixed camera to obtain a corresponding current monitoring image includes: The preset fixed camera is used to collect images of the working area where the currently monitored automatic guided vehicle is located, and the collected initial images are input into the preset target detection model; The preset target detection model is used to detect whether each of the initial images contains the automatic guided vehicle to be monitored, and the initial image containing the automatic guided vehicle to be monitored is screened out from each of the initial images based on the corresponding target detection result to obtain the corresponding current monitoring image.

3. The method for monitoring the position and posture of an automated guided vehicle according to claim 1, characterized in that: Before collecting images of the current automatic guided vehicle to be monitored by a preset fixed camera, the method further includes: Determine a working area corresponding to any automatic guided vehicle based on preset application requirements; A preset laser radar is used to scan the working area corresponding to any automated guided vehicle, and a corresponding target map is constructed for the working area corresponding to any automated guided vehicle based on the corresponding scanning results and the laser real-time positioning and mapping algorithm, so as to obtain the historical accurate posture information according to the laser real-time positioning and mapping algorithm and the target map.

4. The method for monitoring the position and posture of an automated guided vehicle according to claim 1, characterized in that: Before inputting the current monitoring image into the posture estimation model obtained after training based on the preset training set, the method further includes: Labeling the historical monitoring image and the coordinate anchor frame with historical posture information labels corresponding to the historical accurate posture information to obtain the preset training set; The historical monitoring images and the coordinate anchor frames in the preset training set are sent to the input end of the initial pose estimation model for training, and the historical pose information labels are sent to the output end of the initial pose estimation model for training to obtain a trained pose estimation model.

5. The method for monitoring the position and posture of an automated guided vehicle according to claim 4, characterized in that: The training process of the initial pose estimation model also includes: The static object feature information in the working environment of the automatic guided vehicle is determined by the historical monitoring images in the preset training set, and the different working areas in the working environment are identified by using the static object feature information, so as to perform model training according to the corresponding working area identification results.

6. The method for monitoring the position and posture of an automated guided vehicle according to any one of claims 1 to 5, characterized in that: Also includes: The first posture information of the automatic guided vehicle to be monitored outputted in real time by the posture estimation model is stored in a preset storage space; When the automatic guided vehicle to be monitored restarts, the first position and posture information of the automatic guided vehicle to be monitored stored in real time in the preset storage space is read, and the position and posture of the automatic guided vehicle to be monitored is initialized using the read first position and posture information.

7. A method for training a pose estimation model, characterized in that: include: Using the target detection model to detect the historical monitoring images collected by the preset fixed camera to obtain the coordinate anchor frame of any automatic guided vehicle in the historical monitoring images; Determine the historical accurate posture information of the arbitrary automatic guided vehicle based on the laser real-time positioning and mapping algorithm, and determine the historical accurate posture information as the image annotation information corresponding to the historical monitoring image and the coordinate anchor frame; A target training set is constructed according to the historical monitoring image, the coordinate anchor frame and the image annotation information, and an initial pose estimation model is trained by using the target training set to obtain a trained pose estimation model.

8. A posture abnormality monitoring device, characterized in that: include: An image acquisition module is used to acquire images of the current automatic guided vehicle to be monitored through a preset fixed camera to obtain a corresponding current monitoring image; An image input module, used to input the current monitoring image into a posture estimation model obtained after training based on a preset training set; the preset training set includes historical monitoring images of any automatic guided vehicle collected by the preset fixed camera, a coordinate anchor frame of the any automatic guided vehicle obtained after target detection on the historical monitoring image, and corresponding image annotation information, wherein the image annotation information includes accurate historical posture information of the any automatic guided vehicle determined based on a laser instant positioning and mapping algorithm; A posture information acquisition module, used to obtain the first posture information of the automatic guided vehicle to be monitored in the current monitoring image estimated by the posture estimation model, and determine the second posture information of the automatic guided vehicle to be monitored based on the laser real-time positioning and mapping algorithm; The posture information comparison module is used to compare the first posture information with the second posture information to determine whether there is an abnormality in the second posture information currently obtained based on the laser instant positioning and mapping algorithm.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, which implements the method according to any one of claims 1 to 7 when executed by a processor.

Images