Underground vehicle positioning method, device and equipment and storage medium

Abstract

The invention provides an underground vehicle positioning method, device and equipment and a storage medium, and relates to the technical field of coal mining. In some embodiments of the disclosure, vehicle electronic control unit information, vehicle-mounted video stream information and vehicle ultra-wideband positioning information of a target vehicle are acquired; determining initial inertial positioning position information of the target vehicle according to the vehicle electronic control unit information; determining initial visual positioning position information of the target vehicle according to the vehicle-mounted video stream information; performing global coordinate conversion on the initial inertial positioning position information to obtain target inertial positioning position information, and performing global coordinate conversion on the initial visual positioning position information to obtain target visual positioning position information; performing position information fusion on the target inertial positioning position information, the target visual positioning position information and the vehicle ultra-wideband positioning information to obtain vehicle positioning fusion position information of the target vehicle; the positioning accuracy of the vehicle in the motion state is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of coal mining, and particularly relates to an underground vehicle positioning method, device, equipment and storage medium. BACKGROUND

[0002] With the continuous advancement of intelligent construction of coal mines, the automation and intelligence level of underground transportation systems are increasingly improved, and accurate positioning of transportation vehicles such as underground trackless rubber-tyred vehicles and track electric locomotives becomes one of the key technologies to ensure safety production and improve transportation efficiency. Due to the complex environment of coal mine underground, there are problems such as narrow roadway, poor lighting conditions, strong electromagnetic interference and lack of GPS signal, so the traditional positioning method based on wireless signal cannot meet the high-precision and high-stability positioning requirements of underground vehicles in dynamic operation. Therefore, the fusion of multiple positioning methods, especially the introduction of AI vision-based positioning technology, has become an important research direction to improve the positioning ability of underground vehicles.

[0003] The positioning of existing underground vehicles in coal mines mainly relies on inertial navigation system (INS), radio frequency identification (RFID), WIFI fingerprint positioning, UWB (ultra-wideband) and other wireless positioning technologies. Among them, UWB has been widely used in underground positioning due to its high ranging accuracy; some systems use UWB and INS fusion to improve positioning stability through Kalman filtering and other algorithms. In addition, some mines try to introduce visual auxiliary positioning methods, which use fixed markers on the roadway wall or track features to estimate the vehicle position through traditional image processing methods.

[0004] At present, the positioning accuracy of underground vehicle positioning technology in the vehicle motion state is low. SUMMARY

[0005] The present disclosure provides an underground vehicle positioning method, device, equipment and storage medium to at least solve the problem of low positioning accuracy of existing underground vehicles.

[0006] The technical solution of the present disclosure is as follows:

[0007] The present disclosure provides an underground vehicle positioning method, device, equipment and storage medium to at least solve the problem of low positioning accuracy of existing underground vehicles.

[0008] Obtain vehicle electronic control unit information, vehicle video stream information and vehicle ultra-wideband positioning information of a target vehicle;

[0009] Determine initial inertial positioning position information of the target vehicle according to the vehicle electronic control unit information;

[0010] Determine initial visual positioning position information of the target vehicle according to the vehicle video stream information;

[0011] perform global coordinate conversion on the initial inertial positioning position information to obtain target inertial positioning position information, and perform global coordinate conversion on the initial visual positioning position information to obtain target visual positioning position information;

[0012] perform position information fusion on the target inertial positioning position information, the target visual positioning position information, and the vehicle ultra-wideband positioning information to obtain vehicle positioning fusion position information of the target vehicle.

[0013] Optionally, the vehicle electronic control unit information includes specific force and angular velocity; and the determining of the initial inertial positioning position information of the target vehicle according to the vehicle electronic control unit information comprises:

[0014] acquiring a given initial condition;

[0015] inputting the given initial condition, the specific force, and the angular velocity into a track recursion algorithm to obtain the initial inertial positioning position information of the target vehicle.

[0016] Optionally, the determining of the initial visual positioning position information of the target vehicle according to the vehicle video stream information comprises:

[0017] performing marker detection and marker tracking according to the vehicle video stream information to obtain a target marker and a relative position between the target marker and the target vehicle;

[0018] identifying the target marker to obtain a type of the target marker;

[0019] determining the initial visual positioning position information of the target vehicle according to the type of the target marker and the relative position.

[0020] Optionally, the performing of the marker detection and the marker tracking according to the vehicle video stream information to obtain the target marker and the relative position between the target marker and the target vehicle comprises:

[0021] performing data enhancement processing on the vehicle video stream information and initializing a set of anchor presets to obtain an input image for target detection;

[0022] extracting features from the input image by using a feature extraction network to obtain a multi-scale feature map;

[0023] fusing the multi-scale feature map by using a PANet structure to obtain fusion feature map information;

[0024] adjusting parameters of a target detection network by an optimizer based on a classification loss function and a bounding box regression loss function to predict a category and position information of the target marker.

[0025] based on the detection results of the current frame and the historical frames, predicting a motion state of the target marker by using Kalman filtering;

[0026] based on an intersection over union distance between the predicted bounding box and the detection bounding box of the current frame, performing data association by using a Hungarian algorithm to match the detection result with an existing tracker;

[0027] when the detection result does not match the existing tracker, creating a new tracker and initializing a motion state of the new tracker; and when any one tracker is not matched in consecutive T frames, destroying the any one tracker;

[0028] based on the detection result and the tracking result, determining a target marker and a relative position of the target marker and the target vehicle.

[0029] Optionally, the determining of the initial visual positioning position information of the target vehicle according to the type of the target marker and the relative position includes:

[0030] determining position information of the target marker according to the type of the target marker and the relative position; wherein the position information includes a marker coordinate value;

[0031] querying a mapping relationship table of coordinate values and distances according to the marker coordinate value to obtain a distance between the target marker and a vehicle-mounted camera of the target vehicle;

[0032] calculating the initial visual positioning position information of the target vehicle according to the distance between the target marker and the vehicle-mounted camera of the target vehicle.

[0033] Optionally, the global coordinate conversion of the initial inertial positioning position information to obtain target inertial positioning position information and the global coordinate conversion of the initial visual positioning position information to obtain target visual positioning position information include:

[0034] extracting a first relative coordinate of the target vehicle from the initial inertial positioning position information; converting the first relative coordinate into a first global coordinate in a global coordinate system based on a coordinate transformation formula, as the target inertial positioning position information; and

[0035] extracting a second relative coordinate of the target vehicle from the initial visual positioning position information; converting the second relative coordinate into a second global coordinate in the global coordinate system based on the coordinate transformation formula, as the target visual positioning position information.

[0036] Optionally, the target inertial positioning position information, the target visual positioning position information and the vehicle ultra-wideband positioning information are fused to obtain vehicle positioning fusion position information of the target vehicle, and the method comprises the following steps of:

[0037] The target inertial positioning position information, the target visual positioning position information and the vehicle ultra-wideband positioning information are fused by using a federal Kalman filter with an open architecture to obtain vehicle positioning fusion position information of the target vehicle, wherein the federal Kalman filter comprises a plurality of sub-filters and a main filter.

[0038] In the federal Kalman filter, each sub-filter comprises a system state vector and a measurement vector, and state transition matrices and measurement matrices for time points k-1 to k are used to perform time update and measurement update to obtain state estimation values and error covariance matrices.

[0039] The main filter is configured to fuse filter results of the sub-filters to obtain vehicle positioning fusion position information of the target vehicle and predict vehicle positioning fusion position information at a future time point.

[0040] The present disclosure also provides a downhole vehicle positioning device, which comprises:

[0041] An acquisition module is configured to acquire vehicle electronic control unit information, vehicle video stream information and vehicle ultra-wideband positioning information of a target vehicle.

[0042] A first determination module is configured to determine initial inertial positioning position information of the target vehicle according to the vehicle electronic control unit information.

[0043] A second determination module is configured to determine initial visual positioning position information of the target vehicle according to the vehicle video stream information.

[0044] A conversion module is configured to perform global coordinate conversion on the initial inertial positioning position information to obtain target inertial positioning position information, and perform global coordinate conversion on the initial visual positioning position information to obtain target visual positioning position information.

[0045] A fusion module is configured to fuse the target inertial positioning position information, the target visual positioning position information and the vehicle ultra-wideband positioning information to obtain vehicle positioning fusion position information of the target vehicle.

[0046] The present disclosure also provides an electronic device, which comprises:

[0047] A processor.

[0048] a memory for storing the processor-executable instructions;

[0049] The processor is configured to execute the instructions to implement each step in the above method.

[0050] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement each step in the above method.

[0051] The technical solutions provided by the embodiments of the present disclosure have at least the following beneficial effects:

[0052] In some embodiments of the present disclosure, vehicle electronic control unit information, vehicle video stream information and vehicle ultra-wideband positioning information of a target vehicle are acquired; initial inertial positioning position information of the target vehicle is determined according to the vehicle electronic control unit information; initial visual positioning position information of the target vehicle is determined according to the vehicle video stream information; the initial inertial positioning position information is globally coordinate-converted to obtain target inertial positioning position information, and the initial visual positioning position information is globally coordinate-converted to obtain target visual positioning position information; the target inertial positioning position information, the target visual positioning position information and the vehicle ultra-wideband positioning information are fused to obtain vehicle positioning fusion position information of the target vehicle; the present disclosure combines the vehicle electronic control unit information, the vehicle video stream information and the vehicle ultra-wideband positioning information to realize multi-source positioning information fusion of the target vehicle, and improve the positioning accuracy of the vehicle in a motion state.

[0053] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0054] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure, and do not constitute an undue limitation on the present disclosure.

[0055] Figure 1 A flowchart of a downhole vehicle positioning method is provided for an exemplary embodiment of the present disclosure;

[0056] Figure 2 A schematic diagram of a downhole vehicle positioning system is provided for an exemplary embodiment of the present disclosure;

[0057] Figure 3 A flowchart of a marker tracking algorithm is provided for an exemplary embodiment of the present disclosure;

[0058] Figure 4 A schematic diagram of a feedback-free reset federated Kalman filter is provided for an exemplary embodiment of the present disclosure;

[0059] Figure 5 A structural schematic diagram of a downhole vehicle positioning device provided for an exemplary embodiment of the present disclosure is shown in the figure.

[0060] Figure 6 A structural schematic diagram of an electronic device provided for an exemplary embodiment of the present disclosure is shown in the figure. DETAILED DESCRIPTION

[0061] In order for those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below in conjunction with the accompanying drawings.

[0062] It should be noted that the terms "first", "second", and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Rather, they are merely examples of devices and methods consistent with some aspects of the present disclosure.

[0063] It should be noted that the user information involved in the present disclosure includes but is not limited to user device information and user personal information; the collection, storage, use, processing, transmission, provision and disclosure of user information in the present disclosure comply with the provisions of relevant laws and regulations, and do not violate public order and good customs.

[0064] In order to solve the above technical problems, in some embodiments of the present disclosure, vehicle electronic control unit information, vehicle video stream information and vehicle ultra-wideband positioning information of a target vehicle are acquired; initial inertial positioning position information of the target vehicle is determined according to the vehicle electronic control unit information; initial visual positioning position information of the target vehicle is determined according to the vehicle video stream information; the initial inertial positioning position information is globally coordinate-converted to obtain target inertial positioning position information, and the initial visual positioning position information is globally coordinate-converted to obtain target visual positioning position information; the target inertial positioning position information, the target visual positioning position information and the vehicle ultra-wideband positioning information are fused to obtain vehicle positioning fusion position information of the target vehicle; the present disclosure combines the vehicle electronic control unit information, the vehicle video stream information and the vehicle ultra-wideband positioning information to realize multi-source positioning information fusion of the target vehicle, and improve the positioning accuracy of the vehicle in a motion state.

[0065] The technical solutions provided by the embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings.

[0066] Figure 1A flowchart of a downhole vehicle positioning method is provided for the exemplary embodiments of the present disclosure. As shown in Figure 1 The method comprises the following steps:

[0067] S101: Obtain vehicle electronic control unit information, vehicle video stream information and vehicle ultra-wideband positioning information of a target vehicle;

[0068] S102: Determine initial inertial positioning position information of the target vehicle according to the vehicle electronic control unit information;

[0069] S103: Determine initial visual positioning position information of the target vehicle according to the vehicle video stream information;

[0070] S104: Perform global coordinate conversion on the initial inertial positioning position information to obtain target inertial positioning position information, and perform global coordinate conversion on the initial visual positioning position information to obtain target visual positioning position information;

[0071] S105: Perform position information fusion on the target inertial positioning position information, the target visual positioning position information and the vehicle ultra-wideband positioning information to obtain vehicle positioning fusion position information of the target vehicle.

[0072] In some embodiments of the present disclosure, the execution subject of the above method is a terminal device or a server.

[0073] The terminal device includes, but is not limited to, a mobile station (MS), a mobile terminal, a mobile telephone, a handset, a portable equipment, and the like. The terminal device can communicate with one or more core networks through a radio access network (RAN). For example, the terminal device can be a mobile phone (also referred to as a "cellular" phone), a computer with wireless communication function, and the like. The terminal device can also be a computer with wireless transceiver function, a virtual reality (VR) terminal device, an AR terminal device, a wireless terminal in industrial control, a wireless terminal in self driving, a wireless terminal in remote medical treatment, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, and the like. The operating system installed on the terminal device includes, but is not limited to, an IOS, an Android, a windows, a linux, a Mac OS, and the like. The terminal device can be referred to by different names in different networks, such as a user equipment, a mobile station, a subscriber unit, a station, a cellular phone, a personal digital assistant, a wireless modem, a wireless communication device, a handheld device, a laptop computer, a cordless phone, a wireless local loop station, a television, and the like. For the convenience of description, the terminal device is referred to as a terminal device in this embodiment.

[0074] In this embodiment, the implementation form of the server is not limited. For example, the server can be a conventional server, a cloud server, a cloud host, a virtual center, and the like. The server mainly includes a processor, a hard disk, a memory, a system bus, and the like, and has a general computer architecture type.

[0075] In the embodiments of the present disclosure, vehicle electronic control unit information, vehicle video stream information and vehicle ultra-wideband positioning information of a target vehicle are acquired; initial inertial positioning position information of the target vehicle is determined according to the vehicle electronic control unit information; initial visual positioning position information of the target vehicle is determined according to the vehicle video stream information; the initial inertial positioning position information is globally coordinate-converted to obtain target inertial positioning position information, and the initial visual positioning position information is globally coordinate-converted to obtain target visual positioning position information; the target inertial positioning position information, the target visual positioning position information and the vehicle ultra-wideband positioning information are fused to obtain vehicle positioning fusion position information of the target vehicle; the present disclosure combines the vehicle electronic control unit information, the vehicle video stream information and the vehicle ultra-wideband positioning information, realizes multi-source positioning information fusion of the target vehicle, and improves positioning accuracy of the vehicle in a motion state.

[0076] Figure 2 A schematic diagram of a downhole vehicle positioning system is provided for the exemplary embodiments of the present disclosure. As shown in FIG. 1, the downhole vehicle positioning system includes a vehicle electronic control unit 101, a vehicle video stream device 102, a vehicle ultra-wideband positioning device 103 and a vehicle positioning fusion device 104. Figure 2As shown, the lower vehicle positioning system includes: an inertial navigation positioning calculation module, a marker detection tracking module, a marker recognition module, a visual positioning calculation module, a coordinate system conversion module, and a positioning fusion module. The inertial navigation positioning calculation module completes the positioning calculation function based on the vehicle electronic control unit information. The vehicle's inertial positioning data is calculated using the ESKF (Error State Kalman Filter) algorithm through the vehicle's wheel speed, acceleration, direction angle and other information. During the positioning process, the inertial navigation module provides continuity and real-time of the vehicle position. The marker detection tracking module completes the detection and tracking of the set markers. This module detects and tracks the relative motion of the markers in the video stream. The marker recognition module completes the recognition of the markers. Based on the previous detection and tracking module, the module identifies the detected marker image and gives the recognized marker ID. The visual positioning calculation module calculates the position of the vehicle relative to the marker based on the existing marker calibration position information and visual marker recognition information, and calculates the vehicle visual positioning result to give the vehicle visual positioning position information. The coordinate system conversion module is used to convert the relative distance into absolute coordinates, which usually needs to be converted according to the known reference coordinate system (such as GPS coordinates, map coordinates, etc.). This module can accept the distance data output by the visual positioning calculation module and the inertial navigation positioning calculation module, and convert the relative position into global coordinates combined with the coordinate system information. The positioning fusion prediction module generates real-time continuous vehicle positioning fusion position information based on the vehicle ultra-wideband positioning information, vehicle visual positioning position information and inertial positioning position information, and predicts the vehicle's position information after dt time. This module uses visual positioning as calibration information and inertial position as continuous prediction information to realize the output of high-precision continuous position information. dt represents the time delay caused by data acquisition, transmission and processing, and the module will automatically compensate for the positioning error caused by time delay.

[0077] The present disclosure provides a set of fusion precision positioning system based on AI visual positioning. The system no longer needs to separately add positioning base stations, but only needs to integrate existing positioning system data, vehicle ECU data, etc. to realize the precision positioning function of the vehicle through fusion algorithm. The system includes a fusion precision positioning algorithm and an AI platform. The AI platform is mainly used to deploy the fusion precision positioning algorithm. The platform provides functions such as Internet of Things, video cloud, artificial intelligence, and device integration.

[0078] The AI fusion precise positioning system aims to improve the real-time positioning accuracy of transportation vehicles in coal mine tunnels and overcome the shortcomings of existing UWB positioning technology in dynamic environments. Based on advanced technologies such as 5G, AI, and IoT, the system comprehensively utilizes vehicle-mounted video streams, vehicle ECU data, and AI visual positioning models to provide dynamic and accurate real-time positioning information for transportation vehicles. By introducing inertial navigation and visual recognition algorithms and a delay compensation mechanism, the system can achieve continuous positioning and prediction of vehicles in dynamic conditions at a low cost, ensuring a positioning accuracy of no more than 7.3 meters, thereby meeting the safety and efficiency requirements of coal mine production.

[0079] Based on the AI visual positioning fusion precise positioning system application, the precise prediction and positioning of multiple vehicle positions are completed by receiving vehicle-mounted video stream information and vehicle ECU information based on the task management of the AI computing power management platform. First, the vehicle's video stream information and ECU information are transmitted to the AI computing power management platform through the 5G network. Then, the AI fusion precise positioning application is managed by the application image form, and its interface form needs to meet the requirements of the scheduling management platform. Finally, the vehicle position prediction and positioning results are sent to the intelligent human-computer interaction system and the intelligent auxiliary transportation management platform for displaying the vehicle position on the intelligent auxiliary transportation management platform.

[0080] The NTP time server completes the time service of all devices in the system and synchronizes the clock. It also corrects the system timestamp in the video stream and ECU data packet.

[0081] The vehicle electronic control unit information and vehicle-mounted camera video information utilize the characteristics of 5G network, such as large bandwidth, low latency, high reliability, and wide connection IoT, to design two precise positioning auxiliary schemes: AI visual positioning and vehicle electronic control unit information positioning. Through the fusion calculation of the above three positioning data sources, precise positioning of the vehicle in motion and at rest is achieved. This implementation scheme has the following characteristics:

[0082] In terms of scientificity, this scheme combines multiple positioning technologies, including UWB wireless positioning, AI visual positioning, and vehicle electronic control unit information positioning. These technologies have scientific basis and mature theoretical foundation. Through the fusion calculation of the three positioning data sources, the advantages of different technologies can be fully utilized to improve the accuracy and reliability of positioning, with high scientificity.

[0083] In terms of integrity, this scheme not only achieves precise positioning of vehicles in a stationary state but also solves the problem of excessive motion positioning error, meeting the comprehensive needs of coal mine vehicle positioning. From the selection of positioning technology to data fusion algorithms and process optimization in actual application, this scheme covers multiple aspects such as technology, algorithm, and application, with high integrity.

[0084] In terms of advancement, on the one hand, new technologies are introduced, AI visual positioning and vehicle electronic control unit information positioning are relatively advanced positioning technologies. AI visual positioning uses computer vision technology to identify and locate vehicles, with high precision, real-time performance, etc.; vehicle electronic control unit information positioning obtains position information through the vehicle's own electronic control unit, with high stability, high reliability, etc. On the other hand, the scheme integrates multiple advanced technologies to realize the innovation of the positioning system, improve the positioning accuracy and efficiency, and has high advancement.

[0085] In terms of practicality, in view of the problems existing in the original UWB wireless positioning system of the coal mine, the scheme puts forward an effective solution, which can effectively improve the accuracy and reliability of vehicle positioning, and meets the actual needs of coal mine production and management. And the design of the system considers the operation convenience in practical application, adopts the interface mode to crosslink with other systems, and has high practicality.

[0086] In terms of compatibility, first, compatibility with existing equipment: the system fully considers the compatibility with the original UWB wireless positioning system of the coal mine, which can realize the reuse of existing equipment, reducing the cost and difficulty of system deployment. Second, compatibility with other systems: the system considers the compatibility with other management systems of the coal mine in design, adopts standard protocols and interfaces, can realize data sharing and interaction, and improves the overall management level of the coal mine.

[0087] In terms of rationality, first, reasonable technology selection: according to the actual situation and needs of the coal mine, the appropriate positioning technology is selected and reasonably integrated. It not only guarantees the accuracy and reliability of positioning, but also considers the cost and implementation difficulty. Second, reasonable process design: the process design of the system is reasonable, which can realize the real-time collection, processing and transmission of vehicle positioning data, improve the efficiency and accuracy of positioning.

[0088] In terms of feasibility, the technology is feasible: the positioning technology adopted is mature technology with high feasibility. At the same time, the data fusion algorithm has been fully verified and tested, which can ensure the stable operation of the system. The implementation is feasible: the implementation process of the system considers the actual situation and resource conditions of the coal mine, and formulates a reasonable implementation plan and scheme, which has high feasibility.

[0089] In terms of effective use of existing resources, first, equipment reuse: the equipment of the original UWB wireless positioning system of the coal mine is fully utilized, reducing the cost and difficulty of system deployment. Second, data integration and reuse: the existing UWB positioning data is integrated with AI visual positioning and vehicle electronic control unit information positioning data for fusion calculation, improving the utilization value of data.

[0090] In terms of development capabilities, the development team has rich experience and technical strength in positioning system development, and can customize development according to the actual needs of coal mines to ensure the quality and performance of the system. At the same time, the development team also has good after-sales service capability and can timely solve problems that occur during system operation.

[0091] In some embodiments of the present disclosure, vehicle electronic control unit information, vehicle video stream information and vehicle ultra-wideband positioning information of the target vehicle are obtained. For example, by communicating with the vehicle electronic control unit (ECU) through the vehicle OBD (On-Board Diagnostics) interface or the CAN bus interface, the motion state information such as the speed, the heading angle, the acceleration and the steering angle of the vehicle is collected in real time; the vehicle video stream information is collected through the high-definition camera installed at the key visual angle position of the target vehicle and transmitted to the host unit through the image transmission interface (such as LVDS or Ethernet); at the same time, the ultra-wideband (UWB) positioning module is deployed on the target vehicle, and combined with multiple UWB anchors deployed in the environment, the high-precision real-time position data of the vehicle is obtained through the TOA (Time of Arrival) or TDOA (Time Difference of Arrival) positioning algorithm. Through the above-mentioned manner, the system can efficiently and synchronously obtain the multi-source positioning related information of the vehicle, and provide data support for subsequent positioning fusion and state estimation.

[0092] It should be noted that the vehicle electronic control unit information includes but is not limited to specific force, angular velocity, vehicle speed, engine state information, gearbox information, brake system information, steering system information, vehicle body attitude information, environmental perception information, safety system state and vehicle diagnostic information.

[0093] In some embodiments of the present disclosure, the initial inertial positioning position information of the target vehicle is determined according to the vehicle electronic control unit information. One implementable way is to obtain a given initial condition; input the given initial condition, specific force and angular velocity into a track recursion algorithm to obtain the initial inertial positioning position information of the target vehicle. Wherein, the vehicle ECU information (i.e. vehicle electronic control unit information) is used to obtain the specific force and angular velocity information of the vehicle, and the given initial condition is combined to calculate the real-time autonomous positioning system of the speed, position and attitude parameters. The inertial navigation positioning calculation belongs to a kind of reckoning navigation mode, that is, the position of a known point is calculated according to the continuously measured vehicle heading angle and speed, so that the current position of the moving body can be continuously measured.

[0094] Among them, the dead reckoning (DR) algorithm is the main implementation means of inertial navigation positioning calculation. The DR algorithm refers to that the navigation state (state, speed and position) at the last time is known, and the navigation state at the next time is calculated according to the sensor observation value. The DR algorithm includes two parts of attitude arrangement and position arrangement. The dead reckoning algorithm is intended to use the ESKF algorithm, the full name of which is Error state Kalman filter. Compared with the original Kalman filter, its advantages are as follows: the errors are all small quantities, and the second order can be ignored. A three-dimensional small quantity is used to represent the increment of rotation, instead of a quaternion or a rotation matrix. The ESKF is always near the origin, and the principle singularity point.

[0095] The basic flow of the dead reckoning algorithm of the present disclosure is as follows: initialize the state quantity, and initialize the Kalman state quantity. Here, if the gravity g is included, the pose can be initialized as the unit matrix I, otherwise, a reasonable initial pose R(0) needs to be determined. When an ECU data arrives, the ECU inertial navigation solution is performed, the state quantity x is updated, and the error state quantity ax is updated using the state transition matrix. At this time, if the state quantity needs to be returned, only x+ax needs to be returned, but ax=0 (the result will drift quickly without being updated at present). When a heterogenous observation data arrives, such as visual positioning information, the observation degree error state quantity needs to be corrected. The corrected ax' is obtained, and only x+ax' needs to be returned.

[0096] In some embodiments of the present disclosure, the initial visual positioning position information of the target vehicle is determined according to the vehicle video stream information. One implementable way is to perform marker detection and marker tracking according to the vehicle video stream information to obtain a target marker and a relative position between the target marker and the target vehicle; identify the target marker to obtain the type of the target marker; and determine the initial visual positioning position information of the target vehicle according to the type and the relative position of the target marker.

[0097] In the above embodiment, the target marker and the relative position of the target marker and the target vehicle are obtained by performing marker detection and marker tracking according to the vehicle-mounted video stream information. One implementable manner is that data enhancement processing is performed on the vehicle-mounted video stream information, and a set of anchor presets is initialized to obtain an input image for target detection; a feature extraction network is used to perform feature extraction on the input image to obtain a multi-scale feature map; a PANet structure is used to fuse the multi-scale feature map to obtain fused feature map information; based on a classification loss function and a bounding box regression loss function, parameters of a target detection network are adjusted by an optimizer to predict class and position information of a target marker; based on detection results of a current frame and historical frames, a Kalman filter is used to predict a motion state of the target marker; based on an intersection over union distance between a predicted bounding box and a current frame detection bounding box, a Hungarian algorithm is used for data association to match the detection result and an existing tracker; when the detection result and the existing tracker do not match, a new tracker is created and a new tracker motion state is initialized; and when any one tracker is not matched in continuous T frames, any one tracker is destroyed; and based on the detection result and the tracking result, the target marker and the relative position of the target marker and the target vehicle are determined. The marker detection and tracking module is a prerequisite for visual positioning and is divided into two main parts: a detection algorithm and a tracking algorithm.

[0098] To ensure the overall performance of the system, a high marker detection rate is required. Therefore, a target detection algorithm based on deep learning network is adopted. The main structure and function of the network are described as follows: 1) Preparation (in Input): The picture needs to be subjected to data enhancement (especially Mosaic data enhancement), and a set of anchor presets is initialized. 2) Feature extraction (in Backbone): The basic structures of Conv, C3 and SPPF are used for feature extraction of the input picture. Conv is used for down-sampling of the input (a total of 5 times of down-sampling); C3 is used for feature extraction and fusion of the input, enriching the semantic information of the features. In this process, Boottleneck is used to reduce the parameter quantity and calculation amount, and the idea of CSPNet is used to enhance the learning ability of CNN; SPPF uses pooling and feature fusion to enrich the semantic information of the features, so that the deepest feature map has extremely rich semantic information. 3) Processed features (in Neck): The three scale feature maps to be subjected to target detection are fused with shallow features (shallow features are beneficial to detection). PANet is used to fuse the extracted feature maps with shallow features, so that the feature maps have both rich semantic information and accurate position information of objects. 4) Prediction of target (in Head): The processed feature maps are predicted, and the parameter weights are optimized according to the loss function (ClassificitionLoss and BoundingBoxRegeressionLoss) and the optimizer.

[0099] For the marker tracking algorithm, a tracking-by-detection framework is adopted. The tracker only needs to use the detection results of the current and previous frames to implement the tracking algorithm. No appearance features are used in the tracking process, and only the position and size of the detection box are used in motion estimation and data association.

[0100] Figure 3 A flowchart of a marker tracking algorithm provided for the exemplary embodiments of the present disclosure is shown in FIG. 1. As shown in FIG. 1, the algorithm mainly includes four modules: a target detection module; a motion prediction module; a data association module; and a marker establishment and destruction module. Figure 3

[0101] ​The target detection module is the above-mentioned marker detection module, and an output of the target detection module is a marker position and size detected in an image. The motion prediction module, assuming that three markers are detected in a current frame, uses a Kalman filter or the like to predict states of the three objects in a next frame (or several frames). A motion model is used in the prediction process. If a detection result in the next frame is associated with one of the objects, the detection result is used as an observation value to update the state of the object. The data association module answers which object a current detection result is. Assuming that three objects are detected in a previous frame and four objects are detected in a current frame, a 3x4 matrix can be obtained, where each element in the matrix represents an IoU distance (Intersection-over-union distance) between a predicted bounding box and a current bounding box. Based on the matrix, in order to minimize the total IoU distance, a Hungarian algorithm can be used for matching / assignment, so as to complete data association. The marker establishment and destruction module, if an IoU between a detection result and all existing trackers is small, a new tracker is established based on the detection result. A speed of the object is initialized to 0, and a speed-related covariance component is initialized to a large value. If a tracked object (tracker) is not detected and associated for T frames, the tracker is destroyed.

[0102] In the above embodiment, a target marker is identified to obtain a type of the target marker. The marker identification module aims to identify which marker is in a region of interest cropped by the detection and tracking module. In order to improve the identification rate, a deep learning model is used to implement the marker identification, and a network structure such as FaceNet is referred to to complete the marker identification. A training process of the identification network can be roughly divided into the following steps.

[0103] Data preprocessing: images in a marker data set often contain entire markers and part of an environment background, and are often tilted, rotated, or the like. Before the images are input to the model, the marker part of the images needs to be cut, irrelevant background information needs to be removed, and the cut marker images need to be aligned, and then the images can be used for training.

[0104] Loading a data set according to a specific rule: the system uses a new data import method: two matched images (two images of the same marker) and one unmatched image (one image of another marker) are used as a triplet, and the data set is loaded in groups.

[0105] Extracting marker feature information using large backbone network: a suitable deep convolutional neural network is selected, and the output layer is modified for feature extraction. Common deep convolutional neural networks include ResNet series, Inception series, etc.

[0106] L2 norm normalization of marker feature information: the result output by the deep convolutional neural network is a feature vector in 128-dimensional hyperspace, which needs to be normalized to 128-dimensional hypersphere using L2 regularization.

[0107] Loss calculation and gradient update: using a specific "ternary loss function" can make the model quickly optimize in the target direction.

[0108] In the above embodiment, according to the type and relative position of the target marker, the initial visual positioning position information of the target vehicle is determined. One possible way is to determine the position information of the target marker according to the type and relative position of the target marker; wherein the position information includes: marker coordinate value; according to the marker coordinate value, the mapping relationship table of coordinate value and distance is queried to obtain the distance between the target marker and the vehicle-mounted camera of the target vehicle; according to the distance between the target marker and the vehicle-mounted camera of the target vehicle, the initial visual positioning position information of the target vehicle is calculated. The visual positioning calculation module refers to calculating the position of the vehicle relative to the marker based on the existing marker calibration position information and visual marker recognition information, and calculating the visual positioning result of the vehicle. Including three calculation steps: querying the position information x of the current marker calibration; according to the coordinate value of the marker in the image, the distance dx between the marker and the vehicle-mounted camera is calculated by table lookup; the position information of the vehicle is calculated.

[0109] In some embodiments of the present disclosure, the initial inertial positioning position information is globally coordinate-converted to obtain target inertial positioning position information, and the initial visual positioning position information is globally coordinate-converted to obtain target visual positioning position information. One implementable way is to extract first relative coordinates of the target vehicle from the initial inertial positioning position information; convert the first relative coordinates into first global coordinates in a global coordinate system based on a coordinate transformation formula, as the target inertial positioning position information; and extract second relative coordinates of the target vehicle from the initial visual positioning position information; convert the second relative coordinates into second global coordinates in the global coordinate system based on the coordinate transformation formula, as the target visual positioning position information. Wherein, the coordinate conversion module is used to convert the vehicle relative distance data output by the visual positioning calculation module and the inertial navigation positioning calculation module into the vehicle position in the coordinate system. Through this module, the relative position can be combined with the known coordinates of the markers to calculate the actual position of the vehicle in the coordinate system, thereby providing data support for further positioning fusion. Including three calculation steps: obtaining relative distance information; combining the global coordinates of the markers, using appropriate coordinate transformation formulas (such as rotation matrix, translation, etc.), converting the relative coordinates of the vehicle into coordinate values in the global coordinate system. This calculation takes into account the orientation of the vehicle and the known position of the markers, ensuring that the conversion result is consistent with the global coordinate system. The calculated global coordinate information of the vehicle is transmitted to the positioning fusion prediction module for integrating data from different sensors (such as vision, inertial navigation, etc.), improving positioning accuracy, and ultimately obtaining the accurate position of the vehicle.

[0110] In some embodiments of the present disclosure, target inertial positioning position information, target visual positioning position information and vehicle ultra-wideband positioning information are fused to obtain vehicle positioning fusion position information of the target vehicle. One possible implementation is that a federated Kalman filter with an open architecture is used to fuse the target inertial positioning position information, the target visual positioning position information and the vehicle ultra-wideband positioning information to obtain the vehicle positioning fusion position information of the target vehicle; wherein the federated Kalman filter comprises a plurality of sub-filters and a master filter; in the federated Kalman filter, each sub-filter comprises a system state vector and a measurement vector, and a state transition matrix and a measurement matrix representing time instants k-1 to k, for performing time update and measurement update to obtain a state estimate value and an error covariance matrix; the master filter is used to globally fuse the filter results of each sub-filter to obtain the vehicle positioning fusion position information of the target vehicle and predict the vehicle positioning fusion position information at a future time instant. The positioning fusion prediction module is based on the vehicle ultra-wideband positioning information, the visual positioning position information and the inertial positioning position information to generate real-time continuous vehicle positioning fusion position information, and can predict the position fusion error caused by information time delay and automatically compensate the vehicle position at the current time instant. In order to improve the fault tolerance and information source expansion capability of the accurate positioning system, a federated Kalman filter is designed, an open algorithm architecture is used, and typical vehicle positioning information sources are selected for fusion. The algorithm framework takes a common reference system as a benchmark, and respectively forms sub-filters with other information sources. The output results of the sub-filters are input into the master filter for information fusion after fault diagnosis and system reconstruction.

[0111] Figure 4 A schematic diagram of a no-feedback reset federated Kalman filter is provided for the exemplary embodiments of the present disclosure. As shown in Figure 4 , each local filter is independent of each other without mutual influence caused by feedback reset, which provides the highest fault tolerance. The corresponding discretization system error model of the system is as follows:

[0112]

[0113] In the formula, is the system state vector of the i-th sub-filter; is the measurement vector of the i-th sub-filter; Φk|k-1 represents a one-step state transition matrix from time instant k-1 to time instant k; is the measurement matrix. The i sub-filters independently perform time update and measurement update to obtain a state estimate value and an error covariance matrix

[0114] When the recursion in time is completed in the global filter, the fusion position information of the vehicle at the future time needs to be predicted. When the time update is completed, the system needs to calculate the difference between the current algorithm platform clock and the time stamp of each data packet, dt = T - t, T represents the current algorithm platform time, t represents the current data packet time stamp, the fusion data at time t is needed to complete the prediction of the vehicle position at the current time T = t + dt, and the time delay position compensation is completed.

[0115] Through the processes of time update, measurement update, information fusion, and information distribution, part of the information is lost in the noise distribution process of the sub-filter, and the filtering result is suboptimal. In the fusion process of the sub-filter results of each information source, the non-optimality is recombined to obtain a globally optimal estimate.

[0116] The functions of the present disclosure mainly include: high-precision positioning function, safety warning and alarm function, trajectory recording and backtracking function, data storage and query function, and driving map generation function. The specific descriptions are as follows: high-precision positioning function, the system provides high-precision vehicle positioning service through the original UWB positioning system and the newly added AI multi-element fusion and time delay compensation function. In a static environment, the positioning accuracy can be within 0.3 meters; in the process of dynamic driving, the positioning accuracy is not more than 7.3 meters, which meets the positioning needs of coal mine transport vehicles in complex roadway environments. Safety warning and alarm function, the system monitors the relative distance between vehicles and personnel in real time. When the distance between vehicles is less than 150 meters, or the distance between the vehicle and the personnel is less than 100 meters, the system will automatically trigger a safety distance warning to remind the driver and relevant personnel, and ensure the safety of coal mine operations. Trajectory recording and backtracking function, the system provides a recording interface for real-time trajectory and position information of the vehicle, allowing the system to record the running trajectory data of the vehicle and supporting subsequent trajectory backtracking, driving analysis and accident tracking. At the same time, these data can be used to generate a driving map of the vehicle, providing comprehensive analysis support for the management side to understand the running situation of the vehicle in the roadway. Data storage and query function, the system stores the positioning data and running trajectory data of the vehicle in real time, which is convenient for querying the historical position records of the vehicle at any time, and provides support for historical data analysis of vehicle operation, and assists in formulating optimization schemes. These functions together constitute the core module of the system, ensuring the safe and efficient operation of coal mine transport vehicles, and providing reliable data support for management decisions.

[0117] The present disclosure makes full use of the existing UWB positioning system and camera equipment, without the need for additional new positioning base stations. This not only saves the cost of equipment procurement and installation, but also shortens the deployment time of the system. At the same time, with the help of existing infrastructure, the compatibility and stability of the system are ensured, and the technical risks and maintenance complexity that may be brought about by the introduction of new equipment are reduced. By optimizing the use of existing resources, the project realizes efficient resource integration, and improves the economic benefit of the overall system.

[0118] High-precision positioning in static and dynamic environments. In the static state of the vehicle, based on UWB positioning technology, the positioning accuracy is guaranteed to be within 0.3 meters, meeting the requirements of high-precision static positioning in complex environments such as coal mines. In the dynamic driving state of the vehicle, the system uses AI multi-element fusion technology, including target detection and recognition positioning, camera vision assistance, and ECU data for inertial navigation calculation, etc. Multi-source data fusion, to achieve accurate positioning of the vehicle. Through multi-source information fusion, the positioning accuracy in dynamic state reaches 7.3 meters, ensuring the continuity and accuracy of positioning, providing reliable support for complex dynamic application scenarios.

[0119] Multi-source fusion of AI vision and inertial navigation data. Through AI vision technology, the data collected by the camera is detected for target detection, combined with the vehicle acceleration, speed, and other information collected by the ECU for inertial navigation positioning, a multi-source information fusion positioning scheme is constructed. This technology effectively improves the accuracy and stability of positioning in dynamic environments, solving the problem of precision decline of single positioning technology in dynamic scenarios, thereby ensuring the robustness of the system while meeting the dynamic precise positioning requirements.

[0120] Time delay compensation mechanism ensures positioning accuracy. In the system design, a time delay compensation mechanism is introduced to ensure the real-time performance of data transmission and processing. Due to the possibility of delay in network transmission and data processing, the system corrects the positioning data through the time delay compensation algorithm to ensure the accuracy and synchronization of the vehicle position information. Combined with AI multi-element fusion technology, the time delay compensation mechanism can effectively reduce the influence of transmission delay on positioning results, providing more accurate vehicle position data for real-time monitoring, and improving the reliability and response speed of positioning.

[0121] Flexible system scalability. The system is scalable in size; each vehicle data is processed independently using application mirroring, with good scalability. When the coal mine scale expands and the number of management vehicles increases, only by increasing the server computing power and adding the number of application mirrors can the vehicle processing capacity be expanded. The scalability of other data sources, the system design reserves multiple data interfaces, which can easily access other data sources such as sensor data, monitoring video streams, GPS data, etc., supporting seamless integration with existing systems. Through modular and standardized interface design, new data source access can be easily realized through configuration files or plug-in methods. In this way, the system not only adapts to the growth of vehicle data, but also can further enhance the functionality and intelligent level of the system by accessing more data sources to meet changing business needs.

[0122] In the entire vehicle positioning system, the data flow starts from multiple input modules and goes through multiple algorithm processing modules to achieve accurate vehicle positioning. First, dynamic data from the vehicle ECU (such as wheel speed, acceleration, direction angle, etc.) and video stream data from the vehicle-mounted camera are received. ECU data is used for inertial navigation positioning, while video stream is used for detecting surrounding markers. In the algorithm processing module, the inertial navigation positioning calculation module calculates the inertial navigation positioning data of the vehicle by receiving ECU data and using the error state Kalman filter (ESKF) algorithm, ensuring the continuity and real-time of the vehicle position in the case of marker occlusion or absence in the video picture. Then, the marker detection and tracking module detects and tracks the preset markers in the video stream data, identifies the relative position of these markers relative to the vehicle, and transmits the detection results to the next module. The marker recognition module assigns a unique ID to the detected markers to ensure that the system can distinguish between different markers and transmit the recognition results to the visual positioning module. In the visual positioning calculation module, the accurate position of the vehicle relative to the markers is calculated based on the recognition ID and known position of the markers, generating visual positioning data. The inertial navigation positioning and visual positioning data are transmitted to the coordinate conversion module to convert the relative position to an accurate coordinate system position. Finally, in the positioning fusion and prediction module, the converted inertial navigation positioning and visual positioning data are fused to generate high-precision real-time vehicle positioning information. This module also performs time delay compensation to eliminate the delay in data processing and transmission, and uses inertial navigation information to predict the future position of the vehicle in the short term, ensuring continuous output of data. Through multi-source data fusion, positioning algorithm and time delay compensation mechanism, the entire system can provide high-precision vehicle positioning services in complex environments.

[0123] Figure 5 A structural diagram of a downhole vehicle positioning device 50 is provided for the exemplary embodiments of the present disclosure. As shown, the downhole vehicle positioning device 50 includes an acquisition module 51, a first determination module 52, a second determination module 53, a conversion module 54, and a fusion module 55. Figure 5

[0124] The acquisition module 51 is configured to acquire vehicle electronic control unit information, vehicle video stream information, and vehicle ultra-wideband positioning information of a target vehicle.

[0125] The first determination module 52 is configured to determine initial inertial positioning position information of the target vehicle based on the vehicle electronic control unit information.

[0126] The second determination module 53 is configured to determine initial visual positioning position information of the target vehicle based on the vehicle video stream information.

[0127] ​The conversion module 54 is configured to perform global coordinate conversion on the initial inertial positioning position information to obtain target inertial positioning position information, and perform global coordinate conversion on the initial visual positioning position information to obtain target visual positioning position information.

[0128] The fusion module 55 is configured to perform position information fusion on the target inertial positioning position information, the target visual positioning position information, and the vehicle ultra-wideband positioning information to obtain vehicle positioning fusion position information of the target vehicle.

[0129] As to the apparatus in the above embodiments, the specific manners in which the various modules perform operations have been described in detail in the embodiments of the method, and thus will not be described in detail here.

[0130] Figure 6 A structural schematic diagram of an electronic device is provided for the exemplary embodiments of the present disclosure. As shown in the figure, the electronic device includes a memory 61 and a processor 62. In addition, the electronic device also includes a power supply component 63 and a communication component 64. Figure 6

[0131] The memory 61 is configured to store computer programs and can be configured to store other various data to support operations on the electronic device. Examples of the data include instructions of any application program or method for operating on the electronic device.

[0132] The memory 61 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0133] The communication component 64 is configured to perform data transmission with other devices.

[0134] The processor 62 can execute computer instructions stored in the memory 61, so as to: acquire vehicle electronic control unit information, vehicle video stream information and vehicle ultra-wideband positioning information of a target vehicle; determine initial inertial positioning position information of the target vehicle according to the vehicle electronic control unit information; determine initial visual positioning position information of the target vehicle according to the vehicle video stream information; perform global coordinate conversion on the initial inertial positioning position information to obtain target inertial positioning position information, and perform global coordinate conversion on the initial visual positioning position information to obtain target visual positioning position information; and perform position information fusion on the target inertial positioning position information, the target visual positioning position information and the vehicle ultra-wideband positioning information to obtain vehicle positioning fusion position information of the target vehicle.

[0135] ​Accordingly, the embodiments of the present disclosure further provide a computer readable storage medium storing a computer program. When the computer readable storage medium stores the computer program, and the computer program is executed by one or more processors, the one or more processors are caused to perform the steps of the method embodiments. Figure 1 Steps in the method embodiments.

[0136] Accordingly, the embodiments of the present disclosure further provide a computer program product, which includes a computer program / instruction, and the computer program / instruction is executed by a processor to cause the processor to perform the steps of the method embodiments. Figure 1 Steps in the method embodiments.

[0137] The communication component in the above Figure 6 The communication component in the above

[0138] The power supply component in the above Figure 6 The power supply component in the above

[0139] The electronic device in the above

[0140] The display screen in the above

[0141] The audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) that is configured to receive an external audio signal when a device in which the audio component is included is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory or transmitted via the communication component. In some embodiments, the audio component further includes a speaker for outputting audio signals.

[0142] Those skilled in the art will appreciate that embodiments of the disclosure can be supplied as a method, a system, or a computer program product. Accordingly, the disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the disclosure can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code.

[0143] The disclosure is described in reference to the flowchart and / or block diagrams of the method, apparatus (system) and computer program product according to embodiments of the disclosure. It should be understood that each flow and / or block in the flowchart and / or block diagram, and a combination of flows and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions that are executed by the processor of the computer or other programmable data processing apparatus generate an apparatus that implements the flow Figure 1 The flow or flows and / or blocks in a flowchart and / or a combination of flows and / or blocks in a flowchart can represent a module, a segment, or a portion of code, which includes one or more executable instructions for implementing the specified logical function(s). Figure 1 The flow or flows and / or blocks in a flowchart and / or a combination of flows and / or blocks in a flowchart can represent a module, a segment, or a portion of code, which includes one or more executable instructions for implementing the specified logical function(s).

[0144] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which are executed on the computer or other programmable apparatus provide steps for implementing the flow Figure 1 The flow or flows and / or blocks in a flowchart and / or a combination of flows and / or blocks in a flowchart can represent a module, a segment, or a portion of code, which includes one or more executable instructions for implementing the specified logical function(s). Figure 1 The flow or flows and / or blocks in a flowchart and / or a combination of flows and / or blocks in a flowchart can represent a module, a segment, or a portion of code, which includes one or more executable instructions for implementing the specified logical function(s).

[0145] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which are executed on the computer or other programmable apparatus provide steps for implementing the flow Figure 1 The flow or flows and / or blocks in a flowchart and / or a combination of flows and / or blocks in a flowchart can represent a module, a segment, or a portion of code, which includes one or more executable instructions for implementing the specified logical function(s). Figure 1 The flow or flows and / or blocks in a flowchart and / or a combination of flows and / or blocks in a flowchart can represent a module, a segment, or a portion of code, which includes one or more executable instructions for implementing the specified logical function(s).

[0146] In one typical arrangement, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0147] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) having a common memory address space and / or non-volatile memory, such as read-only memory (ROM), flash memory, and the like, embodied in computer-readable media. The memory is an example of computer-readable media.

[0148] Computer-readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.

[0149] It should be noted that, in the present document, relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without limitation, an element preceded by "comprises... a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0150] The foregoing is merely illustrative of the various implementations of the present disclosure and the general principles thereof. Numerous modifications can be made to these illustrations, and equivalents can be substituted therefor, without departing from the scope of the present disclosure. The specific embodiments commensurate with the specific application are intended to be illustrative only and not limiting of the scope of the application as set forth in the following claims.

Claims

1. A method of locating a vehicle down a well, characterised by, The method comprises the following steps: acquiring vehicle electronic control unit information, vehicle video stream information and vehicle ultra-wideband positioning information of a target vehicle; determining initial inertial positioning position information of the target vehicle according to the vehicle electronic control unit information; determining initial visual positioning position information of the target vehicle according to the vehicle video stream information; performing global coordinate conversion on the initial inertial positioning position information to obtain target inertial positioning position information, and performing global coordinate conversion on the initial visual positioning position information to obtain target visual positioning position information; performing position information fusion on the target inertial positioning position information, the target visual positioning position information and the vehicle ultra-wideband positioning information to obtain vehicle positioning fusion position information of the target vehicle.

2. The method of claim 1, wherein, The vehicle electronic control unit information comprises specific force and angular velocity; the determination of the initial inertial positioning position information of the target vehicle according to the vehicle electronic control unit information comprises the following steps: acquiring a given initial condition; inputting the given initial condition, the specific force and the angular velocity into a track recursion algorithm to obtain the initial inertial positioning position information of the target vehicle.

3. The method of claim 1, wherein, The determination of the initial visual positioning position information of the target vehicle according to the vehicle video stream information comprises the following steps: performing marker detection and marker tracking according to the vehicle video stream information to obtain a target marker and a relative position between the target marker and the target vehicle; identifying the target marker to obtain a type of the target marker; determining the initial visual positioning position information of the target vehicle according to the type of the target marker and the relative position.

4. The method of claim 3, wherein, The marker detection and marker tracking according to the vehicle video stream information to obtain a target marker and a relative position between the target marker and the target vehicle comprise the following steps: performing data enhancement processing on the vehicle video stream information and initializing a group of anchor presets to obtain an input image for target detection; extracting features from the input image by using a feature extraction network to obtain multi-scale feature maps; fusing the multi-scale feature maps by using a PANet structure to obtain fusion feature map information; adjusting parameters of a target detection network by using an optimizer based on a classification loss function and a bounding box regression loss function to predict a category and position information of the target marker; predicting a motion state of the target marker by using Kalman filtering based on detection results of a current frame and historical frames; performing data association by using a Hungarian algorithm according to an intersection over union distance between a predicted bounding box and a current frame detection bounding box to match the detection results and existing trackers; when the detection results do not match the existing trackers, creating a new tracker and initializing a motion state of the new tracker; and when any one tracker is not matched in consecutive T frames, destroying the any one tracker; determining a target marker and a relative position between the target marker and the target vehicle according to the detection results and tracking results.

5. The method of claim 3, wherein, The initial visual positioning position information of the target vehicle is determined according to the type of the target marker and the relative position, and the method comprises the steps of: According to the type of the target marker and the relative position, the position information of the target marker is determined; wherein the position information comprises: marker coordinate value; According to the marker coordinate value, the mapping relationship table of coordinate value and distance is queried to obtain the distance between the target marker and the vehicle-mounted camera of the target vehicle; According to the distance between the target marker and the vehicle-mounted camera of the target vehicle, the initial visual positioning position information of the target vehicle is calculated.

6. The method of claim 1, wherein, The initial inertial positioning position information is globally coordinate-converted to obtain target inertial positioning position information, and the initial visual positioning position information is globally coordinate-converted to obtain target visual positioning position information, and the method comprises the steps of: The first relative coordinate of the target vehicle is extracted from the initial inertial positioning position information; the first relative coordinate is converted into the first global coordinate in the global coordinate system based on the coordinate transformation formula, and the target inertial positioning position information is obtained; and The second relative coordinate of the target vehicle is extracted from the initial visual positioning position information; the second relative coordinate is converted into the second global coordinate in the global coordinate system based on the coordinate transformation formula, and the target visual positioning position information is obtained.

7. The method of claim 1, wherein, The target inertial positioning position information, the target visual positioning position information and the vehicle ultra-wideband positioning information are fused to obtain the vehicle positioning fusion position information of the target vehicle, and the method comprises the steps of: An open-architecture federal Kalman filter is used to fuse the target inertial positioning position information, the target visual positioning position information and the vehicle ultra-wideband positioning information to obtain the vehicle positioning fusion position information of the target vehicle; wherein the federal Kalman filter comprises: a plurality of sub-filters and a main filter; In the federal Kalman filter, each sub-filter comprises a system state vector and a measurement vector, and a state transition matrix and a measurement matrix for representing time k-1 to time k, for performing time update and measurement update to obtain state estimation value and error covariance matrix; The main filter is used to globally fuse the filter results of each sub-filter to obtain the vehicle positioning fusion position information of the target vehicle, and predict the vehicle positioning fusion position information at a future time.

8. A downhole vehicle positioning apparatus, characterized by Comprise: An acquisition module is configured to acquire vehicle electronic control unit information, vehicle video stream information and vehicle ultra-wideband positioning information of a target vehicle; A first determination module is configured to determine initial inertial positioning position information of the target vehicle according to the vehicle electronic control unit information; A second determination module is configured to determine initial visual positioning position information of the target vehicle according to the vehicle video stream information; A conversion module is configured to globally coordinate-convert the initial inertial positioning position information to obtain target inertial positioning position information, and globally coordinate-convert the initial visual positioning position information to obtain target visual positioning position information; A fusion module is configured to fuse the target inertial positioning position information, the target visual positioning position information and the vehicle ultra-wideband positioning information to obtain vehicle positioning fusion position information of the target vehicle.

9. An electronic device, comprising: The computer program product comprises: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the steps in the method of any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps in the method of any one of claims 1-7.

Images