Cabin door positioning method and device based on confidence fusion and related equipment

Abstract

The invention provides a cabin door positioning method and device based on confidence fusion and related equipment, and relates to the technical field of cabin door positioning. The method comprises the steps that motion information of the boarding bridge and environment information of the environment where the boarding bridge is located are acquired; determining a plurality of confidence coefficients according to the motion information and / or the environment information; fusing the plurality of confidence coefficients to obtain a comprehensive confidence coefficient; according to the comprehensive confidence coefficient and a set decision-making mechanism, cabin door feature point coordinate information is obtained based on first coordinate information and / or second coordinate information, the first coordinate information is cabin door feature point coordinate information obtained through a visual odometer, and the second coordinate information is cabin door feature point coordinate information obtained based on a cabin door recognition model. According to the method, the advantages of accurately predicting cabin door coordinate information by a model and estimating cabin door coordinates in real time by visual tracking can be reserved, more accurate cabin door coordinate information can be quickly obtained, and both accuracy and high efficiency are considered.

Description

Technical Field

[0001] This disclosure relates to the field of hatch positioning technology, and in particular to a hatch positioning method, apparatus and related equipment based on confidence fusion. Background Technology

[0002] Boarding bridges are used to connect terminal boarding gates to aircraft doors. They are specialized docking devices that serve aircraft and provide a safe and efficient passage for personnel and cargo.

[0003] During the docking of the boarding bridge with the cabin door, it is necessary to quickly and accurately locate the door's position so that the boarding bridge and door can automatically dock. The cabin door positioning methods provided in related technologies suffer from the problem of difficulty in simultaneously achieving real-time performance and positioning accuracy.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This disclosure provides a door positioning method, apparatus, and related equipment based on confidence fusion, which at least to some extent overcomes the technical problem of door positioning methods provided in related technologies, which have difficulty in balancing real-time performance and positioning accuracy.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to one aspect of this disclosure, a door positioning method based on confidence fusion is provided, comprising: acquiring motion information of a boarding bridge and environmental information of the environment in which the boarding bridge is located; determining multiple confidence parameters based on the motion information and / or the environmental information; fusing the multiple confidence parameters to obtain a comprehensive confidence score; and obtaining door feature point coordinate information based on a first coordinate information and / or a second coordinate information based on the comprehensive confidence score and a set decision mechanism, wherein the first coordinate information is door feature point coordinate information obtained by visual odometry, and the second coordinate information is door feature point coordinate information obtained based on a door recognition model.

[0008] In some embodiments, based on the comprehensive confidence level and a set decision mechanism, the door feature point coordinate information is obtained based on the first coordinate information and / or the second coordinate information, including: if the comprehensive confidence level is less than a first threshold, the second coordinate information is directly determined as the fused door feature point coordinate information; if the comprehensive confidence level is greater than the first threshold, the first coordinate information is directly determined as the fused door feature point coordinate information, and the second threshold is greater than the first threshold; if the comprehensive confidence level is greater than or equal to the first threshold and less than or equal to the second threshold, the first coordinate information and the second coordinate information are fused to obtain the fused door feature point coordinate information.

[0009] In some embodiments, fusing the first coordinate information and the second coordinate information to obtain fused hatch feature point coordinate information includes: determining first weight information and second weight information, wherein the first weight information is the weight information corresponding to the first coordinate information, and the second coordinate information is the weight information corresponding to the second coordinate information; and performing a weighted summation on the first coordinate information and the second coordinate information according to the first weight information and the second weight information to obtain fused hatch feature point coordinate information.

[0010] In some embodiments, the first coordinate information and the second coordinate information are weighted and summed using the following formula: ; ;in, This indicates that the coordinates of the fused hatch feature points have been obtained; Indicates the second coordinate information; Indicates the first coordinate information; This indicates the fusion weight of the second coordinate information; This indicates the fusion weight of the first coordinate information; Indicates the overall confidence level; This represents a coefficient, with a value ranging from 0 to 1.

[0011] In some embodiments, the first coordinate information and the second coordinate information are weighted and summed using the following formula: ; ;in, This indicates the coordinate information of the fused hatch feature points; This represents the coordinate information of the hatch feature points derived from the hatch recognition model, i.e., the second coordinate information; This represents the coordinate information of the hatch feature points obtained through visual odometry, i.e., the first coordinate information; This represents the evaluation score of the model's prediction results; The evaluation score represents the visual tracking results; This represents the evaluation score corresponding to coordinate fusion.

[0012] In some embodiments, the method further includes: obtaining the camera rotation matrix and camera translation vector of the current frame hatch image; determining the coordinate information of the hatch feature points in the current frame hatch image based on the camera rotation matrix and camera translation vector of the current frame hatch image, as well as the coordinate information of the hatch feature points in the reference frame hatch image, and the camera rotation matrix and camera translation vector of the reference frame hatch image, to obtain the first coordinate information.

[0013] In some embodiments, the confidence level includes at least one of the following: spatial consistency confidence level, texture sharpness confidence level, historical accuracy confidence level, environmental adaptability confidence level, and binocular matching confidence level.

[0014] In some embodiments, the door positioning method based on confidence fusion provided in this disclosure can calculate the comprehensive confidence level using the following formula: ;in, Indicates the overall confidence level; Indicates the confidence level of spatial consistency; Indicates confidence level in texture sharpness; Indicates the confidence level of historical accuracy; Indicates the confidence level of environmental adaptability; Indicates the confidence level of binocular matching; The fusion weights represent the spatial consistency confidence levels; The fusion weights represent the confidence level of texture sharpness; The fusion weights represent the confidence levels of historical accuracy. The fusion weights represent the confidence levels of environmental adaptability; This represents the fusion weight for binocular matching confidence.

[0015] Spatial consistency confidence is calculated using the following formula. Texture sharpness confidence Historical accuracy confidence level Confidence level of environmental adaptability Binocular matching confidence : ; ; ; ; ; in, Indicates the first Frame pose information; Indicates the first Frame pose information; Indicates the scale parameter; Representing an image The gradient is used to reflect the edge texture information of the image; This represents the maximum value of the image gradient; Represents the norm of the image gradient matrix; Indicates the number of frames in the sliding window; Indicates the frame number of the current frame; Indicates the frame number within the sliding window; Indicates the first The coordinate information of the hatch feature points obtained by frame tracking; Indicates the first The frame reference contains the coordinates of the hatch feature points. This indicates the intensity used to measure rain and fog conditions; Indicates the first The feature point set of a frame-based binocular system; Indicates the first The feature point set of a frame-based binocular system; This indicates intersection, union, and ratio.

[0016] According to another aspect of this disclosure, a door positioning device based on confidence fusion is also provided, comprising: a data acquisition module for acquiring motion information of a boarding bridge and environmental information of the environment in which the boarding bridge is located; a confidence determination module for determining multiple confidence parameters based on the motion information and / or the environmental information; a confidence fusion module for fusing the multiple confidence parameters to obtain a comprehensive confidence score; and a door coordinate information determination module for obtaining door feature point coordinate information based on a first coordinate information and / or a second coordinate information, according to the comprehensive confidence score and a set decision mechanism, wherein the first coordinate information is door feature point coordinate information obtained by visual odometry, and the second coordinate information is door feature point coordinate information obtained based on a door recognition model.

[0017] According to another aspect of this disclosure, an electronic device is also provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the confidence fusion-based door positioning method described above by executing the executable instructions.

[0018] According to another aspect of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the door positioning method based on confidence fusion as described in any one of the preceding claims.

[0019] According to another aspect of this disclosure, a computer program product is also provided, comprising: a computer program or instructions that, when executed by a processor, implement the confidence fusion-based hatch positioning method described in any one of the preceding claims.

[0020] The door positioning method, apparatus, and related equipment based on confidence fusion provided in this disclosure firstly acquire the motion information of the boarding bridge and the environmental information of the environment in which the boarding bridge is located. Then, based on the motion information of the boarding bridge and / or the environmental information of the environment in which the boarding bridge is located, multiple confidence parameters in the time dimension and / or spatial dimension are determined. Then, the multiple confidence parameters are fused to obtain a comprehensive confidence score. Finally, based on the comprehensive confidence score and a set decision mechanism, door feature point coordinate information is obtained based on first coordinate information and / or second coordinate information. The first coordinate information is the door feature point coordinate information obtained by visual odometry, and the second coordinate information is the door feature point coordinate information obtained based on a door recognition model.

[0021] Through the embodiments of this disclosure, based on the motion information of the boarding bridge itself and the environmental information of the environment in which the boarding bridge is located, the door feature point coordinate information obtained by visual odometry and / or the door feature point coordinate information obtained based on the door recognition model are fused. This can retain the advantages of the model accurately predicting the door coordinate information and visual tracking estimating the door coordinate in real time, and quickly obtain more accurate door coordinate information, taking into account both accuracy and efficiency.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0024] Figure 1 This diagram illustrates an application system architecture according to an embodiment of the present disclosure. Figure 2 A flowchart of a door positioning method based on confidence fusion is shown in an embodiment of this disclosure; Figure 3 This document illustrates a flowchart illustrating how a method for determining the coordinates of hatch feature points based on a comprehensive confidence level and a set decision-making mechanism, as described in an embodiment of this disclosure. Figure 4 This illustration shows an optional coordinate fusion information fusion flowchart in an embodiment of the present disclosure; Figure 5 This diagram illustrates a hatch feature point according to an embodiment of the present disclosure; Figure 6 This diagram illustrates a confidence fusion framework according to an embodiment of the present disclosure. Figure 7 This document illustrates a flowchart of an embodiment of the present disclosure that fuses confidence parameters from five dimensions to obtain a comprehensive confidence score. Figure 8 This diagram illustrates a method for dynamically switching coordinate fusion based on comprehensive confidence level according to an embodiment of the present disclosure. Figure 9 A schematic diagram of a door positioning device based on confidence fusion is shown in an embodiment of this disclosure; Figure 10 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0025] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0026] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0027] The specific implementation methods of the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0028] Figure 1 A schematic diagram of an exemplary application system architecture is shown, illustrating an application of the confidence-based door positioning method described in this disclosure. For example... Figure 1 As shown, the system architecture may include terminal device 101, network 102 and server 103.

[0029] Network 102 is a medium used to provide a communication link between terminal device 101 and server 103, and can be a wired network or a wireless network.

[0030] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats, including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPSec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0031] Terminal device 101 can be various electronic devices, including but not limited to smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, wearable devices, augmented reality devices, virtual reality devices, etc.

[0032] Optionally, the client of the application installed on different terminal devices 101 may be the same, or the client of the same type of application based on different operating systems. Depending on the terminal platform, the specific form of the application client may also be different; for example, the application client may be a mobile client, a PC client, etc.

[0033] Server 103 can be a server that provides various services, such as a backend management server that supports the device operated by the user using terminal device 101. The backend management server can analyze and process received requests and other data, and feed the processing results back to the terminal device.

[0034] Optionally, the server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0035] Those skilled in the art will know that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative; any number of terminal devices, networks, and servers can be included depending on actual needs. This disclosure does not limit the scope of the embodiments.

[0036] Under the above system architecture, this disclosure provides a door positioning method based on confidence fusion, which can be executed by any electronic device with computing power.

[0037] In some embodiments, the door positioning method based on confidence fusion provided in this disclosure can be executed by a terminal device in the above-described system architecture; in other embodiments, the door positioning method based on confidence fusion provided in this disclosure can be executed by a server in the above-described system architecture; in still other embodiments, the door positioning method based on confidence fusion provided in this disclosure can be implemented by the terminal device and the server in the above-described system architecture through interaction.

[0038] Figure 2 This diagram illustrates a flowchart of a door positioning method based on confidence fusion, as shown in an embodiment of the present disclosure. Figure 2 As shown, the door positioning method based on confidence fusion provided in this embodiment includes the following steps: S202, Obtain motion information of the boarding bridge and environmental information of the environment in which the boarding bridge is located.

[0039] It should be noted that, in this disclosure, the cabin door refers to a rigid structural component installed on the cabin of an aircraft or other means of transportation, used to connect the interior and exterior spaces. It serves as a passageway for personnel and / or cargo to enter and exit the cabin, and is also the docking target for docking equipment such as boarding bridges. In this disclosure, the various embodiments are primarily described using an aircraft cabin door as an example.

[0040] In the above S202, the motion information of the boarding bridge refers to the motion information of the boarding bridge itself during the docking process with the cabin door; the environmental information of the environment in which the boarding bridge is located refers to various data in the external environment around the boarding bridge that may interfere with the docking accuracy, efficiency or safety, which is used to provide a basis for environmental adaptability adjustment and avoid docking failure due to environmental interference.

[0041] In this embodiment of the disclosure, the acquired motion information may include, but is not limited to, the following information about the boarding bridge: position information (such as the position coordinates of the boarding bridge under specified coordinates), attitude information (such as the tilt angle and rotation angle of the boarding bridge relative to the reference direction), and motion trend information (such as the speed and acceleration of the boarding bridge); the acquired environmental information in this embodiment of the disclosure may include, but is not limited to, the following information about the docking area: weather information (such as temperature, humidity, rainfall, wind force, and light), physical space information (such as the spatial layout of airport parking positions and port berths, obstacles, and ground / water surface conditions), and other object information that affects the identification of the cabin door (such as personnel, equipment, and temporary obstructions).

[0042] S204, determine multiple confidence parameters based on motion information and / or environmental information.

[0043] It should be noted that the confidence level parameters in this embodiment refer to indicators determined based on the motion information (time dimension) of the boarding bridge and / or the environmental information (spatial dimension) of the environment in which the boarding bridge is located. These parameters are configured according to the actual scenario, such as the boarding bridge, to quantify the reliability of related technical results (e.g., door detection / tracking, pose estimation, etc.). After obtaining the motion information (time dimension) of the boarding bridge and the environmental information (spatial dimension) of the environment in which the boarding bridge is located, multiple confidence levels can be determined based on the motion information and / or environmental information. In specific implementations, different confidence level parameters can be configured according to the actual scenario (e.g., the boarding bridge scenario).

[0044] In an optional embodiment, this disclosure employs one or more of the following confidence parameters: spatial consistency confidence, texture sharpness confidence, historical accuracy confidence, environmental adaptability confidence, and binocular matching confidence. Specifically, spatial consistency confidence can be used to quantify the pose stability of the boarding bridge (e.g., calculating the Euclidean distance between key door feature points in adjacent frames); texture sharpness confidence can be used to quantify the saliency of texture features at the door edges; historical accuracy confidence can be used to quantify the historical accuracy of door detection / tracking results based on sliding window error (e.g., performing a moving average of historical errors every 10 frames to infer the accuracy level); environmental adaptability confidence can be used to quantify the attenuation of door detection / tracking performance under complex weather conditions such as rain and fog; and binocular matching confidence can be used to quantify the matching accuracy of the binocular camera for door feature points.

[0045] S206, multiple confidence parameters are fused to obtain the comprehensive confidence score.

[0046] Compared to single-dimensional confidence parameters (such as confidence in historical accuracy focusing only on time, or confidence in environmental adaptability focusing only on space), the comprehensive confidence obtained by fusing multiple confidence parameters can achieve synergy between dynamic continuity in time and environmental constraints / positional accuracy in space. For example, the reliability of the hatch positioning at a certain moment needs to consider both whether the hatch movement was stable in the past few seconds (time dimension) and whether there is interference in the current environment (spatial dimension).

[0047] In some embodiments, when the confidence parameters include spatial consistency confidence, texture sharpness confidence, historical accuracy confidence, environmental adaptability confidence, and binocular matching confidence, the door positioning method based on confidence fusion provided in this disclosure embodiment can calculate the comprehensive confidence using the following formula: (1) in, Indicates the overall confidence level; Indicates the confidence level of spatial consistency; Indicates confidence level in texture sharpness; Indicates the confidence level of historical accuracy; Indicates the confidence level of environmental adaptability; Indicates the confidence level of binocular matching; The fusion weights represent the spatial consistency confidence levels; The fusion weights represent the confidence level of texture sharpness; The fusion weights represent the confidence levels of historical accuracy. The fusion weights represent the confidence levels of environmental adaptability; This represents the fusion weights for binocular matching confidence. In specific implementations, the fusion weights for spatial consistency confidence, texture sharpness confidence, historical accuracy confidence, environmental adaptability confidence, and binocular matching confidence can be pre-configured or dynamically changed; this disclosure does not impose any restrictions on this.

[0048] Based on regression analysis of experimental data, in one embodiment, the following will be used: , , , , The values ​​were selected as 0.4, 0.2, 0.2, 0.1, and 0.1, respectively, and the overall confidence level is expressed as follows: (2) In the comprehensive confidence level shown in Formula (2), the spatial weight is selected as 40% because the aircraft is a rigid body and the pose continuity is the basis of visual tracking; the texture weight is selected as 20% because the edge features of the cabin door are the core basis for the cabin door positioning accuracy; the historical weight is selected as 20% to be used for key long-term observations to suppress SLAM cumulative drift; the environmental weight is selected as 10% to provide quantitative compensation for aviation-specific meteorological interference; and the matching weight is selected as 10% to ensure the geometric consistency of binocular stereo vision.

[0049] In some embodiments, spatial consistency confidence can be calculated using the following formula. Texture sharpness confidence Historical accuracy confidence level Confidence level of environmental adaptability Binocular matching confidence : (3) (4) (5) (6) (7) in, Indicates the first Frame pose information; Indicates the first Frame pose information; This represents a scale parameter used to adjust the degree of influence of pose changes on spatial consistency calculations; Representing an image The gradient is typically a two-dimensional matrix used to reflect the edge texture information of the image; This represents the maximum value of the image gradient, used to normalize the gradient and keep the texture credibility within a reasonable range. Represents the norm of the image gradient matrix; This indicates the number of frames in the sliding window, and can be set to 10 frames. It is used to calculate the average error over a period of time. Indicates the frame number of the current frame; Indicates the frame number within the sliding window; Indicates the first The coordinate information of the hatch feature points obtained by frame tracking; Indicates the first The frame reference contains the coordinates of the hatch feature points. This indicates the intensity of rain and fog environments, and is used to compensate for environmental adaptability. Indicates the first The feature point set of a frame-based binocular system; Indicates the first The feature point set of a frame-based binocular system; The intersection-union ratio (IU) measures the degree of overlap between two sets of feature points, thus reflecting the matching quality of binocular feature points.

[0050] S208, based on the comprehensive confidence level and the established decision-making mechanism, obtain the door feature point coordinate information based on the first coordinate information and / or the second coordinate information, wherein the first coordinate information is the door feature point coordinate information obtained by visual odometry, and the second coordinate information is the door feature point coordinate information obtained based on the door recognition model.

[0051] It should be noted that the overall confidence level in this embodiment reflects the reliability of the hatch positioning at a certain moment. The decision mechanism refers to a pre-configured mechanism for determining the hatch feature point coordinates based on the first coordinate information and / or the second coordinate information according to the overall confidence level. In one embodiment, different coordinate fusion schemes can be adopted depending on the range of the overall confidence level. When the overall confidence level is large at a certain moment, it indicates that the hatch feature point coordinates obtained by the visual odometry are relatively accurate. In order to improve the real-time performance of hatch positioning and reduce computational complexity, the decision mechanism can be directly... The door feature point coordinates are obtained using visual odometry. When the overall confidence level is low at a certain moment, it indicates that the door feature point coordinates obtained by visual odometry are not very accurate. In this case, the door feature point coordinates obtained by visual odometry can be fused with the door feature point coordinates obtained by the door recognition model to obtain more accurate door feature point coordinates. When the overall confidence level is very low at a certain moment, it indicates that the door feature point coordinates obtained by visual odometry are very inaccurate. Therefore, the door feature point coordinates obtained by the door recognition model can be forced to be used.

[0052] It should be noted that the hatch feature points in this embodiment refer to feature points selected on the hatch that can determine the position of the hatch. These can be the outline of the hatch, the edge line of the hatch threshold, or the two corner points formed by the intersection of the edge lines on both sides of the hatch and the edge line of the threshold.

[0053] Taking an aircraft cabin door as an example, after the aircraft docks at the airport, the boarding bridge needs to precisely align with the left / right corner points of the cabin door to ensure safe passage for passengers. Therefore, the cabin door feature points in this embodiment include a first corner point and a second corner point. The first and second corner points are points formed by the intersection of the side edges of the cabin door with the edge of the door sill. When an observer outside the aircraft faces the cabin door, the first corner point can be the left corner point, and the second corner point can be the right corner point. After fusing the door feature point coordinate information accurately predicted by the model and the door feature point coordinate information estimated in real-time by visual tracking based on a comprehensive confidence level, the fused door feature point coordinate information is output to the boarding bridge control system to control the docking of the boarding bridge with the cabin door.

[0054] In this embodiment of the disclosure, a comprehensive confidence level is determined based on the motion information of the boarding bridge and the environmental information of the environment in which the boarding bridge is located. Then, based on the comprehensive confidence level and the set decision mechanism, the door feature point coordinate information is obtained based on the door recognition model and / or the door feature point coordinate information obtained by visual odometry, so as to obtain a more accurate door position and control the docking equipment such as the boarding bridge to be precisely aligned with the door.

[0055] In practical implementation, the boarding bridge can be controlled to perform the cabin door docking operation in the following way: control the boarding bridge to perform rotation mode and forward mode. The rotation mode is the working mode that controls the boarding bridge to rotate by a preset angle, and the forward mode is the working mode that controls the boarding bridge to move forward by a preset distance.

[0056] For example, Figure 3 This document illustrates a flowchart illustrating how a decision-making mechanism is used to determine the coordinates of hatch feature points according to a comprehensive confidence level, as described in an embodiment of this disclosure. Figure 4 As shown, the specific fusion process includes the following steps: S302, if the overall confidence level is less than the first threshold, the second coordinate information is directly determined as the fused hatch feature point coordinate information; S304, if the overall confidence level is greater than the second threshold, then the first coordinate information is directly determined as the fused hatch feature point coordinate information, and the second threshold is greater than the first threshold; S306 If the overall confidence level is greater than or equal to the first threshold and less than or equal to the second threshold, then the first coordinate information and the second coordinate information are fused to obtain the fused hatch feature point coordinate information.

[0057] In this embodiment of the disclosure, the first threshold and the second threshold can be two preset thresholds, wherein the second threshold is greater than the first threshold. Optionally, when the overall confidence level is any number between 0 and 1, the first threshold and the second threshold are also numbers between 0 and 1. In one embodiment, the first threshold is 0.6; the second threshold is 0.85.

[0058] Optionally, the above decision-making mechanism can be a three-level fusion decision-making mechanism, that is, divided into three decision levels according to the range of the comprehensive confidence level, forming a stepped fault-tolerant structure, realizing a progressive fault-tolerant system from trusting the visual tracking results to balanced fusion, and then to forced error correction. The first level (comprehensive confidence level greater than the second threshold) is the tracking-dominated mode (high speed and low power consumption), the second level is Kalman weighted fusion (adaptive balancing), and the third level is forced correction. Optionally, corresponding fault recovery can be performed simultaneously with forced correction.

[0059] In some embodiments, fusing the first coordinate information and the second coordinate information can be achieved by weighted summation of the first coordinate information and the second coordinate information. In specific implementations, such as... Figure 4 As shown, this can be achieved through the following steps: S402, determine the first weight information and the second weight information, wherein the first weight information is the weight information corresponding to the first coordinate information, and the second weight information is the weight information corresponding to the second coordinate information; S404. Based on the first weight information and the second weight information, the first coordinate information and the second coordinate information are weighted and summed to obtain the fused hatch feature point coordinate information.

[0060] In one embodiment, the first coordinate information and the second coordinate information can be weighted and summed using the following formula: (8) (9) in, This indicates that the coordinates of the fused hatch feature points have been obtained; Indicates the second coordinate information; Indicates the first coordinate information; The time-varying gain coefficient of the Kalman filter is used to balance the fusion weights of real-time tracking results and historical detection results. Its physical meaning is to dynamically adjust the proportion of both in the final output through a weighted combination of real-time confidence and historical attenuation terms. In this embodiment, As a fusion weight for the second coordinate information (historical detection results, i.e., the coordinate information of the hatch feature points obtained based on the hatch recognition model); The fusion weight represents the first coordinate information (real-time tracking results, i.e., the coordinate information of the hatch feature points obtained through visual odometry); Indicates the overall confidence level; This represents a coefficient, with a value ranging from 0 to 1.

[0061] In one embodiment, The value of is chosen to be 0.7; then It is expressed as follows: (10) in, This represents the real-time confidence level, reflecting the degree of trust in the tracking results of the current frame. A weight of 0.7 indicates that the real-time confidence level plays a major role in the gain. This indicates the historical decay term, which decays exponentially over time. It reflects that the weight of historical test results gradually decreases over time (the longer the time, the less reference value the historical information has). A weight of 0.3 is an appropriate retention of historical information.

[0062] In this embodiment of the disclosure, As a fusion weight, it represents the tracking result. With test results The dynamic fusion not only ensures the tracking accuracy of the current frame, but also suppresses short-term noise through historical information, thereby improving the stability of the overall output.

[0063] In another embodiment, the first coordinate information and the second coordinate information can be weighted and summed using the following formula: (11) (12) in, This indicates the coordinate information of the fused hatch feature points; This represents the coordinate information of the hatch feature points derived from the hatch recognition model, i.e., the second coordinate information; This represents the coordinate information of the hatch feature points obtained through visual odometry, i.e., the first coordinate information; This represents the evaluation score of the model's prediction results; The evaluation score represents the visual tracking results; This represents the evaluation score corresponding to coordinate fusion.

[0064] In some embodiments, the selected hatch feature points in this disclosure include: a first corner point and a second corner point, wherein the first corner point and the second corner point are points formed by the intersection of the edge lines on both sides of the hatch and the edge lines of the threshold, respectively. The hatch localization method based on confidence fusion provided in this disclosure can determine the evaluation score of the model prediction result and the evaluation score of the visual tracking result through the following formula: (13) (14) in, This represents the evaluation score of the model's prediction results; This indicates the evaluation score of the first corner point; This indicates the evaluation score for the second corner point; This indicates the evaluation score for the lower edge of the hatch; The evaluation score represents the visual tracking results; This indicates the evaluation score for the current frame; This indicates the evaluation score of the reference frame; This represents a curve parameter, with a value between 0 and 1. This represents the distance between the current frame and the reference frame. It can be the average distance between the door coordinates in the current frame and the reference frame. As the distance between the current frame and the reference frame gradually increases, the tracking result score of the current frame also gradually decreases.

[0065] In some embodiments, the confidence fusion-based hatch localization method provided in this disclosure can obtain the second coordinate information of hatch feature points in a hatch image through the following steps: inputting the hatch image into a pre-trained hatch recognition model and outputting the first coordinate information of hatch feature points in the hatch image.

[0066] In some embodiments, the confidence-based hatch localization method provided in this disclosure can obtain the first coordinate information of hatch feature points in a hatch image through the following steps: obtaining the camera rotation matrix and camera translation vector of the current frame hatch image; determining the coordinate information of the hatch feature points in the current frame hatch image based on the camera rotation matrix and camera translation vector of the current frame hatch image, as well as the coordinate information of the hatch feature points in the reference frame hatch image, the camera rotation matrix and camera translation vector of the reference frame hatch image.

[0067] Furthermore, in some embodiments, the door localization method based on confidence fusion provided in this disclosure can determine the coordinate information of door feature points in the current frame door image using the following formula: (15) in, This indicates the coordinate information of the hatch feature points in the current frame hatch image; This indicates the coordinate information of the hatch feature points in the reference frame hatch image; This represents the camera rotation matrix of the current frame's hatch image; Represents the camera translation vector of the current frame's hatch image; The camera rotation matrix represents the reference frame hatch image; This represents the camera translation vector of the reference frame hatch image.

[0068] The confidence-based door positioning method provided in this disclosure can be applied to, but is not limited to, the positioning of aircraft doors. The following section uses aircraft door positioning as an example, combined with… Figures 5 to 8 The door positioning method based on confidence fusion provided in the embodiments of this disclosure will be described in detail.

[0069] In a system that automatically docks a boarding bridge (also known as an airport jet bridge) with an aircraft door, the real-time and accurate positioning of the aircraft door is the core problem to be solved. Boarding bridge navigation refers to continuously providing the boarding bridge control system with the three-dimensional coordinates of door feature points (also known as door control points) over time, so that the boarding bridge docks with the aircraft door. In this embodiment, the selected door feature points can be the two corner points where the extended lines of adjacent sides of the rounded rectangle at the bottom of the door intersect. Figure 5 The three-dimensional coordinates of the left corner point B1 and right corner point B2 shown are based on the integration of various external information, observation information, and historical equipment status to make positioning judgments and generate control commands. The boarding bridge navigation scheme must meet the following requirements: ① Real-time requirement: The docking process must be completed within 1 minute, and the control command delay must be less than 100ms; ② Accuracy requirement: The door positioning error must be controlled within ±10cm; ③ Robustness requirement: It must not fail under extreme scenarios such as heavy rain (50mm / h) and strong light (100,000 lux).

[0070] In some embodiments, the coordinate information of the hatch feature points can be obtained in two ways: one is a model prediction method, which inputs the binocular image into a pre-trained hatch recognition model and outputs the coordinate information of the hatch feature points; the other is a visual tracking method, which calculates the camera pose through visual odometry, transforms the coordinate information of the hatch feature points in the reference frame into the current frame (visual tracking), and obtains the coordinate information of the hatch feature points in the current frame.

[0071] Both methods have their own advantages and disadvantages. The model prediction method has high accuracy (single frame error less than 1cm), but it has high computational complexity and poor real-time performance (computational complexity leads to a delay of more than 200ms. In docking tests, the movement of the boarding bridge caused a sense of jerking, and the measured docking time standard deviation was ±3.6s). The visual tracking method has better real-time performance (single frame processing time less than 20ms), but the cumulative error will increase with the distance between the reference frame and the current frame, affecting the accuracy of coordinate tracking (usually the error can reach 10-15cm after 30 seconds. In docking tests, the endpoint position deviation was 12cm, exceeding the safety threshold).

[0072] To retain the advantages of the two aforementioned hatch positioning methods, this disclosure proposes... Figure 6 The Spatial Temporal Confidence Fusion Framework (STCF) shown takes as input multiple positioning methods (including but not limited to the two door positioning methods mentioned above) and various environmental information collected by environmental sensors; the output of the framework is boarding bridge control commands; the core modules are a confidence assessment engine and a three-level fusion decision model.

[0073] Based on failure analysis from extensive real-world testing, the confidence evaluation engine in this embodiment adopts the following five dimensions of confidence: spatial consistency confidence, texture sharpness confidence, historical accuracy confidence, environmental adaptability confidence, and binocular matching confidence.

[0074] For the aircraft door motion characteristics (rigid body, low speed, linear) and environmental interference factors (rain, fog, strong light, vibration), a confidence function as shown in formulas (3) to (7) is designed to fuse the confidence scores of these five dimensions and obtain a comprehensive confidence score. Figure 7 As shown, after fusing the confidence scores of the five dimensions to obtain the comprehensive confidence score, it is input into the three-level fusion decision module.

[0075] Three-level fusion decision-making refers to a system divided into three decision levels based on the comprehensive confidence level range, forming a tiered fault-tolerant structure. This progressive fault-tolerant system ranges from trust tracking to balanced fusion, and then to forced error correction. The first-level fusion mode is tracking-dominated (high-speed, low-power); the second-level fusion mode is Kalman weighted fusion (adaptive balancing); and the third-level fusion mode is forced correction and fault recovery (safety fallback). In practical implementation, the following can be adopted: Figure 8 The dynamic switching logic is shown. When... In this case, a tracking-dominated mode is adopted, directly using the visual tracking results. To prevent oscillations during implementation, a hysteresis band of 0.02 can be set. Exit tracking-dominant mode at time; When using a weighted fusion mode (such as Kalman weighted fusion), to prevent oscillations, a minimum duration of 5 frames (approximately 167ms) is set to meet the conditions, then the tracking-dominated mode switches to the weighted fusion mode; when... When the signal is received, the system exits the weighted fusion mode and adopts the forced correction mode, which forces the use of the model prediction results (binocular calculation results). To prevent oscillations, confirmation can be performed for three consecutive frames. If the conditions are met for three consecutive frames, the system switches from the weighted fusion mode to the forced correction mode. After the forced correction is completed, the system returns to the tracking-dominated mode.

[0076] Optionally, in one embodiment, the second-level fusion mode described above can also adopt the weighted fusion mode shown in formula (11), which uses the model prediction results to correct the visual tracking results, taking into account both the accuracy of the model prediction and the real-time performance of the visual tracking.

[0077] In summary, the door positioning method based on confidence fusion provided in this disclosure can achieve the following technical effects: ① incorporating environmental adaptability indicators into the fusion decision; ② achieving intelligent decay of historical weights over time through time-varying Kalman gain; ③ using model prediction results to correct visual tracking results, balancing the accuracy of model prediction and the real-time performance of visual tracking.

[0078] It should be noted that the acquisition, storage, use, and processing of data in this disclosed technical solution comply with the relevant provisions of laws and regulations. All types of data, such as personal identity data, operational data, and behavioral data related to individuals, customers, and groups, obtained in this disclosed embodiment have been agreed upon by the users.

[0079] Based on the same inventive concept, this disclosure also provides a door positioning device based on confidence fusion, as described in the following embodiments. Since the principle by which this device embodiment solves the problem is similar to that of the above-described method embodiments, the implementation of this device embodiment can refer to the implementation of the above-described method embodiments, and repeated details will not be elaborated further.

[0080] Figure 9 This diagram illustrates a door positioning device based on confidence fusion according to an embodiment of the present disclosure. Figure 9 As shown, the device includes: a data acquisition module 901, a confidence level determination module 902, a confidence level fusion module 903, and a hatch coordinate information determination module 904.

[0081] The system includes a data acquisition module 901 for acquiring motion information of the boarding bridge and environmental information of the environment in which the boarding bridge is located; a confidence determination module 902 for determining multiple confidence parameters based on motion information and / or environmental information; a confidence fusion module 903 for fusing multiple confidence parameters to obtain a comprehensive confidence score; and a door coordinate information determination module 904 for obtaining door feature point coordinate information based on the comprehensive confidence score and a set decision mechanism, using first coordinate information and / or second coordinate information. The first coordinate information is the door feature point coordinate information obtained through visual odometry, and the second coordinate information is the door feature point coordinate information obtained based on a door recognition model.

[0082] In some embodiments, the hatch coordinate information determination module 904 is further configured to: if the overall confidence level is less than a first threshold, directly determine the second coordinate information as the fused hatch feature point coordinate information; if the overall confidence level is greater than a second threshold, directly determine the first coordinate information as the fused hatch feature point coordinate information, wherein the second threshold is greater than the first threshold; if the overall confidence level is greater than or equal to the first threshold and less than or equal to the second threshold, fuse the first coordinate information and the second coordinate information to obtain the fused hatch feature point coordinate information.

[0083] In some embodiments, the hatch coordinate information determination module 904 is further configured to: determine first weight information and second weight information, wherein the first weight information is the weight information corresponding to the first coordinate information and the second coordinate information is the weight information corresponding to the second coordinate information; and perform a weighted summation of the first coordinate information and the second coordinate information based on the first weight information and the second weight information to obtain the fused hatch feature point coordinate information.

[0084] In some embodiments, the above-mentioned door coordinate information determination module 904 is further used to perform a weighted summation of the first coordinate information and the second coordinate information using formula (8).

[0085] In other embodiments, the above-mentioned door coordinate information determination module 904 is also used to perform a weighted summation of the first coordinate information and the second coordinate information using formula (11).

[0086] Optionally, the hatch feature points include: a first corner point and a second corner point, the first corner point and the second corner point being the points formed by the intersection of the edge lines on both sides of the hatch with the edge lines of the threshold respectively. The hatch coordinate information determination module 904 is also used to determine the evaluation score of the model prediction result and the evaluation score of the visual tracking result through the above formulas (13) and (90).

[0087] In some embodiments, the confidence parameters include, but are not limited to, at least one of the following: spatial consistency confidence, texture sharpness confidence, historical accuracy confidence, environmental adaptability confidence, and binocular matching confidence.

[0088] In some embodiments, the confidence fusion module 903 is further used to calculate the overall confidence level using formula (2).

[0089] It should be noted that the examples and application scenarios implemented by the modules in the above device embodiments and the corresponding steps in the method embodiments are the same, but are not limited to the content disclosed in the above method embodiments. It should also be noted that the above modules, as part of the device, can be executed in a computer system such as a set of computer-executable instructions.

[0090] Those skilled in the art will understand that various aspects of this disclosure can be implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which can be collectively referred to herein as a "circuit", "module" or "system".

[0091] Based on the same inventive concept, this disclosure also provides an electronic device, which includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the confidence fusion-based door positioning method described above by executing the executable instructions. Since the principle by which this electronic device solves the problem is similar to that of the above method embodiments, the implementation of this electronic device embodiment can refer to the implementation of the above method embodiments, and repeated details will not be described again.

[0092] The following reference Figure 10 To describe an electronic device 1000 according to such an embodiment of the present disclosure. Figure 10 The electronic device 1000 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0093] like Figure 10 As shown, the electronic device 1000 is manifested in the form of a general-purpose computing device. The components of the electronic device 1000 may include, but are not limited to: at least one processing unit 1010, at least one storage unit 1020, and a bus 1030 connecting different system components (including storage unit 1020 and processing unit 1010).

[0094] The storage unit stores program code that can be executed by the processing unit 1010, causing the processing unit 1010 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 1010 can perform the following steps of the above method embodiment: acquiring motion information of the boarding bridge and environmental information of the environment in which the boarding bridge is located; determining multiple confidence levels based on the motion information and / or environmental information; fusing the multiple confidence levels to obtain a comprehensive confidence level; and obtaining door feature point coordinate information based on the comprehensive confidence level and a set decision mechanism, based on first coordinate information and / or second coordinate information, wherein the first coordinate information is the door feature point coordinate information obtained by visual odometry, and the second coordinate information is the door feature point coordinate information obtained based on a door recognition model.

[0095] Storage unit 1020 may include readable media in the form of volatile storage units, such as random access memory (RAM) 10201 and / or cache memory 10202, and may further include read-only memory (ROM) 10203.

[0096] Storage unit 1020 may also include a program / utility 10204 having a set (at least one) program module 10205, such program module 10205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0097] Bus 1030 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the multiple bus structures.

[0098] Electronic device 1000 can also communicate with one or more external devices 1040 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1000, and / or any device that enables electronic device 1000 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1050. Furthermore, electronic device 1000 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1060. As shown, network adapter 1060 communicates with other modules of electronic device 1000 via bus 1030. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0099] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0100] Based on the same inventive concept, this disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the confidence-based door positioning method described above. Since the principle by which this computer-readable storage medium solves the problem is similar to that of the above-described method embodiments, the implementation of this computer-readable storage medium embodiment can refer to the implementation of the above-described method embodiments, and repeated details will not be elaborated further.

[0101] More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0102] In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.

[0103] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0104] In practical implementation, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0105] Based on the same inventive concept, this disclosure also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the confidence-based door positioning method of any one of the above method embodiments. Since the principle by which this computer program product embodiment solves the problem is similar to that of the above method embodiments, the implementation of this computer program product embodiment can refer to the implementation of the above method embodiments, and repeated details will not be elaborated further.

[0106] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0107] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0108] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0109] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A hatch positioning method based on confidence fusion, characterized in that, include: Obtain motion information of the boarding bridge and environmental information of the environment in which the boarding bridge is located; Based on the motion information and / or the environmental information, multiple confidence parameters are determined; The multiple confidence parameters are fused to obtain a comprehensive confidence score; Based on the comprehensive confidence level and the established decision-making mechanism, the door feature point coordinate information is obtained based on the first coordinate information and / or the second coordinate information, wherein the first coordinate information is the door feature point coordinate information obtained by visual odometry, and the second coordinate information is the door feature point coordinate information obtained based on the door recognition model.

2. The hatch positioning method based on confidence fusion according to claim 1, characterized in that, The step of obtaining the door feature point coordinate information based on the comprehensive confidence level and the established decision-making mechanism, based on the first coordinate information and / or the second coordinate information, includes: If the overall confidence level is less than the first threshold, then the second coordinate information is directly determined as the door feature point coordinate information; If the overall confidence level is greater than the second threshold, then the first coordinate information is directly determined as the door feature point coordinate information, and the second threshold is greater than the first threshold; If the overall confidence level is greater than or equal to the first threshold and less than or equal to the second threshold, then the first coordinate information and the second coordinate information are fused to obtain the door feature point coordinate information.

3. The hatch positioning method based on confidence fusion according to claim 2, characterized in that, The first coordinate information and the second coordinate information are fused to obtain the door feature point coordinate information, including: Determine first weight information and second weight information, wherein the first weight information is the weight information corresponding to the first coordinate information, and the second coordinate information is the weight information corresponding to the second coordinate information; Based on the first weight information and the second weight information, the first coordinate information and the second coordinate information are weighted and summed to obtain the fused hatch feature point coordinate information.

4. The hatch positioning method based on confidence fusion according to claim 3, characterized in that, The first coordinate information and the second coordinate information are weighted and summed using the following formula: ； ； in, This indicates that the coordinates of the fused hatch feature points have been obtained; Indicates the second coordinate information; Indicates the first coordinate information; This indicates the fusion weight of the second coordinate information; This indicates the fusion weight of the first coordinate information; Indicates the overall confidence level; This represents a coefficient, with a value ranging from 0 to 1.

5. The hatch positioning method based on confidence fusion according to claim 3, characterized in that, The first coordinate information and the second coordinate information are weighted and summed using the following formula: ； ； in, This indicates the coordinate information of the fused hatch feature points; This represents the coordinate information of the hatch feature points derived from the hatch recognition model, i.e., the second coordinate information; This represents the coordinate information of the hatch feature points obtained through visual odometry, i.e., the first coordinate information; This represents the evaluation score of the model's prediction results; The evaluation score represents the visual tracking results; This represents the evaluation score corresponding to coordinate fusion.

6. The hatch positioning method based on confidence fusion according to claim 1, characterized in that, The method further includes: Obtain the camera rotation matrix and camera translation vector of the current frame hatch image; Based on the camera rotation matrix and camera translation vector of the current frame hatch image, and the coordinate information of the hatch feature points in the reference frame hatch image, the coordinate information of the hatch feature points in the current frame hatch image is determined, and the first coordinate information is obtained.

7. The hatch positioning method based on confidence fusion according to any one of claims 1 to 6, characterized in that, The plurality of confidence parameters include at least one of the following: spatial consistency confidence, texture sharpness confidence, historical accuracy confidence, environmental adaptability confidence, and binocular matching confidence.

8. The hatch positioning method based on confidence fusion according to claim 7, characterized in that, The overall confidence level is calculated using the following formula: ； in, Indicates the overall confidence level; Indicates the confidence level of spatial consistency; Indicates confidence level in texture sharpness; Indicates the confidence level of historical accuracy; Indicates the confidence level of environmental adaptability; Indicates the confidence level of binocular matching; The fusion weights represent the spatial consistency confidence levels; The fusion weights represent the confidence level of texture sharpness; The fusion weights represent the confidence levels of historical accuracy. The fusion weights represent the confidence levels of environmental adaptability; This represents the fusion weight for binocular matching confidence.

9. The hatch positioning method based on confidence fusion according to claim 8, characterized in that, Spatial consistency confidence is calculated using the following formula. Texture sharpness confidence Historical accuracy confidence level Confidence level of environmental adaptability Binocular matching confidence : ； ； ； ； ； in, Indicates the first Frame pose information; Indicates the first Frame pose information; Indicates the scale parameter; Representing an image The gradient is used to reflect the edge texture information of the image; This represents the maximum value of the image gradient; Represents the norm of the image gradient matrix; Indicates the number of frames in the sliding window; Indicates the frame number of the current frame; Indicates the frame number within the sliding window; Indicates the first The coordinate information of the hatch feature points obtained by frame tracking; Indicates the first The frame reference contains the coordinates of the hatch feature points. This indicates the intensity used to measure rain and fog conditions; Indicates the first The feature point set of a frame-based binocular system; Indicates the first The feature point set of a frame-based binocular system; This indicates intersection, union, and ratio.

10. A hatch positioning device based on confidence fusion, characterized in that, include: The data acquisition module is used to acquire the motion information of the boarding bridge and the environmental information of the environment in which the boarding bridge is located; A confidence level determination module is used to determine multiple confidence level parameters based on the motion information and / or the environmental information; The confidence fusion module is used to fuse the multiple confidence parameters to obtain a comprehensive confidence score. The hatch coordinate information determination module is used to obtain hatch feature point coordinate information based on the comprehensive confidence level and the set decision mechanism, based on the first coordinate information and / or the second coordinate information, wherein the first coordinate information is the hatch feature point coordinate information obtained by visual odometry, and the second coordinate information is the hatch feature point coordinate information obtained based on the hatch recognition model.

11. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the door positioning method based on confidence fusion as described in any one of claims 1 to 9 by executing the executable instructions.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the door positioning method based on confidence fusion as described in any one of claims 1 to 9.

13. A computer program product, comprising: A computer program or instruction, characterized in that, when executed by a processor, the computer program or instruction implements the door positioning method based on confidence fusion as described in any one of claims 1 to 9.

Images