Navigation fault detection method and electronic equipment

Through the factor graph and index gradual depletion sequential probability ratio detection method, combined with chi-square inspection, the navigation fault detection problem caused by GNSS interference is solved, the positioning accuracy and safety of autonomous driving vehicles are improved, and the fault miss detection rate is reduced.

CN118915099AActive Publication Date: 2024-11-08HUAWEI TECH CO LTD +1
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202310515770.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-08
Publication Date
2024-11-08
Estimated Expiration
2043-05-08

AI Technical Summary

Technical Problem

The prior art cannot effectively detect and identify navigation failures caused by interference from Global Navigation Satellite System (GNSS) and has a high missed detection rate, which affects the positioning accuracy and safety of autonomous vehicles.

Method used

Using a factor graph-based identification suppression algorithm and exponential gradual cancellation sequential probability ratio detection method, combined with chi-square test, monitors slow-change and mutation faults in GNSS interference, improves the accuracy and accuracy of fault detection, and provides real-time location and navigation route information to avoid mislocalization caused by GNSS interference.

Benefits of technology

It improves the accuracy of GNSS interference detection and the positioning accuracy and safety of autonomous driving vehicles, reduces the fault miss detection rate, and ensures the correct navigation information of users when they are affected by GNSS interference.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118915099A_ABST
    Figure CN118915099A_ABST
Patent Text Reader

Abstract

The invention provides a navigation fault detection method and electronic equipment, and the method comprises the steps: obtaining first data, the first data comprising a plurality of observation values at a first moment measured by a plurality of sensors in first equipment; obtaining first state information of the first equipment at the first moment according to the first data; according to the first data and the first state information, detecting whether the first device is interfered by the GNSS; when it is detected that the first device is interfered by the GNSS, a first interface is displayed, the first interface comprises first prompt information, and the first prompt information indicates that the first device is interfered by the GNSS. According to the method and the device, the navigation faults caused by the GNSS interference signals can be detected by adopting different modes according to different change conditions based on the characteristic that the change conditions of the GNSS interference in different time periods are different, so that the accuracy of fault detection is improved, the omission ratio of the faults is reduced, and the positioning precision and the safety of the navigation electronic equipment during automatic driving are effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a navigation fault detection method and electronic equipment. Background Art

[0002] Global Navigation Satellite System (GNSS) interference refers to the jammer simulating and transmitting false GNSS interference signals through software radio and other platforms, causing the target receiver to be invaded and taken over by the interference signal, thereby outputting incorrect positioning results. Incorrect positioning results can cause adverse consequences. For example, in the scenario of autonomous driving, the jammer can design a precise GNSS interference algorithm for the multi-sensor fusion framework of the target vehicle (the output GNSS position information is used for vehicle navigation and autonomous driving), so that the multi-sensor fusion framework is taken over and outputs incorrect / false GNSS position information, causing the vehicle to deviate from the normal planned route and cause an accident. At present, the industry has not proposed effective measures to accurately detect navigation failures caused by GNSS interference signals, and the failure miss rate is high. Summary of the invention

[0003] The present application discloses a navigation fault detection method and electronic device. Based on the different characteristics of GNSS interference changes in different time periods, different methods are used to detect navigation faults caused by GNSS interference signals according to different changes, thereby improving the accuracy of fault detection, reducing the missed detection rate of faults, and effectively improving the positioning accuracy and safety of electronic equipment during autonomous driving.

[0004] In a first aspect, an embodiment of the present application provides a navigation fault detection method, which is applied to a first device, and the method includes: acquiring first data, the first data including multiple observation values ​​at a first moment respectively measured by multiple sensors in the first device; obtaining first status information of the first device at the first moment based on the first data; detecting whether the first device is interfered with by the global navigation satellite system GNSS based on the first data and the first status information; when it is detected that the first device is interfered with by the GNSS, displaying a first interface, the first interface including first prompt information, and the first prompt information indicating that the first device is interfered with by the GNSS.

[0005] In the above method, the electronic device can obtain the first state information of the first device at the first moment based on multiple observation values, and detect whether the electronic device is subject to GNSS interference based on the multiple observation values ​​and the first state information to monitor the start time of the GNSS interference. When interference is detected, the electronic device can promptly alert the user, such as displaying a first prompt message to prompt the user that it is currently subject to GNSS interference, so as to avoid the misplaced positioning information outputted by the GNSS interference affecting the user's use and causing accidents, thereby improving the safety of the electronic device when driving, improving the accuracy of fault detection, and reducing the missed detection rate of faults.

[0006] In a possible implementation, detecting whether the first device is subject to GNSS interference based on the first data and the first status information includes: detecting whether the first device is subject to GNSS interference based on the first data, historical data corresponding to the first data, the first status information, and historical status information corresponding to the first status information.

[0007] In the above method, the electronic device can detect whether the first device is subject to GNSS interference based on the first data and the first status information at the first moment, and the historical data before the first moment (that is, the historical data corresponding to the first data and the historical status information corresponding to the first status information), thereby avoiding the Kalman filter only considering the observation value at the current moment (that is, the first moment mentioned above) and ignoring the constraints of the data at historical moments on the data at the current moment, thereby further improving the accuracy of fault detection.

[0008] In a possible implementation, detecting whether the first device is subject to GNSS interference based on the first data, historical data corresponding to the first data, the first status information, and historical status information corresponding to the first status information includes: obtaining second data based on the first data, historical data corresponding to the first data, the second status information, and historical status information corresponding to the second status information; obtaining a first statistic of the first device at the first moment based on the second data, and determining that the first device is subject to GNSS interference when the first statistic is greater than or equal to a first threshold.

[0009] In the above method, the electronic device can timely detect the slowly varying soft faults in the GNSS interference through the first statistic to monitor the start time of the GNSS interference, improve the accuracy of fault detection, and reduce the missed detection rate of faults.

[0010] In a possible implementation, the method also includes: when it is detected that the first device is not affected by GNSS interference, displaying a second interface, the second interface including a first position and a first route, the first position being the position of the first device at the first moment estimated based on second state information, and the first route being the navigation route of the first device at the first moment estimated based on the second state information.

[0011] In the above method, when it is detected that the electronic device is not affected by GNSS interference, the electronic device can display the position and navigation route at the first moment obtained according to the first data, and automatically drive according to the navigation route, so that the user can view the real-time position and navigation route of the electronic device in the current trip, and the user can also monitor the road conditions to improve driving safety.

[0012] In a possible implementation, the method also includes: when it is detected that the first device is subject to GNSS interference, displaying a third interface, the third interface including a second position and a second route, the second position being the position of the first device at the first moment estimated based on the first status information, and the second route being the navigation route of the first device at the first moment estimated based on the first status information.

[0013] In the above method, when it is detected that the electronic device is subject to GNSS interference, the electronic device can display the second position and the second route at the first moment obtained according to the first status information. The second position is closer to the actual position of the electronic device when it is subject to GNSS interference, thereby preventing the electronic device from deviating from the normally planned route due to erroneous positioning information output by GNSS interference and causing accidents, thereby improving the positioning accuracy and safety of the electronic device during automatic driving.

[0014] In a possible implementation, the first state information is obtained according to a first algorithm, and the second state information is obtained according to a second algorithm, the first algorithm is a recognition suppression algorithm, and the second algorithm is a multi-sensor fusion MSF positioning algorithm.

[0015] In the above method, the position obtained by the factor graph-based identification suppression algorithm is closer to the real position of the electronic device when it is subject to GNSS interference, so that the electronic device can still output correct position information when it is subject to GNSS interference. The MSF positioning algorithm only considers the observation value at the current moment, ignores the constraints of the data at the historical moment on the data at the current moment, and has high requirements for the linearization and noise distribution of the system. When the measurement model is nonlinear and / or the noise does not conform to the Gaussian distribution, the fusion accuracy is not high and the estimated state information is inaccurate.

[0016] In a possible implementation manner, the first state information includes at least one of a position, a posture, a speed, and a time of the first device.

[0017] In a possible implementation, the method also includes: receiving a first operation when displaying the first interface; and displaying a third interface in response to the first operation, the third interface including a second position and a second route, the second position being the position of the first device at the first moment estimated based on the first state information, and the second route being the navigation route of the first device at the first moment estimated based on the first state information.

[0018] In the above method, the electronic device can detect signal anomalies before a dangerous collision occurs, and prompt the user that the current driving safety is about to be threatened. After receiving the user operation, the second position and the second route at the first moment obtained according to the first state information can be displayed. The second position is closer to the actual position of the electronic device when it is subject to GNSS interference, thereby preventing the electronic device from deviating from the normally planned route due to the erroneous positioning information output by the GNSS interference and causing an accident, thereby improving the positioning accuracy and safety of the electronic device during automatic driving.

[0019] In one possible implementation, when it is detected that the first device is subject to GNSS interference, displaying a first interface includes: when it is detected that the first device is subject to GNSS interference, if the defense function is not turned on, displaying the first interface; the method also includes: when it is detected that the first device is subject to GNSS interference, if the defense function is turned on, displaying a third interface, the third interface including a second position and a second route, the second position is the position of the first device at the first moment estimated based on the first state information, and the second route is the navigation route of the first device at the first moment estimated based on the first state information.

[0020] In the above method, if the defense function is not turned on, the electronic device can display the first interface when it encounters GNSS interference to prompt the user to turn on the defense function to improve the safety of driving the electronic device. If the defense function is turned on, the electronic device can output correct location information when it encounters GNSS interference to improve the positioning accuracy of the electronic device when driving.

[0021] In a possible implementation, the method further includes: when the first interface is displayed, the driving mode of the first device is an automatic driving mode; after the first interface is displayed, the method further includes: receiving a second operation, and the second operation is used to switch the automatic driving mode to a manual driving mode.

[0022] In the above method, when the electronic device is subject to GNSS interference, it can switch the automatic driving mode to the manual driving mode according to the user's operation, so as to prevent the electronic device from deviating from the normal planned route and affecting the user's use due to the erroneous positioning information output by the GNSS interference, thereby further improving the safety of the electronic device during driving and ensuring the safety of the user.

[0023] In one possible implementation, the detecting that the first device is subject to GNSS interference includes: when it is detected that the offset distance is greater than or equal to an offset threshold, determining that the first device is subject to GNSS interference, the offset distance is the difference between a first position and a second position, the first position is a position estimated according to a first algorithm, the second position is a position estimated according to a second algorithm, the first algorithm is an identification suppression algorithm, and the second algorithm is an MSF positioning algorithm; when it is detected that the offset distance is less than the offset threshold, determining that the GNSS interference to the first device has ended.

[0024] In the above method, when the offset distance of the electronic device is greater than or equal to the offset threshold, the electronic device may deviate from the normal planned route, which may easily cause an accident. Therefore, when the offset distance is greater than or equal to the offset threshold, the electronic device determines that it is subject to GNSS interference to ensure safety during driving.

[0025] In a possible implementation, the method also includes: when it is detected that the first device is not subject to GNSS interference, the first device is located in a first lane; the offset distance is the distance between the first position estimated by the first algorithm and a lane dividing line, and the lane dividing line is the dividing line between the first lane and an adjacent lane.

[0026] In the above method, when the electronic device switches from the first lane being driven to the adjacent lane, the electronic device deviates from the normal planned route, and therefore, the electronic device determines that it is subject to GNSS interference.

[0027] In a possible implementation, the method also includes: after detecting that the first device is subject to GNSS interference, acquiring second data, the second data including multiple observation values ​​at a second moment respectively measured by the multiple sensors, the second moment being later than the first moment; detecting whether the GNSS interference to the first device is ended based on the second data; and when it is detected that the GNSS interference to the first device is ended, displaying a second prompt message, the second prompt message indicating that the GNSS interference to the first device is ended.

[0028] In the above method, after the electronic device is subjected to GNSS interference, the electronic device can detect whether the GNSS interference suffered by the electronic device has ended based on multiple observation values ​​at the second moment, so as to monitor the end time of the GNSS interference, avoid false alarms caused by untimely detection, so that normal GNSS signals cannot be used for navigation and positioning. When the end of GNSS interference is detected, an alarm end message is displayed to prompt the user that the current driving safety threat has been lifted, thereby improving the safety of electronic devices when driving, improving the accuracy of fault detection, and reducing the missed detection rate of faults.

[0029] In a possible implementation, the method also includes: after detecting that the GNSS interference to the first device has ended, displaying a fourth interface, the fourth interface including a third position and a third route, the third position being the position of the first device at the second moment estimated based on third state information, the third route being the navigation route of the first device at the second moment estimated based on the third state information, the third state information being obtained based on a second algorithm, and the second algorithm being an MSF positioning algorithm.

[0030] In the above method, when the electronic device detects that the GNSS interference has ended, the electronic device can be restored to the third position and the third route estimated by the MSF positioning algorithm, so as to avoid the identification suppression algorithm occupying too much running memory of the electronic device and save memory resources.

[0031] In a possible implementation, after displaying the second interface, the method further includes: when it is detected that the first device is subject to GNSS interference, if the defense function is not turned on, displaying a fifth interface, the fifth interface including a fourth position and a fourth route, the fourth position and the fourth route are obtained according to a second algorithm, the time corresponding to the fourth position is later than the first time, the fourth position is different from the first position, the fourth route is different from the first route, and the second algorithm is an MSF positioning algorithm.

[0032] In the above method, if the electronic device continues to be interfered with by GNSS and the defense function is not turned on, the electronic device will display the fourth position and the fourth route obtained by the MSF positioning algorithm. At this time, the fourth position and the fourth route are inaccurate positioning information, which will cause the electronic device to deviate from the normally planned route and cause an accident.

[0033] In a possible implementation, detecting whether the GNSS interference suffered by the first device has ended based on the second data includes: obtaining third data based on the second data and a first estimated value, the first estimated value is obtained based on part of the observation values ​​in the second data, the first estimated value is obtained based on a first algorithm, the first algorithm is an identification suppression algorithm, the first estimated value corresponds to the second moment, and the third data corresponds to the second moment; when it is detected at a third moment that the first device is affected by GNSS interference, obtaining a second statistic based on the second data, the third moment is a moment before the second moment, and the second statistic corresponds to the second moment; when the second statistic is greater than or equal to a first threshold, obtaining a third statistic based on the third data, the third statistic corresponds to the second moment; when the third statistic is greater than or equal to the second threshold, determining that the GNSS interference suffered by the first device has ended.

[0034] In the above method, the electronic device can combine the second statistic and the third statistic to timely detect sudden hard faults in GNSS interference to monitor the end time of GNSS interference, improve the accuracy of fault detection, and reduce the missed detection rate of faults.

[0035] In a second aspect, the present application provides an electronic device, including a transceiver, a processor and a memory, wherein the memory is used to store a computer program, and the processor calls the computer program to execute the navigation fault detection method in any possible implementation of the first aspect.

[0036] In a third aspect, the present application provides an electronic device, comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the navigation fault detection method in any possible implementation of the first aspect.

[0037] In a fourth aspect, the present application provides a computer storage medium storing a computer program, which, when executed by a processor, implements the navigation fault detection method in any possible implementation of any of the above aspects.

[0038] In a fifth aspect, the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the navigation fault detection method in any possible implementation of the first aspect.

[0039] In a sixth aspect, the present application provides an electronic device, the electronic device comprising a method or device for executing any one of the implementations of the first aspect of the present application. The electronic device is, for example, a chip. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The following is an introduction to the drawings used in this application.

[0041] Figure 1 is a hardware structure diagram of an electronic device 100 provided in this application;

[0042] Figure 2 is a schematic diagram of a software architecture of an electronic device 100 provided in this application;

[0043] Figure 3 is a structural schematic diagram of an electronic device 100 provided in the present application;

[0044] Figure 4-Figure 6 is a schematic diagram of some user interfaces provided by this application;

[0045] Figure 7 It is a flowchart of a navigation fault detection method provided by the present application;

[0046] Figure 8 It is a structural schematic diagram of a factor graph provided by this application;

[0047] Fig. 9 It is a structural schematic diagram of a factor graph corresponding to a GNSS residual sequence provided by this application;

[0048] Fig.10 It is a flowchart of another navigation fault detection method provided by the present application;

[0049] Fig.11 It is a structural schematic diagram of a factor graph corresponding to residual information provided by this application;

[0050] Fig.12 This is a logic diagram of combining the first method and the second method to detect the end of GNSS interference provided by the present application;

[0051] Fig.13 It is a schematic diagram of a GNSS interference detection process provided by this application. DETAILED DESCRIPTION

[0052] The technical solution in the embodiment of the present application will be described below in conjunction with the accompanying drawings. In the description of the embodiment of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiment of the present application, "multiple" means two or more than two.

[0053] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0054] The location information provided by the Global Navigation Satellite System (GNSS) can be applied to scenarios such as autonomous driving and terminal location-based services (such as navigation services). Since the GNSS signal emitted by the satellite itself is not strong, when the GNSS signal is weak and / or the GNSS signal is modulated, the GNSS signal is susceptible to radio frequency interference, resulting in inaccurate / reduced positioning accuracy and even interruption of positioning services. GNSS interference is a kind of radio frequency interference with strong intentions. The jammer can simulate and transmit false GNSS interference signals (such as forged signals with the same parameters as GNSS satellite signals) through platforms such as software radio, so that the target receiver is invaded and taken over by the interference signal, thereby outputting erroneous positioning results. Incorrect positioning results can cause adverse consequences. For example, in autonomous driving scenarios and navigation scenarios, the vehicle deviates from the normal planned route and causes an accident.

[0055] For the convenience of explanation, the following embodiments are described using an autonomous driving scenario as an example.

[0056] In the autonomous driving scenario, the vehicle can navigate and drive autonomously based on the GNSS position information output by the Multi-Sensor Fusion (MSF) framework. The MSF framework can output GNSS position information based on the detection signals of multiple sensors, such as but not limited to GNSS receivers, inertial measurement units (IMUs), light detection and ranging (Lidars), cameras, etc. The jammer can design a sophisticated GNSS jamming algorithm for the target vehicle's multi-sensor fusion framework, so that the multi-sensor fusion framework is taken over and outputs wrong / false GNSS position information, causing the vehicle to deviate from the normal planned route and navigation failures, which is likely to cause safety accidents.

[0057] At present, the multi-sensor fusion framework is based on the Kalman filter to estimate the state information of the vehicle, such as but not limited to position, posture, speed, time and other information. In one embodiment, the Kalman filter can include two stages: prediction and update. In the prediction stage, the Kalman filter can obtain the estimated result of the current state (referred to as the current state) based on the estimated result of the state at the previous moment (referred to as the previous state). In the update stage, the estimated result obtained in the prediction stage is optimized using the observed value of the current state to obtain the optimal estimated result of the current state. For example, the estimated value at time k is obtained based on the estimated value at time k-1, and the estimated value at time k is optimized using the observed value at time k to obtain the optimal estimated value at time k. Among them, the estimated value can be estimated by a measurement model, and the observed value can be measured by multiple sensors. For example, the acceleration and angular velocity of the vehicle are measured by an IMU, and the measurement model can indicate the functional relationship between the observed value and the state to be estimated. The Kalman filter-based integrated navigation system has the problem of inaccurate state information estimation. For example, the Kalman filter only considers the estimated value of the previous state and the observed value of the current state, ignoring the constraints of the state information at the historical moment (which can be referred to as historical information) on the current state information. For another example, the Kalman filter is implemented based on Gaussian noise and linear system, and has high requirements on the linearization and noise distribution of the system. When the measurement model is nonlinear and / or the noise does not conform to the Gaussian distribution, the fusion accuracy of the Kalman filter is not high, and the estimated state information is inaccurate.

[0058] Moreover, GNSS interference is slow-changing in the beginning, and may cause small, slow-changing soft faults. When GNSS interference is about to end, it is sudden, and may cause sudden hard faults. At present, the residual chi-square detection method can be used to detect GNSS interference signals to detect faults. The residual chi-square detection method has a high sensitivity for sudden hard faults, but a low sensitivity for small, slow-changing soft faults. Therefore, the residual chi-square detection method cannot detect the slow-changing soft faults caused by GNSS interference in time, and the missed detection rate of faults is high, and the risk of accidents is high.

[0059] The present application provides a navigation fault detection method, which is applied to electronic devices. The electronic devices can use a recognition and suppression algorithm based on a factor graph to estimate state information to obtain the real-time position / positioning result of the electronic devices, solving the problem of inaccurate state information estimated by the multi-sensor fusion (MSF) positioning algorithm. The electronic device can use a detection method based on an exponentially fading sequential probability ratio to detect a slowly changing soft fault in GNSS interference to monitor the start time of GNSS interference, and the electronic device can use a method of combining an exponentially fading sequential probability ratio and a chi-square test to detect a sudden hard fault in GNSS interference to monitor the end time of GNSS interference. The present application comprehensively considers the slowly changing situation and the sudden situation in GNSS interference, and adopts different methods to detect faults for different situations, improves the accuracy of fault detection, and reduces the missed detection rate of faults. In addition, the real-time position / positioning result obtained by the recognition and suppression algorithm based on the factor graph is closer to the real position of the electronic device when it is subjected to GNSS interference, so that the electronic device can still output the correct position information when it is subjected to GNSS interference, avoiding the electronic device from deviating from the normal planned route due to the wrong positioning information output by the GNSS interference, causing accidents, and improving the positioning accuracy and safety of the electronic device during automatic driving.

[0060] In addition, the electronic device can display the detection process of the GNSS interference signal to the user in a visual form. For example, after the GNSS interference begins, the electronic device can display an alarm message to remind the user that it is currently suffering from GNSS interference. When the GNSS interference ends, the electronic device can display an alarm end message to remind the user that the current GNSS interference has ended. During the GNSS interference process, the electronic device can display the real-time location / positioning results obtained by the identification and suppression algorithm to avoid the user's use being affected by the erroneous positioning information output by the GNSS interference, further improving the safety of the electronic device during automatic driving, thereby ensuring user safety.

[0061] In the present application, the electronic device may be a mobile phone, a tablet computer, a handheld computer, a desktop computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), smart home devices such as smart TVs, wearable devices such as smart bracelets, smart watches, smart glasses, augmented reality (AR), virtual reality (VR), mixed reality (MR) and other extended reality (XR) devices, vehicle-mounted devices or smart city devices, etc. The present application does not impose any special restrictions on the specific types of electronic devices.

[0062] Next, an exemplary electronic device 100 according to an embodiment of the present application is introduced.

[0063] Figure 1 A schematic diagram of the hardware structure of an electronic device 100 is exemplarily shown.

[0064] like Figure 1 As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0065] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0066] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0067] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0068] The processor 110 may also be provided with a memory for storing instructions and data. In one embodiment, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0069] In one embodiment, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0070] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In one embodiment, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface to realize the touch function of the electronic device 100.

[0071] The I2S interface can be used for audio communication. In one embodiment, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In one embodiment, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0072] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In one embodiment, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In one embodiment, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to implement the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0073] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In one embodiment, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In one embodiment, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0074] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In one embodiment, the processor 110 and the camera 193 communicate via the CSI interface to realize the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to realize the display function of the electronic device 100.

[0075] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In one embodiment, the GPIO interface can be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0076] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0077] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0078] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0079] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0080] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0081] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0082] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In one embodiment, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In one embodiment, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0083] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In one embodiment, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0084] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0085] In one embodiment, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0086] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0087] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In one embodiment, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0088] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0089] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In one embodiment, ISP can be set in camera 193.

[0090] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In one embodiment, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0091] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0092] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0093] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0094] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0095] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0096] The electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0097] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In one embodiment, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0098] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0099] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0100] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0101] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0102] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In one embodiment, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In one embodiment, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0103] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In one embodiment, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

[0104] The air pressure sensor 180C is used to measure air pressure. In one embodiment, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0105] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In one embodiment, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover automatic unlocking and other features are set.

[0106] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0107] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In one embodiment, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0108] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0109] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0110] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0111] The temperature sensor 180J is used to detect temperature. In one embodiment, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown due to low temperature.

[0112] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0113] The bone conduction sensor 180M can obtain vibration signals. In one embodiment, the bone conduction sensor 180M can obtain vibration signals of the vibrating bones of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure pulse signals. In one embodiment, the bone conduction sensor 180M can also be set in headphones to form bone conduction headphones. The audio module 170 can parse out voice signals based on the vibration signals of the vibrating bones of the vocal part obtained by the bone conduction sensor 180M to realize the voice function. The application processor can parse the heart rate information based on the blood pressure pulse signal obtained by the bone conduction sensor 180M to realize the heart rate detection function.

[0114] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0115] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0116] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0117] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In one embodiment, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0118] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. For example, the software system of the layered architecture may be an Android system, or a Harmony operating system (OS), or other software systems. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0119] Figure 2 A schematic diagram of the software architecture of an electronic device 100 is exemplarily shown.

[0120] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In one embodiment, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0121] The application layer can include a series of application packages.

[0122] like Figure 2 As shown, the application package may include camera, gallery, music, calendar, short message, call, navigation, Bluetooth, browser, screen projection and other applications.

[0123] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0124] like Figure 2 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0125] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0126] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0127] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0128] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0129] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0130] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0131] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0132] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0133] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0134] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0135] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0136] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0137] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0138] A 2D graphics engine is a drawing engine for 2D drawings.

[0139] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0140] The following is an example of the workflow of the software and hardware of the electronic device 100 in conjunction with the autonomous driving scenario of the navigation application.

[0141] When the touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into an original input event (including touch coordinates, timestamp of the touch operation, and other information). The original input event is stored in the kernel layer. The application framework layer obtains the original input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is a control of the navigation application, the navigation application calls the interface of the application framework layer, and then controls the display driver by calling the kernel layer, and displays the main interface of the navigation application through the display screen 194.

[0142] Figure 3 A schematic structural diagram of an electronic device 100 is exemplarily shown.

[0143] like Figure 3As shown, the electronic device 100 may include a physical unit 101 and a functional unit 102. The physical unit 101 may include a sensor 1011, an onboard computer 1012, and a central control display screen 1013. The functional unit 102 may include a multi-sensor fusion (MSF) positioning algorithm module 1021, an identification suppression algorithm module 1022, and a global navigation satellite system (GNSS) interference detection module 1023. Optionally, the onboard computer 1012 may include at least one module in the functional unit 102, wherein:

[0144] The sensor 1011 may include, but is not limited to, a GNSS receiver, an IMU, an antenna, a laser radar, a camera, etc. The sensor 1011 may be used to obtain multiple observation quantities / observation values ​​of the electronic device 100, such as the position of the electronic device 100 measured by the GNSS receiver, the acceleration and angular velocity of the electronic device 100 measured by the IMU, and the distance measured by the laser radar. The onboard computer 1012 may be used to receive the multiple observation quantities / observation values ​​measured by the sensor 1011, and send the obtained multiple observation quantities / observation values ​​to the MSF positioning algorithm module 1021, the identification and suppression algorithm module 1022, and the GNSS interference detection module 1023. The MSF positioning algorithm module 1021 can obtain the first state information of the electronic device 100 according to the multiple observations / observation values ​​sent by the vehicle-mounted computer 1012. The first state information includes, for example but not limited to, information such as position, posture, speed, time, etc. In some examples, the MSF positioning algorithm module 1021 can estimate the first state information of the electronic device 100 at time k according to the multiple observations / observation values ​​at time k (it can also be referred to as the first state information being an estimated value at time k). The MSF positioning algorithm module 1021 can send the first state information to the vehicle-mounted computer 1012, and the vehicle-mounted computer 1012 can estimate the first real-time position of the electronic device 100 at time k according to the first state information. Optionally, the vehicle-mounted computer 1012 can match the first real-time position with a high-precision map to determine the first lane information of the electronic device 100. The vehicle-mounted computer 1012 can send the first real-time position, and optionally the first lane information of the electronic device 100 to the central control display screen 1013, and the central control display screen 1013 can be used to display the first real-time position, and optionally the first lane information of the electronic device 100 during navigation.

[0145] The identification and suppression algorithm module 1022 can obtain the second state information of the electronic device 100 based on the multiple observations / observation values ​​sent by the onboard computer 1012. The second state information includes, for example but not limited to, position, posture, speed, time and other information. In some examples, the identification and suppression algorithm module 1022 can estimate the second state information of the electronic device 100 at time k based on the multiple observations / observation values ​​at time k (it can also be called the second state information is the estimated value at time k). It can be understood that for multiple observations at the same time (k), the first state information and the second state information can be estimated values ​​obtained using different algorithms. The identification and suppression algorithm module 1022 can send the second state information to the onboard computer 1012, and the onboard computer 1012 can estimate the second real-time position of the electronic device 100 at time k based on the second state information. Optionally, the onboard computer 1012 can match the second real-time position with the high-precision map to determine the second lane information of the electronic device 100.

[0146] The identification and suppression algorithm module 1022 can send the second state information to the GNSS interference detection module 1023. The GNSS interference detection module 1023 can detect whether the current electronic device 100 is subject to GNSS interference based on the second state information and the multiple observation quantities / observation values ​​sent by the vehicle-mounted computer 1012. When GNSS interference is detected, the GNSS interference detection module 1023 can send an alarm message to the vehicle-mounted computer 1012. After the vehicle-mounted computer 1012 receives the alarm message, it can display the alarm message through the central control display screen 1013 to prompt the user that the current GNSS interference is being suffered. When the GNSS interference is detected to be over, the GNSS interference detection module 1023 can send an alarm end message to the vehicle-mounted computer 1012. After the vehicle-mounted computer 1012 receives the alarm end message, it can display the alarm end message through the central control display screen 1013 to prompt the user that the current GNSS interference is over. Optionally, during the GNSS interference process, the vehicle-mounted computer 1012 can display the second real-time position and second lane information of the electronic device 100 during the navigation process through the central control display screen 1013.

[0147] Understandably, Figure 3 The electronic device 100 shown may include more or fewer modules. Any module in the electronic device 100 may be a hardware module or a software module. Any of the above modules may be a separate module, and at least one of the above modules may also be integrated together. For example, the MSF positioning algorithm module 1021, the identification and suppression algorithm module 1022, and the GNSS interference detection module 1023 may be integrated in the on-board computer 1012. The present application does not limit the specific modules of the electronic device 100.

[0148] The following describes an application scenario involved in an embodiment of the present application and a user interface diagram in the scenario.

[0149] See also Figure 4 , Figure 4 A schematic diagram of a user interface for autonomous driving navigation is shown as an example.

[0150] like Figure 4 As shown in (A), the electronic device 100 can display a user interface 410 of a navigation application. The user interface 410 includes a search box 411 and a search box 412. The search box 411 can be used to input the starting point of the navigation route, and the search box 412 can be used to input the end point of the navigation route. The user interface 410 also includes a map 413, and the map 413 displays a location identifier 413A, a location identifier 413B, and a route 413C. The location identifier 413A is used to identify the position of the starting point in the map 413, and the starting point is the position entered in the search box 411. The location identifier 413B is used to identify the position of the end point in the map 413, and the end point is the position entered in the search box 412. Route 413C is used to indicate route information from the above-mentioned starting point to the above-mentioned end point. The user interface 410 also includes navigation information 414. The navigation information 414 may include multiple route information from the above-mentioned starting point to the above-mentioned end point, for example, "recommended" route information and route information of other options. The "recommended" route information is displayed with the characters "32 minutes·11 kilometers", which can represent the time and route distance required for the recommended route, wherein the "recommended" route information has been selected, corresponding to route 413C. Navigation information 414 may also include control 414A. Control 414A is displayed with the characters "Start navigation", which can be used to trigger a navigation interface that displays the above-mentioned selected route information. In one embodiment, the electronic device 100 can respond to a user operation on control 414A (for example, the user operation is a touch operation), automatically drive according to the selected route information, and display the user interface of autonomous driving navigation (also referred to as navigation in autonomous driving mode), for details, see Figure 4 The user interface 420 shown in (B).

[0151] like Figure 4As shown in (B), the user interface 420 may include an autonomous driving navigation interface 421 and navigation information 422. The autonomous driving navigation interface 421 may display multiple lanes, such as lane 421A, lane 421B, and lane 421C. The autonomous driving navigation interface 421 may also include a location identifier 421D and a navigation route 421E. The location identifier 421D may identify the location of the electronic device 100 on the map, and specifically may identify the lane where the electronic device 100 is located (for example, the electronic device 100 is currently in lane 421B). The navigation route 421E may represent the current automatic driving route of the electronic device 100. The navigation information 422 may include driving prompt information, road information, distance and time information. The driving prompt information may indicate the driving of the electronic device 100 (for example, "Please keep straight"); the road information may indicate the road information during the driving process of the electronic device 100 (for example, "2 kilometers into Taizihu North Road"); the distance and time information may indicate the distance between the electronic device 100 and the destination (for example, "11 kilometers"), the remaining driving time (for example, "32 minutes") and the estimated time to arrive at the destination (for example, "expected to arrive at 8:32").

[0152] In one embodiment, during autonomous driving navigation, for example Figure 4 After the implementation shown in (B), if the electronic device 100 is currently in the first preset situation, for example, when the electronic device 100 is interfered with by the GNSS and the safe navigation mode is not turned on, the navigation route and navigation information of the electronic device 100 are changed. At this time, the electronic device 100 can display the navigation interface after being interfered with by the GNSS. For details, see Figure 4 The user interface 430 shown in (C).

[0153] like Figure 4 As shown in (C), the user interface 430 may include a location identifier 431A, a navigation route 431B, and navigation information 432. The location identifier 431A may identify the location of the electronic device 100 on the map, and the navigation route 431B may represent the changed navigation route. Assuming that the electronic device 100 is at the location identified by the location identifier 431A, the electronic device 100 is subject to GNSS interference. At this time, the navigation route of the electronic device 100 is changed from the original navigation route (for example, a route that stays straight in lane 421B) to navigation route 431B. Specifically, the lane in which the electronic device 100 is traveling may change, for example, the electronic device 100 switches from lane 421B to lane 421A, and the driving prompt information included in the navigation information 432 is displayed as "lane switching is about to take place ahead", to instruct the electronic device 100 to change the driving lane 421B to lane 421A.

[0154] In one embodiment, Figure 4After the implementation shown in (C), the electronic device 100 can display the navigation route after the change (for example Figure 4 When the electronic device 100 detects that it is currently in the first preset situation, the electronic device 100 may display a warning user interface to prompt the user to turn on the safe navigation mode. For details, see Figure 5 The user interface 440 shown in (A).

[0155] like Figure 5 As shown in (A), the user interface 440 and Figure 4 The example embodiment is similar to the user interface 430 shown in (C), except that the location marker 441A in the user interface 440 can indicate that the electronic device 100 is currently located at the boundary between lane 421A and lane 421B, rather than in the previous lane 421B. The location marked by the location marker 441A can be a false location on the map when the electronic device 100 is subject to GNSS interference. The navigation information 442 and the navigation route 431B in the user interface 440 can indicate that the electronic device 100 is traveling along the navigation route 431B, for example, from the location indicated by the location marker 431A on the map to the location indicated by the location marker 441A. However, at this time, the actual driving route of the electronic device 100 is the above-mentioned original navigation route (for example, a route that stays straight in lane 421B). Specifically, the electronic device 100 can travel from the location indicated by the location marker 431A to the location indicated by the location marker 441B (not displayed in the user interface 440). The location marked by the location marker 441B can be the actual location on the map when the electronic device 100 is subject to GNSS interference. User interface 440 may further include window 443, which may include prompt information (for example, including characters such as "There is a problem with the current navigation position", "Please confirm whether to enable safe navigation mode"), control 443A and control 443B, control 443A may be used to enable safe navigation mode, and control 443B may be used to refuse to enable safe navigation mode.

[0156] In one embodiment, after the electronic device 100 turns on the safe navigation mode, it can relocate the position of the electronic device 100, update the navigation route, and automatically drive according to the updated navigation route. Figure 5 The user operation of the control 443A in the user interface 440 shown in (A) (for example, the user operation is a touch operation) is displayed as shown in FIG. Figure 5 The user interface 450 shown in (B) is shown in FIG. Figure 5As shown in (B), the user interface 450 may include prompt information 451 and control 452. The prompt information 451 displays the characters "Switched to safe navigation mode" and "Repositioning". The prompt information 451 and control 452 may indicate that the electronic device 100 has turned on the safe navigation mode (for example, switched from the automatic driving mode to the safe navigation mode) and repositioned the position of the electronic device 100 on the map. The user interface after the electronic device 100 repositions and updates the navigation route is, for example, Figure 5 The user interface 460 shown in (C) is shown in FIG. Figure 5 As shown in (C), the user interface 460 and Figure 5 The navigation route 461B in the user interface 460 is similar to the user interface 450 shown in (B), except that the navigation route 461B in the user interface 460 can represent the updated navigation route of the electronic device 100 in the safe navigation mode. For example, the navigation route is updated from: the electronic device 100 switches from lane 421B to lane 421A and keeps going straight in lane 421A to: the electronic device 100 switches from lane 421A to lane 421B and keeps going straight in lane 421B. The navigation route can represent the planned route of the electronic device 100 on the actual driving road. The location identifier 461A in the user interface 460 can identify the position of the electronic device 100 after repositioning, which can represent the position of the electronic device 100 on the actual driving road.

[0157] In one embodiment, if the electronic device 100 detects that it is currently in the second preset situation, for example, when the GNSS interference to the electronic device 100 ends, the electronic device 100 can display a user interface indicating that the alarm has ended to prompt the user to turn on the automatic driving mode. For details, see Figure 6 The user interface 470 shown in (A).

[0158] like Figure 6 As shown in (A), the user interface 470 and Figure 5 The user interface 470 is similar to the user interface shown in (C), except that the user interface 470 also includes a window 471, and the window 471 may include prompt information (for example, including characters "The current navigation position has been restored", "Please confirm whether to turn on the automatic driving mode"), a control 471A and a control 471B, and the control 471A can be used to turn on the automatic driving mode, and the control 471B can be used to refuse to turn on the automatic driving mode.

[0159] In one embodiment, the electronic device 100 may display a user interface for autonomous driving navigation in response to a user operation on the control 471A (for example, the user operation is a touch operation). Figure 6 The user interface 480 shown in (B) is shown in FIG. Figure 6 As shown in (B), the user interface 480 and Figure 4 The user interface 480 is similar to the user interface 420 shown in (B), except that the navigation information 481 in the user interface 480 is different from the navigation information 422 in the user interface 420, and the location of the electronic device 100 on the map in the user interface 480 and the current automatic driving route of the electronic device 100 are different from those in the user interface 420.

[0160] Not limited to the above implementation, in another implementation, the user can set the permissions of the navigation application through the system setting function of the electronic device 100 or the setting function of the navigation application, for example, allowing the navigation application to automatically turn on the safe navigation mode. For details, see Figure 6 The user interface 490 shown in (C) is shown in FIG. Figure 6 As shown in (C), user interface 490 may include setting name 491 and control 492. Setting name 491 displays the characters "safe navigation mode". Control 492 may be used to turn on or off the safe navigation mode of electronic device 100. Control 492 in user interface 490 is in the turned-on state. In some examples, after electronic device 100 turns on safe navigation mode, electronic device 100 may provide: when detecting that it is currently in a first preset situation, automatically turn on the function of safe navigation mode. For specific examples, see Figure 5 The embodiment shown.

[0161] Not limited to the above example, in other examples, when the electronic device 100 does not turn on the safe navigation mode in response to the user operation, the electronic device 100 can turn off the automatic navigation mode. Figure 5 A user operation (for example, a touch operation) of the control 443B in the user interface 440 shown in (A) turns off the automatic navigation mode of the current electronic device 100. At this time, the electronic device 100 stops automatic driving and displays a prompt message to prompt the user to take over the electronic device 100.

[0162] Not limited to the above example, in other examples, when the electronic device 100 does not turn on the automatic driving mode in response to the user operation, the electronic device 100 can continue to automatically drive according to the updated navigation route. At this time, the electronic device 100 can display the user interface of the automatic driving in the safe navigation mode. Figure 6 The user operation of the control 471B in the user interface 470 shown in (A) (for example, the user operation is a touch operation) displays and Figure 6 (B) is similar to the user interface 480 shown in FIG. 1 , except that the navigation route and Figure 6The navigation route is different from the navigation route in the user interface 480 shown in (B), which is the updated navigation route in the safe navigation mode.

[0163] Not limited to the above example, in other examples, when the electronic device 100 does not receive a user operation for turning on or off the safe navigation mode within the first preset time, the electronic device 100 may turn on the safe navigation mode by default. Figure 5 After the user interface 440 shown in (A) is displayed, if no user operation (for example, the user operation is a touch operation) on the control 443A or the control 443B is received within a first preset time (for example, 10 seconds), the electronic device 100 can automatically turn on the safe navigation mode and display the following Figure 5 The user interface 450 is shown in (B).

[0164] Next, the navigation fault detection method provided by the embodiment of the present application is introduced.

[0165] See also Figure 7 , Figure 7 is a flowchart of a navigation fault detection method provided in an embodiment of the present application. The method can be applied to Figure 1 The electronic device 100 shown. The method can be applied to Figure 3 The electronic device 100 is shown. The method may include but is not limited to the following steps:

[0166] S101: A sensor acquires first data.

[0167] In one embodiment, the first data may include multiple observations / observation values ​​measured by sensors at time k, wherein the sensors include, for example but not limited to, GNSS receivers, IMUs, antennas, lidars, cameras, etc., and the multiple observations include, for example but not limited to, the position of the electronic device 100 measured by the GNSS receiver, the acceleration and angular velocity of the electronic device 100 measured by the IMU, and the distance measured by the lidar, etc.

[0168] In one embodiment, the electronic device 100 may receive a user operation and respond to the user operation, for example, in response to a request for Figure 4 The user operates the control 414A in the user interface 410 shown in (A) to turn on the automatic driving / navigation function. At this time, the sensor in the electronic device 100 obtains the first data.

[0169] S102: The sensor sends first data to the MSF positioning algorithm module.

[0170] In one embodiment, the MSF positioning algorithm may be an algorithm based on a Kalman filter.

[0171] S103: The sensor sends first data to the recognition and suppression algorithm module.

[0172] S104: The sensor sends first data to the GNSS interference detection module.

[0173] The order of any two steps in S102, S103 and S104 is not limited.

[0174] S105: The MSF positioning algorithm module obtains first status information according to the first data.

[0175] In one embodiment, the first state information is the state information of the electronic device 100 at the corresponding moment obtained based on the first data. The first state information is the same as the moment corresponding to the first data. The first state information includes, for example but not limited to, information such as the position, posture, and speed of the electronic device 100. In some examples, the MSF positioning algorithm module can estimate the first state information at moment k based on multiple observations received at moment k. It can be understood that the first state information is the state estimate value obtained using the MSF positioning algorithm at moment k.

[0176] S106: The MSF positioning algorithm module obtains first navigation information according to the first state information.

[0177] In one embodiment, the first navigation information includes, for example but not limited to, the first real-time position of the electronic device 100, the first route information, etc., wherein the first real-time position and the first route information are related. For example, the MSF positioning algorithm module can determine the first route information based on the first real-time position of the electronic device 100 at time k and the destination position.

[0178] S107: The MSF positioning algorithm module outputs first navigation information.

[0179] In one implementation, after the MSF positioning algorithm module obtains the first navigation information, the first navigation information may be outputted, for example, by displaying the first navigation information on a display screen of the electronic device 100, for example, displaying Figure 4 The user interface 420 shown in (B), wherein the location identifier 421D in the user interface 420 indicates the first real-time location, and the navigation route 421E indicates the first route information.

[0180] S108: The identification suppression algorithm module obtains second state information according to the first data.

[0181] In one embodiment, the second state information is the state information of the electronic device 100 at a corresponding moment obtained based on the first data. The second state information is the same as the moment corresponding to the first data. The second state information includes, for example but not limited to, information such as the position, posture, and speed of the electronic device 100. In some examples, the recognition and suppression algorithm module can estimate the second state information at moment k based on multiple observations received at moment k. It can be understood that the second state information is the state estimate value obtained using the recognition and suppression algorithm at moment k.

[0182] In one embodiment, the recognition suppression algorithm module can input multiple observations at time k into a multi-sensor fusion framework based on a factor graph, and output the optimal estimate at time k, that is, the above-mentioned second state information. In some examples, the recognition suppression algorithm module can estimate the optimal estimate of the current moment based on the observations at the historical moments and the observations at the current moment, wherein the historical moments can include k-1 different moments within a second preset time length before the current moment, for example, the current moment is k, and the historical moments can be 20 different moments within 20 seconds before k, wherein one second corresponds to one moment.

[0183] In one embodiment, a factor graph is a graph model that can intuitively represent the information transfer relationship between state variables in the state space through nodes and edges (i.e., the connection between nodes). It can be understood that the multi-sensor fusion framework based on the factor graph can combine multiple observations (also called measurement factors) to construct the maximum likelihood estimation problem to obtain the optimal estimate. An example of the factor graph can be seen in Figure 8 .

[0184] like Figure 8 As shown, the node xi is the state factor to be estimated, node b i is the bias factor, node f i,imu is the IMU pre-integration factor, node f i,bias is the IMU bias factor, node f i,2D / 3D is the two-dimensional (2D) / three-dimensional (3D) visual positioning factor, node f i,gps is the GPS factor, node f prior,state is the initial state factor, node f prior,bias is the initial bias factor, where i is a positive integer greater than or equal to 1 and less than or equal to k. Each state factor x to be estimated i It is related to the factors that are connected to it. Take the state factor x1 to be estimated as an example. 1,imu 、f 1,2D / 3D 、f prior,state Related, where f1,imu is related to b1, b1 is related to f 1,bias 、f prior,bias Related.

[0185] In some examples, the factor graph model can be converted into a nonlinear problem using maximum a posteriori probability estimation. The specific implementation process can be seen in the following equations (1), (2) and (3):

[0186] X = argmax x (П j f j (x j )) (1)

[0187]

[0188] Among them, x j represents the jth state variable to be estimated, z j represents the jth observation, f j (x j ) represents the jth factor node, X={x1,x2,...,x k} represents the set of state variables to be estimated, h j (·) represents the jth observation z j The relevant measurement equations, such as h j (x j ) is the state variable x to be estimated j and the observed quantity z j The observation function between j represents the measurement variance, one j corresponds to one moment, j∈[0,n], n represents the sliding window size, which can be understood as the number of historical moments, and n can be k-1. k is the first data at the kth moment, x k is the second state information at the kth moment.

[0189] Formula (1) and formula (2) can be converted into a nonlinear least squares problem. The nonlinear least squares problem can be seen in the following formula (3):

[0190]

[0191] Next, solve formula (3) to obtain the optimal estimated value x k (ie, the second state information).

[0192] The multi-sensor fusion framework based on factor graph can take into account the observations at historical moments and continuously optimize the optimal solution of nonlinear problems through repeated iterations and relinearization to obtain the optimal estimate.

[0193] The order of S105 and S108 is not limited.

[0194] S109: The identification suppression algorithm module obtains second navigation information according to the second state information.

[0195] In one embodiment, the second navigation information includes, for example but not limited to, the second real-time position of the electronic device 100, second route information, etc., wherein the second real-time position and the second route information are related. For example, the identification and suppression algorithm module can determine the second route information based on the second real-time position of the electronic device 100 at time k and the destination position.

[0196] The order of S106 and S109 is not limited, and the order of S107 and S109 is not limited.

[0197] S110: The identification and suppression algorithm module sends second status information to the GNSS interference detection module.

[0198] The order of any two steps in S110, S105 and S108 is not limited.

[0199] S111: The GNSS interference detection module obtains second data according to the first data and the second status information, and detects whether GNSS interference is caused according to the second data.

[0200] In one embodiment, the GNSS interference detection module can obtain the second data according to the first data, the historical data corresponding to the first data, the second state information, and the historical state information corresponding to the second state information, wherein the first data can be the observation value at the k moment, the historical data corresponding to the first data can be the observation value at the historical moment before the k moment (t moment, t is a positive integer greater than or equal to 1 and less than or equal to k-1, k-1 is the number of historical moments), the second state information can be the estimated value at the k moment, and the historical state information corresponding to the second state information can be the estimated value at the historical moment before the k moment (t moment). Optionally, the historical data corresponding to the first data and the historical state information corresponding to the second state information can be stored in the electronic device 100.

[0201] In one embodiment, the second data may be a residual sequence formed by the difference between the observed value and the estimated value at the same time, for example, the difference between the first data at time k and the second state information at time k, and the difference between the first data at the same time before time k (time t) and the second state information at time t. In some examples, it is assumed that the GNSS measurement model can refer to the following equations (4) and (5):

[0202] z k =H k X k +V k +ρk (4)

[0203]

[0204] Among them, z k is the observed value at time k, H k is the discrete measurement matrix, X k is the estimated value at time k, V k is the discrete measurement noise, ρ k is a discrete measurement disturbance.

[0205] Through formula (4) and formula (5), the residual of the GNSS measurement factor can be seen in the following formula (6):

[0206]

[0207] Among them, v k is the GNSS measurement residual at time k, and the GNSS measurement residuals in multiple sliding windows (i.e., multiple historical moments) in the factor graph model are selected to form the residual sequence {v j |j=1,2,3,...,k}, where an example of the factor graph corresponding to the GNSS residual sequence can be found in Fig. 9 . See Fig. 9 , Fig. 9 It is a structural schematic diagram of a factor graph corresponding to a GNSS residual sequence provided in an embodiment of the present application.

[0208] like Fig. 9 As shown, the node xi is the state factor to be estimated, node f i,imu is the IMU pre-integration factor, node f i,2D / 3D is the 2D / 3D visual positioning factor, node f i,gps is the GPS factor. Among them, i is a positive integer greater than or equal to 1 and less than or equal to k, the dotted box represents the sliding window size, and the residual v i Related to IMU pre-integration factor, 2D / 3D visual positioning factor, and GPS factor.

[0209] In one embodiment, the GNSS interference detection module can analyze the second data in a first manner to obtain a first interference statistic. The first manner includes, for example but not limited to, taking the mean, variance, expectation, etc., and comparing the first interference statistic with a first threshold. When the first interference statistic is greater than or equal to the first threshold, it is determined that GNSS interference is present. At this time, S112 is executed; when the first interference statistic is less than the first threshold, it is determined that GNSS interference is not present, and S101-S111 are executed again. At this time, the time corresponding to the first data obtained is different from the k time, and accordingly, the time corresponding to the data obtained according to the first data is also different from the k time.

[0210] In some examples, the residual sequence is a residual sequence composed of GNSS measurement residuals, and the first method is a statistical test method of exponentially fading sequential probability ratio. When there is no interference in the GNSS measurement model, the residual sequence should obey the normal distribution N(0, ∑). At this time, the expectation and variance of the residual can be referred to the following formula (7):

[0211]

[0212] Among them, E(v k ) represents the expected residual at time k, represents the variance of the residual at time k.

[0213] When interference occurs in the GNSS measurement model, the expected value and variance of the residual can be found in the following equation (8):

[0214]

[0215] According to formula (7) and formula (8), the following hypothesis test is proposed: H0: ρ = 0, the GNSS measurement model is normal; H1: ρ ≠ 0, interference occurs in the GNSS measurement model. The probability density of the residual samples under the two probability assumptions can be seen in the following formulas (9) and (10):

[0216]

[0217]

[0218] in,

[0219] According to the probability densities of the two hypotheses in formula (9) and formula (10), the likelihood ratio under the two hypotheses can be calculated as shown in the following formula (11):

[0220]

[0221] Next, for L k By finding the logarithm, we can obtain the first interference statistic of the sequential probability ratio under the factor graph model optimization, which can be seen in the following formula (12):

[0222]

[0223] The change rate of the first interference statistic at each moment can be seen in the following formula (13):

[0224]

[0225] Therefore, the first interference statistic of the sequential probability ratio can also be represented by an iterative form, as shown in the following formula (14):

[0226] λ k =λ k-1 +Δλ k (14)

[0227] The residual mean at time k can be found in the following formula (15):

[0228]

[0229] in, represents the fading weighting factor.

[0230] It can be seen from formula (15) that the introduction of the fading weighting factor can increase the residual v at time k k The weight of , reducing the residual mean of historical moments The weight of can be understood as, when GNSS interference occurs at time k, the weight of the residual at time k in the residual mean is enhanced, and the weight of the residual at the historical time in the residual mean is reduced, so as to enhance the rate of change of the residual mean when GNSS interference occurs, so that the residual mean is more sensitive to the start time of GNSS interference, thereby determining the time when GNSS interference occurs. That is to say, by analyzing the residual sequence, if the change amount of the residual mean at the current time is greater than or equal to the first threshold, it is determined that the current time begins to be subject to GNSS interference, wherein the first interference statistic can indicate the residual mean.

[0231] In one embodiment, when the electronic device 100 begins to be interfered by GNSS, the first real-time position in the first navigation information estimated by the MSF positioning algorithm module cannot represent the real-time position of the electronic device 100 during navigation, so the navigation route information of the electronic device 100 will change. In some examples, the user interface of the electronic device 100 during the autonomous driving process is, for example, Figure 4 The user interface 420 shown in (B) of FIG. 1 is a user interface when the electronic device 100 begins to be interfered with by the GNSS. Figure 4 The user interface 430 shown in (C).

[0232] In one embodiment, when the electronic device 100 turns on the automatic driving / navigation function, the GNSS interference detection module can be automatically turned on to turn on the GNSS interference identification and suppression detection function. For example, the electronic device 100 can receive user operations and set the permissions of the navigation application during the automatic driving process according to the user operations, such as allowing the navigation application to automatically turn on the GNSS interference identification and suppression detection function. Optionally, the electronic device 100 may not turn on the GNSS interference detection module. In this case, the electronic device 100 does not execute S110-S113. Optionally, the electronic device 100 may turn on the GNSS interference detection module during the automatic driving process to turn on the GNSS interference identification and suppression detection function.

[0233] S112: The GNSS interference detection module outputs a first prompt message.

[0234] In one embodiment, when the GNSS interference detection module detects that the electronic device 100 is subject to GNSS interference, the GNSS interference detection module may output a first prompt message, for example, by displaying the first prompt message on the display screen of the electronic device 100. The first prompt message may be used to prompt the user that the current electronic device 100 is subject to GNSS interference, for example, by displaying Figure 5 The user interface 440 shown in (A).

[0235] S113: When the first preset condition is met, the identification suppression algorithm module outputs the second navigation information.

[0236] In one embodiment, the first preset condition may be that the electronic device 100 has turned on the safe navigation mode. When the first preset condition is met, the identification suppression algorithm module can output the second navigation information, for example, display the second navigation information through the display screen of the electronic device 100.

[0237] In some examples, when the electronic device 100 does not enable the safe navigation mode, the first prompt information may also be used to prompt the user to enable the safe navigation mode. The electronic device 100 may respond to the user operation, such as Figure 5 The touch operation of the control 443A in the user interface 440 shown in (A) turns on the safe navigation mode. At this time, the recognition suppression algorithm module can output the second navigation information, for example, the electronic device 100 displays Figure 5 The user interface 460 shown in (C).

[0238] In other examples, when the electronic device 100 turns on the automatic driving / navigation function, the safe navigation mode can be automatically turned on. For example, the electronic device 100 can receive user operations and set the permissions of the navigation application during the automatic driving process according to the user operations, such as allowing the navigation application to automatically turn on the safe navigation mode. For specific examples, see Figure 6In this case, after the GNSS interference detection module outputs the first prompt information, it can directly output the second navigation information.

[0239] S114: When the first preset condition is not met, the MSF positioning algorithm module outputs first navigation information.

[0240] In one embodiment, when the first preset condition is not met, the MSF positioning algorithm module can output the first navigation information. At this time, the electronic device 100 can display a prompt message to remind the user that it is currently subject to GNSS interference, and the navigation positioning and navigation route information may be erroneous. The user can choose to manually drive the electronic device 100.

[0241] In some examples, when the electronic device 100 does not enable the safe navigation mode, for example, the electronic device 100 may respond to the Figure 5 The user operates the control 443B in the user interface 440 shown in (A) to turn off the automatic driving mode and cancel the safety navigation mode. At this time, the electronic device 100 can display a prompt message to remind the user that it is currently subject to GNSS interference and asks the user to take over / manually drive the electronic device 100.

[0242] The above embodiments are described using autonomous driving as an example, but are not limited to this. The present application can also be applied to manual driving scenarios, and the specific description is similar to the above description.

[0243] exist Figure 7 In the method shown, the GNSS interference detection module can use a detection method based on exponentially fading sequential probability ratio to timely detect slowly changing soft faults caused by GNSS interference, so as to monitor the start time of GNSS interference, detect signal anomalies before a dangerous collision occurs in the electronic device, and remind the user that the current driving safety is about to be threatened, thereby improving driving safety, improving the accuracy of fault detection, and reducing the missed detection rate of faults. In addition, the multi-sensor fusion framework based on factor graphs avoids the assumption that the measurement model needs to be linearized and Gaussian noise, and improves the fusion accuracy of multiple sensors, so that the real-time position obtained by the recognition suppression algorithm is closer to the actual position of the electronic device when it is subject to GNSS interference, improving the robustness and stability of the multi-sensor fusion framework, and improving the positioning accuracy of the electronic device during automatic driving.

[0244] In addition, the GNSS interference detection module can distinguish between abnormal errors caused by GNSS interference and errors caused by natural factors (such as GNSS positioning errors caused by multipath, positioning errors caused by sensor failures such as IMU / lidar, etc.). It will only alarm when it is determined that the abnormal error is caused by GNSS interference, thus avoiding false alarms and further improving the accuracy of fault detection.

[0245] See also Fig.10 , Fig.10 is a flowchart of another navigation fault detection method provided in an embodiment of the present application. The method can be applied to Figure 1 The electronic device 100 shown. The method can be applied to Figure 3 The electronic device 100 is shown. The method may include but is not limited to the following steps:

[0246] S201: The sensor acquires first data.

[0247] In one implementation, before S201, S101-S114 are also included. When the electronic device 100 detects that it is subject to GNSS interference, the sensor in the electronic device 100 can continuously obtain the first data. For example, the electronic device 100 detects that it is subject to GNSS interference at time k, and the sensor in the electronic device 100 can obtain the first data at time m (m is greater than k). At this time, the first data may include multiple observations / observation values ​​measured by the sensor at time m. For details, see Figure 7 The description of S101 is not repeated here.

[0248] S202: The sensor sends first data to the GNSS interference detection module.

[0249] S203: The sensor sends first data to the recognition and suppression algorithm module.

[0250] The order of S202 and S203 is not limited.

[0251] S204: The identification suppression algorithm module obtains second state information according to the first data.

[0252] In one embodiment, the description of S204 and Figure 7 The description of S108 is similar, except that the second state information in S204 is an estimated value obtained by using the recognition suppression algorithm at time m.

[0253] S205: The identification suppression algorithm module obtains second navigation information according to the second state information.

[0254] In one embodiment, the description of S204 and Figure 7 The description of S109 is similar to that of S205, except that the second navigation information in S205 is determined based on the second state information at time m.

[0255] S206: The GNSS interference detection module obtains third data according to the first data, and detects whether the GNSS interference is ended according to the third data.

[0256] In one embodiment, the GNSS interference detection module can calculate the third data based on the first data, wherein the first data includes, for example but not limited to, observation values ​​obtained by IMU measurement, observation values ​​obtained by lidar measurement, observation values ​​obtained by camera measurement, etc.

[0257] In one embodiment, the third data may be residual information formed by the difference between the observed value and the estimated value at the same time. Figure 7 The description of the second data in S111 is similar to that in S112, except that the estimated value here is an estimated value estimated based on the observation value of other sensors other than the GNSS receiver (such as IMU and LiDAR), wherein an example of the factor graph corresponding to the residual information can be found in the following Fig.11 . See Fig.11 , Fig.11 It is a structural schematic diagram of a factor graph corresponding to residual information provided in an embodiment of the present application.

[0258] like Fig.11 As shown, Fig.11 and Fig. 9 The structure of the factor graph shown is similar, except that Fig.11 Also includes node f i,lidar Lidar factor, residual information Z i Related to the IMU pre-integration factor, 2D / 3D visual positioning factor, GPS factor, and lidar factor, where i is a positive integer greater than or equal to 1 and less than or equal to m.

[0259] In some examples, it is assumed that the residual information of the measurement model is recalculated / reconstructed according to the state variable nodes of the IMU, the lidar and / or the camera in the factor graph. The residual information can be referred to in the following formula (16):

[0260] Z m =Z gps,m -x imu / lidar,m (16)

[0261] Among them, Z gps,m represents the observation value of the Global Positioning System (GPS) at time m, x imu / lidar,m represents the estimated value at time m, Z m represents the residual at time m.

[0262] In some examples, the residual is the above residual information, and the second method is the residual chi-square detection method. According to formula (16), the chi-square detection statistic (also called the second interference statistic) is established, which can be seen in the following formula (17):

[0263]

[0264] Among them, ∑ -1 Represents the variance of the residual.

[0265] In one implementation, the GNSS interference detection module may detect whether the GNSS interference at the current moment is ended according to the third data and the detection result at the previous moment. In some examples, the GNSS interference detection module may detect whether the GNSS interference is ended in combination with the first method and the second method. Fig.12 , Fig.12 FIG. 1 is a logic diagram of combining the first method and the second method to detect the end of GNSS interference provided by an embodiment of the present application. Fig.12 As shown in FIG. 1 , when the detection result at the previous moment (moment m-1) is that the GNSS interference is being detected (the GNSS interference is not over), the GNSS interference detection module can first analyze the second data at the current moment (moment m) by the first method to obtain the first interference statistic at moment m. For the specific implementation process, see Figure 7 S111 is not described in detail. When the first interference statistic at time m is greater than or equal to the first threshold, the third data is analyzed by a second method to obtain a second interference statistic. The second method includes, for example but not limited to, a residual chi-square detection method, and compares the second interference statistic and the second threshold. When the second interference statistic is less than the second threshold, it is determined that the GNSS interference is over (the end time is time m). At this time, S207 is executed; when the second interference statistic is greater than or equal to the second threshold, it is determined that the GNSS interference is not over (that is, it is still subject to GNSS interference). At this time, S208 is executed, wherein the specific implementation method of the detection result at the previous moment is the same as the implementation method at the current moment (time m), that is, the first method and the second method are combined for detection. Optionally, when the second interference statistic at time m is less than the second threshold, it is determined that the GNSS interference is not over.

[0266] The order of S204 and S206 is not limited, and the order of S205 and S206 is not limited.

[0267] S207: The GNSS interference detection module outputs a second prompt message.

[0268] In one embodiment, when the GNSS interference detection module detects that the GNSS interference has ended, the GNSS interference detection module may output a second prompt message, for example, by displaying the second prompt message on the display screen of the electronic device 100. The second prompt message may be used to prompt the user that the GNSS interference currently suffered by the electronic device 100 has ended, for example, by displaying Figure 6 The user interface 470 shown in (A).

[0269] In one embodiment, the second prompt information can also be used to prompt the user to turn on the automatic driving mode. In some examples, the electronic device 100 can respond to the user operation of the function control for turning on the automatic driving mode, such as Figure 6 The touch operation of the control 471A in the user interface 470 shown in (A) is performed to switch from the safe navigation mode to the automatic driving mode. At this time, the MSF positioning algorithm module can output the first navigation information, for example, the electronic device 100 displays Figure 6 In some other examples, the electronic device 100 may respond to the user interface 480 shown in (B). Figure 6 The user operation of the control 471B in the user interface 470 shown in (A) cancels the automatic driving mode and continues to maintain the safe navigation mode. At this time, the recognition suppression algorithm module can output the second navigation information. Not limited to this, in some other examples, the electronic device 100 can cancel the automatic driving mode and turn off the safe navigation mode according to the user operation. At this time, the user can choose to manually drive the electronic device 100.

[0270] S208: When the first preset condition is met, the identification suppression algorithm module outputs the second navigation information.

[0271] In one embodiment, when the GNSS interference detection module detects that the GNSS interference has not ended, the electronic device 100 is still subject to GNSS interference. When the first preset condition is met, the identification and suppression algorithm module can continue to output the second navigation information. Figure 7 The description of S113 is similar and will not be repeated here.

[0272] In one implementation, after S208, the electronic device 100 may execute S201-S206 again, and the time corresponding to the first data acquired at this time is different from time m, and accordingly, the time corresponding to the data obtained according to the first data is also different from time m.

[0273] Not limited to the above implementation, in another implementation, after S207, if the navigation of the electronic device 100 has not ended, the electronic device 100 may execute again Figure 7 S101-S111.

[0274] In one embodiment, an example diagram of the electronic device 100 detecting GNSS interference and the end of GNSS interference can be seen in Fig.13 . See Fig.13 , Fig.13 It is a schematic diagram of a GNSS interference detection process provided in an embodiment of the present application.

[0275] like Fig.13As shown, assuming that the electronic device 100 is subject to GNSS interference when it is at position 1, the navigation route of the electronic device 100 will change, for example, the route of driving straight in lane b becomes: the route of switching from lane b to lane a. The electronic device 100 will drive according to the changed route (first navigation information). In the process of the electronic device 100 driving from position 1 to position 2, when the electronic device 100 drives to the offset boundary, the electronic device 100 detects that the offset distance A is greater than or equal to the first offset threshold, and the electronic device 100 determines that it is subject to GNSS interference. At this time, the electronic device 100 can display an alarm message to prompt the user that it is currently subject to GNSS interference. Assuming that the electronic device 100 is subject to GNSS interference and the safety navigation mode is not turned on, the electronic device 100 continues to drive according to the changed route (for example, driving from position 2 to position 3). In the process of the electronic device 100 driving from position 3 to position 4, when the electronic device 100 drives to the offset boundary, the electronic device 100 detects that the offset distance B is less than the second offset threshold, and the electronic device 100 determines that the GNSS interference ends. At this time, the electronic device 100 can display an alarm end message to prompt the user that the current GNSS interference ends. Among them, the offset distance A can indicate Figure 7 The first interference statistic in the first offset threshold can indicate Figure 7 The first threshold in the offset distance B can indicate Fig.10 The second interference statistic in the second offset threshold can indicate Fig.10 The second threshold in .

[0276] Not limited to the above examples, in other examples, after the electronic device 100 detects that the offset distance A is greater than or equal to the first offset threshold, if the safe navigation mode has been turned on, it will travel according to the navigation route when there is no GNSS interference (corresponding to the second navigation information above). When the electronic device 100 detects that the offset distance B is less than the second offset threshold, it is determined that the GNSS interference has ended, and the electronic device 100 can display the end of the alarm.

[0277] exist Fig.10 In the method shown, the GNSS interference detection module can use the combined exponential gradual disappearance sequential probability ratio and chi-square test method to timely detect sudden hard faults in GNSS interference, so as to monitor the end time of GNSS interference, avoid false alarms caused by untimely detection, so that normal GNSS signals cannot be used for navigation and positioning, and display the alarm end information when the GNSS interference ends, prompting the user that the current driving safety threat has been lifted. In addition, during the GNSS interference process, the electronic device can display the real-time position obtained by the identification and suppression algorithm to avoid the user's use being affected by the erroneous positioning information output by the GNSS interference, further improve the safety of the electronic device during automatic driving, and thus ensure the safety of the user.

[0278] The methods provided in the embodiments of the present application may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiments of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital video disc (DWD), or a semiconductor medium (e.g., a solid state drive (SSD)). As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application is described in detail with reference to the above embodiments, a person skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A navigation fault detection method, characterized in that: Applied to a first device, the method includes: Acquire first data, where the first data includes a plurality of observation values ​​at a first moment respectively measured by a plurality of sensors in the first device; Obtaining first state information of the first device at the first moment according to the first data; detecting, according to the first data and the first status information, whether the first device is interfered with by a global navigation satellite system GNSS; When it is detected that the first device is subject to GNSS interference, a first interface is displayed, wherein the first interface includes first prompt information, and the first prompt information indicates that the first device is subject to GNSS interference.

2. The method according to claim 1, characterized in that The detecting, according to the first data and the first status information, whether the first device is subject to GNSS interference includes: Whether the first device is subject to GNSS interference is detected according to the first data, the historical data corresponding to the first data, the first status information, and the historical status information corresponding to the first status information.

3. The method according to claim 2, characterized in that The detecting, according to the first data, the historical data corresponding to the first data, the first status information, and the historical status information corresponding to the first status information, whether the first device is subject to GNSS interference includes: Obtaining second data according to the first data, historical data corresponding to the first data, the second state information, and historical state information corresponding to the second state information; A first statistic of the first device at the first moment is obtained according to the second data, and when the first statistic is greater than or equal to a first threshold, it is determined that the first device is subject to GNSS interference.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: When it is detected that the first device is not affected by GNSS interference, a second interface is displayed, and the second interface includes a first position and a first route. The first position is the position of the first device at the first moment estimated based on the second state information, and the first route is the navigation route of the first device at the first moment estimated based on the second state information.

5. The method according to claim 4, characterized in that The method further comprises: When it is detected that the first device is subject to GNSS interference, a third interface is displayed, and the third interface includes a second position and a second route, the second position is the position of the first device at the first moment estimated based on the first status information, and the second route is the navigation route of the first device at the first moment estimated based on the first status information.

6. The method according to claim 5, characterized in that The first state information is obtained according to a first algorithm, and the second state information is obtained according to a second algorithm. The first algorithm is an identification suppression algorithm, and the second algorithm is a multi-sensor fusion MSF positioning algorithm.

7. The method according to claims 1 to 6, characterized in that The first state information includes at least one of the position, posture, speed, and time of the first device.

8. The method according to claim 1, characterized in that The method further comprises: When the first interface is displayed, receiving a first operation; In response to the first operation, a third interface is displayed, and the third interface includes a second position and a second route, the second position is the position of the first device at the first moment estimated based on the first state information, and the second route is the navigation route of the first device at the first moment estimated based on the first state information.

9. The method according to claim 1, characterized in that When it is detected that the first device is interfered with by the GNSS, displaying a first interface includes: When it is detected that the first device is interfered with by GNSS, if the defense function is not enabled, displaying the first interface; The method further comprises: When it is detected that the first device is subject to GNSS interference, if the defense function is turned on, a third interface is displayed, and the third interface includes a second position and a second route. The second position is the position of the first device at the first moment estimated based on the first status information, and the second route is the navigation route of the first device at the first moment estimated based on the first status information.

10. The method according to claim 1, characterized in that The method further comprises: When the first interface is displayed, the driving mode of the first device is the automatic driving mode; After displaying the first interface, the method further includes: A second operation is received, where the second operation is used to switch the automatic driving mode to a manual driving mode.

11. The method according to claim 1, characterized in that: The detecting that the first device is interfered with by the GNSS includes: When it is detected that the offset distance is greater than or equal to the offset threshold, it is determined that the first device is subject to GNSS interference, the offset distance is a difference between a first position and a second position, the first position is a position estimated according to a first algorithm, the second position is a position estimated according to a second algorithm, the first algorithm is an identification suppression algorithm, and the second algorithm is an MSF positioning algorithm; When it is detected that the offset distance is less than the offset threshold, it is determined that the GNSS interference to the first device ends.

12. The method according to claim 11, characterized in that The method further comprises: When it is detected that the first device is not interfered with by the GNSS, the first device is located in a first lane; The offset distance is the distance between the first position estimated by the first algorithm and a lane dividing line, and the lane dividing line is a dividing line between the first lane and an adjacent lane.

13. The method according to any one of claims 1 to 12, characterized in that: The method further comprises: After detecting that the first device is subject to GNSS interference, acquiring second data, where the second data includes a plurality of observation values ​​at a second moment respectively measured by the plurality of sensors, where the second moment is later than the first moment; detecting, according to the second data, whether the GNSS interference to the first device is ended; When it is detected that the GNSS interference suffered by the first device ends, a second prompt information is displayed, and the second prompt information indicates that the GNSS interference suffered by the first device ends.

14. The method according to claim 13, characterized in that The method further comprises: After detecting that the GNSS interference to the first device has ended, a fourth interface is displayed, wherein the fourth interface includes a third position and a third route, wherein the third position is the position of the first device at the second moment estimated based on the third state information, and the third route is the navigation route of the first device at the second moment estimated based on the third state information, and the third state information is obtained based on a second algorithm, and the second algorithm is an MSF positioning algorithm.

15. The method according to claim 4, characterized in that After displaying the second interface, the method further includes: When it is detected that the first device is subject to GNSS interference, if the defense function is not turned on, a fifth interface is displayed, and the fifth interface includes a fourth position and a fourth route. The fourth position and the fourth route are obtained according to a second algorithm. The time corresponding to the fourth position is later than the first time. The fourth position is different from the first position, and the fourth route is different from the first route. The second algorithm is an MSF positioning algorithm.

16. The method according to claim 13, characterized in that The detecting, according to the second data, whether the GNSS interference suffered by the first device is ended includes: Obtaining third data according to the second data and a first estimated value, wherein the first estimated value is obtained according to a portion of observed values ​​in the second data, the first estimated value is obtained according to a first algorithm, the first algorithm is a recognition suppression algorithm, the first estimated value corresponds to the second moment, and the third data corresponds to the second moment; When it is detected at a third moment that the first device is subject to GNSS interference, obtaining a second statistic according to the second data, the third moment being a moment before the second moment, and the second statistic corresponding to the second moment; When the second statistic is greater than or equal to the first threshold, obtaining a third statistic according to the third data, wherein the third statistic corresponds to the second moment; When the third statistic is greater than or equal to a second threshold, it is determined that the GNSS interference to the first device ends.

17. An electronic device, characterized in that: The method comprises a transceiver, a processor and a memory, wherein the memory is used to store a computer program, and the processor calls the computer program to execute the method according to any one of claims 1 to 16.

18. A computer storage medium, characterized in that: The computer storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 17 is implemented.

Citation Information

Patent Citations

  • Design method of UUV multi-sea-area integrated navigation system based on factor graph optimization

    CN113984061A

  • Interference display method and device for positioning navigation, electronic equipment and storage medium

    CN114710220A

  • Vehicle control method and device based on road side unit

    CN114987542A

  • Multi-sensor fusion positioning method and system in unmanned complex scene

    CN115265551A

  • GNSS (Global Navigation Satellite System) sensor fault diagnosis method and device, navigation system and vehicle

    CN115453580A